OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Using private IPs for Hardware Nodes

2007-11-30T11:35:34Z

Bwian: /* An OpenVZ Hardware Node has two Ethernet interfaces */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE is started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create the file <code>/etc/vz/vznet.conf</code> with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> should be executable.(chmod +x /usr/sbin/vznetcfg.custom)}}

==== Setting the route VE → HN ====
To set up a route from the VE to the HN, the custom script has to get a HN IP (the $VE0_IP variable in the script). There are several ways to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Each variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assuming you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no need to make the VE accessible from the HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of the above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

It is nesessary to set a local IP for 'br0' to ensure VE ↔ HN connection availability.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:31:44Z

Bwian: /* Setting the route VE → HN */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE is started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create the file <code>/etc/vz/vznet.conf</code> with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> should be executable.(chmod +x /usr/sbin/vznetcfg.custom)}}

==== Setting the route VE → HN ====
To set up a route from the VE to the HN, the custom script has to get a HN IP (the $VE0_IP variable in the script). There are several ways to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Each variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:29:16Z

Bwian: /* Make the script to be run on a VE start */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE is started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create the file <code>/etc/vz/vznet.conf</code> with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> should be executable.(chmod +x /usr/sbin/vznetcfg.custom)}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:27:25Z

Bwian: /* Create a custom network configuration script */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE is started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:26:25Z

Bwian: /* Edit the VE's configuration */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:24:52Z

Bwian: /* Set up a bridge on a HN */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring the <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To automatically create bridge <code>br0</code> you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> to add the <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add some parameters to the <code>/etc/vz/conf/$VEID.conf</code> which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T11:15:15Z

Bwian: /* (Optional) Add VE↔HN routes */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The above configuration provides the following connections:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that

* The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes)

* The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not)

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To make bridge <code>br0</code> automatically created you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> file to add <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add some parameters to the <code>/etc/vz/conf/$VEID.conf</code> which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T10:49:45Z

Bwian: /* Add the VE's veth interface to the bridge */ readability

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The configuration above provides following connections available:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that
* A VE accessibility from the HN depends on if the local gateway provides NAT or not (probably - yes).
* A HN accessibility from a VE depends on if the ISP gateway is aware about the local network addresses (most probably - no).

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following route rules:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To make bridge <code>br0</code> automatically created you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> file to add <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add some parameters to the <code>/etc/vz/conf/$VEID.conf</code> which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Using private IPs for Hardware Nodes

2007-11-30T10:37:46Z

Bwian: /* Prerequisites */ grammar/readability, minor edit

This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology:

[[Image:PrivateIPs_fig1.gif|An initial network topology]]

== Prerequisites ==
This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template.
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any.

This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities.

This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed.
{{Note|don't assign an IP after VE creation.}}

== An OVZ Hardware Node has the only one Ethernet interface ==
(assume eth0)

=== Hardware Node configuration ===

==== Create a bridge device ====
[HN]# brctl addbr br0

==== Remove an IP from eth0 interface ====
[HN]# ifconfig eth0 0

==== Add eth0 interface into the bridge ====
[HN]# brctl addif br0 eth0

==== Assign the IP to the bridge ====
(the same that was assigned on eth0 earlier)
[HN]# ifconfig br0 10.0.0.2/24

==== Resurrect the default routing ====
[HN]# ip route add default via 10.0.0.1 dev br0

{{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}}

==== A script example ====
<pre>
[HN]# cat /tmp/br_add
#!/bin/bash

brctl addbr br0
ifconfig eth0 0
brctl addif br0 eth0
ifconfig br0 10.0.0.2/24
ip route add default via 10.0.0.1 dev br0
</pre>

[HN]# /tmp/br_add >/dev/null 2>&1 &

=== VE configuration ===

==== Start a VE ====
[HN]# vzctl start 101

==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ====
[HN]# vzctl set 101 --netif_add eth0 --save

==== Set up an IP to the newly created VE's veth interface ====
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26

==== Add the VE's veth interface to the bridge ====
[HN]# brctl addif br0 veth101.0

{{Note|veth interface will work as expected only after it is turned by the bridge into the forwarding state after some delay.
In 2.6.18 kernel it is 15 seconds by default.
}}

==== Set up the default route for the VE ====
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0

==== (Optional) Add VE↔HN routes ====
The configuration above provides following connections available:
* VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN)
* VE ↔ Internet

Note that
* A VE accessibility from the HN depends on if the local gateway provides NAT or not (probably - yes).
* A HN accessibility from a VE depends on if the ISP gateway is aware about the local network addresses (most probably - no).

So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following route rules:

[HN]# ip route add 85.86.87.195 dev br0
[HN]# vzctl exec 101 ip route add 10.0.0.2 dev eth0

=== Resulting OpenVZ Node configuration ===
[[Image:PrivateIPs_fig2.gif|Resulting OpenVZ Node configuration]]

=== Making the configuration persistent ===

==== Set up a bridge on a HN ====
This can be done by configuring <code>ifcfg-*</code> files located in <code>/etc/sysconfig/network-scripts/</code>.

Assuming you had a configuration file (e.g. <code>ifcfg-eth0</code>) like:
<pre>
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

To make bridge <code>br0</code> automatically created you can create <code>ifcfg-br0</code>:
<pre>
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
</pre>

and edit <code>ifcfg-eth0</code> file to add <code>eth0</code> interface into the bridge <code>br0</code>:
<pre>
DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
</pre>

==== Edit the VE's configuration ====
Add some parameters to the <code>/etc/vz/conf/$VEID.conf</code> which will be used during the network configuration:
* Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start)
* Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces)
* Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code>
* Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added)

An example:
<pre>
# Network customization section
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="85.86.87.195/26"
VE_DEFAULT_GATEWAY="85.86.87.193"
BRIDGEDEV="br0"
</pre>

==== Create a custom network configuration script ====
which should be called each time a VE started (e.g. <code>/usr/sbin/vznetcfg.custom</code>):
<pre>
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE

GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/usr/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE

NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done

if [ ! -n "$VETH_IP_ADDRESS" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi

echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0

VEROUTEDEV=$VZHOSTIF

if [ -n "$BRIDGEDEV" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV."
VEROUTEDEV=$BRIDGEDEV
$brctl addif $BRIDGEDEV $VZHOSTIF
fi

# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up

for IP in $VETH_IP_ADDRESS; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME

# removing the netmask
IP_STRIP=${IP%%/*};

echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done

if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi

if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi

exit 0
</pre>

==== Make the script to be run on a VE start ====
In order to run above script on a VE start create <code>/etc/vz/vznet.conf</code> file with the following contents:

EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"

{{Note|<code>/usr/sbin/vznetcfg.custom</code> file should be executable.}}

==== Setting the route VE → HN ====
To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it:

# Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code>
# Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file)
# Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script

Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple).

== An OpenVZ Hardware Node has two Ethernet interfaces ==
Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic.
Let's assign eth0 for the external traffic and eth1 for the local one.

If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration:
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]]
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]]

For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'.

== Putting VEs to different subnetworks ==
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the
[[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]].

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]

[[Category: HOWTO]]
[[Category: Networking]]

Bonding

2007-09-04T06:46:33Z

Bwian: /* Modify eth0 and eth1 config files */

Linux allows binding multiple network interfaces into a single channel/NIC.

== Introduction ==
The Linux bonding driver provides a method for aggregating
multiple network interfaces into a single logical "bonded" interface.
The behavior of the bonded interfaces depends upon the mode; generally
speaking, modes provide either hot standby or load balancing services.
Additionally, link integrity monitoring may be performed.

== Setting up bounding on a RHEL/CentOS 4 system ==

===Create a bond0 configuration file===

Red Hat Linux stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you need to create bond0 config file:

# vi /etc/sysconfig/network-scripts/ifcfg-bond0

Append following lines to it:

====In case of static IP====
<pre>
DEVICE=bond0
IPADDR=x.x.x.x
NETWORK=y.y.y.y
NETMASK=z.z.z.z
BOOTPROTO=none
ONBOOT=yes
</pre>
x.x.x.x is an IP address of HW.

y.y.y.y is an Network address of HW.

z.z.z.z is an net mask address of HW (usually 255.255.255.0).

Replace above IP data with your actual IP address. Save file and exit to shell prompt.

====In case of DHCP====
<pre>
DEVICE=bond0
BOOTPROTO=dhcp
ONBOOT=yes
</pre>

===Modify eth0 and eth1 config files===

Open both configurations using vi (text editor) and make sure the file reads as follows for interface eth0

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
<pre>
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>

Open the eth1 configuration file using vi :

vi /etc/sysconfig/network-scripts/ifcfg-eth1

Make sure the file reads as follows for interface eth1:
<pre>
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>
Save the file and exit to shell prompt.

=== Load bond driver/module ===
Make sure the bonding module is loaded before the channel-bonding interface (bond0) is brought up. You need to modify the kernel modules configuration file:

# vi /etc/modprobe.conf
<pre>
alias bond0 bonding
options bond0 miimon=100
</pre>
You can learn more about bonding options in the kernel source documentation file "Documentation/networking/bonding.txt"

=== Test the configuration ===
First, load the bonding module:
<pre>
# modprobe bonding
</pre>
Restart networking service in order to up bond0 interface:
<pre>
# service network restart
</pre>
Check proc info:

# cat /proc/net/bonding/bond0
<pre>
Ethernet Channel Bonding Driver: v2.6.3 (June 8, 2005)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:c3

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:cd
</pre>

List all interfaces:
<pre>
#ip a

2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc noqueue
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.25/16 brd 10.17.255.255 scope global bond0
6: eth0: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
8: eth1: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
1: venet0: <BROADCAST,POINTOPOINT,NOARP,UP> mtu 1500 qdisc noqueue
link/void
</pre>
Route:
<pre>
# ip r

10.17.0.0/16 dev bond0 proto kernel scope link src 10.17.3.25
169.254.0.0/16 dev bond0 scope link
default via 10.17.0.1 dev bond0
</pre>

== Traffic shaping ==
=== Virtuozzo traffic shaping tools ===
Just replace old netdev to new bonding device (bond0)

vi /etc/sysconfig/vz
<pre>
## Network traffic parameters
TRAFFIC_SHAPING=yes
BANDWIDTH="bond0:102400"
TOTALRATE="bond0:1:4096"
RATE="bond0:1:8"
</pre>
and do the rest as usuall
<pre>
# vzctl set $veid --ratebound $bound --rate $rif:$class:$rate --save
</pre>

=== Traffic shaping with tc ===
Where is no specifics here, see [[Traffic shaping with tc]].

As a result in:
<pre>
# ip a s bond0
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc cbq
##NOTE:Class Based Queueing tc was added ^^^^^^^^^^
link/ether 00:0c:29:07:d4:c3 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.41/16 brd 10.17.255.255 scope global bond0
</pre>

[[Category: HOWTO]]
[[Category: Networking]]

Bonding

2007-09-04T06:44:20Z

Bwian: /* Load bond driver/module */

Linux allows binding multiple network interfaces into a single channel/NIC.

== Introduction ==
The Linux bonding driver provides a method for aggregating
multiple network interfaces into a single logical "bonded" interface.
The behavior of the bonded interfaces depends upon the mode; generally
speaking, modes provide either hot standby or load balancing services.
Additionally, link integrity monitoring may be performed.

== Setting up bounding on a RHEL/CentOS 4 system ==

===Create a bond0 configuration file===

Red Hat Linux stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you need to create bond0 config file:

# vi /etc/sysconfig/network-scripts/ifcfg-bond0

Append following lines to it:

====In case of static IP====
<pre>
DEVICE=bond0
IPADDR=x.x.x.x
NETWORK=y.y.y.y
NETMASK=z.z.z.z
BOOTPROTO=none
ONBOOT=yes
</pre>
x.x.x.x is an IP address of HW.

y.y.y.y is an Network address of HW.

z.z.z.z is an net mask address of HW (usually 255.255.255.0).

Replace above IP data with your actual IP address. Save file and exit to shell prompt.

====In case of DHCP====
<pre>
DEVICE=bond0
BOOTPROTO=dhcp
ONBOOT=yes
</pre>

===Modify eth0 and eth1 config files===

Open both configuration using vi text editor and make sure file read as follows for eth0 interface

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
<pre>
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>

Open eth1 configuration file using vi text editor:

vi /etc/sysconfig/network-scripts/ifcfg-eth1

Make sure file read as follows for eth1 interface:
<pre>
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>
Save file and exit to shell prompt.

=== Load bond driver/module ===
Make sure the bonding module is loaded before the channel-bonding interface (bond0) is brought up. You need to modify the kernel modules configuration file:

# vi /etc/modprobe.conf
<pre>
alias bond0 bonding
options bond0 miimon=100
</pre>
You can learn more about bonding options in the kernel source documentation file "Documentation/networking/bonding.txt"

=== Test the configuration ===
First, load the bonding module:
<pre>
# modprobe bonding
</pre>
Restart networking service in order to up bond0 interface:
<pre>
# service network restart
</pre>
Check proc info:

# cat /proc/net/bonding/bond0
<pre>
Ethernet Channel Bonding Driver: v2.6.3 (June 8, 2005)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:c3

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:cd
</pre>

List all interfaces:
<pre>
#ip a

2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc noqueue
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.25/16 brd 10.17.255.255 scope global bond0
6: eth0: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
8: eth1: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
1: venet0: <BROADCAST,POINTOPOINT,NOARP,UP> mtu 1500 qdisc noqueue
link/void
</pre>
Route:
<pre>
# ip r

10.17.0.0/16 dev bond0 proto kernel scope link src 10.17.3.25
169.254.0.0/16 dev bond0 scope link
default via 10.17.0.1 dev bond0
</pre>

== Traffic shaping ==
=== Virtuozzo traffic shaping tools ===
Just replace old netdev to new bonding device (bond0)

vi /etc/sysconfig/vz
<pre>
## Network traffic parameters
TRAFFIC_SHAPING=yes
BANDWIDTH="bond0:102400"
TOTALRATE="bond0:1:4096"
RATE="bond0:1:8"
</pre>
and do the rest as usuall
<pre>
# vzctl set $veid --ratebound $bound --rate $rif:$class:$rate --save
</pre>

=== Traffic shaping with tc ===
Where is no specifics here, see [[Traffic shaping with tc]].

As a result in:
<pre>
# ip a s bond0
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc cbq
##NOTE:Class Based Queueing tc was added ^^^^^^^^^^
link/ether 00:0c:29:07:d4:c3 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.41/16 brd 10.17.255.255 scope global bond0
</pre>

[[Category: HOWTO]]
[[Category: Networking]]

Bonding

2007-09-04T06:42:28Z

Bwian: /* Load bond driver/module */

Linux allows binding multiple network interfaces into a single channel/NIC.

== Introduction ==
The Linux bonding driver provides a method for aggregating
multiple network interfaces into a single logical "bonded" interface.
The behavior of the bonded interfaces depends upon the mode; generally
speaking, modes provide either hot standby or load balancing services.
Additionally, link integrity monitoring may be performed.

== Setting up bounding on a RHEL/CentOS 4 system ==

===Create a bond0 configuration file===

Red Hat Linux stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you need to create bond0 config file:

# vi /etc/sysconfig/network-scripts/ifcfg-bond0

Append following lines to it:

====In case of static IP====
<pre>
DEVICE=bond0
IPADDR=x.x.x.x
NETWORK=y.y.y.y
NETMASK=z.z.z.z
BOOTPROTO=none
ONBOOT=yes
</pre>
x.x.x.x is an IP address of HW.

y.y.y.y is an Network address of HW.

z.z.z.z is an net mask address of HW (usually 255.255.255.0).

Replace above IP data with your actual IP address. Save file and exit to shell prompt.

====In case of DHCP====
<pre>
DEVICE=bond0
BOOTPROTO=dhcp
ONBOOT=yes
</pre>

===Modify eth0 and eth1 config files===

Open both configuration using vi text editor and make sure file read as follows for eth0 interface

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
<pre>
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>

Open eth1 configuration file using vi text editor:

vi /etc/sysconfig/network-scripts/ifcfg-eth1

Make sure file read as follows for eth1 interface:
<pre>
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
</pre>
Save file and exit to shell prompt.

=== Load bond driver/module ===
Make sure the bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify the kernel modules configuration file:

# vi /etc/modprobe.conf
<pre>
alias bond0 bonding
options bond0 miimon=100
</pre>
You can learn more about all bonding options in the kernel source documentation file "Documentation/networking/bonding.txt"

=== Test the configuration ===
First, load the bonding module:
<pre>
# modprobe bonding
</pre>
Restart networking service in order to up bond0 interface:
<pre>
# service network restart
</pre>
Check proc info:

# cat /proc/net/bonding/bond0
<pre>
Ethernet Channel Bonding Driver: v2.6.3 (June 8, 2005)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:c3

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:07:d4:cd
</pre>

List all interfaces:
<pre>
#ip a

2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc noqueue
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.25/16 brd 10.17.255.255 scope global bond0
6: eth0: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
8: eth1: <BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:0c:29:73:26:19 brd ff:ff:ff:ff:ff:ff
1: venet0: <BROADCAST,POINTOPOINT,NOARP,UP> mtu 1500 qdisc noqueue
link/void
</pre>
Route:
<pre>
# ip r

10.17.0.0/16 dev bond0 proto kernel scope link src 10.17.3.25
169.254.0.0/16 dev bond0 scope link
default via 10.17.0.1 dev bond0
</pre>

== Traffic shaping ==
=== Virtuozzo traffic shaping tools ===
Just replace old netdev to new bonding device (bond0)

vi /etc/sysconfig/vz
<pre>
## Network traffic parameters
TRAFFIC_SHAPING=yes
BANDWIDTH="bond0:102400"
TOTALRATE="bond0:1:4096"
RATE="bond0:1:8"
</pre>
and do the rest as usuall
<pre>
# vzctl set $veid --ratebound $bound --rate $rif:$class:$rate --save
</pre>

=== Traffic shaping with tc ===
Where is no specifics here, see [[Traffic shaping with tc]].

As a result in:
<pre>
# ip a s bond0
4: bond0: <BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc cbq
##NOTE:Class Based Queueing tc was added ^^^^^^^^^^
link/ether 00:0c:29:07:d4:c3 brd ff:ff:ff:ff:ff:ff
inet 10.17.3.41/16 brd 10.17.255.255 scope global bond0
</pre>

[[Category: HOWTO]]
[[Category: Networking]]