Editing Using private IPs for Hardware Nodes
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | + | This article describes how to assign public IPs to VEs running on OVZ Hardware Nodes in case you have a following network topology: | |
− | |||
− | This article describes how to assign public IPs to | ||
[[Image:PrivateIPs_fig1.gif|An initial network topology]] | [[Image:PrivateIPs_fig1.gif|An initial network topology]] | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== Prerequisites == | == Prerequisites == | ||
− | This configuration was tested on a RHEL5 OpenVZ Hardware Node and a | + | This configuration was tested on a RHEL5 OpenVZ Hardware Node and a VE based on a Fedora Core 5 template. |
Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any. | Other host OSs and templates might require some configuration changes, please add corresponding OS specific changes if you've faced any. | ||
This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities. | This article assumes the presence of 'brctl', 'ip' and 'ifconfig' utils. You may need to install missing packages like 'bridge-utils'/'iproute'/'net-tools' or others which contain those utilities. | ||
− | This article assumes you have already [[Quick installation|installed OpenVZ]], | + | This article assumes you have already [[Quick installation|installed OpenVZ]], prepared the [[OS template cache]](s) and have [[Basic_operations_in_OpenVZ_environment|VE(s) created]]. If not, follow the links to perform the steps needed. |
− | prepared the [[OS template cache]](s) and have | + | {{Note|don't assign an IP after VE creation.}} |
− | [[Basic_operations_in_OpenVZ_environment| | ||
− | {{Note|don't assign an IP after | ||
== An OVZ Hardware Node has the only one Ethernet interface == | == An OVZ Hardware Node has the only one Ethernet interface == | ||
Line 26: | Line 16: | ||
=== Hardware Node configuration === | === Hardware Node configuration === | ||
− | |||
− | |||
==== Create a bridge device ==== | ==== Create a bridge device ==== | ||
Line 45: | Line 33: | ||
[HN]# ip route add default via 10.0.0.1 dev br0 | [HN]# ip route add default via 10.0.0.1 dev br0 | ||
− | + | {{Warning|if you are '''configuring''' the node '''remotely''' you '''must''' prepare a '''script''' with the above commands and run it in background with the redirected output or you'll '''lose the access''' to the Node.}} | |
==== A script example ==== | ==== A script example ==== | ||
Line 61: | Line 49: | ||
[HN]# /tmp/br_add >/dev/null 2>&1 & | [HN]# /tmp/br_add >/dev/null 2>&1 & | ||
− | === | + | === VE configuration === |
− | ==== Start a | + | ==== Start a VE ==== |
[HN]# vzctl start 101 | [HN]# vzctl start 101 | ||
− | ==== Add a [[Virtual_Ethernet_device|veth interface]] to the | + | ==== Add a [[Virtual_Ethernet_device|veth interface]] to the VE ==== |
[HN]# vzctl set 101 --netif_add eth0 --save | [HN]# vzctl set 101 --netif_add eth0 --save | ||
− | ==== Set up an IP to the newly created | + | ==== Set up an IP to the newly created VE's veth interface ==== |
[HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26 | [HN]# vzctl exec 101 ifconfig eth0 85.86.87.195/26 | ||
− | ==== Add the | + | ==== Add the VE's veth interface to the bridge ==== |
[HN]# brctl addif br0 veth101.0 | [HN]# brctl addif br0 veth101.0 | ||
− | {{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the | + | {{Note|There will be a delay of about 15 seconds(default for 2.6.18 kernel) while the bridge software runs STP to detect loops and transitions the venet interface to the forwarding state. |
<!-- /sys/class/net/$BR_NAME/bridge/forward_delay in SEC*USER_HZ -->}} | <!-- /sys/class/net/$BR_NAME/bridge/forward_delay in SEC*USER_HZ -->}} | ||
− | ==== Set up the default route for the | + | ==== Set up the default route for the VE ==== |
[HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0 | [HN]# vzctl exec 101 ip route add default via 85.86.87.193 dev eth0 | ||
− | ==== (Optional) Add | + | ==== (Optional) Add VE↔HN routes ==== |
The above configuration provides the following connections: | The above configuration provides the following connections: | ||
− | * | + | * VE X ↔ VE Y (where VE X and VE Y can locate on any OVZ HN) |
− | * | + | * VE ↔ Internet |
Note that | Note that | ||
− | * The accessability of the | + | * The accessability of the VE from the HN depends on the local gateway providing NAT(probably - yes) |
− | * The accessability of the HN from the | + | * The accessability of the HN from the VE depends on the ISP gateway being aware of the local network(probably not) |
− | So to provide | + | |
+ | So to provide VE ↔ HN accessibility despite the gateways' configuration you can add the following routes: | ||
[HN]# ip route add 85.86.87.195 dev br0 | [HN]# ip route add 85.86.87.195 dev br0 | ||
Line 131: | Line 120: | ||
</pre> | </pre> | ||
− | ==== Edit the | + | ==== Edit the VE's configuration ==== |
− | Add these parameters to the <code>/etc/vz/conf/$ | + | Add these parameters to the <code>/etc/vz/conf/$VEID.conf</code> file which will be used during the network configuration: |
− | * Add <code>VETH_IP_ADDRESS="IP/MASK"</code> (a | + | * Add/change <code>CONFIG_CUSTOMIZED="yes"</code> (indicates that a custom script should be run on a VE start) |
− | * Add <code>VE_DEFAULT_GATEWAY=" | + | * Add <code>VETH_IP_ADDRESS="VE IP/MASK"</code> (a VE can have multiple IPs separated by spaces) |
− | * Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the | + | * Add <code>VE_DEFAULT_GATEWAY="VE DEFAULT GATEWAY"</code> |
+ | * Add <code>BRIDGEDEV="BRIDGE NAME"</code> (a bridge name to which the VE veth interface should be added) | ||
An example: | An example: | ||
<pre> | <pre> | ||
# Network customization section | # Network customization section | ||
+ | CONFIG_CUSTOMIZED="yes" | ||
VETH_IP_ADDRESS="85.86.87.195/26" | VETH_IP_ADDRESS="85.86.87.195/26" | ||
VE_DEFAULT_GATEWAY="85.86.87.193" | VE_DEFAULT_GATEWAY="85.86.87.193" | ||
Line 146: | Line 137: | ||
==== Create a custom network configuration script ==== | ==== Create a custom network configuration script ==== | ||
− | which should be called each time a | + | which should be called each time a VE is started (e.g. <code>/usr/sbin/vznetcfg.custom</code>): |
<pre> | <pre> | ||
#!/bin/bash | #!/bin/bash | ||
# /usr/sbin/vznetcfg.custom | # /usr/sbin/vznetcfg.custom | ||
− | # a script to bring up bridged network interfaces (veth's) in a | + | # a script to bring up bridged network interfaces (veth's) in a VE |
GLOBALCONFIGFILE=/etc/vz/vz.conf | GLOBALCONFIGFILE=/etc/vz/vz.conf | ||
− | + | VECONFIGFILE=/etc/vz/conf/$VEID.conf | |
vzctl=/usr/sbin/vzctl | vzctl=/usr/sbin/vzctl | ||
brctl=/usr/sbin/brctl | brctl=/usr/sbin/brctl | ||
Line 159: | Line 150: | ||
ifconfig=/sbin/ifconfig | ifconfig=/sbin/ifconfig | ||
. $GLOBALCONFIGFILE | . $GLOBALCONFIGFILE | ||
− | . $ | + | . $VECONFIGFILE |
NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'` | NETIF_OPTIONS=`echo $NETIF | sed 's/,/\n/g'` | ||
for str in $NETIF_OPTIONS; do \ | for str in $NETIF_OPTIONS; do \ | ||
# getting 'ifname' parameter value | # getting 'ifname' parameter value | ||
− | if | + | if [[ "$str" =~ "^ifname=" ]]; then |
# remove the parameter name from the string (along with '=') | # remove the parameter name from the string (along with '=') | ||
− | + | VEIFNAME=${str#*=}; | |
fi | fi | ||
# getting 'host_ifname' parameter value | # getting 'host_ifname' parameter value | ||
− | if | + | if [[ "$str" =~ "^host_ifname=" ]]; then |
# remove the parameter name from the string (along with '=') | # remove the parameter name from the string (along with '=') | ||
VZHOSTIF=${str#*=}; | VZHOSTIF=${str#*=}; | ||
Line 176: | Line 167: | ||
if [ ! -n "$VETH_IP_ADDRESS" ]; then | if [ ! -n "$VETH_IP_ADDRESS" ]; then | ||
− | echo "According to $CONFIGFILE | + | echo "According to $CONFIGFILE VE$VEID has no veth IPs configured." |
exit 1 | exit 1 | ||
fi | fi | ||
if [ ! -n "$VZHOSTIF" ]; then | if [ ! -n "$VZHOSTIF" ]; then | ||
− | echo "According to $CONFIGFILE | + | echo "According to $CONFIGFILE VE$VEID has no veth interface configured." |
exit 1 | exit 1 | ||
fi | fi | ||
− | if [ ! -n "$ | + | if [ ! -n "$VEIFNAME" ]; then |
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF." | echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF." | ||
exit 1 | exit 1 | ||
fi | fi | ||
− | echo "Initializing interface $VZHOSTIF for | + | echo "Initializing interface $VZHOSTIF for VE$VEID." |
$ifconfig $VZHOSTIF 0 | $ifconfig $VZHOSTIF 0 | ||
− | + | VEROUTEDEV=$VZHOSTIF | |
if [ -n "$BRIDGEDEV" ]; then | if [ -n "$BRIDGEDEV" ]; then | ||
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV." | echo "Adding interface $VZHOSTIF to the bridge $BRIDGEDEV." | ||
− | + | VEROUTEDEV=$BRIDGEDEV | |
$brctl addif $BRIDGEDEV $VZHOSTIF | $brctl addif $BRIDGEDEV $VZHOSTIF | ||
fi | fi | ||
− | # Up the interface $ | + | # Up the interface $VEIFNAME link in VE$VEID |
− | $vzctl exec $VEID $ip link set $ | + | $vzctl exec $VEID $ip link set $VEIFNAME up |
for IP in $VETH_IP_ADDRESS; do | for IP in $VETH_IP_ADDRESS; do | ||
− | echo "Adding an IP $IP to the $ | + | echo "Adding an IP $IP to the $VEIFNAME for VE$VEID." |
− | $vzctl exec $VEID $ip address add $IP dev $ | + | $vzctl exec $VEID $ip address add $IP dev $VEIFNAME |
# removing the netmask | # removing the netmask | ||
IP_STRIP=${IP%%/*}; | IP_STRIP=${IP%%/*}; | ||
− | echo "Adding a route from | + | echo "Adding a route from VE0 to VE$VEID." |
− | $ip route add $IP_STRIP dev $ | + | $ip route add $IP_STRIP dev $VEROUTEDEV |
done | done | ||
− | if [ -n "$ | + | if [ -n "$VE0_IP" ]; then |
− | echo "Adding a route from | + | echo "Adding a route from VE$VEID to VE0." |
− | $vzctl exec $VEID $ip route add $ | + | $vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME |
fi | fi | ||
if [ -n "$VE_DEFAULT_GATEWAY" ]; then | if [ -n "$VE_DEFAULT_GATEWAY" ]; then | ||
− | echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for | + | echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID." |
$vzctl exec $VEID \ | $vzctl exec $VEID \ | ||
− | $ip route add default via $VE_DEFAULT_GATEWAY dev $ | + | $ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME |
fi | fi | ||
exit 0 | exit 0 | ||
</pre> | </pre> | ||
− | |||
− | ==== Make the script to be run on a | + | ==== Make the script to be run on a VE start ==== |
− | In order to run above script on a | + | In order to run above script on a VE start create the file <code>/etc/vz/vznet.conf</code> with the following contents: |
− | <code>/etc/vz/vznet.conf</code> with the following contents: | ||
EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom" | EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom" | ||
− | {{Note|<code>/usr/sbin/vznetcfg.custom</code> should be executable (chmod +x /usr/sbin/vznetcfg.custom | + | {{Note|<code>/usr/sbin/vznetcfg.custom</code> should be executable.(chmod +x /usr/sbin/vznetcfg.custom)}} |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | ==== Setting the route | + | ==== Setting the route VE → HN ==== |
− | To set up a route from | + | To set up a route from VE to HN the custom script has to get a HN IP (the $VE0_IP variable in the script). There can be different approaches to specify it: |
− | # Add an entry | + | # Add an entry VE0_IP="VE0 IP" to the <code>$VEID.conf</code> |
− | # Add an entry | + | # Add an entry VE0_IP="VE0 IP" to the <code>/etc/vz/vz.conf</code> (the global configuration config file) |
− | # Implement some smart algorithm to determine the | + | # Implement some smart algorithm to determine the VE0 IP right in the custom network configuration script |
− | + | Every variant has its pros and cons, nevertheless for HN static IP configuration variant 2 seems to be acceptable (and the most simple). | |
== An OpenVZ Hardware Node has two Ethernet interfaces == | == An OpenVZ Hardware Node has two Ethernet interfaces == | ||
− | + | Assume you have 2 interfaces eth0 and eth1 and want to separate local traffic (10.0.0.0/24) from the external traffic. | |
Let's assign eth0 for the external traffic and eth1 for the local one. | Let's assign eth0 for the external traffic and eth1 for the local one. | ||
− | If there is no | + | If there is no aim to make VE accessible from HN and vice versa, it's enough to replace 'br0' with 'eth1' in the following steps of above configuration: |
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]] | * Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Assign_the_IP_to_the_bridge|Assign the IP to the bridge]] | ||
* Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]] | * Hardware Node configuration → [[Using_private_IPs_for_Hardware_Nodes#Resurrect_the_default_routing|Resurrect the default routing]] | ||
− | + | For the VE ↔ HN connections availability it is nesessary to set an IP (local) to the 'br0'. | |
− | == Putting | + | == Putting VEs to different subnetworks == |
It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the | It's enough to set up the correct $VETH_IP_ADDRESS and $VE_DEFAULT_GATEWAY values in the | ||
− | [[Using_private_IPs_for_Hardware_Nodes# | + | [[Using_private_IPs_for_Hardware_Nodes#Edit_the_VE.27s_configuration|above configuration]]. |
== See also == | == See also == |