Virtual Ethernet device
Virtual ethernet device is an ethernet-like device which can be used inside a VE. Unlike venet network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to ethX or other device and VE user fully sets up his networking himself, including IPs, gateways etc.
Virtual ethernet device consist of two ethernet devices - one in CT0 and another one in VE. These devices are connected to each other, so if a packet goes to one device it will come out from the other device.
Contents
Virtual ethernet device usage
Kernel module
First of all, make sure the vzethdev module is loaded:
# lsmod | grep vzeth vzethdev 8224 0 vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
In case it is not loaded, load it:
# modprobe vzethdev
You might want to add the module to /etc/init.d/vz script, so it will be loaded during startup.
|
Note: since vzctl version 3.0.11, vzethdev is loaded by /etc/init.d/vz |
MAC addresses
In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions.
MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.
There is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this:
chmod +x easymac.sh ./easymac.sh -R
Adding veth to a VE
syntax vzctl version < 3.0.14
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
Here
- dev_name is the ethernet device name that you are creating on the host system
- dev_addr is its MAC address
- ve_dev_name is the corresponding ethernet device name you are creating on the VE
- ve_dev_addr is its MAC address
|
|
Note: that this option is incremental, so devices are added to already existing ones. |
NB there are no spaces after the commas
Example:
vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
After executing this command veth device will be created for VE 101 and veth configuration will be saved to a VE configuration file. Host-side ethernet device will have veth101.0 name and 00:12:34:56:78:9A MAC address. VE-side ethernet device will have eth0 name and 00:12:34:56:78:9B MAC address.
syntax vzctl version >= 3.0.14
Read Update infos about vzctl 3.0.14
vzctl set <VEID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]
Here
- ifname is the ethernet device name in the VE
- mac is its MAC address in the VE
- host_ifname is the ethernet device name on the host (CT0)
- host_mac is its MAC address on the host (CT0)
|
|
Note: All parameters except ifname are optional and are automatically generated if not specified. |
Example:
vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save
Removing veth from a VE
syntax vzctl version < 3.0.14
vzctl set <VEID> --veth_del <dev_name>
Here dev_name is the ethernet device name in the host system.
Example:
vzctl set 101 --veth_del veth101.0 --save
After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.
syntax vzctl version >= 3.0.14
vzctl set <VEID> --netif_del <dev_name>|all
Here
dev_nameis the ethernet device name in the VE.
|
|
Note: If you want to remove all ethernet devices in VE, use all.
|
Example:
vzctl set 101 --netif_del eth0 --save
Common configurations with virtual ethernet devices
Module vzethdev must be loaded to operate with veth devices.
Simple configuration with virtual ethernet device
Start a VE
[host-node]# vzctl start 101
Add veth device to VE
old way
[host-node] ifconfig eth0 ... HWaddress 00:12:34:56:78:9B ...
[host-node] easymac.sh -R 00:12:34:56:78:9A
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
new way
[host-node]# vzctl set 101 --netif_add veth101 --save
This allocates a mac address, and associates it with the host eth0 port
Configure devices in CT0
[host-node]# ifconfig veth101.0 0 [host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp [host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
Configure device in VE
[host-node]# vzctl enter 101 [ve-101]# /sbin/ifconfig eth0 0 [ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0 [ve-101]# /sbin/ip route add default dev eth0
Add route in CT0
[host-node]# ip route add 192.168.0.101 dev veth101.0
Virtual ethernet device with IPv6
Start VE
[host-node]# vzctl start 101
Add veth device to VE
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
Configure devices in CT0
[host-node]# ifconfig veth101.0 0 [host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
Configure device in VE
[host-node]# vzctl enter 101 [ve-101]# /sbin/ifconfig eth0 0
Start router advertisement daemon (radvd) for IPv6 in CT0
First you need to edit radvd configuration file. Here is a simple example of /etc/radv.conf:
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;
prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;
prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
Then, start radvd:
[host-node]# /etc/init.d/radvd start
Add IPv6 addresses to devices in CT0
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64 [host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
Virtual ethernet devices can be joined in one bridge
Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices
Create bridge device
[host-node]# brctl addbr vzbr0
Add veth devices to bridge
[host-node]# brctl addif vzbr0 veth101.0 ... [host-node]# brctl addif vzbr0 veth101.n [host-node]# brctl addif vzbr0 veth102.0 ... ... [host-node]# brctl addif vzbr0 vethXXX.N
Configure bridge device
[host-node]# ifconfig vzbr0 0 [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
Add routes in CT0
[host-node]# ip route add 192.168.101.1 dev vzbr0 ... [host-node]# ip route add 192.168.101.n dev vzbr0 [host-node]# ip route add 192.168.102.1 dev vzbr0 ... ... [host-node]# ip route add 192.168.XXX.N dev vzbr0
Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes.
Making a veth-device persistent
At the moment, it is not possible to have the commands needed for a persistent veth being made automatically be vzctl. A bugreport ( http://bugzilla.openvz.org/show_bug.cgi?id=301 ) has already been made. Until then, here's a way to make the above steps persistent.
1. First, edit the VE's configuration to specify what the veth's IP address(es) should be, and to indicate that a custom script should be run when starting up a VE.
- Open up /etc/vz/conf/VEID.conf
- Comment out any IP_ADDRESS entries to prevent a VENET-device from being created in the VE
- Add or change the entry CONFIG_CUSTOMIZED="yes"
- Add an entry VETH_IP_ADDRESS="<VE IP>" The VE IP can have multiple IPs, separated by spaces
2. Now to create that "custom script". The following helper script will check the configuration file for IP addresses and for the veth interface, and configure the IP routing accordingly. Create the script /usr/sbin/vznetaddroute to have the following, and then chmod 0500 /usr/sbin/vznetaddroute to make it executable.
#!/bin/bash # /usr/sbin/vznetaddroute # a script to bring up virtual network interfaces (veth's) in a VE CONFIGFILE=/etc/vz/conf/$VEID.conf . $CONFIGFILE VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` if [ ! -n "$VETH_IP_ADDRESS" ]; then echo "According to $CONFIGFILE VE$VEID has no veth IPs configured." exit 1 fi if [ ! -n "$VZHOSTIF" ]; then echo "According to $CONFIGFILE VE$VEID has no veth interface configured." exit 1 fi for IP in $VETH_IP_ADDRESS; do echo "Adding interface $VZHOSTIF and route $IP for VE$VEID to CT0" /sbin/ifconfig $VZHOSTIF 0 echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding /sbin/ip route add $IP dev $VZHOSTIF done exit 0
3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddroute which you just created.
#!/bin/bash EXTERNAL_SCRIPT="/usr/sbin/vznetaddroute"
4. Of course, the VE's operating system will need to be configured with those IP address(es) as well. Consult the manual for your VE's OS for details.
That's it! At this point, when you restart the VE you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the VE and use the network.
Making a bridged veth-device persistent
Like the above example, here it is how to add the veth device to a bridge in a persistent way. vzctl doesn't offer an automatic function to do this.
1. First, edit the VE's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a VE.
- Open up /etc/vz/conf/VEID.conf
- Comment out any IP_ADDRESS entries to prevent a VENET-device from being created in the VE
- Add or change the entry CONFIG_CUSTOMIZED="yes"
- Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend.
2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then chmod 0500 /usr/sbin/vznetaddbr to make it executable.
#!/bin/bash # /usr/sbin/vznetaddbr # a script to add virtual network interfaces (veth's) in a VE to a bridge on CT0 CONFIGFILE=/etc/vz/conf/$VEID.conf . $CONFIGFILE VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` if [ ! -n "$VZHOSTIF" ]; then echo "According to $CONFIGFILE VE$VEID has no veth interface configured." exit 1 fi if [ ! -n "$VZHOSTBR" ]; then echo "According to $CONFIGFILE VE$VEID has no bridge interface configured." exit 1 fi echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for VE$VEID" /sbin/ifconfig $VZHOSTIF 0 echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding /usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF exit 0
3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created.
#!/bin/bash EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
4. Of course, the VE's operating system will need to have . Consult the manual for your VE's OS for details.
When the VE is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing brctl show
Inside the VE you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan.
Virtual ethernet devices + VLAN
This configuration can be done by adding vlan device to the previous configuration.