Virtual Ethernet device
Virtual ethernet device is an ethernet-like device which can be used inside a VE. Unlike venet network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to ethX or other device and VE user fully sets up his networking himself, including IPs, gateways etc.
Virtual ethernet device consist of two ethernet devices - one in VE0 and another one in VE. These devices are connected to each other, so if a packet goes to one device it will come out from the other device.
Contents
Virtual ethernet device usage
Kernel module
First of all, make sure the vzethdev module is loaded:
# lsmod | grep vzeth vzethdev 8224 0 vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
In case it is not loaded, load it:
# modprobe vzethdev
You might want to add the module to /etc/init.d/vz script, so it will be loaded during startup.
|
Note: since vzctl version 3.0.11, vzethdev is loaded by /etc/init.d/vz |
Adding veth to a VE
|
|
Note: Use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions and MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. |
syntax vzctl version < 3.0.14
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
Here
- dev_name is the ethernet device name that you are creating on the host system
- dev_addr is its MAC address
- ve_dev_name is the corresponding ethernet device name you are creating on the VE
- ve_dev_addr is its MAC address
|
|
Note: that this option is incremental, so devices are added to already existing ones. |
NB there are no spaces after the commas
Example:
vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
After executing this command veth device will be created for VE 101 and veth configuration will be saved to a VE configuration file. Host-side ethernet device will have veth101.0 name and 00:12:34:56:78:9A MAC address. VE-side ethernet device will have eth0 name and 00:12:34:56:78:9B MAC address.
syntax vzctl version >= 3.0.14
Read Update infos about vzctl 3.0.14
vzctl set <VEID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]
Here
- ifname is the ethernet device name in the VE
- mac is its MAC address in the VE
- host_ifname is the ethernet device name on the host (VE0)
- host_mac is its MAC address on the host (VE0)
|
|
Note: All parameters except ifname are optional and are automatically generated if not specified. |
Example:
vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save
Removing veth from a VE
syntax vzctl version < 3.0.14
vzctl set <VEID> --veth_del <dev_name>
Here dev_name is the ethernet device name in the host system.
Example:
vzctl set 101 --veth_del veth101.0 --save
After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.
syntax vzctl version >= 3.0.14
vzctl set <VEID> --netif_del <dev_name>|<all>
Here
- dev_name is the ethernet device name in the VE.
|
|
Note: If you want to remove all ethernet devices in VE, use all. |
Example:
vzctl set 101 --netif_del eth0 --save
Common configurations with virtual ethernet devices
Module vzethdev must be loaded to operate with veth devices.
Simple configuration with virtual ethernet device
Start a VE
[host-node]# vzctl start 101
Add veth device to VE
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
Configure devices in VE0
[host-node]# ifconfig veth101.0 0 [host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp [host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
Configure device in VE
[host-node]# vzctl enter 101 [ve-101]# /sbin/ifconfig eth0 0 [ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0 [ve-101]# /sbin/ip route add default dev eth0
Add route in VE0
[host-node]# ip route add 192.168.0.101 dev veth101.0
Virtual ethernet device with IPv6
Start VE
[host-node]# vzctl start 101
Add veth device to VE
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
Configure devices in VE0
[host-node]# ifconfig veth101.0 0 [host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
Configure device in VE
[host-node]# vzctl enter 101 [ve-101]# /sbin/ifconfig eth0 0
Start router advertisement daemon (radvd) for IPv6 in VE0
First you need to edit radvd configuration file. Here is a simple example of /etc/radv.conf:
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;
prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;
prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
Then, start radvd:
[host-node]# /etc/init.d/radvd start
Add IPv6 addresses to devices in VE0
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64 [host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
Virtual ethernet devices can be joined in one bridge
Perform steps 1 - 4 from Simple configuration chapter for several VEs and/or veth devices
Create bridge device
[host-node]# brctl addbr vzbr0
Add veth devices to bridge
[host-node]# brctl addif vzbr0 veth101.0 ... [host-node]# brctl addif vzbr0 veth101.n [host-node]# brctl addif vzbr0 veth102.0 ... ... [host-node]# brctl addif vzbr0 vethXXX.N
Configure bridge device
[host-node]# ifconfig vzbr0 0 [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
Add routes in VE0
[host-node]# ip route add 192.168.101.1 dev vzbr0 ... [host-node]# ip route add 192.168.101.n dev vzbr0 [host-node]# ip route add 192.168.102.1 dev vzbr0 ... ... [host-node]# ip route add 192.168.XXX.N dev vzbr0
Thus you'll have more convinient configuration, i.e. all routes to VEs will be through this bridge and VEs can communicate with each other even without these routes.
Making a veth-device persistent
At the moment, it is not possible to have the commands needed for a persistent veth being made automatically be vzctl. A bugreport ( http://bugzilla.openvz.org/show_bug.cgi?id=301 ) has already been made. Until then, here's a way to make the above steps persistent (for a debian based system in this example).
Cleaning ${VEID}.conf
Option A)
Open up /etc/vz/conf/VEID.conf and comment out any IP_ADDRESS-entries to prevent a VENET-device from being created in the VE. Add or change the entry CONFIG_CUSTOMIZED="yes".
Option B)
Follow Option A and add in addition a VETH_IP_ADDRESS="<your VE IP>" entry to your /etc/vz/conf/VEID.conf including the IP Address you want to set.
Adding an external script to VE0
Copy and paste the following code (either Option A or B) into /usr/sbin/vznetaddroute:
Option A)
#!/bin/bash
#
# This script adds the appropriate VE0-route for veth-enabled VEs.
# See http://wiki.openvz.org/Virtual_Ethernet_device for more information.
#
# check the VEID
if [ "${VEID}" == 101 ]; then
echo "Adding interface veth101.0 and route 192.168.0.101 for VE101 to VE0"
/sbin/ifconfig veth101.0 0
echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
/sbin/ip route add 192.168.0.101 dev veth101.0
elsif [ "${VEID}" == 102 ]; then
echo "Adding interface veth102.0 and route 192.168.0.102 for VE101 to VE0"
/sbin/ifconfig veth101.0 0
echo 1 > /proc/sys/net/ipv4/conf/veth102.0/forwarding
echo 1 > /proc/sys/net/ipv4/conf/veth102.0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
/sbin/ip route add 192.168.0.102 dev veth102.0
elsif [ "${VEID}" == YOUR_VE ]; then
# same as above with the vethYOUR_VE.0 device and the appropriate ip
fi
exit
Add one elsif-section for every veth-enabled VE you'd like to have automatically configured. Remember to run
chmod +x /usr/sbin/vznetaddroute
to make the script executable.
Option B)
#!/bin/bash
VZCONFDIR=/etc/vz
VZHOSTIF=$BASH_ARGV
. $VZCONFDIR/conf/$VEID.conf
if [ -n $VETH_IP_ADDRESS ]; then
echo "Adding interface $VZHOSTIF and route $VETH_IP_ADDRESS for VE$VEID to VE0"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/sbin/ip route add $VETH_IP_ADDRESS dev $VZHOSTIF
else
echo "found no VETH_IP_ADDRESS in $VZCONFDIR/conf/$VEID.conf!"
exit 1;
fi
exit
Remember to run
chmod +x /usr/sbin/vznetaddroute
to make the script executable.
in addition you need to edit your /etc/network/interfaces and add the sysctl statements for forwardings and proxy_arp like this:
auto eth0
iface eth0 inet static
address 10.1.1.1
netmask 255.255.255.0
network 10.1.1.0
broadcast 10.1.1.255
gateway 10.215.1.254
dns-nameservers 10.215.1.20
dns-search prod.your.domain
up sysctl -w net.ipv4.conf.eth0.proxy_arp=1
up sysctl -w net.ipv4.conf.eth0.forwarding=1
afterwards execute both statements manualy:
sysctl -w net.ipv4.conf.eth0.proxy_arp=1 sysctl -w net.ipv4.conf.eth0.forwarding=1
Make vzctl run the script
To make vzctl run the script, copy and paste the following line to /etc/vz/vznet.conf:
#!/bin/bash EXTERNAL_SCRIPT="/usr/sbin/vznetaddroute"
The script will now run every time a veth-enabled VE is started.
Adding a script to VE
Now we're done with VE0, we still need to add a route to the VE itself. So we start up the VE with
vzctl start 101
, get into it with
vzctl enter 101
and create a new file /etc/init.d/route-up in the VE with the following content:
#!/bin/bash /sbin/ip route add default dev eth0
Make the script executable with
chmod +x /etc/init.d/route-up
and add it to the runlevels:
ve101:/# update-rc.d route-up defaults Adding system startup for /etc/init.d/route-up ... /etc/rc0.d/K20route-up -> ../init.d/route-up [...]
Checking
Now to see if everything worked, leave the VE with
exit
, stop the VE via
vzctl stop 101
and restart it with
vzctl start 101
. Still in VE0, check the route for the VE:
ve0:/# ip route ls 192.168.0.101 dev veth101.0 scope link [...] ve0:/# ping 192.168.0.101 -c 4 -q [...] --- 192.168.0.101 ping statistics --- 4 packets transmitted, 4 recieved, 0% packet loss, time 0ms
If somethings not working, check the contents of the files we just created or changed. Now get into the VE via
vzctl enter 101
and check the routing there:
ve101:/# ifconfig
eth0 Link encap:Ethernet HWaddr 00:12:34:56:78:9B
inet addr:192.168.0.101 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:92 errors:0 dropped:0 overruns:0 frame:0
TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6757 (6.5 KiB) TX bytes:10396 (10.1 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
ve101:/# ip route ls
default dev eth0 scope link
ve101:/# ping 192.168.0.101 -c 4 -q
[...]
--- 192.168.0.101 ping statistics ---
4 packets transmitted, 4 recieved, 0% packet loss, time 0ms
If you have problems getting it persistent, please comment.
Virtual ethernet devices + VLAN
This configuration can be done by adding vlan device to the previous configuration.