Virtual Ethernet device

From OpenVZ Virtuozzo Containers Wiki
Revision as of 10:39, 7 May 2007 by Kir (talk | contribs) (→‎syntax vzctl version >= 3.0.14)
Jump to: navigation, search

Virtual ethernet device is an ethernet-like device which can be used inside a VE. Unlike venet network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to ethX or other device and VE user fully sets up his networking himself, including IPs, gateways etc.

Virtual ethernet device consist of two ethernet devices - one in VE0 and another one in VE. These devices are connected to each other, so if a packet goes to one device it will come out from the other device.

Virtual ethernet device usage

Kernel module

First of all, make sure the vzethdev module is loaded: 

# lsmod | grep vzeth
vzethdev                8224  0
vzmon                  35164  5 vzethdev,vznetdev,vzrst,vzcpt
vzdev                   3080  4 vzethdev,vznetdev,vzmon,vzdquota

In case it is not loaded, load it: 

# modprobe vzethdev

You might want to add the module to /etc/init.d/vz script, so it will be loaded during startup.

Yellowpin.svg Note: since vzctl version 3.0.11, vzethdev is loaded by /etc/init.d/vz


Adding veth to a VE

Yellowpin.svg Note: Use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions and MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.

syntax vzctl version < 3.0.14

vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>

Here

  • dev_name is the ethernet device name that you are creating on the host system
  • dev_addr is its MAC address
  • ve_dev_name is the corresponding ethernet device name you are creating on the VE
  • ve_dev_addr is its MAC address
Yellowpin.svg Note: that this option is incremental, so devices are added to already existing ones.

NB there are no spaces after the commas

Example: 

vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

After executing this command veth device will be created for VE 101 and veth configuration will be saved to a VE configuration file. Host-side ethernet device will have veth101.0 name and 00:12:34:56:78:9A MAC address. VE-side ethernet device will have eth0 name and 00:12:34:56:78:9B MAC address.


syntax vzctl version >= 3.0.14

Read Update infos about vzctl 3.0.14 

vzctl set <VEID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]

Here

  • ifname is the ethernet device name in the VE
  • mac is its MAC address in the VE
  • host_ifname is the ethernet device name on the host (VE0)
  • host_mac is its MAC address on the host (VE0)
Yellowpin.svg Note: All parameters except ifname are optional and are automatically generated if not specified.

Example: 

vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save


Removing veth from a VE

syntax vzctl version < 3.0.14

vzctl set <VEID> --veth_del <dev_name>

Here dev_name is the ethernet device name in the host system.

Example: 

vzctl set 101 --veth_del veth101.0 --save

After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.


syntax vzctl version >= 3.0.14

vzctl set <VEID> --netif_del <dev_name>|all

Here

  • dev_name is the ethernet device name in the VE.
Yellowpin.svg Note: If you want to remove all ethernet devices in VE, use all.

Example: 

vzctl set 101 --netif_del eth0 --save

Common configurations with virtual ethernet devices

Module vzethdev must be loaded to operate with veth devices.

Simple configuration with virtual ethernet device

Start a VE

[host-node]# vzctl start 101

Add veth device to VE

[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

Configure devices in VE0

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

Configure device in VE

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0

Add route in VE0

[host-node]# ip route add 192.168.0.101 dev veth101.0

Virtual ethernet device with IPv6

Start VE

[host-node]# vzctl start 101

Add veth device to VE

[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

Configure devices in VE0

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

Configure device in VE

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0

Start router advertisement daemon (radvd) for IPv6 in VE0

First you need to edit radvd configuration file. Here is a simple example of /etc/radv.conf: 

interface veth101.0
{
        AdvSendAdvert on;
        MinRtrAdvInterval 3;
        MaxRtrAdvInterval 10;
        AdvHomeAgentFlag off;

        prefix 3ffe:2400:0:0::/64
        {
                AdvOnLink on;
                AdvAutonomous on;
                AdvRouterAddr off;
        };
};

interface eth0
{
        AdvSendAdvert on;
        MinRtrAdvInterval 3;
        MaxRtrAdvInterval 10;
        AdvHomeAgentFlag off;

        prefix 3ffe:0302:0011:0002::/64
        {
                AdvOnLink on;
                AdvAutonomous on;
                AdvRouterAddr off;
        };
};

Then, start radvd: 

[host-node]# /etc/init.d/radvd start

Add IPv6 addresses to devices in VE0

[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64

Virtual ethernet devices can be joined in one bridge

Perform steps 1 - 4 from Simple configuration chapter for several VEs and/or veth devices

Create bridge device

[host-node]# brctl addbr vzbr0

Add veth devices to bridge

[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N

Configure bridge device

[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp

Add routes in VE0

[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0

Thus you'll have more convinient configuration, i.e. all routes to VEs will be through this bridge and VEs can communicate with each other even without these routes.



Making a veth-device persistent

At the moment, it is not possible to have the commands needed for a persistent veth being made automatically be vzctl. A bugreport ( http://bugzilla.openvz.org/show_bug.cgi?id=301 ) has already been made. Until then, here's a way to make the above steps persistent (for a debian based system in this example).

Cleaning ${VEID}.conf

Option A)

Open up /etc/vz/conf/VEID.conf and comment out any IP_ADDRESS-entries to prevent a VENET-device from being created in the VE. Add or change the entry CONFIG_CUSTOMIZED="yes".


Option B)

Follow Option A and add in addition a VETH_IP_ADDRESS="<your VE IP>" entry to your /etc/vz/conf/VEID.conf including the IP Address you want to set.

Adding an external script to VE0

Copy and paste the following code (either Option A or B) into /usr/sbin/vznetaddroute:

Option A)
#!/bin/bash
#
# This script adds the appropriate VE0-route for veth-enabled VEs.
# See http://wiki.openvz.org/Virtual_Ethernet_device for more information.
#

# check the VEID
if [ "${VEID}" == 101 ]; then
  echo "Adding interface veth101.0 and route 192.168.0.101 for VE101 to VE0"
  /sbin/ifconfig veth101.0 0
  echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
  echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
  echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
  echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
  /sbin/ip route add 192.168.0.101 dev veth101.0
elsif [ "${VEID}" == 102 ]; then
  echo "Adding interface veth102.0 and route 192.168.0.102 for VE101 to VE0"
  /sbin/ifconfig veth101.0 0
  echo 1 > /proc/sys/net/ipv4/conf/veth102.0/forwarding
  echo 1 > /proc/sys/net/ipv4/conf/veth102.0/proxy_arp
  echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
  echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
  /sbin/ip route add 192.168.0.102 dev veth102.0
elsif [ "${VEID}" == YOUR_VE ]; then
  # same as above with the vethYOUR_VE.0 device and the appropriate ip
fi
exit

Add one elsif-section for every veth-enabled VE you'd like to have automatically configured. Remember to run 

chmod +x /usr/sbin/vznetaddroute

to make the script executable.

Option B)
#!/bin/bash

VZCONFDIR=/etc/vz

. $VZCONFDIR/conf/$VEID.conf

VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'`

if [ -n $VETH_IP_ADDRESS ]; then
        echo "Adding interface $VZHOSTIF and route $VETH_IP_ADDRESS for VE$VEID to VE0"
        /sbin/ifconfig $VZHOSTIF 0
        echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
        echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
        /sbin/ip route add $VETH_IP_ADDRESS dev $VZHOSTIF
else
        echo "found no VETH_IP_ADDRESS in  $VZCONFDIR/conf/$VEID.conf!"
        exit 1;
fi
exit

Remember to run 

chmod +x /usr/sbin/vznetaddroute

to make the script executable.

in addition you need to edit your /etc/network/interfaces and add the sysctl statements for forwardings and proxy_arp like this: 

auto eth0
iface eth0 inet static
        address 10.1.1.1
        netmask 255.255.255.0
        network 10.1.1.0
        broadcast 10.1.1.255
        gateway 10.215.1.254
        dns-nameservers 10.215.1.20
        dns-search prod.your.domain
        up sysctl -w net.ipv4.conf.eth0.proxy_arp=1
        up sysctl -w net.ipv4.conf.eth0.forwarding=1

afterwards execute both statements manualy: 

sysctl -w net.ipv4.conf.eth0.proxy_arp=1
sysctl -w net.ipv4.conf.eth0.forwarding=1

Make vzctl run the script

To make vzctl run the script, copy and paste the following line to /etc/vz/vznet.conf: 

#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddroute"

The script will now run every time a veth-enabled VE is started.

Adding a script to VE

Now we're done with VE0, we still need to add a route to the VE itself. So we start up the VE with 

vzctl start 101

, get into it with 

vzctl enter 101

and create a new file /etc/init.d/route-up in the VE with the following content: 

#!/bin/bash
/sbin/ip route add default dev eth0

Make the script executable with 

chmod +x /etc/init.d/route-up

and add it to the runlevels: 

ve101:/# update-rc.d route-up defaults
 Adding system startup for /etc/init.d/route-up ...
  /etc/rc0.d/K20route-up -> ../init.d/route-up
  [...]

Checking

Now to see if everything worked, leave the VE with 

exit

, stop the VE via 

vzctl stop 101

and restart it with 

vzctl start 101

. Still in VE0, check the route for the VE: 

ve0:/# ip route ls
192.168.0.101 dev veth101.0  scope link
[...]
ve0:/# ping 192.168.0.101 -c 4 -q
[...]
--- 192.168.0.101 ping statistics ---
4 packets transmitted, 4 recieved, 0% packet loss, time 0ms

If somethings not working, check the contents of the files we just created or changed. Now get into the VE via 

vzctl enter 101

and check the routing there: 

ve101:/# ifconfig 
eth0      Link encap:Ethernet  HWaddr 00:12:34:56:78:9B  
          inet addr:192.168.0.101  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:92 errors:0 dropped:0 overruns:0 frame:0
          TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6757 (6.5 KiB)  TX bytes:10396 (10.1 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
ve101:/# ip route ls
default dev eth0  scope link
ve101:/# ping 192.168.0.101 -c 4 -q
[...]
--- 192.168.0.101 ping statistics ---
4 packets transmitted, 4 recieved, 0% packet loss, time 0ms

If you have problems getting it persistent, please comment.

Virtual ethernet devices + VLAN

This configuration can be done by adding vlan device to the previous configuration.

See also

External links

Retrieved from "https://wiki.openvz.org/index.php?title=Virtual_Ethernet_device&oldid=3068"