<translate><!--T:1-->'''Virtual ethernet Ethernet device''' is an ethernetEthernet-like device which that can be used inside a [[VEcontainer]]. Unlikea [[venet]] network device, a [[veth ]] device has a MAC address. Due to thisTherefore, it can be used in more configurations, when . When veth is bridged to ethX or other device and VE a [[CT0]] network interface (e.g., eth0), the container can act as anindependent host on the network. The container's user fully sets can set up his all of the networking himself, including IPs, gateways , etc.

Virtual ethernet <!--T:2-->A virtual Ethernet device consist consists of two ethernet Ethernet devices - ,one in [[VE0CT0]] (e.g., vethN.0) and another a corresponding one in VECT (e. These devices g., eth0) that are connected to each other, so if . If a packet goes is sent to onedevice it will come out from the other device.

== Virtual ethernet Ethernet device usage ==<!--T:3-->

=== Kernel module ===<!--T:4-->First of all, make sure the The <code>vzethdev</code> module is should be loaded:. You can check it with the following commands.

You might want to add the module to === MAC addresses === <code>/etc/init.d/vz script</code!--T:6-->The following steps to generate a MAC address are not necessary, so it since newer versionsof vzctl will be loaded during startupautomatically generate a MAC address for you. These steps are providedin case you want to set a MAC address manually.

{{Note|since vzctl version 3<!--T:7-->You should use a random MAC address when adding a network interface to a container.0.11Do not use MAC addresses of real eth devices, vzethdev is loaded by /etc/initbecause this can lead to collisions.d/vz}}

=== Adding veth to <!--T:9-->There is a VE ===utility script available for generating MAC addresses: https://github.com/moutai/eucalyptus-utils/blob/master/easymac.sh. It is used like this:

{{Note|Use random MAC addresses <!--T:10-->chmod +x easymac.sh . Do not use MAC addresses of real eth devices, because this can lead to collisions and MAC addresses must be entered in XX:XX:XX:XX:XX:XX format/easymac.}}sh -R

==== syntax vzctl version < 3.0.14 =Adding veth to a CT ===<!--T:11-->

<pre!--T:12-->vzctl set <VEIDCTID> --veth_add netif_add <dev_nameifname>[,<dev_addrmac>,<ve_dev_namehost_ifname>,<ve_dev_addrhost_mac>,</prebridge>]

<!--T:13-->Here * <tt>dev_nameifname</tt> is the ethernet Ethernet device name that you are creating on in the [[VE0|host system]]CT* <tt>dev_addrmac</tt> is its MAC addressin the CT* <tt>ve_dev_namehost_ifname</tt> is the corresponding ethernet Ethernet device name you are creating on the VEhost ([[CT0]])* <tt>ve_dev_addrhost_mac</tt> is its MAC addresson the host ([[CT0]]), if you want independent communication with the Container through the bridge, you should explicitly specify multicast MAC address here (FE:FF:FF:FF:FF:FF).* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. (See the reference to the vznetaddbr script below and persistent bridge configurations.)

<!--T:14-->{{Note| that this option is incrementalAll parameters except <code>ifname</code> are optional. Missing parameters, except for bridge, so devices are added to already existing onesautomatically generated, if not specified.}} NB there are no spaces after the commas

<pre>vzctl set 101 !--veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78T:9B 16--save</pre>After executing this command <tt>veth</tt> device will be created for VE vzctl set 101 and veth configuration will be saved to a VE configuration file.Host-side ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.VE-side ethernet device will have <tt>netif_add eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.  ==== syntax vzctl version >= 3.0.14 ==== Read Update infos about [http://openvz.org/news/updates/vzctl-3.0.14-1 vzctl 3.0.14] <pre>vzctl set <VEID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]</pre> Here* <tt>ifname</tt> is the ethernet device name in the VE* <tt>mac</tt> is its MAC address in the VE* <tt>host_ifname</tt> is the ethernet device name on the host ([[VE0]])* <tt>host_mac</tt> is its MAC address on the host ([[VE0]]) {{Note|All parameters except ifname are optional and are automatically generated if not specified.}}save

Example<!--T:17-->If you want to specify everything:

<pre!--T:18-->

==== syntax vzctl version < 3.0.14 ==== <pre>vzctl set <VEID> !--T:21--veth_del <dev_name></pre>Here <tt>dev_name</tt> is If you want to specify the ethernet device name in bridge and autogenerate the [[VE0|host system]].other values:

Example <!--T:<pre22-->vzctl set 101 --veth_del veth101.0 netif_add eth0,,,,vzbr0 --save</pre>After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.

==== syntax vzctl version >= 3.0.14 ==== <pre!--T:24-->vzctl set <VEIDCTID> --netif_del <dev_name>|<all></pre>

* <ttcode>dev_name</ttcode> is the ethernet Ethernet device name in the [[VECT]].

<!--T:26-->{{Note|If you want to remove all ethernet Ethernet devices in VECT, use <code>all</code>.}}

<pre!--T:28-->

== Common configurations with virtual ethernet devices ==Module <tt!--T:31-->vzethdev<Assuming that 192.168.0.0/tt> must be loaded 24 is being used on your LAN, the following sections show how to operate with configure a container for the LAN using veth devices.

=== Simple configuration with virtual ethernet device =Start a CT ==== <!--T:32-->

==== Start a VE ==== <pre!--T:33-->

==== Add veth device to VE CT ====<!--T:34-->  <pre!--T:35-->[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,netif_add eth0,00:12:34:56:78:9B --save </pre!--T:36-->This allocates a MAC address and associates it with the host eth0 port.

==== Configure devices in VE0 CT0 ====<!--T:37-->The following steps are needed when the [[CT]] is '''not''' bridged to a [[CT0]] network interface. That is because the [[CT]] is connected to a virtual network that is "behind" [[CT0]]. [[CT0]] must forward packets between its physical network interface and the virtual network interface where [[CT]] is located. The first step below to configure the interface is not necessary if the container has been started, since the device will have been initialized.

==== Configure device in VE CT ====<!--T:38-->The following steps show an example of a quick manual configuration of the [[CT]] network interface. Typically, you would configure the network settings in /etc/network/interfaces (Debian, see below) or however it is normally configured on your distribution. You can also comment or remove the configuration for venet0, if it exists, because that device will not be used.

<!--T:39-->Notes:* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier* 192.168.0.101 is chosen to be an [[unrouteable private ip address]]. Where 101 reminds you that it is node 101.* The "ip route" tells all traffic to head to "device eth0"* In theory you could [[use dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it** http://openvz.org/pipermail/users/2005-November/000020.html ==== Add route in [[VE0CT0]] ====<pre!--T:40-->Since [[CT0]] is acting as a router between its physical network interface and the virtual network interface of the [[CT]], we need to add a route to the [[CT]] to direct traffic to the right destination. [host-node]# ip route add 192.168.0.101 dev veth101.0 === Using a directly routed IPv4 with virtual Ethernet device === <!--T:41--> ==== Situation ==== <!--T:42-->Hardware Node (HN/CT0) has 192.168.0.1/pre24 with router 192.168.0.254. <!--T:43-->We also know that IPv4 10.0.0.1/32 is directly routed to 192.168.0.1 (this is called a ''fail-over IP''). <!--T:44-->We want to give this directly routed IPv4 address to a container (CT).

=== Virtual ethernet device with IPv6 =Start container ==== <!--T:45-->

==== Start [[VE]] ==== <pre!--T:46-->

==== Add veth device to CT ==== <!--T:47-->  <!--T:48-->[[VE]host-node] # vzctl set 101 --netif_add eth0 --save <!--T:49-->This allocates a MAC address and associates it with the host eth0 port. ==== Configure device and add route in CT0 ====<!--T:50--> <!--T:51-->

[host-node]# vzctl set 101 ifconfig veth101.0 0[host--veth_add node]# ip route add 10.0.0.1 dev veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices <!--T:52-->You can automatize this at VPS creation by using a mount script <tt>$VEID.mount</tt>. <!--T:53-->The problem here is that the ''veth'' interface appears in [[VE0]] ====CT0 '''after''' VPS has started, therefore we cannot directly use the commands in the mount script. We launch a shell script (enclosed by { }) in background (operator '''&''') that waits for the interface to be ready and then adds the IP route. <!--T:54-->Contents of the mount script <tt>/etc/vz/conf/101.mount</tt>:

[host#!/bin/bash# This script source VPS configuration files in the same order as vzctl does <!--T:55--node]># ifconfig veth101.0 0if one of these files does not exist then something is really broken[host-node]# echo 1 > f /procetc/sys/net/ipv6vz/vz.conf/veth101.0/forwarding] || exit 1[host-nodef $VE_CONFFILE ]# echo || exit 1  <!--T:56--> # source both files. Note the order, it is important. /procetc/sys/net/ipv6vz/vz.conf/eth0/forwarding[host. $VE_CONFFILE <!--T:57--node]># echo Configure veth with IP after VPS has started{ IP=X.Y.Z.T DEV=veth101.0 while sleep 1 > ; do /procsbin/sysifconfig $DEV 0 >/netdev/ipv6null 2>&1 if [ $? -eq 0 ]; then /confsbin/all/forwardingip route add $IP dev $DEV break fi done} &

==== Configure device Make sure IPv4 forwarding is enabled in [[VE]] CT0 ====<!--T:58--> <!--T:59-->

[host-node]# vzctl enter 101echo 1 > /proc/sys/net/ipv4/ip_forward[vehost-101node]# echo 1 > /sbinproc/sys/net/ipv4/conf/ifconfig eth0 /forwarding[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding

==== Start router advertisement daemon (radvd) for IPv6 Configure device in VE0 CT ====First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>!--T:<pre60-->interface veth101.0{ AdvSendAdvert on; MinRtrAdvInterval 3; MaxRtrAdvInterval 10; AdvHomeAgentFlag off;

prefix 3ffe<!--T:2400:0:0::/64 { AdvOnLink on; AdvAutonomous on; AdvRouterAddr off; };61-->};1. Configure IP address

prefix 3ffe<!--T:0302:0011:0002::/64 { AdvOnLink on; AdvAutonomous on; AdvRouterAddr off;63--> };};</pre>3. Add default route

Then, start radvd<!--T:64-->

[hostve-node101]# /etcsbin/initifconfig eth0 10.d0.0.1 netmask 255.255.255.255[ve-101]# /sbin/ip route add 192.168.0.1 dev eth0[ve-101]# /radvd startsbin/ip route add default via 192.168.0.1

[host-node]# ip addr add dev veth101auto eth0iface eth0 inet static address 10.0.0 3ffe:2400::212:34ff:fe56:789a.1 netmask 255.255.255.255 up /sbin/64[host-node]# ip addr route add 192.168.0.1 dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455 up /sbin/64ip route add default via 192.168.0.1

=== Virtual ethernet devices can be joined Ethernet device with IPv6 === <!--T:66--> <!--T:67-->See the [[VEs and HNs in one same subnets]] article. === Independent Virtual Ethernet communication through the bridge ===<!--T:68-->Perform steps 1 - 4 from Simple Bridging a [[CT]] interface to a [[CT0]] interface is the magic that allows the [[CT]] to be an independent host on the network with its own IP address, gateway, etc. [[CT0]] does not need any configuration chapter for several VEs and/forwarding packets to the [[CT]] or performing proxy arp for the [[CT]] or veth devicesevent the routing.

<!--T:69-->To manually configure a bridge and add devices to it, perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices using FE:FF:FF:FF:FF:FF as a [[CT0]] veth side MAC address and then follow these steps. ==== Create bridge device ====<!--T:70-->

==== Add veth devices to bridge ====<!--T:71-->

==== Configure bridge device ====<!--T:72-->

Thus you'll have more convinient configuration=== Automating the bridge === <!--T:73-->The most convenient method is to automatically create the bridge at boot as a network interface, iadd the physical interface from [[CT0]] and then add the interface from each [[CT]] as it starts.e. all routes All devices are connected to VEs will be through this bridge a virtual switch, and VEs can communicate with each containers directly access the network just as any other even host without these routesadditional configuration on [[CT0]].

<!--T:75--> === Making The bridge forwarding delay is set to 0 seconds so that forwarding begins immediately when a veth-device persistent ===At the moment, it new interface is not possible added to have the commands needed for a persistent veth being made automatically be vzctlbridge. A bugreport ( http://bugzilla.openvz.org/show_bug.cgi?id=301 ) has already been made. Until thenThe default delay is 30 seconds, here's a way during which the bridge pauses all traffic to make the above steps persistent (for a debian based system in this example). ==== Cleaning ${VEID}.conf =========Option A)=====Open up /etc/vz/conf/VEID.conf listen and comment figure out any IP_ADDRESS-entries where devices are. This can interrupt services when a container is added to prevent a VENET-device from being created in the VEbridge. Add or change If you aren't running the entry CONFIG_CUSTOMIZED="yes".  =====Option Bspanning tree protocol (off by default)=====Follow Option A and add the bridge does not create a loop in addition your network, then there is no need for a VETH_IP_ADDRESS="<your VE IP>" entry to your /etc/vz/conf/VEID.conf including the IP Address you want to setforwarding delay. ====Adding an external script to VE0 ====Copy and paste the following code (either Option A or B) into /usr/sbin/vznetaddroute:=====Option A)=====

# check the VEIDif [ "${VEID}" == 101 ]; then echo "Adding interface veth101.0 and route 192.168.0.101 for VE101 to VE0" /sbin/ifconfig veth101.0 0 echo 1 <!--T:76--> /proc/sys/net/ipv4/conf/veth101.0/forwarding echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arpauto vzbr0 echo 1 > /proc/sys/net/ipv4/conf/eth0/forwardingiface vzbr0 inet static echo 1 > /proc/sys/net/ipv4/conf/ bridge_ports eth0/proxy_arp /sbin/ip route add 192.168.0.101 dev veth101. bridge_fd 0elsif [ "${VEID}" == 102 ]; then echo "Adding interface veth102.0 and route address 192.168.01.102 for VE101 to VE0"100 /sbin/ifconfig veth101 netmask 255.0 0 echo 1 > /proc/sys/net/ipv4/conf/veth102255.0/forwarding echo 1 > /proc/sys/net/ipv4/conf/veth102255.0/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp /sbin/ip route add gateway 192.168.0.102 dev veth102.0elsif [ "${VEID}" == YOUR_VE ]; then # same as above with the vethYOUR_VE1.0 device and the appropriate ipfiexit254

Add one elsif-section for every veth-enabled VE you'd like to have automatically configured. Remember to run <pre>chmod +x /usr/sbin/vznetaddroute</pre> to make the script executable.====Making a veth-device persistent =Option B)=====<pre!--T:77-->#!/bin/bashThese steps are no longer necessary, as the veth device is automatically created when the container is started. They remain here as a reference.

VZCONFDIR<!--T:78-->According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/etc/vz--veth_del are introduced"

<!--T:79-->See http://wiki.openvz. $VZCONFDIRorg/confw/$VEIDindex.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section.conf

VZHOSTIF=`echo $NETIF |sed <!--T:80-->That's/^.*host_ifname=\(it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added.*\)And you should be able to ping the host,and to enter the CT and use the network.*$/\1/g'`

if [ === Making a bridged veth-n $VETH_IP_ADDRESS ]; then echo "Adding interface $VZHOSTIF and route $VETH_IP_ADDRESS for VE$VEID to VE0" /sbin/ifconfig $VZHOSTIF 0 echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding /sbin/ip route add $VETH_IP_ADDRESS dev $VZHOSTIFelse echo "found no VETH_IP_ADDRESS in $VZCONFDIR/conf/$VEID.confdevice persistent === <!" exit 1;fiexit</pre>Remember to run <pre>chmod +x /usr/sbin/vznetaddroute</pre--T:81--> to make the script executable.

in addition you need to edit your /etc/network/interfaces and add the sysctl statements for forwardings and proxy_arp like this:<pre>auto eth0iface eth0 inet static address 10.1.1.1 netmask 255.255.255.0 network 10.1.1.0 broadcast 10.1.1.255 gateway 10.215.1.254 dns!--nameservers 10.215.1.20 dnsT:82-search prod.your.domain up sysctl -w net.ipv4.conf.eth0.proxy_arp=1> up sysctl -w net.ipv4.confLike the above example, here it is how to add the veth device to a bridge in a persistent way.eth0.forwarding=1</pre>

afterwards execute both statements manualy<!--T:<pre83-->sysctl vzctl includes a 'vznetaddbr' script, which makes use of the ''bridge'' parameter of the -w net.ipv4.conf.eth0.proxy_arp=1sysctl -w netnetif_add switch.ipv4.conf.eth0.forwarding=1</pre>

====Make vzctl run <!--T:84-->Just create /etc/vz/vznet.conf containing the script====following.

To make vzctl run the script, copy and paste the following line to /etc/vz/vznet.conf<!--T:85-->

#!/bin/bashEXTERNAL_SCRIPT="/usr/sbin/vznetaddroutevznetaddbr"

Now we're done with VE0, we still need to add a route to the VE itself. So we start up the VE with <pre>vzctl start 101</pre>, get into it with <pre>vzctl enter 101</pre> and create a new file /etc/init.d/route!--up in the VE with the following contentT:86-->Or just run command

#!/binecho 'EXTERNAL_SCRIPT="/bashusr/sbin/ip route add default dev eth0</pre> Make the script executable with <pre>chmod +x /etc/init.d/route-up</prevznetaddbr"' > and add it to the runlevels:<pre>ve101:/# update-rc.d route-up defaults Adding system startup for /etc/init.d/route-up ... /etc/rc0.d/K20route-up -> ../init.d/route-up [...]</pre> ==== Checking ====Now to see if everything worked, leave the VE with <pre>exit</pre>, stop the VE via <pre>vzctl stop 101</pre> and restart it with <pre>vzctl start 101</pre>. Still in VE0, check the route for the VE:<pre>ve0:/# ip route ls192.168.0.101 dev veth101.0 scope link[...]ve0:/# ping 192.168.0.101 -c 4 -q[...]--- 192.168.0.101 ping statistics ---4 packets transmitted, 4 recieved, 0% packet loss, time 0ms</pre> If somethings not working, check the contents of the files we just created or changed. Now get into the VE via <pre>vzctl enter 101</pre> and check the routing there:<pre>ve101:/# ifconfig eth0 Link encap:Ethernet HWaddr 00:12:34:56:78:9B inet addr:192.168.0.101 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:92 errors:0 dropped:0 overruns:0 frame:0 TX packets:94 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:6757 (6.5 KiB) TX bytes:10396 (10.1 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)ve101:/# ip route lsdefault dev eth0 scope linkve101:vz/# ping 192.168.0.101 -c 4 -q[...]--- 192.168.0vznet.101 ping statistics ---4 packets transmitted, 4 recieved, 0% packet loss, time 0msconf

If you have problems getting it persistent, please comment<!--T:87-->The script uses 'vmbr0' as default bridge name when no bridge is specified.

=== Virtual ethernet Ethernet devices + VLAN ===<!--T:88-->

== See also ==<!--T:89-->

== External links ==<!--T:90-->