<translate><!--T:1-->'''Virtual ethernet Ethernet device''' is an ethernetEthernet-like device which that can be used inside a [[VEcontainer]]. Unlikea [[venet]] network device, a [[veth ]] device has a MAC address. Due to thisTherefore, it can be used in more configurations, when . When veth is bridged to ethX or other device and VE a [[CT0]] network interface (e.g., eth0), the container can act as anindependent host on the network. The container's user fully sets can set up his all of the networking himself, including IPs, gateways , etc.

Virtual ethernet <!--T:2-->A virtual Ethernet device consist consists of two ethernet Ethernet devices - ,one in [[CT0]] (e.g., vethN.0) and another a corresponding one in VECT (e. These devices g., eth0) that are connected to each other, so if . If a packet goes is sent to onedevice it will come out from the other device.

== Virtual ethernet Ethernet device usage ==<!--T:3-->

=== Kernel module ===<!--T:4-->First of all, make sure the The <code>vzethdev</code> module is should be loaded:. You can check it with the following commands.

You might want to add the module to === MAC addresses === <code>/etc/init.d/vz script</code!--T:6-->The following steps to generate a MAC address are not necessary, so it since newer versionsof vzctl will be loaded during startupautomatically generate a MAC address for you. These steps are providedin case you want to set a MAC address manually.

{{Note|since vzctl version 3.0.11, vzethdev is loaded by /etc/init.d/vz}}<!--T:7--> === MAC addresses ===In the below commands, you You should use a random MAC addressesaddress when adding a network interface to a container. Do not use MAC addresses of real eth devices, because this can lead to collisions.

YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl <!--T:9--netif_add MAY GENERATE THEM AUTOMATICALLY AS NECESSARY.> Nevertheless, there There is a utility script available for generating MAC addresses: httphttps://www.easyvmxgithub.com/softwaremoutai/eucalyptus-utils/blob/master/easymac.sh. It is to be used like this:

<!--T:10-->chmod +x easymac.sh

=== Adding veth to a VE CT === ==== syntax vzctl version <!--T:11-->= 3.0.14 ====

Read Update infos about [http <!--T://openvz.org/news/updates/vzctl12-3.0.14-1 vzctl 3.0.14] <pre>vzctl set <VEIDCTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac]>,</prebridge>]

* <tt>ifname</tt> is the ethernet Ethernet device name in the VECT* <tt>mac</tt> is its MAC address in the VECT* <tt>host_ifname</tt> is the ethernet Ethernet device name on the host ([[CT0]])* <tt>host_mac</tt> is its MAC address on the host ([[CT0]]), if you want independent communication with the Container through the bridge, you should explicitly specify multicast MAC address here (FE:FF:FF:FF:FF:FF).* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. (See the reference to the vznetaddbr script below and persistent bridge configurations.)

<!--T:14-->{{Note|All parameters except <code>ifname </code> are optional and . Missing parameters, except for bridge, are automatically generated , if not specified.}}

<pre!--T:16-->vzctl set 101 --netif_add eth0</pre>--save

Or, if <!--T:17-->If you want to specify everything:

<pre!--T:18-->

==== syntax vzctl version < 3.0.14 ====!--T:19-->If you want to use independent communication through the bridge:

<pre!--T:20-->vzctl set <VEID> 101 --veth_add <dev_name>netif_add eth0,00:12:34:56:78:9A,<dev_addr>veth101.0,<ve_dev_name>FE:FF:FF:FF:FF:FF,<ve_dev_addr></pre>vzbr0 --save

Here * <tt!--T:21-->dev_name</tt> is the ethernet device name that If you are creating on the [[CT0|host system]]* <tt>dev_addr</tt> is its MAC address* <tt>ve_dev_name</tt> is want to specify the corresponding ethernet device name you are creating on bridge and autogenerate the VE* <tt>ve_dev_addr</tt> is its MAC addressother values:

{{Note| that this option is incremental, so devices are added to already existing ones.}} NB there are no spaces after the commas Example: <pre>[host!--node] ifconfig eth0...HWaddress 00:12:34:56T:78:9B...</pre> <pre>[host22-node] easymac.sh -R00:12:34:56:78:9A</pre> <pre>vzctl set 101 --veth_add veth101.0netif_add eth0,,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B vzbr0 --save</pre>After executing this command <tt>veth</tt> device will be created for VE 101 and veth configuration will be saved to a VE configuration file.Host-side ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.VE-side ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

=== Removing veth from a VE CT ===<!--T:23-->

==== syntax vzctl version >= 3.0.14 ==== <pre!--T:24-->vzctl set <VEIDCTID> --netif_del <dev_name>|all</pre>

* <code>dev_name</code> is the ethernet Ethernet device name in the [[VECT]].

<!--T:26-->{{Note|If you want to remove all ethernet Ethernet devices in VECT, use <code>all</code>.}}

<pre!--T:28-->

==== syntax vzctl version < 3.0.14 =Simple configuration with virtual Ethernet device ===<!--T:30-->

<pre>vzctl set <VEID> !--T:31--veth_del <dev_name></pre>Here <tt>dev_name</tt> is the ethernet device name in the [[CT0|host system]]Assuming that 192. Example:<pre>vzctl set 101 --veth_del veth101168.0 --save</pre>After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and /24 is being used on your LAN, the following sections show how to configure a container for the LAN using veth configuration will be updated in VE config file.

== Common configurations with virtual ethernet devices ==Module <tt>vzethdev</tt> must be loaded to operate with veth devices. === Simple configuration with virtual ethernet device === ==== Start a VE ====<pre!--T:33-->

==== Add veth device to VE CT ====<!--T:34-->

<pre!--T:35-->[host-node]# vzctl set 101 --netif_add veth101 eth0 --save</pre>

<!--T:36-->This allocates a mac MAC address, and associates it with the host eth0 port.

==== Configure devices in CT0 ====<!--T:37-->The following steps are needed when the [[CT]] is '''not''' bridged to a [[CT0]] network interface. That is because the [[CT]] is connected to a virtual network that is "behind" [[CT0]]. [[CT0]] must forward packets between its physical network interface and the virtual network interface where [[CT]] is located. The first step below to configure the interface is not necessary if the container has been started, since the device will have been initialized.

Notes:* These files did not exist for me when trying ([[User:Mrjcleaver|Mrjcleaver]] 14:04, 31 May 2008 (EDT)) ==== Configure device in VE CT ====<!--T:38-->The following steps show an example of a quick manual configuration of the [[CT]] network interface. Typically, you would configure the network settings in /etc/network/interfaces (Debian, see below) or however it is normally configured on your distribution. You can also comment or remove the configuration for venet0, if it exists, because that device will not be used.

* In theory you could [[use dhcpcd dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it** http://openvz.org/pipermail/users/2005-November/000020.html ==== Add route in [[CT0]] ==== <!--T:40-->Since [[CT0]] is acting as a router between its physical network interface and the virtual network interface of the [[CT]], we need to add a route to the [[CT]] to direct traffic to the right destination. [host-node]# ip route add 192.168.0.101 dev veth101.0 === Using a directly routed IPv4 with virtual Ethernet device === <!--T:41--> ==== Situation ==== <!--T:42-->Hardware Node (HN/CT0) has 192.168.0.1/24 with router 192.168.0.254.

==== Add route in [[CT0]] ====<pre!--T:43-->[host-node]# ip route add We also know that IPv4 10.0.0.1/32 is directly routed to 192.168.0.101 dev veth1011 (this is called a ''fail-over IP'').0 </pre!--T:44-->We want to give this directly routed IPv4 address to a container (CT).

=== Virtual ethernet device with IPv6 =Start container ==== <!--T:45-->

==== Start [[VE]] ==== <pre!--T:46-->

==== Add veth device to CT ==== <!--T:47-->  <!--T:48-->[[VE]host-node] # vzctl set 101 --netif_add eth0 --save <!--T:49-->This allocates a MAC address and associates it with the host eth0 port. ==== Configure device and add route in CT0 ====<!--T:50--> <!--T:51-->

[host-node]# vzctl set 101 ifconfig veth101.0 0[host--veth_add node]# ip route add 10.0.0.1 dev veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices <!--T:52-->You can automatize this at VPS creation by using a mount script <tt>$VEID.mount</tt>. <!--T:53-->The problem here is that the ''veth'' interface appears in [[CT0]] ===='''after''' VPS has started, therefore we cannot directly use the commands in the mount script. We launch a shell script (enclosed by { }) in background (operator '''&''') that waits for the interface to be ready and then adds the IP route. <!--T:54-->Contents of the mount script <tt>/etc/vz/conf/101.mount</tt>:

[host#!/bin/bash# This script source VPS configuration files in the same order as vzctl does <!--T:55--node]># ifconfig veth101.0 0if one of these files does not exist then something is really broken[host-node]# echo 1 > f /procetc/sys/net/ipv6vz/vz.conf/veth101.0/forwarding] || exit 1[host-nodef $VE_CONFFILE ]# echo || exit 1  <!--T:56--> # source both files. Note the order, it is important. /procetc/sys/net/ipv6vz/vz.conf/eth0/forwarding[host. $VE_CONFFILE <!--T:57--node]># echo Configure veth with IP after VPS has started{ IP=X.Y.Z.T DEV=veth101.0 while sleep 1 > ; do /procsbin/sysifconfig $DEV 0 >/netdev/ipv6null 2>&1 if [ $? -eq 0 ]; then /confsbin/all/forwardingip route add $IP dev $DEV break fi done} &

==== Configure device Make sure IPv4 forwarding is enabled in [[VE]] CT0 ====<!--T:58--> <!--T:59-->

[host-node]# vzctl enter 101echo 1 > /proc/sys/net/ipv4/ip_forward[vehost-101node]# echo 1 > /sbinproc/sys/net/ipv4/conf/ifconfig eth0 /forwarding[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding

==== Start router advertisement daemon (radvd) for IPv6 Configure device in CT0 CT ====First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>!--T:<pre60-->interface veth101.0{ AdvSendAdvert on; MinRtrAdvInterval 3; MaxRtrAdvInterval 10; AdvHomeAgentFlag off;

prefix 3ffe<!--T:2400:0:0::/64 { AdvOnLink on; AdvAutonomous on; AdvRouterAddr off; };61-->};1. Configure IP address

prefix 3ffe<!--T:0302:0011:0002::/64 { AdvOnLink on; AdvAutonomous on; AdvRouterAddr off;63--> };};</pre>3. Add default route

Then, start radvd<!--T:64-->

[hostve-node101]# /etcsbin/initifconfig eth0 10.d0.0.1 netmask 255.255.255.255[ve-101]# /sbin/ip route add 192.168.0.1 dev eth0[ve-101]# /radvd startsbin/ip route add default via 192.168.0.1

[host-node]# ip addr add dev veth101auto eth0iface eth0 inet static address 10.0.0 3ffe:2400::212:34ff:fe56:789a.1 netmask 255.255.255.255 up /sbin/64[host-node]# ip addr route add 192.168.0.1 dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455 up /sbin/64ip route add default via 192.168.0.1

=== Virtual ethernet devices can be joined in one bridge Ethernet device with IPv6 ===Perform steps 1 <!- 4 from Simple configuration chapter for several containers and/or veth devices-T:66-->

<!--T:67-->See the [[VEs and HNs in same subnets]] article. === Independent Virtual Ethernet communication through the bridge === <!--T:68-->Bridging a [[CT]] interface to a [[CT0]] interface is the magic that allows the [[CT]] to be an independent host on the network with its own IP address, gateway, etc. [[CT0]] does not need any configuration for forwarding packets to the [[CT]] or performing proxy arp for the [[CT]] or event the routing. <!--T:69-->To manually configure a bridge and add devices to it, perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices using FE:FF:FF:FF:FF:FF as a [[CT0]] veth side MAC address and then follow these steps. ==== Create bridge device ====<!--T:70-->

==== Add veth devices to bridge ====<!--T:71-->

==== Configure bridge device ====<!--T:72-->

==== Add routes in [[CT0]] =Automating the bridge ===<pre!--T:73-->The most convenient method is to automatically create the bridge at boot as a network interface, add the physical interface from [[host-nodeCT0]]# ip route and then add 192.168.101.1 dev vzbr0...the interface from each [[host-nodeCT]]# ip route add 192as it starts.168.101.n dev vzbr0All devices are connected to a virtual switch, and containers directly access the network just as any other host without additional configuration on [[host-nodeCT0]# ip route add 192.168.102.1 dev vzbr0......[host-node]# ip route add 192.168.XXX.N dev vzbr0</pre>

Thus you'll have more convinient configuration<!--T:74-->In Debian, iconfigure the network interface on [[CT0]] to plug into a bridge in /etc/network/interfaces.eThe [[CT0]] physical device is added to the bridge as the "uplink" port to the physical network. all routes You need to containers will be through have bridge-utils installed for this bridge and containers can communicate with each other even without these routesto work.

=== Making a veth<!-device persistent ===-T:78-->

<!--T:79-->See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989 #Making_a_veth-device_persistent for a workaround that used to be described in this section.

<!--T:80-->That's it! At this point, when you restart the VE CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the VE CT and use the network.

=== Making a bridged veth-device persistent ===Like the above example, here it is how to add the veth device to a bridge in a persistent way. vzctl doesn't offer an automatic function to do this.<!--T:81-->

1. First<!--T:82-->Like the above example, edit the VE's configuration here it is how to specify what is add the host bridge , and veth device to indicate that a custom script should be run when starting up bridge in a VE.* Open up /etc/vz/conf/VEID.conf* Comment out any IP_ADDRESS entries to prevent a VENET-device from being created in the VE* Add or change the entry CONFIG_CUSTOMIZED="yes"* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extendpersistent way.

2. Now to create that "custom <!--T:83-->vzctl includes a 'vznetaddbr' script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to which makes use of the ''bridge. Create '' parameter of the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable--netif_add switch.

#!/bin/bash# EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr# a script to add virtual network interfaces (veth's) in a VE to a bridge on CT0 CONFIGFILE=/etc/vz/conf/$VEID.conf. $CONFIGFILEVZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` if [ ! -n "$VZHOSTIF" ]; then echo "According to $CONFIGFILE VE$VEID has no veth interface configured." exit 1fi if [ ! -n "$VZHOSTBR" ]; then echo "According to $CONFIGFILE VE$VEID has no bridge interface configured." exit 1fi echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for VE$VEID"/sbin/ifconfig $VZHOSTIF 0echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arpecho 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF exit 0

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you <!--T:86-->Or just created.run command

#!/bin/bashecho 'EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"' > /etc/vz/vznet.conf

4. Of course, the VE<!--T:87-->The script uses 's operating system will need to have . Consult the manual for your VEvmbr0's OS for details. When the VE is started, the veth specified in the NETIF value as default bridge name when no bridge is added to the bridge specified. You can check this by doing <code>brctl show</code>

Inside the VE you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan. === Virtual ethernet Ethernet devices + VLAN ===<!--T:88-->

== See also ==<!--T:89-->

== External links ==<!--T:90-->

* [http://viresosysadmin-ivanov.blogspot.com/2008/02/2-veth-with-2-brindgesbridges-on-openvz-at.html 2 veth with 2 bridges setup]* [https://forum.proxmox.com/threads/physical-host-with-2-nics-each-with-different-gateways.1733/#post-9287 Non default gateway for CentOS OpenVZ container] - this applies to BlueOnyx in Proxmox as well. | [[Media:TwoGWsPVECentOS.pdf|Cache]] </translate>