20
edits
Changes
m
Virtual <!--T:2-->A virtual Ethernet device consist consists of two Ethernet devices --,the one in [[CT0]] (e.g., vethN.0) and another a corresponding one in CT(e. These devices g., eth0) that are connectedto each other, so if . If a packet goes is sent to onedevice it will come out from the other device.
{{Note|in vzctl === MAC addresses === < 3.0.11!--T:6-->The following steps to generate a MAC address are not necessary, vzethdev is not autoloaded by <code>/etc/initsince newer versionsof vzctl will automatically generate a MAC address for you.d/vz</code> script, so These steps are providedin case you have to edit it want to load this moduleset a MAC address manually.}}
=== MAC addresses ===<!--T:7-->In the below commands, you You should use a random MAC addressesaddress when adding a network interface to a container. Do not use MAC addresses of real eth devices, because this can lead to collisions.
YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl <!--T:9--veth_add>MAY GENERATE THEM AUTOMATICALLY AS NECESSARYThere is a utility script available for generating MAC addresses: https://github.com/moutai/eucalyptus-utils/blob/master/easymac.sh.It is used like this:
Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this <!--T:10--> chmod +x easymac.sh
==== syntax vzctl version <!--T:12--> 3.0.22 ==== vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>]
Or, if <!--T:17-->If you want to specify everything:
[host<!-node] easymac.sh -RT:19--> 00:12:34:56:78If you want to use independent communication through the bridge:9A
After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file.Host!--side Ethernet device will have <tt>veth101.0</tt> name and <tt>00T:12:34:56:78:9A</tt21--> MAC address.CT-side Ethernet device will have <tt>eth0</tt> name If you want to specify the bridge and <tt>00:12:34:56:78autogenerate the other values:9B</tt> MAC address.
=== Removing veth from a CT === <!--T:22-->vzctl set 101 --netif_add eth0,,,,vzbr0 --save
=== Using a directly routed IPv4 with virtual Ethernet device === ==== Situation ====<!--T:42-->
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
Thus you'll have more convinient configuration<!--T:74-->In Debian, iconfigure the network interface on [[CT0]] to plug into a bridge in /etc/network/interfaces.eThe [[CT0]] physical device is added to the bridge as the "uplink" port to the physical network. all routes You need to containers will be through have bridge-utils installed for this bridge and containers can communicate with each other even without these routesto work.
=== Making a veth<!-device persistent ===-T:78-->
==== method for vzctl version <!--T:83--> 3.0.22 ==== Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <''bridge> '' parameter of the --netif_add switch.
#!/bin/bash
The script uses 'vmbr0' as default bridge name when no bridge is specified. ==== method for vzctl version <= 3.0.22 ==== Older vzctl doesn't offer an automatic function to do this. 1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT.* Open up /etc/vz/conf/CTID.conf* Comment out any IP_ADDRESS entries to prevent a CTNET!--T:86--device from being created in the CT* Add or change the entry CONFIG_CUSTOMIZED="yes"* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend. 2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable.Or just run command
#!/bin/bash# echo 'EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr# a script to add virtual network interfaces (veth"'s) in a CT to a bridge on CT0 CONFIGFILE=> /etc/vz/conf/$VEIDvznet.conf. $CONFIGFILEVZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` if [ ! -n "$VZHOSTIF" ]; then echo "According to $CONFIGFILE CT$VEID has no veth interface configured." exit 1fi if [ ! -n "$VZHOSTBR" ]; then echo "According to $CONFIGFILE CT$VEID has no bridge interface configured." exit 1fi echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID"/sbin/ifconfig $VZHOSTIF 0/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF exit 0
3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom <!--T:87-->The script" uses 'vmbr0' as being the vznetaddbr which you just createddefault bridge name when no bridge is specified. <pre>#!/bin/bashEXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"</pre>
This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it: <pre>#!/bin/bash CONFIGFILE="/etc/vz/conf/$VEID.conf" if [ -f "$CONFIGFILE" ]then . "$CONFIGFILE" VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` export VZHOSTIF export VZHOSTBR # Fork into the background and try a few times, # until the host side of the interface appears: /bin/bash -c 'for i in 5 10 20 40 80 160 do if ifconfig -a | grep -q "$VZHOSTIF" then exec /usr/sbin/vznetaddbr else sleep "$i" fi done ' & # In the meantime, let the CT's start process continue, # or else the interface will never appear: exit 0else $0: Config file "$CONFIGFILE" does not exist. exit 1fi</pre> 4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details. When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code> Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan. === Virtual Ethernet devices + VLAN ===<!--T:88-->
no edit summary
<translate><!--T:1-->'''Virtual Ethernet device''' is an Ethernet-like device which that can be usedinside a [[container]]. Unlike a [[venet]] network device, a [[veth]] devicehas a MAC address. Therefore, therefore it can be used in more configurations, when . When vethis bridged to ethX or other device and a [[CT0]] network interface (e.g., eth0), the container can act as anindependent host on the network. The container's user fully sets can set upall of the networkinghis networking himself, including IPs, gateways , etc.
== Virtual Ethernet device usage ==<!--T:3-->
=== Kernel module ===<!--T:4-->First of all, make sure the The <code>vzethdev</code> module is should be loaded:. You can check it with the following commands.
<pre>
# lsmod | grep vzeth
</pre>
<!--T:5-->
In case it is not loaded, load it:
<pre>
</pre>
<!--T:8-->
MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.
./easymac.sh -R
=== Adding veth to a CT ===<!--T:11-->
<!--T:13-->
Here
* <tt>ifname</tt> is the Ethernet device name in the CT
* <tt>mac</tt> is its MAC address in the CT
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]])
* <tt>host_mac</tt> is its MAC address on the host ([[CT0]]), if you want independent communication with the Container through the bridge, you should explicitly specify multicast MAC address here (FE:FF:FF:FF:FF:FF).* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge.(See the reference to the vznetaddbr script below and persistent bridge configurations.)
<!--T:14-->{{Note|All parameters except <code>ifname</code> are optional and . Missing parameters, except for bridge, are automatically generated , if not specified.}}
<!--T:15-->
Example:
<!--T:16-->vzctl set 101 --netif_add eth0 --save
<!--T:18-->vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save Or, if you want to specify the bridge and leave the other values autogenerated: vzctl set 101 --netif_add eth0,,,,vmbr1 --save ==== syntax vzctl version >= 3.0.14 ==== Syntax is the same as above, but without a <bridge> parameter. ==== syntax vzctl version < 3.0.14 ==== vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr> Here * <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]]* <tt>dev_addr</tt> is its MAC address* <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT* <tt>ve_dev_addr</tt> is its MAC address {{Note|this option is incremental, so devices are added to already existing ones.}} NB there should no spaces after the commas. Example:<pre>[host-node] ifconfig eth0...HWaddress 00:12:34:56:78:9B...</pre>
<!--T:20-->vzctl set 101 --veth_add veth101.0netif_add eth0,00:12:34:56:78:9A,eth0veth101.0,00FE:12FF:34FF:56FF:78FF:9B FF,vzbr0 --save
==== syntax vzctl version >= 3.0.14 =Removing veth from a CT ===<!--T:23-->
<!--T:24-->vzctl set <CTID> --netif_del <dev_name>|all
<!--T:25-->
Here
* <code>dev_name</code> is the Ethernet device name in the [[CT]].
<!--T:26-->
{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}}
<!--T:27-->
Example:
<!--T:28-->vzctl set 101 --netif_del eth0 --save
==Common configurations with virtual Ethernet devices == syntax vzctl version < 3.0.14 ==== vzctl set <CTID> !--veth_del <dev_name> Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]]. ExampleT: vzctl set 101 29--veth_del veth101.0 --save After executing this command veth device with host-side Ethernet name<code>veth101.0</code> will be removed from CT101 and veth configurationwill be updated in CT config file. == Common configurations with virtual Ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.
=== Simple configuration with virtual Ethernet device ===<!--T:30-->
<!--T:31-->Assuming you have that 192.168.0.0/24 is being used on your LAN, you will learn the following sections show how to integrate configure a container in this for the LAN using veth.
==== Start a CT ====<!--T:32-->
<!--T:33-->[host-node]# vzctl start 101
==== Add veth device to CT ====<!--T:34-->
<!--T:35-->[host-node]# vzctl set 101 --netif_add eth0 --save
<!--T:36-->
This allocates a MAC address and associates it with the host eth0 port.
==== Configure devices in CT0 ====<!--T:37-->The following steps are needed when the [[CT]] is '''not''' bridged to a [[CT0]] network interface. That is because the [[CT]] is connected to a virtual network that is "behind" [[CT0]]. [[CT0]] must forward packets between its physical network interface and the virtual network interface where [[CT]] is located. The first step below to configure the interface is not necessary if the container has been started, since the device will have been initialized.
<pre>
[host-node]# ifconfig veth101.0 0
</pre>
==== Configure device in CT ====<!--T:38-->The following steps show an example of a quick manual configuration of the [[CT]] network interface. Typically, you would configure the network settings in /etc/network/interfaces (Debian, see below) or however it is normally configured on your distribution. You can also comment or remove the configuration for venet0, if it exists, because that device will not be used.
<pre>
[host-node]# vzctl enter 101
</pre>
<!--T:39-->
Notes:
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier
** http://openvz.org/pipermail/users/2005-November/000020.html
==== Add route in [[CT0]] ====<!--T:40-->Since [[CT0]] is acting as a router between its physical network interface and the virtual network interface of the [[CT]], we need to add a route to the [[CT]] to direct traffic to the right destination.
[host-node]# ip route add 192.168.0.101 dev veth101.0
=== Using a directly routed IPv4 with virtual Ethernet device === <!--T:41-->
Hardware Node (HN/CT0) has 192.168.0.1/24 with router 192.168.0.254.
<!--T:43-->
We also know that IPv4 10.0.0.1/32 is directly routed to 192.168.0.1 (this is called a ''fail-over IP'').
<!--T:44-->
We want to give this directly routed IPv4 address to a container (CT).
==== Start container ====<!--T:45-->
<!--T:46-->[host-node]# vzctl start 101
==== Add veth device to CT ====<!--T:47-->
<!--T:48-->[host-node]# vzctl set 101 --netif_add eth0 --save
<!--T:49-->
This allocates a MAC address and associates it with the host eth0 port.
==== Configure device and add route in CT0 ====<!--T:50-->
<!--T:51-->
<pre>
[host-node]# ifconfig veth101.0 0
</pre>
<!--T:52-->
You can automatize this at VPS creation by using a mount script <tt>$VEID.mount</tt>.
<!--T:53-->
The problem here is that the ''veth'' interface appears in CT0 '''after''' VPS has started, therefore we cannot directly use the commands in the mount script. We launch a shell script (enclosed by { }) in background (operator '''&''') that waits for the interface to be ready and then adds the IP route.
<!--T:54-->
Contents of the mount script <tt>/etc/vz/conf/101.mount</tt>:
<pre>
# This script source VPS configuration files in the same order as vzctl does
<!--T:55-->
# if one of these files does not exist then something is really broken
[ -f /etc/vz/vz.conf ] || exit 1
[ -f $VE_CONFFILE ] || exit 1
<!--T:56-->
# source both files. Note the order, it is important
. /etc/vz/vz.conf
. $VE_CONFFILE
<!--T:57-->
# Configure veth with IP after VPS has started
{
</pre>
==== Make sure IPv4 forwarding is enabled in CT0 ====<!--T:58-->
<!--T:59-->
<pre>
[host-node]# echo 1 > /proc/sys/net/ipv4/ip_forward
You can permanently set this by using <tt>/etc/sysctl.conf</tt>.
==== Configure device in CT ====<!--T:60-->
<!--T:61-->
1. Configure IP address
<!--T:62-->
2. Add gateway
<!--T:63-->
3. Add default route
<!--T:64-->
<pre>
[ve-101]# /sbin/ifconfig eth0 10.0.0.1 netmask 255.255.255.255
[ve-101]# /sbin/ip route add 192.168.0.1 dev eth0
[ve-101]# /sbin/ip route add default via 192.168.0.1
</pre>
<!--T:65-->
In a Debian container, you can configure this permanently by using <tt>/etc/network/interfaces</tt>:
<pre>
</pre>
=== Virtual Ethernet device with IPv6 ===<!--T:66-->
<!--T:67-->
See the [[VEs and HNs in same subnets]] article.
=== Independent Virtual Ethernet devices can be joined in one communication through the bridge ===<!--T:68-->Perform Bridging a [[CT]] interface to a [[CT0]] interface is the magic that allows the [[CT]] to be an independent host on the network with its own IP address, gateway, etc. [[CT0]] does not need any configuration for forwarding packets to the [[CT]] or performing proxy arp for the [[CT]] or event the routing. <!--T:69-->To manually configure a bridge and add devices to it, perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devicesusing FE:FF:FF:FF:FF:FF as a [[CT0]] veth side MAC address and then follow these steps.
==== Create bridge device ====<!--T:70-->
<pre>
[host-node]# brctl addbr vzbr0
</pre>
==== Add veth devices to bridge ====<!--T:71-->
<pre>
[host-node]# brctl addif vzbr0 veth101.0
</pre>
==== Configure bridge device ====<!--T:72-->
<pre>
[host-node]# ifconfig vzbr0 0
</pre>
==== Add routes in [[CT0]] =Automating the bridge ===<pre!--T:73-->The most convenient method is to automatically create the bridge at boot as a network interface, add the physical interface from [[host-nodeCT0]]# ip route and then add 192.168.101.1 dev vzbr0...the interface from each [[host-nodeCT]]# ip route add 192as it starts.168.101.n dev vzbr0All devices are connected to a virtual switch, and containers directly access the network just as any other host without additional configuration on [[host-nodeCT0]# ip route add 192.168.102.1 dev vzbr0......[host-node]# ip route add 192.168.XXX.N dev vzbr0</pre>
<!--T:75-->
The bridge forwarding delay is set to 0 seconds so that forwarding begins immediately when a new interface is added to a bridge. The default delay is 30 seconds, during which the bridge pauses all traffic to listen and figure out where devices are. This can interrupt services when a container is added to the bridge. If you aren't running the spanning tree protocol (off by default) and the bridge does not create a loop in your network, then there is no need for a forwarding delay.
<pre>
iface eth0 inet manual
<!--T:76-->
auto vzbr0
iface vzbr0 inet static
bridge_ports eth0
bridge_fd 0
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.254
</pre>
Follow the steps below for making a veth bridge persistent with the included script. That will automatically add each container to the bridge when it is started. Finally, specify vzbr0 as the bridge when adding the network interface to a container, as describe above. No configuration is needed on [[CT0]] for forwarding packets, proxy arp or additional routes. The interface in each [[CT]] can be configured as desired. Everything "just works" according to normal network interface configuration and default routing rules. Note that as discussed in the troubleshooting section below, bridged packets by default pass through the FORWARD iptables chain. Take care when adding rules to that table that bridged packets are not mistakenly blocked. This behavior can be disabled, if desired (sysctl: <code>net.bridge.bridge-nf-call-iptables</code>).
=== Making a veth-device persistent === <!--T:77-->
These steps are no longer necessary, as the veth device is automatically created when the container is started. They remain here as a reference.
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced"
<!--T:79-->
See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section.
<!--T:80-->
That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network.
=== Making a bridged veth-device persistent ===<!--T:81-->
<!--T:82-->
Like the above example, here it is how to add the veth device to a bridge in a persistent way.
<!--T:84-->
Just create /etc/vz/vznet.conf containing the following.
<!--T:85-->
<pre>
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>
<pre>
</pre>
This configuration can be done by adding vlan device to the previous configuration.
== See also ==<!--T:89-->
* [[Virtual network device]]
* [[Differences between venet and veth]]
* Troubleshooting: [[Bridge doesn't forward packets]]
== External links ==<!--T:90-->
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]
* [http://viresosysadmin-ivanov.blogspot.com/2008/02/2-veth-with-2-brindgesbridges-on-openvz-at.html 2 veth with 2 bridges setup]* [https://forum.proxmox.com/threads/physical-host-with-2-nics-each-with-different-gateways.1733/#post-9287 Non default gateway for CentOS OpenVZ container] - this applies to BlueOnyx in Proxmox as well. | [[Media:TwoGWsPVECentOS.pdf|Cache]] </translate>
[[Category: Networking]]
[[Category: HOWTO]]
