Editing Virtual Ethernet device
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | + | '''Virtual Ethernet device''' is an Ethernet-like device which can be used | |
− | + | inside a [[container]]. Unlike [[venet]] network device, [[veth]] device | |
− | '''Virtual Ethernet device''' is an Ethernet-like device | + | has a MAC address, therefore it can be used in configurations, when veth |
− | inside a [[container]]. Unlike | + | is bridged to ethX or other device and container's user fully sets up |
− | has a MAC address | + | his networking himself, including IPs, gateways etc. |
− | is bridged to | ||
− | |||
− | himself, including IPs, gateways | ||
− | + | Virtual Ethernet device consist of two Ethernet devices -- | |
− | + | the one in [[CT0]] and another one in CT. These devices are connected | |
− | one in [[CT0]] | + | to each other, so if a packet goes to one |
− | + | device it will come out from the other device. | |
− | == Virtual Ethernet device usage == | + | == Virtual Ethernet device usage == |
− | === Kernel module === | + | === Kernel module === |
− | + | First of all, make sure the <code>vzethdev</code> module is loaded: | |
<pre> | <pre> | ||
# lsmod | grep vzeth | # lsmod | grep vzeth | ||
Line 24: | Line 21: | ||
</pre> | </pre> | ||
− | |||
In case it is not loaded, load it: | In case it is not loaded, load it: | ||
<pre> | <pre> | ||
Line 30: | Line 26: | ||
</pre> | </pre> | ||
− | + | {{Note|in vzctl < 3.0.11, vzethdev is not autoloaded by <code>/etc/init.d/vz</code> script, so you have to edit it to load this module.}} | |
− | |||
− | |||
− | |||
− | + | === MAC addresses === | |
− | + | In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions. | |
− | |||
MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. | MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. | ||
− | + | YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl --veth_add | |
− | + | MAY GENERATE THEM AUTOMATICALLY AS NECESSARY. | |
− | + | Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this: | |
− | chmod +x easymac.sh | + | |
+ | chmod +x easymac.sh | ||
./easymac.sh -R | ./easymac.sh -R | ||
− | === Adding veth to a CT === | + | === Adding veth to a CT === |
− | + | ==== syntax vzctl version > 3.0.22 ==== | |
− | vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>] | + | |
+ | vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>] | ||
− | |||
Here | Here | ||
* <tt>ifname</tt> is the Ethernet device name in the CT | * <tt>ifname</tt> is the Ethernet device name in the CT | ||
* <tt>mac</tt> is its MAC address in the CT | * <tt>mac</tt> is its MAC address in the CT | ||
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]]) | * <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]]) | ||
− | * <tt>host_mac</tt> is its MAC address on the host ([[CT0]]) | + | * <tt>host_mac</tt> is its MAC address on the host ([[CT0]]) |
− | * <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. | + | * <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. |
− | + | {{Note|All parameters except <code>ifname</code> are optional and are automatically generated if not specified.}} | |
− | {{Note|All parameters except <code>ifname</code> are optional | ||
− | |||
Example: | Example: | ||
− | + | vzctl set 101 --netif_add eth0 --save | |
− | vzctl set 101 --netif_add eth0 --save | + | |
+ | Or, if you want to specify everything: | ||
+ | |||
+ | vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save | ||
+ | |||
+ | Or, if you want to specify the bridge and leave the other values autogenerated: | ||
+ | |||
+ | vzctl set 101 --netif_add eth0,,,,vmbr1 --save | ||
+ | |||
+ | ==== syntax vzctl version >= 3.0.14 ==== | ||
− | < | + | Syntax is the same as above, but without a <bridge> parameter. |
− | |||
− | + | ==== syntax vzctl version < 3.0.14 ==== | |
− | |||
− | < | + | vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr> |
− | |||
− | |||
− | |||
− | < | + | Here |
− | + | * <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]] | |
+ | * <tt>dev_addr</tt> is its MAC address | ||
+ | * <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT | ||
+ | * <tt>ve_dev_addr</tt> is its MAC address | ||
− | + | {{Note|this option is incremental, so devices are added to already existing ones.}} | |
− | |||
− | + | NB there should no spaces after the commas. | |
− | + | Example: | |
− | vzctl set <CTID> --netif_del <dev_name>|all | + | <pre> |
+ | [host-node] ifconfig eth0 | ||
+ | ... | ||
+ | HWaddress 00:12:34:56:78:9B | ||
+ | ... | ||
+ | </pre> | ||
+ | |||
+ | [host-node] easymac.sh -R | ||
+ | 00:12:34:56:78:9A | ||
+ | |||
+ | vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save | ||
+ | |||
+ | After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file. | ||
+ | Host-side Ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address. | ||
+ | CT-side Ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address. | ||
+ | |||
+ | === Removing veth from a CT === | ||
+ | |||
+ | ==== syntax vzctl version >= 3.0.14 ==== | ||
+ | |||
+ | vzctl set <CTID> --netif_del <dev_name>|all | ||
− | |||
Here | Here | ||
* <code>dev_name</code> is the Ethernet device name in the [[CT]]. | * <code>dev_name</code> is the Ethernet device name in the [[CT]]. | ||
− | |||
{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}} | {{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}} | ||
− | |||
Example: | Example: | ||
− | + | vzctl set 101 --netif_del eth0 --save | |
− | vzctl set 101 --netif_del eth0 --save | ||
− | == | + | ==== syntax vzctl version < 3.0.14 ==== |
+ | |||
+ | vzctl set <CTID> --veth_del <dev_name> | ||
+ | |||
+ | Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]]. | ||
+ | |||
+ | Example: | ||
+ | |||
+ | vzctl set 101 --veth_del veth101.0 --save | ||
+ | |||
+ | After executing this command veth device with host-side Ethernet name | ||
+ | <code>veth101.0</code> will be removed from CT101 and veth configuration | ||
+ | will be updated in CT config file. | ||
+ | |||
+ | == Common configurations with virtual Ethernet devices == | ||
Module <tt>vzethdev</tt> must be loaded to operate with veth devices. | Module <tt>vzethdev</tt> must be loaded to operate with veth devices. | ||
− | === Simple configuration with virtual Ethernet device === | + | === Simple configuration with virtual Ethernet device === |
− | + | ==== Start a CT ==== | |
− | |||
− | + | [host-node]# vzctl start 101 | |
− | |||
− | |||
− | ==== Add veth device to CT ==== | + | ==== Add veth device to CT ==== |
− | + | [host-node]# vzctl set 101 --netif_add eth0 --save | |
− | [host-node]# vzctl set 101 --netif_add eth0 --save | ||
− | |||
This allocates a MAC address and associates it with the host eth0 port. | This allocates a MAC address and associates it with the host eth0 port. | ||
− | ==== Configure devices in CT0 ==== | + | ==== Configure devices in CT0 ==== |
− | |||
<pre> | <pre> | ||
[host-node]# ifconfig veth101.0 0 | [host-node]# ifconfig veth101.0 0 | ||
Line 137: | Line 158: | ||
</pre> | </pre> | ||
− | ==== Configure device in CT ==== | + | Notes: |
− | + | * These files did not exist for me when trying ([[User:Mrjcleaver|Mrjcleaver]] 14:04, 31 May 2008 (EDT)) | |
+ | |||
+ | ==== Configure device in CT ==== | ||
<pre> | <pre> | ||
[host-node]# vzctl enter 101 | [host-node]# vzctl enter 101 | ||
Line 146: | Line 169: | ||
</pre> | </pre> | ||
− | |||
Notes: | Notes: | ||
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier | * Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier | ||
Line 154: | Line 176: | ||
** http://openvz.org/pipermail/users/2005-November/000020.html | ** http://openvz.org/pipermail/users/2005-November/000020.html | ||
− | ==== Add route in [[CT0]] ==== | + | ==== Add route in [[CT0]] ==== |
− | + | ||
[host-node]# ip route add 192.168.0.101 dev veth101.0 | [host-node]# ip route add 192.168.0.101 dev veth101.0 | ||
− | |||
− | === | + | === Virtual Ethernet device with IPv6 === |
− | |||
− | + | ==== Start a [[CT]] ==== | |
− | |||
− | + | [host-node]# vzctl start 101 | |
− | |||
− | ==== | + | ==== Add veth device to the [[CT]] ==== |
− | + | [host-node]# vzctl set 101 --veth_add eth0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save | |
− | [host-node]# vzctl | ||
− | ==== | + | ==== Configure devices in the [[CT0]] ==== |
− | + | [host-node]# ifconfig veth101.0 0 | |
− | [host-node]# | + | [host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding |
+ | [host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding | ||
+ | [host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding | ||
− | + | ==== Configure device in [[CT]] ==== | |
− | |||
− | + | [host-node]# vzctl enter 101 | |
+ | [ve-101]# /sbin/ifconfig eth0 0 | ||
− | < | + | ==== Start router advertisement daemon (radvd) for IPv6 in CT0 ==== |
+ | First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>: | ||
<pre> | <pre> | ||
− | + | interface veth101.0 | |
− | + | { | |
− | + | AdvSendAdvert on; | |
− | + | MinRtrAdvInterval 3; | |
− | + | MaxRtrAdvInterval 10; | |
− | + | AdvHomeAgentFlag off; | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | prefix 3ffe:2400:0:0::/64 | |
− | + | { | |
− | + | AdvOnLink on; | |
− | + | AdvAutonomous on; | |
+ | AdvRouterAddr off; | ||
+ | }; | ||
+ | }; | ||
− | + | interface eth0 | |
− | |||
{ | { | ||
− | + | AdvSendAdvert on; | |
− | + | MinRtrAdvInterval 3; | |
− | + | MaxRtrAdvInterval 10; | |
− | + | AdvHomeAgentFlag off; | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | prefix 3ffe:0302:0011:0002::/64 | |
− | + | { | |
− | + | AdvOnLink on; | |
− | + | AdvAutonomous on; | |
− | + | AdvRouterAddr off; | |
− | + | }; | |
− | + | }; | |
</pre> | </pre> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | Then, start radvd: | |
− | |||
− | + | [host-node]# /etc/init.d/radvd start | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | ==== Add IPv6 addresses to devices in [[CT0]] ==== | |
− | |||
<pre> | <pre> | ||
− | + | [host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64 | |
− | + | [host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64 | |
− | |||
− | |||
− | |||
− | |||
</pre> | </pre> | ||
− | === Virtual Ethernet | + | === Virtual Ethernet devices can be joined in one bridge === |
− | + | Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | ==== Create bridge device ==== | |
− | |||
− | |||
− | ==== Create bridge device ==== | ||
<pre> | <pre> | ||
[host-node]# brctl addbr vzbr0 | [host-node]# brctl addbr vzbr0 | ||
</pre> | </pre> | ||
− | ==== Add veth devices to bridge ==== | + | ==== Add veth devices to bridge ==== |
<pre> | <pre> | ||
[host-node]# brctl addif vzbr0 veth101.0 | [host-node]# brctl addif vzbr0 veth101.0 | ||
Line 293: | Line 267: | ||
</pre> | </pre> | ||
− | ==== Configure bridge device ==== | + | ==== Configure bridge device ==== |
<pre> | <pre> | ||
[host-node]# ifconfig vzbr0 0 | [host-node]# ifconfig vzbr0 0 | ||
+ | [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding | ||
+ | [host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp | ||
</pre> | </pre> | ||
− | === | + | ==== Add routes in [[CT0]] ==== |
− | + | <pre> | |
+ | [host-node]# ip route add 192.168.101.1 dev vzbr0 | ||
+ | ... | ||
+ | [host-node]# ip route add 192.168.101.n dev vzbr0 | ||
+ | [host-node]# ip route add 192.168.102.1 dev vzbr0 | ||
+ | ... | ||
+ | ... | ||
+ | [host-node]# ip route add 192.168.XXX.N dev vzbr0 | ||
+ | </pre> | ||
− | + | Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | === Making a veth-device persistent === | |
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced" | According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced" | ||
− | |||
See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section. | See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section. | ||
− | |||
That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network. | That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network. | ||
− | === Making a bridged veth-device persistent === | + | === Making a bridged veth-device persistent === |
− | |||
Like the above example, here it is how to add the veth device to a bridge in a persistent way. | Like the above example, here it is how to add the veth device to a bridge in a persistent way. | ||
− | + | ==== method for vzctl version > 3.0.22 ==== | |
− | vzctl includes a 'vznetaddbr' script, which makes use of the | + | |
+ | Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <bridge> parameter of the --netif_add switch. | ||
− | |||
Just create /etc/vz/vznet.conf containing the following. | Just create /etc/vz/vznet.conf containing the following. | ||
− | |||
<pre> | <pre> | ||
+ | #!/bin/bash | ||
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr" | EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr" | ||
</pre> | </pre> | ||
− | < | + | The script uses 'vmbr0' as default bridge name when no bridge is specified. |
− | + | ||
+ | ==== method for vzctl version <= 3.0.22 ==== | ||
+ | |||
+ | Older vzctl doesn't offer an automatic function to do this. | ||
+ | |||
+ | 1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT. | ||
+ | * Open up /etc/vz/conf/CTID.conf | ||
+ | * Comment out any IP_ADDRESS entries to prevent a CTNET-device from being created in the CT | ||
+ | * Add or change the entry CONFIG_CUSTOMIZED="yes" | ||
+ | * Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend. | ||
+ | |||
+ | 2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable. | ||
+ | |||
<pre> | <pre> | ||
− | echo 'EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr" | + | #!/bin/bash |
+ | # /usr/sbin/vznetaddbr | ||
+ | # a script to add virtual network interfaces (veth's) in a CT to a bridge on CT0 | ||
+ | |||
+ | CONFIGFILE=/etc/vz/conf/$VEID.conf | ||
+ | . $CONFIGFILE | ||
+ | VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` | ||
+ | |||
+ | if [ ! -n "$VZHOSTIF" ]; then | ||
+ | echo "According to $CONFIGFILE CT$VEID has no veth interface configured." | ||
+ | exit 1 | ||
+ | fi | ||
+ | |||
+ | if [ ! -n "$VZHOSTBR" ]; then | ||
+ | echo "According to $CONFIGFILE CT$VEID has no bridge interface configured." | ||
+ | exit 1 | ||
+ | fi | ||
+ | |||
+ | echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID" | ||
+ | /sbin/ifconfig $VZHOSTIF 0 | ||
+ | echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp | ||
+ | echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding | ||
+ | /usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF | ||
+ | |||
+ | exit 0 | ||
+ | </pre> | ||
+ | |||
+ | 3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created. | ||
+ | |||
+ | <pre> | ||
+ | #!/bin/bash | ||
+ | EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr" | ||
+ | </pre> | ||
+ | |||
+ | This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it: | ||
+ | |||
+ | <pre> | ||
+ | #!/bin/bash | ||
+ | |||
+ | CONFIGFILE="/etc/vz/conf/$VEID.conf" | ||
+ | |||
+ | if [ -f "$CONFIGFILE" ] | ||
+ | then | ||
+ | . "$CONFIGFILE" | ||
+ | VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=\(.*\),.*$/\1/g'` | ||
+ | export VZHOSTIF | ||
+ | export VZHOSTBR | ||
+ | |||
+ | # Fork into the background and try a few times, | ||
+ | # until the host side of the interface appears: | ||
+ | /bin/bash -c 'for i in 5 10 20 40 80 160 | ||
+ | do | ||
+ | if ifconfig -a | grep -q "$VZHOSTIF" | ||
+ | then | ||
+ | exec /usr/sbin/vznetaddbr | ||
+ | else | ||
+ | sleep "$i" | ||
+ | fi | ||
+ | done | ||
+ | ' & | ||
+ | |||
+ | # In the meantime, let the CT's start process continue, | ||
+ | # or else the interface will never appear: | ||
+ | exit 0 | ||
+ | else | ||
+ | $0: Config file "$CONFIGFILE" does not exist. | ||
+ | exit 1 | ||
+ | fi | ||
</pre> | </pre> | ||
− | < | + | 4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details. |
− | + | ||
+ | When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code> | ||
+ | |||
+ | Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan. | ||
− | === Virtual Ethernet devices + VLAN === | + | === Virtual Ethernet devices + VLAN === |
This configuration can be done by adding vlan device to the previous configuration. | This configuration can be done by adding vlan device to the previous configuration. | ||
− | == See also == | + | == See also == |
* [[Virtual network device]] | * [[Virtual network device]] | ||
* [[Differences between venet and veth]] | * [[Differences between venet and veth]] | ||
* [[Using private IPs for Hardware Nodes]] | * [[Using private IPs for Hardware Nodes]] | ||
− | |||
* Troubleshooting: [[Bridge doesn't forward packets]] | * Troubleshooting: [[Bridge doesn't forward packets]] | ||
− | == External links == | + | == External links == |
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd] | * [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd] | ||
− | * [http:// | + | * [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html 2 veth with 2 bridges setup] |
− | |||
− | |||
− | |||
[[Category: Networking]] | [[Category: Networking]] | ||
[[Category: HOWTO]] | [[Category: HOWTO]] |