OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Monitoring openvz resources using yabeda

2010-01-29T11:45:53Z

Lithium: /* What's yabeda, after all? */

[[Category: Monitoring]]

== What's yabeda, after all? ==

Yabeda is an OpenVZ failcnt complainer which tends to be lightweight, flexible and easily extendable.

Should be used on host machines (via some cron-job) to generate alerts when failcnt gets increased. Failcnt is the counter used in openvz kernels to tell whether the needed parameter reached its limit.

The program is being written using Ruby aiming to have as little dependancies as possible.

* Sends mails once any of your VE actually reached the barrier.
* Some other stuff is planned, check out the [http://yabeda.cryo.net.ru project home page]

VPN via the TUN/TAP device

2009-09-30T12:46:38Z

Lithium: /* Kernel TUN/TAP support */

This article describes how to use VPN via the TUN/TAP device inside a [[container]].

== Kernel TUN/TAP support ==
OpenVZ supports VPN inside a container via kernel TUN/TAP module and device.
To allow container #101 to use the TUN/TAP device the following should be done:

Make sure the '''tun''' module has been already loaded on the [[hardware node]]:
<pre>
# lsmod | grep tun
</pre>

If it is not there, use the following command to load '''tun''' module:
<pre>
# modprobe tun
</pre>

To make sure that '''tun''' module will be automatically loaded on every reboot you can also add it or into /etc/modules.conf (on RHEL see /etc/sysconfig/modules/ directory) or into /etc/sysconfig/vz-scripts/''CTID''.mount. (echo 'modprobe tun' >> /etc/sysconfig/vz-scripts/''CTID''.mount)

== Granting container an access to TUN/TAP ==
Allow your container to use the tun/tap device by running the following commands on the host node:

vzctl set 101 --devices c:10:200:rw --save
vzctl set 101 --capability net_admin:on --save

And create the character device file inside the container (execute the following on the host node):

vzctl exec 101 mkdir -p /dev/net
vzctl exec 101 mknod /dev/net/tun c 10 200
vzctl exec 101 chmod 600 /dev/net/tun

== Configuring VPN inside container ==
After the configuration steps above are done it is possible to use VPN software working with TUN/TAP inside
container just like on a usual standalone linux box.

The following software can be used for VPN with TUN/TAP:
* Virtual TUNnel (http://vtun.sourceforge.net)
* OpenVPN (http://openvpn.sourceforge.net)

== Troubleshooting ==
If NAT is needed within the VE, this error will occur on attempts to use NAT:

# iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o venet0 -j MASQUERADE
iptables v1.4.3.2: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

The solution is given here:

http://kb.parallels.com/en/5228

Also see page 69-70 of:

http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf

Note that the above steps do not solve the problem if a gentoo VE sits on a Centos HN; it's still an unsolved mystery.

== External links ==
* [http://vtun.sourceforge.net Virtual TUNnel]
* [http://openvpn.sourceforge.net OpenVPN]

[[Category: HOWTO]]
[[Category: Networking]]

VPN via the TUN/TAP device

2009-09-30T12:42:12Z

Lithium: add notes about autoload tun module

This article describes how to use VPN via the TUN/TAP device inside a [[container]].

== Kernel TUN/TAP support ==
OpenVZ supports VPN inside a container via kernel TUN/TAP module and device.
To allow container #101 to use the TUN/TAP device the following should be done:

Make sure the '''tun''' module has been already loaded on the [[hardware node]]:
<pre>
# lsmod | grep tun
</pre>

If it is not there, use the following command to load '''tun''' module:
<pre>
# modprobe tun
</pre>

To make sure that '''tun''' module will be automatically loaded on every reboot you can also add it or into /etc/modules.conf (on RHEL: echo 'modprobe tun' >> /etc/sysconfig/modules/tun.modules) or into /etc/sysconfig/vz-scripts/''VEID''.mount. (echo 'modprobe tun' >> /etc/sysconfig/vz-scripts/''VEID''.mount)

== Granting container an access to TUN/TAP ==
Allow your container to use the tun/tap device by running the following commands on the host node:

vzctl set 101 --devices c:10:200:rw --save
vzctl set 101 --capability net_admin:on --save

And create the character device file inside the container (execute the following on the host node):

vzctl exec 101 mkdir -p /dev/net
vzctl exec 101 mknod /dev/net/tun c 10 200
vzctl exec 101 chmod 600 /dev/net/tun

== Configuring VPN inside container ==
After the configuration steps above are done it is possible to use VPN software working with TUN/TAP inside
container just like on a usual standalone linux box.

The following software can be used for VPN with TUN/TAP:
* Virtual TUNnel (http://vtun.sourceforge.net)
* OpenVPN (http://openvpn.sourceforge.net)

== Troubleshooting ==
If NAT is needed within the VE, this error will occur on attempts to use NAT:

# iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o venet0 -j MASQUERADE
iptables v1.4.3.2: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

The solution is given here:

http://kb.parallels.com/en/5228

Also see page 69-70 of:

http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf

Note that the above steps do not solve the problem if a gentoo VE sits on a Centos HN; it's still an unsolved mystery.

== External links ==
* [http://vtun.sourceforge.net Virtual TUNnel]
* [http://openvpn.sourceforge.net OpenVPN]

[[Category: HOWTO]]
[[Category: Networking]]

OS template cache preparation

2009-08-21T08:31:11Z

Lithium: add note about locale setup

This article describes the procedure of an OS template cache creation. It assumes you already have OpenVZ installed and running. The steps needed to achieve it are documented in the [[Quick installation]] document.

== Terminology ==

Please make sure you understand the following terms.

{{:OS template}}
{{:OS template metadata}}
{{:OS template cache}}

== Creating an OS template cache ==
You can create an [[OS template cache]] using template utilities and [[OS template metadata]] right on your [[hardware node]]. The process is automated and will take from about 10 minutes to a few hours, depending on the network speed, and the result will be most up-to-date template cache.

=== Installing template utilities ===

You have to install a few packages in order to be able to create/update OS template cache(s).

==== Using yum ====
<pre>
# yum install vzpkg vzyum vzrpm43-python vzrpm44-python vzctl-lib
</pre>

==== Using rpm ====
Packages are available from [http://openvz.org/download/template/utils/ Download » Templates » Utilities]. You need both <tt>vzpkg</tt> and <tt>vzyum</tt> packages, as well as one or both <tt>vzrpm43</tt>/<tt>vzrpm44</tt> (including their <tt>-python</tt> counterparts), depending on the OS templates being used.

Install these utilities using rpm:

<pre>
# rpm -ihv vzpkg*.rpm vzyum*.rpm vzrpm44*.rpm
</pre>

In Red Hat Enterprise Linux, to install <tt>vzyum</tt> you will need <tt>[http://rpmfind.net/linux/rpm2html/search.php?query=python-elementtree&system=redhat python-elementtree]</tt>, <tt>[http://rpmfind.net/linux/rpm2html/search.php?query=python-sqlite&system=redhat python-sqlite]</tt>, and <tt>[http://rpmfind.net/linux/rpm2html/search.php?query=python-urlgrabber&system=redhat python-urlgrabber]</tt>. These packages might have dependencies of their own. For example, <tt>python-sqlite</tt> needs <tt>[http://rpmfind.net/linux/rpm2html/search.php?query=sqlite&system=redhat sqlite]</tt>.

=== Installing OS template metadata ===

To create an [[OS template cache]], you need to get the [[OS template metadata|metadata]] for the chosen OS template(s).

==== Using yum ====
To see which templates are available, run
<pre>
# yum search vztmpl
</pre>

To install some of the templates, run
<pre>
# yum install vztmpl-XXX [...]
</pre>

==== Using rpm ====
Get the chosen <tt>vztmpl-*</tt> packages from [http://openvz.org/download/template/metadata/ Downloads » Templates » Metadata] (or directly from [http://download.openvz.org/template/metadata/ download.openvz.org/template/metadata] or one of the [[Download mirrors|mirrors]] and install them:

<pre>
# rpm -ihv vztmpl-*.rpm
</pre>

=== Running <tt>vzpkgcache</tt> ===

Run the <tt>vzpkgcache</tt> utility; see the vzpkgcache(8) man page for details. It will create or update the caches of all the templates for which the corresponding metadata exist. Before cache creation be sure that you are set up all necessary locales for rpm (see [[Locales inside VE]]).

# vzpkgcache centos-4-i386-minimal

== Alternative: use precreated template cache ==

As an alternative to creating a cache using template metadata, you can use precreated template cache taken from [http://openvz.org/download/template/cache Downloads » Templates » Precreated], or directly from [http://download.openvz.org/template/precreated/ download.openvz.org/template/precreated], or from one of the [[Download mirrors|mirrors]].

Precreated templates can be easily updated using the following algorithm:
# create temporary [[container]] based on template
# update [[container]] using OS-specific tools (yum, apt or similar)
# pack [[container]] as a new template
Examples of this procedure are described in details at [[Updating Ubuntu template]], [[Updating Debian template]], [[Fedora template update]]

In order to use precreated template cache files, download files for chosen OS distributions and place them as-is (no unpacking needed) to the <tt>/vz/template/cache</tt> directory.

'''NOTE:''' On Ubuntu and probably recent Debian distros, the path is <tt>/var/lib/vz/template/cache</tt> if you installed OpenVZ from the repositories.

'''NOTE:''' If you use precreated CentOS-4 templates and wish to install software using vzyum, you will probably run into error like this:

[root@localhost tmp]# vzyum CTID install mypackage
[root@localhost tmp]# ERROR: No such OS template: install

This might apply to Fedora also. To fix this problem, install the appropriate [[OS template metadata]] on the OpenVZ host, for example

yum install vztmpl-centos-4

[Unverified note] : After creating a new OSTemplate cache called "centos4-i386-[ASINGLEWORD].tar.gz from a Container (on which vzyum worked) based on centos4-i386-default.tar.gz, vzyum failed to work (showing "No such OS template: install").

Here is the "solution" I worked out :
The new OSTemplate should be named "centos4-i386-[ASINGLEWORD].tar.gz", and in /vz/template/centos/4/i386/config you should copy the default.list to [ASINGLEWORD].list. vzyum then works (for me!). I have tried to verify this by creating templates with [MULTIPLEWORDS], which fail, and by creating templates without a corresponding [ASINGLEWORLD].conf file, which also fail - indicating that using a single word seems to be important (no hyphens etc). Good luck.

== Next step ==

Follow on to the [[container creation]] article.

[[Category: Installation]]
[[Category: Templates]]

Virtual network device

2009-07-29T12:30:41Z

Lithium: added notes about filtering src/dst ip-addresses

Virtual network device (<code>venet</code>) is the default network device for a [[container]]. This network device looks like a point-to-point connection between [[container]] and the [[CT0|host system]]. It does packet switching based on IP header. This is a default network device for container (an alternative is [[veth]] device).

Venet drop ip-packets '''from''' the container with a source address, and '''in''' the container with the destination address, which is not corresponding to an ip-address of the container.

Venet device is created automatically on [[container]] start. Vzctl scripts set up an appropriate IP address and other settings on venet inside a container.

== Usage ==

== Kernel module ==
First of all, check that <code>vznetdev</code> module is loaded:
<pre>
# lsmod | grep vznetdev
</pre>

If it is not, load the module:
<pre>
# modprobe vznetdev
</pre>

You might want to check /etc/init.d/vz script to make sure the module gets loaded during startup.

=== Adding IP address to a container ===
<pre>
vzctl set <CTID> --ipadd <IP1>[,<IP2>,...] [--save]
</pre>

{{Note|This option is incremental, so IP addresses are added to already existing ones.}}

==== Example ====
<pre>
vzctl set 101 --ipadd 10.0.0.1 --save
</pre>
After executing this command IP address 10.0.0.1 will be added to container 101 and IP configuration will be saved to a container configuration file.

=== Removing IP address from a container ===
<pre>
vzctl set <CTID> --ipdel <IP1>[,<IP2>,...] [--save]
vzctl set <CTID> --ipdel all [--save]
</pre>

==== Example ====
<pre>
vzctl set 101 --ipdel 10.0.0.1
</pre>
After executing this command IP address 10.0.0.1 will be removed from container 101, but IP configuration will not be changed in container config file. And after container reboot IP address 10.0.0.1 will be assigned to this container again.

== Sysctl ==

You will need to configure some sysctl parameters to get your venet devices working.
Please have a look at the [[Installation_on_Debian#sysctl]] section.

== IPv6 ==

To setup IPv6 networking with venet you'll need to enable the following in your sysctl.conf:

<code>
# IPv6 Packet Forwarding and Proxy NDP
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.proxy_ndp = 1
net.ipv6.conf.all.proxy_ndp = 1
</code>

In IPv6 you can't control forwarding per device, forwarding control has to take place in ip6tables, so all interfaces will forward IPv6 traffic.

If you enable IPv6 forwarding for your interfaces, Linux assumes your host to act like a router and will ignore 'Router Advertisments'
(see [http://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol Neighbor Discovery Protocol] or [http://mirrors.bieringer.de/Linux+IPv6-HOWTO/proc-sys-net-ipv6.html Linux IPv6 Howto]).

You will as well need to configure a new v6 default gateway for your host:

<code>
ip addr add 2620:0:2d0:1::193/64 dev eth0
route -6 add default gw 2620:0:2d0:1::1
</code>

You can add these commands to your existing network configuration on Debian/Linux:

<code>
iface eth0 inet static
address 64.131.90.7
netmask 255.255.255.240
network 64.131.90.0
broadcast 64.131.90.15
gateway 64.131.90.1
up ip addr add 2620:0:2d0:1::193/64 dev eth0
up route -6 add default gw 2620:0:2d0:1::1
down ip addr del 3620:0:2d0:1::193/64 dev eth0
</code>

== See also ==
* [[Veth]]
* [[Differences between venet and veth]]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual Ethernet device

2009-07-29T11:40:35Z

Lithium: /* Making a veth-device persistent */

'''Virtual Ethernet device''' is an Ethernet-like device which can be used
inside a [[container]]. Unlike [[venet]] network device, [[veth]] device
has a MAC address, therefore it can be used in configurations, when veth
is bridged to ethX or other device and container's user fully sets up
his networking himself, including IPs, gateways etc.

Virtual Ethernet device consist of two Ethernet devices --
the one in [[CT0]] and another one in CT. These devices are connected
to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual Ethernet device usage ==

=== Kernel module ===
First of all, make sure the <code>vzethdev</code> module is loaded:
<pre>
# lsmod | grep vzeth
vzethdev 8224 0
vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt
vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
</pre>

In case it is not loaded, load it:
<pre>
# modprobe vzethdev
</pre>

{{Note|in vzctl < 3.0.11, vzethdev is not autoloaded by <code>/etc/init.d/vz</code> script, so you have to edit it to load this module.}}

=== MAC addresses ===
In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions.

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.

YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl --veth_add
MAY GENERATE THEM AUTOMATICALLY AS NECESSARY.

Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this:

chmod +x easymac.sh
./easymac.sh -R

=== Adding veth to a CT ===

==== syntax vzctl version > 3.0.22 ====

vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>]

Here
* <tt>ifname</tt> is the Ethernet device name in the CT
* <tt>mac</tt> is its MAC address in the CT
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]])
* <tt>host_mac</tt> is its MAC address on the host ([[CT0]])
* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge.

{{Note|All parameters except <code>ifname</code> are optional and are automatically generated if not specified.}}

Example:

vzctl set 101 --netif_add eth0 --save

Or, if you want to specify everything:

vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save

Or, if you want to specify the bridge and leave the other values autogenerated:

vzctl set 101 --netif_add eth0,,,,vmbr1 --save

==== syntax vzctl version >= 3.0.14 ====

Syntax is the same as above, but without a <bridge> parameter.

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>

Here
* <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT
* <tt>ve_dev_addr</tt> is its MAC address

{{Note|this option is incremental, so devices are added to already existing ones.}}

NB there should no spaces after the commas.

Example:
<pre>
[host-node] ifconfig eth0
...
HWaddress 00:12:34:56:78:9B
...
</pre>

[host-node] easymac.sh -R
00:12:34:56:78:9A

vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file.
Host-side Ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
CT-side Ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

=== Removing veth from a CT ===

==== syntax vzctl version >= 3.0.14 ====

vzctl set <CTID> --netif_del <dev_name>|all

Here
* <code>dev_name</code> is the Ethernet device name in the [[CT]].

{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}}

Example:

vzctl set 101 --netif_del eth0 --save

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_del <dev_name>

Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]].

Example:

vzctl set 101 --veth_del veth101.0 --save

After executing this command veth device with host-side Ethernet name
<code>veth101.0</code> will be removed from CT101 and veth configuration
will be updated in CT config file.

== Common configurations with virtual Ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual Ethernet device ===

==== Start a CT ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure devices in CT0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

Notes:
* These files did not exist for me when trying ([[User:Mrjcleaver|Mrjcleaver]] 14:04, 31 May 2008 (EDT))

==== Configure device in CT ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

Notes:
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier
* 192.168.0.101 is chosen to be an [[unrouteable private ip address]]. Where 101 reminds you that it is node 101.
* The "ip route" tells all traffic to head to "device eth0"
* In theory you could [[use dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it
** http://openvz.org/pipermail/users/2005-November/000020.html

==== Add route in [[CT0]] ====

[host-node]# ip route add 192.168.0.101 dev veth101.0

=== Virtual Ethernet device with IPv6 ===

==== Start a [[CT]] ====

[host-node]# vzctl start 101

==== Add veth device to the [[CT]] ====

[host-node]# vzctl set 101 --veth_add eth0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices in the [[CT0]] ====

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

==== Configure device in [[CT]] ====

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0

==== Start router advertisement daemon (radvd) for IPv6 in CT0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:

[host-node]# /etc/init.d/radvd start

==== Add IPv6 addresses to devices in [[CT0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual Ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[CT0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes.

=== Making a veth-device persistent ===
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced"

See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section.

That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network.

=== Making a bridged veth-device persistent ===

Like the above example, here it is how to add the veth device to a bridge in a persistent way.

==== method for vzctl version > 3.0.22 ====

Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <bridge> parameter of the --netif_add switch.

Just create /etc/vz/vznet.conf containing the following.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

The script uses 'vmbr0' as default bridge name when no bridge is specified.

==== method for vzctl version <= 3.0.22 ====

Older vzctl doesn't offer an automatic function to do this.

1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT.
* Open up /etc/vz/conf/CTID.conf
* Comment out any IP_ADDRESS entries to prevent a CTNET-device from being created in the CT
* Add or change the entry CONFIG_CUSTOMIZED="yes"
* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend.

2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable.

<pre>
#!/bin/bash
# /usr/sbin/vznetaddbr
# a script to add virtual network interfaces (veth's) in a CT to a bridge on CT0

CONFIGFILE=/etc/vz/conf/$VEID.conf
. $CONFIGFILE
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE CT$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VZHOSTBR" ]; then
echo "According to $CONFIGFILE CT$VEID has no bridge interface configured."
exit 1
fi

echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF

exit 0
</pre>

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it:

<pre>
#!/bin/bash

CONFIGFILE="/etc/vz/conf/$VEID.conf"

if [ -f "$CONFIGFILE" ]
then
. "$CONFIGFILE"
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`
export VZHOSTIF
export VZHOSTBR

# Fork into the background and try a few times,
# until the host side of the interface appears:
/bin/bash -c 'for i in 5 10 20 40 80 160
do
if ifconfig -a | grep -q "$VZHOSTIF"
then
exec /usr/sbin/vznetaddbr
else
sleep "$i"
fi
done
' &

# In the meantime, let the CT's start process continue,
# or else the interface will never appear:
exit 0
else
$0: Config file "$CONFIGFILE" does not exist.
exit 1
fi
</pre>

4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details.

When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code>

Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan.

=== Virtual Ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]
* [[Using private IPs for Hardware Nodes]]
* Troubleshooting: [[Bridge doesn't forward packets]]

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]
* [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html 2 veth with 2 bridges setup]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual Ethernet device

2009-07-24T13:25:50Z

Lithium:

'''Virtual Ethernet device''' is an Ethernet-like device which can be used
inside a [[container]]. Unlike [[venet]] network device, [[veth]] device
has a MAC address, therefore it can be used in configurations, when veth
is bridged to ethX or other device and container's user fully sets up
his networking himself, including IPs, gateways etc.

Virtual Ethernet device consist of two Ethernet devices --
the one in [[CT0]] and another one in CT. These devices are connected
to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual Ethernet device usage ==

=== Kernel module ===
First of all, make sure the <code>vzethdev</code> module is loaded:
<pre>
# lsmod | grep vzeth
vzethdev 8224 0
vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt
vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
</pre>

In case it is not loaded, load it:
<pre>
# modprobe vzethdev
</pre>

{{Note|in vzctl < 3.0.11, vzethdev is not autoloaded by <code>/etc/init.d/vz</code> script, so you have to edit it to load this module.}}

=== MAC addresses ===
In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions.

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.

YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl --veth_add
MAY GENERATE THEM AUTOMATICALLY AS NECESSARY.

Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this:

chmod +x easymac.sh
./easymac.sh -R

=== Adding veth to a CT ===

==== syntax vzctl version > 3.0.22 ====

vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>]

Here
* <tt>ifname</tt> is the Ethernet device name in the CT
* <tt>mac</tt> is its MAC address in the CT
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]])
* <tt>host_mac</tt> is its MAC address on the host ([[CT0]])
* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge.

{{Note|All parameters except <code>ifname</code> are optional and are automatically generated if not specified.}}

Example:

vzctl set 101 --netif_add eth0 --save

Or, if you want to specify everything:

vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save

Or, if you want to specify the bridge and leave the other values autogenerated:

vzctl set 101 --netif_add eth0,,,,vmbr1 --save

==== syntax vzctl version >= 3.0.14 ====

Syntax is the same as above, but without a <bridge> parameter.

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>

Here
* <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT
* <tt>ve_dev_addr</tt> is its MAC address

{{Note|this option is incremental, so devices are added to already existing ones.}}

NB there should no spaces after the commas.

Example:
<pre>
[host-node] ifconfig eth0
...
HWaddress 00:12:34:56:78:9B
...
</pre>

[host-node] easymac.sh -R
00:12:34:56:78:9A

vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file.
Host-side Ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
CT-side Ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

=== Removing veth from a CT ===

==== syntax vzctl version >= 3.0.14 ====

vzctl set <CTID> --netif_del <dev_name>|all

Here
* <code>dev_name</code> is the Ethernet device name in the [[CT]].

{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}}

Example:

vzctl set 101 --netif_del eth0 --save

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_del <dev_name>

Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]].

Example:

vzctl set 101 --veth_del veth101.0 --save

After executing this command veth device with host-side Ethernet name
<code>veth101.0</code> will be removed from CT101 and veth configuration
will be updated in CT config file.

== Common configurations with virtual Ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual Ethernet device ===

==== Start a CT ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure devices in CT0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

Notes:
* These files did not exist for me when trying ([[User:Mrjcleaver|Mrjcleaver]] 14:04, 31 May 2008 (EDT))

==== Configure device in CT ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

Notes:
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier
* 192.168.0.101 is chosen to be an [[unrouteable private ip address]]. Where 101 reminds you that it is node 101.
* The "ip route" tells all traffic to head to "device eth0"
* In theory you could [[use dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it
** http://openvz.org/pipermail/users/2005-November/000020.html

==== Add route in [[CT0]] ====

[host-node]# ip route add 192.168.0.101 dev veth101.0

=== Virtual Ethernet device with IPv6 ===

==== Start a [[CT]] ====

[host-node]# vzctl start 101

==== Add veth device to the [[CT]] ====

[host-node]# vzctl set 101 --veth_add eth0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices in the [[CT0]] ====

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

==== Configure device in [[CT]] ====

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0

==== Start router advertisement daemon (radvd) for IPv6 in CT0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:

[host-node]# /etc/init.d/radvd start

==== Add IPv6 addresses to devices in [[CT0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual Ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[CT0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes.

=== Making a veth-device persistent ===
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced"

See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989 for a workaround that used to be described in this section.

That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network.

=== Making a bridged veth-device persistent ===

Like the above example, here it is how to add the veth device to a bridge in a persistent way.

==== method for vzctl version > 3.0.22 ====

Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <bridge> parameter of the --netif_add switch.

Just create /etc/vz/vznet.conf containing the following.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

The script uses 'vmbr0' as default bridge name when no bridge is specified.

==== method for vzctl version <= 3.0.22 ====

Older vzctl doesn't offer an automatic function to do this.

1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT.
* Open up /etc/vz/conf/CTID.conf
* Comment out any IP_ADDRESS entries to prevent a CTNET-device from being created in the CT
* Add or change the entry CONFIG_CUSTOMIZED="yes"
* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend.

2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable.

<pre>
#!/bin/bash
# /usr/sbin/vznetaddbr
# a script to add virtual network interfaces (veth's) in a CT to a bridge on CT0

CONFIGFILE=/etc/vz/conf/$VEID.conf
. $CONFIGFILE
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE CT$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VZHOSTBR" ]; then
echo "According to $CONFIGFILE CT$VEID has no bridge interface configured."
exit 1
fi

echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF

exit 0
</pre>

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it:

<pre>
#!/bin/bash

CONFIGFILE="/etc/vz/conf/$VEID.conf"

if [ -f "$CONFIGFILE" ]
then
. "$CONFIGFILE"
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`
export VZHOSTIF
export VZHOSTBR

# Fork into the background and try a few times,
# until the host side of the interface appears:
/bin/bash -c 'for i in 5 10 20 40 80 160
do
if ifconfig -a | grep -q "$VZHOSTIF"
then
exec /usr/sbin/vznetaddbr
else
sleep "$i"
fi
done
' &

# In the meantime, let the CT's start process continue,
# or else the interface will never appear:
exit 0
else
$0: Config file "$CONFIGFILE" does not exist.
exit 1
fi
</pre>

4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details.

When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code>

Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan.

=== Virtual Ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]
* [[Using private IPs for Hardware Nodes]]
* Troubleshooting: [[Bridge doesn't forward packets]]

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]
* [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html 2 veth with 2 bridges setup]

[[Category: Networking]]
[[Category: HOWTO]]

VPN via the TUN/TAP device

2009-07-23T08:26:18Z

Lithium:

This article describes how to use VPN via the TUN/TAP device inside a [[container]].

== Kernel TUN/TAP support ==
OpenVZ supports VPN inside a container via kernel TUN/TAP module and device.
To allow container #101 to use the TUN/TAP device the following should be done:

Make sure the '''tun''' module has been already loaded on the [[hardware node]]:
<pre>
# lsmod | grep tun
</pre>

If it is not there, use the following command to load '''tun''' module:
<pre>
# modprobe tun
</pre>

You can also add it into /etc/modules.conf to make sure it will be loaded on every reboot automatically. (On RHEL create a script /etc/sysconfig/modules/tun.modules).

== Granting container an access to TUN/TAP ==
Allow your container to use the tun/tap device by running the following commands on the host node:

vzctl set 101 --devices c:10:200:rw --save
vzctl set 101 --capability net_admin:on --save

And create the character device file inside the container (execute the following on the host node):

vzctl exec 101 mkdir -p /dev/net
vzctl exec 101 mknod /dev/net/tun c 10 200
vzctl exec 101 chmod 600 /dev/net/tun

== Configuring VPN inside container ==
After the configuration steps above are done it is possible to use VPN software working with TUN/TAP inside
container just like on a usual standalone linux box.

The following software can be used for VPN with TUN/TAP:
* Virtual TUNnel (http://vtun.sourceforge.net)
* OpenVPN (http://openvpn.sourceforge.net)

== Troubleshooting ==
If NAT is needed within the VE, this error will occur on attempts to use NAT:

# iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o venet0 -j MASQUERADE
iptables v1.4.3.2: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

The solution is given here:

http://kb.parallels.com/en/5228

Also see page 69-70 of:

http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf

Note that the above steps do not solve the problem if a gentoo VE sits on a Centos HN; it's still an unsolved mystery.

== External links ==
* [http://vtun.sourceforge.net Virtual TUNnel]
* [http://openvpn.sourceforge.net OpenVPN]

[[Category: HOWTO]]
[[Category: Networking]]