OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Introduction to virtualization

2010-04-12T15:45:39Z

Ginkyo: 78.110.1.175 is an idiot > spam : revert to kir version

Virtualization is a framework or methodology of dividing the resources of a computer into multiple execution environments. Virtualization techniques create multiple isolated partitions — Virtual Machines (VM) or [[Virtual Environment]]s ([[VE]]s) — on a single physical server.

== Techniques ==
There are several kinds of virtualization techniques which provide similar features but differ in the degree of abstraction and the methods used for virtualization.

=== Virtual machines (VMs) ===
'''Virtual machines''' emulate some real or fictional hardware, which in turn requires real resources from the ''host'' (the machine running the VMs). This approach, used by most ''system emulators'', allows the emulator to run an arbitrary ''guest operating system'' without modifications because guest OS is not aware that it is not running on real hardware. The main issue with this approach is that some CPU instructions require additional privileges and may not be executed in user space thus requiring a ''virtual machines monitor'' (VMM) to analyze executed code and make it safe on-the-fly. Hardware emulation approach is used by [http://www.vmware.com/ VMware] products, [http://fabrice.bellard.free.fr/qemu/ QEMU], [http://parallels.com/ Parallels] and [http://www.microsoft.com/windowsserversystem/virtualserver/default.mspx Microsoft Virtual Server].

=== Paravirtualization ===
This technique also requires a VMM, but most of its work is performed in the ''guest OS'' code, which in turn is ''modified'' to support this VMM and avoid unnecessary use of privileged instructions. The paravirtualization technique also enables running different OSs on a single server, but requires them to be ported, i.e. they should «know» they are running under the hypervisor. The paravirtualization approach is used by products such as [http://www.xensource.com/xen/ Xen] and [http://user-mode-linux.sourceforge.net/ UML].

=== Virtualization on the OS level, a.k.a. containers virtualization ===
Most applications running on a server can easily share a machine with others, if they could be isolated and secured. Further, in most situations, different operating systems are not required on the same server, merely multiple instances of a single ''operating system''. OS-level virtualization systems have been designed to provide the required isolation and security to run multiple applications or copies of the same OS (but different distributions of the OS) on the same server. [http://openvz.org/ OpenVZ], [http://www.swsoft.com/products/virtuozzo Virtuozzo], [http://linux-vserver.org/ Linux-VServer], [http://www.sun.com/bigadmin/content/zones/ Solaris Zones] and [http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/jail.html FreeBSD Jails] are examples of OS-level virtualization.

== Short comparison ==
The three techniques differ in complexity of implementation, breadth of OS support, performance in comparison with standalone server, and level of access to common resources. For example, VMs have wider scope of usage, but poorer performance. Para-VMs have better performance, but can support fewer OSs because one has to modify the original OS.

Virtualization on the OS level provides the best performance and scalability compared to other approaches. Performance difference of such systems can be as low as 1…3%, comparing with that of a standalone server. [[Virtual Environment]]s are usually also much simpler to administer as all of them can be accessed and administered from the host system. Generally, such systems are the best choice for server consolidation of same OS workloads.

== Where to go further ==
If you've decided to try OpenVZ virtualization solution go to [[Quick installation|OpenVZ installation]] section.

== External links ==
* [http://en.wikipedia.org/wiki/Operating_system-level_virtualization wikipedia: Operating system-level virtualization]
* [http://www.hpl.hp.com/techreports/2007/HPL-2007-59R1.pdf HP Labs: Performance Evaluation of Virtualization Technologies for Server Consolidation]

[[Category: Technology]]
[[Category: Concepts]]

Virtual network device

2010-01-25T13:05:50Z

Ginkyo: IPv6 support

Virtual network device (<code>venet</code>) is the default network device for a [[container]]. Due to [[w:Network_Layer|Layer 3]] employed by OpenVZ's venet, this network device looks like a point-to-point connection between [[container]] and the [[CT0|host system]]. It does packet switching based on IP header. This is a default network device for container (an alternative is [[veth]] device).

Venet drop ip-packets '''from''' the container with a source address, and '''in''' the container with the destination address, which is not corresponding to an ip-address of the container.

Venet device is created automatically on [[container]] start. Vzctl scripts set up an appropriate IP address and other settings on venet inside a container.

== Usage ==

== Kernel module ==
First of all, check that <code>vznetdev</code> module is loaded:
<pre>
# lsmod | grep vznetdev
</pre>

If it is not, load the module:
<pre>
# modprobe vznetdev
</pre>

You might want to check /etc/init.d/vz script to make sure the module gets loaded during startup.

=== Adding IP address to a container ===
<pre>
vzctl set <CTID> --ipadd <IP1>[,<IP2>,...] [--save]
</pre>

{{Note|This option is incremental, so IP addresses are added to already existing ones.}}

==== Example ====
<pre>
vzctl set 101 --ipadd 10.0.0.1 --save
</pre>
After executing this command IP address 10.0.0.1 will be added to container 101 and IP configuration will be saved to a container configuration file.

=== Removing IP address from a container ===
<pre>
vzctl set <CTID> --ipdel <IP1>[,<IP2>,...] [--save]
vzctl set <CTID> --ipdel all [--save]
</pre>

==== Example ====
<pre>
vzctl set 101 --ipdel 10.0.0.1
</pre>
After executing this command IP address 10.0.0.1 will be removed from container 101, but IP configuration will not be changed in container config file. And after container reboot IP address 10.0.0.1 will be assigned to this container again.

== Specific aspects of venet network device ==

{{Note|If you require a feature which venet is lacking (from the list below), please consider using [[veth]] device (which have [[w:Data_Link_Layer|layer 2]] support.)}}

=== No [[w:Address_Resolution_Protocol|ARP]] protocol support ===
Venet network device is explicitly NOARP, so there is no MAC address.
Consequently, it's not possible to make broadcasts inside a [[container]], so software like Samba server or DHCP server will not function (under a container with a venet network device).

=== No [[w:Network_bridge|bridge]] support ===
Venet network device cannot be bridged together and/or with other devices.

=== No possiblity to assign an IP from the CT ===
With venet device, only OpenVZ [[hardware node]] administrator can assign an IP address to a [[container]].

=== No full support of IPv6 stack ===

venet devices are not fully IPv6 compliant. They do not properly support MAC addresses and consequently link local addresses and can not play nice with neighbor discovery or router advertisements, router discovery, or auto-conf. They also require additional modifications to the layer 3 forwarding behaviour of the host via sysctl, to get your venet devices working.
Please have a look at the [[Quick installation#sysctl]] section.

veth devices do require iptables and ip6tables exceptions on the host for each VE address.

You'll need to use the veth bridging device if you want full IPv6 compliance. See the [[VEs and HNs in same subnets]] article for an example.

== See also ==
* [[Veth]]
* [[Differences between venet and veth]]

[[Category: Networking]]
[[Category: HOWTO]]

Monitoring bandwidth of containers

2010-01-25T00:05:03Z

Ginkyo: append Category Networking & $VE_PRIVATE

{{DISPLAYTITLE:Monitoring bandwith of containers}}

Use the /proc/dev/net inside a container, dump stats into a round robin database (rrd)

<pre>
#!/bin/bash
#
# Bandwidth collection script for OpenVZ by Eric 'phpfreak' Rosebrock
# http://www.serverpowered.com / http://www.thewebfreaks.com
# Please read: http://www.serverpowered.com/openvz-bandwidth-accounting.php
#
#
# No warranties on this script, use it at your own risk!
#
# Special thanks to Rick Blundell for helping me to get this thing going.
#
# Discussions: http://forum.openvz.org/index.php?t=tree&goto=1350

# BWVE is the bandwidth storage VE. This would have been created by the
# tutorial you followed as listed above.

BWVE=1002

# BWDIR is the directory inside the bandwdith VE where the .rrd files
# will be stored.

BWDIR=var/rrd/vz

## Default folder for containers
VE_PRIVATE=/var/lib/vz/private

# You should not need to edit anything below this line.

# Get the date from the first of this month to create new rrd files
TDATE=`date +%Y-%m-01`;
ETIME=`date +%s -d $TDATE`

# Define the full path to the bandwidth VE directory and the storage dir.
CBWDIR=$VE_PRIVATE/$BWVE/$BWDIR

# Test and make sure that directory is there, if not create it.
if ! test -d $CBWDIR; then
echo $CBWDIR does not exist, creating it.
mkdir $CBWDIR

fi

# Get a list of all of the VE's created on the system, not just the running ones.

for i in `ls $CBWDIR`; do
VES=`echo $i | cut -f2 -d- | cut -f1 -d.`;

# if the VE was there before, an .rrd file will exist. Now if the VE is gone,
# Delete the .rrd file because its obviously not needed anymore.

if ! test -d $VE_PRIVATE/$VES; then
echo $i will be deleted
rm -rf $CBWDIR/vps-$VES.rrd
fi
done

# Time to do the data collection.
for i in `/usr/sbin/vzlist -Ho veid`; do

RRDFILE=$CBWDIR/vps-$i.rrd

if ! test -e $RRDFILE; then
echo $RRDFILE does not exist, creating.
/usr/bin/rrdtool create $RRDFILE -s 300 \
DS:ds0:DERIVE:600:0:1125000000 \
DS:ds1:DERIVE:600:0:1125000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \
RRA:MIN:0.5:1:600 \
RRA:MIN:0.5:6:700 \
RRA:MIN:0.5:24:775 \
RRA:MIN:0.5:288:797
fi

X=`/usr/sbin/vzctl exec $i "grep venet0 /proc/net/dev"`
eval `echo $X | cut -f2 -d: | awk '{printf"IN=%s\nOUT=%s\n", $1, $9}'`

for g in IN OUT;do
if [ -z ${!g} ]; then
echo foo
eval $g=0
fi
done
/usr/bin/rrdtool update $RRDFILE N:$IN:$OUT
done
</pre>

[[Category: HOWTO]]
[[Category: Monitoring]]
[[Category: Networking]]

Containers/Networking

2010-01-24T16:21:40Z

Ginkyo: append Category Networking

[[Category: Containers]]
[[Category: Networking]]

There are several approaches of how to virtualize networking for containers. Those are desribed below.

__TOC__

== Layer 3 virtualized network interface ==

This one is employed by OpenVZ's venet. See [[venet]].

== Layer 2 virtualized network interface ==

This one is employed by OpenVZ's veth. See [[veth]].

== Layer 3 isolated network (bind filtering) ==

This one is implemented in Linux-VServer. Basically, when a container calls <code>bind()</code> with <code>INADDR_ANY</code>, kernel actually binds the socket to some specific IP address(es). Some more details (not much) can be found at http://linux-vserver.org/Paper#Network_Separation

== See also ==
* [[Differences between venet and veth]]

Using veth and brctl for protecting HN and saving IP addresses

2010-01-24T16:13:55Z

Ginkyo: append Category HOWTO & Networking

Configuration described below has been suggested by Ugo123. Thank you.

Consider we are facing the following task:

# We have limited range of IP addresses granted by ISP. We want to assign as many granted IPs to containers as possible. We do not want to protect containers from Internet.
# We want to protect the [[HN]] OS ([[CT]]0) from Internet and make it possible to manage containers from [[CT0]] within local area network.

Assume we have a [[HN]] with 2 Ethernet cards (interfaces eth0 and eth1), OpenVZ kernel 2.6.18-028stab033, vzctl version 3.0.16,
bridge-utils version 1.1. OpenVZ installation process is covered in [[quick installation]].

This task can be effectively performed by setting up the configuration presented in Figure 1.

Figure 1: Effective configuration. 10.0.98.96-10.0.98.X - range of IP addresses granted by ISP, 192.168.1.136 - IP address from LAN

[[Image:fig.jpg]]

Initial ifconfig output of HN is the following:
<pre>
[HN]# ifconfig
eth0 Link encap:Ethernet HWaddr XX:XX:XX:XX:XX:34
inet addr:192.168.1.136 Bcast:192.168.3.255 Mask:255.255.252.0
inet6 addr: fe80::230:48ff:fe5b:ab34/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3122 errors:0 dropped:0 overruns:0 frame:0
TX packets:246 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:325879 (318.2 KiB) TX bytes:57278 (55.9 KiB)
Interrupt:20

eth1 Link encap:Ethernet HWaddr XX:XX:XX:XX:XX:35
inet addr:192.168.0.32 Bcast:192.168.3.255 Mask:255.255.252.0
inet6 addr: fe80::213:d4ff:fe90:4d50/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:603734 errors:0 dropped:0 overruns:0 frame:0
TX packets:36627 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:21

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1376 errors:0 dropped:0 overruns:0 frame:0
TX packets:1376 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2078718 (1.9 MiB) TX bytes:2078718 (1.9 MiB)
</pre>
Let us step through the setup process.

1) Create 2 containers on the HN as described in http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf.
For testing purposes I've used opensuse-10 precreated template from openvz.org:
<pre>
[HN]# cd /vz/template/cache
[HN]# wget http://download.openvz.org/template/precreated/opensuse-10-i386-default.tar.gz
</pre>
Create container 101 and assign it one of the IP addresses obtained from ISP:
<pre>
[HN]# vzctl create 101 --ostemplate opensuse-10-i386-default --ipadd 10.0.98.96
[HN]# vzctl set 101 --userpasswd root:XXX --save
</pre>
And do the same for CT 102 ... CT N. When ready - start containers:
<pre>
[HN]# vzctl start 101
[HN]# vzlist -a
CTID NPROC STATUS IP_ADDR HOSTNAME
101 4 running 10.0.98.96 -
102 4 running 10.0.98.97 -
</pre>
2) By default containers use venet device for networking (see [[venet]]). But current
configuration requires using alternative networking - through veth devices (see [[Virtual Ethernet device]]).
Switch CT 101 to veth by doing the following:

MAC address needed by eth0 of CT 101 and veth101.0 should be generated by easymac:
<pre>
[HN]# wget http://www.easyvmx.com/software/easymac.sh
[HN]# chmod 0777 easymac.sh
[HN]# ./easymac.sh -R
00:0C:29:70:BB:34
[HN]# ./easymac.sh -R
00:0C:29:C0:2E:07
</pre>
Replace venet by veth device on HN:
<pre>
[HN]# ifconfig venet0:0 down
[HN]# vzctl set 101 --netif_add eth0,00:0C:29:70:BB:34,veth101.0,00:0C:29:C0:2E:07 --save
[HN]# ifconfig veth101.0 0
[HN]# echo 0 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[HN]# echo 0 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
</pre>
Enter the container and tune ifconfig within the container:
<pre>
[CT 101]# vzctl enter 101
[CT 101]# ifconfig venet0:0 down
[CT 101]# ifconfig venet0 down
[CT 101]# ifconfig eth0 0
[CT 101]# ip addr add 10.0.98.96 dev eth0
[CT 101]# ip route add default dev eth0
</pre>
The same (whole item 2) should be done for CT 102 ... CT N.
3) Now we should eliminate the IP address on eth1:

[HN]# vim /etc/sysconfig/network-scripts/ifcfg-eth1

Edit like this:
DEVICE=eth1
#BOOTPROTO=dhcp <<== comment out
HWADDR=XX:XX:XX:XX:XX:XX
ONBOOT=yes

and save changes (:wq).

[HN]# /etc/init.d/network restart

And turn off forwarding and proxy_arp for eth1.

[HN]# ifconfig eth1 0
[HN]# echo 0 > /proc/sys/net/ipv4/conf/eth1/forwarding
[HN]# echo 0 > /proc/sys/net/ipv4/conf/eth1/proxy_arp
</pre>
4) Create br0 bridge uniting eth1, veth101.0, ..., vethN.0:
<pre>
[HN]# brctl addbr br0
[HN]# brctl addif br0 eth1
[HN]# brctl addif br0 veth101.0
..., veth102.0, vethN.0 etc.
[HN]# ifconfig br0 0
</pre>
And turn off frowarding and proxy_arp for br0:
<pre>
[HN]# echo 0 > /proc/sys/net/ipv4/conf/br0/forwarding
[HN]# echo 0 > /proc/sys/net/ipv4/conf/br0/proxy_arp
</pre>
This is very important action. If skipped, network
can be broken on further steps due to incoming arp-requests provoked storm.

As a result of above listed actions the ifconfig output like the following should be listed:
<pre>
[HN]# ifconfig
br0 Link encap:Ethernet HWaddr 00:0C:29:A7:A9:D9
inet6 addr: fe80::20c:29ff:fea7:a9d9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:79 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2972 (2.9 KiB) TX bytes:4390 (4.2 KiB)

eth0 Link encap:Ethernet HWaddr 00:30:48:5B:AB:34
inet addr:192.168.1.136 Bcast:192.168.3.255 Mask:255.255.252.0
inet6 addr: fe80::230:48ff:fe5b:ab34/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:347855 errors:0 dropped:0 overruns:0 frame:0
TX packets:4778 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:35964081 (34.2 MiB) TX bytes:698801 (682.4 KiB)
Interrupt:20

eth1 Link encap:Ethernet HWaddr 00:30:48:5B:AB:35
inet6 addr: fe80::230:48ff:fe5b:ab35/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:322 errors:0 dropped:0 overruns:0 frame:0
TX packets:182 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41943 (40.9 KiB) TX bytes:21338 (20.8 KiB)
Interrupt:21

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1376 errors:0 dropped:0 overruns:0 frame:0
TX packets:1376 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2078718 (1.9 MiB) TX bytes:2078718 (1.9 MiB)

veth101.0 Link encap:Ethernet HWaddr 00:0C:29:C0:2E:07
inet6 addr: fe80::20c:29ff:fec0:2e07/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:363 errors:0 dropped:0 overruns:0 frame:0
TX packets:397 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:31134 (30.4 KiB) TX bytes:31440 (30.7 KiB)

veth102.0 Link encap:Ethernet HWaddr 00:0C:29:A7:A9:D9
inet6 addr: fe80::20c:29ff:fea7:a9d9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:36 errors:0 dropped:0 overruns:0 frame:0
TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1840 (1.7 KiB) TX bytes:2350 (2.2 KiB)
</pre>
5) That is all. It's time to test the obtained configuration.
Now plug eth1 of HN into network wall outlet provided by ISP and carry out the following testing:

- It should be tested that containers are accessible from Internet:
<pre>
[INET]# ssh root@10.0.98.96
[CT 101]# ...
</pre>
- HN is not accessible from Internet:
<pre>
[INET]# ssh root@192.168.1.136
inaccessible
</pre>
- containers can be managed from HN:
<pre>
[HN]# vzctl enter 101
[CT 101]# ...
</pre>
- containers CT 101, CT 102 .. CT N "see" each other (ping).

If all the steps are done as written, it should work.
Enjoy.

[[Category: HOWTO]]
[[Category:Networking]]

Virtual Ethernet device

2010-01-01T20:59:03Z

Ginkyo: Undo revision 8038 by Ginkyo (Talk) sorry it's > and not >=

'''Virtual Ethernet device''' is an Ethernet-like device which can be used
inside a [[container]]. Unlike [[venet]] network device, [[veth]] device
has a MAC address, therefore it can be used in configurations, when veth
is bridged to ethX or other device and container's user fully sets up
his networking himself, including IPs, gateways etc.

Virtual Ethernet device consist of two Ethernet devices --
the one in [[CT0]] and another one in CT. These devices are connected
to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual Ethernet device usage ==

=== Kernel module ===
First of all, make sure the <code>vzethdev</code> module is loaded:
<pre>
# lsmod | grep vzeth
vzethdev 8224 0
vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt
vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
</pre>

In case it is not loaded, load it:
<pre>
# modprobe vzethdev
</pre>

{{Note|in vzctl < 3.0.11, vzethdev is not autoloaded by <code>/etc/init.d/vz</code> script, so you have to edit it to load this module.}}

=== MAC addresses ===
In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions.

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.

YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl --veth_add
MAY GENERATE THEM AUTOMATICALLY AS NECESSARY.

Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this:

chmod +x easymac.sh
./easymac.sh -R

=== Adding veth to a CT ===

==== syntax vzctl version > 3.0.22 ====

vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>]

Here
* <tt>ifname</tt> is the Ethernet device name in the CT
* <tt>mac</tt> is its MAC address in the CT
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]])
* <tt>host_mac</tt> is its MAC address on the host ([[CT0]])
* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge.

{{Note|All parameters except <code>ifname</code> are optional and are automatically generated if not specified.}}

Example:

vzctl set 101 --netif_add eth0 --save

Or, if you want to specify everything:

vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save

Or, if you want to specify the bridge and leave the other values autogenerated:

vzctl set 101 --netif_add eth0,,,,vmbr1 --save

==== syntax vzctl version >= 3.0.14 ====

Syntax is the same as above, but without a <bridge> parameter.

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>

Here
* <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT
* <tt>ve_dev_addr</tt> is its MAC address

{{Note|this option is incremental, so devices are added to already existing ones.}}

NB there should no spaces after the commas.

Example:
<pre>
[host-node] ifconfig eth0
...
HWaddress 00:12:34:56:78:9B
...
</pre>

[host-node] easymac.sh -R
00:12:34:56:78:9A

vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file.
Host-side Ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
CT-side Ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

=== Removing veth from a CT ===

==== syntax vzctl version >= 3.0.14 ====

vzctl set <CTID> --netif_del <dev_name>|all

Here
* <code>dev_name</code> is the Ethernet device name in the [[CT]].

{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}}

Example:

vzctl set 101 --netif_del eth0 --save

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_del <dev_name>

Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]].

Example:

vzctl set 101 --veth_del veth101.0 --save

After executing this command veth device with host-side Ethernet name
<code>veth101.0</code> will be removed from CT101 and veth configuration
will be updated in CT config file.

== Common configurations with virtual Ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual Ethernet device ===

Assuming you have 192.168.0.0/24 on your LAN, you will learn how to integrate a container in this LAN using veth.

==== Start a CT ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure devices in CT0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

==== Configure device in CT ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

Notes:
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier
* 192.168.0.101 is chosen to be an [[unrouteable private ip address]]. Where 101 reminds you that it is node 101.
* The "ip route" tells all traffic to head to "device eth0"
* In theory you could [[use dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it
** http://openvz.org/pipermail/users/2005-November/000020.html

==== Add route in [[CT0]] ====

[host-node]# ip route add 192.168.0.101 dev veth101.0

=== Using a directly routed IPv4 with virtual Ethernet device ===

==== Situation ====
Hardware Node (HN/CT0) has 192.168.0.1/24 with router 192.168.0.254.

We also know that IPv4 10.0.0.1/32 is directly routed to 192.168.0.1 (this is called a ''fail-over IP'').

We want to give this directly routed IPv4 address to a container (CT).

==== Start container ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure device and add route in CT0 ====

<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# ip route add 10.0.0.1 dev veth101.0
</pre>

You can automatize this at VPS creation by using a mount script <tt>$VEID.mount</tt>.

The problem here is that the ''veth'' interface appears in CT0 '''after''' VPS has started, therefore we cannot directly use the commands in the mount script. We launch a shell script (enclosed by { }) in background (operator '''&''') that waits for the interface to be ready and then adds the IP route.

Contents of the mount script <tt>/etc/vz/conf/101.mount</tt>:
<pre>
#!/bin/bash
# This script source VPS configuration files in the same order as vzctl does

# if one of these files does not exist then something is really broken
[ -f /etc/vz/vz.conf ] || exit 1
[ -f $VE_CONFFILE ] || exit 1

# source both files. Note the order, it is important
. /etc/vz/vz.conf
. $VE_CONFFILE

# Configure veth with IP after VPS has started
{
IP=X.Y.Z.T
DEV=veth101.0
while sleep 1; do
/sbin/ifconfig $DEV 0 >/dev/null 2>&1
if [ $? -eq 0 ]; then
/sbin/ip route add $IP dev $DEV
break
fi
done
} &
</pre>

==== Make sure IPv4 forwarding is enabled in CT0 ====

<pre>
[host-node]# echo 1 > /proc/sys/net/ipv4/ip_forward
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
</pre>
You can permanently set this by using <tt>/etc/sysctl.conf</tt>.

==== Configure device in CT ====

1. Configure IP address

2. Add gateway

3. Add default route

<pre>
[ve-101]# /sbin/ifconfig eth0 10.0.0.1 netmask 255.255.255.255
[ve-101]# /sbin/ip route add 192.168.0.1 dev eth0
[ve-101]# /sbin/ip route default via 192.168.0.1
</pre>

In a Debian container, you can configure this permanently by using <tt>/etc/network/interfaces</tt>:
<pre>
auto eth0
iface eth0 inet static
address 10.0.0.1
netmask 255.255.255.255
up /sbin/ip route add 192.168.0.1 dev eth0
up /sbin/ip route add default via 192.168.0.1
</pre>

=== Virtual Ethernet device with IPv6 ===

==== Start a [[CT]] ====

[host-node]# vzctl start 101

==== Add veth device to the [[CT]] ====

[host-node]# vzctl set 101 --veth_add eth0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices in the [[CT0]] ====

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

==== Configure device in [[CT]] ====

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0

==== Start router advertisement daemon (radvd) for IPv6 in CT0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:

[host-node]# /etc/init.d/radvd start

==== Add IPv6 addresses to devices in [[CT0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual Ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[CT0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes.

=== Making a veth-device persistent ===
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced"

See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section.

That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network.

=== Making a bridged veth-device persistent ===

Like the above example, here it is how to add the veth device to a bridge in a persistent way.

==== method for vzctl version > 3.0.22 ====

Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <bridge> parameter of the --netif_add switch.

Just create /etc/vz/vznet.conf containing the following.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

The script uses 'vmbr0' as default bridge name when no bridge is specified.

==== method for vzctl version <= 3.0.22 ====

Older vzctl doesn't offer an automatic function to do this.

1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT.
* Open up /etc/vz/conf/CTID.conf
* Comment out any IP_ADDRESS entries to prevent a CTNET-device from being created in the CT
* Add or change the entry CONFIG_CUSTOMIZED="yes"
* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend.

2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable.

<pre>
#!/bin/bash
# /usr/sbin/vznetaddbr
# a script to add virtual network interfaces (veth's) in a CT to a bridge on CT0

CONFIGFILE=/etc/vz/conf/$VEID.conf
. $CONFIGFILE
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE CT$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VZHOSTBR" ]; then
echo "According to $CONFIGFILE CT$VEID has no bridge interface configured."
exit 1
fi

echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF

exit 0
</pre>

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it:

<pre>
#!/bin/bash

CONFIGFILE="/etc/vz/conf/$VEID.conf"

if [ -f "$CONFIGFILE" ]
then
. "$CONFIGFILE"
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`
export VZHOSTIF
export VZHOSTBR

# Fork into the background and try a few times,
# until the host side of the interface appears:
/bin/bash -c 'for i in 5 10 20 40 80 160
do
if ifconfig -a | grep -q "$VZHOSTIF"
then
exec /usr/sbin/vznetaddbr
else
sleep "$i"
fi
done
' &

# In the meantime, let the CT's start process continue,
# or else the interface will never appear:
exit 0
else
$0: Config file "$CONFIGFILE" does not exist.
exit 1
fi
</pre>

4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details.

When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code>

Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan.

=== Virtual Ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]
* [[Using private IPs for Hardware Nodes]]
* Patch: [[Disable venet interface]]
* Troubleshooting: [[Bridge doesn't forward packets]]

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]
* [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html 2 veth with 2 bridges setup]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual Ethernet device

2010-01-01T20:54:12Z

Ginkyo: /* syntax vzctl version > 3.0.22 */ no bridge support in 3.0.22

'''Virtual Ethernet device''' is an Ethernet-like device which can be used
inside a [[container]]. Unlike [[venet]] network device, [[veth]] device
has a MAC address, therefore it can be used in configurations, when veth
is bridged to ethX or other device and container's user fully sets up
his networking himself, including IPs, gateways etc.

Virtual Ethernet device consist of two Ethernet devices --
the one in [[CT0]] and another one in CT. These devices are connected
to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual Ethernet device usage ==

=== Kernel module ===
First of all, make sure the <code>vzethdev</code> module is loaded:
<pre>
# lsmod | grep vzeth
vzethdev 8224 0
vzmon 35164 5 vzethdev,vznetdev,vzrst,vzcpt
vzdev 3080 4 vzethdev,vznetdev,vzmon,vzdquota
</pre>

In case it is not loaded, load it:
<pre>
# modprobe vzethdev
</pre>

{{Note|in vzctl < 3.0.11, vzethdev is not autoloaded by <code>/etc/init.d/vz</code> script, so you have to edit it to load this module.}}

=== MAC addresses ===
In the below commands, you should use random MAC addresses. Do not use MAC addresses of real eth devices, because this can lead to collisions.

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format.

YOU MAY NOT NEED TO GENERATE MAC ADDRESSES BY HAND BECAUSE vzctl --veth_add
MAY GENERATE THEM AUTOMATICALLY AS NECESSARY.

Nevertheless, there is a utility script available for generating MAC addresses: http://www.easyvmx.com/software/easymac.sh. It is to be used like this:

chmod +x easymac.sh
./easymac.sh -R

=== Adding veth to a CT ===

==== syntax vzctl version > 3.0.23 ====

vzctl set <CTID> --netif_add <ifname>[,<mac>,<host_ifname>,<host_mac>,<bridge>]

Here
* <tt>ifname</tt> is the Ethernet device name in the CT
* <tt>mac</tt> is its MAC address in the CT
* <tt>host_ifname</tt> is the Ethernet device name on the host ([[CT0]])
* <tt>host_mac</tt> is its MAC address on the host ([[CT0]])
* <tt>bridge</tt> is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge.

{{Note|All parameters except <code>ifname</code> are optional and are automatically generated if not specified.}}

Example:

vzctl set 101 --netif_add eth0 --save

Or, if you want to specify everything:

vzctl set 101 --netif_add eth0,00:12:34:56:78:9A,veth101.0,00:12:34:56:78:9B --save

Or, if you want to specify the bridge and leave the other values autogenerated:

vzctl set 101 --netif_add eth0,,,,vmbr1 --save

==== syntax vzctl version >= 3.0.14 ====

Syntax is the same as above, but without a <bridge> parameter.

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>

Here
* <tt>dev_name</tt> is the Ethernet device name that you are creating on the [[CT0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is the corresponding Ethernet device name you are creating on the CT
* <tt>ve_dev_addr</tt> is its MAC address

{{Note|this option is incremental, so devices are added to already existing ones.}}

NB there should no spaces after the commas.

Example:
<pre>
[host-node] ifconfig eth0
...
HWaddress 00:12:34:56:78:9B
...
</pre>

[host-node] easymac.sh -R
00:12:34:56:78:9A

vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

After executing this command <tt>veth</tt> device will be created for CT 101 and veth configuration will be saved to a CT configuration file.
Host-side Ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
CT-side Ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.

=== Removing veth from a CT ===

==== syntax vzctl version >= 3.0.14 ====

vzctl set <CTID> --netif_del <dev_name>|all

Here
* <code>dev_name</code> is the Ethernet device name in the [[CT]].

{{Note|If you want to remove all Ethernet devices in CT, use <code>all</code>.}}

Example:

vzctl set 101 --netif_del eth0 --save

==== syntax vzctl version < 3.0.14 ====

vzctl set <CTID> --veth_del <dev_name>

Here <tt>dev_name</tt> is the Ethernet device name in the [[CT0|host system]].

Example:

vzctl set 101 --veth_del veth101.0 --save

After executing this command veth device with host-side Ethernet name
<code>veth101.0</code> will be removed from CT101 and veth configuration
will be updated in CT config file.

== Common configurations with virtual Ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual Ethernet device ===

Assuming you have 192.168.0.0/24 on your LAN, you will learn how to integrate a container in this LAN using veth.

==== Start a CT ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure devices in CT0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

==== Configure device in CT ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

Notes:
* Until you ifconfig eth0 it won't appear. When you do it will use the mac address netif_add added earlier
* 192.168.0.101 is chosen to be an [[unrouteable private ip address]]. Where 101 reminds you that it is node 101.
* The "ip route" tells all traffic to head to "device eth0"
* In theory you could [[use dhcpd with OpenVZ]] and dhclient to pick up an DHCP address from your router instead of hardwiring it
** http://openvz.org/pipermail/users/2005-November/000020.html

==== Add route in [[CT0]] ====

[host-node]# ip route add 192.168.0.101 dev veth101.0

=== Using a directly routed IPv4 with virtual Ethernet device ===

==== Situation ====
Hardware Node (HN/CT0) has 192.168.0.1/24 with router 192.168.0.254.

We also know that IPv4 10.0.0.1/32 is directly routed to 192.168.0.1 (this is called a ''fail-over IP'').

We want to give this directly routed IPv4 address to a container (CT).

==== Start container ====

[host-node]# vzctl start 101

==== Add veth device to CT ====

[host-node]# vzctl set 101 --netif_add eth0 --save

This allocates a MAC address and associates it with the host eth0 port.

==== Configure device and add route in CT0 ====

<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# ip route add 10.0.0.1 dev veth101.0
</pre>

You can automatize this at VPS creation by using a mount script <tt>$VEID.mount</tt>.

The problem here is that the ''veth'' interface appears in CT0 '''after''' VPS has started, therefore we cannot directly use the commands in the mount script. We launch a shell script (enclosed by { }) in background (operator '''&''') that waits for the interface to be ready and then adds the IP route.

Contents of the mount script <tt>/etc/vz/conf/101.mount</tt>:
<pre>
#!/bin/bash
# This script source VPS configuration files in the same order as vzctl does

# if one of these files does not exist then something is really broken
[ -f /etc/vz/vz.conf ] || exit 1
[ -f $VE_CONFFILE ] || exit 1

# source both files. Note the order, it is important
. /etc/vz/vz.conf
. $VE_CONFFILE

# Configure veth with IP after VPS has started
{
IP=X.Y.Z.T
DEV=veth101.0
while sleep 1; do
/sbin/ifconfig $DEV 0 >/dev/null 2>&1
if [ $? -eq 0 ]; then
/sbin/ip route add $IP dev $DEV
break
fi
done
} &
</pre>

==== Make sure IPv4 forwarding is enabled in CT0 ====

<pre>
[host-node]# echo 1 > /proc/sys/net/ipv4/ip_forward
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
</pre>
You can permanently set this by using <tt>/etc/sysctl.conf</tt>.

==== Configure device in CT ====

1. Configure IP address

2. Add gateway

3. Add default route

<pre>
[ve-101]# /sbin/ifconfig eth0 10.0.0.1 netmask 255.255.255.255
[ve-101]# /sbin/ip route add 192.168.0.1 dev eth0
[ve-101]# /sbin/ip route default via 192.168.0.1
</pre>

In a Debian container, you can configure this permanently by using <tt>/etc/network/interfaces</tt>:
<pre>
auto eth0
iface eth0 inet static
address 10.0.0.1
netmask 255.255.255.255
up /sbin/ip route add 192.168.0.1 dev eth0
up /sbin/ip route add default via 192.168.0.1
</pre>

=== Virtual Ethernet device with IPv6 ===

==== Start a [[CT]] ====

[host-node]# vzctl start 101

==== Add veth device to the [[CT]] ====

[host-node]# vzctl set 101 --veth_add eth0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save

==== Configure devices in the [[CT0]] ====

[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

==== Configure device in [[CT]] ====

[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0

==== Start router advertisement daemon (radvd) for IPv6 in CT0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:

[host-node]# /etc/init.d/radvd start

==== Add IPv6 addresses to devices in [[CT0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual Ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several containers and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[CT0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to containers will be through this bridge and containers can communicate with each other even without these routes.

=== Making a veth-device persistent ===
According to http://bugzilla.openvz.org/show_bug.cgi?id=301 , a bug that stopped the veth device persistent was "Obsoleted now when --veth_add/--veth_del are introduced"

See http://wiki.openvz.org/w/index.php?title=Virtual_Ethernet_device&diff=5990&oldid=5989#Making_a_veth-device_persistent for a workaround that used to be described in this section.

That's it! At this point, when you restart the CT you should see a new line in the output, indicating that the interface is being configured and a new route being added. And you should be able to ping the host, and to enter the CT and use the network.

=== Making a bridged veth-device persistent ===

Like the above example, here it is how to add the veth device to a bridge in a persistent way.

==== method for vzctl version > 3.0.22 ====

Newer versions of vzctl includes a 'vznetaddbr' script, which makes use of the new <bridge> parameter of the --netif_add switch.

Just create /etc/vz/vznet.conf containing the following.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

The script uses 'vmbr0' as default bridge name when no bridge is specified.

==== method for vzctl version <= 3.0.22 ====

Older vzctl doesn't offer an automatic function to do this.

1. First, edit the CT's configuration to specify what is the host bridge , and to indicate that a custom script should be run when starting up a CT.
* Open up /etc/vz/conf/CTID.conf
* Comment out any IP_ADDRESS entries to prevent a CTNET-device from being created in the CT
* Add or change the entry CONFIG_CUSTOMIZED="yes"
* Add an entry VZHOSTBR="<bridge if>" which is the bridge interface (already configured and up), you want to extend.

2. Now to create that "custom script". The following helper script will check the configuration file for the bridge interface name and for the veth interface, and add the interface to the bridge. Create the script /usr/sbin/vznetaddbr to have the following, and then <code>chmod 0500 /usr/sbin/vznetaddbr</code> to make it executable.

<pre>
#!/bin/bash
# /usr/sbin/vznetaddbr
# a script to add virtual network interfaces (veth's) in a CT to a bridge on CT0

CONFIGFILE=/etc/vz/conf/$VEID.conf
. $CONFIGFILE
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`

if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE CT$VEID has no veth interface configured."
exit 1
fi

if [ ! -n "$VZHOSTBR" ]; then
echo "According to $CONFIGFILE CT$VEID has no bridge interface configured."
exit 1
fi

echo "Adding interface $VZHOSTIF to bridge $VZHOSTBR on CT0 for CT$VEID"
/sbin/ifconfig $VZHOSTIF 0
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
/usr/sbin/brctl addif $VZHOSTBR $VZHOSTIF

exit 0
</pre>

3. Now create /etc/vz/vznet.conf containing the following. This is what defines the "custom script" as being the vznetaddbr which you just created.

<pre>
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
</pre>

This may not work for particularily old versions of vzctl, e.g., the version 3.0.11 that ships with Debian Etch. For those versions, you can try a hack: Use the custom script <code>/etc/vz/conf/$VID.mount</code> which is available, even in these old versions. But it gets called too early, before the networking has been set up. But it can start some background process, which waits and occasionally polls until $VZHOSTIF has become available. Here is one way to go about it:

<pre>
#!/bin/bash

CONFIGFILE="/etc/vz/conf/$VEID.conf"

if [ -f "$CONFIGFILE" ]
then
. "$CONFIGFILE"
VZHOSTIF=`echo $NETIF |sed 's/^.*host_ifname=$.*$,.*$/\1/g'`
export VZHOSTIF
export VZHOSTBR

# Fork into the background and try a few times,
# until the host side of the interface appears:
/bin/bash -c 'for i in 5 10 20 40 80 160
do
if ifconfig -a | grep -q "$VZHOSTIF"
then
exec /usr/sbin/vznetaddbr
else
sleep "$i"
fi
done
' &

# In the meantime, let the CT's start process continue,
# or else the interface will never appear:
exit 0
else
$0: Config file "$CONFIGFILE" does not exist.
exit 1
fi
</pre>

4. Of course, the CT's operating system will need to have . Consult the manual for your CT's OS for details.

When the CT is started, the veth specified in the NETIF value is added to the bridge specified. You can check this by doing <code>brctl show</code>

Inside the CT you can configure the interface statically or using dhcp, as a real interface attached to a switch on the lan.

=== Virtual Ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== See also ==
* [[Virtual network device]]
* [[Differences between venet and veth]]
* [[Using private IPs for Hardware Nodes]]
* Patch: [[Disable venet interface]]
* Troubleshooting: [[Bridge doesn't forward packets]]

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]
* [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html 2 veth with 2 bridges setup]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual network device

2010-01-01T20:07:05Z

Ginkyo: /* Limitations */ => Specific aspects

Virtual network device (<code>venet</code>) is the default network device for a [[container]]. This network device looks like a point-to-point connection between [[container]] and the [[CT0|host system]]. It does packet switching based on IP header. This is a default network device for container (an alternative is [[veth]] device).

Venet drop ip-packets '''from''' the container with a source address, and '''in''' the container with the destination address, which is not corresponding to an ip-address of the container.

Venet device is created automatically on [[container]] start. Vzctl scripts set up an appropriate IP address and other settings on venet inside a container.

== Usage ==

== Kernel module ==
First of all, check that <code>vznetdev</code> module is loaded:
<pre>
# lsmod | grep vznetdev
</pre>

If it is not, load the module:
<pre>
# modprobe vznetdev
</pre>

You might want to check /etc/init.d/vz script to make sure the module gets loaded during startup.

=== Adding IP address to a container ===
<pre>
vzctl set <CTID> --ipadd <IP1>[,<IP2>,...] [--save]
</pre>

{{Note|This option is incremental, so IP addresses are added to already existing ones.}}

==== Example ====
<pre>
vzctl set 101 --ipadd 10.0.0.1 --save
</pre>
After executing this command IP address 10.0.0.1 will be added to container 101 and IP configuration will be saved to a container configuration file.

=== Removing IP address from a container ===
<pre>
vzctl set <CTID> --ipdel <IP1>[,<IP2>,...] [--save]
vzctl set <CTID> --ipdel all [--save]
</pre>

==== Example ====
<pre>
vzctl set 101 --ipdel 10.0.0.1
</pre>
After executing this command IP address 10.0.0.1 will be removed from container 101, but IP configuration will not be changed in container config file. And after container reboot IP address 10.0.0.1 will be assigned to this container again.

== Sysctl ==

You will need to configure some sysctl parameters to get your venet devices working.
Please have a look at the [[Installation_on_Debian#sysctl]] section.

== IPv6 ==

To setup IPv6 networking with venet you'll need to enable the following in your sysctl.conf:

<code>
# IPv6 Packet Forwarding and Proxy NDP
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.proxy_ndp = 1
net.ipv6.conf.all.proxy_ndp = 1
</code>

In IPv6 you can't control forwarding per device, forwarding control has to take place in ip6tables, so all interfaces will forward IPv6 traffic.

If you enable IPv6 forwarding for your interfaces, Linux assumes your host to act like a router and will ignore 'Router Advertisments'
(see [http://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol Neighbor Discovery Protocol] or [http://mirrors.bieringer.de/Linux+IPv6-HOWTO/proc-sys-net-ipv6.html Linux IPv6 Howto]).

You will as well need to configure a new v6 default gateway for your host:

<code>
ip addr add 2620:0:2d0:1::193/64 dev eth0
route -6 add default gw 2620:0:2d0:1::1
</code>

You can add these commands to your existing network configuration on Debian/Linux:

<code>
iface eth0 inet static
address 64.131.90.7
netmask 255.255.255.240
network 64.131.90.0
broadcast 64.131.90.15
gateway 64.131.90.1
up ip addr add 2620:0:2d0:1::193/64 dev eth0
up route -6 add default gw 2620:0:2d0:1::1
down ip addr del 3620:0:2d0:1::193/64 dev eth0
</code>
== Specific aspects of venet network device ==
=== No [http://en.wikipedia.org/wiki/Address_Resolution_Protocol ARP] protocol support ===
Venet network device is explicitly NOARP, so there is no MAC address.
Consequently, it's not possible to make broadcasts inside a [[CT|container]], so software like Samba server or DHCP server, will not function (under a container with a venet network device). 
Please consider use of [[veth]] network device if you need this feature

=== No bridge support ===
Venet network device cannot be bridged together and/or with other devices. 
Please consider use of [[veth]] network device if you need this feature

=== No possiblity to assign an IP from the CT ===
With venet device, only OpenVZ [[Hardware_Node|hardware node]] administrator can assign an IP address to a [[CT|container]]. 
Please consider use of [[veth]] network device if you need this feature

== See also ==
* [[Veth]]
* [[Differences between venet and veth]]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual network device

2010-01-01T20:04:08Z

Ginkyo: Adding a new chapter: limitations

Virtual network device (<code>venet</code>) is the default network device for a [[container]]. This network device looks like a point-to-point connection between [[container]] and the [[CT0|host system]]. It does packet switching based on IP header. This is a default network device for container (an alternative is [[veth]] device).

Venet drop ip-packets '''from''' the container with a source address, and '''in''' the container with the destination address, which is not corresponding to an ip-address of the container.

Venet device is created automatically on [[container]] start. Vzctl scripts set up an appropriate IP address and other settings on venet inside a container.

== Usage ==

== Kernel module ==
First of all, check that <code>vznetdev</code> module is loaded:
<pre>
# lsmod | grep vznetdev
</pre>

If it is not, load the module:
<pre>
# modprobe vznetdev
</pre>

You might want to check /etc/init.d/vz script to make sure the module gets loaded during startup.

=== Adding IP address to a container ===
<pre>
vzctl set <CTID> --ipadd <IP1>[,<IP2>,...] [--save]
</pre>

{{Note|This option is incremental, so IP addresses are added to already existing ones.}}

==== Example ====
<pre>
vzctl set 101 --ipadd 10.0.0.1 --save
</pre>
After executing this command IP address 10.0.0.1 will be added to container 101 and IP configuration will be saved to a container configuration file.

=== Removing IP address from a container ===
<pre>
vzctl set <CTID> --ipdel <IP1>[,<IP2>,...] [--save]
vzctl set <CTID> --ipdel all [--save]
</pre>

==== Example ====
<pre>
vzctl set 101 --ipdel 10.0.0.1
</pre>
After executing this command IP address 10.0.0.1 will be removed from container 101, but IP configuration will not be changed in container config file. And after container reboot IP address 10.0.0.1 will be assigned to this container again.

== Sysctl ==

You will need to configure some sysctl parameters to get your venet devices working.
Please have a look at the [[Installation_on_Debian#sysctl]] section.

== IPv6 ==

To setup IPv6 networking with venet you'll need to enable the following in your sysctl.conf:

<code>
# IPv6 Packet Forwarding and Proxy NDP
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.proxy_ndp = 1
net.ipv6.conf.all.proxy_ndp = 1
</code>

In IPv6 you can't control forwarding per device, forwarding control has to take place in ip6tables, so all interfaces will forward IPv6 traffic.

If you enable IPv6 forwarding for your interfaces, Linux assumes your host to act like a router and will ignore 'Router Advertisments'
(see [http://en.wikipedia.org/wiki/Neighbor_Discovery_Protocol Neighbor Discovery Protocol] or [http://mirrors.bieringer.de/Linux+IPv6-HOWTO/proc-sys-net-ipv6.html Linux IPv6 Howto]).

You will as well need to configure a new v6 default gateway for your host:

<code>
ip addr add 2620:0:2d0:1::193/64 dev eth0
route -6 add default gw 2620:0:2d0:1::1
</code>

You can add these commands to your existing network configuration on Debian/Linux:

<code>
iface eth0 inet static
address 64.131.90.7
netmask 255.255.255.240
network 64.131.90.0
broadcast 64.131.90.15
gateway 64.131.90.1
up ip addr add 2620:0:2d0:1::193/64 dev eth0
up route -6 add default gw 2620:0:2d0:1::1
down ip addr del 3620:0:2d0:1::193/64 dev eth0
</code>
== Limitations ==
=== No [http://en.wikipedia.org/wiki/Address_Resolution_Protocol ARP] protocol support ===
Venet network device is explicitly NOARP, so there is no MAC address.
Consequently, it's not possible to make broadcasts inside a [[CT|container]], so software like Samba server or DHCP server, will not function (under a container with a venet network device). 
Please consider use of [[veth]] network device if you need this feature

=== No bridge support ===
Venet network device cannot be bridged together and/or with other devices. 
Please consider use of [[veth]] network device if you need this feature

=== No possiblity to assign an IP from the CT ===
With venet device, only OpenVZ [[Hardware_Node|hardware node]] administrator can assign an IP address to a [[CT|container]]. 
Please consider use of [[veth]] network device if you need this feature

== See also ==
* [[Veth]]
* [[Differences between venet and veth]]

[[Category: Networking]]
[[Category: HOWTO]]

Container enter failed

2009-10-13T13:24:31Z

Ginkyo: Undo revision 7576 by 91.214.44.121 (Talk) =>spam link

'''Problem''': container created succesfully and started.
But when trying to do
<pre>
vzctl enter 101
</pre>
you get
<pre>
container enter failed(?)
</pre>

Using strace, you see:
<pre>
# strace -ff vzctl enter
....
fstat64(...st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0)...) fail
....
</pre>

'''Solution''':
Recompile the kernel with the following option:
<pre>
CONFIG_LEGACY_PTYS=y
</pre>

'''Other solutions''':

1) enter the VE manually creating the LEGACY_PTYS devices

<pre>
vzctl exec 101 /sbin/MAKEDEV tty
vzctl exec 101 /sbin/MAKEDEV pty
vzctl enter 101
</pre>

2A) If you want udev in VE, save the changes forcing udev to make LEGACY_PTYS:

<pre>
cat > /etc/udev/makedev.d/51-udev.nodes
# These device have to be created manually
tty0
tty1
tty2
tty3
....
ttyp0
ttyp1
ttyp2
ttyp3
....
ptyp0
ptyp1
ptyp2
ptyp3
....
</pre>

2B) If you think is better disable udev in VE, comment out in the VE the line:
<pre>
/sbin/start_udev
</pre>
in
<pre>
/etc/rc.d/rc.sysinit
</pre>

.. however updates to the package which owns this file may revert your changes, so you must take steps to guard against this.

Restart the VE and make the devices with MAKEDEV:
<pre>
vzctl exec 101 /sbin/MAKEDEV tty
vzctl exec 101 /sbin/MAKEDEV pty
vzctl enter 101
</pre>

== See also ==

* {{Bug|130}}
* {{Bug|578}}

[[Category: Troubleshooting]]

Monitoring openvz resources using nagios and snmp

2008-07-08T10:26:39Z

Ginkyo: /* snmpd configuration */ spelling corrections

== snmpd configuration ==
Debian Etch example:
<pre>
apt-get install snmpd
</pre>

edit '''/etc/default/snmpd''' : remove ''-u snmp'' and replace ''127.0.0.1'' with your ip (ie : 207.46.250.119), Full'''/etc/default/snmpd''' example:
<pre>
export MIBDIRS=/usr/share/snmp/mibs
SNMPDRUN=yes
SNMPDOPTS='-Lsd -Lf /dev/null -I -smux -p /var/run/snmpd.pid 207.46.250.119'
TRAPDRUN=no
TRAPDOPTS='-Lsd -p /var/run/snmptrapd.pid'
</pre>

For Debian 4.x:
<pre>
export MIBDIRS=/usr/share/snmp/mibs
SNMPDRUN=yes
SNMPDOPTS='-Lsd -Lf /dev/null -I -smux -p /var/run/snmpd.pid'
TRAPDRUN=no
TRAPDOPTS='-Lsd -p /var/run/snmptrapd.pid'
</pre>

Create user(my_username) and add new mib. Password need a min. of 8 charactes. Username only characters:
<pre>
/etc/init.d/snmpd stop
echo rouser my_username priv >> /etc/snmp/snmpd.conf
echo "extend .1.3.6.1.4.1.2021.51 beancounters /bin/cat /proc/user_beancounters" >> /etc/snmp/snmpd.conf
echo "extend .1.3.6.1.4.1.2021.52 vzquota /bin/cat /proc/vz/vzquota" >> /etc/snmp/snmpd.conf
echo createUser my_username MD5 my_password DES >> /var/lib/snmp/snmpd.conf
/etc/init.d/snmpd start
</pre>

Testing snmp:
<pre>
snmpwalk -v 3 -u my_username -l authPriv -a MD5 -A my_password -x DES -X my_password $(hostname -i)
</pre>

Warning: the minimum pass phrase length is 8 characters.

== nagios configuration ==
=== example nagios configuration ===
add to configuration:
<pre>
define command {
command_name check_snmp_openvz_on_port
# command_line /usr/local/bin/check_snmp_openvz.sh $HOSTADDRESS$ PORT USER PASSWORD
command_line /usr/local/bin/check_snmp_openvz.sh $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$
}
</pre>

<pre>
define host {
host_name openvz-server
alias Serwer Openvz
address 207.46.250.119
use generic-host
contact_groups admins
}
</pre>

<pre>
define service{
use generic-service
host_name openvz-server
service_description Virtual Machines Limits
check_command check_snmp_openvz_on_port!161!my_username!my_password
max_check_attempts 1
}

</pre>

=== nagios plugin ===
It is shell script:
<source lang="bash">
# cat /usr/local/bin/check_snmp_openvz.sh
#!/bin/bash
HOST=$1
PORT=$2
USER=$3
PASS=$4
export FILE=/tmp/$HOST.beancounters
RET=0

DATA_TMP=`snmpwalk -v 3 -u $USER -l authPriv -a MD5 -A $PASS -x DES -X $PASS $HOST:$PORT .1.3.6.1.4.1.2021.51.4`
if [ "$?" != "0" ]; then
echo "Unknown snmp error"
exit 1
fi

DATA=`echo "$DATA_TMP"| perl -ne '/"(.*)"/ ; print "$1\n" ;'`

if [ -f $FILE ]; then
echo "$DATA" | perl -n -e'
use Data::Dumper;
my $file=$ENV{"FILE"};
my $ret=0 ;
my $vid ;
my $resource ;
my $held ;
my $maxheld ;
my $barrier ;
my $limit ;
my $failcnt ;
my %beancounters ;
my %beancounters_old ;
while(<STDIN>){
my %vmachine;
if ( /\D*(\d+):.*/ ){ $vid=$1; $beancounters{$vid}=\%vmachine ; }
if ( /^[\W\d]+([a-z]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ) {
$resource=$1 ;
$held=$2 ;
$maxheld=$3 ;
$barrier=$4 ;
$limit=$5 ;
$failcnt=$6 ;
${beancounters{$vid}}{$resource}=[$held , $maxheld , $barrier , $limit ,$failcnt ];
if ( ($held > $barrier) && ($barrier != 0) ) {
print "WARNING: Limits on $vid: $resource held->$held , barrier->$barrier ( limit->$limit ) " ;
$ret=1;
}
}
}

# read and parse old data
open(MYINPUTFILE, "<$file");
while(<MYINPUTFILE>){
my %vmachine;
if ( /\D*(\d+):.*/ ){ $vid=$1; $beancounters_old{$vid}=\%vmachine ; }
if ( /^[\W\d]+([a-z]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ) {
$resource=$1 ;
$held=$2 ;
$maxheld=$3 ;
$barrier=$4 ;
$limit=$5 ;
$failcnt=$6 ;
${beancounters_old{$vid}}{$resource}=[$held , $maxheld , $barrier , $limit ,$failcnt ];
}
}

foreach my $vmachine_id (keys %beancounters) {
foreach my $resource (keys %{$beancounters{$vmachine_id}} ) {
if ( defined($beancounters{$vmachine_id}{$resource}[4]) && defined($beancounters_old{$vmachine_id}{$resource}[4]) ){
my $failcnt=$beancounters{$vmachine_id}{$resource}[4];
my $failcnt_old=$beancounters_old{$vmachine_id}{$resource}[4];
my $held=$beancounters{$vmachine_id}{$resource}[0];
my $maxheld=$beancounters{$vmachine_id}{$resource}[1];
my $barrier=$beancounters{$vmachine_id}{$resource}[2];
my $limit=$beancounters{$vmachine_id}{$resource}[3];
if ( $failcnt_old < $failcnt ){
print "CRITICAL: Incrased failcnt $vmachine_id: $resource from $failcnt_old to $failcnt (held->$held , maxheld->$maxheld , barrier->$barrier , limit->$limit ) " ;
$ret=2;
}
}
}

}

# if ($ret == 0 ) { print "Ok. \n" ; }
# print Dumper(%beancounters_old) ;
# print "\n";
exit($ret);
'

RET1=$?
fi

echo "$DATA" > $FILE
#####################################################################################
######### quota check
#####################################################################################

DATA=`snmpwalk -v 3 -u $USER -l authPriv -a MD5 -A $PASS -x DES -X $PASS $HOST:$PORT .1.3.6.1.4.1.2021.52.4 \
| perl -ne '/"(.*)"/ ; print "$1\n" ;'`

if [ "$?" != "0" ]; then
echo "Unknown snmp error"
exit 1
fi

echo "$DATA" | perl -n -e'
my $vid ;
my $ret=0 ;
while(<STDIN>){
my %vid;
if ( /\D*(\d+):.*/ ){ $vid=$1; }
if ( /\s*(\S+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ){
$resource=$1 ;
$usage=$2 ;
$softlimit=$3 ;
$hardlimit=$4 ;
$time=$5 ;
$expire=$6 ;
if ( $usage >= $softlimit ){
print "WARNING: VZquota limit exceeded on $vid: $resource usage->$usage, softlimit->$softlimit, hardlimit->$hardlimit, time->$time, expire->$expire " ;
$ret=1;
}
}
}
exit($ret);
'
RET2=$?

#####################################################################################
########### return
#####################################################################################

if [ $RET1 -gt $RET2 ]; then
RET=$RET1
else
RET=$RET2
fi

if [ $RET = 0 ]; then
echo Ok.
fi
exit $RET
</source>
=== check_vzquota Without SNMP ===
<source lang="bash">
#!/bin/bash
RET=0
DATA=`echo;sudo /usr/sbin/vzlist -1 | xargs -I {} bash -c "echo {}:;sudo /usr/sbin/vzquota stat {}"`
echo "$DATA" | perl -n -e'
my $vid ;
my $ret=0 ;
while(<STDIN>){
my %vid;
if ( /^(\d+):.*/ ){ $vid=$1; }
if ( /\D*(\d+):.*/ ){ $vid=$1; }
if ( /\s*(\S+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ){
$resource=$1 ;
$usage=$2 ;
$softlimit=$3 ;
$hardlimit=$4 ;
if ( $usage >= $softlimit ){
print "WARNING: VZquota limit exceeded on $vid: $resource usage->$usage, softlimit->$softlimit, hardlimit->$hardlimit, time->$time, expire->$expire " ;
$ret=1;
}
print "$vid:$resource $usage/$softlimit ";
}
}
exit($ret);
'
RET=$?
echo
exit $RET

</source>

=== check_ubc Without SNMP ===
<source lang="bash">
#!/bin/bash
export FILE=/tmp/check_ubc
RET=0

DATA=`cat /proc/user_beancounters`

if [ -f $FILE ]; then
echo "$DATA" | perl -n -e'
use Data::Dumper;
my $file=$ENV{"FILE"};
my $ret=0 ;
my $vid ;
my $resource ;
my $held ;
my $maxheld ;
my $barrier ;
my $limit ;
my $failcnt ;
my %beancounters ;
my %beancounters_old ;
while(<STDIN>){
my %vmachine;
if ( /\D*(\d+):.*/ ){ $vid=$1; $beancounters{$vid}=\%vmachine ; }
if ( /^[\W\d]+([a-z]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ) {
$resource=$1 ;
$held=$2 ;
$maxheld=$3 ;
$barrier=$4 ;
$limit=$5 ;
$failcnt=$6 ;
${beancounters{$vid}}{$resource}=[$held , $maxheld , $barrier , $limit ,$failcnt ];
if ( ($held > $barrier) && ($barrier != 0) ) {
print "WARNING: Limits on $vid: $resource held->$held , barrier->$barrier ( limit->$limit ) " ;
$ret=1;
}
#print "$vid:$resource $held Barrier:$barrier ";
}
}

# read and parse old data
open(MYINPUTFILE, "<$file");
while(<MYINPUTFILE>){
my %vmachine;
if ( /\D*(\d+):.*/ ){ $vid=$1; $beancounters_old{$vid}=\%vmachine ; }
if ( /^[\W\d]+([a-z]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+).*/ ) {
$resource=$1 ;
$held=$2 ;
$maxheld=$3 ;
$barrier=$4 ;
$limit=$5 ;
$failcnt=$6 ;
${beancounters_old{$vid}}{$resource}=[$held , $maxheld , $barrier , $limit ,$failcnt ];
}
}

foreach my $vmachine_id (keys %beancounters) {
foreach my $resource (keys %{$beancounters{$vmachine_id}} ) {
if ( defined($beancounters{$vmachine_id}{$resource}[4]) && defined($beancounters_old{$vmachine_id}{$resource}[4]) ){
my $failcnt=$beancounters{$vmachine_id}{$resource}[4];
my $failcnt_old=$beancounters_old{$vmachine_id}{$resource}[4];
my $held=$beancounters{$vmachine_id}{$resource}[0];
my $maxheld=$beancounters{$vmachine_id}{$resource}[1];
my $barrier=$beancounters{$vmachine_id}{$resource}[2];
my $limit=$beancounters{$vmachine_id}{$resource}[3];
if ( $failcnt_old < $failcnt ){
print "CRITICAL: Incrased failcnt $vmachine_id: $resource from $failcnt_old to $failcnt (held->$held , maxheld->$maxheld , barrier->$barrier , limit->$limit ) " ;
$ret=2;
}
#print "$vmachine_id: Old_Failcnt: $failcnt_old Failcnt: $failcnt \n";
}
}

}

if ($ret == 0 ) { print "OK. \n" ; }
# print Dumper(%beancounters_old) ;
# print "\n";
exit($ret);
'

RET=$?
fi

echo "$DATA" > $FILE
exit $RET
</source>

Differences between venet and veth

2008-05-19T12:27:00Z

Ginkyo: some cleaning (link, bold, italic, order, VE=>CT, misspelling)

OpenVZ provides you to use either [[veth]] (Virtual eTHernet) or [[venet]] (Virtual NETwork) devices (or both) for in-[[CT]] networking. Here we describe the differences between those devices.

* ''veth'' allows broadcasts in CT, so you can use even a DHCP server inside a CT, or a samba server with domain broadcasts or other such stuff.
* ''veth'' has some security implications, so is not recommended in untrusted environments like HSP. This is due to broadcasts, traffic sniffing, possible IP collisions etc. i.e. CT's user can actually ruin your ethernet network with such direct access to ethernet layer.
* With ''veth'' device, only node administrator can assign an IP to a CT. With ''veth'' device, network settings can be fully done on CT side. CT should setup correct gateway, IP/netmask etc. and then a [[HN|node]] admin can only choose where your traffic goes.
* ''veth'' devices can be bridged together and/or with other devices. For example, in host system admin can bridge ''veth'' from 2 CTs with some VLAN eth0.X. In this case, these 2 CTs will be connected to this VLAN.
* ''veth'' device is a bit faster and more efficient.
* With ''veth'' devices, IPv6 auto generates an address from MAC.

The brief summary:
{| class="wikitable" style="text-align: center;"
|+ '''Differences between veth and venet'''
! Feature !! [[veth]] !! [[venet]]
|-
! MAC address
| {{yes}} || {{no}}
|-
! Broadcasts inside CT
| {{yes}} || {{no}}
|-
! Traffic sniffing
| {{yes}} || {{no}}
|-
! Network security
| style="background: #ffdddd" | Low <ref>Due to broadcasts, sniffing and possible IP collisions etc.</ref>
| style="background: #ddffdd" | High
|-
! Can be used in bridges
| {{yes}} || {{no}}
|-
! Performance
| style="background: #ffdddd" | Fast
| style="background: #ddffdd" | Fastest
|-
|}
<references/>

[[Category: Networking]]

VEID

2008-05-19T11:13:07Z

Ginkyo: sorry I forget : Category: Definitions

#REDIRECT:[[CTID]]
[[Category: Definitions]]

CTID

2008-05-19T11:12:36Z

Ginkyo: sorry I forget : Category: Definitions

#REDIRECT:[[container]]
[[Category: Definitions]]

VEID

2008-05-19T11:10:36Z

Ginkyo: #REDIRECT:CTID

#REDIRECT:[[CTID]]

CTID

2008-05-19T11:09:58Z

Ginkyo: #REDIRECT:container

#REDIRECT:[[container]]

Container

2008-05-19T11:05:31Z

Ginkyo: Add a note for CTID

A '''container''' (otherwise known as CT, Virtual Environment (VE), Virtual Private Server (VPS) etc.) is one of the main concepts of OpenVZ.

Container is an isolated entity which performs and executes exactly like a stand-alone server. Container can be rebooted independently and have root access, users/groups, IP address(es), memory, processes, files, applications, system libraries and configuration files.

OpenVZ allows to have multiple CTs (up to as many as several hundreds) on a single [[Hardware Node]].

{{Note|in this Wiki, terms ''VE'' and ''container'' are used interchangeably. For new articles, ''container'' is the preferable term.}}

If you want to manage your container, you must use her identifer : a ConTainer's IDentifer (CTID). 
Please note that a quota ID is not always the same as a CTID

{{Note|in this Wiki, terms ''VEID'' and ''CTID'' are used interchangeably. For new articles, ''CTID'' is the preferable term.}}

[[Category: Definitions]]

Disk quota

2008-05-19T09:15:48Z

Ginkyo: add command for space available in VE

Basic disk quota management:

To set disk space, run the following commands:
<pre>
vzctl set CTID --diskspace $SoftLimit$:$HardLimit$ --save
</pre>

Example:
<pre>
[host-node]# vzctl set 101 --diskspace 6G:7G --save
</pre>

You could verify the space available with this command (ie : CTID =101)
<pre>
[host-node]# vzctl exec 101 df -h
</pre>

[[Category: Disk quota]]