OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Talk:Virtual Ethernet device

2008-03-27T12:58:18Z

Major: /* Multiple persistent Veth interfaces */

Under Common Configurations -> Simple configuration -> Configure devices in VE0, the example shows

<pre>
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

being run on the host node, VE0. I don't think this can be correct, because eth0 exists in VE 101, not VE0.
-- {{unsigned|Andrex}}

: This is correct, because we need to enable forwarding on both network interfaces in VE0 (host-node eth0 and veth101.0) to allow the network packets on one network interface (host-node eth0) to be forwarded to another network interface (veth101.0). The same thing with proxy ARP, we need to enable it on host-node eth0 and veth101.0 network interfaces. --[[User:Major|Major]] 08:42, 17 July 2006 (EDT)

==Multiple persistent Veth interfaces==
The script for making a persistent bridge get added to a particular host-bridge on startup VPS/VE, doesn't seem to account for having multiple bridges. I have a bridge to a private lan for administrative functions and a bridge to the public network for inbound connections for services. How might one ensure that the vethVEID.x's are hooked up to the bridges on the host every time the VE is started? --[[User:Btrotter|Btrotter]] 05:34, 17 March 2008 (EDT)

: Please take a look on articles [http://wiki.openvz.org/Using_private_IPs_for_Hardware_Nodes Using private IPs for Hardware Nodes] and [http://vireso.blogspot.com/2008/02/2-veth-with-2-brindges-on-openvz-at.html Bridged Networks for OpenVZ]. There you will find scripts which will help you to manage 2 bridges, and you can simply extend this script to use more then 2 bridges. --[[User:Major|Major]] 15:59, 27 March 2008 (MSK)

Talk:Virtual Ethernet device

2008-03-27T12:57:18Z

Major: /* Answer: Multiple persistent Veth interfaces */

VLAN

2007-02-05T09:53:35Z

Major: VLAN

A virtual LAN, commonly known as a '''vLAN''' or as a '''VLAN''', is a method of creating independent logical networks within a physical network. Several VLANs can co-exist within such a network. This helps in reducing the broadcast domain and administratively separating logical segments of LAN (like company departments) which should not exchange data using LAN (they still can by routing).

A VLAN consists of a network of computers that behave as if connected to the same wire - even though they may actually be physically connected to different segments of a LAN. Network administrators configure VLANs through software rather than hardware, which makes them extremely flexible. One of the biggest advantages of VLANs emerges when physically moving a computer to another location: it can stay on the same VLAN without the need for any hardware reconfiguration.

VLAN 1 is the default VLAN; it can never be deleted. All untagged traffic falls into this VLAN by default.

==Advantages of VLAN==
* Increases the number of '''broadcast domains''' but reduces the size of each '''broadcast domain''', which in turn reduces network traffic and increases network security (both of which are hampered in case of single large broadcast domain)
* Reduces management effort to create subnetworks
* Reduces hardware requirement, as networks can be logically instead of physically separated
* Increases control over multiple traffic types.

== Common VLAN configurations for VE ==
VLAN can be used in following ways:
* Create VLAN device on physical network interface (eth0) and move it (VLAN device) to VE:
<pre>
host # vconfig add eth0 <vlan_id>
host # vzctl set <VEID> --netdev_add eth0.<vlan_id> --save
</pre>
* Create VLAN device inside VE on veth device
<pre>
ve # vconfig add eth0 <vlan_id>
</pre>

The second option is available only in kernel with virtualized VLAN (since 2.6.18-028test005 version).

Demo scripts

2006-10-30T16:59:33Z

Major: /* Massive VE load */

The following demo scripts (scenarios) can be used to show advantages of OpenVZ.

== Full VE lifecycle ==

Create VE, set IP, start, add user, enter, exec, show ps -axf output inside VE, stop, and destroy. It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!"). During the demonstration, describe what's happening and why.

Here are the example commands needed:

# VE=123
# IP=10.1.1.123
# sed -i "/$IP /d" ~/.ssh/
# time vzctl create $VE --ostemplate fedora-core-5-i386-default
# vzctl set $VE --ipadd $IP --hostname newVE --save
# vzctl start $VE
# vzctl exec $VE ps axf
# vzctl set $VE --userpasswd guest:secret --save
# ssh guest@$IP
[newVE]# ps axf
[newVE]# logout
# vzctl stop $VE
# vzctl destroy $VE

== Massive VE creation ==

Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

Here are the example commands needed:

<pre>
# time for ((VE=200; VE<250; VE++)); do \
> time vzctl create $VE --ostemplate fedora-core-5-i386-default; \
> vzctl start $VE; \
> done
</pre>

== Massive VE load ==

Use VEs from previous item — load those by <code>ab</code> or <code>http_load</code>. This demo shows that multiple VEs are working just fine, with low response time etc.

<pre>
# for ((VE=200; VE<250; VE++)); do \
> vzctl set $VE --ipadd 10.1.1.$VE --save; \
> done
</pre>
On another machine:
<pre>
# rpm -ihv http_load
#
</pre>
FIXME: http_load commands

== Live migration ==

If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

FIXME: commands, setup, vnc template.

== Resource management ==
Below scenarios aims to show how OpenVZ resource management works.

=== [[UBC]] protection ===

==== fork() bomb ====
<pre>
# while [ true ]; do \
> while [ true ]; do \
> echo " " > /dev/null;
> done &
> done
</pre>
We can see that number of processes inside VE will not be grow up. We will see only increase of <code>numproc</code> or <code>kmemsize</code> fail counters in <code>/proc/user_beancounters</code>.

==== dentry cache eat up ====
FIXME

=== CPU scheduler ===
Create 3 VEs:
<pre>
# vzctl create 101
# vzctl create 102
# vzctl create 103
</pre>

Set VEs weights:
<pre>
# vzctl set 101 --cpuunits 1000 --save
# vzctl set 102 --cpuunits 2000 --save
# vzctl set 103 --cpuunits 3000 --save
</pre>

We set next cpu sharing <code>VE101 : VE102 : VE103 = 1 : 2 : 3</code>

Run VEs:
<pre>
# vzctl start 101
# vzctl start 102
# vzctl start 103
</pre>

Run busy loops in VEs:
<pre>
# vzctl enter 101
[ve101]# while [ true ]; do true; done
# vzctl enter 102
[ve102]# while [ true ]; do true; done
# vzctl enter 103
[ve103]# while [ true ]; do true; done
</pre>

Check in top that sharing works:
<pre>
# top
COMMAND %CPU
bash 48.0
bash 34.0
bash 17.5
</pre>

So, we see that CPU time is given to VEs in proportion ~ 1 : 2 : 3.

=== Disk quota ===
<pre>
# vzctl set VEID --diskspace 1048576:1153434 --save
# vzctl start VEID
# vzctl enter VEID
[ve]# dd if=/dev/zero of=/tmp/tmp.file bs=1048576 count=1000
dd: writing `/tmp/tmp.file': Disk quota exceeded
</pre>

Demo scripts

2006-10-30T16:13:34Z

Major: /* Massive VE creation */

The following demo scripts (scenarios) can be used to show advantages of OpenVZ.

== Full VE lifecycle ==

Create VE, set IP, start, add user, enter, exec, show ps -axf output inside VE, stop, and destroy. It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!"). During the demonstration, describe what's happening and why.

Here are the example commands needed:

# VE=123
# IP=10.1.1.123
# sed -i "/$IP /d" ~/.ssh/
# time vzctl create $VE --ostemplate fedora-core-5-i386-default
# vzctl set $VE --ipadd $IP --hostname newVE --save
# vzctl start $VE
# vzctl exec $VE ps axf
# vzctl set $VE --userpasswd guest:secret --save
# ssh guest@$IP
[newVE]# ps axf
[newVE]# logout
# vzctl stop $VE
# vzctl destroy $VE

== Massive VE creation ==

Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

Here are the example commands needed:

<pre>
# time for ((VE=200; VE<250; VE++)); do \
> time vzctl create $VE --ostemplate fedora-core-5-i386-default; \
> vzctl start $VE; \
> done
</pre>

== Massive VE load ==

Use VEs from previous item — load those by <code>ab</code> or <code>http_load</code>. This demo shows that multiple VEs are working just fine, with low response time etc.

FIXME: commands, ab/http_load setup.

== Live migration ==

If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

FIXME: commands, setup, vnc template.

== Resource management ==
Below scenarios aims to show how OpenVZ resource management works.

=== [[UBC]] protection ===

==== fork() bomb ====
<pre>
# while [ true ]; do \
> while [ true ]; do \
> echo " " > /dev/null;
> done &
> done
</pre>
We can see that number of processes inside VE will not be grow up. We will see only increase of <code>numproc</code> or <code>kmemsize</code> fail counters in <code>/proc/user_beancounters</code>.

==== dentry cache eat up ====
FIXME

=== CPU scheduler ===
Create 3 VEs:
<pre>
# vzctl create 101
# vzctl create 102
# vzctl create 103
</pre>

Set VEs weights:
<pre>
# vzctl set 101 --cpuunits 1000 --save
# vzctl set 102 --cpuunits 2000 --save
# vzctl set 103 --cpuunits 3000 --save
</pre>

We set next cpu sharing <code>VE101 : VE102 : VE103 = 1 : 2 : 3</code>

Run VEs:
<pre>
# vzctl start 101
# vzctl start 102
# vzctl start 103
</pre>

Run busy loops in VEs:
<pre>
# vzctl enter 101
[ve101]# while [ true ]; do true; done
# vzctl enter 102
[ve102]# while [ true ]; do true; done
# vzctl enter 103
[ve103]# while [ true ]; do true; done
</pre>

Check in top that sharing works:
<pre>
# top
COMMAND %CPU
bash 48.0
bash 34.0
bash 17.5
</pre>

So, we see that CPU time is given to VEs in proportion ~ 1 : 2 : 3.

=== Disk quota ===
<pre>
# vzctl set VEID --diskspace 1048576:1153434 --save
# vzctl start VEID
# vzctl enter VEID
[ve]# dd if=/dev/zero of=/tmp/tmp.file bs=1048576 count=1000
dd: writing `/tmp/tmp.file': Disk quota exceeded
</pre>

Demo scripts

2006-10-30T16:08:06Z

Major: /* fork() bomb */

The following demo scripts (scenarios) can be used to show advantages of OpenVZ.

== Full VE lifecycle ==

Create VE, set IP, start, add user, enter, exec, show ps -axf output inside VE, stop, and destroy. It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!"). During the demonstration, describe what's happening and why.

Here are the example commands needed:

# VE=123
# IP=10.1.1.123
# sed -i "/$IP /d" ~/.ssh/
# time vzctl create $VE --ostemplate fedora-core-5-i386-default
# vzctl set $VE --ipadd $IP --hostname newVE --save
# vzctl start $VE
# vzctl exec $VE ps axf
# vzctl set $VE --userpasswd guest:secret --save
# ssh guest@$IP
[newVE]# ps axf
[newVE]# logout
# vzctl stop $VE
# vzctl destroy $VE

== Massive VE creation ==

Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

Here are the example commands needed:

# VE=200
# time while [ $VE -lt 250 ]; do \
> time vzctl create $VE --ostemplate fedora-core-5-i386-default; \
> vzctl start $VE; \
> let VE++; \
> done

== Massive VE load ==

Use VEs from previous item — load those by <code>ab</code> or <code>http_load</code>. This demo shows that multiple VEs are working just fine, with low response time etc.

FIXME: commands, ab/http_load setup.

== Live migration ==

If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

FIXME: commands, setup, vnc template.

== Resource management ==
Below scenarios aims to show how OpenVZ resource management works.

=== [[UBC]] protection ===

==== fork() bomb ====
<pre>
# while [ true ]; do \
> while [ true ]; do \
> echo " " > /dev/null;
> done &
> done
</pre>
We can see that number of processes inside VE will not be grow up. We will see only increase of <code>numproc</code> or <code>kmemsize</code> fail counters in <code>/proc/user_beancounters</code>.

==== dentry cache eat up ====
FIXME

=== CPU scheduler ===
Create 3 VEs:
<pre>
# vzctl create 101
# vzctl create 102
# vzctl create 103
</pre>

Set VEs weights:
<pre>
# vzctl set 101 --cpuunits 1000 --save
# vzctl set 102 --cpuunits 2000 --save
# vzctl set 103 --cpuunits 3000 --save
</pre>

We set next cpu sharing <code>VE101 : VE102 : VE103 = 1 : 2 : 3</code>

Run VEs:
<pre>
# vzctl start 101
# vzctl start 102
# vzctl start 103
</pre>

Run busy loops in VEs:
<pre>
# vzctl enter 101
[ve101]# while [ true ]; do true; done
# vzctl enter 102
[ve102]# while [ true ]; do true; done
# vzctl enter 103
[ve103]# while [ true ]; do true; done
</pre>

Check in top that sharing works:
<pre>
# top
COMMAND %CPU
bash 48.0
bash 34.0
bash 17.5
</pre>

So, we see that CPU time is given to VEs in proportion ~ 1 : 2 : 3.

=== Disk quota ===
<pre>
# vzctl set VEID --diskspace 1048576:1153434 --save
# vzctl start VEID
# vzctl enter VEID
[ve]# dd if=/dev/zero of=/tmp/tmp.file bs=1048576 count=1000
dd: writing `/tmp/tmp.file': Disk quota exceeded
</pre>

Demo scripts

2006-10-30T14:30:52Z

Major: /* CPU scheduler */

The following demo scripts (scenarios) can be used to show advantages of OpenVZ.

== Full VE lifecycle ==

Create VE, set IP, start, add user, enter, exec, show ps -axf output inside VE, stop, and destroy. It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!"). During the demonstration, describe what's happening and why.

Here are the example commands needed:

# VE=123
# IP=10.1.1.123
# sed -i "/$IP /d" ~/.ssh/
# time vzctl create $VE --ostemplate fedora-core-5-i386-default
# vzctl set $VE --ipadd $IP --hostname newVE --save
# vzctl start $VE
# vzctl exec $VE ps axf
# vzctl set $VE --userpasswd guest:secret --save
# ssh guest@$IP
[newVE]# ps axf
[newVE]# logout
# vzctl stop $VE
# vzctl destroy $VE

== Massive VE creation ==

Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

Here are the example commands needed:

# VE=200
# time while [ $VE -lt 250 ]; do \
> time vzctl create $VE --ostemplate fedora-core-5-i386-default; \
> vzctl start $VE; \
> let VE++; \
> done

== Massive VE load ==

Use VEs from previous item — load those by <code>ab</code> or <code>http_load</code>. This demo shows that multiple VEs are working just fine, with low response time etc.

FIXME: commands, ab/http_load setup.

== Live migration ==

If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

FIXME: commands, setup, vnc template.

== Resource management ==
Below scenarios aims to show how OpenVZ resource management works.

=== [[UBC]] protection ===

==== fork() bomb ====
<pre>
# while [ true ]; do \
> while [ true ]; do \
> echo " " > /dev/null;
> done &
> done
</pre>

==== dentry cache eat up ====
FIXME

=== CPU scheduler ===
Create 3 VEs:
<pre>
# vzctl create 101
# vzctl create 102
# vzctl create 103
</pre>

Set VEs weights:
<pre>
# vzctl set 101 --cpuunits 1000 --save
# vzctl set 102 --cpuunits 2000 --save
# vzctl set 103 --cpuunits 3000 --save
</pre>

We set next cpu sharing <code>VE101 : VE102 : VE103 = 1 : 2 : 3</code>

Run VEs:
<pre>
# vzctl start 101
# vzctl start 102
# vzctl start 103
</pre>

Run busy loops in VEs:
<pre>
# vzctl enter 101
[ve101]# while [ true ]; do true; done
# vzctl enter 102
[ve102]# while [ true ]; do true; done
# vzctl enter 103
[ve103]# while [ true ]; do true; done
</pre>

Check in top that sharing works:
<pre>
# top
COMMAND %CPU
bash 48.0
bash 34.0
bash 17.5
</pre>

So, we see that CPU time is given to VEs in proportion ~ 1 : 2 : 3.

=== Disk quota ===
<pre>
# vzctl set VEID --diskspace 1048576:1153434 --save
# vzctl start VEID
# vzctl enter VEID
[ve]# dd if=/dev/zero of=/tmp/tmp.file bs=1048576 count=1000
dd: writing `/tmp/tmp.file': Disk quota exceeded
</pre>

Demo scripts

2006-10-27T12:02:30Z

Major: /* CPU scheduler */

The following demo scripts (scenarios) can be used to show advantages of OpenVZ.

== Full VE lifecycle ==

Create VE, set IP, start, add user, enter, exec, show ps -axf output inside VE, stop, and destroy. It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!"). During the demonstration, describe what's happening and why.

Here are the example commands needed:

# VE=123
# IP=10.1.1.123
# sed -i "/$IP /d" ~/.ssh/
# time vzctl create $VE --ostemplate fedora-core-5-i386-default
# vzctl set $VE --ipadd $IP --hostname newVE --save
# vzctl start $VE
# vzctl exec $VE ps axf
# vzctl set $VE --userpasswd guest:secret --save
# ssh guest@$IP
[newVE]# ps axf
[newVE]# logout
# vzctl stop $VE
# vzctl destroy $VE

== Massive VE creation ==

Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

Here are the example commands needed:

# VE=200
# time while [ $VE -lt 250 ]; do \
> time vzctl create $VE --ostemplate fedora-core-5-i386-default; \
> vzctl start $VE; \
> let VE++; \
> done

== Massive VE load ==

Use VEs from previous item — load those by <code>ab</code> or <code>http_load</code>. This demo shows that multiple VEs are working just fine, with low response time etc.

FIXME: commands, ab/http_load setup.

== Live migration ==

If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

FIXME: commands, setup, vnc template.

== Resource management ==
Below scenarios aims to show how OpenVZ resource management works.

=== [[UBC]] protection ===

==== fork() bomb ====
<pre>
# while [ true ]; do \
> while [ true ]; do \
> echo " " > /dev/null;
> done &
> done
</pre>

==== dentry cache eat up ====
FIXME

=== CPU scheduler ===
Create 3 VPSes:
<pre>
# vzctl create 101
# vzctl create 102
# vzctl create 103
</pre>

Set VPS weights:
<pre>
# vzctl set 101 --cpuunits 1000 --save
# vzctl set 102 --cpuunits 2000 --save
# vzctl set 103 --cpuunits 3000 --save
</pre>

We set next cpu sharing <code>VPS101 : VPS102 : VPS103 = 1 : 2 : 3</code>

Run VPSes:
<pre>
# vzctl start 101
# vzctl start 102
# vzctl start 103
</pre>

Run busy loops in VPSes:
<pre>
# vzctl enter 101
[ve101]# while [ true ]; do true; done
# vzctl enter 102
[ve102]# while [ true ]; do true; done
# vzctl enter 103
[ve103]# while [ true ]; do true; done
</pre>

Check in top that sharing works:
<pre>
# top
COMMAND %CPU
bash 48.0
bash 34.0
bash 17.5
</pre>

=== Disk quota ===
<pre>
# vzctl set VEID --diskspace 1048576:1153434 --save
# vzctl start VEID
# vzctl enter VEID
[ve]# dd if=/dev/zero of=/tmp/tmp.file bs=1048576 count=1000
dd: writing `/tmp/tmp.file': Disk quota exceeded
</pre>

Demo scripts

2006-10-27T10:47:13Z

Major: /* Disk quota */

Demo scripts

2006-10-27T10:04:23Z

Major: /* edit fork() bomb */

Demo scripts

2006-10-25T13:36:55Z

Major: Small corrections

Demo scripts which can be used to show advantages of OpenVZ:

* Full VE lifecycle (create, set ip, start, add user, enter, exec, show ps -axf output inside VE), stop, destroy). It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!")

* Massive VE creation. Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

* Use VEs from prev. item — load those by <code>ab</code> or <code>http_load</code> — shows that many VE are working quite fine, with low response time etc.

* If you have two boxes, do "<code>vzmigrate --online</code>" from one box to another. You can use, say, <code>xvnc</code> in a VE and <code>vncclient</code> to connect to it, then run <code>xscreensaver-demo</code> and while the picture is moving do a live migration. You'll show <code>xscreensaver</code> stalls for a few seconds but then keeps running — on another machine! That looks amazing, to say at least.

Demo scripts

2006-10-25T13:32:35Z

Major: Demo scripts

Demo scripts which can be used to show advantages of OpenVZ:

* Full VE lifecycle (create, set ip, start, add user, enter, exec, show ps -axf output inside VE), stop, destroy). It should take two minutes ("compare that to a time you need to deploy a new (non-virtual) server!")

* Massive VE creation. Create/start 50 or 100 VEs in a shell loop. Shows fast deployment and high density.

* Use VEs from prev. item -- load those by ab or http_load -- shows that many VE are working quite fine, with low response time etc.

* If you have two boxes, do vzmigrate --online from one box to another. You can use, say, xvnc in a VE and vncclient to connect to it, then run xscreensaver-demo and while the picture is moving do a live migration. You'll show xscreensaver stalls for a few seconds but then keeps running -- on another machine! That looks amazing, to say at least.

Checkpointing internals

2006-09-06T13:51:55Z

Major: Checkpointing internals

= Checkpointing internals =

Process checkpoint/restore consists of two phases. The first phase is to save the running state of a process. This usually includes register set, address space, allocated resources, and other process private data. The second phase is to re-construct the original running process from the saved image and resume the execution exactly the point, where it was suspended.

There are several problems with existing checkpoint/restore systems. First, except some written-from-scratch process migration operating systems (such as Sprite), they can not preserve opened network connections. Second, general-purpose operating systems such as Unix were not designed to support process migration, so checkpoint/restore systems built on top of existing OSes usually only support a limited set of applications. Third, all systems do not guarantee processes restoration on other side because of resource conflicts (e.g. there can be a process with such pid). OpenVZ gives a unique chance to solve all those problems and to implement full-fledged universal checkpoint/restore system, its intrinsic capability to isolate and to virtualize groups of processes allows to define a self-consistent state essentially for any configurations of VEs using all the kinds of resources, which are available inside VE.

The primary contributions of OpenVZ checkpoint/restore system are:
* No run time overhead besides actual checkpoint/restore
* Network connection migration support
* Virtualization of pids
* Image size minimization

== Modular Structure ==

Main functionality (checkpoint and restore functions) implemented as two separate kernel modules:

'''vzcpt''' - provides checkpoint functionality;

'''vzrst''' - provides restore functionality.

Checkpoint and restore are controlled via <code>ioctl()</code> calls on regular pseudo-files <code>/proc/cpt</code> and <code>/proc/rst</code> created in <code>procfs</code>. Ioctl commands are listed in <code><linux/cpt_ioctl.h></code>.

== Checkpoint Module ==

Checkpoint module (<code>vzcpt</code>) provides the following general functionality by ioctls:
* <code>CPT_SUSPEND</code> – moving processes to frozen state (VE suspending);
* <code>CPT_DUMP</code> – collecting and saving all VPS data to image file (VE dumping);
* <code>CPT_KILL</code> – killing VE;
* <code>CPT_RESUME</code> – resuming processes from frozen state to running state.

Freezing all the processes before saving VE state is necessary because processes inside VE can be connected via IPC, can send signals, share files, virtual memory and another objects. To guarantee self-consistency of saved state all the processes must be suspended and network connections must be stopped.

== Restore Module ==

Restore module (<code>vzrst</code>) provides the following general functionality by ioctls:
* <code>CPT_UNDUMP</code> – reconstructing processes and VE private data from image file (VE undumping);
* <code>CPT_KILL</code> – killing VE;
* <code>CPT_RESUME</code> – resuming processes from frozen state to running state.

After reconstructing all necessary kernel structures processes are placed in an uninterruptible state, so that processes cannot run before reconstruction of full VE will be completed. Only after the whole VE is restored it is possible to resume network connectivity and to wake up the processes. It is necessary to emphasize, it is impossible to reduce latency of migration waking up some processes before all the VE is restored.

== Virtualization of pids ==

A process has an identifier (<code>PID</code>), which is unaltered while process lifecycle. So, it is necessary to restore pid after migration. But it is impossible to do this if there is another process with the same pid. This problem was solved in the following way.

Processes created inside VE are assigned with pair of pids: one is traditional pid, i.e. a global value which uniquely identifies the process in host OS. Another is virtual pid which is unique only inside VE but can be used by several VEs. Processes inside VE communicate using only their virtual pids, so that provided virtual pids are preserved while checkpointing/restore the whole VE can be transferred to another hardware node not causing pid conflicts. Migrated processes get another global pids but this pid is invisible from inside VE.

Main drawback of this solution is that it is necessary to maintain mapping of virtual pids to global pids, which introduces additional overhead for all the syscalls using a pid as an argument or a return value (f.e. <code>kill()</code>, <code>wait4()</code>, <code>fork()</code>). The overhead (~0.3%) is visible in the tests, which essentially do nothing but forking and stopping processes. This overhead appears only for online migrated VEs. There are no overhead at all for VEs, which never migrated.

== Image size minimization ==

CPT needs to save all the VE data to a file after VE was suspended and to transfer this file to the destination node before VE can be restored. It means that migration latency is proportional to total size of this image file. Though actual image sizes are surprisingly small for typical tasks 
2 Mb – idle apache with 8 preforked children 
20 Mb – screen + bash + cxoffice + winword + small document 
24 Mb – screen + bash + mozilla + Java VM 
0.7 Mb – screen + 1 bash 
3 Mb – screen + 8 bashes in chain 
13 Mb – screen + acroread on 7 Mb pdf file 
2 Mb – running full Zeus-4.2r4 
0.9 Mb – sshd with one forked child and bash 
3 Mb – mysqld 
1 Mb – postgresql server 
2.5 Mb – CommuniGate Pro 4.0.6 
25 Mb – phhttps with LinuxThreads. Doc set is /var/www/manual/*.en 

they can be much larger, when the VE to be migrated runs processes which use lots of virtual memory.

== Limitations ==

CPT implements migration of almost all kernel objects, but not all of them. When CPT sees that a process in VE makes use of an unimplemented facility it does not allow migration.

Another kind of limitation is when a process uses some facilities, which are not available at target node. For example, a process can auto detect CPU at runtime and start using some instructions specific for this CPU: SSE2, CMOV instructions etc. In this case migration is possible only when destination node also supports those facilities.

Third kind of limitations is caused by applications, which use non-virtual capabilities, which are directly accessible at user level. F.e. a process could use CPU timestamps to calculate some timings, or it could use SMP CPU ID to optimize memory accesses. In this case completely transparent migration would be possible only using virtualization techniques provided by the latest Intel CPUs and it is not even clear, whether unavoidable overhead introduced at this level of virtualization is worth of maintaining such exotic applications.

Checkpointing and live migration

2006-09-06T12:42:23Z

Major: Checkpointing and live migration

CPT is an extension to OpenVZ kernel which allows to save full state of a running VE and to restore it later on the same or on a different host in a way transparent for running applications and network connections. This technique has several applications, the most important being live (zero-downtime) migration of VEs and taking an instant snapshot of a running VE for later resume, i. e. CheckPoinTing.

Before CPT, it was only possible to migrate a VE through shutdown and subsequent reboot. The procedure not only introduces quite a long downtime of network services, it is not transparent for clients using the VE, making impossible migration, when clients runs some tasks which are not tolerant to shutdowns.

Comparing to this old scheme, CPT allows to migrate a VE in a way, essentially invisible both for users of this VE and for external clients, using network services located inside VE. It still introduces a short delay in service, required for actual checkpoint/restore of the processes, but this delay is indistinguishable from a short interruption of network connectivity.

== Online migration ==

There is special utility vzmigrate in OpenVZ distribution intended to support VE migration. With it help one can perform live (zero-downtime) migration, i.e. while migration VPS hangs for a while and after migration it continues work as though nothing has happened. Online migration can be performed by
<pre>vzmigrate --online <host> VEID</pre>
command. During online migration all VE private data saved to an image file, which is transferred to target host.

== Manual Checkpoint and Restore Functions ==

<code>vzmigrate</code> is not strictly required to perform online migration. <code>vzctl</code> utility, accompanied with some file system backup tools, provides enough of power to do all the tasks.

VE can be checkpointed with command:
<pre>vzctl chkpnt VEID --dumpfile <path></pre>
This command saves all the state of running VE to dump file and stops the VE. If the option <code>--dumpfile</code> is not set, <code>vzctl</code> uses default path <code>/var/tmp/Dump.VEID</code>.

After this it is possible to restore the VE exactly in the same state executing:
<pre>vzctl restore VEID --dumpfile <path></pre>
If dump file and file system is transferred to another HW node, the same command can restore VE there with the same success.

It is critical requirement that file system at the moment of restore must be identical to the file system at the moment of checkpointing. If this requirement is not held, depending on severity of changes process of restoration can be aborted or the processes inside VE can see this as an external corruption of open files. When VE is restored on the same node where it was checkpointed, it is enough not to touch file system accessible by the VE. When VE is transferred to another node it is necessary to synchronize VE file system before restore. <code>vzctl</code> does not provide this functionality and external tools (f.e. <code>rsync</code>) are required.

== Step-by-step Checkpoint and Restore ==

Process of checkpointing can be performed by stages. It consists of three steps.

First step – suspending VE. At this stage CPT moves all the processes to special beforehand known state and stops VE network interfaces. This stage can be done by
<pre>vzctl chkpnt VEID --suspend</pre>
command. Second step – dumping VE. At this phase CPT saves state of processes and global state of VE to image file. All the process private data need to be saved: address space, register set, opened files/pipes/sockets, System V IPC structures, current working directory, signal handlers, timers, terminal settings, user identities (uid, gid, etc), process identities (pid, pgrp, sid, etc), rlimit and other data. This stage can be done by
<pre>vzctl chkpnt VEID --dump --dumpfile <path></pre>
command. Third step – killing or resuming processes. If the migration succeeds VE can be stopped with the command:
<pre>vzctl chkpnt VEID --kill</pre>
If migration failed by some reason or if the goal was taking a snapshot of VE state for later restore, CPT can resume VE with:
<pre>vzctl chkpnt VEID --resume</pre>

Process of restoring consists of two steps. The first step is to restore processes and to leave them in a special frozen state. After this step processes are ready to continue execution, however, in some cases CPT has to do some operations after process is woken up, therefore CPT sets process return point to function in our module. This stage can be done by
<pre>vzctl restore VEID --undump --dumpfile <path></pre>
command. Second step – waking up processes or killing them if restore process failed. After CPT wakes up process, it performs necessary operations in our function and continues execution. This stages can be done by
<pre>vzctl restore VEID --resume
vzctl restore VEID --kill</pre>
commands.

Talk:Virtual Ethernet device

2006-07-17T12:42:37Z

Major:

Virtual Ethernet device

2006-06-28T14:39:00Z

Major: Differences between venet and veth moved to separate topic

Virtual ethernet device is ethernet device which can be used inside a [[VE]]. Unlike
venet network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to
ethX or other device and VPS user fully setups his networking himself,
including IPs, gateways etc.

Virtual ethernet device consist of two ethernet devices - one in [[VE0]] and another one
in VE. These devices are connected to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual ethernet device usage ==

=== Adding veth to a VE ===
<pre>
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
</pre>
Here
* <tt>dev_name</tt> is ethernet device name in the [[VE0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is an ethernet device name in the VE
* <tt>ve_dev_addr</tt> is its MAC address

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. Note that this option
is incremental, so devices are added to already existing ones.

==== Examples ====
<pre>
vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>
After executing this command <tt>veth</tt> device will be created for VE 101 and veth configuration will be saved to a VE configuration file.
Host-side ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
VE-side ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.
{{Note|Use random MAC addresses. Do not use MAC addresses of real eth devices, beacuse this can lead to collisions.}}

=== Removing veth from a VE ===
<pre>
vzctl set <VEID> --veth_del <dev_name>
</pre>
Here <tt>dev_name</tt> is the ethernet device name in the [[VE0|host system]].

==== Example ====
<pre>
vzctl set 101 --veth_del veth101.0 --save
</pre>
After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.

== Common configurations with virtual ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual ethernet device ===

==== Start a VE ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to VE ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in VE0 ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

==== Configure device in VE ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

==== Add route in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.0.101 dev veth101.0
</pre>

=== Virtual ethernet device with IPv6 ===

==== Start [[VE]] ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to [[VE]] ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in [[VE0]] ====
<pre>
[host-node]# ifconfig veth101.0 0
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
</pre>

==== Configure device in [[VE]] ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 0
</pre>

==== Start router advertisement daemon (radvd) for IPv6 in VE0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:
<pre>
[host-node]# /etc/init.d/radvd start
</pre>

==== Add IPv6 addresses to devices in [[VE0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several VEs and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 0
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to VEs will be through this bridge and VEs can communicate with each other even without these routes.

=== Virtual ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]

[[Category: Networking]]
[[Category: HOWTO]]

Differences between venet and veth

2006-06-28T14:38:06Z

Major: Category changed

= Differences between venet and veth =
* veth allows broadcasts in VE, so you can use even dhcp server inside VE or samba server with domain broadcasts or other such stuff.
* veth has some security implications, so is not recommended in untrusted environments like HSP. This is due to broadcasts, traffic sniffing, possible IP collisions etc. i.e. VE user can actually ruin your ethernet network with such direct access to ethernet layer.
* With venet device, only node administrator can assign an IP to a VE. With veth device, network settings can be fully done on VE side. VE should setup correct GW, IP/mask etc and node admin then can only choose where your traffic goes.
* veth devices can be bridged together and/or with other devices. For example, in host system admin can bridge veth from 2 VEs with some VLAN eth0.X. In this case, these 2 VEs will be connected to this VLAN.
* venet device is a bit faster and more efficient.
* With veth devices IPv6 auto generates an address from MAC.

The brief summary:
{| class="wikitable" style="text-align: center;"
|+ '''Differences between veth and venet'''
! Feature !! veth !! venet
|-
! MAC address
| {{yes}} || {{no}}
|-
! Broadcasts inside VE
| {{yes}} || {{no}}
|-
! Traffic sniffing
| {{yes}} || {{no}}
|-
! Network security
| low <ref>Due to broadcasts, sniffing and possible IP collisions etc.</ref> || hi
|-
! Can be used in bridges
| {{yes}} || {{no}}
|-
! Performance
| fast || fastest
|-
|}
<references/>

[[Category: Networking]]

Virtual network device

2006-06-28T14:37:33Z

Major: Category changed

Vitual network device (venet) is the default network device for [[VE]]. This network device looks like a ppp connection between [[VE]] and [[VE0|host system]]. It does packet switching based on IP header.

venet is created automatically on [[VE]] start. vzctl scripts setup appropriate IP and settings on venet inside VPS.

= Virtual network device usage =

== Adding IP address to a VE ==
<pre>
vzctl set <VEID> --ipadd <IP1>[,<IP2>,...] [--save]
</pre>

{{Note|This option is incremental, so IP addresses are added to already existing ones.}}

=== Example ===
<pre>
vzctl set 101 --ipadd 10.0.0.1 --save
</pre>
After executing this command IP address 10.0.0.1 will be added to VE 101 and IP configuration will be saved to a VE configuration file.

== Removing IP address from a VE ==
<pre>
vzctl set <VEID> --ipdel <IP1>[,<IP2>,...] [--save]
</pre>

=== Example ===
<pre>
vzctl set 101 --ipadd 10.0.0.1
</pre>
After executing this command IP address 10.0.0.1 will be removed from VE 101, but IP configuration will not be changed in VE config file. And after VE reboot IP address 10.0.0.1 will be assigned to this VE again.

[[Category: Networking]]
[[Category: HOWTO]]

Differences between venet and veth

2006-06-28T14:34:27Z

Major: Differences between venet and veth

Virtual network device

2006-06-28T14:32:25Z

Major: Virtual network device page added

Virtual Ethernet device

2006-06-09T07:56:26Z

Major: General info updated

Virtual ethernet device is ethernet device which can be used inside a [[VE]]. Unlike
venet network device, veth device has a MAC address. Due to this, it can be used in configurations, when veth is bridged to
ethX or other device and VPS user fully setups his networking himself,
including IPs, gateways etc.

Virtual ethernet device consist of two ethernet devices - one in [[VE0]] and another one
in VE. These devices are connected to each other, so if a packet goes to one
device it will come out from the other device.

List of differences with venet device:
- veth allows broadcasts in VPS, so you can use even dhcp server
inside VPS or samba server with domain broadcasts or other such stuff.
- veth has some security implications, so is not recommended in
untrusted environments like HSP. This is due to broadcasts,
traffic sniffing, possible IP collisions etc. i.e. VPS user can
actually ruin your ethernet network with such direct access to
ethernet layer.
- with venet device, only node administrator can assign IP to VPS.
With veth device, network settings can be fully done on VPS side.
VPS should setup correct GW, IP/mask etc and node admin then can
only choose where your traffic goes.
- veth devices can be bridged together and/or with other devices.
For example, in host system admin can bridge veth from 2 VPSs with
some VLAN eth0.X. In this case, these 2 VPSs will be connected to
this VLAN.
- venet device is a bit faster and more efficient.
- with veth devices IPv6 auto generates an address from MAC.

The brief summary:
veth venet
MAC address + -
broadcasts inside VPS + -
traffic sniffing + -
network security low hi
(due to broadcasts,
sniffing and possible IP
collisions etc.)
can be used in bridges + -
performance fast fastest

== Virtual ethernet device usage ==

=== Adding veth to a VE ===
<pre>
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
</pre>
Here
* <tt>dev_name</tt> is ethernet device name in the [[VE0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is an ethernet device name in the VE
* <tt>ve_dev_addr</tt> is its MAC address

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. Note that this option
is incremental, so devices are added to already existing ones.

==== Examples ====
<pre>
vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>
After executing this command <tt>veth</tt> device will be created for VE 101 and veth configuration will be saved to a VE configuration file.
Host-side ethernet device will have <tt>veth101.0</tt> name and <tt>00:12:34:56:78:9A</tt> MAC address.
VE-side ethernet device will have <tt>eth0</tt> name and <tt>00:12:34:56:78:9B</tt> MAC address.
{{Note|Use random MAC addresses. Do not use MAC addresses of real eth devices, beacuse this can lead to collisions.}}

=== Removing veth from a VE ===
<pre>
vzctl set <VEID> --veth_del <dev_name>
</pre>
Here <tt>dev_name</tt> is the ethernet device name in the [[VE0|host system]].

==== Example ====
<pre>
vzctl set 101 --veth_del veth101.0 --save
</pre>
After executing this command veth device with host-side ethernet name veth101.0 will be removed from VE 101 and veth configuration will be updated in VE config file.

== Common configurations with virtual ethernet devices ==
Module <tt>vzethdev</tt> must be loaded to operate with veth devices.

=== Simple configuration with virtual ethernet device ===

==== Start a VE ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to VE ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in VE0 ====
<pre>
[host-node]# ifconfig veth101.0 up
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/veth101.0/proxy_arp
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
</pre>

==== Configure device in VE ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 up
[ve-101]# /sbin/ip addr add 192.168.0.101 dev eth0
[ve-101]# /sbin/ip route add default dev eth0
</pre>

==== Add route in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.0.101 dev veth101.0
</pre>

=== Virtual ethernet device with IPv6 ===

==== Start [[VE]] ====
<pre>
[host-node]# vzctl start 101
</pre>

==== Add veth device to [[VE]] ====
<pre>
[host-node]# vzctl set 101 --veth_add veth101.0,00:12:34:56:78:9A,eth0,00:12:34:56:78:9B --save
</pre>

==== Configure devices in [[VE0]] ====
<pre>
[host-node]# ifconfig veth101.0 up
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/veth101.0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/eth0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
</pre>

==== Configure device in [[VE]] ====
<pre>
[host-node]# vzctl enter 101
[ve-101]# /sbin/ifconfig eth0 up
</pre>

==== Start router advertisement daemon (radvd) for IPv6 in VE0 ====
First you need to edit radvd configuration file. Here is a simple example of <tt>/etc/radv.conf</tt>:
<pre>
interface veth101.0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:2400:0:0::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};

interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 10;
AdvHomeAgentFlag off;

prefix 3ffe:0302:0011:0002::/64
{
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr off;
};
};
</pre>

Then, start radvd:
<pre>
[host-node]# /etc/init.d/radvd start
</pre>

==== Add IPv6 addresses to devices in [[VE0]] ====
<pre>
[host-node]# ip addr add dev veth101.0 3ffe:2400::212:34ff:fe56:789a/64
[host-node]# ip addr add dev eth0 3ffe:0302:0011:0002:211:22ff:fe33:4455/64
</pre>

=== Virtual ethernet devices can be joined in one bridge ===
Perform steps 1 - 4 from Simple configuration chapter for several VEs and/or veth devices

==== Create bridge device ====
<pre>
[host-node]# brctl addbr vzbr0
</pre>

==== Add veth devices to bridge ====
<pre>
[host-node]# brctl addif vzbr0 veth101.0
...
[host-node]# brctl addif vzbr0 veth101.n
[host-node]# brctl addif vzbr0 veth102.0
...
...
[host-node]# brctl addif vzbr0 vethXXX.N
</pre>

==== Configure bridge device ====
<pre>
[host-node]# ifconfig vzbr0 up
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/forwarding
[host-node]# echo 1 > /proc/sys/net/ipv4/conf/vzbr0/proxy_arp
</pre>

==== Add routes in [[VE0]] ====
<pre>
[host-node]# ip route add 192.168.101.1 dev vzbr0
...
[host-node]# ip route add 192.168.101.n dev vzbr0
[host-node]# ip route add 192.168.102.1 dev vzbr0
...
...
[host-node]# ip route add 192.168.XXX.N dev vzbr0
</pre>

Thus you'll have more convinient configuration, i.e. all routes to VEs will be through this bridge and VEs can communicate with each other even without these routes.

=== Virtual ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

== External links ==
* [http://www.tldp.org/HOWTO/Linux+IPv6-HOWTO/hints-daemons-radvd.html Linux IPv6 HOWTO, a chapter about radvd]

[[Category: Networking]]
[[Category: HOWTO]]

Virtual Ethernet device

2006-06-07T14:03:17Z

Major:

Virtual ethernet device is ethernet device which can be used inside a [[VE]]. Unlike
venet network device, veth device has a MAC address.

Virtual ethernet device consist of two ethernet devices - one in [[VE0]] and another one
in VE. These devices are connected to each other, so if a packet goes to one
device it will come out from the other device.

== Virtual ethernet device usage ==

=== Adding veth to a VE ===
<pre>
vzctl set <VEID> --veth_add <dev_name>,<dev_addr>,<ve_dev_name>,<ve_dev_addr>
</pre>
Here
* <tt>dev_name</tt> is ethernet device name in the [[VE0|host system]]
* <tt>dev_addr</tt> is its MAC address
* <tt>ve_dev_name</tt> is an ethernet device name in the VE
* <tt>ve_dev_addr</tt> is its MAC address

MAC addresses must be entered in XX:XX:XX:XX:XX:XX format. Note that this option
is incremental, so devices are added to already existing ones.

=== Removing veth from a VE ===
<pre>
vzctl set <VEID> --veth_del <dev_name>
</pre>
Here <tt>dev_name</tt> is the ethernet device name in the [[VE0|host system]].

== Common configurations with virtual ethernet devices ==

=== Virtual ethernet device can be used with IPv6 ===
You'll need to setup IPv6 address on ethernet device inside a VE, add default route inside a VE
and add route to this address via host-side veth in host system.
Do not forget to enable forwarding and proxy_arp on host-side veth device.

=== Virtual ethernet devices can be joined in one bridge ===
Thus you'll have more convinient configuration, i.e. all routes to VEs will be
through this bridge and VEs can communicate with each other even without these routes.

=== Virtual ethernet devices + VLAN ===
This configuration can be done by adding vlan device to the previous configuration.

[[Category: Networking]]
[[Category: HOWTO]]