OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Migration from Linux-VServer to OpenVZ

2008-04-28T10:46:13Z

Toutoune25: wikif

{{Roughstub}}

Current document describes the migration from Linux-VServer based virtualization solution to OpenVZ.

Description of challenge:

The challenge is migration from Linux-Vserver to OpenVZ by booting the OpenVZ kernel and updating the existing configs of
utility level in purpose to make the existing guest OSes work over OpenVZ kernel.

Details of migration process. Step by step:

1. Initial conditions: the following example of Linux-VServer based solution was used for the experiment:

* Kernel linux-2.6.17.13 was patched by the patch-2.6.17.13-vs2.0.2.1.diff and rebuild;
* Util-vserver-0.30.211 tools were used for creating containers;

<code>
# vserver-info
Versions:
Kernel: 2.6.17.13-vs2.0.2.1
VS-API: 0x00020002
util-vserver: 0.30.211; Dec 5 2006, 17:10:21

Features:
CC: gcc, gcc (GCC) 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)
CXX: g++, g++ (GCC) 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)
CPPFLAGS: ''
CFLAGS: '-g -O2 -std=c99 -Wall -pedantic -W -funit-at-a-time'
CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W -fmessage-length=0 -funit-at-a-time'
build/host: i686-pc-linux-gnu/i686-pc-linux-gnu
Use dietlibc: yes
Build C++ programs: yes
Build C99 programs: yes
Available APIs: v13,net
ext2fs Source: kernel
syscall(2) invocation: alternative
vserver(2) syscall#: 273/glibc

Paths:
prefix: /usr/local
sysconf-Directory: ${prefix}/etc
cfg-Directory: ${prefix}/etc/vservers
initrd-Directory: $(sysconfdir)/init.d
pkgstate-Directory: ${prefix}/var/run/vservers
vserver-Rootdir: /vservers
#
</code>

VServer v345 was built using vserver vX build utility and populated by using the tarballed template of Fedora Core 4.

<code>
# vserver v345 start
Starting system logger: [ OK ]
Initializing random number generator: [ OK ]
Starting crond: l: [ OK ]
Starting atd: [ OK ]
# vserver v345 enter
[/]# ls -l
total 44
drwxr-xr-x 2 root root 4096 Oct 26 2004 bin
drwxr-xr-x 3 root root 4096 Dec 8 17:16 dev
drwxr-xr-x 27 root root 4096 Dec 8 15:21 etc
-rw-r--r-- 1 root root 0 Dec 8 15:33 halt
drwxr-xr-x 2 root root 4096 Jan 24 2003 home
drwxr-xr-x 7 root root 4096 Oct 26 2004 lib
drwxr-xr-x 2 root root 4096 Jan 24 2003 mnt
drwxr-xr-x 3 root root 4096 Oct 26 2004 opt
-rw-r--r-- 1 root root 0 Dec 7 20:17 poweroff
dr-xr-xr-x 80 root root 0 Dec 8 11:38 proc
drwxr-x--- 2 root root 4096 Dec 7 20:17 root
drwxr-xr-x 2 root root 4096 Oct 26 2004 sbin
drwxrwxrwt 2 root root 40 Dec 8 17:16 tmp
drwxr-xr-x 15 root root 4096 Jul 27 2004 usr
drwxr-xr-x 17 root root 4096 Oct 26 2004 var
[/]# sh
sh-2.05b#
.........
</code>

As a result we obtain running virtual environment v345:

<code>
# vserver-stat

CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME
0 51 90.9M 26.3M 0m58s75 2m42s57 33m45s93 root server
49153 4 10.2M 2.8M 0m00s00 0m00s11 21m45s42 v345

#
</code>

2. Starting migration to OpenVZ: downloading and installing the stable OpenVZ kernel.

Install the OpenVZ kernel, as described in [[quick installation]].

After the kernel is installed, reboot the machine. After rebooting and logging in you will see the following reply on vserver-stat call:

<code>
# vserver-stat
can not change context: migrate kernel feature missing and 'compat' API disabled: Function not implemented
#
</code>

It is a natural thing that now virtual environment v345 is unavailable. The following steps will be devoted to making it
work over OpenVZ kernel.

3. Downloading and installing vzctl package

OpenVZ solution requires installing a set of tools: vzctl and vzquota packages. Download and install it, as described in [[quick installation]].

If rpm complains about unresolved dependencies, you'll have to satisfy them first, then repeat the installation.
Then launch the OpenVZ:

<code>
# /sbin/service vz start
Starting OpenVZ: [ OK ]
Bringing up interface venet0: [ OK ]
Configuring interface venet0: [ OK ]
#
</code>

Currently vzlist utility is unable to find any containers:

<code>
# vzlist
Containers not found
#
</code>

4. Updating different configurations in purpose to make existing templates work

Move the existing templates of guest OSs to the right place:

<code>
# cd /vz
# mkdir private
# mkdir 345
# mv /vservers/v345 /vz/private/345
</code>

Now it is time for creating configuration files for OpenVZ container. Use the basic sample
configuration presented in /etc/sysconfig/vz-scripts/ve-vps.basic.conf-sample file:

<code>
# cd /etc/sysconfig/vz-scripts
# cp ve-vps.basic.conf-sample 345.conf
</code>

Update the ON_BOOT string in 345.conf file by typing:

<code>
.....
ONBOOT="yes"
.....
</code>
to make it boot on node restart, and add a couple of strings related to the
particular container 345:
<code>
.....
VE_ROOT="/vz/root/345"
VE_PRIVATE="/vz/private/345"
ORIGIN_SAMPLE="vps.basic"
HOSTNAME="test345.my.org"
IP_ADDRESS="192.168.0.145"
.....
</code>

And reboot the machine:

<code>
# reboot
</code>

5. Testing how the guest OSs successfully work over OpenVZ. Reference to Users Guide of OpenVZ (vzctl).

After rebooting you will be able to see running container 345 that have been
migrated from vserver:

<code>
# vzlist -a
CTID NPROC STATUS IP_ADDR HOSTNAME
345 5 running 192.168.0.145 test345.my.org
#
</code>

And run commands on it:

<code>
# vzctl exec 345 ls -l
total 48
drwxr-xr-x 2 root root 4096 Oct 26 2004 bin
drwxr-xr-x 3 root root 4096 Dec 11 12:42 dev
drwxr-xr-x 27 root root 4096 Dec 11 12:44 etc
-rw-r--r-- 1 root root 0 Dec 11 12:13 fastboot
-rw-r--r-- 1 root root 0 Dec 8 15:33 halt
drwxr-xr-x 2 root root 4096 Jan 24 2003 home
drwxr-xr-x 7 root root 4096 Oct 26 2004 lib
drwxr-xr-x 2 root root 4096 Jan 24 2003 mnt
drwxr-xr-x 3 root root 4096 Oct 26 2004 opt
-rw-r--r-- 1 root root 0 Dec 7 20:17 poweroff
dr-xr-xr-x 70 root root 0 Dec 11 12:42 proc
drwxr-x--- 2 root root 4096 Dec 7 20:17 root
drwxr-xr-x 2 root root 4096 Dec 11 12:13 sbin
drwxrwxrwt 2 root root 4096 Dec 8 12:40 tmp
drwxr-xr-x 15 root root 4096 Jul 27 2004 usr
drwxr-xr-x 17 root root 4096 Oct 26 2004 var
#
</code>

Potential issues:
This work has in progress status. Some issues may take place with tuning the network for containers. To be continued.

[[Category:HOWTO]]

Features

2008-04-28T08:32:34Z

Toutoune25: typo

The architecture of OpenVZ is different from the traditional virtual machines architecture because it always runs the same OS kernel as the host system (while still allowing multiple Linux distributions in individual [[container]]s). This single-kernel implementation technology enables running [[container]]s with a near-zero overhead. Thus, OpenVZ offer an order of magnitude higher efficiency and manageability than traditional virtualization technologies.

== OS Virtualization ==
From the point of view of applications and [[container]] users, each container is an independent system. This independence is provided by a virtualization layer in the kernel of the host OS. Note that only a negligible part of the CPU resources is spent on virtualization (around 1-2%). The main features of the virtualization layer implemented in OpenVZ are the following:

* A [[container]] looks and behaves like a regular Linux system. It has standard startup scripts; software from vendors can run inside a container without OpenVZ-specific modifications or adjustment;
* A user can change any configuration file and install additional software;
* [[Containers]] are completely isolated from each other (file system, processes, Inter Process Communication (IPC), sysctl variables);
* Processes belonging to a container are scheduled for execution on all available CPUs. Consequently, [[CT]]s are not bound to only one CPU and can use all available CPU power.

== Network virtualization ==

The OpenVZ network virtualization layer is designed to isolate [[CT]]s from each other and from the physical network:

* Each [[CT]] has its own IP address; multiple IP addresses per CT are allowed;
* Network traffic of a CT is isolated from the other CTs. In other words, containers are protected from each other in the way that makes traffic snooping impossible;
* Firewalling may be used inside a CT (the user can create rules limiting access to some services using the canonical iptables tool inside a CT). In other words, it is possible to set up firewall rules from inside a CT;
* Routing table manipulations and advanced routing features are supported for individual containers. For example, setting different maximum transmission units (MTUs) for different destinations, specifying different source addresses for different destinations, and so on.

== Resource Management ==

OpenVZ [[resource management]] controls the amount of resources available for containers. The controlled resources include such parameters as CPU power, disk space, a set of memory-related parameters, etc. Resource management allows OpenVZ to:

* Effectively share available [[host system]] resources among CTs
* Guarantee Quality-of-Service (QoS)
* Provide performance and resource isolation and protect from denial-of-service attacks
* Collect usage information for system health monitoring

Resource management is much more important for OpenVZ than for a standalone computer since computer resource utilization in a OpenVZ-based system is considerably higher than that in a typical system.
As all the CTs are using the same kernel, resource management is of paramount importance. Really, each CT should stay within its boundaries and not affect other CTs in any way — and this is what resource management does.

OpenVZ resource management consists of four main components: two-level disk quota, fair CPU scheduler, disk I/O scheduler, and user beancounters. Please note that all those resources can be changed during CT runtime, there is no need to reboot. Say, if you want to give your CT less memory, you just change the appropriate parameters on the fly. This is either very hard to do or not possible at all with other virtualization approaches such as VM or hypervisor.

=== Two-Level Disk Quota ===
[[Host system]] administrator ([[HW]] root) can set up a per-container [[disk quota]]s, in terms of disk blocks and inodes (roughly number of files). This is the first level of disk quota. In addition to that, a container administrator ([[CT]] root) can employ usual quota tools inside own CT to set standard UNIX per-user and per-group [[disk quota]]s.

If one want to give a CT more disk space, you just increase its disk quota. No need to resize disk partitions etc.

=== Fair CPU scheduler ===
CPU scheduler in OpenVZ is a two-level implementation of [[fair-share scheduling]] strategy.

On the first level scheduler decides which CT is give the CPU time slice to, based on per-CT cpuunits values. On the second level the standard Linux scheduler decides which process to run in that container, using standard Linux process priorities and such.

OpenVZ administrator can set up different values of <code>cpuunits</code> for different containers, and the CPU time will be given to those proportionally.

Also there is a way to limit CPU time, e.g. say that this container is limited to, say, 10% of CPU time available.

=== I/O scheduler ===
Similar to the Fair CPU scheduler described above, I/O scheduler in OpenVZ is also two-level, utilizing Jens Axboe's CFQ I/O scheduler on its second level.

Each container is assigned an I/O priority, and the I/O scheduler distributes the available I/O bandwidth according to the priorities assigned. Thus no single container can saturate an I/O channel.

=== User Beancounters ===

[[User beancounters]] is a set of per-CT counters, limits, and guarantees. There is a set of about 20 parameters which are carefully chosen to cover all the aspects of CT operation, so no single container can abuse any resource which is limited for the whole node and thus do harm to another CTs.

Resources accounted and controlled are mainly memory and various in-kernel objects such as IPC shared memory segments, network buffers etc. etc. Each resource can be seen from <code>/proc/user_beancounters</code> and has five values assiciated with it: current usage, maximum usage (for the lifetime of a container), barrier, limit, and fail counter. The meaning of barrier and limit is parameter-dependant; in short, those can be thought of as a soft limit and a hard limit. If any resource hits the limit, fail counter for it is increased, so CT administrator can see if something bad is happening by analyzing the output of <code>/proc/user_beancounters</code> in her container.

== Checkpointing and live migration ==

{{Main|Checkpointing and live migration}}

A live migration and checkpointing feature was released for OpenVZ in the middle of April 2006. It allows to migrate a container from one physical server to another without a need to shutdown/restart a container. The process is known as checkpointing: a CT is frozen and its whole state is saved to the file on disk. This file can then be transferred to another machine and a CT can be unfrozen (restored) there. The delay is about a few seconds, and it is not a downtime, just a delay.

Since every piece of the container state, including opened network connections, is saved, from the user's perspective it looks like a delay in response: say, one database transaction takes a longer time than usual, when it continues as normal and user doesn't notice that his database is already running on the another machine.

That feature makes possible scenarios such as upgrading your server without any need to reboot it: if your database needs more memory or CPU resources, you just buy a newer better server and live migrate your container to it, then increase its limits. If you want to add more RAM to your server, you migrate all containers to another one, shut it down, add memory, start it again and migrate all containers back.

[[Category: Concepts]]
[[Category: Technology]]