OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Checkpointing and live migration

2006-10-06T10:17:19Z

Christoph: /* Manual Checkpoint and Restore Functions */

CPT is an extension to OpenVZ kernel which allows to save full state of a running VE and to restore it later on the same or on a different host in a way transparent for running applications and network connections. This technique has several applications, the most important being live (zero-downtime) migration of VEs and taking an instant snapshot of a running VE for later resume, i. e. CheckPoinTing.

Before CPT, it was only possible to migrate a VE through shutdown and subsequent reboot. The procedure not only introduces quite a long downtime of network services, it is not transparent for clients using the VE, making impossible migration, when clients runs some tasks which are not tolerant to shutdowns.

Comparing to this old scheme, CPT allows to migrate a VE in a way, essentially invisible both for users of this VE and for external clients, using network services located inside VE. It still introduces a short delay in service, required for actual checkpoint/restore of the processes, but this delay is indistinguishable from a short interruption of network connectivity.

== Online migration ==

There is special utility vzmigrate in OpenVZ distribution intended to support VE migration. With it help one can perform live (a.k.a. online) migration, i. e. for migration VE “freezes” for some time, and after migration it continues to work as though nothing has happened. Online migration can be performed by
<pre>vzmigrate --online <host> VEID</pre>
command. During online migration all VE private data saved to an image file, which is transferred to target host.

In order for vzmigrate to work without asking for a password, ssh public keys from the source host should be placed to destination host's <code>authorized_keys</code>. In other words, command <code>ssh root@host</code> should not ask you for a password. See [[ssh keys]] for more info.

== Manual Checkpoint and Restore Functions ==

<code>vzmigrate</code> is not strictly required to perform online migration. <code>vzctl</code> utility, accompanied with some file system backup tools, provides enough of power to do all the tasks.

VE can be checkpointed with command:
<pre>vzctl chkpnt VEID --dumpfile <path></pre>
This command saves all the state of running VE to dump file and stops the VE. If the option <code>--dumpfile</code> is not set, <code>vzctl</code> uses default path <code>/vz/dump/Dump.VEID</code>.

After this it is possible to restore the VE exactly in the same state executing:
<pre>vzctl restore VEID --dumpfile <path></pre>
If dump file and file system is transferred to another HW node, the same command can restore VE there with the same success.

It is critical requirement that file system at the moment of restore must be identical to the file system at the moment of checkpointing. If this requirement is not held, depending on severity of changes process of restoration can be aborted or the processes inside VE can see this as an external corruption of open files. When VE is restored on the same node where it was checkpointed, it is enough not to touch file system accessible by the VE. When VE is transferred to another node it is necessary to synchronize VE file system before restore. <code>vzctl</code> does not provide this functionality and external tools (f.e. <code>rsync</code>) are required.

== Step-by-step Checkpoint and Restore ==

Process of checkpointing can be performed by stages. It consists of three steps.

First step – suspending VE. At this stage CPT moves all the processes to special beforehand known state and stops VE network interfaces. This stage can be done by
<pre>vzctl chkpnt VEID --suspend</pre>
command. Second step – dumping VE. At this phase CPT saves state of processes and global state of VE to image file. All the process private data need to be saved: address space, register set, opened files/pipes/sockets, System V IPC structures, current working directory, signal handlers, timers, terminal settings, user identities (uid, gid, etc), process identities (pid, pgrp, sid, etc), rlimit and other data. This stage can be done by
<pre>vzctl chkpnt VEID --dump --dumpfile <path></pre>
command. Third step – killing or resuming processes. If the migration succeeds VE can be stopped with the command:
<pre>vzctl chkpnt VEID --kill</pre>
If migration failed by some reason or if the goal was taking a snapshot of VE state for later restore, CPT can resume VE with:
<pre>vzctl chkpnt VEID --resume</pre>

Process of restoring consists of two steps. The first step is to restore processes and to leave them in a special frozen state. After this step processes are ready to continue execution, however, in some cases CPT has to do some operations after process is woken up, therefore CPT sets process return point to function in our module. This stage can be done by
<pre>vzctl restore VEID --undump --dumpfile <path></pre>
command. Second step – waking up processes or killing them if restore process failed. After CPT wakes up process, it performs necessary operations in our function and continues execution. This stages can be done by
<pre>vzctl restore VEID --resume
vzctl restore VEID --kill</pre>
commands.

HA cluster with DRBD and Heartbeat

2006-10-04T15:17:39Z

Christoph:

This article shows how to setup a OpenVZ high availability (HA) cluster using the data replication software DRBD and the cluster manager Heartbeat. In this example the two machines builing the cluster run on CentOS 4.3. The article also shows how to do kernel updates in the cluster, including necessary steps like recompiling of new DRBD userspace tools. For this purpose, kernel 2.6.8-022stab078.10 (containing DRBD module 0.7.17) is used as initial kernel version, and kernel 2.6.8-022stab078.14 (containing DRBD module 0.7.20) as updated kernel version.

Additional information about clustering of virtual machines can be found in the following paper: http://www.linuxtag.org/2006/fileadmin/linuxtag/dvd/12080-paper.pdf

Some other additional information can be found in the documentation of the Thomas-Krenn.AG cluster (The author of this howto is working in the cluster development there, that is the reason why he was able to write this howto :-). The full documentation with interesting illustrations is currently only available in German:
http://my.thomas-krenn.com/service_support/index.php/page.242

== Prerequisites ==
The OpenVZ kernel already includes the DRBD module. The DRBD userspace tools and the cluster manager Heartbeat must be provided seperately. As the API version of the DRBD userspace tools must exactly match the API version of the module, compile them yourself. Also compile Heartbeat yourself, as at the time of this writing the CentOS extras repository only contained an old CVS version of Heartbeat.

On a hardware node for production use there should not be any application that is not really needed for running OpenVZ (any things which are not needed by OpenVZ should run in a VE for security reasons). As a result, compile DRBD and Heartbeat on another machine running CentOS 4.3 (in this example I used a virtual machine on a VMware Server).

=== Compiling Heartbeat ===
Heartbeat version 1.2.* has successfully been used in a lot of two-node-clusters around the world. As the codebase used in version 1.2.* is in production use for many years now, the code is very stable. At the time of writing, Heartbeat version 1.2.4 is the current version of the 1.2.* branch.

Get the tar.gz of the current version of the 1.2.* branch from http://linux-ha.org/download/index.html, at the time of this writing this is http://linux-ha.org/download/heartbeat-1.2.4.tar.gz. Use rpmbuild to build the package:
<pre>rpmbuild -ta heartbeat-1.2.4.tar.gz</pre>
After that, you find four rpm packes in /usr/src/redhat/RPMS/i386 (heartbeat-1.2.4-1.i386.rpm, heartbeat-ldirectord-1.2.4-1.i386.rpm, heartbeat-pils-1.2.4-1.i386.rpm, heartbeat-stonith-1.2.4-1.i386.rpm). In this example only heartbeat-1.2.4-1.i386.rpm, heartbeat-pils-1.2.4-1.i386.rpm, and heartbeat-stonith-1.2.4-1.i386.rpm are needed.

=== Compiling DRBD userspace tools ===
When compiling the DRBD userspace tools, you have to take care to take the version that matches the DRBD version that is included in the OpenVZ kernel you want to use. If you are unsure about the version, do the following steps while running the OpenVZ kernel that you want to use on a test machine (I used another virtual machine on a VMware server to try this):
<pre>
[root@testmachine ~]# cat /proc/version
Linux version 2.6.8-022stab078.10 (root@rhel4-32) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 Wed Jun 21 12:01:20 MSD 2006
[root@testmachine ~]# modprobe drbd
[root@testmachine ~]# cat /proc/drbd
version: 0.7.17 (api:77/proto:74)
SVN Revision: 2093 build by phil@mescal, 2006-03-06 15:04:12
0: cs:Unconfigured
1: cs:Unconfigured
[root@testmachine ~]# rmmod drbd
[root@testmachine ~]#
</pre>
Here the version of the DRBD module is 0.7.17. So the userspace tools for 0.7.17 are neccessary.

Back on the buildmachine, do the following to create the rpm:
<pre>
[root@buildmachine ~]# yum install kernel-devel gcc bison flex
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
Parsing package install arguments
Nothing to do
[root@buildmachine ~]# tar xfz drbd-0.7.17.tar.gz
[root@buildmachine ~]# cd drbd-0.7.17
[root@buildmachine drbd-0.7.17]# make rpm
[...]
You have now:
-rw-r--r-- 1 root root 288728 Jul 30 10:40 dist/RPMS/i386/drbd-0.7.17-1.i386.rpm
-rw-r--r-- 1 root root 518369 Jul 30 10:40 dist/RPMS/i386/drbd-km-2.6.9_34.0.2.EL-0.7.17-1.i386.rpm
[root@buildmachine drbd-0.7.17]#
</pre>
Note that in this way the kernel-devel from CentOS is used, but this does not matter as the created drbd-km rpm will not be used (the DRBD kernel module is already included in OpenVZ kernel). If the kernel-devel package is not the same version as the kernel package that is currently running, it is possible to execute 'make rpm KDIR=/usr/src/kernels/2.6.9-34.0.2.EL-i686/' to directly point to the kernel sources.

== Installing the two nodes ==
Install the two machines in the same way as you would install them for a normal OpenVZ installation, but do not create a filesystem for the /vz. This filesystem will be installed later on on top of DRBD.
{| class="wikitable"
|+'''Example installation configuration'''
! Parameter !! node1 !! node2
|-
! hostname
| ovz-node1
| ovz-node2
|-
! / filesystem
| hda1, 10 GB
| hda1, 10 GB
|-
! swap space
| hda2, 2048 MB
| hda2, 2048 MB
|-
! public LAN
| eth0, 192.168.1.201
| eth0, 192.168.1.202
|-
! private LAN
| eth1, 192.168.255.1 (Gbit Ethernet)
| eth1, 192.168.255.2 (Gbit Ethernet)
|-
! other install options
| no firewall, no SELinux
| no firewall, no SELinux
|-
! package groups
| deactivated everything, only kept vim-enhanced
| deactivated everything, only kept vim-enhanced
|}

== Installing OpenVZ ==

Get the OpenVZ kernel and utilities and install them on both nodes, as described in [[quick installation]]. Update grub configuration to use the OpenVZ kernel by default. Disable starting of OpenVZ on system boot on both nodes (OpenVZ will be started and stopped by Heartbeat):
<pre>
[root@ovz-node1 ~]# chkconfig --del vz
[root@ovz-node1 ~]#
</pre>
Then reboot both machines.

== Setting up DRBD ==

On each of the two nodes create a partition that acts as underlying DRBD device. The partitions should have exactly the same size (I created a 10 GB partition hda3 using fdisk on each node for this example). Note that it might be necessary to reboot the machines to re-read the partition table.

Install the rpm of the DRBD userspace tools on both nodes:
<pre>
[root@ovz-node1 ~]# rpm -ihv drbd-0.7.17-1.i386.rpm
Preparing... ########################################### [100%]
1:drbd ########################################### [100%]
[root@ovz-node1 ~]#
</pre>
Then create the drbd.conf configuration file and copy it to /etc/drbd.conf on both nodes. Below is the example configuration file that is used in this article:
<pre>
resource r0 {
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";

startup {
degr-wfc-timeout 120;
}

net {
on-disconnect reconnect;
}

disk {
on-io-error detach;
}

syncer {
rate 30M;
group 1;
al-extents 257;
}

on ovz-node1 {
device /dev/drbd0;
disk /dev/hda3;
address 192.168.255.1:7788;
meta-disk internal;
}

on ovz-node2 {
device /dev/drbd0;
disk /dev/hda3;
address 192.168.255.2:7788;
meta-disk internal;
}

}
</pre>
Start DRBD on both nodes:
<pre>
[root@ovz-node1 ~]# /etc/init.d/drbd start
Starting DRBD resources: [ d0 s0 n0 ].
[root@ovz-node1 ~]#
</pre>
Then check the status of /proc/drbd:
<pre>
[root@ovz-node1 ~]# cat /proc/drbd
version: 0.7.17 (api:77/proto:74)
SVN Revision: 2093 build by phil@mescal, 2006-03-06 15:04:12
0: cs:Connected st:Secondary/Secondary ld:Inconsistent
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
[root@ovz-node1 ~]#
</pre>
Both nodes are now Secondary and Inconsistent. The latter is because the underlying storage is not yet in-sync, and DRBD has no way to know whether you want the initial sync from ovz-node1 to ovz-node2, or ovz-node2 to ovz-node1. As there is no data below it yet, it does not matter.

To start the sync from ovz-node1 to ovz-node2, do the following on ovz-node1:
<pre>
[root@ovz-node1 ~]# drbdadm -- --do-what-I-say primary all
[root@ovz-node1 ~]# cat /proc/drbd
version: 0.7.17 (api:77/proto:74)
SVN Revision: 2093 build by phil@mescal, 2006-03-06 15:04:12
0: cs:SyncSource st:Primary/Secondary ld:Consistent
ns:627252 nr:0 dw:0 dr:629812 al:0 bm:38 lo:640 pe:0 ua:640 ap:0
[=>..................] sync'ed: 6.6% (8805/9418)M
finish: 0:04:51 speed: 30,888 (27,268) K/sec
[root@ovz-node1 ~]#
</pre>
As you see, DRBD syncs with about 30 MB per second, as we told it so in /etc/drbd.conf. On the SyncSource (ovz-node1 in this case) the DRBD device is already useable (although it is syncing in the background).

So you can immediately create the filesystem:
<pre>
[root@ovz-node1 ~]# mkfs.ext3 /dev/drbd0
[...]
[root@ovz-node1 ~]#
</pre>

=== Copy necessary OpenVZ files to DRBD device ===

Move the original /vz directory to /vz.orig and recreate the /vz directory to have it as a mount point '''(do this on both nodes)''':
<pre>
[root@ovz-node1 ~]# mv /vz /vz.orig
[root@ovz-node1 ~]# mkdir /vz
[root@ovz-node1 ~]#
</pre>

Afterwards move the necessary OpenVZ directories (/etc/vz, /etc/sysconfig/vz-scripts, /var/vzquota) and replace them with symbolic links '''(do this on both nodes)''':
<pre>
[root@ovz-node1 ~]# mv /etc/vz /etc/vz.orig
[root@ovz-node1 ~]# mv /etc/sysconfig/vz-scripts /etc/sysconfig/vz-scripts.orig
[root@ovz-node1 ~]# mv /var/vzquota /var/vzquota.orig
[root@ovz-node1 ~]# ln -s /vz/cluster/etc/vz /etc/vz
[root@ovz-node1 ~]# ln -s /vz/cluster/etc/sysconfig/vz-scripts /etc/sysconfig/vz-scripts
[root@ovz-node1 ~]# ln -s /vz/cluster/var/vzquota /var/vzquota
[root@ovz-node1 ~]#
</pre>

Currently, ovz-node1 is still Primary of /dev/drbd0. You can now mount it and copy the necessary files to it '''(only on ovz-node1!)''':
<pre>
[root@ovz-node1 ~]# mount /dev/drbd0 /vz
[root@ovz-node1 ~]# cp -a /vz.orig/* /vz/
[root@ovz-node1 ~]# mkdir -p /vz/cluster/etc
[root@ovz-node1 ~]# mkdir -p /vz/cluster/etc/sysconfig
[root@ovz-node1 ~]# mkdir -p /vz/cluster/var
[root@ovz-node1 ~]# cp -a /etc/vz /vz/cluster/etc/
[root@ovz-node1 ~]# cp -a /etc/sysconfig/vz-scripts /vz/cluster/etc/sysconfig/
[root@ovz-node1 ~]# cp -a /var/vzquota /vz/cluster/var/
[root@ovz-node1 ~]# umount /dev/drbd0
[root@ovz-node1 ~]#
</pre>

== Setting up Heartbeat ==

Install the neccessary Heartbeat rpms on both nodes:
<pre>
[root@ovz-node1 ~]# rpm -ihv heartbeat-1.2.4-1.i386.rpm heartbeat-pils-1.2.4-1.i386.rpm heartbeat-stonith-1.2.4-1.i386.rpm
Preparing... ########################################### [100%]
1:heartbeat-pils ########################################### [ 33%]
2:heartbeat-stonith ########################################### [ 67%]
3:heartbeat ########################################### [100%]
[root@ovz-node1 ~]#
</pre>
Create the Heartbeat configuration file ha.cf and copy it to /etc/ha.d/ha.cf on both nodes. Details about this file can be found at http://www.linux-ha.org/ha.cf. Below is an example configuration which uses the two network connections and also a serial connection for heartbeat packets:
<pre>
# Heartbeat logging configuration
logfacility daemon

# Heartbeat cluster members
node ovz-node1
node ovz-node2

# Heartbeat communication timing
keepalive 1
warntime 10
deadtime 30
initdead 120

# Heartbeat communication paths
udpport 694
ucast eth1 192.168.255.1
ucast eth1 192.168.255.2
ucast eth0 192.168.1.201
ucast eth0 192.168.1.202
baud 19200
serial /dev/ttyS0

# Don't fail back automatically
auto_failback off

# Monitoring of network connection to default gateway
ping 192.168.1.1
respawn hacluster /usr/lib64/heartbeat/ipfail
</pre>
Create the Heartbeat configuration file authkeys and copy it to /etc/ha.d/authkeys on both nodes. Set the permissions of this file to 600. Details about this file can be found at http://www.linux-ha.org/authkeys. Below is an example:
<pre>
auth 1
1 sha1 PutYourSuperSecretKeyHere
</pre>
Create the Heartbeat configuration file haresources and copy it to /etc/ha.d/haresources on both nodes. Details about this file can be found at http://www.linux-ha.org/haresources. Below is an example:
<pre>
ovz-node1 drbddisk::r0 Filesystem::/dev/drbd0::/vz::ext3 vz MailTo::youremail@yourdomain.tld
</pre>
Finally, you can now start heartbeat on both nodes:
<pre>
[root@ovz-node1 ~]# /etc/init.d/heartbeat start
Starting High-Availability services:
[ OK ]
[root@ovz-node1 ~]#
</pre>

== How to do OpenVZ kernel updates when it contains a new DRBD version ==

As mentioned above, it is important to use the correct version of the DRBD userspace tools. When an OpenVZ kernel contains a new DRBD version, it is important that the DRBD API version of the userspace tools matches the API version of the DRBD module that is included in the OpenVZ kernel. The API versions can be found at http://svn.drbd.org/drbd/branches/drbd-0.7/ChangeLog. The best way is to always use the version of the DRBD userspace tools that matches the version of the DRBD module that is included in the OpenVZ kernel.

In this example the initial cluster installation contained OpenVZ kernel 2.6.8-022stab078.10, which contains the DRBD module 0.7.17. The steps below show the update procedure to OpenVZ kernel 2.6.8-022stab078.14, which contains the DRBD module 0.7.20.
In the first step build the DRBD userspace tools version 0.7.20 on your buildmachine. Then stop Heartbeat and DRBD on the passive node (hint: you can use 'cat /proc/drbd' to get a hint which node is active and which one is passive):
<pre>
[root@ovz-node2 ~]# cat /proc/drbd
version: 0.7.17 (api:77/proto:74)
SVN Revision: 2093 build by phil@mescal, 2006-03-06 15:04:12
0: cs:Connected st:Secondary/Primary ld:Consistent
ns:60 nr:136 dw:196 dr:97 al:3 bm:3 lo:0 pe:0 ua:0 ap:0
[root@ovz-node2 ~]# /etc/init.d/heartbeat stop
Stopping High-Availability services:
[ OK ]
[root@ovz-node2 ~]# cat /proc/drbd
version: 0.7.17 (api:77/proto:74)
SVN Revision: 2093 build by phil@mescal, 2006-03-06 15:04:12
0: cs:Connected st:Secondary/Primary ld:Consistent
ns:60 nr:136 dw:196 dr:97 al:3 bm:3 lo:0 pe:0 ua:0 ap:0
[root@ovz-node2 ~]# /etc/init.d/drbd stop
Stopping all DRBD resources.
[root@ovz-node2 ~]# cat /proc/drbd
cat: /proc/drbd: No such file or directory
[root@ovz-node2 ~]#
</pre>
Then install the new kernel and the DRBD userspace tools on this node:
<pre>
[root@ovz-node2 ~]# rpm -ihv ovzkernel-2.6.8-022stab078.14.i686.rpm
warning: ovzkernel-2.6.8-022stab078.14.i686.rpm: V3 DSA signature: NOKEY, key ID a7a1d4b6
Preparing... ########################################### [100%]
1:ovzkernel ########################################### [100%]
[root@ovz-node2 ~]# rpm -Uhv drbd-0.7.20-1.i386.rpm
Preparing... ########################################### [100%]
1:drbd ########################################### [100%]
/sbin/service
Stopping all DRBD resources.
[root@ovz-node2 ~]#
</pre>
Now set the new kernel as default kernel in /etc/grub.conf and then reboot this node.

After the reboot, the new DRBD version is visible:
<pre>
[root@ovz-node2 ~]# cat /proc/drbd
version: 0.7.20 (api:79/proto:74)
SVN Revision: 2260 build by phil@mescal, 2006-07-04 15:18:57
0: cs:Connected st:Secondary/Primary ld:Consistent
ns:0 nr:28 dw:28 dr:0 al:0 bm:2 lo:0 pe:0 ua:0 ap:0
[root@ovz-node2 ~]#
</pre>

To update the other node, switch-over the services to make the current active node the passive node. Execute the following on the still active node (it could be that the hb_standby command is located in /usr/lib/heartbeat):
<pre>
[root@ovz-node1 ~]# /usr/lib64/heartbeat/hb_standby
2006/08/03_21:09:41 Going standby [all].
[root@ovz-node1 ~]#
</pre>

Now do the same steps on the new passive node to update it: stop Heartbeat and DRBD, install the new kernel and the new DRBD userspace tools, set the new kernel as default kernel in /etc/grub.conf and reboot the node.

== How to do updates of vzctl, vzctl-lib, and vzquota ==

Ensure after every update of OpenVZ tools that OpenVZ is not started on system boot. To disable starting of OpenVZ on system boot execute on both nodes:
<pre>
[root@ovz-node1 ~]# chkconfig --del vz
[root@ovz-node1 ~]#
</pre>

[[Category: HOWTO]]

Quick installation (legacy)

2006-09-21T09:56:29Z

Christoph: /* Conntracks */

This document briefly describes the steps needed to install OpenVZ on your machine.

This document is also available in the following languages: [http://forum.openvz.org/index.php?t=tree&goto=35&#msg_35 French], [http://forum.openvz.org/index.php?t=tree&goto=1805&#msg_1805 German].

OpenVZ consists of a kernel, user-level tools, and VE templates. This guide tells how to install the kernel and the tools.

== Requirements ==
This guide assumes you are running recent release of Fedora Core (like FC5) or RHEL/CentOS 4. Currently, OpenVZ kernel tries to support the same hardware that Red Hat kernels support. For full hardware compatibility list, see [http://www.virtuozzo.com/en/products/virtuozzo/hcl/ Virtuozzo HCL].

=== rpm or yum? ===

In case you have yum utility available on you system, you may want to use it effectively to install and update OpenVZ packages. In case you don't have yum, or don't want to use it, you can use plain old rpm. Instructions for both rpm and yum are provided below.

=== yum pre-setup ===
If you want to use yum, you should set up OpenVZ yum repository first.

Download [http://download.openvz.org/openvz.repo openvz.repo] file and put it to your <code>/etc/yum.repos.d/</code> repository. This can be achieved by the following commands, as root:
<pre>
# cd /etc/yum.repos.d
# wget http://download.openvz.org/openvz.repo
</pre>

In case you can not cd to /etc/yum.repos.d, it means either yum is not installed on your system, or yum version is too old. In that case, just stick to rpm installation method.

== Kernel installation ==

{{Note|In case you want to recompile the kernel yourself rather than use the one provided by OpenVZ, see [[kernel build]].}}

First, you need to choose what “flavor” of the kernel you want to install. Please refer to [[Kernel flavors]] for more information.

=== Using yum ===
Run the following command
<pre>
# yum install ovzkernel[-flavor]
</pre>

Here <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

=== Using rpm ===
Get the kernel binary RPM from the [http://openvz.org/download/kernel/ Download » Kernel] page, or directly from [http://download.openvz.org/kernel/ download.openvz.org/kernel], or from one of its [[Download mirrors|mirrors]]. You need only one kernel RPM so please [[Kernel flavors|choose the appropriate one]] depending on your hardware.

Next, install the kernel RPM you chose:

<pre>
# rpm -ihv ovzkernel[-flavor]*.rpm
</pre>

Here <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

{{Note|<tt>rpm -U</tt> (where <tt>-U</tt> stands for ''upgrade'') should '''not''' be used, otherwise all currently installed kernels will be uninstalled.}}

== Configuring the bootloader ==

In case GRUB is used as the boot loader, it will be configured automatically: lines similar to these will be added to the <tt>/boot/grub/grub.conf</tt> file:

<pre>
title Fedora Core (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5 quiet rhgb vga=0x31B
initrd /initrd-2.6.8-022stab029.1.img
</pre>
Change <tt>Fedora Core</tt> to <tt>OpenVZ</tt> (just for clarity reasons, so the OpenVZ kernels will not be mixed up with non OpenVZ ones). Remove extra arguments from the kernel line, leaving only the <tt>root=...</tt> parameter. The modifed portion of <tt>/etc/grub.conf</tt> should look like this:

<pre>
title OpenVZ (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5
initrd /initrd-2.6.8-022stab029.1.img
</pre>

== Configuring ==

Please make sure the following steps are performed before rebooting into OpenVZ kernel.

=== sysctl ===

There is a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in <tt>/etc/sysctl.conf</tt> file. Here is the relevant part of the file; please edit it accordingly.

<pre>
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv4.conf.default.proxy_arp = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enables the magic-sysrq key
kernel.sysrq = 1
# TCP Explict Congestion Notification
#net.ipv4.tcp_ecn = 0
# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
</pre>

=== SELinux ===

SELinux should be disabled. To that effect, put the following line to <code>/etc/sysconfig/selinux</code>:
<pre>
SELINUX=disabled
</pre>

=== Conntracks ===

In the stable OpenVZ kernels (those that are 2.6.8-based) netfilter connection tracking for [[VE0]] is disabled by default. If you have a stateful firewall enabled on the host node (it is there by default) you should either disable it, or enable connection tracking for [[VE0]].

To enable conntracks for VE0, add the following line to <code>/etc/modprobe.conf</code> file:
<pre>
options ip_conntrack ip_conntrack_enable_ve0=1
</pre>

{{Note|in kernels later than 2.6.8, connection tracking is enabled by default}}

== Rebooting into OpenVZ kernel ==

Now reboot the machine and choose "OpenVZ" on the boot loader menu. If the OpenVZ kernel has been booted successfully, proceed to installing the user-level tools for OpenVZ.

== Installing the utilities ==

OpenVZ needs some user-level tools installed. Those are:

; vzctl
: A utility to control OpenVZ VPSs (create, destroy, start, stop, set parameters etc.)
; vzquota
: A utility to manage quotas for VPSs. Mostly used indirectly (by vzctl).

=== Using yum ===

<pre>
# yum install vzctl vzquota
</pre>

=== Using rpm ===

Download the binary RPMs of these utilities from [http://openvz.org/download/utils/ Download » Utils], or directly from [http://download.openvz.org/utils/ download.openvz.org/utils], or from one of its [[Download mirrors|mirrors]]. Install them:

<pre>
# rpm -Uhv vzctl*.rpm vzquota*.rpm
</pre>

If rpm complains about unresolved dependencies, you'll have to satisfy them first, then repeat the installation.

When all the tools are installed, start the OpenVZ subsystem.

== Starting OpenVZ ==

As root, execute the following command:

<pre>
# /sbin/service vz start
</pre>

This will load all the needed OpenVZ kernel modules. This script should also start all the VPSs marked to be auto-started on machine boot (there aren't any yet).

During the next reboot, this script should be executed automatically.

== Next steps ==

OpenVZ is now set up on your machine. To load OpenVZ kernel by default, edit the default line in the /boot/grub/grub.conf file to point to the OpenVZ kernel. For example, if the OpenVZ kernel is the first kernel mentioned in the file, put it as default 0. See man grub.conf for more details.

The next step is to prepare the [[OS template]]: please continue to [[OS template cache preparation]] document.

[[Category: Installation]]
[[Category: HOWTO]]