OpenVZ Virtuozzo Containers Wiki - User contributions [en]

User talk:Kir

2013-01-29T10:33:35Z

Grin: /* G+ */ new section

Hi Kir - thanks for your updates to the [[DHCP]] page. Do you use #openvz on freenode? [[User:Mrjcleaver|Mrjcleaver]] 10:53, 1 June 2008 (EDT)
: Yes, I am usually in the channel during work hours (my timezone is GMT+3, Moscow) --[[User:Kir|Kir]] 21:33, 1 June 2008 (EDT)

Hi Kir, please add the following warning to the list of precreated templates page. Cheers, Chris.

<blockquote>
WARNING: DO NOT USE the CentOS 6 template: it has this bug:

https://www.virtualmin.com/node/21124
</blockquote>

== G+ ==

Your G+ forwarder doesn't. ;-) --[[User:Grin|Grin]] ([[User talk:Grin|talk]]) 05:33, 29 January 2013 (EST)

UBC interdependencies table

2013-01-29T10:19:23Z

Grin: /* See also */ code here too

{{UBC toc}}

{| class="wikitable"
! Interdependency !! Importance
|-
| <math>kmemsize_{bar} \ge 40KB \cdot avnumproc + dcachesize_{lim}</math> || {{mandatory}}
|-
| <math>privvmpages_{bar} \ge vmguarpages_{bar}</math> || {{recommended}}
|-
| <math>tcpsndbuf_{lim} - tcpsndbuf_{bar} \ge 2.5KB \cdot numtcpsock</math> || {{mandatory}}
|-
| <math> othersockbuf_{lim} - othersockbuf_{bar} \ge 2.5KB \cdot numothersock</math> || {{mandatory}}
|-
| <math>tcprcvbuf_{lim} - tcprcvbuf_{bar} \ge 2.5KB \cdot numtcpsock</math> || {{recommended}}
|-
| <math>tcprcvbuf_{bar} \ge 64KB</math> || {{recommended}}
|-
| <math>tcpsndbuf_{bar} \ge 64KB</math> || {{recommended}}
|-
| <math>dgramrcvbuf_{bar} \ge 129KB</math> || {{optional}}
|-
| <math>othersockbuf_{bar} \ge 129KB</math> || {{optional}}
|-
| <math>numfile \ge avnumproc \cdot 32</math> || {{recommended}}
|-
| <math>dcachesize_{bar} \ge numfile \cdot 384\ \rm(bytes)</math> || {{recommended}}
|-
| <math>barrier \le limit</math> || {{mandatory}}
|}

== See also ==
* [[UBC consistency check]]
* [[user:grin/openvz checker.pl]]

[[Category:Troubleshooting]]

UBC consistency check

2013-01-29T10:15:34Z

Grin: code

{{UBC toc}}

System resource control parameters have certain interdependencies. Constraints on the parameter settings are listed below. Indexes <code>bar</code> and <code>lim</code> in the formulae below mean the barrier and the limit of the parameters, respectively.

Configuration of resource control parameters for a container
is invalid if these constraints are not satisfied. The best way to ensure the
validity of the configuration is to use the {{man|vzcfgvalidate|8}} utility.

All the interdependencies discussed below and their importance are summarized in [[UBC interdependencies table]].

The configured limits can be checked through
* <code>/proc/user_beancounters</code> interface;
* container configuration files in <code>/etc/vz/conf/</code> directory.

== kmemsize should be enough for the expected number of processes ==
<math>kmemsize_{bar} \ge 40KB \cdot avnumproc + dcachesize_{lim}</math>

(<code>avnumproc</code> here stands for the expected average number of processes).

This constraint is important for applications to work reliably in the
container.
If it is not satisfied, applications will start to fail in the middle of
operations instead of failing while spawning more processes
and the application's ability to handle resource shortage will be very
limited.

== Memory allocation limits should not be less than the guarantee ==
<math> privvmpages_{bar} \ge vmguarpages_{bar}</math>

If this constraint is not satisfied, <code>vmguarpages</code> will not work.

== Send buffers should have enough space for all sockets ==
<math>tcpsndbuf_{lim} - tcpsndbuf_{bar} \ge 2.5KB \cdot numtcpsock</math>

<math>othersockbuf_{lim} - othersockbuf_{bar} \ge 2.5KB \cdot numothersock</math>

These constraints are also important.
If they are not satisfied, transmission of data over the sockets
may hang in some circumstances.

== Other TCP socket buffers should be big enough ==
<math>tcprcvbuf_{lim} - tcprcvbuf_{bar} \ge 2.5KB \cdot numtcpsock</math>

<math>tcprcvbuf_{bar} \ge 64KB</math>

<math>tcpsndbuf_{bar} \ge 64KB</math>

Selecting the left side equal to the right side in the inequalities
above ensures minimal performance of network communications.
Increasing the left side will increase performance to a certain extent.

== UDP socket buffers should be large enough if the system is not tight on memory ==
<math>dgramrcvbuf_{bar} \ge 129KB</math>

<math>othersockbuf_{bar} \ge 129KB</math>

These constraints are desired, but not essential.
Large enough buffers for UDP sockets improve reliability of datagram delivery.
However, note that if the UDP traffic is so bursty that it needs larger
buffers, the datagrams will likely be lost not because of resource control
limits, but because of other memory and performance limitations.

== Number of files limit should be adequate for the expected number of processes ==
<math>numfile \ge avnumproc \cdot 32</math>

Note that each process after a <code>execve(2)</code> system call
requires a file for each loaded shared library.
A too low <code>numfile</code> limit will increase the chances of failures
during <code>execve(2)</code> calls with error messages that are not clear
to the users.

== The limit on the total size of <code>dentry</code> and <code>inode</code> structures locked in memory should be adequate for allowed number of files ==

<math>dcachesize_{bar} \ge numfile \cdot 384\ \rm(bytes)</math>

A too low <code>dcachesize</code> limit will increase the chances of
file operation refusals resulting in unexpected application failures.

== Barrier should be less or equal than limit ==
In addition to the conditions listed above,

<math>barrier \le limit</math>

should be maintained for each parameter.

== Code ==
See [[User:Grin/openvz checker.pl]] for a contributed code to check the consistency.

[[Category:Troubleshooting]]

UBC

2013-01-29T10:13:14Z

Grin: /* See also */ program to check the settings

'''UBC''', or '''User Beancounters''' — a set of limits and guarantees controlled per [[container]]. UBC is the major component of OpenVZ [[resource management]].

{| class="wikitable"
{{UBC toc/body}}
|}

== See also ==
* http://blog.maxgarrick.com/understanding-openvz-resource-limits/
* [[User:Grin/openvz checker.pl]]

[[Category: UBC]]
[[Category: Definitions]]

User:Grin/openvz checker.pl

2013-01-29T10:12:29Z

Grin: sharing is caring

<pre>
#!/usr/bin/perl
#$Id: openvz_checker.pl,v 1c29ac915688 2009/12/09 21:17:13 grin $
#(c)Peter "grin" Gervai, 2009; grin(a)grin(dot)hu
# GPLv2 / CC-BY-SA 3.0
#
# check openvz resource settings
# based on wiki pages
#
# Usage: just run it on the VE0 (host) system as root.

use strict;
use warnings;

use IO::File;
use Data::Dumper;

my $dir_bc = "/proc/bc";

my $CUR=0;
my $MAX=1;
my $BAR=2;
my $LIM=3;

# ia32...
my $PAGESIZE=4096;

my @allsocketbuf = qw( tcprcvbuf tcpsndbuf dgramrcvbuf othersockbuf );

opendir( DIR, $dir_bc );
my @veids = grep { /^\d+/ && -d "$dir_bc/$_" } readdir(DIR);
closedir( DIR );

my %res;
for my $veid ( @veids ) {
next if $veid == 0;
my $f = new IO::File "< $dir_bc/$veid/resources" or next;
while( <$f> ) {
chomp;
# trim spaces
s/^\s*(.*?)\s*$/$1/;
my @data = split /\s+/;
# held, maxheld, barrier, limit, failcnt
$res{$veid}{$data[0]} = [ @data[1..5] ];
}
}

## total ram
my $ram = qx( free | grep Mem: ); chomp $ram;
$ram = (split /\s+/, $ram)[1];
print "RAM=$ram\n";
$ram *= 1024;

## swap
my $swap = qx( free | grep Swap: ); chomp $swap;
$swap = (split /\s+/, $swap)[1];
print "SWAP=$swap\n";
$swap *= 1024;

my $sum;
### low mem x86_32
# max lowram 832MB
my $lowram = $ram > 832000000 ? 832000000 : $ram;

###############################
####### system wide config
###############################

###############################
## utilisation
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'kmemsize'}->[$CUR];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$CUR];
}
$sum += $b;
}

print "Low memory (x86_32) Utilisation: ", &pretty($sum / (0.4 * $lowram));
print " ( <1 OK; <2 unsafe; >2 dangerous)\n";

###############################
## commitment level
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'kmemsize'}->[$LIM];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$LIM];
}
$sum += $b;
}

print "Low memory (x86_32) Commitment: ", &pretty($sum / (0.4 * $lowram));
print " ( <1 OK; <1.5 acceptable; >2 not recommended)\n";

############################
#### total ram
## utilisation
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'physpages'}->[$CUR] * $PAGESIZE;
$b += $res{$veid}{'kmemsize'}->[$CUR];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$CUR];
}
$sum += $b;
}
print "Total RAM utilisation: ", &pretty($sum / $ram);
print " ( <0.8 low; <1 ok; >1 impossible)\n";

############################
#### total ram+swap
## utilisation
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'oomguarpages'}->[$CUR] * $PAGESIZE;
$b += $res{$veid}{'kmemsize'}->[$CUR];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$CUR];
}
$sum += $b;
}
print "RAM+swap utilisation: ", &pretty($sum / ($ram+$swap));
my $low_bound = &pretty($ram / ($ram+$swap));
my $hi_bound = &pretty(($ram + 0.5*$swap) / ($ram+$swap));
print " ( <$low_bound low; between ok; >$hi_bound bad)\n";

## commitment
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'oomguarpages'}->[$BAR] * $PAGESIZE;
$b += $res{$veid}{'kmemsize'}->[$LIM];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$LIM];
}
$sum += $b;
}
print "RAM+swap commitment: ", &pretty($sum / ($ram+$swap));
print " ( <0.8 low; <1 ok; >1 not recommended)\n";

## allocations
$sum=0;
foreach my $veid (keys %res) {
my $b = $res{$veid}{'privvmpages'}->[$LIM] * $PAGESIZE;
$b += $res{$veid}{'kmemsize'}->[$LIM];
foreach my $sb (@allsocketbuf) {
$b += $res{$veid}{$sb}->[$LIM];
}
$sum += $b;
}
print "RAM+swap allocations: ", &pretty($sum / ($ram+$swap));
print " ( <1.5 low; <4 ok; >4 not recommended [oom])\n";

###############################
####### per VE configs
###############################
foreach my $veid (sort { $a <=> $b } keys %res) {
# <numproc> should be average process num
&compare( $veid, $res{$veid}{'kmemsize'}->[$BAR],
40*1024*$res{$veid}{'numproc'}->[$CUR] + $res{$veid}{'dcachesize'}->[$LIM],
"kmemsize(bar)", "numproc~dcachesize", "kmemsize low for # of processes" );

&compare( $veid, $res{$veid}{'privvmpages'}->[$BAR], $res{$veid}{'vmguarpages'}->[$BAR],
"privvmpages","vmguarpages", "mem alloc less than guarantee" );

&compare( $veid, $res{$veid}{'tcpsndbuf'}->[$LIM] - $res{$veid}{'tcpsndbuf'}->[$BAR],
2.5*1024*$res{$veid}{'numtcpsock'}->[$LIM],
"tcpsndbuf(lim-bar)", "numtcpsock(lim{bytes})", "tcp may hang");

&compare( $veid, $res{$veid}{'othersockbuf'}->[$LIM] - $res{$veid}{'othersockbuf'}->[$BAR],
2.5*1024*$res{$veid}{'numothersock'}->[$LIM],
"othersockbuf(lim-bar)", "numothersock{bytes}", "othersock may hang");

&compare( $veid, $res{$veid}{'tcprcvbuf'}->[$LIM] - $res{$veid}{'tcprcvbuf'}->[$BAR],
2.5*1024*$res{$veid}{'numothersock'}->[$LIM],
"tcprcvbuf(lim-bar)","numothersock{bytes}","may slow tcp" );

if( $res{$veid}{'tcprcvbuf'}->[$BAR] < 65535 ) {
print "$veid: tcprcvbuf too low (<64k)\n";
}

if( $res{$veid}{'tcpsndbuf'}->[$BAR] < 65535 ) {
print "$veid: tcpsndbuf too low (<64k)\n";
}

if( $res{$veid}{'dgramrcvbuf'}->[$BAR] < 131200 ) {
print "$veid: dgramrcvbuf too low (<129k)\n";
}

if( $res{$veid}{'othersockbuf'}->[$BAR] < 131200 ) {
print "$veid: othersockbuf too low (<129k)\n";
}

# numfile < avgnumproc*32
# ...

if( $res{$veid}{'dcachesize'}->[$BAR] <
$res{$veid}{'numfile'}->[$MAX] * 384 ) {
print "$veid: dcachesize likely low for dentry and inode allocs\n";
}

foreach my $param (keys %{$res{$veid}}) {
&compare( $veid, $res{$veid}{$param}->[$LIM], $res{$veid}{$param}->[$BAR],
"limit", "barrier", "problem with $param" );
}
}

sub compare {
my ($veid, $large, $small, $ltext, $stext, $text) = @_;

if( $large < $small ) {
# bad...
print "VE $veid: $text ($ltext=$large < $stext=$small)\n";
}
return;
}

sub pretty($) {
return sprintf("%6.4f", shift);
}
</pre>

User:Grin

2013-01-29T09:40:46Z

Grin: /* resource management */ source code ;)

== About me ==
See me [http://hu.wikipedia.org/wiki/user:grin there] on Wikipedia (or in the [http://hu.wikipedia.org/wiki/user:grin english] one).

== About my openvz ==
I use OpenVZ to completely separate @#$%^&*!ing ''PHP'' users from the rest of the sane uncracked world. I run ''Debian/GNU Linux'' and try to send my changes upstream. ;-)

=== Why do I use OpenVZ ===
* I do not require full virtualisation
* I do not require different operating systems, kernels or actually environment for VEs
* I need a lightweight solution
* I need easy remote management
* I need open source, libre software
* I need statistics (''small lie, big lie, statistics'', but anyway :))
* I need an upstream who is arrogant enough that their changes ''can'' find their way into the mainline kernel ;-)

=== My opinion about OpenVZ ===
So far I believe it delivers what it promises. Maybe the secret is not to promise much. ;=)

== My opinion about this wiki ==
Extremely unstructured, extremely hard to use, badly needs some organisation, which I plan to try someday. Something along a FAQ what's thematically organised and linking everywhere from the questions.

----
== Wiki structure planning ==
=== Better entry points ===
* [[One more entry point]]

=== resource management ===
* [[/openvz_checker.pl]]

==== quotas ====
* [[OpenVZ disk quota, df and stat weird behaviour]]
* [[Checking disk quota]]
* [[Resource shortage]]
* [[FAQ#What_filesystems_should_I_choose_for_saving_my_VEs.3F]] (no support for xfs, fsck it)
* [[Resource management]]
* [[Disk quota]]

Talk:Physical to container

2009-12-09T20:22:16Z

Grin: /* /proc in fstab */ fine

== RSync Command ==
Please check the following command, Ive used with success:
rsync -arvpz --numeric-ids --exclude dev --exclude proc --exclude tmp --exclude mnt --exclude sys -e "ssh -l root@a.b.c.d" root@a.b.c.d:/ /vz/private/123/

== Migration Script ==

I composed a little Script to migrate a Debian Sarge Box to OpenVZ. (Some System specific steps have been removed)

<pre>#!/bin/sh
echo "Stopping VE 300..."
vzctl stop 300
echo "Creating base filesystem /dev /proc ..."
mknod /vz/private/300/dev/ptmx c 5 2
mkdir /vz/private/300/dev/pts
rm -f /vz/private/300/dev/null
mknod /vz/private/300/dev/null c 1 3
chmod o+rw /vz/private/300/dev/null
echo "Copy the tty's to VE and Set Permissions"
cp -r /dev/ttyp* /dev/ptyp* /vz/private/300/dev/
chmod o+wx /vz/private/300/dev/*typ*
echo "Creating /dev/random and Set Permissions"
mknod -m 644 /vz/private/300/dev/random c 1 8
mknod -m 644 /vz/private/300/dev/urandom c 1 9
chown root:root /vz/private/300/dev/random /vz/private/300/dev/urandom
echo "clearing mtab / fstab..."
echo -n > /vz/private/300/etc/fstab
rm /vz/private/300/etc/mtab
ln -s /proc/mounts /vz/private/300/etc/mtab
</pre>

== Use Dummy ==

Having been completely clueless about Openvz three days ago I first created a dummy container based on the same distribution I wanted to migrate. This helped me greatly to understand what was needed and also allowed to use the dummy as a template for /dev and the conf file. Disregarding the time to setup the host system the actual migration from the physical machine with tar ball took me two hours and the migrated container started up on second attempt after first complaining about disk size. I increased that and it runs. I am very impressed by the ease of it all.

== /proc in fstab ==

Often it is ''wise'' to keep proc mount in fstab, if the OS doesn't make it by itself. --[[User:Grin|Grin]] 11:49, 9 December 2009 (UTC)

: Thanks for noting this! I have modified the page accordingly, please check --[[User:Kir|Kir]] 11:56, 9 December 2009 (UTC)

:: Seems fine, thanks, Kir. --[[User:Grin|Grin]] 20:22, 9 December 2009 (UTC)

Talk:Physical to container

2009-12-09T11:49:59Z

Grin: /* /proc in fstab */ new section

Vestat

2009-03-23T15:50:08Z

Grin: let's meet reality when converting measures

[[category:resource management]]
The '''/proc/vz/vestat''' file contains statistics for VE CPU usage.

The ''guessed'' content (may the Gods - blessed thy fingers - acknowledge or reject my humble info here):

* '''VEID''': The VE ID, obviously.
* '''user, nice, system, uptime''': usage in [[jiffy|jiffies]]. These are the equivalents of <code>/proc/stat</code>, but there is no ''idle'' because it cannot be measured this way.
* '''idle, strv, uptime, used''': usage in [[cycle|cycles]]. ''idle'' and ''uptime'' are obvious, ''strv'' isn't used, and ''used'' is the used cycles by VE on all CPUs.
* '''maxlat, totlat, numsched''': latency statistics in cycles. ''maxlat'' is max latency in cycles meaning how long VE process has to wait before it actually got CPU time; ''totlat/numsched'' gives average scheduling latency. '''These do not seem work in OpenVZ, only in Virtuozzo.'''

== Measure conversions ==
Using jiffies:
*seconds = ''measurement'' / jiffies_per_seconds

Using cycles:
*seconds = ''measurement'' / frequency_of_your_cpu

Variables:
* ''jiffies_per_second'' = 1000 '''(more often ''100'', unless you have changed that in kernel config, check CONFIG_HZ= in kernel .config or /proc/config.gz)'''
* ''cycles_per_jiffy'' = ''frequency_of_your_cpu'' / ''jiffies_per_second''
* ''frequency_of_your_cpu'' (in Hz) can be read from /proc/cpuinfo, as:
** <code>cpu MHz : ''megahertz''</code>, and thus ''frequency_of_your_cpu'' = ''megahertz'' * 1048576 (for multiCPU machines this is based on the speed of one cpu only)

Vestat

2009-03-23T12:39:35Z

Grin: fix tipography a bit

[[category:resource management]]
The '''/proc/vz/vestat''' file contains statistics for VE CPU usage.

The ''guessed'' content (may the Gods - blessed thy fingers - acknowledge or reject my humble info here):

* '''VEID''': The VE ID, obviously.
* '''user, nice, system, uptime''': usage in [[jiffy|jiffies]]. These are the equivalents of <code>/proc/stat</code>, but there is no ''idle'' because it cannot be measured this way.
* '''idle, strv, uptime, used''': usage in [[cycle|cycles]]. ''idle'' and ''uptime'' are obvious, ''strv'' isn't used, and ''used'' is the used cycles by VE on all CPUs.
* '''maxlat, totlat, numsched''': latency statistics in cycles. ''maxlat'' is max latency in cycles meaning how long VE process has to wait before it actually got CPU time; ''totlat/numsched'' gives average scheduling latency. '''These do not seem work in OpenVZ, only in Virtuozzo.'''

== Measure conversions ==
Using jiffies:
*seconds = ''measurement'' / jiffies_per_seconds

Using cycles:
*seconds = ''measurement'' / cycles_per_jiffy / jiffies_per_seconds.

Variables:
* ''jiffies_per_second'' = 1000 '''(unless you have changed that in kernel config?)'''
* ''cycles_per_jiffy'' = ''frequency_of_your_cpu'' / ''jiffies_per_second''
* ''frequency_of_your_cpu'' (in Hz) can be read from /proc/cpuinfo, as:
** <code>cpu MHz : ''frequency_of_your_cpu''</code>

Disk quota

2008-09-18T15:09:12Z

Grin: dont work for xfs

Basic disk quota management:

To set disk space, run the following commands:
<pre>
vzctl set CTID --diskspace $SoftLimit$:$HardLimit$ --save
</pre>

Example:
<pre>
[host-node]# vzctl set 101 --diskspace 6G:7G --save
</pre>

You could verify the space available with this command (ie : CTID =101)
<pre>
[host-node]# vzctl exec 101 df -h
</pre>

If you want remove disk quota:
<pre>
DISK_QUOTA=no
</pre>

== Filesystem dependency ==
If your host filesystem is not ext2/3 (like ''XFS'') the disk quotas will not work inside containers.

[[Category: Disk quota]]

CPU Fair scheduler

2008-05-30T07:56:08Z

Grin: stub

The '''Fair scheduler''' distributes CPU resources among the VEs, and controls CPU resource management.

It is a two-level implementation of [[fair-share scheduling]] strategy.

On the first level scheduler decides which VE is give the CPU time slice to, based on per-VE <code>cpuunits</code> values. On the second level the standard Linux scheduler decides which process to run in that container, using standard Linux process priorities and such.

OpenVZ administrator can set up different values of <code>cpuunits</code> for different containers, and the CPU time will be given to those proportionally.

Also there is a way to limit CPU time, e.g. say that this container is limited to, say, 10% of CPU time available.

''...Stay tuned for more info...''

== Monitoring ==
The scheduler can be monitored by using the <code>/proc/vz/[[vestat]]</code> file.

Vestat

2008-05-30T07:49:44Z

Grin: category

[[category:resource management]]
The '''/proc/vz/vestat''' file contains statistics for VE CPU usage.

The ''guessed'' content (may the Gods - blessed thy fingers - acknowledge or reject my humble info here):

* '''VEID''': The VE ID, obviously.
* '''user, nice, system, uptime''': usage in [[jiffy|jiffies]]. These are the equivalents of <code>/proc/stat</code>, but there is no ''idle'' because it cannot be measured this way.
* '''idle, strv, uptime, used''': usage in [[cycle|cycles]]. idle and uptime are obvious, strv isn't used, and used is the used cycles by VE on all CPUs.
* '''maxlat, totlat, numsched''': latency statistics in ''cycles''. maxlat is max latency in cycles meaning how long VE process has to wait before it actually got CPU time; totlat/numsched gives average scheduling latency. '''These do not seem work in OpenVZ, only in Virtuozzo.'''

== Measure conversions ==
Using jiffies:
*seconds = ''measurement'' / jiffies_per_seconds

Using cycles:
*seconds = ''measurement'' / cycles_per_jiffy / jiffies_per_seconds.

Variables:
* ''jiffies_per_second'' = 1000 '''(unless you have changed that in kernel config?)'''
* ''cycles_per_jiffy'' = ''frequency_of_your_cpu'' / ''jiffies_per_second''
* ''frequency_of_your_cpu'' (in Hz) can be read from /proc/meminfo, as:
** <code>cpu MHz : ''frequency_of_your_cpu''</code>

Vestat

2008-05-30T07:45:11Z

Grin: let it be

The '''/proc/vz/vestat''' file contains statistics for VE CPU usage.

The ''guessed'' content (may the Gods - blessed thy fingers - acknowledge or reject my humble info here):

* '''VEID''': The VE ID, obviously.
* '''user, nice, system, uptime''': usage in [[jiffy|jiffies]]. These are the equivalents of <code>/proc/stat</code>, but there is no ''idle'' because it cannot be measured this way.
* '''idle, strv, uptime, used''': usage in [[cycle|cycles]]. idle and uptime are obvious, strv isn't used, and used is the used cycles by VE on all CPUs.
* '''maxlat, totlat, numsched''': latency statistics in ''cycles''. maxlat is max latency in cycles meaning how long VE process has to wait before it actually got CPU time; totlat/numsched gives average scheduling latency. '''These do not seem work in OpenVZ, only in Virtuozzo.'''

== Measure conversions ==
Using jiffies:
*seconds = ''measurement'' / jiffies_per_seconds

Using cycles:
*seconds = ''measurement'' / cycles_per_jiffy / jiffies_per_seconds.

Variables:
* ''jiffies_per_second'' = 1000 '''(unless you have changed that in kernel config?)'''
* ''cycles_per_jiffy'' = ''frequency_of_your_cpu'' / ''jiffies_per_second''
* ''frequency_of_your_cpu'' (in Hz) can be read from /proc/meminfo, as:
** <code>cpu MHz : ''frequency_of_your_cpu''</code>

UBC systemwide configuration

2008-05-30T06:48:43Z

Grin: backlink to ubc

{{UBC toc}}

The [[UBC consistency check]] article discussed validation of a [[resource management|resource control]] configuration for a single container. This article discusses checks that the configuration of the
'''whole''' system is '''valid'''.

Configurations where resources allowed for containers exceed system
capacity<ref>More precisely, configurations with excessive
overcommitment, as explained below.</ref> are not valid
and dangerous from stability point of view.
They may result in abnormal termination of the applications,
bad responsiveness of the system and, sometimes, system hangs.
Whereas the configuration validation discussed in [[UBC consistency check]]
addressed application functionality,
the validation considered in this section is aimed at security and stability
of the whole system.

The best way to make sure that the configuration of the whole system
is valid is to run periodic automatic checks, based on the formulae described
below.

== Resource utilization and commitment level ==

Several resources of the whole system (such as RAM) are discussed below
in terms of '''utilization''' and '''commitment level'''.

'''Utilization''' shows the amount of resources consumed by all
containers at the given time.
In general, low utilization values mean that the system is under-utilized.
Often, it means that the system is capable of supporting more Virtual
Environment if the existing containers continue to maintain
the same load and resource consumption level.
High utilization values (in general, more than <code>1</code>)
mean the the system is overloaded and the service level
of the containers is degraded.

'''Commitment level''' shows how much resources are “promised” to the existing
containers. Low commitment levels mean that the system
is capable of supporting more containers.
Commitment levels more than <code>1</code> mean that the
containers are promised more resources
than the system has, and in this case the system is said to be
overcommitted.
If the system runs a lot of containers,
it is usually acceptable to have some overcommitment, because it is unlikely that all Virtual
Environments will request resources at the same time.
However, higher commitment levels (as discussed below for each resource
individually) will cause containerss to experience failures to
allocate and use the resources promised to them.

== “Low memory” (x86 specific) ==
Because of specifics of architecture of Intel's x86 processors, the RAM of the
computer can't be used uniformly.
The most important memory area is so called “low memory”, a part of memory
residing at lower addresses and directly accessible by the kernel.
For current Linux kernels, the size of low memory area is 832MB
(or, if the computer has less RAM than 832MB, the size of the RAM).

Note that 64-bit kernels (i.e. x86_64, ia64 etc.) can access all memory
directly and are not using “low memory” area.
=== Utilization ===
The lower bound estimation of low memory utilization is

<math>
\frac{\displaystyle\sum_{all\ containers}
(kmemsize_{cur}+allsocketbuf_{cur})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm,
</math>

where

<math>
allsocketbuf=tcprcvbuf+tcpsndbuf+dgramrcvbuf+othersockbuf\rm.
</math>

Utilization of low memory below <math>1</math> is normal.
Utilization above <math>1</math> is not safe, and utilization above <math>2</math>
is dangerous is very likely to cause bad system responsiveness,
application stalls for seconds or more and termination of some applications.

=== Commitment level ===
The commitment level can be computed as

<math>
\frac{\displaystyle\sum_{all\ containers}
(kmemsize_{lim}+allsocketbuf_{lim})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm.
</math>

Commitment levels below <math>1</math> are normal.
Levels between <math>1</math> and <math>1.2</math> are usually acceptable for systems with about
100 containers.
Systems with more containers may have the commitment level
increased, to about <math>1.5</math>—<math>2</math> for 400 containers.
Higher commitment levels for this resource are not recommended,
because the consequences of exceeding the low memory capacities
are severe and affect the whole system and all the containers.

== Total RAM ==
This subsection discusses usage of the whole RAM and its utilization.
Usage of swap space and the sum of used RAM and swap space are discussed
below in subsection [[{{PAGENAME}}#Memory and swap space|Memory and swap space]].

Current version of OpenVZ can't guarantee availability of certain
amount of memory (in opposite to the sum of memory and swap space),
so the commitment level is not applicable to the total RAM.
Such guarantees will be implemented in future versions.

=== Utilization ===

The amount of RAM consumed by all containers can be computed as

<math>
\sum_{all\ containers}
(physpages_{cur}\cdot4096+kmemsize_{cur}+allsocketbuf_{cur})\rm.
</math>

The difference between memory usage shown by <code>free(1)</code>
or <code>/proc/meminfo</code> and the total amount of RAM consumed by
containers is the memory used by system daemons
and different caches.

The memory utilization can be computed as

<math>
\frac{\displaystyle\sum_{all\ containers}
(physpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size}\rm.
</math>

Utilization levels from <math>0.8</math> to <math>1</math> are normal.
Lower utilization means that the system is under-utilized, and,
if other system resources and their commitment levels permit,
the system can host more containers.
By the nature of the accounting of <code>physpages</code> and other parameters,
total RAM utilization can't be bigger than <math>1</math>.

== Memory and swap space ==
The main resource of the computer determining the amount of memory
applications can use is the sum of RAM and swap sizes.
If the total size of the used memory exceeds the RAM size,
Linux kernel moves some data to swap and and loads it back when the
application needs it.
More frequently used data tends to stay in RAM, less frequently used
data spends more time in swap.

Swap-in and swap-out activity reduces the system performance to some extent.
However, if this activity is not excessive, the performance decrease is not
very noticeable. On the other hand, the benefits of using swap space are
quite big, allowing to increase the number of containers in the
system about 2 times.

Swap space is essential for handling system load bursts.
System with enough swap just slows down at high load bursts,
whereas the system without swap reacts to high load bursts by refusing
memory allocations (causing applications to refuse to accept clients or
terminate) and by direct killing of some applications.
Additionally, the presence of swap space helps the system to better
balance memory and move data between “low memory” and the rest of the RAM.

In all OpenVZ installations it is strongly recommended
to have swap of size not less than the RAM size.

Also, it is not recommended to create swap space of the size
of more than 4 times RAM size because of performance degradation
related to swap-in and swap-out activity.
That is, the system should be configured so that
<math>RAM\ size \le swap\ size \le 4\cdot RAM\ size\rm.</math>
The optimal configuration is when swap size is twice more than the RAM size.

=== Utilization ===
<math>
\frac{\displaystyle\sum_{all\ containers}
(oomguarpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size + swap\ size}\rm.
</math>

The normal utilization of memory plus swap ranges between
<math>\frac{RAM\ size}{RAM\ size + swap\ size}</math>
and
<math>\frac{RAM\ size + \frac12swap\ size}{RAM\ size + swap\ size}.</math>

Lower utilization means that the system memory is under-utilized
at the moment of checking the utilization.
Higher utilization is likely to cause gradual performance degradation because
of swap-in and swap-out activity and is a sign of overloading of the system.

=== Commitment level ===
<math>
\frac{\displaystyle\sum_{all\ containers}
(oomguarpages_{bar}\cdot4096+
kmemsize_{lim}+allsocketbuf_{lim})}
{RAM\ size + swap\ size}\rm.
</math>

The normal commitment level is about <math>0.8</math>—<math>1</math>.

Commitment levels more than <math>1</math> means that the Virtual Environmens are
guaranteed more memory than the system has.
Such overcommitment is strongly not recommended, because in that case
if all the memory is consumed, random applications, including the ones
belonging to the host system, may be killed and the system may become
inaccessible by <code>ssh(1)</code> and lose other important functionality.

It is better to guarantee containers less and have less commitment
levels than to accidently overcommit the system by memory plus swap.
If the system has spare memory and swap, containers will
transparently be able to use the memory and swap above their guarantees.
Guarantees given to containers should not be big, and it is normal
if memory and swap usage for some containers stays above their
guarantee.
It is also normal to give guarantees only to containers with
preferred service.
But administrators should not guarantee container more
than the system actually has.

== Allocated memory ==

This subsection considers standard memory allocations made by
applications in the container.
The allocations for each container are controlled by
two parameters: <code>vmguarpages</code> and <code>privvmpages</code>,
discussed in sections FIXME.

Allocated memory is a more “virtual” system resource than the RAM or RAM
plus swap size.
Applications can allocate memory but start to use it only later, and
the amount of system's free memory will decrease only at the moment of the
use.
The sum of allocated memory size of all containers is an estimation
of how much physical memory will be used when (and if) all applications
claim the allocated memory.

=== Utilization ===
<math>
\frac{\displaystyle\sum_{all\ containers}
(privvmpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size + swap\ size}\rm.
</math>

This utilization level is the ratio of the amount of allocated memory
to the capacity of the system.

Low utilization level means that the system can support more
containers, if other resources permit.
High utilization levels may, but doesn't necessarily mean that
the system is overloaded.
As it was explaned above, not all applications use all the allocated memory,
so this utilization level may exceed <math>1</math>.

Computing this utilization level is useful for comparing
it with the commitment level and the level of memory allocation restrictions,
discussed below, to configure memory allocation restrictions
for the container.

=== Commitment level ===
Allocation guarantee commitment level

<math>
\frac{\displaystyle\sum_{all\ containers}
(vmguarpages_{bar}\cdot4096+
kmemsize_{lim}+allsocketbuf_{lim})}
{RAM\ size + swap\ size}
</math>

is the ratio of the memory space guaranteed to be available for
allocations to the capacity of the system.
Similarly to the commitment level of memory plus swap space (as discussed
in subsection [[{{PAGENAME}}#Memory and swap space|Memory and swap space]]),
this level should be kept below <math>1</math>.
If the level is above <math>1</math>, it significantly increases the chances of
applications to be killed instead of be notified in case the system
experiences a memory shortage.

It's better to provide lower guarantees than accidently guarantee more than
the system has, because containers are allowed to allocated memory
above their guarantee if the system is not tight on memory.
It is also normal to give guarantees only to containers with
preferred service.

== Limiting memory allocations ==
In addition to providing allocation guarantees, it is possible
to impose restrictions on the amount of memory allocated by
containers.

If a system has multiple containers,
it is important to make sure that for each container

<math>privvmpages_{lim}\cdot4096 \le 0.6\cdot {RAM\ size}\rm.</math>

If this condition is not satisfied, a single container may easily
cause an excessive swap-out and very bad performance of the whole system.
Usually, for each container <code>privvmpages</code> limitations are
to values much less than the size of the RAM.

The resource control parameters should be configured in a way,
so that in case of memory shortage applications are given chance
to notice the shortage and exit gracefully, instead of being terminated
by the kernel.
For this purpose, it is recommended to maintain reasonable
total level of memory allocation restrictions, computed as

<math>
\frac{\displaystyle\sum_{all\ containers}
(privvmpages_{lim}\cdot4096+
kmemsize_{lim}+allsocketbuf_{lim})}
{RAM\ size + swap\ size}\rm.
</math>

This number shows how much memory applications are allowed to allocate
in comparison with the capacity of the system.

In practice, a lot of applications do not use the memory very efficiently
and, sometimes, allocated memory will never be used later.
For example, Apache Web server at start time it allocates about
20–30%% more memory that it will ever use.
Some multi-threaded applications are especially bad at using their memory,
and their rate of allocated to used memory may happen to be 1000%.

The bigger the level of memory allocation restrictions,
the more chances are that applications will be killed instead
of getting an error on next memory allocation in case
the system experiences memory shortage.
The levels ranging in <math>1.5</math>–<math>4</math> can be considered acceptable.
Administrators can find experimentally the optimal setting for their load,
basing on the frequency of messages “Out of Memory: killing process”
in system logs, saved by <code>klogd(8)</code> and <code>syslogd(8)</code>.
However, for stability-critical applications, it's better to keep
the level not exceeding <math>1</math>.

Downloading and installing

2008-05-27T12:30:02Z

Grin: /* Downloading the kernel */ internal DL link

This page organises the pages related to downloading ''kernel'' and ''user-space components'', and pages about ''installation'' on different systems.

== Downloading the kernel ==
* [http://download.openvz.org/ The download directory]
* [[Download]] / [[Download/kernel|kernel]] - the download instructions page and the kernel download page, pointing to the latest versions
* [[Kernel build#Rebuilding kernel from sources]] - generic instructions on getting and compiling the kernel
* [[Kernel configuration]] - instructions on configuring the kernel source
* [[Kernel download structure]] - description of the whole OpenVZ download directory structure and [[Kernel download structure|versioning system]]

=== Debian specific ===
* [[Compiling the OpenVZ kernel (the Debian way)]] - using pre-packaged kernel source and openvz patch

=== RPM specific ===
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Kernel build#Rebuilding kernel from SRPM]] - getting and compiling the kernel from SRPMs

=== Bare Metal ===
* [[Bare Metal Installer]] - installing OpenVZ on an "empty" system (with no operating system) booting a DVD

== Installation and setup ==
* [[Installation on Debian]], [[Debian template creation]]
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Ubuntu template]]
* [[Known problems]] - with kernels and their solutions

Downloading and installing

2008-05-27T10:20:41Z

Grin: /* Downloading the kernel */ extlink of the Dl dir

This page organises the pages related to downloading ''kernel'' and ''user-space components'', and pages about ''installation'' on different systems.

== Downloading the kernel ==
* [http://download.openvz.org/ The download directory]
* [[Kernel build#Rebuilding kernel from sources]] - generic instructions on getting and compiling the kernel
* [[Kernel configuration]] - instructions on configuring the kernel source
* [[Kernel download structure]] - description of the whole OpenVZ download directory structure and [[Kernel download structure|versioning system]]

=== Debian specific ===
* [[Compiling the OpenVZ kernel (the Debian way)]] - using pre-packaged kernel source and openvz patch

=== RPM specific ===
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Kernel build#Rebuilding kernel from SRPM]] - getting and compiling the kernel from SRPMs

=== Bare Metal ===
* [[Bare Metal Installer]] - installing OpenVZ on an "empty" system (with no operating system) booting a DVD

== Installation and setup ==
* [[Installation on Debian]], [[Debian template creation]]
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Ubuntu template]]
* [[Known problems]] - with kernels and their solutions

One more entry point

2008-05-27T10:18:34Z

Grin: added DL and install page link

This entry point of OpenVZ wiki is created in order to arrange all the information on this site more logically. General idea of arrangement is the following. Here you can find something like table of contents. Each chapter contents collection of pages, that pass to the subject of the chapter. Of course there can be articles on the same subject in one chapter: one writes article in own way, another person does it in the other way. It is important, that pages in chapters go in the order of complexity of the material.
It allows the novices at OpenVZ to read only first pages of chapters to start some feature working. Advanced users can quickly go through this table of contents and find more specific information they actually need.

It is greatly appreciated if you will create points in this table (even without writing the article), in order for the community to know what topics are popular and need to be described. Writing articles is also very useful in fact.

== Table of Contents ==
{| width="100%"
| valign="top" width="50%" |
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Introduction to Virtualization Technology
|}
* [[Introduction to virtualization]]
* [[Virtualization implementation comparison]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Basic concepts of OpenVZ
|}
* [[VE]] = [[Virtual Environment]]
* [[HN]] = [[HW]] = [[Hardware Node]] = [[Host system]]
* [[CT0]]
* [[OS template]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Choosing and Installing OpenVZ
|}
* [[Quick installation]] -- [[Downloading and installing]]
* [[OpenVZ components]]
* [[Kernel versioning]]
* [[Different kernel flavors (UP, SMP, ENTERPRISE, ENTNOSPLIT)]]
* [[Features]]
* [[Security]]
* [[HA cluster with DRBD and Heartbeat]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Templates
|}
* [[OS template]]
* [[OS template cache]]
* [[OS template metadata]]
* [[Adding OS template to the installation]]
* [[OS template cache preparation]]
* [[Debian template creation]]
* [[Gentoo template creation]]
* [[Slackware template creation]]
* [[Ubuntu template]]
* [[Creating your own template]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | First steps inside OpenVZ environment
|}
* [[VE creation]]
* [[Starting VE]]
* [[Entering VE]]
* [[Running a process in VE]]
* [[Stoping VE]]
* [[Destroying VE]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Resource accounting
|}
* [[User beancounters]]
* [[OpenVZ quota]]
* [[CPU Fairscheduler]]
* [[Resource shortage]]
|}
| valign="top" | 
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Network
|}
* [[Setting up network in VE]]
* [[Virtual network device]]
* [[Virtual Ethernet device]]
* [[iptables in OpenVZ environment]]
* [[Differences between venet and veth]]
* [[Using NAT for VE with private IPs]]
* [[Source based routing]]
* [[VPN via the TUN/TAP device]]
* [[Traffic accounting with iptables]]
* [[Traffic shaping with tc]]
* [[Using bridges in OpenVZ]]
* [[NFS server inside VE]]
* [[X inside VE]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Checkpointing and Migration
|}
* [[What is checkpointing]]
* [[Live migration]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Working with OpenVZ code
|}
* [[Porting the kernel]]
* [[Kernel build]]
* [[Containers/Pidcache]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Troubleshooting
|}
* [[Magic SysRq Key]]
* [[Remote console setup]]
* [[SysRq debugger]]
* [[When you have an oops]]
* [[Modifying initrd image]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Using specific tools in VE
|}
* [[Asterisk in VE with debian stable]]
* [[Plesk in VE]]
* [[Yum]]
* [[Shared webhosting]]
* [[CPanel in VE]]
* [[Zimbra on OpenVZ on CentOS]]
|}
{| style="border:1px solid #ccc;background-color:#f7fff7;margin-top:+.7em" cellpadding="10" width=100%
|
{| style="background-color:#e5ffe5;border:1px solid #ccc;" width=100%
| align=center | Other
|}
* [[Processes scope and visibility]]
* [[Setting up a mirror]]
* [[MAX ULONG]]
* [[Memory page]]
* [[Artwork]]
* [[FAQ]]
|}
|}

Downloading and installing

2008-05-27T10:16:35Z

Grin: another one categorising the mess

This page organises the pages related to downloading ''kernel'' and ''user-space components'', and pages about ''installation'' on different systems.

== Downloading the kernel ==
* [[Kernel build#Rebuilding kernel from sources]] - generic instructions on getting and compiling the kernel
* [[Kernel configuration]] - instructions on configuring the kernel source
* [[Kernel download structure]] - description of the whole OpenVZ download directory structure and [[Kernel download structure|versioning system]]

=== Debian specific ===
* [[Compiling the OpenVZ kernel (the Debian way)]] - using pre-packaged kernel source and openvz patch

=== RPM specific ===
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Kernel build#Rebuilding kernel from SRPM]] - getting and compiling the kernel from SRPMs

=== Bare Metal ===
* [[Bare Metal Installer]] - installing OpenVZ on an "empty" system (with no operating system) booting a DVD

== Installation and setup ==
* [[Installation on Debian]], [[Debian template creation]]
* [[Quick installation]] - getting kernel and userspace tools on RPM based systems
* [[Ubuntu template]]
* [[Known problems]] - with kernels and their solutions

User:Grin

2008-05-06T13:12:04Z

Grin: /* Wiki structure planning */ better entry points already existing

== About me ==
See me [http://hu.wikipedia.org/wiki/user:grin there] on Wikipedia (or in the [http://hu.wikipedia.org/wiki/user:grin english] one).

== About my openvz ==
I use OpenVZ to completely separate @#$%^&*!ing ''PHP'' users from the rest of the sane uncracked world. I run ''Debian/GNU Linux'' and try to send my changes upstream. ;-)

=== Why do I use OpenVZ ===
* I do not require full virtualisation
* I do not require different operating systems, kernels or actually environment for VEs
* I need a lightweight solution
* I need easy remote management
* I need open source, libre software
* I need statistics (''small lie, big lie, statistics'', but anyway :))
* I need an upstream who is arrogant enough that their changes ''can'' find their way into the mainline kernel ;-)

=== My opinion about OpenVZ ===
So far I believe it delivers what it promises. Maybe the secret is not to promise much. ;=)

== My opinion about this wiki ==
Extremely unstructured, extremely hard to use, badly needs some organisation, which I plan to try someday. Something along a FAQ what's thematically organised and linking everywhere from the questions.

----
== Wiki structure planning ==
=== Better entry points ===
* [[One more entry point]]

=== resource management ===
==== quotas ====
* [[OpenVZ disk quota, df and stat weird behaviour]]
* [[Checking disk quota]]
* [[Resource shortage]]
* [[FAQ#What_filesystems_should_I_choose_for_saving_my_VEs.3F]] (no support for xfs, fsck it)
* [[Resource management]]
* [[Disk quota]]

User:Grin

2008-04-29T07:57:34Z

Grin: organise the bastard

== About me ==
See me [http://hu.wikipedia.org/wiki/user:grin there] on Wikipedia (or in the [http://hu.wikipedia.org/wiki/user:grin english] one).

== About my openvz ==
I use OpenVZ to completely separate @#$%^&*!ing ''PHP'' users from the rest of the sane uncracked world. I run ''Debian/GNU Linux'' and try to send my changes upstream. ;-)

=== Why do I use OpenVZ ===
* I do not require full virtualisation
* I do not require different operating systems, kernels or actually environment for VEs
* I need a lightweight solution
* I need easy remote management
* I need open source, libre software
* I need statistics (''small lie, big lie, statistics'', but anyway :))
* I need an upstream who is arrogant enough that their changes ''can'' find their way into the mainline kernel ;-)

=== My opinion about OpenVZ ===
So far I believe it delivers what it promises. Maybe the secret is not to promise much. ;=)

== My opinion about this wiki ==
Extremely unstructured, extremely hard to use, badly needs some organisation, which I plan to try someday. Something along a FAQ what's thematically organised and linking everywhere from the questions.

----
== Wiki structure planning ==

=== resource management ===
==== quotas ====
* [[OpenVZ disk quota, df and stat weird behaviour]]
* [[Checking disk quota]]
* [[Resource shortage]]
* [[FAQ#What_filesystems_should_I_choose_for_saving_my_VEs.3F]] (no support for xfs, fsck it)
* [[Resource management]]
* [[Disk quota]]

User talk:Grin

2008-04-29T07:48:24Z

Grin: just to keep order

== Messages ==
Here you can leave a message for me, and since all feedback are welcome you're supported to do so.

It is useful, however, that you use the ''"E-mail this user"'' feature in the left menu as well to notify me that a message is waiting, otherwise I'll reply only when I log in the next time.

== Leave your message here ==

Shared webhosting

2008-03-31T20:03:18Z

Grin: change infinite width pre into italics

:''This document describes creating a "secure shared web hosting service" on « HN (Host Node) » It is « NOT » about per container shared web hosting.

{{roughstub}}

== The problem ==

One of the problems with shared web hosting (i.e. different people with each his/her own webpages) is that modern script languages such as PHP, Python, or Perl are too powerful. For example take the following PHP script:

<pre>
<?php

function get_content($filename) {
$handle = fopen($filename, 'r');
echo fread($handle, filesize($filename));
fclose($handle);
}

get_content('/home/ppuk34/www/forum/config.inc.php');

?>
</pre>

With PHP you could use open_basedir to prevent this, but there are more ways. For example [http://mgeisler.net/php-shell/ PHP Shell], a script that is [http://mgeisler.net/downloads/phpshell/SECURITY often mis-used] by people with not-so-good intentions. Or think about the [http://www.f-secure.com/v-descs/santy_a.shtml Santy-worm] which mis-used phpBB. Again there is a solution in the form of safe_mode, but lots of PHP scripts break if you enable this. For Python, Perl, or CGI-scripts there are no easy ways and you have to use wrappers or other tricks to chroot these.

== The solution ==

[[Image:Shared hosting1.png|345px|right|The OpenVZ way of shared webhosting]]

You can waste hours of time in securing all the possible things you don't want in your shared webhosting environment. And unless you are very familiar with all the things modern scripting languages can do, you probably miss dozens of alternative routes. In this process you frustrate your clients, because security always means that legitimate things break. As a side effect of your hard work, you can waste hours of extra time in educating your users. But in the end most users don't care about security, unless they are themselves victims of a compromised host. Learning the hard way is by far the most effective method. One possible solution is dedicated webhosting, but most users don't have the experience to maintain a server or it is way to expensive for them.

The main problem with shared webhosting is that by its very nature all files which are served through the web are public. Apache for example uses only one account to read all files. As said, you can use tricks with CGI wrappers to execute the scripting languages under its own credentials. However this kind of security depends on the wrappers ability to securely separate the users. We all know that if this is broken — and most often it will be broken — the result is a higher clearance on the underlying filesystem. For most systems you need more than one wrapper, so the number of possible security problems grow. The ultimate user separation is in the kernel and you can view the modifications OpenVZ has done in this light. Instead of CGI wrappers we go one step higher and give every user its own minimal server. In the rest of this article we describe how shared webhosting with OpenVZ could be implemented.

=== Minimal server ===

Create an CTx with your favorite distro. Give it an internal IP-address in one of the ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. Then strip away all unnecessary init.d scripts so only the bare minimum is started. That means as a minimum syslogd and ssh so the account holder can upload his/her files through SCP/SFTP in his/her own minimal server. For this to work you need to set up [[Using NAT for container with private IPs|destination NAT on CT0]] from high numbered ports to port 22 on the given private IP address:

<pre>
dnat="-j DNAT --to-destination"

iptables -t nat -P PREROUTING ACCEPT
iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22
iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22
...
</pre>

The other thing you want for webhosting is of course a webserver as well. To minimize the amount of needed memory, we choose [http://www.lighttpd.net/ Lighttpd] instead of the common Apache. Then configure the scripting language of your choice to run under this webserver. It is possible to use different languages/setups for different accounts as well. Also problematic CGI-scripts are not problematic anymore...

=== MySQL server ===

Most webhosting accounts use MySQL, but if you prefer another database server, go ahead. Create a new CTx with a lot more resources and again an internal IP-address. Now configure the accounts. As an extra security measure you can use the internal IP-address as well.

==== MySQL socket sharing ====

You can also share the socket of an CTx running MySQL, which is alot faster than TCP/IP.
e.g.:
<pre>
ln /var/lib/vz/private/101/var/run/mysqld/mysqld.sock /var/lib/vz/private/102/var/run/mysqld/mysqld.sock
</pre>
Inside of 102:
<pre>
$ mysql -u root -p -S /var/run/mysqld/mysqld.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9 to server version: 5.0.32-Debian_7etch1-log
</pre>

==== Refreshing links to MySQL socket ====

Sharing the MySQL socket works really well until the MySQL database is restarted or the container running MySQL is restarted. When this happens, the socket file is removed and recreated. In most cases, a different inode will be used, causing existing hard links to the mysql.sock file to no longer work. The solution is to relink these sockets.

There are more elegant solutions to this problem, but the following script is a decent hack that can be run via a cron job every minute or two. It will loop through all containers between START_CTID and STOP_CTID and make sure that the links point to the correct socket. If they do not, the link will be recreated.

<pre>
#!/bin/sh

# Created by Tauren Mills (tauren at tauren dot com) 2007-11-15

###################################
# Start of Configuration settings #
###################################

# Location of private containers:
PRIVATE=/vz/private

# Starting CT ID. CTIDs with this ID or greater will have mysql.sock link created
START_CTID=1001

# Stopping CT ID. CTIDs with this ID or less will have mysql.sock link created
STOP_CTID=2000

# Shared Mysql CT ID:
MYSQL_CTID=201

# Location of mysql socket file
MYSQL_SOCK_DIR=/var/lib/mysql

# Mysql socket file name
MYSQL_SOCK=mysql.sock

#################################
# End of Configuration settings #
#################################

# Display output if quiet is 0
QUIET=0

if [ $# -eq 1 -a "$1" = "--quiet" ]; then
QUIET=1
fi

# Full path to socket
MYSQL_SOCK_FILE=${PRIVATE}/${MYSQL_CTID}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}

[ $QUIET -eq 0 ] && echo
[ $QUIET -eq 0 ] && echo "Relinking process starting..."

# Get current location so we can set it back later
oldDirectory=`pwd`

# Check to see if MySQL container socket exists
if [ -S "${MYSQL_SOCK_FILE}" ]; then

# Get inode of MySQL container socket
mysql_inode=`ls -i ${MYSQL_SOCK_FILE} | awk '{ print $1;}'`

# Search through containers
cd $PRIVATE
for i in * ; do
# The current container to process
veid=$i

# Check if container should be processed
if [ $veid -ne $MYSQL_CTID -a $veid -ge $START_CTID -a $veid -le $STOP_CTID ]; then

# Get this container's socket
vesock=${PRIVATE}/${veid}${MYSQL_SOCK_DIR}/${MYSQL_SOCK}

# Make sure folder exists
mkdir -p ${PRIVATE}/${veid}${MYSQL_SOCK_DIR}

# Check to see if this container has a socket already
if [ -S "${vesock}" ]; then
# Get inode of this container socket
ve_inode=`ls -i ${vesock} | awk '{ print $1;}'`

# Test if sockets are the same
if [ $mysql_inode -eq $ve_inode ]; then
# No action required
[ $QUIET -eq 0 ] && echo "$veid VALID: socket ${vesock}"
else
# Remove existing file if any
if [ -a "${vesock}" ]; then
rm ${vesock}
fi

# Create hardlink to mysql socket file
ln ${MYSQL_SOCK_FILE} ${vesock}

[ $QUIET -eq 0 ] && echo "$veid FIXED: socket ${vesock}"
fi
else
# Socket didn't exist or file wasn't a socket

# Remove existing file if any
if [ -a "${vesock}" ]; then
rm ${vesock}
fi

# Create hardlink to mysql socket file
ln ${MYSQL_SOCK_FILE} ${vesock}

[ $QUIET -eq 0 ] && echo "$veid FIXED: socket ${vesock}"
fi
else
[ $QUIET -eq 0 ] && echo "$veid SKIPPED"
fi
done
else
[ $QUIET -eq 0 ] && echo "${MYSQL_SOCK_FILE} does not exist. Is MySQL running?"
fi

cd $oldDirectory

[ $QUIET -eq 0 ] && echo "Relinking process complete."
[ $QUIET -eq 0 ] && echo

</pre>

The following post on the OpenVZ forum has some other suggestions on how to deal with this issue:
[http://forum.openvz.org/index.php?t=rview&goto=23305#msg_23305 Shared webhosting - problem with mysql socket]

=== Proxy webserver ===

Because we have only one public IP-address, we need an trick to access every minimal server based on the hostname in the HTTP request. For SSH we used different ports, but that is not an option for websites. Again we create an CTx with an internal IP-address. On this server we install Lighttpd as well, because the proxying is very simple. First we must forward port 80 to this server:

<pre>
dnat="-j DNAT --to-destination"

iptables -t nat -P PREROUTING ACCEPT
iptables -t nat -A PREROUTING -p TCP -d <external IP-address> --dport 80 $dnat 192.168.13.11:80
iptables -t nat -A PREROUTING -p TCP --dport 10122 $dnat 192.168.13.101:22
iptables -t nat -A PREROUTING -p TCP --dport 10222 $dnat 192.168.13.102:22
...
</pre>

Then we create for every website an section in /etc/lighttpd/lighttpd.conf as follows:

<pre>
$HTTP["host"] == "ve101.armorica.tk" {
proxy.server = ( "" => ( ( "host" => "192.168.13.101" ) ) )
}
</pre>

You can map more names to the same IP-address if needed. The last step is to add mod_proxy to the server.modules section.

For apache add a VirtualHost directive

<pre>
<VirtualHost external-IP-address:80>
ServerName mydomainnameishere.com
RewriteEngine On
RewriteRule ^(.*)$ http://192.168.2.101$1 [P]
RewriteRule ^(.*)$ http://mydomainnameishere.com$1 [P]
</VirtualHost>

<VirtualHost external-IP-address:80>
ServerName mydomainnameishere.com
RewriteEngine On
RewriteRule ^(.*)$ http://192.168.2.101$1 [P]
RewriteRule ^(.*)$ http://www.mydomainnameishere.com$1 [P]
</VirtualHost>
</pre>

=== Other applications ===

Create for other applications as mail, make sure that the minimal servers use this one for sending mail from webpages, DNS etc. CTx as needed. The resulting server is shown in the figure above.

[[Category:HOWTO]]

User:Grin

2008-01-17T16:05:52Z

Grin: adding links for myself

see me [http://hu.wikipedia.org/wiki/user:grin there].

----
== resource management ==
=== quotas ===
* [[OpenVZ disk quota, df and stat weird behaviour]]
* [[Checking disk quota]]
* [[Resource shortage]]
* [[FAQ#What_filesystems_should_I_choose_for_saving_my_VEs.3F]] (no support for xfs, fsck it)
* [[Resource management]]
* [[OpenVZ disk quota system]]

User:Grin

2007-11-27T00:10:02Z

Grin: there i am

see me [http://hu.wikipedia.org/wiki/user:grin there].

Installation on Debian/old

2007-11-27T00:06:13Z

Grin: /* Install the toolset */ it needs the repo

= Sarge-Dapper (OldStable) =
The OpenVZ packages at http://debian.systs.org/ aimed to install OpenVZ in a easy way, some tasks are even completed during the install process!

== edit apt source settings ==
Add to your "/etc/apt/sources.list"

<pre>
deb http://debian.systs.org/debian sarge openvz
</pre>

and get the new package lists

<pre>
# apt-get update
</pre>

== precompiled kernel images at debian.systs.org (dso) ==
The kernel-images on debian.systs.org (dso) use the same kernel-config taken from OpenVZ.
(most kernel-modules are built-in!)

If there is more than one CPU available (or a CPU with hyperthreading), use the kernel-smp deb.
If there is more than 4 Gb of RAM available, use the kernel-enterprise deb.
Otherwise, use the plain kernel deb (kernel).

{| class="wikitable"
|+'''Kernel flavors list'''
! Kernel type !! Description !! Hardware !! Use case
|-
! -
| uniprocessor
| up to 4GB of RAM
|
|-
! -smp
| symmetric multiprocessor
| up to 4 GB of RAM
| 10-20 VPSs
|-
! -entnosplit
| SMP + PAE support
| up to 64 GB of RAM
| 10-30 VPSs
|-
! -enterprise
| SMP + PAE support + 4/4GB split
| up to 64 GB of RAM
| >20-30 VPSs
|}

kernel-image: i368 and amd64
<pre>
ovzkernel-2.6.9
ovzkernel-2.6.9-smp
</pre>

kernel-image: i386 only:
<pre>
ovzkernel-2.6.9-enterprise
ovzkernel-2.6.9-entnosplit
</pre>

OpenVZ tool(s) for i386 and amd64
<pre>
vzctl
vzquota
vzprocps
vzdump
</pre>

template(s) for i368 and amd64 : Debian 3.1 Minimal
<pre>
vzctl-ostmpl-debian
</pre>

== installing the kernel-images, toolset and debian-os-template ==
Example: install the stable OpenVZ kernel, tools and Debian OS Template

# aptitude install ovzkernel-2.6.9 vzctl vzquota vzdump vzctl-ostmpl-debian

If you are using GRUB, maybe you need to update the /boot/grub/menu.lst file
(can be configured at /etc/kernel-img.conf):

# /sbin/grub-update

Reboot in your new Debian Sarge OpenVZ System

# reboot

That's all :-)

Now it's time to setup your VEs with the minimal Debian-3.1 Template, create new one or download another precreated OS-Template.

= Etch (Stable) =
OpenVZ is now a part of Debian Etch repository. The packages are 'vzctl' and 'vzquota'.

== install the kernel-image ==

=== precompiled kernel images at download.openvz.org ===

A Debian OpenVZ kernel repository is online, for direct access http://download.openvz.org/kernel/debian/etch/

add to your "/etc/apt/sources.list"
<pre>
deb http://download.openvz.org/debian etch main
</pre>

Update package lists
<pre>
# apt-get update
</pre>

List downloadable OpenVZ linux-images
<pre>
# apt-cache search linux-image-2.6.18-openvz
</pre>

Install a kernel
<pre>
# apt-get install <linux-image>
</pre>

=== precompiled kernel images at debian.systs.org ===

Add to your "/etc/apt/sources.list"

<pre>
deb http://debian.systs.org/ etch openvz
</pre>

Add the signing key of debian.systs.org (dso) apt-keyring, (need root permissions)
<pre>
# wget http://debian.systs.org/dso_archiv_signing_key.asc -q -O - | apt-key add -
</pre>

and get the new package lists

<pre>
# apt-get update
</pre>

Choose a linux image (version 028stab048.1) :
<pre>
ovzkernel-2.6.18 (i386 and amd64)
ovzkernel-2.6.18-smp (i386 and amd64)
ovzkernel-2.6.18-enterprise only (i386)
</pre>

# apt-get install <linux-image>

=== Build your own kernel-image (debian way) ===

==== Installing sources ====

To install the kernel-source and the OpenVZ kernel patch, run:
# apt-get install kernel-package linux-source-2.6.18 kernel-patch-openvz libncurses5-dev

==== Unpacking ====

Unpack the kernel source:
<pre>
# cd /usr/src
# tar xjf linux-source-2.6.18.tar.bz2
# cd linux-source-2.6.18
</pre>

==== Kernel config ====

You need a kernel config.
You can use the config of the debian-kernel:
# cp /boot/config-2.6.18-5-686 .config

'''Or''' get a 2.6.18 kernel configuration from http://download.openvz.org/kernel/branches/2.6.18/current/configs/ (depending on your architecture; the below example is for i686):
# wget http://download.openvz.org/kernel/branches/2.6.18/current/configs/kernel-2.6.18-i686.config.ovz -O .config

==== Patching and configuring ====

Now you can apply the openvz kernel patch and modify your kernel-config:
# ../kernel-patches/all/apply/openvz
# make menuconfig

You need the following OpenVZ kernel config settings:
<pre>
(taken from OpenVZ Kernel 2.6.18-028test010.1 on 686)

Filesystem
\_ [*] Second extended fs support (CONFIG_EXT2_FS)
\_ [*] Ext3 journalling file system support (CONFIG_EXT3_FS)
\_ [M] Quota Support (CONFIG_QUOTA)
\_ [*] Compatibility with older quotactl interface (CONFIG_QUOTA_COMPAT)
\_ [*]Quota format v2 support (CONFIG_QFMT_V2)
\_ [*] VPS filesystem (CONFIG_SIM_FS)
\_ [M] Virtuozzo Disk Quota support (CONFIG_VZ_QUOTA)
\-> [*] Per-user and per-group quota in Virtuozzo quota partitions (VZ_QUOTA_UGID)

Security
\->[ ] Enable different security models

OpenVZ ... (what else :-)
\_[*] Virtual Environment support (CONFIG_VE)
\_ <M> VE calls interface (CONFIG_VE_CALLS)
\_ <M> VE networking (CONFIG_VE_NETDEV)
\_ <M> Virtual ethernet device (CONFIG_VE_ETHDEV)
\_ <M> VE device (CONFIG_VZ_DEV)
\_ [*] VE netfiltering (CONFIG_VE_IPTABLES)
\_ <M> VE watchdog module (CONFIG_VZ_WDOG)
\_ <M> Checkpointing & restoring Virtual Environments (CONFIG_VZ_CHECKPOINT)

User resources ... (User Beancounters)
\_ [*] Enable user resource accounting (CONFIG_USER_RESOURCE)
\_ [*] Account physical memory usage ( CONFIG_USER_RSS_ACCOUNTING)
\_ [*] Account disk IO (CONFIG_UBC_IO_ACCT)
\_ [*] Account swap usage (CONFIG_USER_SWAP_ACCOUNTING)
\_ [*] Report resource usage in /proc (CONFIG_USER_RESOURCE_PROC)
\_ [*] User resources debug features (CONFIG_UBC_DEBUG)
\_ [*] Debug kmemsize with cache counters (CONFIG_UBC_DEBUG_KMEM)
</pre>

{{Note|better to build the kernel-headers as well, so afterward other kernel-modules can
be built without whole kernel tree (e.g. drbd -> drbd0.7-module-source)}}

See also : "make-kpkg --targets"

==== Compiling ====

Compile your kernel (as user root, or you need the --rootcmd!)
# make-kpkg --append_to_version=-1-openvz --added_patches=openvz --revision=1 --initrd binary-arch
or all above with one step
# make-kpkg --append_to_version=-1-openvz --added_patches=openvz --revision=1 --initrd --config menuconfig binary-arch

==== Installing ====
Install the kernel and update initramfs:

# dpkg -i ../linux-image-2.6.18-1-openvz_1_i386.deb
# update-initramfs -c -k 2.6.18-1-openvz

{{Note|update-initramfs is done, when make-kpkg is use with --initrd option}}

{{Note|update-grub can be configured by /etc/kernel-img.conf}}

==== Bootloader ====

Update the bootloader (if not done before)

GRUB :

# /usr/sbin/update-grub

{{Note|since the Debian Etch release the location of update-grub is moved from /sbin/update-grub to /usr/sbin/update-grub}}

== Install the toolset ==

You need the toolset for managing OpenVZ Virtual Environments (VE) (You will need ''deb http://debian.systs.org/ etch openvz'' repository for that.)

<pre>
# apt-get install vzctl vzquota vzdump vzctl-ostmpl-debian
</pre>

= modify needed settings =

If you want network access for the virtual server then you need to enable IP forwarding.

An old (before Etch) Debian Way: set "ip_forward" to yes in /etc/network/option.

# editor /etc/network/options

The new (from Etch) standard way is to use sysctl for this (see below).

In some cases you may need to enable proxy_arp for the network devices that you want your virtual hosts to be accessible on.
You can add this to a specific interface in the network configuration (/etc/network/interfaces) by the following lines, replace %DEV% with your device name (ie. eth0).

Example:

<pre>
[...]
# device: %DEV%
iface %DEV% inet static
address 192.168.0.2
netmask 255.255.255.0
network 192.168.2.0
broadcast 192.168.2.255
gateway 192.168.2.1

up sysctl -w net.ipv4.conf.%DEV%.proxy_arp=100
pre-down sysctl -w net.ipv4.conf.%DEV%.proxy_arp=0
[...]
</pre>

or use the /etc/network/if-up/ and /etc/network/if-down.d/ directories.

<pre>
INFO: # man 5 interfaces (to read more about debian's network interface configuration for ifup and ifdown)
INFO: It is recommanded to add the magic-sysrq key, to your /etc/sysctl.conf
</pre>

a (plain) OpenVZ Linux Way:

Add settings to "/etc/sysctl.conf"

<pre>
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv4.conf.default.proxy_arp = 0

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# TCP Explict Congestion Notification
# net.ipv4.tcp_ecn = 0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
</pre>

<pre>
INFO: Suggestion: Please make a symlink from /var/lib/vz to /vz as backward compability to Main OpenVZ
(Debian vz root directory is installed FHS-like to /var/lib/vz)

# ln -s /var/lib/vz /vz
</pre>

'''Before you restart your Server, keep in mind, that your system has all needed modules enabled; booting from your harddisk (e.g. hardware modules, raid system(s), lvm2 etc). May you need a INITRD (initramdisk) or compile needed kernel modules statically in.'''

# reboot

That's all!

Now it's time to create a OS Template or download another precreated OS-Template.

INFO: Suggestions: Setup your default OS Template in /etc/vz/vz.conf

[[Category: HOWTO]]
[[Category: Installation]]

Quick installation (legacy)

2007-11-26T23:31:41Z

Grin: open our eyes... link to debian based system install ;-)

This document briefly describes the steps needed to install OpenVZ on your (RPM based) machine. For '''Debian''' based systems see [[Installation on Debian]].

This document is also available in the following languages: [http://forum.openvz.org/index.php?t=tree&goto=35&#msg_35 French], [http://forum.openvz.org/index.php?t=tree&goto=1805&#msg_1805 German],
[http://wiki.openvz.jp Japanese],
[[Quick_installation_(Spanish)|Spanish]].

OpenVZ consists of a kernel, user-level tools, and VE templates. This guide tells how to install the kernel and the tools.

== Requirements ==
This guide assumes you are running recent release of Fedora Core (like FC5) or RHEL/CentOS 4. Currently, OpenVZ kernel tries to support the same hardware that Red Hat kernels support. For full hardware compatibility list, see [http://www.swsoft.com/en/products/virtuozzo/hcl/ Virtuozzo HCL].

=== Filesystems ===
It is recommended to use a separate partition for VEs private directories (by default /vz/private/<veid>). The reason why you should do so is that if you wish to use OpenVZ per-VE disk quota, you won't be able to use usual Linux disk quotas on the same partition. Bear in mind, that per-VE quota in this context includes not only pure per-VE quota, but also usual Linux disk quota used in VE, not on [[HN]].

At least try to avoid using root partition for VEs, because the root user of VE will be able to overcome 5% disk space barrier in some situations. This way HN root partition can be completely filled and it will break the system.

OpenVZ per-VE disk quota is supported only for ext2/ext3 filesystems. So use one of these filesystems (ext3 is recommended) if you need per-VE disk quota.

=== rpm or yum? ===

In case you have yum utility available on your system, you may want to use it effectively to install and update OpenVZ packages. In case you don't have yum, or don't want to use it, you can use plain old rpm. Instructions for both rpm and yum are provided below.

=== yum pre-setup ===
If you want to use yum, you should set up OpenVZ yum repository first.

Download [http://download.openvz.org/openvz.repo openvz.repo] file and put it to your <code>/etc/yum.repos.d/</code> repository. This can be achieved by the following commands, as root:
<pre>
# cd /etc/yum.repos.d
# wget http://download.openvz.org/openvz.repo
# rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
</pre>

In case you can not cd to /etc/yum.repos.d, it means either yum is not installed on your system, or yum version is too old. In that case, just stick to rpm installation method.

== Kernel installation ==

{{Note|In case you want to recompile the kernel yourself rather than use the one provided by OpenVZ, see [[kernel build]].}}

First, you need to choose what “flavor” of the kernel you want to install. Please refer to [[Kernel flavors]] for more information.

=== Using yum ===
Run the following command
<pre>
# yum install ovzkernel[-flavor]
</pre>

Here <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

=== Using rpm ===
Get the kernel binary RPM from the [http://openvz.org/download/kernel/ Download » Kernel] page, or directly from [http://download.openvz.org/kernel/ download.openvz.org/kernel], or from one of its [[Download mirrors|mirrors]]. You need only one kernel RPM so please [[Kernel flavors|choose the appropriate one]] depending on your hardware.

Next, install the kernel RPM you chose:

<pre>
# rpm -ihv ovzkernel[-flavor]*.rpm
</pre>

Here <code>[-flavor]</code> is optional, and can be <code>-smp</code> or <code>-enterprise</code>. Refer to [[kernel flavors]] for more info.

{{Note|<tt>rpm -U</tt> (where <tt>-U</tt> stands for ''upgrade'') should '''not''' be used, otherwise all currently installed kernels will be uninstalled.}}

== Configuring the bootloader ==

In case GRUB is used as the boot loader, it will be configured automatically: lines similar to these will be added to the <tt>/boot/grub/grub.conf</tt> file:

<pre>
title Fedora Core (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5 quiet rhgb vga=0x31B
initrd /initrd-2.6.8-022stab029.1.img
</pre>
Change <tt>Fedora Core</tt> to <tt>OpenVZ</tt> (just for clarity reasons, so the OpenVZ kernels will not be mixed up with non-OpenVZ ones). Remove extra arguments from the kernel line, leaving only the <tt>root=...</tt> parameter. The modifed portion of <tt>/etc/grub.conf</tt> should look like this:

<pre>
title OpenVZ (2.6.8-022stab029.1)
root (hd0,0)
kernel /vmlinuz-2.6.8-022stab029.1 ro root=/dev/sda5
initrd /initrd-2.6.8-022stab029.1.img
</pre>

== Configuring ==

Please make sure the following steps are performed before rebooting into OpenVZ kernel.

=== sysctl ===

There are a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in <tt>/etc/sysctl.conf</tt> file. Here is the relevant part of the file; please edit it accordingly.

<pre>
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv4.conf.default.proxy_arp = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enables the magic-sysrq key
kernel.sysrq = 1
# TCP Explict Congestion Notification
#net.ipv4.tcp_ecn = 0
# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
</pre>

=== SELinux ===

SELinux should be disabled. To that effect, put the following line to <code>/etc/sysconfig/selinux</code>:
<pre>
SELINUX=disabled
</pre>

=== Conntracks ===

In the stable OpenVZ kernels (those that are 2.6.8-based) netfilter connection tracking for [[VE0]] is disabled by default. If you have a stateful firewall enabled on the host node (it is there by default) you should either disable it, or enable connection tracking for [[VE0]].

To enable conntracks for VE0, add the following line to <code>/etc/modprobe.conf</code> file:
<pre>
options ip_conntrack ip_conntrack_enable_ve0=1
</pre>

{{Note|In kernels later than 2.6.8, connection tracking is enabled by default.}}

== Rebooting into OpenVZ kernel ==

Now reboot the machine and choose "OpenVZ" on the boot loader menu. If the OpenVZ kernel has been booted successfully, proceed to installing the user-level tools for OpenVZ.

== Installing the utilities ==

OpenVZ needs some user-level tools installed. Those are:

; vzctl
: A utility to control OpenVZ VPSs (create, destroy, start, stop, set parameters etc.)
; vzquota
: A utility to manage quotas for VPSs. Mostly used indirectly (by vzctl).

=== Using yum ===

<pre>
# yum install vzctl vzquota
</pre>

=== Using rpm ===

Download the binary RPMs of these utilities from [http://openvz.org/download/utils/ Download » Utils], or directly from [http://download.openvz.org/utils/ download.openvz.org/utils], or from one of its [[Download mirrors|mirrors]]. Install them:

<pre>
# rpm -Uhv vzctl*.rpm vzquota*.rpm
</pre>

If rpm complains about unresolved dependencies, you'll have to satisfy them first, then repeat the installation.

When all the tools are installed, start the OpenVZ subsystem.

== Starting OpenVZ ==

As root, execute the following command:

<pre>
# /sbin/service vz start
</pre>

This will load all the needed OpenVZ kernel modules. This script should also start all the VPSs marked to be auto-started on machine boot (there aren't any yet).

During the next reboot, this script should be executed automatically.

== Next steps ==

OpenVZ is now set up on your machine. To load OpenVZ kernel by default, edit the default line in the /boot/grub/grub.conf file to point to the OpenVZ kernel. For example, if the OpenVZ kernel is the first kernel mentioned in the file, put it as default 0. See man grub.conf for more details.

The next step is to prepare the [[OS template]]: please continue to [[OS template cache preparation]] document.

[[Category: Installation]]
[[Category: HOWTO]]