OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Debian template creation

2013-06-23T20:24:43Z

TimSmall: /* Preparing for and packing template cache */ Use system default editor (whatever that's been set to) instead of insisting on nano (!)

These are rough instructions of how to manually create basic Debian template cache, which can be used to create OpenVZ [[VE]]s based on Debian.

'''Notes:'''
* You shouldn't be running as root, but as a user that is permitted to use sudo instead. It's a dangerous idea, run as root at your peril.
* Anywhere you see <tt>/vz</tt>, you might really need to use <tt>/var/lib/vz</tt> instead, especially on a Debian Etch host.
* Anywhere you see <tt>http://http.us.debian.org/debian/</tt>, you can substitute your favorite Debian mirror. ([http://www.debian.org/mirror/list List of official Debian Mirrors])
* See also: <tt>/usr/share/doc/vzctl/README.Debian</tt> in the ''vzctl'' Debian package

== Prerequisites ==

{{Warning|if you want to use <code>ext4</code> file system for <code>/vz</code>, use <code>nodelalloc</code> option in <code>/etc/fstab</code>, otherwise it will crash. See {{Bug|1509}} and its duplicates for details.}}

You need to have a working copy of <tt>debootstrap</tt> running on your hardware node.

For Debian:
sudo apt-get install debootstrap

For Gentoo:
sudo emerge debootstrap

For Fedora (at least Fedora 8 have it, not sure about earlier versions):
sudo yum install debootstrap

For other distros you might need to install it from sources, or search for an appropriate package for your distribution. An RPM is available on the [http://forum.openvz.org/index.php?t=tree&th=142&mid=584 OpenVZ Forum].

== Bootstrapping Debian ==

You can install different releases of Debian into a VE's private directory using the debootstrap command.

The command parameters are:

debootstrap --arch ARCH NAME DIRECTORY URL

Specify your architecture instead of <tt>i386</tt> if you're using something other than i386/x86. For example, for AMD64/x86_64, use <tt>amd64</tt> or for ia64, use <tt>ia64</tt>. You can use http or ftp in the URL.

We use VE ID of 777 for this example, but it can be any unused ID.

=== Wheezy (current stable) ===

debootstrap --arch i386 wheezy /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 wheezy /vz/private/777 http://ftp.us.debian.org/debian/

=== Squeeze (current oldstable) ===

debootstrap --arch i386 squeeze /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 squeeze /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Lenny (old release) ===

debootstrap --arch i386 lenny /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 lenny /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Etch (old release) ===

debootstrap --arch i386 etch /vz/private/777 http://http.us.debian.org/debian/

=== Sarge (very old release) ===

debootstrap sarge /vz/private/777 http://archive.debian.org/debian

== Preparing the HN network ==
Append the following lines to /etc/sysctl.conf, adjust to taste and then execute "sysctl -p" for them to take effect.
### OpenVZ settings

# On Hardware Node enable packet forwarding to forward
# packets between the HN network interfaces and venet.
# Proxy arp is needed when CT is in a different subnet
# or when using veth AND veth is not bridged to a HN
# interface. When veth is bridged to a HN interface,
# the CT handles its own arps.

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# TCP Explict Congestion Notification
net.ipv4.tcp_ecn = 0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

== Preparing and starting the VE ==

=== Setting VE config ===
First, we need a config for the [[VE]]:
sudo vzctl set 777 --applyconfig vps.basic --save

On debian squeeze only the following worked for me (''confirmed''), because the standard template names in /etc/vz/conf have changed.
sudo vzctl set 777 --applyconfig basic --save

=== Setting VE OSTEMPLATE ===
Also, we need <tt>OSTEMPLATE</tt> to be set in VE configuration file, for [[vzctl]] to work properly.

sudo sh -c 'echo OSTEMPLATE=\"debian-6.0\"' >> /etc/vz/conf/777.conf

=== Setting VE IP address ===
For the [[VE]] to be able to download updates from the Internet, we need a valid IP address for it:
sudo vzctl set 777 --ipadd x.x.x.x --save

{{Note|if you use private IP for the VE, you might have to set up NAT as described in [[Using NAT for VE with private IPs]].}}

=== Setting DNS server for VE ===
For the [[VE]] to be able to download updates from the Internet, we also need to specify a DNS for it:
sudo vzctl set 777 --nameserver x.x.x.x --save

=== Creating /dev/ptmx ===
The ptmx character device should normally exist, but if it doesn't, create one.
sudo mknod --mode 666 /var/lib/vz/private/777/dev/ptmx c 5 2

=== Starting VE ===
Now start the VE:
sudo vzctl start 777

== Customizing the installation ==
A few things need to be done inside a newly created VE for it to become suitable for OpenVZ. Enter the VE to begin the configuration (note: if running a wheezy container on a squeeze hardware node, you'll need to manually install a newer version of vzctl (the one from wheezy will be fine - http://packages.debian.org/wheezy/vzctl) due to this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683454 - without making this change, the enter command will hang). Exporting the path is optional.
sudo vzctl enter 777
export PATH=/sbin:/usr/sbin:/bin:/usr/bin

{{Warning|Do not run the commands below on the hardware node, they are only to be run within the VE!}}

=== Set Debian repositories ===
The list shown is for wheezy, and downloading from US located servers - adjust your release name and mirror location as necessary
cat <<EOF > /etc/apt/sources.list
deb http://http.us.debian.org/debian wheezy main contrib
deb http://security.debian.org wheezy/updates main contrib
deb http://http.us.debian.org/debian wheezy-updates main
## backports - ONLY IF YOU KNOW WHAT YOU DO
# deb http://http.us.debian.org/debian-backports/ wheezy-backports main
EOF

=== Get new security updates ===
apt-get update
apt-get upgrade

=== Install some more packages ===
Installing packages could be an interactive process so the system might ask some questions. You can install more packages if you'd like. For example:
apt-get install ssh quota less

=== Set sane permissions for <tt>/root</tt> directory ===
chmod 700 /root

=== Disable root login ===
This will disable root login by default.
usermod -L root

=== Disable getty ===
Disable running <tt>getty</tt>s on terminals as a VE does not have any:
sed -i -e '/getty/d' /etc/inittab

=== Disable <tt>sync()</tt> for syslog ===
Turn off doing <tt>sync()</tt> on every write for <tt>syslog</tt>'s log files, to improve I/O performance:
<pre>sed -i -e 's@$[[:space:]]$$/var/log/$@\1-\2@' /etc/*syslog.conf</pre>

=== Fix <tt>/etc/mtab</tt> ===
Link <tt>/etc/mtab</tt> to <tt>/proc/mounts</tt>, so <tt>df</tt> and friends will work:
rm -f /etc/mtab
ln -s /proc/mounts /etc/mtab

=== Remove some unneeded packages ===
If you have any packages you'd like to remove, now's the time for it. Here's an example — note that not all of those packages are installed by default in Debian Squeeze (although they were in earlier versions):
dpkg --purge modutils ppp pppoeconf pppoe pppconfig module-init-tools

=== Disable services ===
Do not start some services, stick to bare minimum:
update-rc.d -f klogd remove
update-rc.d -f quotarpc remove
update-rc.d -f exim4 remove
update-rc.d -f inetd remove

For dependency-based boot sequence introduced with Squeeze type:

update-rc.d-insserv -f klogd remove
update-rc.d-insserv -f quotarpc remove
update-rc.d-insserv -f exim4 remove
update-rc.d-insserv -f inetd remove

=== Fix SSH host keys ===
This is only useful if you installed SSH. Each individual [[VE]] should have its own pair of SSH host keys. The code below will wipe out the existing SSH keys and instruct the newly-created [[VE]] to create new SSH keys on first boot.


<source lang="bash">
rm -f /etc/ssh/ssh_host_*
cat << EOF > /etc/rc2.d/S15ssh_gen_host_keys
#!/bin/bash
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ''
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ''
rm -f \$0
EOF
chmod a+x /etc/rc2.d/S15ssh_gen_host_keys
</source>

{{Note|This will not work using the dependency-based boot sequence introduced with Squeeze. See the section below. }}

=== Fix SSH host keys in Squeeze when using dependency-based booting ===

rm -f /etc/ssh/ssh_host_*

<source lang="bash">
cat << EOF > /etc/init.d/ssh_gen_host_keys
#!/bin/sh
### BEGIN INIT INFO
# Provides: Generates new ssh host keys on first boot
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: Generates new ssh host keys on first boot
# Description: Generates new ssh host keys on first boot
### END INIT INFO
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ""
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ""
insserv -r /etc/init.d/ssh_gen_host_keys
rm -f \$0
EOF
</source>
chmod a+x /etc/init.d/ssh_gen_host_keys
insserv /etc/init.d/ssh_gen_host_keys

=== Change timezone ===

You might want to change timezone if you do not live in $UTC. The following example is for Germany

<source lang="bash">
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
</source>
or even better
<source lang="bash">
dpkg-reconfigure tzdata
</source>

=== Clean packages ===
After installing packages, you'll have some junk packages laying around in your cache. Since you don't want your template to have those, this command will wipe them out.
apt-get --purge clean

Now everything is done. Exit from the template and go back to the hardware node.
exit

== Preparing for and packing template cache ==

We don't need an IP for the VE anymore, and we definitely do not need it in template cache, so remove it:
sudo vzctl set 777 --ipdel all --save

Also, remove DNS server and search domain information from ''/etc/resolv.conf'' file '''in VE''':
sudo editor /vz/private/777/etc/resolv.conf

Also, remove ''/etc/hostname'' file '''in VE''':
sudo rm -f /vz/private/777/etc/hostname

Stop the VE:
sudo vzctl stop 777

Go to the VE directory:
cd /vz/private/777

Now create a cached OS tarball. In the command below, you'll want to replace <tt>i386</tt> with your architecture (i386, amd64, ia64, etc).
sudo tar --numeric-owner -zcf /vz/template/cache/debian-5.0-i386-minimal.tar.gz .

Look at the resulting tarball to see its size is sane:
# ls -lh /vz/template/cache
-rw-r--r-- 1 root root 51M Apr 10 03:16 debian-5.0-i386-minimal.tar.gz

== Checking if template cache works ==
We can now create a VE based on the just-created template cache. Be sure to change <tt>i386</tt> to your architecture just like you did when you named the tarball above.
sudo vzctl create 123456 --ostemplate debian-5.0-i386-minimal

Now make sure that it works:
sudo vzctl start 123456
sudo vzctl exec 123456 ps ax

You should see that a few processes are running.

== Final cleanup ==
Stop and remove the test VE you just created:
sudo vzctl stop 123456
sudo vzctl destroy 123456
sudo rm /etc/vz/conf/123456.conf.destroyed

Finally, let's remove the VE we used for OS template cache creation:
sudo vzctl destroy 777
sudo rm /etc/vz/conf/777.conf.destroyed

You might want to edit /etc/vz/vz.conf and change DEF_OSTEMPLATE to the name of the template you use most often so that you don't have to specify the template when creating a VE.
DEF_OSTEMPLATE="debian-6.0-i386-minimal"
If you use iptables, you might want to include additional modules in the list for IPTABLES in /etc/vz/vz.conf. See ''man vzctl'' for a list of available modules.

[[Category: HOWTO]]
[[Category: Templates]]
[[Category: Debian]]

Debian template creation

2013-06-23T20:12:29Z

TimSmall: /* Set up Debian repositories */ Add note re release name and mirror customisation

These are rough instructions of how to manually create basic Debian template cache, which can be used to create OpenVZ [[VE]]s based on Debian.

'''Notes:'''
* You shouldn't be running as root, but as a user that is permitted to use sudo instead. It's a dangerous idea, run as root at your peril.
* Anywhere you see <tt>/vz</tt>, you might really need to use <tt>/var/lib/vz</tt> instead, especially on a Debian Etch host.
* Anywhere you see <tt>http://http.us.debian.org/debian/</tt>, you can substitute your favorite Debian mirror. ([http://www.debian.org/mirror/list List of official Debian Mirrors])
* See also: <tt>/usr/share/doc/vzctl/README.Debian</tt> in the ''vzctl'' Debian package

== Prerequisites ==

{{Warning|if you want to use <code>ext4</code> file system for <code>/vz</code>, use <code>nodelalloc</code> option in <code>/etc/fstab</code>, otherwise it will crash. See {{Bug|1509}} and its duplicates for details.}}

You need to have a working copy of <tt>debootstrap</tt> running on your hardware node.

For Debian:
sudo apt-get install debootstrap

For Gentoo:
sudo emerge debootstrap

For Fedora (at least Fedora 8 have it, not sure about earlier versions):
sudo yum install debootstrap

For other distros you might need to install it from sources, or search for an appropriate package for your distribution. An RPM is available on the [http://forum.openvz.org/index.php?t=tree&th=142&mid=584 OpenVZ Forum].

== Bootstrapping Debian ==

You can install different releases of Debian into a VE's private directory using the debootstrap command.

The command parameters are:

debootstrap --arch ARCH NAME DIRECTORY URL

Specify your architecture instead of <tt>i386</tt> if you're using something other than i386/x86. For example, for AMD64/x86_64, use <tt>amd64</tt> or for ia64, use <tt>ia64</tt>. You can use http or ftp in the URL.

We use VE ID of 777 for this example, but it can be any unused ID.

=== Wheezy (current stable) ===

debootstrap --arch i386 wheezy /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 wheezy /vz/private/777 http://ftp.us.debian.org/debian/

=== Squeeze (current oldstable) ===

debootstrap --arch i386 squeeze /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 squeeze /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Lenny (old release) ===

debootstrap --arch i386 lenny /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 lenny /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Etch (old release) ===

debootstrap --arch i386 etch /vz/private/777 http://http.us.debian.org/debian/

=== Sarge (very old release) ===

debootstrap sarge /vz/private/777 http://archive.debian.org/debian

== Preparing the HN network ==
Append the following lines to /etc/sysctl.conf, adjust to taste and then execute "sysctl -p" for them to take effect.
### OpenVZ settings

# On Hardware Node enable packet forwarding to forward
# packets between the HN network interfaces and venet.
# Proxy arp is needed when CT is in a different subnet
# or when using veth AND veth is not bridged to a HN
# interface. When veth is bridged to a HN interface,
# the CT handles its own arps.

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# TCP Explict Congestion Notification
net.ipv4.tcp_ecn = 0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

== Preparing and starting the VE ==

=== Setting VE config ===
First, we need a config for the [[VE]]:
sudo vzctl set 777 --applyconfig vps.basic --save

On debian squeeze only the following worked for me (''confirmed''), because the standard template names in /etc/vz/conf have changed.
sudo vzctl set 777 --applyconfig basic --save

=== Setting VE OSTEMPLATE ===
Also, we need <tt>OSTEMPLATE</tt> to be set in VE configuration file, for [[vzctl]] to work properly.

sudo sh -c 'echo OSTEMPLATE=\"debian-6.0\"' >> /etc/vz/conf/777.conf

=== Setting VE IP address ===
For the [[VE]] to be able to download updates from the Internet, we need a valid IP address for it:
sudo vzctl set 777 --ipadd x.x.x.x --save

{{Note|if you use private IP for the VE, you might have to set up NAT as described in [[Using NAT for VE with private IPs]].}}

=== Setting DNS server for VE ===
For the [[VE]] to be able to download updates from the Internet, we also need to specify a DNS for it:
sudo vzctl set 777 --nameserver x.x.x.x --save

=== Creating /dev/ptmx ===
The ptmx character device should normally exist, but if it doesn't, create one.
sudo mknod --mode 666 /var/lib/vz/private/777/dev/ptmx c 5 2

=== Starting VE ===
Now start the VE:
sudo vzctl start 777

== Customizing the installation ==
A few things need to be done inside a newly created VE for it to become suitable for OpenVZ. Enter the VE to begin the configuration (note: if running a wheezy container on a squeeze hardware node, you'll need to manually install a newer version of vzctl (the one from wheezy will be fine - http://packages.debian.org/wheezy/vzctl) due to this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683454 - without making this change, the enter command will hang). Exporting the path is optional.
sudo vzctl enter 777
export PATH=/sbin:/usr/sbin:/bin:/usr/bin

{{Warning|Do not run the commands below on the hardware node, they are only to be run within the VE!}}

=== Set Debian repositories ===
The list shown is for wheezy, and downloading from US located servers - adjust your release name and mirror location as necessary
cat <<EOF > /etc/apt/sources.list
deb http://http.us.debian.org/debian wheezy main contrib
deb http://security.debian.org wheezy/updates main contrib
deb http://http.us.debian.org/debian wheezy-updates main
## backports - ONLY IF YOU KNOW WHAT YOU DO
# deb http://http.us.debian.org/debian-backports/ wheezy-backports main
EOF

=== Get new security updates ===
apt-get update
apt-get upgrade

=== Install some more packages ===
Installing packages could be an interactive process so the system might ask some questions. You can install more packages if you'd like. For example:
apt-get install ssh quota less

=== Set sane permissions for <tt>/root</tt> directory ===
chmod 700 /root

=== Disable root login ===
This will disable root login by default.
usermod -L root

=== Disable getty ===
Disable running <tt>getty</tt>s on terminals as a VE does not have any:
sed -i -e '/getty/d' /etc/inittab

=== Disable <tt>sync()</tt> for syslog ===
Turn off doing <tt>sync()</tt> on every write for <tt>syslog</tt>'s log files, to improve I/O performance:
<pre>sed -i -e 's@$[[:space:]]$$/var/log/$@\1-\2@' /etc/*syslog.conf</pre>

=== Fix <tt>/etc/mtab</tt> ===
Link <tt>/etc/mtab</tt> to <tt>/proc/mounts</tt>, so <tt>df</tt> and friends will work:
rm -f /etc/mtab
ln -s /proc/mounts /etc/mtab

=== Remove some unneeded packages ===
If you have any packages you'd like to remove, now's the time for it. Here's an example — note that not all of those packages are installed by default in Debian Squeeze (although they were in earlier versions):
dpkg --purge modutils ppp pppoeconf pppoe pppconfig module-init-tools

=== Disable services ===
Do not start some services, stick to bare minimum:
update-rc.d -f klogd remove
update-rc.d -f quotarpc remove
update-rc.d -f exim4 remove
update-rc.d -f inetd remove

For dependency-based boot sequence introduced with Squeeze type:

update-rc.d-insserv -f klogd remove
update-rc.d-insserv -f quotarpc remove
update-rc.d-insserv -f exim4 remove
update-rc.d-insserv -f inetd remove

=== Fix SSH host keys ===
This is only useful if you installed SSH. Each individual [[VE]] should have its own pair of SSH host keys. The code below will wipe out the existing SSH keys and instruct the newly-created [[VE]] to create new SSH keys on first boot.


<source lang="bash">
rm -f /etc/ssh/ssh_host_*
cat << EOF > /etc/rc2.d/S15ssh_gen_host_keys
#!/bin/bash
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ''
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ''
rm -f \$0
EOF
chmod a+x /etc/rc2.d/S15ssh_gen_host_keys
</source>

{{Note|This will not work using the dependency-based boot sequence introduced with Squeeze. See the section below. }}

=== Fix SSH host keys in Squeeze when using dependency-based booting ===

rm -f /etc/ssh/ssh_host_*

<source lang="bash">
cat << EOF > /etc/init.d/ssh_gen_host_keys
#!/bin/sh
### BEGIN INIT INFO
# Provides: Generates new ssh host keys on first boot
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: Generates new ssh host keys on first boot
# Description: Generates new ssh host keys on first boot
### END INIT INFO
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ""
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ""
insserv -r /etc/init.d/ssh_gen_host_keys
rm -f \$0
EOF
</source>
chmod a+x /etc/init.d/ssh_gen_host_keys
insserv /etc/init.d/ssh_gen_host_keys

=== Change timezone ===

You might want to change timezone if you do not live in $UTC. The following example is for Germany

<source lang="bash">
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
</source>
or even better
<source lang="bash">
dpkg-reconfigure tzdata
</source>

=== Clean packages ===
After installing packages, you'll have some junk packages laying around in your cache. Since you don't want your template to have those, this command will wipe them out.
apt-get --purge clean

Now everything is done. Exit from the template and go back to the hardware node.
exit

== Preparing for and packing template cache ==

We don't need an IP for the VE anymore, and we definitely do not need it in template cache, so remove it:
sudo vzctl set 777 --ipdel all --save

Also, remove DNS server and search domain information from ''/etc/resolv.conf'' file '''in VE''':
sudo nano /vz/private/777/etc/resolv.conf

Also, remove ''/etc/hostname'' file '''in VE''':
sudo rm -f /vz/private/777/etc/hostname

Stop the VE:
sudo vzctl stop 777

Go to the VE directory:
cd /vz/private/777

Now create a cached OS tarball. In the command below, you'll want to replace <tt>i386</tt> with your architecture (i386, amd64, ia64, etc).
sudo tar --numeric-owner -zcf /vz/template/cache/debian-5.0-i386-minimal.tar.gz .

Look at the resulting tarball to see its size is sane:
# ls -lh /vz/template/cache
-rw-r--r-- 1 root root 51M Apr 10 03:16 debian-5.0-i386-minimal.tar.gz

== Checking if template cache works ==
We can now create a VE based on the just-created template cache. Be sure to change <tt>i386</tt> to your architecture just like you did when you named the tarball above.
sudo vzctl create 123456 --ostemplate debian-5.0-i386-minimal

Now make sure that it works:
sudo vzctl start 123456
sudo vzctl exec 123456 ps ax

You should see that a few processes are running.

== Final cleanup ==
Stop and remove the test VE you just created:
sudo vzctl stop 123456
sudo vzctl destroy 123456
sudo rm /etc/vz/conf/123456.conf.destroyed

Finally, let's remove the VE we used for OS template cache creation:
sudo vzctl destroy 777
sudo rm /etc/vz/conf/777.conf.destroyed

You might want to edit /etc/vz/vz.conf and change DEF_OSTEMPLATE to the name of the template you use most often so that you don't have to specify the template when creating a VE.
DEF_OSTEMPLATE="debian-6.0-i386-minimal"
If you use iptables, you might want to include additional modules in the list for IPTABLES in /etc/vz/vz.conf. See ''man vzctl'' for a list of available modules.

[[Category: HOWTO]]
[[Category: Templates]]
[[Category: Debian]]

Debian template creation

2013-06-23T20:08:59Z

TimSmall: /* Customizing the installation */ Add note re fix for using wheezy containers on squeeze HNs.

These are rough instructions of how to manually create basic Debian template cache, which can be used to create OpenVZ [[VE]]s based on Debian.

'''Notes:'''
* You shouldn't be running as root, but as a user that is permitted to use sudo instead. It's a dangerous idea, run as root at your peril.
* Anywhere you see <tt>/vz</tt>, you might really need to use <tt>/var/lib/vz</tt> instead, especially on a Debian Etch host.
* Anywhere you see <tt>http://http.us.debian.org/debian/</tt>, you can substitute your favorite Debian mirror. ([http://www.debian.org/mirror/list List of official Debian Mirrors])
* See also: <tt>/usr/share/doc/vzctl/README.Debian</tt> in the ''vzctl'' Debian package

== Prerequisites ==

{{Warning|if you want to use <code>ext4</code> file system for <code>/vz</code>, use <code>nodelalloc</code> option in <code>/etc/fstab</code>, otherwise it will crash. See {{Bug|1509}} and its duplicates for details.}}

You need to have a working copy of <tt>debootstrap</tt> running on your hardware node.

For Debian:
sudo apt-get install debootstrap

For Gentoo:
sudo emerge debootstrap

For Fedora (at least Fedora 8 have it, not sure about earlier versions):
sudo yum install debootstrap

For other distros you might need to install it from sources, or search for an appropriate package for your distribution. An RPM is available on the [http://forum.openvz.org/index.php?t=tree&th=142&mid=584 OpenVZ Forum].

== Bootstrapping Debian ==

You can install different releases of Debian into a VE's private directory using the debootstrap command.

The command parameters are:

debootstrap --arch ARCH NAME DIRECTORY URL

Specify your architecture instead of <tt>i386</tt> if you're using something other than i386/x86. For example, for AMD64/x86_64, use <tt>amd64</tt> or for ia64, use <tt>ia64</tt>. You can use http or ftp in the URL.

We use VE ID of 777 for this example, but it can be any unused ID.

=== Wheezy (current stable) ===

debootstrap --arch i386 wheezy /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 wheezy /vz/private/777 http://ftp.us.debian.org/debian/

=== Squeeze (current oldstable) ===

debootstrap --arch i386 squeeze /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 squeeze /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Lenny (old release) ===

debootstrap --arch i386 lenny /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 lenny /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Etch (old release) ===

debootstrap --arch i386 etch /vz/private/777 http://http.us.debian.org/debian/

=== Sarge (very old release) ===

debootstrap sarge /vz/private/777 http://archive.debian.org/debian

== Preparing the HN network ==
Append the following lines to /etc/sysctl.conf, adjust to taste and then execute "sysctl -p" for them to take effect.
### OpenVZ settings

# On Hardware Node enable packet forwarding to forward
# packets between the HN network interfaces and venet.
# Proxy arp is needed when CT is in a different subnet
# or when using veth AND veth is not bridged to a HN
# interface. When veth is bridged to a HN interface,
# the CT handles its own arps.

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# TCP Explict Congestion Notification
net.ipv4.tcp_ecn = 0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

== Preparing and starting the VE ==

=== Setting VE config ===
First, we need a config for the [[VE]]:
sudo vzctl set 777 --applyconfig vps.basic --save

On debian squeeze only the following worked for me (''confirmed''), because the standard template names in /etc/vz/conf have changed.
sudo vzctl set 777 --applyconfig basic --save

=== Setting VE OSTEMPLATE ===
Also, we need <tt>OSTEMPLATE</tt> to be set in VE configuration file, for [[vzctl]] to work properly.

sudo sh -c 'echo OSTEMPLATE=\"debian-6.0\"' >> /etc/vz/conf/777.conf

=== Setting VE IP address ===
For the [[VE]] to be able to download updates from the Internet, we need a valid IP address for it:
sudo vzctl set 777 --ipadd x.x.x.x --save

{{Note|if you use private IP for the VE, you might have to set up NAT as described in [[Using NAT for VE with private IPs]].}}

=== Setting DNS server for VE ===
For the [[VE]] to be able to download updates from the Internet, we also need to specify a DNS for it:
sudo vzctl set 777 --nameserver x.x.x.x --save

=== Creating /dev/ptmx ===
The ptmx character device should normally exist, but if it doesn't, create one.
sudo mknod --mode 666 /var/lib/vz/private/777/dev/ptmx c 5 2

=== Starting VE ===
Now start the VE:
sudo vzctl start 777

== Customizing the installation ==
A few things need to be done inside a newly created VE for it to become suitable for OpenVZ. Enter the VE to begin the configuration (note: if running a wheezy container on a squeeze hardware node, you'll need to manually install a newer version of vzctl (the one from wheezy will be fine - http://packages.debian.org/wheezy/vzctl) due to this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683454 - without making this change, the enter command will hang). Exporting the path is optional.
sudo vzctl enter 777
export PATH=/sbin:/usr/sbin:/bin:/usr/bin

{{Warning|Do not run the commands below on the hardware node, they are only to be run within the VE!}}

=== Set Debian repositories ===
cat <<EOF > /etc/apt/sources.list
deb http://http.us.debian.org/debian squeeze main contrib
deb http://security.debian.org squeeze/updates main contrib
deb http://http.us.debian.org/debian squeeze-updates main
## backports - ONLY IF YOU KNOW WHAT YOU DO
# deb http://http.us.debian.org/debian-backports/ squeeze-backports main
EOF

=== Get new security updates ===
apt-get update
apt-get upgrade

=== Install some more packages ===
Installing packages could be an interactive process so the system might ask some questions. You can install more packages if you'd like. For example:
apt-get install ssh quota less

=== Set sane permissions for <tt>/root</tt> directory ===
chmod 700 /root

=== Disable root login ===
This will disable root login by default.
usermod -L root

=== Disable getty ===
Disable running <tt>getty</tt>s on terminals as a VE does not have any:
sed -i -e '/getty/d' /etc/inittab

=== Disable <tt>sync()</tt> for syslog ===
Turn off doing <tt>sync()</tt> on every write for <tt>syslog</tt>'s log files, to improve I/O performance:
<pre>sed -i -e 's@$[[:space:]]$$/var/log/$@\1-\2@' /etc/*syslog.conf</pre>

=== Fix <tt>/etc/mtab</tt> ===
Link <tt>/etc/mtab</tt> to <tt>/proc/mounts</tt>, so <tt>df</tt> and friends will work:
rm -f /etc/mtab
ln -s /proc/mounts /etc/mtab

=== Remove some unneeded packages ===
If you have any packages you'd like to remove, now's the time for it. Here's an example — note that not all of those packages are installed by default in Debian Squeeze (although they were in earlier versions):
dpkg --purge modutils ppp pppoeconf pppoe pppconfig module-init-tools

=== Disable services ===
Do not start some services, stick to bare minimum:
update-rc.d -f klogd remove
update-rc.d -f quotarpc remove
update-rc.d -f exim4 remove
update-rc.d -f inetd remove

For dependency-based boot sequence introduced with Squeeze type:

update-rc.d-insserv -f klogd remove
update-rc.d-insserv -f quotarpc remove
update-rc.d-insserv -f exim4 remove
update-rc.d-insserv -f inetd remove

=== Fix SSH host keys ===
This is only useful if you installed SSH. Each individual [[VE]] should have its own pair of SSH host keys. The code below will wipe out the existing SSH keys and instruct the newly-created [[VE]] to create new SSH keys on first boot.


<source lang="bash">
rm -f /etc/ssh/ssh_host_*
cat << EOF > /etc/rc2.d/S15ssh_gen_host_keys
#!/bin/bash
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ''
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ''
rm -f \$0
EOF
chmod a+x /etc/rc2.d/S15ssh_gen_host_keys
</source>

{{Note|This will not work using the dependency-based boot sequence introduced with Squeeze. See the section below. }}

=== Fix SSH host keys in Squeeze when using dependency-based booting ===

rm -f /etc/ssh/ssh_host_*

<source lang="bash">
cat << EOF > /etc/init.d/ssh_gen_host_keys
#!/bin/sh
### BEGIN INIT INFO
# Provides: Generates new ssh host keys on first boot
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: Generates new ssh host keys on first boot
# Description: Generates new ssh host keys on first boot
### END INIT INFO
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ""
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ""
insserv -r /etc/init.d/ssh_gen_host_keys
rm -f \$0
EOF
</source>
chmod a+x /etc/init.d/ssh_gen_host_keys
insserv /etc/init.d/ssh_gen_host_keys

=== Change timezone ===

You might want to change timezone if you do not live in $UTC. The following example is for Germany

<source lang="bash">
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
</source>
or even better
<source lang="bash">
dpkg-reconfigure tzdata
</source>

=== Clean packages ===
After installing packages, you'll have some junk packages laying around in your cache. Since you don't want your template to have those, this command will wipe them out.
apt-get --purge clean

Now everything is done. Exit from the template and go back to the hardware node.
exit

== Preparing for and packing template cache ==

We don't need an IP for the VE anymore, and we definitely do not need it in template cache, so remove it:
sudo vzctl set 777 --ipdel all --save

Also, remove DNS server and search domain information from ''/etc/resolv.conf'' file '''in VE''':
sudo nano /vz/private/777/etc/resolv.conf

Also, remove ''/etc/hostname'' file '''in VE''':
sudo rm -f /vz/private/777/etc/hostname

Stop the VE:
sudo vzctl stop 777

Go to the VE directory:
cd /vz/private/777

Now create a cached OS tarball. In the command below, you'll want to replace <tt>i386</tt> with your architecture (i386, amd64, ia64, etc).
sudo tar --numeric-owner -zcf /vz/template/cache/debian-5.0-i386-minimal.tar.gz .

Look at the resulting tarball to see its size is sane:
# ls -lh /vz/template/cache
-rw-r--r-- 1 root root 51M Apr 10 03:16 debian-5.0-i386-minimal.tar.gz

== Checking if template cache works ==
We can now create a VE based on the just-created template cache. Be sure to change <tt>i386</tt> to your architecture just like you did when you named the tarball above.
sudo vzctl create 123456 --ostemplate debian-5.0-i386-minimal

Now make sure that it works:
sudo vzctl start 123456
sudo vzctl exec 123456 ps ax

You should see that a few processes are running.

== Final cleanup ==
Stop and remove the test VE you just created:
sudo vzctl stop 123456
sudo vzctl destroy 123456
sudo rm /etc/vz/conf/123456.conf.destroyed

Finally, let's remove the VE we used for OS template cache creation:
sudo vzctl destroy 777
sudo rm /etc/vz/conf/777.conf.destroyed

You might want to edit /etc/vz/vz.conf and change DEF_OSTEMPLATE to the name of the template you use most often so that you don't have to specify the template when creating a VE.
DEF_OSTEMPLATE="debian-6.0-i386-minimal"
If you use iptables, you might want to include additional modules in the list for IPTABLES in /etc/vz/vz.conf. See ''man vzctl'' for a list of available modules.

[[Category: HOWTO]]
[[Category: Templates]]
[[Category: Debian]]

Debian template creation

2013-06-23T20:03:50Z

TimSmall: /* Bootstrapping Debian */ Add wheezy (Debian 7.0)

These are rough instructions of how to manually create basic Debian template cache, which can be used to create OpenVZ [[VE]]s based on Debian.

'''Notes:'''
* You shouldn't be running as root, but as a user that is permitted to use sudo instead. It's a dangerous idea, run as root at your peril.
* Anywhere you see <tt>/vz</tt>, you might really need to use <tt>/var/lib/vz</tt> instead, especially on a Debian Etch host.
* Anywhere you see <tt>http://http.us.debian.org/debian/</tt>, you can substitute your favorite Debian mirror. ([http://www.debian.org/mirror/list List of official Debian Mirrors])
* See also: <tt>/usr/share/doc/vzctl/README.Debian</tt> in the ''vzctl'' Debian package

== Prerequisites ==

{{Warning|if you want to use <code>ext4</code> file system for <code>/vz</code>, use <code>nodelalloc</code> option in <code>/etc/fstab</code>, otherwise it will crash. See {{Bug|1509}} and its duplicates for details.}}

You need to have a working copy of <tt>debootstrap</tt> running on your hardware node.

For Debian:
sudo apt-get install debootstrap

For Gentoo:
sudo emerge debootstrap

For Fedora (at least Fedora 8 have it, not sure about earlier versions):
sudo yum install debootstrap

For other distros you might need to install it from sources, or search for an appropriate package for your distribution. An RPM is available on the [http://forum.openvz.org/index.php?t=tree&th=142&mid=584 OpenVZ Forum].

== Bootstrapping Debian ==

You can install different releases of Debian into a VE's private directory using the debootstrap command.

The command parameters are:

debootstrap --arch ARCH NAME DIRECTORY URL

Specify your architecture instead of <tt>i386</tt> if you're using something other than i386/x86. For example, for AMD64/x86_64, use <tt>amd64</tt> or for ia64, use <tt>ia64</tt>. You can use http or ftp in the URL.

We use VE ID of 777 for this example, but it can be any unused ID.

=== Wheezy (current stable) ===

debootstrap --arch i386 wheezy /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 wheezy /vz/private/777 http://ftp.us.debian.org/debian/

=== Squeeze (current oldstable) ===

debootstrap --arch i386 squeeze /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 squeeze /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Lenny (old release) ===

debootstrap --arch i386 lenny /vz/private/777 http://http.us.debian.org/debian/
or
debootstrap --arch amd64 lenny /vz/private/777 ftp://ftp.us.debian.org/debian/

=== Etch (old release) ===

debootstrap --arch i386 etch /vz/private/777 http://http.us.debian.org/debian/

=== Sarge (very old release) ===

debootstrap sarge /vz/private/777 http://archive.debian.org/debian

== Preparing the HN network ==
Append the following lines to /etc/sysctl.conf, adjust to taste and then execute "sysctl -p" for them to take effect.
### OpenVZ settings

# On Hardware Node enable packet forwarding to forward
# packets between the HN network interfaces and venet.
# Proxy arp is needed when CT is in a different subnet
# or when using veth AND veth is not bridged to a HN
# interface. When veth is bridged to a HN interface,
# the CT handles its own arps.

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter = 1

# Enables the magic-sysrq key
kernel.sysrq = 1

# TCP Explict Congestion Notification
net.ipv4.tcp_ecn = 0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

== Preparing and starting the VE ==

=== Setting VE config ===
First, we need a config for the [[VE]]:
sudo vzctl set 777 --applyconfig vps.basic --save

On debian squeeze only the following worked for me (''confirmed''), because the standard template names in /etc/vz/conf have changed.
sudo vzctl set 777 --applyconfig basic --save

=== Setting VE OSTEMPLATE ===
Also, we need <tt>OSTEMPLATE</tt> to be set in VE configuration file, for [[vzctl]] to work properly.

sudo sh -c 'echo OSTEMPLATE=\"debian-6.0\"' >> /etc/vz/conf/777.conf

=== Setting VE IP address ===
For the [[VE]] to be able to download updates from the Internet, we need a valid IP address for it:
sudo vzctl set 777 --ipadd x.x.x.x --save

{{Note|if you use private IP for the VE, you might have to set up NAT as described in [[Using NAT for VE with private IPs]].}}

=== Setting DNS server for VE ===
For the [[VE]] to be able to download updates from the Internet, we also need to specify a DNS for it:
sudo vzctl set 777 --nameserver x.x.x.x --save

=== Creating /dev/ptmx ===
The ptmx character device should normally exist, but if it doesn't, create one.
sudo mknod --mode 666 /var/lib/vz/private/777/dev/ptmx c 5 2

=== Starting VE ===
Now start the VE:
sudo vzctl start 777

== Customizing the installation ==
A few things need to be done inside a newly created VE for it to become suitable for OpenVZ. Enter the VE to begin the configuration. Exporting the path is optional.
sudo vzctl enter 777
export PATH=/sbin:/usr/sbin:/bin:/usr/bin

{{Warning|Do not run the commands below on the hardware node, they are only to be run within the VE!}}

=== Set Debian repositories ===
cat <<EOF > /etc/apt/sources.list
deb http://http.us.debian.org/debian squeeze main contrib
deb http://security.debian.org squeeze/updates main contrib
deb http://http.us.debian.org/debian squeeze-updates main
## backports - ONLY IF YOU KNOW WHAT YOU DO
# deb http://http.us.debian.org/debian-backports/ squeeze-backports main
EOF

=== Get new security updates ===
apt-get update
apt-get upgrade

=== Install some more packages ===
Installing packages could be an interactive process so the system might ask some questions. You can install more packages if you'd like. For example:
apt-get install ssh quota less

=== Set sane permissions for <tt>/root</tt> directory ===
chmod 700 /root

=== Disable root login ===
This will disable root login by default.
usermod -L root

=== Disable getty ===
Disable running <tt>getty</tt>s on terminals as a VE does not have any:
sed -i -e '/getty/d' /etc/inittab

=== Disable <tt>sync()</tt> for syslog ===
Turn off doing <tt>sync()</tt> on every write for <tt>syslog</tt>'s log files, to improve I/O performance:
<pre>sed -i -e 's@$[[:space:]]$$/var/log/$@\1-\2@' /etc/*syslog.conf</pre>

=== Fix <tt>/etc/mtab</tt> ===
Link <tt>/etc/mtab</tt> to <tt>/proc/mounts</tt>, so <tt>df</tt> and friends will work:
rm -f /etc/mtab
ln -s /proc/mounts /etc/mtab

=== Remove some unneeded packages ===
If you have any packages you'd like to remove, now's the time for it. Here's an example — note that not all of those packages are installed by default in Debian Squeeze (although they were in earlier versions):
dpkg --purge modutils ppp pppoeconf pppoe pppconfig module-init-tools

=== Disable services ===
Do not start some services, stick to bare minimum:
update-rc.d -f klogd remove
update-rc.d -f quotarpc remove
update-rc.d -f exim4 remove
update-rc.d -f inetd remove

For dependency-based boot sequence introduced with Squeeze type:

update-rc.d-insserv -f klogd remove
update-rc.d-insserv -f quotarpc remove
update-rc.d-insserv -f exim4 remove
update-rc.d-insserv -f inetd remove

=== Fix SSH host keys ===
This is only useful if you installed SSH. Each individual [[VE]] should have its own pair of SSH host keys. The code below will wipe out the existing SSH keys and instruct the newly-created [[VE]] to create new SSH keys on first boot.


<source lang="bash">
rm -f /etc/ssh/ssh_host_*
cat << EOF > /etc/rc2.d/S15ssh_gen_host_keys
#!/bin/bash
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ''
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ''
rm -f \$0
EOF
chmod a+x /etc/rc2.d/S15ssh_gen_host_keys
</source>

{{Note|This will not work using the dependency-based boot sequence introduced with Squeeze. See the section below. }}

=== Fix SSH host keys in Squeeze when using dependency-based booting ===

rm -f /etc/ssh/ssh_host_*

<source lang="bash">
cat << EOF > /etc/init.d/ssh_gen_host_keys
#!/bin/sh
### BEGIN INIT INFO
# Provides: Generates new ssh host keys on first boot
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: Generates new ssh host keys on first boot
# Description: Generates new ssh host keys on first boot
### END INIT INFO
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N ""
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -N ""
insserv -r /etc/init.d/ssh_gen_host_keys
rm -f \$0
EOF
</source>
chmod a+x /etc/init.d/ssh_gen_host_keys
insserv /etc/init.d/ssh_gen_host_keys

=== Change timezone ===

You might want to change timezone if you do not live in $UTC. The following example is for Germany

<source lang="bash">
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
</source>
or even better
<source lang="bash">
dpkg-reconfigure tzdata
</source>

=== Clean packages ===
After installing packages, you'll have some junk packages laying around in your cache. Since you don't want your template to have those, this command will wipe them out.
apt-get --purge clean

Now everything is done. Exit from the template and go back to the hardware node.
exit

== Preparing for and packing template cache ==

We don't need an IP for the VE anymore, and we definitely do not need it in template cache, so remove it:
sudo vzctl set 777 --ipdel all --save

Also, remove DNS server and search domain information from ''/etc/resolv.conf'' file '''in VE''':
sudo nano /vz/private/777/etc/resolv.conf

Also, remove ''/etc/hostname'' file '''in VE''':
sudo rm -f /vz/private/777/etc/hostname

Stop the VE:
sudo vzctl stop 777

Go to the VE directory:
cd /vz/private/777

Now create a cached OS tarball. In the command below, you'll want to replace <tt>i386</tt> with your architecture (i386, amd64, ia64, etc).
sudo tar --numeric-owner -zcf /vz/template/cache/debian-5.0-i386-minimal.tar.gz .

Look at the resulting tarball to see its size is sane:
# ls -lh /vz/template/cache
-rw-r--r-- 1 root root 51M Apr 10 03:16 debian-5.0-i386-minimal.tar.gz

== Checking if template cache works ==
We can now create a VE based on the just-created template cache. Be sure to change <tt>i386</tt> to your architecture just like you did when you named the tarball above.
sudo vzctl create 123456 --ostemplate debian-5.0-i386-minimal

Now make sure that it works:
sudo vzctl start 123456
sudo vzctl exec 123456 ps ax

You should see that a few processes are running.

== Final cleanup ==
Stop and remove the test VE you just created:
sudo vzctl stop 123456
sudo vzctl destroy 123456
sudo rm /etc/vz/conf/123456.conf.destroyed

Finally, let's remove the VE we used for OS template cache creation:
sudo vzctl destroy 777
sudo rm /etc/vz/conf/777.conf.destroyed

You might want to edit /etc/vz/vz.conf and change DEF_OSTEMPLATE to the name of the template you use most often so that you don't have to specify the template when creating a VE.
DEF_OSTEMPLATE="debian-6.0-i386-minimal"
If you use iptables, you might want to include additional modules in the list for IPTABLES in /etc/vz/vz.conf. See ''man vzctl'' for a list of available modules.

[[Category: HOWTO]]
[[Category: Templates]]
[[Category: Debian]]

Installation on Debian/old

2013-03-22T12:53:04Z

TimSmall:

OpenVZ consists of a kernel, user-level tools, and container templates.

This guide tells how to install the kernel and the tools on [http://www.debian.org Debian] Etch or Lenny/Squeeze.

For Squeeze, use the Lenny directions.

For Wheezy (7.0), use the vzctl package included in wheezy, together with the Wheezy OpenVZ kernels from [http://download.openvz.org/debian/ http://download.openvz.org/debian/]. Alternatively reduced functionality may be possible using the stock Debian Wheezy kernel (based on kernel.org version 3.2) and [[Vzctl_for_upstream_kernel]].

{{Note|The majority of the content on this page only applies to older, unsupported Debian versions and should be archived.}}

You may also wish to check the information on [http://wiki.debian.org/OpenVz the Debian wiki].

For Etch users, this document explains how to partially upgrade to Debian Lenny and install from lenny repositories ('''use this options at your risk''').

== Requirements ==

=== Filesystems ===
It's recommended that you use a separate partition for container private
directories (by default <code>/var/lib/vz/private/<CTID></code>). The reason for this is that if you wish to use the OpenVZ per-container disk quota, you won't be able to use usual Linux disk quotas on the same partition. Bear in mind that "per-container quota" in this context includes not only pure per-container quota but also the usual Linux disk quota used in container, not on the [[HN]].

At the very least try to avoid using the root partition for containers, because the root user of a container will be able to overcome the 5% disk space barrier in some situations. If the HN root partition is completely filled, it will break the system.

OpenVZ per-container disk quota is supported only for ext2/ext3 filesystems; therefore it makes sense to use one of these filesystems (ext3 is recommended) if you need per-container disk quota.

=== Repository setup (Etch only) ===

'''If you are using Debian Lenny, this step in no longer required. Openvz kernel packages and tools are available on main repository.'''

==== 1. Using openvz.org repositories ====

At the moment two different repositories are online at http://download.openvz.org:

; by Ola Lundqvist <opal@debian.org>
: (OpenVZ kernels only)
: apt-uri http://download.openvz.org/debian

; by Thorsten Schifferdecker <tsd@debian.systs.org>
: apt-uri http://download.openvz.org/debian-systs
: (Mirror of OpenVZ Repository from http://debian.systs.org/)

{{Note|The next steps use the repository at http://download.openvz.org/debian-systs; the actual OpenVZ Tools for Debian exist only as unstable builds, see http://packages.debian.org/vzctl}}

{{Note|By default, on Ubuntu systems root tasks are executed with [https://help.ubuntu.com/community/RootSudo sudo]}}

This can be done via the following commands, as root or as privileged "sudo" user
<pre>
# echo -e "\ndeb http://download.openvz.org/debian-systs etch openvz" >> /etc/apt/sources.list
# wget -q http://download.openvz.org/debian-systs/dso_archiv_signing_key.asc -O- | apt-key add - && apt-get update
</pre>

==== 2. Using Debian repositories (upgrade to lenny) ====

There is even a '''lenny''' repository with kernel 2.6.28. '''Use it at your own risk!'''

Add lenny repositories to your '''/etc/apt/sources.list'''
<pre>
deb http://DEBIAN-MIRROR/debian/ testing main
deb http://DEBIAN-MIRROR/debian-security/ testing/updates main
</pre>

Enlarge apt-cache adding to '''/etc/apt/apt.conf''' this line:
<pre>
APT::Cache-Limit "100000000";
</pre>

Give etch package priority over lenny packages. Edit '''/etc/apt/preferences''' and set like this:
<pre>
Package: *
Pin: release a=etch
Pin-Priority: 700

Package: *
Pin: release a=lenny
Pin-Priority: 650
</pre>

Then '''apt-get update && apt-get dist-upgrade''' to upgrade to lenny.

== Kernel installation ==

=== Wheezy and Lenny ===

{{Note|The best kernel to use is [[Download/kernel/rhel6|RHEL6-based]]. Please see [[Install_kernel_from_RPM_on_Debian_6.0]]}}

=== Etch ===

==== 1. Using openvz kernel repositories ====

{{Note|In case you want to recompile the OpenVZ kernel yourself on Debian, see [[Compiling the OpenVZ kernel (the Debian way)]].}}

First, you need to choose what kernel you want to install.

{| class="wikitable"
|+'''OpenVZ Kernel list built with kernel config from http://download.openvz.org'''
! Kernel !! Description !! Hardware !! Debian Architecture
|-
! ovzkernel-2.6.18
| uniprocessor
| up to 4GB of RAM
| i386 and amd64
|-
! ovzkernel-2.6.18-smp
| symmetric multiprocessor
| up to 4 GB of RAM
| i386 and amd64
|-
! ovzkernel-2.6.18-enterprise
| SMP + PAE support + 4/4GB split
| up to 64 GB of RAM
| i386 only
|}

{| class="wikitable"
|+'''OpenVZ Kernel list built with official Debian kernel config and OpenVZ Settings'''
! Kernel !! Description !! Hardware !! Debian Architecture
|-
! fzakernel-2.6.18-686
| uni- and multiprocessor
| up to 4GB of RAM
| i386
|-
! fzakernel-2.6.18-686-bigmem
| symmetric multiprocessor
| up to 64 GB of RAM
| i386
|-
! fzakernel-2.6.18-amd64
| uni- and multiprocessor
|
| amd64
|-
|}

<pre>
# apt-get install <kernel>
</pre>

===== Configuring the bootloader =====

In case GRUB is used as the boot loader, it will be configured automatically, or execute update-grub; lines similar to these will be added to the <tt>/boot/grub/menu.lst</tt> file:

<pre>
[...]
title Debian GNU/Linux, kernel 2.6.18-ovz-028stab051.1-686
root (hd0,1)
kernel /vmlinuz-2.6.18-ovz-028stab051.1-686 root=/dev/sda5 ro vga=791
initrd /initrd.img-2.6.18-ovz-028stab051.1-686
savedefault
[...]
</pre>

{{Note|per default on debian/ubuntu, a 2.6.22 kernel will boot before a 2.6.18, please check manually the grub boot order. See man update-grub for more details}}

===== Installing the user-level tools =====

OpenVZ needs some user-level tools installed. Those are:

; vzctl
: A utility to control OpenVZ containers (create, destroy, start, stop, set parameters etc.)
; vzquota
: A utility to manage quotas for containers. Mostly used indirectly (by vzctl).

<pre>
# [sudo] apt-get install vzctl vzquota
</pre>

==== 2 Using Debian lenny repositories ====

If you upgrade to lenny, you can search openvz kernel and can install with:
<pre>
apt-get install linux-image-openvz-686
</pre>
this command will install latest kernel and all required packages like:
<pre>
apt-get install iproute libatm1 linux-image-2.6.26-1-openvz-686 linux-image-openvz-686 rsync vzctl vzquota libcgroup-dev
</pre>
and will arrange grub bootloader properly.

=== Rebooting into OpenVZ kernel ===

{{Warning|Before you restart your Server, verify that your system has all needed modules enabled in order to boot your harddisk (e.g. hardware modules, raid system(s), lvm2 etc). You may need an INITRD (initramdisk) or to compile needed kernel modules statically.}}

Now reboot the machine and choose the OpenVZ Linux Kernel on the boot loader menu. If the OpenVZ kernel has been booted successfully, proceed to installing the user-level tools for OpenVZ.

=== Confirm proper installation ===

1. Kernel:
<pre>
# uname -r
2.6.26-1-openvz-686
#
</pre>

2. Openvz kernel facility:
<pre>
# ps ax | grep vz
2349 ? S 0:00 [vzmond]
</pre>

3. A network interface for containers:
<pre>
# ifconfig
venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
</pre>

== Configuring ==

=== sysctl ===

There are a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in <tt>/etc/sysctl.conf</tt> file. Here is the relevant part of the file; please edit it accordingly.

{{Note|vzctl version from debian-systs, automatically inserts these options at the last of <tt>/etc/sysctl.conf</tt>, except for net.ipv4.ip_forward}}

<pre>
[...]

# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp=0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter=1

# Enables the magic-sysrq key
kernel.sysrq=1

# TCP Explict Congestion Notification
#net.ipv4.tcp_ecn=0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects=1
net.ipv4.conf.all.send_redirects=0

[...]
</pre>

# [sudo] sysctl -p

{{Note|You can make a symlink from /var/lib/vz to /vz as backward
compatibility to OpenVZ as installed in other distributions
(Debian vz root directory is /var/lib/vz to be FHS-compliant.}}

# [sudo] ln -s /var/lib/vz /vz

=== OS templates ===

{{Note|Support of OS templates on 64 bit hosts is somewhat limited for the time being, so that not all tools or features are available - please see [[Making template tools to work on x86_64]] and [[Install OpenVZ on a x86 64 system Centos-Fedora]] for additional details and information on possible workarounds}}

To install a container, you need OS template(s).

Precreated templates can be found [http://wiki.openvz.org/Download/template/precreated here] and [http://download.openvz.org/contrib/template/precreated/ here].

You can create your own templates, see
[[Debian template creation]], [[Ubuntu Gutsy template creation]] and [[:Category: Templates]].

{{Note|Setup your prefered standard OS Template : edit the /etc/vz/vz.conf}}

# [sudo] apt-get install vzctl-ostmpl-debian-5.0-i386-minimal

== Additional User Tools ==

; vzprocps
: A set of utilities to provide system information (vzps and vztop)

; [[vzdump]]
: A utility to backup and restore container.

# [sudo] apt-get install vzprocps vzdump

On Debian squeeze, vzdump seems packaged in standard aptline. For lenny, See [[Backup_of_a_running_container_with_vzdump]]

== Secure it ==

If you want to secure your container with individual firewall rules (instead or additionally to securing the host node) then you must run iptables inside the container. This works slightly different than on a physical server. So make sure that you check that iptables rules are indeed applied as expected inside the container.

Iptables modules required by the container must be specified in the general vz.conf file or the vzXXX.conf file of the container.

Add the following line into vz.conf to activate the respective iptables modules for all containers.

IPTABLES="ip_tables ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl
ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_LOG ipt_conntrack ipt_helper ipt_state iptable_nat ip_nat_ftp ip_nat_irc ipt_TOS"

[[http://wiki.debian.org/DebianFirewall][Configure]] your iptable rules inside the container.

{{Warning|Note that iptables rules inside the container are not applied automatically as on a physical server by starting the iptables module! Follow the instructions below}}

To make sure the iptables rules are applied on a startup/reboot we'll create a new file:

nano /etc/network/if-pre-up.d/iptables

Add these lines to it:

#!/bin/bash
/sbin/iptables-restore < /etc/iptables.up.rules

The file needs to be executable so change the permissions:

chmod +x /etc/network/if-pre-up.d/iptables

Start iptables

/etc/init.d/iptables start

If the startup shows errors then you have probably not activated the needed iptables modules. See above.

Check inside the container that your iptables rules are indeed applied:

iptables -L

If the rules do not show up as you would expect on a physical server then you might not have activated the needed iptables modules.

== Start it! ==

# [sudo] /etc/init.d/vz start

This does not make the vz system automatically start at boot time. For automatic start:

# [sudo] update-rc.d vz defaults 98

== Use it! ==

After installing the OpenVZ kernel, user tools and a minimal OS template
to create a first container and do some [[basic operations in OpenVZ environment]]. Read the [[download:doc/OpenVZ-Users-Guide.pdf]], browse this wiki.

[[Category: HOWTO]]
[[Category: Debian]]
[[Category: Installation]]

Installation on Debian/old

2013-03-18T11:23:29Z

TimSmall: Note for relevance, add link to debian.org wiki page.

OpenVZ consists of a kernel, user-level tools, and container templates.

This guide tells how to install the kernel and the tools on [http://www.debian.org Debian] Etch or Lenny/Squeeze.

For Squeeze, use the Lenny directions.

{{Note|The majority of the content on this page only applies to older, unsupported Debian versions and should be archived.}}

You may also wish to check the information on [http://wiki.debian.org/OpenVz the Debian wiki].

For Etch users, this document explains how to partially upgrade to Debian Lenny and install from lenny repositories ('''use this options at your risk''').

== Requirements ==

=== Filesystems ===
It's recommended that you use a separate partition for container private
directories (by default <code>/var/lib/vz/private/<CTID></code>). The reason for this is that if you wish to use the OpenVZ per-container disk quota, you won't be able to use usual Linux disk quotas on the same partition. Bear in mind that "per-container quota" in this context includes not only pure per-container quota but also the usual Linux disk quota used in container, not on the [[HN]].

At the very least try to avoid using the root partition for containers, because the root user of a container will be able to overcome the 5% disk space barrier in some situations. If the HN root partition is completely filled, it will break the system.

OpenVZ per-container disk quota is supported only for ext2/ext3 filesystems; therefore it makes sense to use one of these filesystems (ext3 is recommended) if you need per-container disk quota.

=== Repository setup (Etch only) ===

'''If you are using Debian Lenny, this step in no longer required. Openvz kernel packages and tools are available on main repository.'''

==== 1. Using openvz.org repositories ====

At the moment two different repositories are online at http://download.openvz.org:

; by Ola Lundqvist <opal@debian.org>
: (OpenVZ kernels only)
: apt-uri http://download.openvz.org/debian

; by Thorsten Schifferdecker <tsd@debian.systs.org>
: apt-uri http://download.openvz.org/debian-systs
: (Mirror of OpenVZ Repository from http://debian.systs.org/)

{{Note|The next steps use the repository at http://download.openvz.org/debian-systs; the actual OpenVZ Tools for Debian exist only as unstable builds, see http://packages.debian.org/vzctl}}

{{Note|By default, on Ubuntu systems root tasks are executed with [https://help.ubuntu.com/community/RootSudo sudo]}}

This can be done via the following commands, as root or as privileged "sudo" user
<pre>
# echo -e "\ndeb http://download.openvz.org/debian-systs etch openvz" >> /etc/apt/sources.list
# wget -q http://download.openvz.org/debian-systs/dso_archiv_signing_key.asc -O- | apt-key add - && apt-get update
</pre>

==== 2. Using Debian repositories (upgrade to lenny) ====

There is even a '''lenny''' repository with kernel 2.6.28. '''Use it at your own risk!'''

Add lenny repositories to your '''/etc/apt/sources.list'''
<pre>
deb http://DEBIAN-MIRROR/debian/ testing main
deb http://DEBIAN-MIRROR/debian-security/ testing/updates main
</pre>

Enlarge apt-cache adding to '''/etc/apt/apt.conf''' this line:
<pre>
APT::Cache-Limit "100000000";
</pre>

Give etch package priority over lenny packages. Edit '''/etc/apt/preferences''' and set like this:
<pre>
Package: *
Pin: release a=etch
Pin-Priority: 700

Package: *
Pin: release a=lenny
Pin-Priority: 650
</pre>

Then '''apt-get update && apt-get dist-upgrade''' to upgrade to lenny.

== Kernel installation ==

=== Wheezy and Lenny ===

{{Note|The best kernel to use is [[Download/kernel/rhel6|RHEL6-based]]. Please see [[Install_kernel_from_RPM_on_Debian_6.0]]}}

=== Etch ===

==== 1. Using openvz kernel repositories ====

{{Note|In case you want to recompile the OpenVZ kernel yourself on Debian, see [[Compiling the OpenVZ kernel (the Debian way)]].}}

First, you need to choose what kernel you want to install.

{| class="wikitable"
|+'''OpenVZ Kernel list built with kernel config from http://download.openvz.org'''
! Kernel !! Description !! Hardware !! Debian Architecture
|-
! ovzkernel-2.6.18
| uniprocessor
| up to 4GB of RAM
| i386 and amd64
|-
! ovzkernel-2.6.18-smp
| symmetric multiprocessor
| up to 4 GB of RAM
| i386 and amd64
|-
! ovzkernel-2.6.18-enterprise
| SMP + PAE support + 4/4GB split
| up to 64 GB of RAM
| i386 only
|}

{| class="wikitable"
|+'''OpenVZ Kernel list built with official Debian kernel config and OpenVZ Settings'''
! Kernel !! Description !! Hardware !! Debian Architecture
|-
! fzakernel-2.6.18-686
| uni- and multiprocessor
| up to 4GB of RAM
| i386
|-
! fzakernel-2.6.18-686-bigmem
| symmetric multiprocessor
| up to 64 GB of RAM
| i386
|-
! fzakernel-2.6.18-amd64
| uni- and multiprocessor
|
| amd64
|-
|}

<pre>
# apt-get install <kernel>
</pre>

===== Configuring the bootloader =====

In case GRUB is used as the boot loader, it will be configured automatically, or execute update-grub; lines similar to these will be added to the <tt>/boot/grub/menu.lst</tt> file:

<pre>
[...]
title Debian GNU/Linux, kernel 2.6.18-ovz-028stab051.1-686
root (hd0,1)
kernel /vmlinuz-2.6.18-ovz-028stab051.1-686 root=/dev/sda5 ro vga=791
initrd /initrd.img-2.6.18-ovz-028stab051.1-686
savedefault
[...]
</pre>

{{Note|per default on debian/ubuntu, a 2.6.22 kernel will boot before a 2.6.18, please check manually the grub boot order. See man update-grub for more details}}

===== Installing the user-level tools =====

OpenVZ needs some user-level tools installed. Those are:

; vzctl
: A utility to control OpenVZ containers (create, destroy, start, stop, set parameters etc.)
; vzquota
: A utility to manage quotas for containers. Mostly used indirectly (by vzctl).

<pre>
# [sudo] apt-get install vzctl vzquota
</pre>

==== 2 Using Debian lenny repositories ====

If you upgrade to lenny, you can search openvz kernel and can install with:
<pre>
apt-get install linux-image-openvz-686
</pre>
this command will install latest kernel and all required packages like:
<pre>
apt-get install iproute libatm1 linux-image-2.6.26-1-openvz-686 linux-image-openvz-686 rsync vzctl vzquota libcgroup-dev
</pre>
and will arrange grub bootloader properly.

=== Rebooting into OpenVZ kernel ===

{{Warning|Before you restart your Server, verify that your system has all needed modules enabled in order to boot your harddisk (e.g. hardware modules, raid system(s), lvm2 etc). You may need an INITRD (initramdisk) or to compile needed kernel modules statically.}}

Now reboot the machine and choose the OpenVZ Linux Kernel on the boot loader menu. If the OpenVZ kernel has been booted successfully, proceed to installing the user-level tools for OpenVZ.

=== Confirm proper installation ===

1. Kernel:
<pre>
# uname -r
2.6.26-1-openvz-686
#
</pre>

2. Openvz kernel facility:
<pre>
# ps ax | grep vz
2349 ? S 0:00 [vzmond]
</pre>

3. A network interface for containers:
<pre>
# ifconfig
venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
</pre>

== Configuring ==

=== sysctl ===

There are a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in <tt>/etc/sysctl.conf</tt> file. Here is the relevant part of the file; please edit it accordingly.

{{Note|vzctl version from debian-systs, automatically inserts these options at the last of <tt>/etc/sysctl.conf</tt>, except for net.ipv4.ip_forward}}

<pre>
[...]

# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.default.proxy_arp=0
net.ipv4.ip_forward=1

# Enables source route verification
net.ipv4.conf.all.rp_filter=1

# Enables the magic-sysrq key
kernel.sysrq=1

# TCP Explict Congestion Notification
#net.ipv4.tcp_ecn=0

# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects=1
net.ipv4.conf.all.send_redirects=0

[...]
</pre>

# [sudo] sysctl -p

{{Note|You can make a symlink from /var/lib/vz to /vz as backward
compatibility to OpenVZ as installed in other distributions
(Debian vz root directory is /var/lib/vz to be FHS-compliant.}}

# [sudo] ln -s /var/lib/vz /vz

=== OS templates ===

{{Note|Support of OS templates on 64 bit hosts is somewhat limited for the time being, so that not all tools or features are available - please see [[Making template tools to work on x86_64]] and [[Install OpenVZ on a x86 64 system Centos-Fedora]] for additional details and information on possible workarounds}}

To install a container, you need OS template(s).

Precreated templates can be found [http://wiki.openvz.org/Download/template/precreated here] and [http://download.openvz.org/contrib/template/precreated/ here].

You can create your own templates, see
[[Debian template creation]], [[Ubuntu Gutsy template creation]] and [[:Category: Templates]].

{{Note|Setup your prefered standard OS Template : edit the /etc/vz/vz.conf}}

# [sudo] apt-get install vzctl-ostmpl-debian-5.0-i386-minimal

== Additional User Tools ==

; vzprocps
: A set of utilities to provide system information (vzps and vztop)

; [[vzdump]]
: A utility to backup and restore container.

# [sudo] apt-get install vzprocps vzdump

On Debian squeeze, vzdump seems packaged in standard aptline. For lenny, See [[Backup_of_a_running_container_with_vzdump]]

== Secure it ==

If you want to secure your container with individual firewall rules (instead or additionally to securing the host node) then you must run iptables inside the container. This works slightly different than on a physical server. So make sure that you check that iptables rules are indeed applied as expected inside the container.

Iptables modules required by the container must be specified in the general vz.conf file or the vzXXX.conf file of the container.

Add the following line into vz.conf to activate the respective iptables modules for all containers.

IPTABLES="ip_tables ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl
ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_LOG ipt_conntrack ipt_helper ipt_state iptable_nat ip_nat_ftp ip_nat_irc ipt_TOS"

[[http://wiki.debian.org/DebianFirewall][Configure]] your iptable rules inside the container.

{{Warning|Note that iptables rules inside the container are not applied automatically as on a physical server by starting the iptables module! Follow the instructions below}}

To make sure the iptables rules are applied on a startup/reboot we'll create a new file:

nano /etc/network/if-pre-up.d/iptables

Add these lines to it:

#!/bin/bash
/sbin/iptables-restore < /etc/iptables.up.rules

The file needs to be executable so change the permissions:

chmod +x /etc/network/if-pre-up.d/iptables

Start iptables

/etc/init.d/iptables start

If the startup shows errors then you have probably not activated the needed iptables modules. See above.

Check inside the container that your iptables rules are indeed applied:

iptables -L

If the rules do not show up as you would expect on a physical server then you might not have activated the needed iptables modules.

== Start it! ==

# [sudo] /etc/init.d/vz start

This does not make the vz system automatically start at boot time. For automatic start:

# [sudo] update-rc.d vz defaults 98

== Use it! ==

After installing the OpenVZ kernel, user tools and a minimal OS template
to create a first container and do some [[basic operations in OpenVZ environment]]. Read the [[download:doc/OpenVZ-Users-Guide.pdf]], browse this wiki.

[[Category: HOWTO]]
[[Category: Debian]]
[[Category: Installation]]

Monitoring openvz resources using munin

2010-12-13T13:09:29Z

TimSmall: Add openvzcpu plugin from Munin Exchange

[[Category: Monitoring]]

[http://exchange.munin-monitoring.org/plugins/openvzcpu/details There is a plugin available on the Munin Exchange web site] which tracks CPU usage accross different containers.

[http://exchange.munin-monitoring.org/plugins/openvz_/details A second is also available via the Munin plugin exchange] which tracks beancounter values.

Additionally, there are several plugins available on this page to monitor beancounter values in [http://munin.projects.linpro.no/ Munin].
The third one tries to combine the other two. (FIXME, maybe these should live in the Munin Exchange too?)

= "Simple" munin plugin =

The plugin listed below grabs all the beancounters' values.

<source lang="bash">
#!/bin/sh
#
# plugin to monitor OpenVZ bean counters.
#
#
#%# family=auto
#%# capabilities=autoconf suggest

ATTR=`basename $0 | sed 's/^vebc_//g'`

if [ "$1" = "autoconf" ]; then
if [ -r /proc/bc/0/resources ]; then
echo yes
exit 0
else
echo "no (/proc/bc/0/resources not found)"
exit 1
fi
fi

if [ "$1" = "suggest" ]; then
if [ -r /proc/bc/0/resources ]; then
cat /proc/bc/0/resources |
while read str; do
vals=($str)
echo ${vals[0]}
done
exit 0
else
exit 1
fi
fi

if [ "$1" = "config" ]; then
# echo "graph_order down up"
echo "graph_title $ATTR beancounter for containers"
echo 'graph_category system'
echo "graph_info 'Containers beancounters info'"

for CTID in `ls -d1 /proc/bc/???`; do
id=`basename $CTID`
grep $ATTR $CTID/resources |
while read str; do

vals=($str)
name=${vals[0]}
echo ${id}.label $id
echo "${id}.warning ${vals[3]}"
echo "${id}.critical ${vals[4]}"
done
done

exit 0
fi;

for CTID in `ls -d1 /proc/bc/???`; do
id=`basename $CTID`
grep $ATTR $CTID/resources |
while read str; do
vals=($str)
name=${vals[0]}
echo "$id.value ${vals[1]}"
done
done
</source>

= Extended Version for old system using user_beancounter =

Put it with the munin plugins and make a link for every
graph which should be produced named like:

vebc_VALUENAME1_VALUENAME2_..._CTID

e.g.: vebc_numflock_numpty_numsiginfo_101

<source lang="bash">
#!/bin/sh
#
# plugin to monitor OpenVZ bean counters.
#
#
#%# family=auto
#%# capabilities=autoconf suggest

ATTR=`basename $0 | sed -e 's/^vebc_.*_//'`
STATS=`basename $0 | sed -e 's/^vebc_//' -e 's/_[0-9]*$//' -e 's/_/ /g'`

if [ "$1" = "autoconf" ]; then
if [ -r /proc/bc/0/resources ]; then
echo yes
exit 0
else
echo "no (/proc/bc/0/resources not found)"
exit 1
fi
fi

if [ "$1" = "suggest" ]; then
if [ -r /proc/bc/0/resources ]; then
cat /proc/bc/0/resources |
while read str; do
vals=($str)
echo ${vals[0]}
done
exit 0
else
exit 1
fi
fi

if [ "$1" = "config" ]; then
# echo "graph_order down up"
echo "graph_title beancounter for CT$ATTR: $STATS"
echo "graph_category CT$ATTR"
echo "graph_info 'Container bean counters info'"

readme="false"
cat /proc/user_beancounters | while read myid stuff; do
line=""
if [ "$myid" == "$ATTR:" ]; then
readme="true"
line="$stuff"
echo $line
else
loid=`echo $myid | sed -e 's/.*:/:/'`
if [ "$loid" == ":" ]; then
readme="false"
fi
if [ "$readme" == "true" ]; then
line="$myid $stuff"
echo $line
fi
fi
done | while read name value top warn max; do
okname="dummy"
for statname in $STATS; do
if [ "$name" == "$statname" ]; then
okname=$name
fi
done

if [ "$okname" != "dummy" ]; then
echo $okname.label $name
echo $okname.warning $warn
echo $okname.critical $max
fi
done
exit 0
fi;

readme="false"
cat /proc/user_beancounters | while read myid stuff; do
line=""
if [ "$myid" == "$ATTR:" ]; then
readme="true"
line="$stuff"
echo $line
else
loid=`echo $myid | sed -e 's/.*:/:/'`
if [ "$loid" == ":" ]; then
readme="false"
fi
if [ "$readme" == "true" ]; then
line="$myid $stuff"
echo $line
fi
fi
done | while read name value x; do
okname="dummy"
for statname in $STATS; do
if [ "$name" == "$statname" ]; then
okname=$name
fi
done
if [ "$okname" != "dummy" ]; then
echo $okname.value $value
fi

done
</source>

This is not too performant but should do and the graphs are much
more readable then the first solution.

= Extended plugin from Jan Tomasek =
* Jan has posted another plugin on http://forum.openvz.org/index.php?t=msg&goto=15122, where I've fixed two things:
*# "exit 0" in the "config" block
*# Replaced "vals=($str); echo ${vals[0]}" with "echo ${str%% *}" (the former was causing problems I don't remember anymore)
* v1.3.2 (2008/08/09)
*# If only 1 variable is graphed, also display maxheld, barrier and limit
<source lang="bash">
#!/bin/sh
#
# Munin's plugin to monitor OpenVZ bean counters.
#
# $Log$
# Revision 1.3 2007/07/19 12:57:00 Jan Tomasek <jan@tomasek.cz>
# * rewrited to work with /proc/bc/<VEID>/resources instead of
# /proc/user_beancounters, that simplified code and result
# is also bit faster.
# * added references to OpenVZ wiki
# Revision 1.3.1 2008/05/13 01:26:00 Daniel Hahler <http://daniel.hahler.de/>
# * Minor fixes
# - "exit 0" in "config" block
# - Use "echo ${str%% *}" in "suggest", instead of "vals=($str); echo ${vals[0]}"
#
# Revision 1.3.2 2008/08/09 12:30:00 Christian Rubbert <crubbert@xrc.de>
# * Feature
# - If only 1 variable is graphed, also display maxheld, barrier and limit
#
# Original revision taken from:
# http://wiki.openvz.org/Monitoring_openvz_resources_using_munin
#
#%# family=auto
#%# capabilities=autoconf suggest

VEID=`basename $0 | sed -e 's/^vebc_.*_//'`;
STATS=`basename $0 | sed -e 's/^vebc_//' -e 's/_[0-9]*$//' -e 's/_/ /g'`
x=0; STATSCNT=`for i in $STATS; do x=$[$x+1]; done; echo $x`

if [ "$1" = "autoconf" ]; then
if [ -r /proc/bc/0/resources ]; then
echo yes
exit 0
else
echo "no (/proc/bc/0/resources not found)"
exit 1
fi
fi

if [ "$1" = "suggest" ]; then
if [ -r /proc/bc/0/resources ]; then
cat /proc/bc/0/resources | while read str; do
# Print everything before " "
echo ${str%% *}
done
exit 0
else
exit 1
fi
fi

if [ ! -f /proc/bc/$VEID/resources ]; then
exit 0;
fi

if [ "$1" = "config" ]; then
#echo "graph_order down up"
echo "graph_title VE$VEID: $STATS"
echo "graph_vlabel bean counters"
echo "graph_category VE$VEID"

# Note on URLs. General graph info is by munin version 1.2.5
# accepted even with HTML code. But for value.info it escapes URL,
# I expect that authors of munin will note that in future and put
# escaping even for graph.info.
echo "graph_info VE bean counters info. Documentation of the OpenVZ resource management is located at <a href=\"
http://wiki.openvz.org/UBC\">http://wiki.openvz.org/UBC</a>."

cat /proc/bc/$VEID/resources | while read name value top warn max stuff ; do
for statname in $STATS; do
if [ "$name" = "$statname" ]; then
URL="http://wiki.openvz.org/$name"
if [ "$warn" = "0" ]; then
warn=$max
fi
echo $name.label $name
echo $name.warning $warn
echo $name.critical $max
echo $name.info Description of this resource is located at $URL
fi
done
done

if [ "$STATSCNT" == "1" ]; then
echo maxheld.label Maxheld
echo maxheld.draw LINE2
echo maxheld.info Maximum value
echo barrier.label Barrier
echo barrier.draw LINE2
echo barrier.info Barrier
echo limit.label Limit
echo limit.draw LINE2
echo limit.info Limit
fi
exit 0
fi;

cat /proc/bc/$VEID/resources | while read name value top warn max stuff ; do
for statname in $STATS; do
if [ "$name" = "$statname" ]; then
echo $name".value "$value;

if [ "$STATSCNT" == "1" ]; then
echo maxheld.value $top
echo barrier.value $warn
echo limit.value $max
fi
fi
done
done</source>

= Munin plugin setup =
== Run as root ==
Please note, you have to configure plugin to run as root. Therefore, add the following to /etc/munin/plugin-conf.d/ somewhere:
[vebc*]
user root

== Installing the plugins ==
There's a single plugin file, which can be installed several times and can put several values into the same graph.
You should install the plugin from above to e.g. <code>/usr/local/share/munin/plugins/vebc_</code> and then put symlinks to there from /etc/munin/plugins.
The following script allows you to handle this easily:
To install this, you can use the following script:
<source lang="bash">
#!/bin/bash

FILE=`mktemp /tmp/ln-vebc-XXXXXX`

cd /etc/munin/plugins

for resources in kmemsize \
lockedpages_privvmpages_shmpages_physpages_vmguarpages_oomguarpages \
numproc \
numtcpsock_numflock_numpty_numsiginfo_numothersock_numiptent \
tcpsndbuf_tcprcvbuf_othersockbuf_dgramrcvbuf \
dcachesize \
numfile
do
for VE in 0 `/usr/sbin/vzlist | sed "s/^ *//" |grep '^[0-9]' | cut -f 1 -d " "` ; do
ln -sf /usr/local/share/munin/plugins/vebc_ "vebc_"$resources"_"$VE
echo "vebc_"$resources"_"$VE >> $FILE
done
done

# remove no longer deserved links (ie. links pointing to machines
# which were destroyed or stoped)

find -type l -name vebc_\* | sed "s/\.\///" | while read LN; do
if grep ^$LN$ $FILE >/dev/null; then
true
else
rm $LN
fi
done

rm $FILE
</source>

= Alternative: Using vzlist =
This simple script will generate an overview of the requested stat from all VE's.
When using vzlist your saved from any changes to the beancounters.

== Usage ==
Simply append the variable you want to monitor:
<source lang="bash">ln -s /usr/share/munin/plugins/openvz_ /etc/munin/plugins/openvz_physpages</source>

<source lang="bash">
#!/bin/sh
#
# Munin's plugin to monitor OpenVZ bean counters.
#
# $Log$
# 2008/08/14 Rene Weselowski <http://www.dead.at>
#
#%# family=auto
#%# capabilities=autoconf

ATTRIBUTE=`basename $0 | sed 's/^openvz_//g'`

if [ "$1" = "autoconf" ]; then
echo yes
exit 0
fi

if [ "$1" = "config" ]; then
echo "graph_title $ATTRIBUTE"
echo "graph_args --base 1000 -l 0"
echo "graph_scale yes"
echo "graph_vlabel $ATTRIBUTE Value"
echo "graph_category openvz"
echo "graph_info This graph shows OpenVZ: $ATTRIBUTE"
vzlist -a -H -o hostname | awk '{gsub(/\./,"_",$1)
print("'$ATTRIBUTE'"$1".label "$1"\n" \
"'$ATTRIBUTE'"$1".info '$ATTRIBUTE' for VE"$1)}'
exit 0
fi

vzlist -a -H -o hostname,$ATTRIBUTE | awk '{gsub(/\./,"_",$1)
print("'$ATTRIBUTE'"$1".value "$2)}'

</source>

Physical to container

2010-10-29T10:40:21Z

TimSmall: /* Using udev anyway */ /dev/random was missing

A rough description of how to migrate existing physical server into a [[container]].

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

To make things easier you may like to first follow the basic instructions elsewhere and create a dummy container based on the same Linux distribution you want to migrate. That way you can take that dummy as a template and then copy to your new migrated container and modify. You can later discard this dummy.

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

Hint: Now comes the dummy container handy mentioned above: Simply copy the xxx.conf file of the dummy to your new yyy.conf and modify it.

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude=/dev --exclude=/proc --exclude=/tmp -e ssh root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

{{Note|Do decrease the downtime, you can use double rsync approach. Run rsync for the first time before stopping most of the services, and then for the second time after stopping services. That way most of the data will be transferred while your server is fully working, and the second rsync will just "catch the latest changes" which is faster.}}

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar --numeric-owner -cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

If you copied from the dummy container then this step is already accomplished.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e 's/^[0-9].*getty.*tty/#&/g' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

ln -sf /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the lines for <code>/dev/pts</code>, <code>/proc</code>, <code>/sys</code> and such):

mv /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
egrep '/dev/pts|/dev/shm|/proc|/sys' /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

For Suse 11.0, It is found in /etc/init.d/boot

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

Hint:
Now comes the dummy container handy mentioned above: Simply copy the entire /dev directory of the dummy to your new migrated container - worked in my case at least with Debian Etch.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

==== Using udev anyway ====
CentOS 5 can run in a container with udev enabled. You need to create /etc/udev/devices, containing the above device nodes. Also, the following will create the extra device nodes you need
mkdir /vz/private/123/etc/udev/devices
/sbin/MAKEDEV -d /vz/private/123/dev {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx random urandom zero ram0
/sbin/MAKEDEV -d /vz/private/123/etc/udev/devices {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx random urandom zero ram0

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig SERVICENAME off </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

If the files /vz/private/{CTID}/etc/sysconfig/network-scripts/ifdown-venet or
/vz/private/{CTID}/etc/sysconfig/network-scripts/ifup-venet exist, make sure they won't be used. These two files might exist if the physical server had OpenVZ installed. One way to do this is to rename them, like so:
mv ifdown-venet SKIP.ifdown-venet

Failing to do this will prevent networking from starting up correctly in the container.

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

===== Ubuntu server 8.x =====

Here what I have done for my Ubuntu server JEOS 8.04.2

<pre>
rm /vz/private/123/etc/network/if-up.d/ntpdate
rm /vz/private/123/etc/event.d/tty{1,2,3,4,5,6}
vzctl exec 123 update-rc.d -f klogd remove
vzctl exec 123 update-rc.d -f udev remove

</pre>

==== openSUSE/SLES ====

Use Yast.

=== Disable udev if you create DEVNODES devices ===

If you are creating devices for the container with a DEVNODES statement in a veid.conf file then these devices may be overwritten/deleted by udev when the container starts. As udev cannot "see" the device from within the container it disables it. Therefore, if you have DEVNODES statements in veid.conf then disable udev.

In Fedora, Redhat, Centos, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
Comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 1777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success stories ==
{{Note|please add your line to the bottom of this list, and do not forget to sign it using <code><nowiki>--~~~~</nowiki></code>}}

* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]] 16:16, 23 March 2007 (EST)
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]] 23:38, 26 February 2008 (EST)
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]] 17:35, 10 March 2008 (EST)
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]] 19:46, 10 March 2008 (EST)
* Debian 1:3.3.5-13 with apache2, PHP, etc. --[[User:Spawrks|spawrks]] 23:36, 10 April 2008 (EST)
* Debian Etch with apache2, MySQL, etc. --[[User:Zhafrance|zhafrance]] 16:29, 20 April 2008 (EST)
* Debian Etch i386 with apache2, MySQL, etc. --[[User:geejay|geejay]] 17:29, 26 May 2008 (GMT)
* Centos 4.6 with apache2, MySQL, Qmail etc. --[[User:Bharathchari|Bharathchari]] 08:06, 13 June 2008 (EDT)
* Centos 4.6 with cPanel/WHM (Apache2, Mysql, Exim, etc) --[[User:Zccopwrx|Zccopwrx]] 08:16, 30 July 2008 (EDT)
* SlackWare 10.1 (Qmail) --[[User:defiancenl|defiancenl]]
* SlackWare 10.0 (Qmail) --[[User:defiancenl|defiancenl]]
* Ubuntu 8.04.3 LTS JEOS (Apache2, Mysql) --[[User:bougui|bougui]] Fri Aug 28 10:40:41 EDT 2009
* CentOS 5.3 (Apache2, Mysql, Cacti) --[[User:kofl|kofl]] September 12 2009
* Scientific Linux 3.0.9 (Macrovision FLEXlm) {{unsigned|137.226.90.94|11:34, 4 November 2009}}
* Red Hat Enterprise Linux 4 (rhel4) --[[User:Bpuklich|Bpuklich]] 17:20, 15 February 2010 (UTC)
* Debian SID up-to-date with apache2, MySQL, posgrey etc. --nyquist 14:04, 06 July 2010 (UTC)
* Centos 5.x with Plesk -- 05:33, 17 August 2010 (UTC)
* Redhat 4 -- 20:32, 18 August 2010 (UTC)
* Fedora 4 -- 15:06, 20 August 2010 (UTC)
* Fedora 9 x64 with FDS and samba PDC --burn 23:20 10 October 2010
* Fedora 3 x32 with Plesk -- 23 October 2010 --[[User:Rexwickham|Rex Wickham (2020media.com)]] 13:15, 23 October 2010 (UTC)
[[Category:HOWTO]]

Physical to container

2010-10-29T10:37:48Z

TimSmall: /* /etc/init.d services */ Fix chkconfig invocation - previous invocation would result in the service being re-enabled when the package was upgraded (e.g. security fix etc.)....

A rough description of how to migrate existing physical server into a [[container]].

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

To make things easier you may like to first follow the basic instructions elsewhere and create a dummy container based on the same Linux distribution you want to migrate. That way you can take that dummy as a template and then copy to your new migrated container and modify. You can later discard this dummy.

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

Hint: Now comes the dummy container handy mentioned above: Simply copy the xxx.conf file of the dummy to your new yyy.conf and modify it.

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude=/dev --exclude=/proc --exclude=/tmp -e ssh root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

{{Note|Do decrease the downtime, you can use double rsync approach. Run rsync for the first time before stopping most of the services, and then for the second time after stopping services. That way most of the data will be transferred while your server is fully working, and the second rsync will just "catch the latest changes" which is faster.}}

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar --numeric-owner -cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

If you copied from the dummy container then this step is already accomplished.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e 's/^[0-9].*getty.*tty/#&/g' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

ln -sf /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the lines for <code>/dev/pts</code>, <code>/proc</code>, <code>/sys</code> and such):

mv /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
egrep '/dev/pts|/dev/shm|/proc|/sys' /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

For Suse 11.0, It is found in /etc/init.d/boot

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

Hint:
Now comes the dummy container handy mentioned above: Simply copy the entire /dev directory of the dummy to your new migrated container - worked in my case at least with Debian Etch.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

==== Using udev anyway ====
CentOS 5 can run in a container with udev enabled. You need to create /etc/udev/devices, containing the above device nodes. Also, the following will create the extra device nodes you need
mkdir /vz/private/123/etc/udev/devices
/sbin/MAKEDEV -d /vz/private/123/dev {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0
/sbin/MAKEDEV -d /vz/private/123/etc/udev/devices {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig SERVICENAME off </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

If the files /vz/private/{CTID}/etc/sysconfig/network-scripts/ifdown-venet or
/vz/private/{CTID}/etc/sysconfig/network-scripts/ifup-venet exist, make sure they won't be used. These two files might exist if the physical server had OpenVZ installed. One way to do this is to rename them, like so:
mv ifdown-venet SKIP.ifdown-venet

Failing to do this will prevent networking from starting up correctly in the container.

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

===== Ubuntu server 8.x =====

Here what I have done for my Ubuntu server JEOS 8.04.2

<pre>
rm /vz/private/123/etc/network/if-up.d/ntpdate
rm /vz/private/123/etc/event.d/tty{1,2,3,4,5,6}
vzctl exec 123 update-rc.d -f klogd remove
vzctl exec 123 update-rc.d -f udev remove

</pre>

==== openSUSE/SLES ====

Use Yast.

=== Disable udev if you create DEVNODES devices ===

If you are creating devices for the container with a DEVNODES statement in a veid.conf file then these devices may be overwritten/deleted by udev when the container starts. As udev cannot "see" the device from within the container it disables it. Therefore, if you have DEVNODES statements in veid.conf then disable udev.

In Fedora, Redhat, Centos, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
Comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 1777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success stories ==
{{Note|please add your line to the bottom of this list, and do not forget to sign it using <code><nowiki>--~~~~</nowiki></code>}}

* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]] 16:16, 23 March 2007 (EST)
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]] 23:38, 26 February 2008 (EST)
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]] 17:35, 10 March 2008 (EST)
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]] 19:46, 10 March 2008 (EST)
* Debian 1:3.3.5-13 with apache2, PHP, etc. --[[User:Spawrks|spawrks]] 23:36, 10 April 2008 (EST)
* Debian Etch with apache2, MySQL, etc. --[[User:Zhafrance|zhafrance]] 16:29, 20 April 2008 (EST)
* Debian Etch i386 with apache2, MySQL, etc. --[[User:geejay|geejay]] 17:29, 26 May 2008 (GMT)
* Centos 4.6 with apache2, MySQL, Qmail etc. --[[User:Bharathchari|Bharathchari]] 08:06, 13 June 2008 (EDT)
* Centos 4.6 with cPanel/WHM (Apache2, Mysql, Exim, etc) --[[User:Zccopwrx|Zccopwrx]] 08:16, 30 July 2008 (EDT)
* SlackWare 10.1 (Qmail) --[[User:defiancenl|defiancenl]]
* SlackWare 10.0 (Qmail) --[[User:defiancenl|defiancenl]]
* Ubuntu 8.04.3 LTS JEOS (Apache2, Mysql) --[[User:bougui|bougui]] Fri Aug 28 10:40:41 EDT 2009
* CentOS 5.3 (Apache2, Mysql, Cacti) --[[User:kofl|kofl]] September 12 2009
* Scientific Linux 3.0.9 (Macrovision FLEXlm) {{unsigned|137.226.90.94|11:34, 4 November 2009}}
* Red Hat Enterprise Linux 4 (rhel4) --[[User:Bpuklich|Bpuklich]] 17:20, 15 February 2010 (UTC)
* Debian SID up-to-date with apache2, MySQL, posgrey etc. --nyquist 14:04, 06 July 2010 (UTC)
* Centos 5.x with Plesk -- 05:33, 17 August 2010 (UTC)
* Redhat 4 -- 20:32, 18 August 2010 (UTC)
* Fedora 4 -- 15:06, 20 August 2010 (UTC)
* Fedora 9 x64 with FDS and samba PDC --burn 23:20 10 October 2010
* Fedora 3 x32 with Plesk -- 23 October 2010 --[[User:Rexwickham|Rex Wickham (2020media.com)]] 13:15, 23 October 2010 (UTC)
[[Category:HOWTO]]

Physical to container

2010-10-28T16:11:02Z

TimSmall: /* /etc/fstab */

A rough description of how to migrate existing physical server into a [[container]].

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

To make things easier you may like to first follow the basic instructions elsewhere and create a dummy container based on the same Linux distribution you want to migrate. That way you can take that dummy as a template and then copy to your new migrated container and modify. You can later discard this dummy.

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

Hint: Now comes the dummy container handy mentioned above: Simply copy the xxx.conf file of the dummy to your new yyy.conf and modify it.

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude=/dev --exclude=/proc --exclude=/tmp -e ssh root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

{{Note|Do decrease the downtime, you can use double rsync approach. Run rsync for the first time before stopping most of the services, and then for the second time after stopping services. That way most of the data will be transferred while your server is fully working, and the second rsync will just "catch the latest changes" which is faster.}}

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar --numeric-owner -cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

If you copied from the dummy container then this step is already accomplished.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e 's/^[0-9].*getty.*tty/#&/g' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

ln -sf /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the lines for <code>/dev/pts</code>, <code>/proc</code>, <code>/sys</code> and such):

mv /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
egrep '/dev/pts|/dev/shm|/proc|/sys' /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

For Suse 11.0, It is found in /etc/init.d/boot

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

Hint:
Now comes the dummy container handy mentioned above: Simply copy the entire /dev directory of the dummy to your new migrated container - worked in my case at least with Debian Etch.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

==== Using udev anyway ====
CentOS 5 can run in a container with udev enabled. You need to create /etc/udev/devices, containing the above device nodes. Also, the following will create the extra device nodes you need
mkdir /vz/private/123/etc/udev/devices
/sbin/MAKEDEV -d /vz/private/123/dev {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0
/sbin/MAKEDEV -d /vz/private/123/etc/udev/devices {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

If the files /vz/private/{CTID}/etc/sysconfig/network-scripts/ifdown-venet or
/vz/private/{CTID}/etc/sysconfig/network-scripts/ifup-venet exist, make sure they won't be used. These two files might exist if the physical server had OpenVZ installed. One way to do this is to rename them, like so:
mv ifdown-venet SKIP.ifdown-venet

Failing to do this will prevent networking from starting up correctly in the container.

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

===== Ubuntu server 8.x =====

Here what I have done for my Ubuntu server JEOS 8.04.2

<pre>
rm /vz/private/123/etc/network/if-up.d/ntpdate
rm /vz/private/123/etc/event.d/tty{1,2,3,4,5,6}
vzctl exec 123 update-rc.d -f klogd remove
vzctl exec 123 update-rc.d -f udev remove

</pre>

==== openSUSE/SLES ====

Use Yast.

=== Disable udev if you create DEVNODES devices ===

If you are creating devices for the container with a DEVNODES statement in a veid.conf file then these devices may be overwritten/deleted by udev when the container starts. As udev cannot "see" the device from within the container it disables it. Therefore, if you have DEVNODES statements in veid.conf then disable udev.

In Fedora, Redhat, Centos, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
Comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 1777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success stories ==
{{Note|please add your line to the bottom of this list, and do not forget to sign it using <code><nowiki>--~~~~</nowiki></code>}}

* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]] 16:16, 23 March 2007 (EST)
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]] 23:38, 26 February 2008 (EST)
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]] 17:35, 10 March 2008 (EST)
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]] 19:46, 10 March 2008 (EST)
* Debian 1:3.3.5-13 with apache2, PHP, etc. --[[User:Spawrks|spawrks]] 23:36, 10 April 2008 (EST)
* Debian Etch with apache2, MySQL, etc. --[[User:Zhafrance|zhafrance]] 16:29, 20 April 2008 (EST)
* Debian Etch i386 with apache2, MySQL, etc. --[[User:geejay|geejay]] 17:29, 26 May 2008 (GMT)
* Centos 4.6 with apache2, MySQL, Qmail etc. --[[User:Bharathchari|Bharathchari]] 08:06, 13 June 2008 (EDT)
* Centos 4.6 with cPanel/WHM (Apache2, Mysql, Exim, etc) --[[User:Zccopwrx|Zccopwrx]] 08:16, 30 July 2008 (EDT)
* SlackWare 10.1 (Qmail) --[[User:defiancenl|defiancenl]]
* SlackWare 10.0 (Qmail) --[[User:defiancenl|defiancenl]]
* Ubuntu 8.04.3 LTS JEOS (Apache2, Mysql) --[[User:bougui|bougui]] Fri Aug 28 10:40:41 EDT 2009
* CentOS 5.3 (Apache2, Mysql, Cacti) --[[User:kofl|kofl]] September 12 2009
* Scientific Linux 3.0.9 (Macrovision FLEXlm) {{unsigned|137.226.90.94|11:34, 4 November 2009}}
* Red Hat Enterprise Linux 4 (rhel4) --[[User:Bpuklich|Bpuklich]] 17:20, 15 February 2010 (UTC)
* Debian SID up-to-date with apache2, MySQL, posgrey etc. --nyquist 14:04, 06 July 2010 (UTC)
* Centos 5.x with Plesk -- 05:33, 17 August 2010 (UTC)
* Redhat 4 -- 20:32, 18 August 2010 (UTC)
* Fedora 4 -- 15:06, 20 August 2010 (UTC)
* Fedora 9 x64 with FDS and samba PDC --burn 23:20 10 October 2010
* Fedora 3 x32 with Plesk -- 23 October 2010 --[[User:Rexwickham|Rex Wickham (2020media.com)]] 13:15, 23 October 2010 (UTC)
[[Category:HOWTO]]

Physical to container

2010-10-28T16:09:53Z

TimSmall: /* /etc/mtab */ Tidy, remove useless rm

A rough description of how to migrate existing physical server into a [[container]].

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

To make things easier you may like to first follow the basic instructions elsewhere and create a dummy container based on the same Linux distribution you want to migrate. That way you can take that dummy as a template and then copy to your new migrated container and modify. You can later discard this dummy.

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

Hint: Now comes the dummy container handy mentioned above: Simply copy the xxx.conf file of the dummy to your new yyy.conf and modify it.

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude=/dev --exclude=/proc --exclude=/tmp -e ssh root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

{{Note|Do decrease the downtime, you can use double rsync approach. Run rsync for the first time before stopping most of the services, and then for the second time after stopping services. That way most of the data will be transferred while your server is fully working, and the second rsync will just "catch the latest changes" which is faster.}}

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar --numeric-owner -cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

If you copied from the dummy container then this step is already accomplished.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e 's/^[0-9].*getty.*tty/#&/g' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

ln -sf /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the lines for <code>/dev/pts</code>, <code>/proc</code>, <code>/sys</code> and such):

cp /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
egrep '/dev/pts|/dev/shm|/proc|/sys' /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

For Suse 11.0, It is found in /etc/init.d/boot

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

Hint:
Now comes the dummy container handy mentioned above: Simply copy the entire /dev directory of the dummy to your new migrated container - worked in my case at least with Debian Etch.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

==== Using udev anyway ====
CentOS 5 can run in a container with udev enabled. You need to create /etc/udev/devices, containing the above device nodes. Also, the following will create the extra device nodes you need
mkdir /vz/private/123/etc/udev/devices
/sbin/MAKEDEV -d /vz/private/123/dev {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0
/sbin/MAKEDEV -d /vz/private/123/etc/udev/devices {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

If the files /vz/private/{CTID}/etc/sysconfig/network-scripts/ifdown-venet or
/vz/private/{CTID}/etc/sysconfig/network-scripts/ifup-venet exist, make sure they won't be used. These two files might exist if the physical server had OpenVZ installed. One way to do this is to rename them, like so:
mv ifdown-venet SKIP.ifdown-venet

Failing to do this will prevent networking from starting up correctly in the container.

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

===== Ubuntu server 8.x =====

Here what I have done for my Ubuntu server JEOS 8.04.2

<pre>
rm /vz/private/123/etc/network/if-up.d/ntpdate
rm /vz/private/123/etc/event.d/tty{1,2,3,4,5,6}
vzctl exec 123 update-rc.d -f klogd remove
vzctl exec 123 update-rc.d -f udev remove

</pre>

==== openSUSE/SLES ====

Use Yast.

=== Disable udev if you create DEVNODES devices ===

If you are creating devices for the container with a DEVNODES statement in a veid.conf file then these devices may be overwritten/deleted by udev when the container starts. As udev cannot "see" the device from within the container it disables it. Therefore, if you have DEVNODES statements in veid.conf then disable udev.

In Fedora, Redhat, Centos, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
Comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 1777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success stories ==
{{Note|please add your line to the bottom of this list, and do not forget to sign it using <code><nowiki>--~~~~</nowiki></code>}}

* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]] 16:16, 23 March 2007 (EST)
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]] 23:38, 26 February 2008 (EST)
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]] 17:35, 10 March 2008 (EST)
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]] 19:46, 10 March 2008 (EST)
* Debian 1:3.3.5-13 with apache2, PHP, etc. --[[User:Spawrks|spawrks]] 23:36, 10 April 2008 (EST)
* Debian Etch with apache2, MySQL, etc. --[[User:Zhafrance|zhafrance]] 16:29, 20 April 2008 (EST)
* Debian Etch i386 with apache2, MySQL, etc. --[[User:geejay|geejay]] 17:29, 26 May 2008 (GMT)
* Centos 4.6 with apache2, MySQL, Qmail etc. --[[User:Bharathchari|Bharathchari]] 08:06, 13 June 2008 (EDT)
* Centos 4.6 with cPanel/WHM (Apache2, Mysql, Exim, etc) --[[User:Zccopwrx|Zccopwrx]] 08:16, 30 July 2008 (EDT)
* SlackWare 10.1 (Qmail) --[[User:defiancenl|defiancenl]]
* SlackWare 10.0 (Qmail) --[[User:defiancenl|defiancenl]]
* Ubuntu 8.04.3 LTS JEOS (Apache2, Mysql) --[[User:bougui|bougui]] Fri Aug 28 10:40:41 EDT 2009
* CentOS 5.3 (Apache2, Mysql, Cacti) --[[User:kofl|kofl]] September 12 2009
* Scientific Linux 3.0.9 (Macrovision FLEXlm) {{unsigned|137.226.90.94|11:34, 4 November 2009}}
* Red Hat Enterprise Linux 4 (rhel4) --[[User:Bpuklich|Bpuklich]] 17:20, 15 February 2010 (UTC)
* Debian SID up-to-date with apache2, MySQL, posgrey etc. --nyquist 14:04, 06 July 2010 (UTC)
* Centos 5.x with Plesk -- 05:33, 17 August 2010 (UTC)
* Redhat 4 -- 20:32, 18 August 2010 (UTC)
* Fedora 4 -- 15:06, 20 August 2010 (UTC)
* Fedora 9 x64 with FDS and samba PDC --burn 23:20 10 October 2010
* Fedora 3 x32 with Plesk -- 23 October 2010 --[[User:Rexwickham|Rex Wickham (2020media.com)]] 13:15, 23 October 2010 (UTC)
[[Category:HOWTO]]

Physical to container

2010-10-28T16:08:33Z

TimSmall: /* /etc/inittab */ Change sed to comment out, not nuke in case you ever want to reverse etc.

A rough description of how to migrate existing physical server into a [[container]].

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

To make things easier you may like to first follow the basic instructions elsewhere and create a dummy container based on the same Linux distribution you want to migrate. That way you can take that dummy as a template and then copy to your new migrated container and modify. You can later discard this dummy.

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

Hint: Now comes the dummy container handy mentioned above: Simply copy the xxx.conf file of the dummy to your new yyy.conf and modify it.

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude=/dev --exclude=/proc --exclude=/tmp -e ssh root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

{{Note|Do decrease the downtime, you can use double rsync approach. Run rsync for the first time before stopping most of the services, and then for the second time after stopping services. That way most of the data will be transferred while your server is fully working, and the second rsync will just "catch the latest changes" which is faster.}}

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar --numeric-owner -cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

If you copied from the dummy container then this step is already accomplished.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e 's/^[0-9].*getty.*tty/#&/g' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

rm -f /vz/private/123/etc/mtab
ln -s /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the lines for <code>/dev/pts</code>, <code>/proc</code>, <code>/sys</code> and such):

cp /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
egrep '/dev/pts|/dev/shm|/proc|/sys' /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

For Suse 11.0, It is found in /etc/init.d/boot

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

Hint:
Now comes the dummy container handy mentioned above: Simply copy the entire /dev directory of the dummy to your new migrated container - worked in my case at least with Debian Etch.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

==== Using udev anyway ====
CentOS 5 can run in a container with udev enabled. You need to create /etc/udev/devices, containing the above device nodes. Also, the following will create the extra device nodes you need
mkdir /vz/private/123/etc/udev/devices
/sbin/MAKEDEV -d /vz/private/123/dev {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0
/sbin/MAKEDEV -d /vz/private/123/etc/udev/devices {p,t}ty{a,p}{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f} console core full kmem kmsg mem null port ptmx urandom zero ram0

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

If the files /vz/private/{CTID}/etc/sysconfig/network-scripts/ifdown-venet or
/vz/private/{CTID}/etc/sysconfig/network-scripts/ifup-venet exist, make sure they won't be used. These two files might exist if the physical server had OpenVZ installed. One way to do this is to rename them, like so:
mv ifdown-venet SKIP.ifdown-venet

Failing to do this will prevent networking from starting up correctly in the container.

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

===== Ubuntu server 8.x =====

Here what I have done for my Ubuntu server JEOS 8.04.2

<pre>
rm /vz/private/123/etc/network/if-up.d/ntpdate
rm /vz/private/123/etc/event.d/tty{1,2,3,4,5,6}
vzctl exec 123 update-rc.d -f klogd remove
vzctl exec 123 update-rc.d -f udev remove

</pre>

==== openSUSE/SLES ====

Use Yast.

=== Disable udev if you create DEVNODES devices ===

If you are creating devices for the container with a DEVNODES statement in a veid.conf file then these devices may be overwritten/deleted by udev when the container starts. As udev cannot "see" the device from within the container it disables it. Therefore, if you have DEVNODES statements in veid.conf then disable udev.

In Fedora, Redhat, Centos, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
Comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 1777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success stories ==
{{Note|please add your line to the bottom of this list, and do not forget to sign it using <code><nowiki>--~~~~</nowiki></code>}}

* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]] 16:16, 23 March 2007 (EST)
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]] 23:38, 26 February 2008 (EST)
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]] 17:35, 10 March 2008 (EST)
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]] 19:46, 10 March 2008 (EST)
* Debian 1:3.3.5-13 with apache2, PHP, etc. --[[User:Spawrks|spawrks]] 23:36, 10 April 2008 (EST)
* Debian Etch with apache2, MySQL, etc. --[[User:Zhafrance|zhafrance]] 16:29, 20 April 2008 (EST)
* Debian Etch i386 with apache2, MySQL, etc. --[[User:geejay|geejay]] 17:29, 26 May 2008 (GMT)
* Centos 4.6 with apache2, MySQL, Qmail etc. --[[User:Bharathchari|Bharathchari]] 08:06, 13 June 2008 (EDT)
* Centos 4.6 with cPanel/WHM (Apache2, Mysql, Exim, etc) --[[User:Zccopwrx|Zccopwrx]] 08:16, 30 July 2008 (EDT)
* SlackWare 10.1 (Qmail) --[[User:defiancenl|defiancenl]]
* SlackWare 10.0 (Qmail) --[[User:defiancenl|defiancenl]]
* Ubuntu 8.04.3 LTS JEOS (Apache2, Mysql) --[[User:bougui|bougui]] Fri Aug 28 10:40:41 EDT 2009
* CentOS 5.3 (Apache2, Mysql, Cacti) --[[User:kofl|kofl]] September 12 2009
* Scientific Linux 3.0.9 (Macrovision FLEXlm) {{unsigned|137.226.90.94|11:34, 4 November 2009}}
* Red Hat Enterprise Linux 4 (rhel4) --[[User:Bpuklich|Bpuklich]] 17:20, 15 February 2010 (UTC)
* Debian SID up-to-date with apache2, MySQL, posgrey etc. --nyquist 14:04, 06 July 2010 (UTC)
* Centos 5.x with Plesk -- 05:33, 17 August 2010 (UTC)
* Redhat 4 -- 20:32, 18 August 2010 (UTC)
* Fedora 4 -- 15:06, 20 August 2010 (UTC)
* Fedora 9 x64 with FDS and samba PDC --burn 23:20 10 October 2010
* Fedora 3 x32 with Plesk -- 23 October 2010 --[[User:Rexwickham|Rex Wickham (2020media.com)]] 13:15, 23 October 2010 (UTC)
[[Category:HOWTO]]

Container enter failed

2009-05-08T16:02:47Z

TimSmall:

'''Problem''': container created succesfully and started.
But when trying to do
<pre>
vzctl enter 101
</pre>
you get
<pre>
container enter failed(?)
</pre>

Using strace, you see:
<pre>
# strace -ff vzctl enter
....
fstat64(...st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0)...) fail
....
</pre>

'''Solution''':
Recompile the kernel with the following option:
<pre>
CONFIG_LEGACY_PTYS=y
</pre>

'''Other solutions''':

1) enter the VE manually creating the LEGACY_PTYS devices

<pre>
vzctl exec 101 /sbin/MAKEDEV tty
vzctl exec 101 /sbin/MAKEDEV pty
vzctl enter 101
</pre>

2A) If you want udev in VE, save the changes forcing udev to make LEGACY_PTYS:

<pre>
cat > /etc/udev/makedev.d/51-udev.nodes
# These device have to be created manually
tty0
tty1
tty2
tty3
....
ttyp0
ttyp1
ttyp2
ttyp3
....
ptyp0
ptyp1
ptyp2
ptyp3
....
</pre>

2B) If you think is better disable udev in VE, comment out in the VE the line:
<pre>
/sbin/start_udev
</pre>
in
<pre>
/etc/rc.d/rc.sysinit
</pre>

.. <b>however</b> updates to the package which owns this file may revert your changes, so you must take steps to guard against this.

Restart the VE and make the devices with MAKEDEV:
<pre>
vzctl exec 101 /sbin/MAKEDEV tty
vzctl exec 101 /sbin/MAKEDEV pty
vzctl enter 101
</pre>
I have used this method also to create zap dummy devices for asterisk in VE:
<pre>
/dev/zap/ctl
/dev/zap/pseudo
/dev/zap/channel
/dev/zap/timer
</pre>

2C) The devices can be setup also with a line in VE config

<pre>
grep DEV /etc/vz/conf/101.conf
DEVNODES="zap/ctl:rw zap/channel:rw zap/pseudo:rw zap/timer:rw "
</pre>

== See also ==

* {{Bug|130}}
* {{Bug|578}}

[[Category: Troubleshooting]]

Creating a CentOS 5.0 Template

2008-10-16T16:53:40Z

TimSmall: Add link to page with instructions for bootstrapping a centos rootfs using yum.

[[Category: HOWTO]]
[[Category: Templates]]
{{wikify}}

# Install a system (virtual or physical) with the default CentOS installation that you wish to package. This could be a minimal distro or the default distro or even something custom.
## It's generally quicker to bootstrap a CentOS system on another yum/rpm-based, or Debian based system by following the instructions in [http://faiwiki.informatik.uni-koeln.de/index.php/FAI_multi-distribution#bootstrapping_the_base_images__for_other_distributions]
# Once the OS has been installed, you need to '''tar''' the contents of the OS.
## First, create a file called '''/tmp/exclude''' and add the following lines to it:
#: .bash_history
#: lost+found
#: /dev/*
#: /mnt/*
#: /tmp/*
#: /proc/*
#: /sys/*
#: /usr/src/*
## Now, tar the OS file up by typing: '''tar –czvf /tmp/centos-5.0-<ARCH>-<DISTRO>-image.tar.gz –X /tmp/exclude''' where <'''ARCH'''> represents the system architecture ('''i386''' or
#: '''x86_64''') and <'''DISTRO'''> represents the distribution (default, minimal, etc.).
## Now transfer the file over to the OpenVZ server into '''/vz/template/cache''' folder.
# On the OpenVZ server create a “dummy” container by creating a folder called '''/vz/private/50'''
#: and copying the '''/etc/vz/conf/ve-vps.basic.conf-sample''' to '''/etc/vz/conf/50.conf'''.
#: '''NOTE''': 50 is the CTID for the container. You can choose any unused CTID on the OpenVZ server.
# Now create a new folder called '''/vz/template/centos/5.0/<ARCH>/config''' again where
#: <'''ARCH'''> represents the system architecture.
# Create a file in the folder called rpm and add the following line:
#: 43
# Create a file called '''yum.conf''' in the folder and add the following lines:
#: [main]
#:
#: cachedir=/vz/template/centos/5.0/<ARCH>/yum-cache/
#:
#: reposdir=/dev/null
#:
#: installonlypkgs=
#:
#: [centos5-base]
#: name=CentOS 5 - <ARCH> - Base
#:
#: baseurl=http://mirror.centos.org/centos/5/os/<ARCH>/
#:
#: enabled=1
#:
#: gpgcheck=1
#:
#: [centos5-updates-released]
#:
#: name=CentOS 5 - <ARCH> - Released Updates
#:
#: baseurl=http://mirror.centos.org/centos/5/updates/<ARCH>/
#:
#: enabled=1
#:
#: gpgcheck=1
# Copy '''/etc/vz/dists/centos-4.conf''' to '''/etc/vz/dists/centos-5.0.conf'''.
# Change to the '''/vz/private/50''' folder and then run the command '''gunzip –dc /vz/template/cache/centos-5.0-<ARCH>-<DISTRO>-image.tar.gz | tar –xvf''' – to unpack the base image to the folder.
# Make sure you are in the '''/vz/private/50''' folder.
# Edit '''etc/shadow''' and remove the replace the '''root''' password with !! instead of the hashed value.
# Edit the '''etc/inittab''' file and comment out the lines that respawn '''/sbin/mingetty''' on '''tty1''' through '''tty6'''. Just put a # at the beginning of the line.
# Remove the '''etc/mtab''' file and then create a symbolic link by typing '''ln –s /proc/mounts etc/mtab'''.
# Remove all of the lines from '''etc/fstab''' except for the line that mounts '''/dev/pts'''.
# Edit '''etc/rc.d/rc.sysinit''' and comment out the line that starts '''/sbin/start_udev''' by placing a # at the beginning of the line.
# Now create device nodes by typing:
#: mknod dev/ptmx c 5 2
#: mkdir dev/pts
#: /sbin/MAKEDEV –d /vz/private/50/dev ttyp ptyp
#: mknod dev/null c 1 3
#: mknod dev/urandom c 1 9
# Create the '''var/lock/rpm''' folder.
# If you wish to disable IPv6, do the following:
## Edit '''etc/sysconfig/network''' and set '''NETWORKING_IPV6''' to '''no'''.
## Add the following lines to '''etc/modprobe.d/blacklist''':
#: blacklist ipv6
#: blacklist net-pf-10
# Disable any physical NICs by modifying the '''etc/sysconfig/network-scripts/ifcfg-ethX''' files (where '''X''' is the interface number starting from '''0''') and setting '''ONBOOT''' to '''no'''.
# Now you’re ready to start the template. Type '''vzctl start 50''' and wait for it to start.
# You can install additional packages into the container by typing '''vzyum 50 install <package>''' at the prompt where <'''package'''> represents the name of the software package you wish to install.
# Finally, you should turn off unnecessary services.
## Enter the container by typing '''vzctl enter 50'''.
## View the services that are set to run at startup by typing '''chkconfig --list | grep 5:on'''.
## Disable any unwanted service by typing '''chkconfig --levels 2345 <service>''' off where <'''service'''> represents the service to disable.
#: Services that you can (and should) turn off without harm are acpid, apmd, kudzu, and microcode_ctl.
# Exit the container by typing '''exit''' at the prompt.
# Stop the container by typing '''vzctl stop 50'''.
# Finally, package up the new template by typing '''tar –czvf /vz/template/cache/centos-5.0-<ARCH>-<DISTRO>.tar.gz'''.
# The template is ready for use.