OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Physical to container

2008-04-10T19:36:51Z

Spawrks: /* Success Stories */

A rough description of how to migrate existing physical server into a [[container]].

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude dev --exclude proc --exclude tmp -e "ssh -l root@a.b.c.d" root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e '/getty/d' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

rm -f /vz/private/123/etc/mtab
ln -s /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the line for /dev/pts):

cp /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
grep devpts /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

==== openSUSE/SLES ====

Use Yast.

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success Stories ==
* Debian 1:3.3.5-13 with apache2, PHP, etc. [spawrks , april 10, 2008]
* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]]
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]]
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]]
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]]

[[Category:HOWTO]]

Physical to container

2008-04-10T19:13:07Z

Spawrks:

A rough description of how to migrate existing physical server into a [[container]].

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude dev --exclude proc --exclude tmp -e "ssh -l root@a.b.c.d" root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e '/getty/d' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

rm -f /vz/private/123/etc/mtab
ln -s /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the line for /dev/pts):

cp /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
grep devpts /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

==== openSUSE/SLES ====

Use Yast.

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==

===PHP not serving pages / random issues===

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success Stories ==
* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]]
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]]
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]]
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]]

[[Category:HOWTO]]

Physical to container

2008-04-10T19:11:43Z

Spawrks:

A rough description of how to migrate existing physical server into a [[container]].

== Prepare a new “empty” container ==
For OpenVZ this would mean the following (assume you chose CT ID of 123):

mkdir /vz/root/123 /vz/private/123
cat /etc/vz/conf/ve-vps.basic.conf-sample > /etc/vz/conf/123.conf

== Preparing to migrate ==

Stop most services on a machine to be migrated. “Most” means services such as web server, databases and the like — so you will not lose your data. Just leave the bare minimum (including ssh daemon).

== Copying the data ==

Copy all your data from the machine to an OpenVZ box. Say you'll be using container with ID of 123, then all the data should be placed to <code>/vz/private/123/</code> directory (so there will be directories such as <code>/vz/private/123/bin</code>, <code>etc</code>, <code>var</code> and so on). This could be done in several ways:

=== rsync ===
rsync example (run from the new HN):
rsync -arvpz --numeric-ids --exclude dev --exclude proc --exclude tmp -e "ssh -l root@a.b.c.d" root@a.b.c.d:/ /vz/private/123/

'''Advantage:''' Your system doesn't really go down.

=== Live CD ===
Another way to do is using a live cd, booting up and use tar to dump the complete disk in a tar you save over the network or on a USB device.

=== Tar ===
Another approach is using tar and excluding some dirs, you could do it like this:

Create a file /tmp/excludes.excl with these contents:
.bash_history
/dev/*
/mnt/*
/tmp/*
/proc/*
/sys/*
/usr/src/*

Then create the tar. But remember, when the system is 'not' using udev, you have to look into /proc/ after creating your container because some devices might not exist. (/dev/ptmx or others)

# tar cjpf /tmp/mysystem.tar.bz2 / -X /tmp/excludes.excl

Naturally, you can only do this when the critical services (MySQL, apache, ..) are stopped and your /tmp filesystem is big enough to contain your tar.

'''Advantage:''' You don't need to boot from a live cd, so your system doesn't really go down.

== Setting container parameters ==

=== OSTEMPLATE ===
You have to add <code>OSTEMPLATE=xxx</code> line to <code>/etc/vz/conf/123.conf</code> file, where <code>xxx</code> would be distribution name (like <code>debian-3.0</code>) for vzctl to be able to make changes specific for this distribution.

=== IP address(es) ===
Also, you have to supply an IP for a new container:

vzctl set 123 --ipadd x.x.x.x --save

=== venet vs. veth ===
You may use veth interface instead of venet if you need just bring old server up for seamless migration of services.
It may be nessessary if server you are migrating is badly configured and it is hard to find all hard-coded net interfaces settings and so on.

veth inteface may be included into bridge to allow seamless old installation access.

== Making adjustments ==
Since container is a bit different to a real physical server, you have to edit some files inside your new container.

=== /etc/inittab ===
A container does not have real ttys, so you have to disable getty in <code>/etc/inittab</code> (i. e. <code>/vz/private/123/etc/inittab</code>).

sed -i -e '/getty/d' /vz/private/123/etc/inittab

=== /etc/mtab ===
Link <code>/etc/mtab</code> to <code>/proc/mounts</code>, for <code>df</code> to work properly:

rm -f /vz/private/123/etc/mtab
ln -s /proc/mounts /vz/private/123/etc/mtab

{{out|The problem here is container's root filesystem (<code>/</code>) is mounted not from the container itself, but rather from the host system. That leaves <code>/etc/mtab</code> in container without a record for <code>/</code> being mounted, thus df doesn't show it. By linking <code>/etc/mtab → /proc/mounts</code> we make sure /etc/mtab shows what is really mounted in a container.

Sure this is not the only way to fix df; you can just manually add a line to <code>/etc/mtab</code> telling <code>/</code> is mounted, and make sure this line will be there after a reboot.}}

=== /etc/fstab ===
Since you do not have any real disk partitions in a container, /etc/fstab (or most part of it) is no longer needed. Empty it (excluding the line for /dev/pts):

cp /vz/private/123/etc/fstab /vz/private/123/etc/fstab.old
grep devpts /vz/private/123/etc/fstab.old > /vz/private/123/etc/fstab

You can also mount a devpts in a running (but not fully functional) container:
vzctl exec 123 mount -t devpts none /dev/pts

=== /dev ===

==== Introduction: static /dev ====
In order for container to work, some nodes should be present in container's <code>/dev</code><code></code>. For modern distributions, udev is taking care of it. For a variety of reasons udev doesn't make much sense in a container, so the best thing to do is to disable udev and create needed device nodes manually.

Note that in some distributions <code>/dev</code> is mounted on <code>tmpfs</code> — this will not work in case of static <code>/dev</code>. So what you need to do is find out where <code>/dev</code> is being mounted on <code>tmpfs</code> and remove this. This is highly distribution-dependent; please add info for your distro here.

After you made sure your <code>/dev</code> is static, populate it with needed device nodes.

Please pay attention to the access permissions of the device files being created: a default file mode for newly created files is affected by <code>umask</code> ([[w:umask]]). You can use --mode option for <code>mknod</code> to set the desired permissions.

==== tty device nodes ====

In order for vzctl enter to work, a container needs to have some entries in /dev. This can either be /dev/ttyp* and /dev/ptyp*, or /dev/ptmx and mounted /dev/pts.

===== /dev/ptmx =====
Check that /dev/ptmx exists. If it does not, create with:
mknod --mode 666 /vz/private/123/dev/ptmx c 5 2

===== /dev/pts/ =====
Check that /dev/pts exists. It's a directory, if it does not exist, create with:
mkdir /vz/private/123/dev/pts

===== /dev/ttyp* and /dev/ptyp* =====
Check that /dev/ttyp* and /dev/ptyp* files are there. If not, you have to create those, either by using /sbin/MAKEDEV, or by copying them from the host system.

To copy:
cp -a /dev/ttyp* /dev/ptyp* /vz/private/123/dev/

To recreate with MAKEDEV, either
/sbin/MAKEDEV -d /vz/private/123/dev ttyp ptyp
or
cd /vz/private/123/dev && /sbin/MAKEDEV ttyp

====/dev/null====
Make sure sure /dev/null is not a file or directory; if unsure remove and recreate. If this is not correct sshd will not start correctly.
rm -f /vz/private/123/dev/null
mknod --mode 666 /vz/private/123/dev/null c 1 3

==== /dev/urandom ====
Check that /dev/urandom exists. If it does not, create with:
mknod --mode 444 /vz/private/123/dev/urandom c 1 9

===/proc===
Make sure the /proc directory exists:
ls -la /vz/private/123/ | grep proc

If it doesn't, create it:
mkdir /vz/private/123/proc

=== /etc/init.d services ===

Some system services can (or in some cases should) be disabled. A few good candidates are:

* acpid, amd (not needed)
* checkfs, checkroot (no filesystem checking is required in container)
* clock (no clock setting is required/allowed in container)
* consolefont (container does not have a console)
* hdparm (container does not have real hard drives)
* klogd (unless you use iptables to LOG some packets)
* keymaps (container does not have a real keyboard)
* kudzu (container does not have real hardware)
* lm_sensors (container does not have access to hardware sensors)
* microcodectl (container can not update CPU microcode)
* netplugd (container does not have real Ethernet device)

To see which services are enabled:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --list</code>
* Debian: Use '<code>rcconf</code>' (ncurses) or <code>update-rc.d</code>
( See: http://www.debianadmin.com/manage-linux-init-or-startup-scripts.html )
* Gentoo: <code>/sbin/rc-update show</code>

To disable the service:
* RedHat/Fedora/SUSE: <code>/sbin/chkconfig --del SERVICENAME </code>
* Debian: <code>' update-rc.d -f hdparm remove '</code>
* Gentoo: <code>/sbin/rc-update del SERVICENAME</code>

=== Disable old network interface ===
You should disable your old physical network interface from starting at boot time. This is distribution-dependant.

==== Fedora/CentOS/Red Hat ====
Edit /vz/private/{CTID}/etc/sysconfig/network-scripts/ifcfg-eth''x''

Make the following look like this:
ONBOOT=no

==== Debian/Ubuntu ====
Edit /etc/network/interfaces

<pre>
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo eth0
iface lo inet loopback

iface eth0 inet dhcp
address 10.0.0.4
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.0.0.255
</pre>

You can either comment out the eth* interface stanza(s), or take it out of the "auto" line(s).

==== openSUSE/SLES ====

Use Yast.

=== Other adjustments ===
There might be other adjustments needed. Please add those here (just above this section) if you have more info.

== Starting a new container ==

Try to start your new container:

vzctl start 123

Now check that everything works fine. If not, see [[#Troubleshooting]] below.

== Troubleshooting ==
==PHP not serving pages / random issues==

Make sure that /tmp and /var/tmp are created if you rsynced over your data and that they have proper permissions

mkdir tmp
chmod 777 tmp

=== Can't enter container ===

If you can not enter your container (using <code>vzctl enter</code>), you should be able to at least execute commands in it.

First, see the [[#tty device nodes]] section above.

Next, check if devpts is mounted:
vzctl exec 123 mount | grep pts

If it is not mounted, mount it:
vzctl exec 123 mount -t devpts none /dev/pts

Then, add the appropriate mount command to container's startup scripts. On some distros, you need to have the appropriate line in container's /etc/fstab.

In Fedora, try commenting out any '''udev''' entries in /vz/private/{CTID}/etc/rc.sysinit
vi /vz/private/{CTID}/etc/rc.sysinit
Locate the '''udev''' entry from within vim
/udev
Then comment the line similar to this:
#[ -x /sbin/start_udev ] && /sbin/start_udev

=== Other problems ===
If anything goes wrong, try to find out why and fix. If you have enough Linux experience, it can be handled. Also check out IRC and please report back on this page.

== Success Stories ==
* Debian 3.1 Sarge with MySQL, apache2, PowerDNS --[[User:Stoffell|stoffell]] 08:41, 8 February 2007 (EST)
* Red Hat 7.2 with MySQL 3.23, apache, Chilisoft --[[User:Stoffell|stoffell]] 13:26, 9 February 2007 (EST)
* Gentoo with Courier, Postfix, MySQL, Apache2 --[[User:bfrackie|bfrackie]] 19:00, 18 March 2007 (EST)
* AltLinux Master with qmail, MySQL, Apache, etc - to Debian/testing with OpenVZ --[[User:alexkuklin|alexkuklin]]
* Centos 4.4 with apache2, SVN, TRAC, etc. --[[User:bitherder|bitherder]]
* Centos 4.6 with apache2, Tomcat 5.0.x, postgresql, etc on CentOS 5.1 64bit Host --[[User:laslos|laslos]]
* Debian Etch with apache2 etc... on CentOS 4.6 Host --[[User:laslos|laslos]]

[[Category:HOWTO]]