OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-07T07:17:56Z

Kirshil: /* Technical details */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

Note: before proceed with commands below, one needs to disable SE Linux on the host, otherwise rpm operations
in cdroot-ed environment described later on will fail due to inability to restore security file context. So, please,
do:
<pre>
# setenforce 0
</pre>
This will put SE linux in so-called 'permissive' mode, where all the blockers will be substituted by warnings.
If you wanna get rid of the warnings (rather annoying), the only way to do is to edit <tt>/etc/sysconfig/selinux</tt>
to read <tt>SELINUX=disabled</tt> and then reboot the host.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* dev/
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ livecd-root/tmp/
# chroot livecd-root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm livecd-root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir livecd-root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs livecd-root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp livecd-root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../livecd-root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-07T07:06:57Z

Kirshil: /* Vmlinux, initrd and isolinux.cfg */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* dev/
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ livecd-root/tmp/
# chroot livecd-root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm livecd-root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir livecd-root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs livecd-root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp livecd-root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../livecd-root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-07T06:39:37Z

Kirshil: /* Compressing '''booted''' LiveCD filesystem */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* dev/
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ livecd-root/tmp/
# chroot livecd-root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm livecd-root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir livecd-root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs livecd-root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-07T06:38:58Z

Kirshil: /* Install all these finally */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* dev/
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ livecd-root/tmp/
# chroot livecd-root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm livecd-root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir livecd-root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-06T16:07:22Z

Kirshil: /* Creating booted LiveCD root filesystem */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* dev/
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

Creating OpenVZ LiveCD based on Centos 4.4

2008-07-06T15:54:35Z

Kirshil: /* Creating booted LiveCD root filesystem */

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd -
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd -
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# for f in bin etc lib root sbin usr var; do \
unsquashfs ovz-livecd/base/$f.mo; \
mv squashfs-root/$f/ livecd-root; \
rm -rf squashfs-root; \
done
</pre>

Be patient, this will take a few minutes.

Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# cd livecd-root
# mkdir dev
# cp -a /dev/* root/dev
# mkdir proc
# mount -t proc proc proc/
# mkdir sys
# mount -t sysfs sysfs sys/
# mkdir tmp
# mkdir boot
# cd -
</pre>

That's it. We have '''bootstrapped''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this patch (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.

==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# for f in bin etc lib root sbin usr var vz; do \
mksquashfs root/$f ovz-livecd/base/$f.mo -keep-as-directory; \
done
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mke2fs initrd
initrd is not a block special device.
Proceed anyway? (y,n) y
...
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quiet</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

See more at [[remote console setup]].

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

[[Category: HOWTO]]
[[Category: Live CD]]

X inside VE

2008-01-22T13:51:27Z

Kirshil: /* XDM */

There are several ways to run X applications inside your [[VE]].

== X forwarding ==

=== Single application ===

To run an X application inside a [[VE]], one need simply to connect to a VE with '''ssh -X''':
<pre>
host# ssh -2 -c blowfish -X user@address
</pre>

After login to VE check that <code>$DISPLAY</code> variable is set and X11 forwarding is enabled:
<pre>
ve# echo $DISPLAY
localhost:10.0
</pre>

In case <code>$DISPLAY</code> is not set, make sure that X forwarding is enabled in <code>sshd</code> config inside VE. In most Linux distros sshd configuration is stored in <code>/etc/ssh/sshd_config</code>. You should set parameter <code>X11Forwarding</code> to <code>yes</code>. Also VE should contain <code>xauth</code> package, thus install <code>xauth</code> if it is missing (in Debian this is part of the xbase-clients package). After that, restart your sshd daemon:
<pre>
ve# /etc/init.d/sshd restart
</pre>

{{Note|Don't forget to reconnect after this}}

Now you can run X applications from your VE:
<pre>
ve# firefox
</pre>

=== Desktop ===

Note : If you want to run complete X window environment (including window manager), you should kill local window manager and run only pure X server. Secondly you should use '''-Y''' option when invoking ssh. And if you want to run gnome/kde/..., don't forget to increase [[UBC]] limits, 'cause default values are certainly too small for these monsters. ;)

You can run a desktop with ''xinit'' (on the node):

<pre>
xinit -e ssh -XCc blowfish user@ip_address "/usr/bin/xfce4-session &" -- :1 & disown
</pre>

* Substitute the window manager of choice
* Once xfce has started, you can then close the xterm if you like

Your node will be on Ctrl-Alt-F7

And your VM on Ctrl-Alt-F8

== VNC for X desktop ==
First, one need to run '''Xvnc''' server inside VE. The easiest way for this is to run
'''vncserver''' script. This scripts starts all the required services
and small http daemon which provides graphical web access to your desktop (via Java applet).

<pre>
ve# vncserver -name mydesktop
New 'mydekstop' desktop is ve:1

Starting applications specified in ~/.vnc/xstartup
Log file is ~/.vnc/ve:1.log
</pre>

Now when your desktop is up and running you can connect to it
using '''vncviewer''' command:
<pre>
host# vncviewer <VE_IP>:1
</pre>

If the VNC desktop is the same size or larger than your X desktop, you will see scroll bars on the bottom and the side. This is often inconvenient. You may reduce your VNC desktop to a more reasonable size like this:
<pre>vncserver -geometry 1000x650</pre>
This setting works quite well for a 1024 by 768 X desktop setting.

=== Starting KDE desktop with VNC ===
To start KDE desktop instead of default twm one replace <code>twm &</code> line with <code>startkde &</code> in user's
<code>~/.vnc/xstartup</code> file on the VE.

=== Connecting with VNC from firewalled network ===
VNC uses 590x TCP ports for its connections. These ports can be firewalled in
many networks so in order to be able to connect to remote side one need to tunnel VNC connections somehow.
A usual '''ssh''' can be used for tunneling VNC connections as described below.

<pre>
localhost# ssh -L 5900:localhost:5900 <remote host>
</pre>

where <remote host> is the name of the system you want to connect to. When you are asked for a username and password enter your normal username and password. Then start the VNC session to localhost, i.e.

<pre>
localhost# vncviewer localhost
</pre>

== Using Xephyr ==

Xephyr gives you nested X windows.

First, install Xephyr on your host.

Start Xephyr

<pre>Xephyr -ac -screen 1280x1024 -br -reset -terminate :1 &</pre>

Change your display settings (don't forget to change them back after you establish a connection)

<pre>DISPLAY=:1,0</pre>

Forward your application or desktop over ssh to Xephyr

<pre>ssh -XfC -c blowfish user@server xfce4-session</pre>

If you use an alternate window manager, substitute "startkde", "gnome-session", "startfluxbox", etc. as needed.

[http://cafelinux.org/OptickleArt/albums/userpics/Xephyr.png Screenshot]

== Using XDM with XDMCP ==
This method will give you a graphical login prompt remotely similar to VNC, but with some differences. Most notably, XDMCP is faster than VNC (due to the way each deals with screen handling) and but you CANNOT connect to existing sessions like you can with VNC. Each time you logout, your programs are closed. XDMCP is better suited in situations where you don't have a local display (or dumb terminals/clients) but want to run X11 programs

=== VE Configuration ===
Install your desktop environment as you'd like (kde/gnome/xfce/etc) and ensure you install at least XDM. You can opt to use GDM/KDM as they also do the same job as XDM. The configuration for KDM/GDM is different than XDM's and I was only able to find one link on configuring GDM (more below).

===== XDM =====
Configuring XDM requires editing 3 files: /etc/X11/xdm/xdm-config, /etc/X11/xdm/Xservers, /etc/X11/xdm/Xaccess

In xdm-config, comment out the line where it says '''DisplayManager.requestPort: 0'''

In Xservers, comment out the line ''':0 local /usr/bin/X :0 vt7'''' (this starts a local X server, which will fail)

In Xaccess, uncomment the line with ''' * #any host can get a login window'''
(Please keep in mind the security implications by the above line. Read the comments found in the file and set it appropriately)

To provide the possibility to run sound applications from VE, /dev/dsp device file needs to be exported in VE:
<pre>
vzctl set 221 --devnodes dsp:rw --save
</pre>

Finally, if you intend to make xdm invoking a heavy desktop like kde, it is reasonable to increase the amount of memory available for allocation inside this VE:
<pre>
vzctl set 221 --privvmpages 500M:600M --save
</pre>
For light desktop like icewm this is not needed.

Once these changes have been made, start your xdm server by the appropriate startup script (typically similar to /etc/init.d/xdm start). That concludes the XDM setup in the VE.

===== GDM =====
Edit the '''gdm.conf''' file and in xdmcp section, comment out the 0=standard line under the [servers] section - this will prevent gdm from trying to launch an X server on the local machine - it will simply listen for xdmcp requests. Also, change from VCAllocation=true to VTAllocation=false and comment out the FirstVT=7 line. Change the access restrictions (if any) to suit your needs and then start GDM.

=== Hostnode Configuration ===
On the HN, you need to install ONLY a bare Xserver as per your OS instructions (use yum, emerge, apt, whatever). A desktop environment like XFCE/GNOME/KDE are entirely optional (and not recommended as it'll unnecessarily just chew up resources) on the HN.

To get access to the XDM server, just start an Xserver with '''X -query <remote IP> :0''' (where :0 is the local display...set to :1 if you already have an X session running, or use Xnest)

There are guides online on howto tunnel the XDMCP settings over ssh giving you security, in an otherwise INSECURE protocol.

=== Errata ===
On a side note, as of December 2007, I was never able to successfully get an Xserver to run inside a VE and have the display output onto virtual-terminal 7 (the Xserver default), However, you can get an Xserver running on the hostnode to display output on virtual-terminal 7 without any special configuration (as the hostnode has direct access to all necessary devices).

== External links ==
* http://forum.openvz.org/index.php?t=tree&th=235&mid=1115&&rev=&reveal=
* http://ait.web.psi.ch/services/linux/kde-desktop-sharing.htm
* http://ait.web.psi.ch/services/ssh/vnc-ssh.html
* http://www.vnc.com/pipermail/vnc-list/2002-July/031831.html
* http://www.linuxjournal.com/article/5499
* http://www.realvnc.com/pipermail/vnc-list/2002-March/029502.html
* http://www.redhat.com/archives/rhl-list/2003-December/msg01859.html
* http://faq.gotomyvnc.com/fom-serve/cache/56.html

[[Category: HOWTO]]
[[Category: Networking]]

Asterisk from source

2008-01-14T12:15:04Z

Kirshil:

==General==
Asterisk is free and open source code to create software PBX server. See [http://www.asterisk.org] for details. This package perfectly runs inside OpenVZ container. Some users run up to 60 containers with Asterisk deployed for production per single hardware node. Although the easiest way to install Asterisk into container is to use pre-build package from Linux distribution, occasionally one may need to have a possibility to build it from source tarball available on developer's site.

In order to do it the following remarks are worth reading:

==Building Asterisk in CT==
Asterisk PBX server itself is compiled out of the shelf in CT provided that develop application template is installed on this CT. The functionality of the resulting executable is enough to support simple VoIP telephony.

<pre>
wget http://downloads.digium.com/pub/asterisk/releases/asterisk-x.x.xx.tar.gz
tar xzf asterisk-x.x.xx.tar.gz
cd asterisk-x.x.xx.tar.gz
./configure
make
make install
make samples
</pre>

The last command is not needed in case you have your own configuration as it installs some sample configuration files.
To configure Asterisk itself see for example [http://www.digium.com/elqNow/elqRedir.htm?ref=http://downloads.oreilly.com/books/9780596510480.pdf].

==MeetMe problem==
Unfortunately, one particular module called MeetMe (conferencing tool) will be switched off from compilation. This happens due to external dependency on 'zaptel' package. Zaptel provides support for some hardware cards for FXO/FXS analog telephony marketed by Digium (the company behind Asterisk), and on top of that supplies so called ztdummy kernel module. Ztdummy works like a simple metronome which is required to synchronize multiple sound streams in case of conference call.

If you do not plan to use analog telephone lines, hence don't like in install the hardware, nothing is lost provided you run your HN with 2.6.XX kernel. You just need to play a little trick with Asterisk make system: download zaptel tarball from the same location as Asterisk itself, and copy it's header zaptel.h to location /usr/include/zaptel/zaptel.h in CT where you plan to build Asterisk. This tweaks MeetMe for installation.

==HN configuration==
Finally you need to make sure that on HN ztdummy kernel module is loaded and the access to /dev/zap/pseudo device file is granted to
CT:
<pre>
modprobe ztdummy
vzctl set 240 --devnodes zap/pseudo:rw --save
</pre>

[[Category:HOWTO]]

Asterisk from source

2008-01-14T12:00:06Z

Kirshil:

==General==
Asterisk is free and open source code to create software PBX server. See [http://www.asterisk.org] for details. This package perfectly runs inside OpenVZ container. Some users run up to 60 containers with Asterisk deployed for production per single hardware node. Although the easiest way to install Asterisk into container is to use pre-build package from Linux distribution, occasionally one may need to have a possibility to build it from source tarball available on developer's site.

In order to do it the following remarks are worth reading:

==Building Asterisk in CT==
Asterisk PBX server itself is compiled out of the shelf in CT provided that develop application template is installed on this CT. The functionality of the resulting executable is enough to support simple VoIP telephony.

<pre>
wget http://downloads.digium.com/pub/asterisk/releases/asterisk-x.x.xx.tar.gz
tar xzf asterisk-x.x.xx.tar.gz
cd asterisk-x.x.xx.tar.gz
./configure
make
make install
make samples
</pre>

The last command is not needed in case you have your own configuration as it installs some sample configuration files.

==MeetMe problem==
Unfortunately, one particular module called MeetMe (conferencing tool) will be switched off from compilation. This happens due to external dependency on 'zaptel' package. Zaptel provides support for some hardware cards for FXO/FXS analog telephony marketed by Digium (the company behind Asterisk), and on top of that supplies so called ztdummy kernel module. Ztdummy works like a simple metronome which is required to synchronize multiple sound streams in case of conference call.

If you do not plan to use analog telephone lines, hence don't like in install the hardware, nothing is lost provided you run your HN with 2.6.XX kernel. You just need to play a little trick with Asterisk make system: download zaptel tarball from the same location as Asterisk itself, and copy it's header zaptel.h to location /usr/include/zaptel/zaptel.h in CT where you plan to build Asterisk. This tweaks MeetMe for installation.

==HN configuration==
Finally you need to make sure that on HN ztdummy kernel module is loaded and the access to /dev/zap/pseudo device file is granted to
CT:
<pre>
modprobe ztdummy
vzctl set 240 --devnodes zap/pseudo:rw --save
</pre>

[[Category:HOWTO]]

Asterisk from source

2008-01-14T11:36:48Z

Kirshil: New page: Asterisk is free and open source code to create software PBX server. See [http://www.asterisk.org] for details. This package perfectly runs inside OpenVZ container. Some users run up to 60...

Asterisk in container with Debian stable

2008-01-14T11:36:18Z

Kirshil:

Below is an example of how to install Asterisk into OpenVZ VE based on debian stable aka. "sarge":

<pre>
$ apt-get install asterisk

$ nano /etc/defaults/asterisk
</pre>

change:
<pre>
RUNASTERISK=no -> yes
AST_REALTIME=yes -> no (for realtime, you need

$ /etc/init.d/asterisk start
</pre>
modify the config files for asterisk, and done!

if you need capi or/and zaptel/zaphfc:

* for capi: you need the /dev files on you hw-node and vps:
<pre>
crw-rw---- 1 root dialout 68, 0 Jan 3 2006 /dev/capi20
crw-rw---- 1 root dialout 68, 1 Jan 3 2006 /dev/capi20.00
crw-rw---- 1 root dialout 68, 2 Jan 3 2006 /dev/capi20.01
crw-rw---- 1 root dialout 68, 3 Jan 3 2006 /dev/capi20.02
[...]
</pre>
/* if the files doesn' exist use mknod for creating it*/

and in your vps config add for directly access:
<pre>
DEVICES="c:68:0:rw, c:68:1:rw, c:68:2:rw"
</pre>
* for zaptel/zaphfc:
<pre>
crw-rw---- 1 root dialout 196, 0 Jan 4 2006 ctl
crw-rw---- 1 root dialout 196, 1 Jan 4 2006 1
crw-rw---- 1 root dialout 196, 2 Jan 4 2006 2
crw-rw---- 1 root dialout 196, 3 Jan 4 2006 3
crw-rw---- 1 root dialout 196, 4 Jan 4 2006 4
[...]
crw-rw---- 1 root dialout 196, 253 Jan 4 2006 timer
crw-rw---- 1 root dialout 196, 255 Jan 4 2006 pseudo
crw-rw---- 1 root dialout 196, 254 Jan 4 2006 channel
</pre>
/* if the files doesn' exist use mknod for creating it*/

and in your vps config
<pre>
DEVICES="c:196:0:rw, c:196:2:rw, c:196:1:rw, c:196:253:rw,c:196:254:rw,c:196:255:rw"
</pre>

if need both (capi and zaphfc)
<pre>
DEVICES="c:68:0:rw, c:68:1:rw, c:68:2:rw c:196:0:rw, c:196:2:rw, c:196:1:rw, c:196:253:rw,c:196:254:rw,c:196:255:rw"
</pre>

[[category:HOWTO]]

Asterisk in container with Debian stable

2008-01-11T14:25:10Z

Kirshil: how to compile asterisk inside CT

Asterisk is free and open source code to create software PBX server. See [http://www.asterisk.org] for details.
This package perfectly runs in side OpenVZ container. Some users run up to 60 containers with Asterisk deployed for
production per single hardware node. Although the easiest way to install Asterisk into container is to use pre-build package
from Linux distribution, occasionally one may need to have a possibility to build it from source tarball available on developer's site.

In order to do it the following remarks are worth reading:

Asterisk PBX server itself is compiled out of the shelf in CT provided that develop application template is installed for this CT.
The functionality of the resulting executable is enough to support simple VoIP telephony.

Unfortunately, one particular module called MeetMe (conferencing tool) will be switched off from compilation. This happens due to external dependency on 'zaptel' package.
This package provides support for some hardware cards for FXO/FXS analog telephony marketed by Digium (the company behind Asterisk), and on top of that supplies so called ztdummy kernel module. Ztdummy works like a simple metronome which is required to synchronize multiple sound streams in case of conference call. Initially this module requires 1 kHz clock which is avail for example on all Digium cards.

If you do not have this card (or equivalent from other vendors) as in case if you do not plan to use analog telephone lines, nothing is lost. You just need to play a little trick with Asterisk make system and HN ovz kernel.
First one needs to download zaptel tarball from the same location as Asterisk itself, and copy it's header zaptel.h to location
/usr/include/zaptel/zaptel.h in the CT where you plan to build Asterisk. This tweaks MeetMe for installation. Then you need to build
ztdummy.ko in HN and make it auto-loaded.

Below is an example of how to install Asterisk into OpenVZ VE based on debian stable aka. "sarge":

<pre>
$ apt-get install asterisk

$ nano /etc/defaults/asterisk
</pre>

change:
<pre>
RUNASTERISK=no -> yes
AST_REALTIME=yes -> no (for realtime, you need

$ /etc/init.d/asterisk start
</pre>
modify the config files for asterisk, and done!

if you need capi or/and zaptel/zaphfc:

* for capi: you need the /dev files on you hw-node and vps:
<pre>
crw-rw---- 1 root dialout 68, 0 Jan 3 2006 /dev/capi20
crw-rw---- 1 root dialout 68, 1 Jan 3 2006 /dev/capi20.00
crw-rw---- 1 root dialout 68, 2 Jan 3 2006 /dev/capi20.01
crw-rw---- 1 root dialout 68, 3 Jan 3 2006 /dev/capi20.02
[...]
</pre>
/* if the files doesn' exist use mknod for creating it*/

and in your vps config add for directly access:
<pre>
DEVICES="c:68:0:rw, c:68:1:rw, c:68:2:rw"
</pre>
* for zaptel/zaphfc:
<pre>
crw-rw---- 1 root dialout 196, 0 Jan 4 2006 ctl
crw-rw---- 1 root dialout 196, 1 Jan 4 2006 1
crw-rw---- 1 root dialout 196, 2 Jan 4 2006 2
crw-rw---- 1 root dialout 196, 3 Jan 4 2006 3
crw-rw---- 1 root dialout 196, 4 Jan 4 2006 4
[...]
crw-rw---- 1 root dialout 196, 253 Jan 4 2006 timer
crw-rw---- 1 root dialout 196, 255 Jan 4 2006 pseudo
crw-rw---- 1 root dialout 196, 254 Jan 4 2006 channel
</pre>
/* if the files doesn' exist use mknod for creating it*/

and in your vps config
<pre>
DEVICES="c:196:0:rw, c:196:2:rw, c:196:1:rw, c:196:253:rw,c:196:254:rw,c:196:255:rw"
</pre>

if need both (capi and zaphfc)
<pre>
DEVICES="c:68:0:rw, c:68:1:rw, c:68:2:rw c:196:0:rw, c:196:2:rw, c:196:1:rw, c:196:253:rw,c:196:254:rw,c:196:255:rw"
</pre>

[[category:HOWTO]]