Creating OpenVZ LiveCD based on Centos 4.4

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD. As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18, rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able to create VEs, start them, enter them - i.e. perform common OpenVZ activity.

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

General actions

It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:

• Get a copy of CentOS 4.4 LiveCD image
• Use unsquashfs tool to extract the contents of *.mo files, located on the image
• Construct booted LiveCD root filesystem from extracted files
• Create OpenVZ kernel RPM-packages with squashfs support
• Compile unionfs kernel modules against OpenVZ kernels
• Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
• Remove some RPM-packages from booted LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
• Install OpenVZ kernels, tools and templates in booted LiveCD root filesystem
• Using mksquashfs tool compress booted CD root filesystem
• Modify initrd image: add there kernel modules for OpenVZ kernels
• Introduce small modifications to isolinux configuration file
• Create new iso image

Technical details

This part contains detailed description of LiveCD creation. I've removed the output of some commands in order to simplify the reading.

Creating booted LiveCD root filesystem

Create a directory, that will be our "workbench":

# cd ~
# mkdir workbench
# cd workbench

First of all we need some raw material - CentOS 4.4 LiveCD image:

# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd 
CentOS-4.4-i386-LiveCD.iso: OK

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem. I named this directory "ovz-livecd":

# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount

You can see, that ovz-livecd/base/ directory owns several *.mo files - images of squashfs filesystem, and each image holds one directory of booted LiveCD root filesystem: bin, usr and so on:

# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo

In order to decompress these images we need unsquashfs tool. squashfs source archive contains patches for various kernel versions (to be able mount squashfs), and user-space tools for compressing usual filesytem into squashfs filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install squashfs user-space tools:

# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench

Well, having the intruments in hand, why not extract the images? Create a directory, called livecd-root, that will store the root filesystem of booted LiveCD. I.e. it will contain /bin, /sbin, /etc and other directories, which you will see after booting the LiveCD. Unfortunately unsquashfs tool is not very convenient, and I found only this way to use it:

# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo 
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo 
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo 
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo 
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo 
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo    # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root

Yet we have nearly complete root filesystem of booted LiveCD in livecd-root directory. However some things are missing, and I guess you know what are they: /dev, /proc, /sys, /tmp and /boot directories. We need them, because some tools (rpm, bash, ...) will not work without them in chrooted environment later.

# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot

That's it. We have booted LiveCD root filesystem in livecd-root directory. Now we can chroot in this directory and install all needed packages. But what packages do we need?

• OpenVZ kernels
• OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
• OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
• Some OpenVZ precreated templates
• Some CentOS tool updates: sqlite package

Tools and templates related packages can be simply downloaded and installed. But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't support squashfs. The next chapter explains how to create RPM-package with OpenVZ kernel, that supports squashfs

Building OpenVZ kernels

Let's consider, how to build 2.6.18 OpenVZ kernel RPM-package with squashfs support. First download and install OpenVZ source RPM:

# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm

squashfs sources contain the patch against 2.6.18. Let's copy this patch to RPM build directory:

# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/

Now we need to modify the spec-file (/usr/src/redhat/SPECS/kernel-ovz.spec) in order to add squashfs patch to build process. Below are the changes you should introduce in diff format. The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be based on SMP kernel) and secondly, apply squashfs patch before compilation.

--- kernel-ovz.spec.orig        2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec     2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

 # normal/debug/debuginfo
 %define buildnormal 1
-%define builddebug 1
+%define builddebug 0
 %define builddebuginfo 0

 # kernels we want to build
-%define buildup 1
+%define buildup 0
 %define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
 %define buildentnosplit 0

 # versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
 Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
 Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
 # ----------------------------------------------------------------------------

 BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
 %patch2105 -p1
 %patch2106 -p1

+%patch2107 -p1
+
 # ----------------------------------------------------------------------------

 mkdir configs

We need one more tiny hack in order RPM-build process to walk smoothly. There are config files for various OpenVZ kernel flavours in /usr/src/redhat/SOURCES directory:

# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz             /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz  /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz         /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz

All these files should contain information about squashfs. In fact, only two additional lines are necessary:

CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n

I did it with the following command:

# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/

After all efforts, building the RPM-package is very easy:

# rpmbuild --target=i686 -ba kernel-ovz.spec

If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with squashfs support. Let's copy it to our local directory rpms:

# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir rpms
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm rpms/

OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels, that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description below are based on 028stab035 kernel, that has been just build above.

Installing RPMs

We have the RPM package for the kernel. But we also need

<

To be continued...




=== Obrezki ===
TODO
rpms/
rpms/kernel
rpms/tools
rpms/update

Uninstall <tt>lsctp-tools</tt> package, because these tools they need kernel version prior to 2.6.10:
<pre>
rpm -e lksctp-tools-1.0.2-6.4E.1.i386

Uninstall kernel-module-xfs package, because these modules are compiled against original

rpm -e kernel-module-xfs-2.6.9-42.livecd.c4-0.1-3.i686

cd /tmp/rpms/kernel rpm -ivh kernel-smp-2.6.18-ovz028stab033.1.i686.rpm rpm -e kernel-2.6.9-42.livecd.c4

cd /tmp/rpms/updates rpm -Uvh sqlite-3.3.6-2.i386.rpm

1. rpm -ivh vz*

Preparing... ########################################### [100%]

  1:vzrpm44                ########################################### [  8%]
  2:vzrpm44-python         ########################################### [ 15%]
  3:vzrpm43-python         ########################################### [ 23%]
  4:vzrpm43                ########################################### [ 31%]
  5:vzyum                  ########################################### [ 38%]
  6:vzquota                ########################################### [ 46%]
  7:vzctl-lib              ########################################### [ 54%]
  8:vzctl                  ########################################### [ 62%]
  9:vzpkg                  ########################################### [ 69%]
 10:vztmpl-centos-4        ########################################### [ 77%]
 11:vztmpl-fedora-core-3   ########################################### [ 85%]
 12:vztmpl-fedora-core-4   ########################################### [ 92%]
 13:vztmpl-fedora-core-5   ########################################### [100%]

External Links

• Official CentOS community website
• Linux kernel website
• RedHat Inc. website
• squashfs website
• uninonfs official homepage