Creating OpenVZ LiveCD
The idea of creating OpenVZ LiveCD for a long time was in the air. Such CD would be useful for various demonstrations, test-drives and some other purposes. Additionally we received reports that some problems persist in creating LiveCD based on OpenVZ kernel. In order to eliminate these problems and give users the ability to use OpenVZ technology in LiveCD manner we decided to create it. This article describes how I was doing about it. Probably there are some easier and more effective ways, so feel free to improve the article. We chose KNOPPIX as a base for our CD for two reasons. Firstly KNOPPIX is probably the most widespread LiveCD in the world. Secondly KNOPPIX was interested in integrating OpenVZ in one of its special releases. The article starts from conceptual actions required to create OpenVZ LiveCD and then step-by-step describes technical aspects.
Contents
Concepts
- Download KNOPPIX iso and get access to its content
- Extract KNOPPIX compressed image
- Create OpenVZ Debian package and some other kernel-related packages
- Uninstall old Linux kernel and related packages from extracted tree and
- install OpenVZ kernel there
- Uninstall some packages from extracted tree in order to free some space for
- the next step
- Install vzctl/vzquota/templates in the tree
- Compress the tree
- Copy required kernel modules in appropriate places on the disc
- Create new ISO image
Step-by-step actions
First we need to download KNOPPIX iso image. Here is a list of mirrors where you can download it. I used KNOPPIX 5.1.1 from the nearest mirror:
mkdir openvz-livecd cd openvz-livecd wget ftp://ftp.chg.ru/pub/Linux/knoppix/KNOPPIX_V5.1.1CD-2007-01-04-EN.iso
Let's copy contents of iso to to hard drive:
mkdir mnt mount -o loop KNOPPIX_V5.1.1CD-2007-01-04-EN.iso mnt mkdir cd-contents cp -a mnt/* cd-contents/ umount mnt rm -f mnt
When you're booted in KNOPPIX LiveCD you can observe a lot of files and use variouse applications. On CD all this information is stored in one file:
ls -lh cd-contents/KNOPPIX/KNOPPIX -rw-r--r-- 1 vass vass 691M Jan 4 03:50 cd-contents/KNOPPIX/KNOPPIX
As you can see its size is 691M, but after uncompressing it'll be approximately 2Gb. In order to uncompress this file we need special tools: cloop-utils. You can download them from this site. I used cloop-2.05-1, because exactly this version is installed on KNOPPIX 5.1.1.
wget http://debian-knoppix.alioth.debian.org/sources/cloop_2.05-1.tar.gz tar xzf cloop_2.05-1.tar.gz cd cloop_2.05-1 make extract_compressed_fs make create_compressed_fs # this tool will be required later, when compressing modified tree cd ..
After we have extract_compressed_fs tool in hand, let's extract KNOPPIX image:
cloop_2.05-1/extract_compressed_fs cd-contents/KNOPPIX/KNOPPIX > KNOPPIX.img ls -lh KNOPPIX.img -rw-rw-r-- 1 vass vass 2.0G Apr 12 17:38 KNOPPIX.img
KNOPPIX.img file can be mounted using usual '-o loop' option. Next our step is to create a copy of file system tree on hard drive. Note, that we can't use 'cp' tool here, because of hardlinks and special files. We need to use rsynс:
mkdir mnt mount -o loop KNOPPIX.img mnt mkdir KNOPPIX-content rsync -Hav mnt KNOPPIX-content umount mnt rm -r mnt
Now we should set up OpenVZ environment in obtained tree: kernel, vzctl, vzquota, template. For vzctl and vzquota we can use precompiled packages from http://debian.systs.org. But we have to create kernel Debian package by self, because at the moment we need additional patches in order all to work smoothly (but this will be unnecessary soon, when patches will be included in main OpenVZ patch). I created a Debian VE on my machine and install there kernel-package package, that allows to create Debian kernel packages easily. After that the creation of package is something like that:
mkdir ovz-kernel cd ovz-kernel wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2 wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab027.1/patches/patch-8.el5.028stab027.1-combined.gz wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab027.1/configs/kernel-2.6.18-i686.config.ovz wget http://download.openvz.org/~vass/diff-ubc-mm-null-pointer-for-kthread-20070406 # this patch will be not necessary soon. tar xjf linux-2.6.18.tar.bz2 zcat patch-8.el5.028stab027.1-combined.gz | patch -p1 -d linux-2.6.18 patch -p1 -d linux-2.6.18 < diff-ubc-mm-null-pointer-for-kthread-20070406 mv linux-2.6.18 linux-2.6.18-8.el5-028test027 vim linux-2.6.18-8.el5-028test027/Makefile # here we add right release: -8.el5-028test027 cp kernel-2.6.18-i686.config.ovz linux-2.6.18/.config make-kpkg --revision=.1 kernel_image cd ..
If everything went smoothly, in the top directory we'll be able to see linux-image file:
ls -l linux-image-2.6.18-8.el5-028stab027_.1_i386.deb -rw-r--r-- 1 vass vass 13364308 Apr 16 11:06 linux-image-2.6.18-8.el5-028stab027_.1_i386.deb
Don't delete sources and compilation of this kernel. A bit later we'll need to build aufs and cloop kernel modules against this kernel. Now I download vzctl and vzquota Debian packages from http://debian.systs.org and on template:
wget http://debian.systs.org/debian/pool/openvz/v/vzctl/vzctl_3.0.16-1dso2~sarge0_i386.deb wget http://debian.systs.org/debian/pool/openvz/v/vzquota/vzquota_3.0.9-1dso1_i386.deb wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
Now copy packages to KNOPPIX tree and chroot there:
cp linux-image-2.6.18-8.el5-028stab027_.1_i386.deb KNOPPIX-content/root cp vzctl_3.0.16-1dso2~sarge0_i386.deb KNOPPIX-content/root cp vzquota_3.0.9-1dso1_i386.deb KNOPPIX-content/root cp debian-3.1-i386-minimal.tar.gz KNOPPIX-content/root chroot KNOPPIX-content mount -t proc proc /proc mount -t tmpfs tmpfs /tmp # in order not to think about trash in /tmp later cd /root
In chrooted environment we need to remove all kernel related packages. In KNOPPIX 5.1.1 2.6.19 vanilla kernel is used, so we can easily identify all kernel-related packages and eliminate them using this knowledge:
dpkg-query --list | grep 2.6.19 apt-get remove aufs-cvs-modules-2.6.19 apt-get remove cloop-module-2.6.19 apt-get remove madwifi-modules-2.6.19 apt-get remove ndiswrapper-modules-2.6.19 apt-get remove rt2x00-cvs-modules-2.6.19 apt-get remove loop-aes-modules-2.6.19 apt-get remove linuxt-image-2.6.19
And now install new tools (don't be afraid about warnings: they are ok)
apt-get install linux-image-2.6.18-8.el5-028stab027_.1_i386.deb apt-get install vzquota_3.0.9-1dso1_i386.deb apt-get install vzctl_3.0.16-1dso2~sarge0_i386.deb mv debian-3.1-i386-minimal.tar.gz /var/lib/vz/template/cache/
[TODO: remove other packages to free disk space] Cleaning all up:
umount /proc umount /tmp rm -rf /root/* exit
That's greate, now we have all the tree ready, let's compress it in new image using create_compressed_fs tool:
mkisofs -R -iso-level 4 -force-rr -J ./KNOPPIX-content-modif/ > KNOPPIX-modif.img ./cloop-2.05/create_compressed_fs KNOPPIX-modif.img 65536 > KNOPPIX-modif
Now we need to compile aufs and cloop kernel modules for our new kernel. Let's start from cloop module. We uncomment one definition in source, because we're using RHEL-based kernel. The patch below shows what to do:
--- cloop-2.05/compressed_loop.c.rh 2006-10-13 23:39:41.000000000 +0400 +++ cloop-2.05/compressed_loop.c 2007-04-12 17:21:24.000000000 +0400 @@ -31,7 +31,7 @@ #endif /* Define this if you are using Greenshoe Linux */ -/* #define REDHAT_KERNEL */ + #define REDHAT_KERNEL #include <linux/kernel.h> #include <linux/version.h>
OK, let's compile cloop
wget http://debian-knoppix.alioth.debian.org/sources/cloop_2.05-1.tar.gz tar xzf cloop_2.05-1.tar.gz cd cloop-2.05/ vim compressed_loop.c # uncommenting define here make KERNEL_DIR=../linux-2.6.18-8.el5-028stab027/ module
After that you should have cloop.ko file in current directory. Later we copy it to appropriate place. What about aufs? Get this module from cvs. I use this version (20061113), because this version is used in KNOPPIX 5.1.1. The following changes are required in AUFS, because we use rhel5-based kernel:
--- aufs/fs/aufs/cpup.c.orig 2006-10-23 12:58:33.000000000 +0000
+++ aufs/fs/aufs/cpup.c 2007-04-16 14:46:51.000000000 +0000
@@ -85,9 +85,6 @@ void cpup_attr_all(struct inode *inode)
inode->i_rdev = hidden_inode->i_rdev;
}
inode->i_blkbits = hidden_inode->i_blkbits;
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
- inode->i_blksize = hidden_inode->i_blksize;
-#endif
}
/* ---------------------------------------------------------------------- */
--- aufs/fs/aufs/dir.c.orig 2006-10-23 12:59:29.000000000 +0000
+++ aufs/fs/aufs/dir.c 2007-04-16 14:58:34.000000000 +0000
@@ -223,7 +223,7 @@ struct test_empty_arg {
};
static int test_empty_cb(void *__arg, const char *__name, int namelen,
- loff_t offset, filldir_ino_t ino, unsigned int d_type)
+ loff_t offset, u64 ino, unsigned int d_type)
{
struct test_empty_arg *arg = __arg;
char *name = (void*)__name;
--- aufs/fs/aufs/vdir.c.orig 2006-10-23 13:02:20.000000000 +0000
+++ aufs/fs/aufs/vdir.c 2007-04-16 15:01:44.000000000 +0000
@@ -389,7 +389,7 @@ struct fillvdir_arg {
};
static int fillvdir(void *__arg, const char *__name, int namelen, loff_t offset,
- filldir_ino_t hidden_ino, unsigned int d_type)
+ u64 hidden_ino, unsigned int d_type)
{
struct fillvdir_arg *arg = __arg;
ino_t ino;
TODO: (FOR NEWER AUFS ONE MORE PATCH with "(deleted)"
cvs -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs login cvs -z3 -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs co -D20061113 aufs cd aufs make KDIR=../../linux-2.6.18-8.el5-028stab027/ -f local.mk
After this step you have an aufs.ko file in current directory. Next step is too create add this (and some other) modules to initrd. It'll be tomorrow ;)
[TODO] CREATE GOOD CONTENTS TABLE OF THIS PART.