Creating OpenVZ LiveCD
The idea of creating OpenVZ LiveCD for a long time was in the air. Such CD would be useful for various demonstrations, test-drives and some other purposes. Additionally we received reports that some problems persist in creating LiveCD based on OpenVZ kernel. In order to eliminate these problems and give users the ability to use OpenVZ technology in LiveCD manner we decided to create it. This article describes how I was doing about it. Probably there are some easier and more effective ways, so feel free to improve the article. We chose KNOPPIX as a base for our CD for two reasons. Firstly KNOPPIX is probably the most widespread LiveCD in the world. Secondly KNOPPIX was interested in integrating OpenVZ in one of its special releases. The article starts from conceptual actions required to create OpenVZ LiveCD and then step-by-step describes technical aspects. I want to note, that technical aspects depend a bit on certain version of kernel, aufs and cloop you're using. For example some small changes in code (functions argument type change and alike) are necessary. I used aufs-20061113 and cloop-2.05-1 because these versions were used on in KNOPPIX 5.1.1. 2.6.18-8.el5-028stab027 OpenVZ kernel was used.
Contents
Concepts
- Download KNOPPIX iso and get access to its content
- Extract KNOPPIX compressed image
- Somehow obtain OpenVZ Debian packages: kernel, vzctl, vzquota and template
- Uninstall previous Linux kernel and related packages from extracted tree
- Install OpenVZ Debian packages there
- Compress the tree
- Build aufs and cloop kernel modules against OpenVZ kernel
- Copy these modules in appropriate places on the disc
- Modify initrd image of KNOPPIX
- Create new ISO image
Step-by-step actions
First we need to download KNOPPIX iso image. Here is a list of mirrors where you can download it. I used KNOPPIX 5.1.1 from the nearest mirror:
mkdir openvz-livecd cd openvz-livecd wget ftp://ftp.chg.ru/pub/Linux/knoppix/KNOPPIX_V5.1.1CD-2007-01-04-EN.iso
Let's copy contents of this iso to to the hard drive:
mkdir mnt mount -o loop KNOPPIX_V5.1.1CD-2007-01-04-EN.iso mnt mkdir cd-contents cp -a mnt/* cd-contents/ umount mnt rm -r mnt
When you're booted in KNOPPIX LiveCD you can observe a lot of files and use various applications. On CD all this information is stored in a single file:
ls -lh cd-contents/KNOPPIX/KNOPPIX -rw-r--r-- 1 vass vass 691M Jan 4 03:50 cd-contents/KNOPPIX/KNOPPIX
As you can see its size is 691M, but after uncompressing it'll be approximately 2Gb. In order to uncompress this file we need special tools: cloop-utils. You can download them from this site. I used cloop-2.05-1, because exactly this version is installed on KNOPPIX 5.1.1.
wget http://debian-knoppix.alioth.debian.org/sources/cloop_2.05-1.tar.gz tar xzf cloop_2.05-1.tar.gz cd cloop_2.05-1 make extract_compressed_fs make create_compressed_fs # this tool will be required later, when compressing modified tree cd ..
After we have extract_compressed_fs tool in hand, let's extract KNOPPIX image:
cloop_2.05-1/extract_compressed_fs cd-contents/KNOPPIX/KNOPPIX > KNOPPIX.img ls -lh KNOPPIX.img -rw-rw-r-- 1 vass vass 2.0G Apr 12 17:38 KNOPPIX.img
KNOPPIX.img file can be mounted using usual '-o loop' option. Next our step is to create a copy of KNOPPIX file system tree on the hard disk. Note, that we can't use 'cp' tool here, because of hardlinks and special files. We should use rsynс:
mkdir mnt mount -o loop KNOPPIX.img mnt mkdir KNOPPIX-content rsync -Hav mnt KNOPPIX-content umount mnt rm -r mnt
Now we should set up OpenVZ environment in obtained tree: kernel, vzctl, vzquota, template. For vzctl and vzquota we can use precompiled packages from http://debian.systs.org. But at the moment we have to create kernel Debian package by self, because we need additional patches in order all to work smoothly (but it will be unnecessary soon, when patches will be included in main OpenVZ patch). This is important to use KNOPPIX default kernel config! The creators of LiveCD take a lot of care about customizing config file in such manner, that CD will suite as many users as possible. We shouldn't throw away such great job. We do `make oldconfig` and press enter all the time. Additionally some config options should be turn off at the moment. (http://bugzilla.openvz.org/show_bug.cgi?id=433): CONFIG_MATH_EMULATION, CONFIG_SJCD, CONFIG_OPTCD, CONFIG_AZTCD, CONFIG_BLK_DEV_XD, CONFIG_AUTOFS_FS, CONFIG_SND_WAVEFRONT. I created a Debian VE on my machine and install there kernel-package package, that allows to create Debian kernel packages easily. After that the creation of package is something like that:
mkdir ovz-kernel cd ovz-kernel wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2 wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab027.1/patches/patch-8.el5.028stab027.1-combined.gz wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab027.1/configs/kernel-2.6.18-i686.config.ovz wget http://download.openvz.org/~vass/diff-ubc-mm-null-pointer-for-kthread-20070406 # this patch will be not necessary soon. tar xjf linux-2.6.18.tar.bz2 zcat patch-8.el5.028stab027.1-combined.gz | patch -p1 -d linux-2.6.18 patch -p1 -d linux-2.6.18 < diff-ubc-mm-null-pointer-for-kthread-20070406 mv linux-2.6.18 linux-2.6.18-8.el5-028test027 vim linux-2.6.18-8.el5-028test027/Makefile # here we add right extraversion: -8.el5-028test027 cp linux-2.6.19 linux-2.6.18/.config cd linux-2.6.18-8.el5-028test027 make oldconfig # pressing enter here vim .config # switching off some config options make-kpkg --revision=.1 kernel_image cd .. cd ..
If everything went smoothly, in the top directory we'll be able to see linux-image file:
ls -l linux-image-2.6.18-8.el5-028stab027_.1_i386.deb -rw-r--r-- 1 vass vass 13364308 Apr 16 11:06 linux-image-2.6.18-8.el5-028stab027_.1_i386.deb
Don't delete sources and compilation of this kernel. A bit later we'll need to build aufs and cloop kernel modules against this kernel. Now I download vzctl and vzquota Debian packages from http://debian.systs.org and a template:
wget http://debian.systs.org/debian/pool/openvz/v/vzctl/vzctl_3.0.16-1dso2~sarge0_i386.deb wget http://debian.systs.org/debian/pool/openvz/v/vzquota/vzquota_3.0.9-1dso1_i386.deb wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
Copying packages to KNOPPIX tree and chroot there:
cp linux-image-2.6.18-8.el5-028stab027_.1_i386.deb KNOPPIX-content/root cp vzctl_3.0.16-1dso2~sarge0_i386.deb KNOPPIX-content/root cp vzquota_3.0.9-1dso1_i386.deb KNOPPIX-content/root cp debian-3.1-i386-minimal.tar.gz KNOPPIX-content/root chroot KNOPPIX-content mount -t proc proc /proc mount -t tmpfs tmpfs /tmp # in order not to think about trash in /tmp later cd /root
In chrooted environment we need to remove all kernel related packages. In KNOPPIX 5.1.1 2.6.19 vanilla kernel is used, so we can easily identify all kernel-related packages and eliminate them using this knowledge:
dpkg-query --list | grep 2.6.19 apt-get remove aufs-cvs-modules-2.6.19 apt-get remove cloop-module-2.6.19 apt-get remove madwifi-modules-2.6.19 apt-get remove ndiswrapper-modules-2.6.19 apt-get remove rt2x00-cvs-modules-2.6.19 apt-get remove loop-aes-modules-2.6.19 apt-get remove linux-image-2.6.19
Installing OpenVZ tools (don't be afraid about warnings: they are ok)
apt-get install linux-image-2.6.18-8.el5-028stab027_.1_i386.deb apt-get install vzquota_3.0.9-1dso1_i386.deb apt-get install vzctl_3.0.16-1dso2~sarge0_i386.deb mv debian-3.1-i386-minimal.tar.gz /var/lib/vz/template/cache/
Here I want to note one thing. We added some data to the KNOPPIX image, so its size increased. If you want to burn iso on CD/DVD later you should think of removing some packages. I miss this point, but you're aware. After all we need to clean up chrooted environment:
umount /proc umount /tmp rm -rf /root/* exit
That's great, now we have all the tree ready, let's compress it in the new image using create_compressed_fs tool and copy to the right place:
mkisofs -R -iso-level 4 -force-rr -J ./KNOPPIX-content-modif/ > KNOPPIX-modif.img ./cloop-2.05/create_compressed_fs KNOPPIX-modif.img 65536 > KNOPPIX-modif cp KNOPPIX-modif cd-contents/KNOPPIX/KNOPPIX
The next our step is to compile aufs and cloop kernel modules against OpenVZ kernel. Let's start from cloop module. A definition in source must be commented, because we're using RHEL-based kernel. The patch below shows what to do:
--- cloop-2.05/compressed_loop.c.rh 2006-10-13 23:39:41.000000000 +0400 +++ cloop-2.05/compressed_loop.c 2007-04-12 17:21:24.000000000 +0400 @@ -31,7 +31,7 @@ #endif /* Define this if you are using Greenshoe Linux */ -/* #define REDHAT_KERNEL */ + #define REDHAT_KERNEL #include <linux/kernel.h> #include <linux/version.h>
Getting the sources, building:
wget http://debian-knoppix.alioth.debian.org/sources/cloop_2.05-1.tar.gz tar xzf cloop_2.05-1.tar.gz cd cloop-2.05/ vim compressed_loop.c # uncommenting define here make KERNEL_DIR=../linux-2.6.18-8.el5-028stab027/ module
After that you should have cloop.ko file in current directory. Later we will copy it to appropriate place. What about aufs? Get the sources from cvs. I used 20061113 version, because this version is used in KNOPPIX 5.1.1. The following changes are required in aufs, because we're using rhel5-based kernel:
--- aufs/fs/aufs/cpup.c.orig 2006-10-23 12:58:33.000000000 +0000
+++ aufs/fs/aufs/cpup.c 2007-04-16 14:46:51.000000000 +0000
@@ -85,9 +85,6 @@ void cpup_attr_all(struct inode *inode)
inode->i_rdev = hidden_inode->i_rdev;
}
inode->i_blkbits = hidden_inode->i_blkbits;
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
- inode->i_blksize = hidden_inode->i_blksize;
-#endif
}
/* ---------------------------------------------------------------------- */
--- aufs/fs/aufs/dir.c.orig 2006-10-23 12:59:29.000000000 +0000
+++ aufs/fs/aufs/dir.c 2007-04-16 14:58:34.000000000 +0000
@@ -223,7 +223,7 @@ struct test_empty_arg {
};
static int test_empty_cb(void *__arg, const char *__name, int namelen,
- loff_t offset, filldir_ino_t ino, unsigned int d_type)
+ loff_t offset, u64 ino, unsigned int d_type)
{
struct test_empty_arg *arg = __arg;
char *name = (void*)__name;
--- aufs/fs/aufs/vdir.c.orig 2006-10-23 13:02:20.000000000 +0000
+++ aufs/fs/aufs/vdir.c 2007-04-16 15:01:44.000000000 +0000
@@ -389,7 +389,7 @@ struct fillvdir_arg {
};
static int fillvdir(void *__arg, const char *__name, int namelen, loff_t offset,
- filldir_ino_t hidden_ino, unsigned int d_type)
+ u64 hidden_ino, unsigned int d_type)
{
struct fillvdir_arg *arg = __arg;
ino_t ino;
Building aufs:
cvs -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs login cvs -z3 -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs co -D20061113 aufs cd aufs vim fs/aufs/cpup.c # vim fs/aufs/dir.c # Introducing required changes vim fs/aufs/vdir.c # make KDIR=../../linux-2.6.18-8.el5-028stab027/ -f local.mk
Now you have aufs.ko file in current directory. Next we need to add these modules to the disc:
cp aufs.ko cloop.ko cd-contents/KNOPPIX/modules/
When booting KNOOPIX uses initrd image,
where it stores some kernel modules required to get access to the CD-drive.
We need to remove old modules and add new one to initrd image. Extracting and
mounting initrd:
mkdir initrd cd initrd cp cd-contents/boot/isolinux/minirt.gz ./ gunzip minirt.gz mkdir mnt mount -o loop minirt mnt
The modules directory on initrd contents kernel modules. Now we should replace these modules by equivalents from OpenVZ kernel. Note, that some modules can missing, because we use other kernel. Just remove modules, that are missing. After this operation is done copy new minirt.gz to the appropriate place:
umount mnt gzip minirt cp minirt.gz ../cd-contents/boot/isolinux/ cd ..
The last step is to copy vmlinuz to isolinux directory:
cp vmlinuz-2.6.18-8.el5-028stab027 cd-contents/boot/isolinux/linux
Wow! We're ready to create iso with KNOPPIX OpenVZ CD:
chmod -R u+w cd-contents rm cd-contents/boot/isolinux/boot.cat mkisofs -b boot/isolinux/isolinux.bin -o KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso -no-emul-boot \ -l -J -r -boot-load-size 4 -boot-info-table KNOPPIX_V5.1.1CD-2007-01-04-EN.iso-content
Result
[todo]
