Difference between revisions of "How to use OpenVZ as a XEN guest OS (for x86 platform)"
(wikification) |
m (→Installing from tarball: tar and gzip to one command) |
||
Line 31: | Line 31: | ||
# cd xen | # cd xen | ||
# wget http://bits.xensource.com/oss-xen/release/3.1.0/bin.tgz/xen-3.1.0-install-x86_32p.tgz | # wget http://bits.xensource.com/oss-xen/release/3.1.0/bin.tgz/xen-3.1.0-install-x86_32p.tgz | ||
− | # | + | # tar xvzf xen-3.1.0-install-x86_32p.tgz |
− | |||
</pre> | </pre> | ||
Run the install.sh script within ./dist folder: | Run the install.sh script within ./dist folder: |
Revision as of 09:21, 1 August 2007
Contents
Intro
Recently released RHEL5 kernel supports compatibility with XEN 3.0. It means that RHEL5 kernel built with config that enables XEN can be used as guest OS in both privileged Dom0 and unprivileged DomU XEN domains. RHEL5 - based OpenVZ kernel 028stab033 also contains support for XEN compatibility. Current article reports on how to virtualize xen DomU by using xen-compatible OpenVZ kernel el5.028stab034.
Detailed description of XEN 3.0, user guides and howtos on XEN 3.0 can be found here: http://www.xensource.com/products/xen/documentation.html. But we'll give here some common information on XEN 3.0 in purpose to make a story more clear. XEN system consists of 3 parts:
1) Xen hypervisor - lowest level part of system that virtualizes drivers and architecture dependent part of the system.
2) XenLinux kernel - guest OS that works over hypervisor in privileged domain Dom0 or unprivileged domain DomU. Privileged Dom0 domain is used for creating, destroying and supervising of unprivileged domains (DomUs). XenLinux kernel that is running in Dom0 contains 2 sets of drivers: physical and virtual. DomU XenLinux kernel can contain the only set of virtual drivers.
3) XM Tool - userland program that is used from Dom0 for creating, destroying and supervising DomUs.
As OpenVZ is an OS level virtualization solution and do not affect drivers, it is possible to run OpenVZ within XEN DomU. Text listed below guides through XEN 3.0, XM tool and OpenVZ-XenLinux PAE kernel installation on x86. Guide assumes RHEL5 Linux is used as an OS on your hardware.
Well. Login as root. Create 2 directories:
# mkdir xen # mkdir openvz
Installing XEN itself, Dom0 and XEN tools
XEN 3.1 and XM tool can be installed in 2 ways: either from tarball containing prebuilt binaries or by building from sources.
Installing from tarball
Download XEN 3.1 32 bit PAE SMP tarball from http://bits.xensource.com/ and unpack:
# cd xen # wget http://bits.xensource.com/oss-xen/release/3.1.0/bin.tgz/xen-3.1.0-install-x86_32p.tgz # tar xvzf xen-3.1.0-install-x86_32p.tgz
Run the install.sh script within ./dist folder:
# ./dist/install.sh Installing Xen from './dist/install' to '/'... - installing for udev-based system - modifying permissions All done. Checking to see whether prerequisite tools are installed... Xen CHECK-INSTALL Thu May 24 14:30:20 MSD 2007 Checking check_brctl: OK Checking check_crypto_lib: OK Checking check_iproute: OK Checking check_libvncserver: unused, OK Checking check_python: OK Checking check_python_xml: OK Checking check_sdl: unused, OK Checking check_udev: OK Checking check_zlib_lib: OK All done. #
install.sh checks the presence of the required packages and installs hypervisor xen.gz and XenLinux vmlinuz-2.6.16.33-xen to /boot and xm tool to /usr/sbin. Now update /etc/grub.conf by adding the xen entry in the following way:
/etc/grub.conf:
title Xen Linux (2.6.16.33-xen) root (hd0,1) kernel /boot/xen.gz dom0_mem=1048576 module /boot/vmlinuz-2.6.16.33-xen ro root=LABEL=/ console=ttyS0,115200 console=tty silencelevel=8
Now you can reboot the machine. Choosing the Xen Linux (2.6.16.33-xen) from grub sequence will cause to booting xen hypervisor and Dom0 running Xen Linux (2.6.16.33-xen). In case of success you'll see the usual login prompt. If kernel panic takes place during boot - reboot the machine into your usual working Linux and check if the xen related /etc/grub.conf entry is correct. If /etc/grub.conf entry is correct and xen kernel is unable to boot,- the reason is prebuild vmlinuz-2.6.16.33-xen does not contain some drivers for your hardware, that are needed to be loaded by using initrd. In this case follow instructions below - "Building from sources".
Building from sources
Download the package containing sources of Xen-3.1 and unpack it:
# wget http://bits.xensource.com/oss-xen/release/3.1.0/src.tgz/xen-3.1.0-src.tgz # gunzip xen-3.1.0-src.tgz # tar -xvf xen-3.1.0-src.tar
Change directory to ./xen-3.1.0-src:
# cd ./xen-3.1.0-src
and start building:
# make XEN_TARGET_X86_PAE=y
Building process will do the following: - build hypervizor xen.gz - download linux kernel sources 2.6.16 from kernel.org, patch them with a set of xen patches - from sources obtained build the vmlinuz-2.6.16.33-xen XenLinux kernel that will be able to boot both in Dom0 and DomU - build and install xm tool
After building is complete run ./install.sh script:
# ./install.sh
As a result vmlinuz-2.6.16.33-xen, xen.gz will be placed to boot, xm tool will be placed to /usr/sbin. Now it is time to create initrd for vmlinuz-2.6.16.33-xen. Check that /lib/modules contain recently created directory 2.6.16.33-xen. If it is, type:
# mkinitrd -f -v /boot/initrd-2.6.16.33-xen.img 2.6.16.33-xen
After initrd-2.6.16.33-xen.img is succesfully created update corresponding strings in /etc/grub.conf:
title Xen Linux (2.6.16.33-xen) root (hd0,1) kernel /boot/xen.gz dom0_mem=1048576 module /boot/vmlinuz-2.6.16.33-xen ro root=LABEL=/ console=ttyS0,115200 console=tty silencelevel=8 module /boot/initrd-2.6.16.33-xen.img
Xen 3.0 doesn't support TLS. To make xen work correct disable TLS on your machine bu renaming /lib/tls to /lib/tls.disabled:
# mv /lib/tls /lib/tls.disabled
Check that kernel command line options are correct (after ro ...) and reboot the machine. In grub loader menu select "Xen Linux (2.6.16.33-xen)". In case of success - machine boots into Xen's Dom0 that is running vmlinuz-2.6.16.33-xen XenLinux. Now it is time to prepare OpenVZ XenLinux kernel for DomU.
Installing OpenVZ XenLinux kernel
OpenVZ XenLinux kernel can be installed in 2 ways: either from rpm containing or by building from sources.
Install from rpm
Change directory to openvz and download rpm package with OpenVZ XenLinux kernel for x86:
# cd # cd openvz # wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab034.1/ovzkernel-PAE-2.6.18-8.el5.xen.028stab034.i686.rpm
Install downloaded rpm:
# rpm -ihv ovzkernel-PAE-2.6.18-8.el5.xen.028stab033.i686.rpm
If something goes wrong during installation the prebuilt kernel - you can build the OpenVZ XenLinux kernel from sources as described below.
install from sources
Download rpm with OpenVZ kernel sources:
# wget http://download.openvz.org/kernel/branches/rhel5-2.6.18/028stab034.1/ovzkernel-2.6.18-8.el5.028stab034.1.src.rpm
Unpack downloaded rpm package:
# rpm2cpio ovzkernel-2.6.18-8.el5.028stab034.1.src.rpm > ovzkernel-2.6.18-8.el5.028stab034.1.src.cpio # cpio -i < ovzkernel-2.6.18-8.el5.028stab034.1.src.cpio
Check that ./kernel-ovz.spec contains the following strings:
... # Whether to build the Xen kernels, disable if you want. %define buildxen 1 ...
If "buildxen is set to 0" - update kernel-ovz.spec using text editor, set buildxen to 1.
Prepare sources for building:
# rpmbuild -bp --define "_topdir $PWD" --define "_sourcedir $PWD" --define "_builddir $PWD" --target i686-linux kernel-ovz.spec # cd ./ovzkernel-2.6.18/linux-2.6.18.i686
Use xen config file:
# cp configs/kernel-2.6.18-i686-PAE-xen.config.ovz ./.config
Update Makefile: change the value of EXTRAVERSION from "-prep" to something better, for example "-openvzxen":
EXTRAVERSION = -openvzxen
And build the OpenVZ XenLinux:
# make oldconfig # make
After build is complete, - install modules and the kernel:
# make modules_install # make install
Check that /lib/modules contains the directory 2.6.18-openvzxen. And prepare initrd image for built OpenVZ XenLinux:
# mkinitrd -f -v --omit-scsi-modules --preload=xenblk /boot/initrd-2.6.18-openvzxen.img 2.6.18-openvzxen
OpenVZ XenLinux kernel prepared. Now OpenVZ tools are to be installed. In case you faced problem while building the kernel turn to article http://wiki.openvz.org/Kernel_build .
install OpenVZ tools
Now download and install latest versions of OpenVZ tools vzctl, vzquota that are necessary for starting/stopping VEs:
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm . # wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm . # wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm . # rpm -Uhv --nodeps vzctl-3.0.16-1.i386.rpm vzctl-lib-3.0.16-1.i386.rpm vzquota-3.0.9-1.i386.rpm
More information on installing OpenVZ tools can be found here: http://wiki.openvz.org/Quick_installation.
download OpenVZ guest template
To make it possible to create VEs download one of the precreated OpenVZ templates and place it to /vz/template/cache:
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz # mv fedora-core-5-i386-minimal.tar.gz /vz/template/cache/
Preparing guest partition and configuring XEN
Xen users manual http://www.xensource.com/products/xen/documentation.html describes a set of different ways of creating disk for DomU. Disk can be created using loopback or LVM or physical partition. Consider we have a standalone physical partition /dev/sda7 that will be used for creating DomU disk. Assume /dev/sda7 is empty and it contains ext3 filesystem.
Prepare disk with DomU guest Linux:
# mount -t ext3 /dev/sda7 /mnt # cp -a /bin /mnt # cp -a /dev /mnt # cp -a /etc /mnt # cp -a /lib /mnt # cp -a /net /mnt # cp -a /opt /mnt # cp -a /root /mnt # cp -a /sbin /mnt # cp -a /srv /mnt # cp -a /tmp /mnt # cp -a /usr /mnt # cp -a /var /mnt # cp -a /vz /mnt # mkdir /mnt/sys # mkdir /mnt/proc # mkdir /mnt/mnt # mkdir /mnt/home
Now create configuration file for starting DomU:
# touch /etc/xen/xmDomU # vim /etc/xen/xmDomU
and update xmDomU in the following way:
# Kernel image file. kernel = "/boot/vmlinuz-2.6.18-openvzxen" # Optional ramdisk. ramdisk = "/boot/initrd-2.6.18-openvzxen.img" # Initial memory allocation (in megabytes) for the new domain. # # WARNING: Creating a domain with insufficient memory may cause out of # memory errors. The domain needs enough memory to boot kernel # and modules. Allocating less than 32MBs is not recommended. memory = 256 # A name for your domain. All domains must have different names. name = "Domain-U" # Define network interfaces. vif = [ '' ] # Define the disk devices you want the domain to have access to, and # what you want them accessible as. # Each disk entry is of the form phy:UNAME,DEV,MODE # where UNAME is the device, DEV is the device name the domain will see, # and MODE is r for read-only, w for read-write. disk = [ 'phy:sda7,xvda1,w' ] # Set root device. root = "/dev/xvda1" # Sets runlevel 4. extra = "4 debug"
Update /mnt/etc/fstab in the following way:
/dev/xvda1 / ext3 defaults 1 1
Umount /dev/sda7:
# umount /mnt
Preparations completed. At this point we have everything that is necessary for cretaing DomU and starting OpenVZ XenLinux within DomU. Reboot into Xen: select Xen Linux (2.6.16.33-xen) item from grub menu. After reboot machine will be running vmlinuz-2.6.16.33-xen (see above) in Dom0 over hypervisor. Follow instructions below.
Starting OpenVZ guest in DomU
starting
Start xend daemon to make xm tool work:
# xend start
List the set of running domains:
[root@dhcp0-131 ~]# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 53.2 [root@dhcp0-131 ~]#
Now only privileged domain Dom0 is running. Create DomU:
[root@dhcp0-131 ~]# xm create -c /etc/xen/xmDomU
This call culminates by showing us the DomU login prompt. Current console is now DomU console.
Open another terminal on the machine (correspond to Dom0). And list the machines:
# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 75.0 Domain-U 1 255 1 r----- 19.5 #
Switch back to DomU console and test OpenVZ.
DomU console:
[root@localhost ~]# uname Linux localhost.localdomain 2.6.18-ovzxen-4-uxen #13 SMP Tue May 22 22:22:35 MSD 2007 i686 i686 i386 GNU/Linux [root@localhost ~]# vzctl create 134 --ostemplate fedora-core-5-i386-minimal Creating VE private area (fedora-core-5-i386-minimal) Performing postcreate actions VE private area was created [root@localhost ~]# vzlist -a VEID NPROC STATUS IP_ADDR HOSTNAME 134 - stopped - - [root@localhost ~]# vzctl start 134 Starting VE ... VE is mounted Setting CPU units: 1000 VE start in progress... [root@localhost ~]# vzlist -a VEID NPROC STATUS IP_ADDR HOSTNAME 134 - running - - [root@localhost ~]# vzctl enter 134 entered into VE 134 [root@localhost /]# ... some actions within VE 134 ... [root@localhost /]# exit exited from VE 156 [root@localhost ~]# vzctl stop 134 Stopping VE ... VE was stopped VE is unmounted [root@localhost ~]#
It works.
next steps
As OpenVZ works in DomU it is the best time for configuring network both in DomU and VEs. This process is described in http://www.xensource.com/products/xen/documentation.html (DomU) and http://wiki.openvz.org/Category:Networking (VEs). After this all of us will feel the full power of virtualization!
RHEL5 based OpenVZ in Dom0. Known problems
OpenVZ XenLinux kernel is able to work also in Dom0. It can be tested just by updating /etc/grub.conf on the hardware node. But in this case it will be impossible to start DomUs. It is a known bug and it is related not to OpenVZ, but to RHEL5 kernel. Hope it is fixed in next RHEL5 update.
Appreciates
Thanks to Sergey Ya. Korshunoff seyko@ for starting the topic and contribution.