OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Migration fails

2007-12-02T22:08:01Z

Vass: /* Workaround */ formatting fix

This article contents known reasons of migration failures.

== Problem #1 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Problem #2 ==
=== Symptoms ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
* for 2.6.18 rhel5 based kernels: Update till <tt>028stab035</tt> or newer kernel
* for 2.6.18 mainstream based kernels: Update till <tt>028stab043</tt> or newer kernel

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Problem #3 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE till <tt>2.6.18-028stab049</tt> and <tt>2.6.9-023stab044.14</tt> kernels. This device is usually used for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
* for 2.6.18 based kernels: update till <tt>028stab049</tt> or newer kernel
* for 2.6.9 based kernels: update till <tt>023stab044.14</tt> or newer kernel

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

Migration fails

2007-12-02T22:06:02Z

Vass: /* Reason */ formatting fix

This article contents known reasons of migration failures.

== Problem #1 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Problem #2 ==
=== Symptoms ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
* for 2.6.18 rhel5 based kernels: Update till <tt>028stab035</tt> or newer kernel
* for 2.6.18 mainstream based kernels: Update till <tt>028stab043</tt> or newer kernel

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Problem #3 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE till <tt>2.6.18-028stab049</tt> and <tt>2.6.9-023stab044.14</tt> kernels. This device is usually used for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
Just use 2.6.18-028stab049 or 2.6.9-023stab044.14 (and later) OpenVZ kernels.

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

Migration fails

2007-12-02T22:04:09Z

Vass: /* Workaround */ TUN is now supported.

This article contents known reasons of migration failures.

== Problem #1 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Problem #2 ==
=== Symptoms ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
* for 2.6.18 rhel5 based kernels: Update till <tt>028stab035</tt> or newer kernel
* for 2.6.18 mainstream based kernels: Update till <tt>028stab043</tt> or newer kernel

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Problem #3 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE till 028stab049 kernel. This device is usually used
for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
Just use 2.6.18-028stab049 or 2.6.9-023stab044.14 (and later) OpenVZ kernels.

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

Migration fails

2007-12-02T22:02:22Z

Vass: /* Reason */ TUN is supported now!

This article contents known reasons of migration failures.

== Problem #1 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Problem #2 ==
=== Symptoms ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
* for 2.6.18 rhel5 based kernels: Update till <tt>028stab035</tt> or newer kernel
* for 2.6.18 mainstream based kernels: Update till <tt>028stab043</tt> or newer kernel

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Problem #3 ==
=== Symptoms ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE till 028stab049 kernel. This device is usually used
for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
You should not use <tt>tun</tt> device in VE to migrate it. If you're using VPN in VE, just stop VPN, then produce the online migration and after that start the VPN. Of course, all current connections to VPN will be lost :(.

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

UBC primary parameters

2007-09-05T02:30:30Z

Vass: /* vmguarpages */ x86_64 page size added

{{UBC toc}}

The most important parameters determining the resources available to
Virtual Environment are explained below. The meaning of the parameters
is illustrated assuming that the Virtual Environment runs some network
server applications.

== numproc ==
Maximum number of processes and kernel-level threads allowed for this Virtual Environment.

Many server applications (like Apache Web server, FTP and mail servers)
spawn a process to handle each client, so the limit on number of processes
defines how many clients the application will be able to handle in parallel.
However, the number of processes doesn't limit how “heavy” the application is
and whether the server will be able to serve heavy requests from clients.

Configuring resource control system, it is important to estimate both
the maximum number of processes and the average number of processes
(referred to as avnumproc later). Other (dependent) resource
control parameters may depend both on the limit and the average number (see [[UBC consistency check]]).

Barrier of numproc doesn't provide additional control and should be set equal to the limit.

There is a restriction on the total number of processes in the system.
More than about 16000 processes start to cause poor responsiveness of
the system, worsening when the number grows. Total number of processes
exceeding 32000 is very likely to cause hang of the system.

Note that in practice the number of processes is usually less. Each
process consumes some memory, and the available memory and the
"low memory" (see [[UBC systemwide configuration#“Low memory”|“Low memory”]]) limit the number of processes to lower
values. With typical processes, it is normal to be able to run only up
to 8000 processes in a system.

== numtcpsock ==
Maximum number of TCP sockets.

This parameter limits the
number of TCP connections and, thus, the number of clients the server
application can handle in parallel.
Barrier of this parameter should be set equal to the limit.
If each Virtual Environment has it's own set of IP addresses (which is
the only way a OpenVZ system can be configured), there are no direct
limits on the total number of TCP sockets in the system. The number
of sockes needs to be controlled because each socket needs certain
amount of memory for receive and transmit buffers (see descriptions
of [[<code>tcpsndbuf</code>]] and [[<code>tcprcvbuf</code>]]), and
the memory is a limited resource.

== numothersock ==
Maximum number of non-TCP sockets (local sockets, UDP and other types of sockets).

Local (UNIX-domain) sockets are used for communications inside the
system. Multi-tier applications (for example, a Web application with a
database server as a back-end) may need one or multiple local sockets
to serve each client. Straightforward applications (for example, most
mail servers, with the exception of Postfix) do not use local sockets.

UDP sockets are used for Domain Name Service (DNS) queries, but
the number of such sockets opened simultaneously is low. UDP and
other sockets may also be used in some very special applications
(SNMP agents and others).

<code>Barrier</code> of this parameter should be set equal to the <code>limit</code>.
The number of local sockets in a system is not limited. The number of
UDP sockets in a system, similarly to TCP sockets, is not limited in
OpenVZ systems.

Similarly to <code>numtcpsock</code> parameter discussed above, the number of
non-TCP sockets needs to be controlled because each socket needs certain
amount of memory for its buffers, and the memory is a limited
resource.

== vmguarpages ==
Memory allocation guarantee.

This parameter controls how much memory is available to the Virtual
Environment (i.e. how much memory its applications can allocate by
<code>malloc(3)</code> or other standard Linux memory allocation mechanisms).
The more clients are served or the more “heavy” the application is, the
more memory it needs.

The amount of memory that Virtual Environment's applications are
guaranteed to be able to allocate is specified as the barrier of
<code>vmguarpages</code> parameter. The current amount of allocated memory space
is accounted into <code>privvmpages</code> parameter, and <code>vmguarpages</code>
parameter does not have its own accounting. The barrier and the limit of
<code>privvmpages</code> parameter impose an upper limit on the memory allocations
(see [[privvmpages]]). The meaning of the <code>limit</code> for the
<code>vmguarpages</code> parameter is unspecified in the current version and should be set
to the maximal allowed value ([[MAX_ULONG]]).

If the current amount of allocated memory space does not exceed the
guaranteed amount (the <code>barrier</code> of <code>vmguarpages</code>),
memory allocations of Virtual Environment's applications always succeed.
If the current amount of allocated memory space exceeds the guarantee but below
the <code>barrier</code> of <code>privvmpages</code>, allocations may or may not succeed,
depending on the total amount of available memory in the system.

Starting from the <code>barrier</code> of <code>privvmpages</code>,
normal priority allocations and, starting from the <code>limit</code>
of <code>privvmpages</code>, all memory allocations
made by the applications fail.
The memory allocation guarantee (<code>vmguarpages</code>) is a primary tool for
controlling the memory available to Virtual Environments, because
it allows administrators to provide Service Level Agreements — agreements
guaranteeing certain quality of service, certain amount of resources
and general availability of the service. The unit of measurement
of vmguarpages values is memory pages (4KB on x86 and x86_64 processors).
The total memory allocation guarantees given to Virtual Environments
are limited by the physical resources of the computer — the size of RAM
and the swap space — as discussed in [[UBC systemwide configuration]].

Resource management

2007-09-05T01:24:38Z

Vass:

'''OpenVZ resource management''' is a set of controls to provide per-[[VE]] limits and guarantees in a way that multiple VEs will coexist on a single physical server. It consists three primary controllers:

* [[User beancounters]]
* [[OpenVZ disk quota system]]
* [[CPU Fairscheduler]]
* [[Traffic shaping with tc]]

FIXME: add more text
FIXME: create a right-hand navigation menu for all of the resource management stuff

[[Category:Definitions]]
[[Category:Kernel]]

Installing udev in VE based on Debian-4.0

2007-07-12T08:06:35Z

Vass: Initial edition of the article

When you install <tt>udev</tt> package in [[VE]] based on Debian-4.0 template, you can see the following message:
<pre>
Setting up udev (0.105-4) ...
/var/lib/dpkg/info/udev.postinst: line 125: echo: write error: Operation not permitted
</pre>

The problem is that postinstall script tries to write to <tt>/proc/sys/kernel/hotplug</tt> file, and this operation is prohibited in VE.
To work around this problem, just comment the 125th line in <tt>/var/lib/dpkg/info/udev.postinst</tt> file, that writes to <tt>/proc/sys/kernel/hotplug</tt> file. After that <tt>udev</tt> can be installed smoothly.

P.S. Don't forget to enable <tt>sysfs</tt> feature for VE in question (by vzctl), in order <tt>udev</tt> to be able to work.

[[Category:Troubleshooting]]

Migration fails

2007-07-12T07:52:27Z

Vass: /* Workaround */ add more detailed workaround: stop VPN

This article contents known reasons of migration failures.

== Reason #1 ==
=== Symptomps ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Reason #2 ==
=== Symptomps ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
Update till <tt>028stab035</tt> or newer kernel.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Reason #3 ==
=== Symptomps ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE, which is usually used
for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
You should not use <tt>tun</tt> device in VE to migrate it. If you're using VPN in VE, just stop VPN, then produce the online migration and after that start the VPN. Of course, all current connections to VPN will be lost :(.

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

Migration fails

2007-07-11T12:48:54Z

Vass: reason 3 added: tun device in VE

This article contents known reasons of migration failures.

== Reason #1 ==
=== Symptomps ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Error: undump failed: Invalid argumentStopping VE ...
Restoring failed:
iptables-restore exited with 1
rst_restore_net: -22
</pre>

=== Reason ===
The VE which you wish to migrate uses some iptables modules, that are not
loaded on destination node. Consequently <tt>iptables-restore</tt> fails.

=== Workaround ===
Load all iptables kernel modules that are used by VE on destination node.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=631
* http://forum.openvz.org/index.php?t=tree&th=2756&mid=14198&&rev=&reveal=

== Reason #2 ==
=== Symptomps ===
The system runs <tt>028stab031</tt> or older kernels and
after giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Live migrating VE
Suspending VE
Setting up checkpoint...
suspend...
Can not suspend VE: Device or resource busy
file on unsupported FS: magic 0bad1dea
Checkpointing failed
Error: Failed to suspend VE
</pre>

=== Reason ===
The VE which you wish to migrate uses inotify mechanism (magic 0bad1dea), that is not
supported by <tt>028stab031</tt> and older kernels.

=== Workaround ===
Update till <tt>028stab035</tt> or newer kernel.

=== Links ===

* http://bugzilla.openvz.org/show_bug.cgi?id=552
* http://forum.openvz.org/index.php?t=tree&goto=14780&#msg_14780

== Reason #3 ==
=== Symptomps ===
After giving <tt>vzmigrate -v --online <ip address> <veid></tt>
the following message appears:
<pre>
Dumping VE
Setting up checkpoint...
join context..
dump...
Can not dump VE: Device or resource busy
unsupported netdevice tun0
Checkpointing failed
Error: Failed to dump VE
</pre>

=== Reason ===
Checkpointing doesn't support <tt>tun</tt> device inside VE, which is usually used
for [[VPN via the TUN/TAP device]] in VE.

=== Workaround ===
You should not use <tt>tun</tt> device in VE to migrate it.

=== Links ===
* http://bugzilla.openvz.org/show_bug.cgi?id=642
* http://forum.openvz.org/index.php?t=tree&goto=14826&#msg_14826

[[Category:Troubleshooting]]

Testimonials

2007-07-11T09:34:44Z

Vass: one more added

'''Following are some comments we've received from OpenVZ users.'''

----

I just wanted to congratulate you and the rest of the OpenVZ team for your
excellent, excellent work. We are now have an OpenVZ subject in our 120
hours Total Linux course here at the Bluepoint Foundation, and our
students really appreciate the work you've done. I'm learning a lot from
lurking in the devel list too. Thanks again and keep 'em coming! :D

''Engels Antonio, [http://bluepoint.com.ph/ Bluepoint Foundation]''

----

I would say, OpenVZ is '''United States of Linux''' because i can run all my lovely different type of GNU/Linux distributions(States) on '''one single machine''' with '''one kernel''' model. As stable as guaranteed for production server and handy for software development (well no more dual, tri, quad boot partition). After all, it is just a good quality software piece for Linux User. Again, welcome to '''United States of Linux''' world!!

''Victor, System Admin, [http://www.kholix.com kholix.com]''

----

Thanks for producing OpenVZ, we use it ourselves at PlanetMirror and
find that it's fantastic.

''Robert McLeay, PlanetMirror.com stuff''

----

OpenVZ has me seriously impressed, I was looking to try virtualisation technology and got OpenVZ up and running in only 3 trips to the coffee machine! ''-barf''

----

Hello all, just downloaded and installed OpenVZ, and i must say its a big improvement over other VPS systems that i have tested IMHO.
[http://forum.openvz.org/index.php?t=tree&goto=646&#msg_646]

----

Virtuozzo and openvz are wonderful - I don't know why more people aren't
using them. I hear a lot of hype for xen and usermode but
virtuozzo/openvz is so great for many common needs.
[http://forum.openvz.org/index.php?t=rview&goto=650#msg_650]

----

For my needs, OpenVZ is better than Xen. The one-kernel approach conserves memory, leaving more for applications. And having all VPS in one disk partition saves disk space. A surprise bonus was the template cache management with yum. The ease of keeping templates updated and quickly installing new operating environments is yummy!
[http://forum.openvz.org/index.php?t=rview&goto=3119#msg_3119]

----

Last week when we were in limbo about what to do, it was decided to try out XEN Virtualization. From what is written in the press the Xen system has alot of promise, Features such as opensource with live migration and backups sounds great; but was far too complicated to get working in our configuration. OpenVZ was the only virtual server system that was simple to install and get working.
[http://forum.openvz.org/index.php?t=msg&goto=568#msg_568]

----

It still amazes me how well OpenVZ works.
[http://forum.openvz.org/index.php?t=msg&th=368&#msg_2086]

----

[03:30:15] <pookey> well, I've been using openvz for all of about an hour, and I'm pretty impressed so far :)<br/>
''from #openvz IRC channel''

----

I am playing around for years with all major virtualization environments like all VMware products, Microsoft and several xen based solutions. But OpenVZ is the overall winner. [http://blog.openvz.org/14313.html]

----

I messed around with VMWare and Xen. I found VMWare to be somewhat over kill and Xen was damned hard to get working. The user community on the Xen mailing list is rather hostile. It looked like A number of people hang out there for the sole purpose of flaming the clueless.

I've found the OpenVZ folks to be very helpful and knowledgeable. OpenVZ is simple to install and get running. It pretty much satisfies my needs. [http://blog.openvz.org/14313.html?thread=17897#t17897]

----

I'm a sysadm and have always used xen/qemu/virtualbox/vmware server for my VMs and my customer's ones. Yesterday I decided to give OpenVZ a try and... I'm amazed. It's fast, easy and reliable.Live migration works like a charm and the use of rsync assures low data transfers. Great. [http://forum.openvz.org/index.php?t=tree&goto=14724&#msg_14724]

Migration fails

2007-07-11T07:53:48Z

Vass: second reason added: inotidy in VE

Download/live CD

2007-07-10T12:20:39Z

Vass: /* Downloading */

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.18
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=684 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=68 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-based CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt> or <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to <tt>/vz/private/</tt> for CentOS-based CD and to <tt>/var/lib/vz/private</tt> for KNOPPIX-based CD.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}.
You can work around it by mounting some filesystem (<tt>ext2</tt>, <tt>ext3</tt>, <tt>tmpfs</tt>) to <tt>/var/lib/vz/private</tt> directory.
On CentOS-based LiveCD <tt>tmpfs</tt> is mounted on this directory automatically during the booting.

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T09:52:26Z

Vass: /* CentOS LiveCD contents */ vzctl -> 3.0.18

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.18
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-based CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt> or <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to <tt>/vz/private/</tt> for CentOS-based CD and to <tt>/var/lib/vz/private</tt> for KNOPPIX-based CD.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}.
You can work around it by mounting some filesystem (<tt>ext2</tt>, <tt>ext3</tt>, <tt>tmpfs</tt>) to <tt>/var/lib/vz/private</tt> directory.
On CentOS-based LiveCD <tt>tmpfs</tt> is mounted on this directory automatically during the booting.

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-06T09:20:54Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ removing modules from initrd

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
I also delete some not necessary kernel modules from initrd to free space.
They are stil on CD, so init sctipts will be able to load them later.

initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-06T08:54:56Z

Vass: /* Peculiarities of building 2.6.20 */ added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* Only <tt>unionfs</tt> <tt>2.0</tt> is available for 2.6.20 kernel and it is distributed as a patch. Consequently modify <tt>spec</tt>-file and configs while building the kernel
* <tt>unionfs</tt> <tt>2.0</tt> doesn't support unionctl. Consequently you have to modify <tt>liblinuxlive</tt> file on <tt>initrd</tt> image to mount all branches at once, but don't add them by ioctls.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Getting started with OpenVZ live CD

2007-07-06T07:12:05Z

Vass: moved information about shmpages to appropriate place

This article is written for OpenVZ LiveCD and assumes that the reader only starts using OpenVZ. ([[Download live CD]])

== Introduction ==
So, as you probably know, OpenVZ allows the user to create [[VE]]s, or Virtual Environments, which seem very much
like real computers. Real computer can run various distributions: Debian, Gentoo, Red Hat and Novell products, etc.
In the same way, a VE can be based on various [[OS template|OS (Operating System) templates]]. On the LiveCD only few minimal OS templates are installed because of disk space limit. Each VE is identified by its number -- a '''VEID'''.

== VE creation ==
So, how to create a VE with VEID of 101 based on Debian template? Very easy. Just type the following commands in your
terminal (you must be root):
<pre>
# vzctl create 101 --ostemplate debian-3.1-i386-minimal
Creating VE private area (debian-3.1-i386-minimal)
Performing postcreate actions
VE private area was created
</pre>

'''vzctl''' is the tool that manages VEs. Look in <tt>/vz/template/cache/</tt> (CentOS LiveCD)
or in <tt>/var/lib/vz/template/cache/</tt> (KNOPPIX LiveCD) directories for other OS templates available on LiveCD:
<pre>
# ls -1 /var/lib/vz/template/cache/
centos-4-i386-minimal.tar.gz
debian-3.1-i386-minimal.tar.gz
fedora-core-5-i386-minimal.tar.gz
</pre>

== List of VEs ==
You can get the list of all created VEs on '''HN''' (Hardware Node) using '''vzlist''' command:
<pre>
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 - stopped - -
</pre>

As you see, VE 101 is in stopped state now.

== Starting VE ==
Let's start it:
<pre>
# vzctl start 101
Starting VE ...
VE is mounted
Setting CPU units: 1000
VE start in progress...
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 5 running -
</pre>

== Executing commands in VE ==
From the "vzlist" command you see that 5 processes are running inside VE 101. (The "NPROC" field indicates the number of Processes, or PIDs, that are active in the VE -- not the number of Processors, or CPUs.) Being on usual [[hardware node]] you can use <code>ps</code> command to identify those, and the same command can be used here. The only difference is that this command should be called inside VE.

In order to perform any command inside VE `vzctl exec` is used:
<pre>
# vzctl exec 101 ps
PID TTY TIME CMD
1 ? 00:00:00 init
7672 ? 00:00:00 rc
7674 ? 00:00:00 S10sysklogd
7677 ? 00:00:00 syslogd
7678 ? 00:00:00 syslogd
7683 ? 00:00:00 ps
</pre>

== Entering VE ==
Any self-respected OS provides a shell for the user. This is how you can get the VE's shell:
<pre>
# vzctl enter 101
entered into VE 101
#
</pre>

In this shell you can do almost all you can do on the real HN. For example create a new user:
<pre>
# useradd new-user
# passwd new-user
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
# mkdir /home/new-user
# chown new-user /home/new-user/
# su new-user
$ cd ~
$ pwd
/home/new-user
exit
#
</pre>

In order to exit from VEs shell, just type exit:
<pre>
# exit
logout
exited from VE 101
#
</pre>

== Setting up VE networking ==
Let's set up networking in VE.

<pre>
# echo 1 > /proc/sys/net/ipv4/ip_forward
# ifconfig venet0 up
# vzctl set 101 --ipadd 10.1.1.1 --save
Adding IP address(es): 10.1.1.1
Saved parameters for VE 1
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 4 running 10.1.1.1 -
</pre>

Now your [[Hardware Node]] can ping VE and VE can ping HN:
<pre>
# ping 10.1.1.1
PING 10.1.1.1 (10.1.1.1) 56(84) bytes of data.
64 bytes from 10.1.1.1: icmp_seq=1 ttl=64 time=3.80 ms

--- 10.1.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.804/3.804/3.804/0.000 ms
#
# vzctl exec 101 ping 192.168.0.244
PING 192.168.0.244 (192.168.0.244) 56(84) bytes of data.
64 bytes from 192.168.0.244: icmp_seq=1 ttl=64 time=0.508 ms

#
</pre>

However, it is not possible to ping other computers in the network: for it we need tot
set up NAT (Network Address Translation) and set the nameserver.

Assume that you've set up network on HN (for example via DHCP) and the IP address
of your node is 192.168.0.244 and nameserver IP address is 192.168.1.1.
<pre>
# iptables -t nat -A POSTROUTING -s 10.1.1.1 -o eth0 -j SNAT --to 192.168.0.244
# vzctl set 101 --nameserver 192.168.1.1 --save
File resolv.conf was modified
Saved parameters for VE 101
# vzctl exec 101 ping google.com
PING google.com (64.233.167.99) 56(84) bytes of data.
64 bytes from py-in-f99.google.com (64.233.167.99): icmp_seq=1 ttl=241 time=23.0 ms
</pre>

== Installing software inside VE ==
I guess you've noted that there is not so many packages in VE. It is because minimal template was used.
But of course, you can install any software in VE by yourself. For example, in Debian usual <tt>apt-get</tt> tool can be used.

Now, for example, we can install gcc inside VE 101 for developing purposes:
<pre>
# vzctl enter 101
entered into VE 101
#
# apt-get install gcc
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
binutils cpp cpp-3.3 gcc-3.3
Suggested packages:
binutils-doc cpp-doc make manpages-dev autoconf automake libtool flex bison gdb gcc-doc gcc-3.3-doc
Recommended packages:
libc-dev libc6-dev
The following NEW packages will be installed:
binutils cpp cpp-3.3 gcc gcc-3.3
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 5220kB of archives.
After unpacking 13.6MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.freenet.de stable/main binutils 2.15-6 [2221kB]
Get:2 http://ftp.freenet.de stable/main cpp-3.3 1:3.3.5-13 [1393kB]
Get:3 http://ftp.freenet.de stable/main cpp 4:3.3.5-3 [29.6kB]
Get:4 http://ftp.freenet.de stable/main gcc-3.3 1:3.3.5-13 [1570kB]
Get:5 http://ftp.freenet.de stable/main gcc 4:3.3.5-3 [4906B]
Fetched 5220kB in 10s (507kB/s)
Selecting previously deselected package binutils.
(Reading database ... 7436 files and directories currently installed.)
Unpacking binutils (from .../binutils_2.15-6_i386.deb) ...
Selecting previously deselected package cpp-3.3.
Unpacking cpp-3.3 (from .../cpp-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package cpp.
Unpacking cpp (from .../cpp_4%3a3.3.5-3_i386.deb) ...
Selecting previously deselected package gcc-3.3.
Unpacking gcc-3.3 (from .../gcc-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package gcc.
Unpacking gcc (from .../gcc_4%3a3.3.5-3_i386.deb) ...
Setting up binutils (2.15-6) ...

Setting up cpp-3.3 (3.3.5-13) ...
Setting up cpp (3.3.5-3) ...
Setting up gcc-3.3 (3.3.5-13) ...
Setting up gcc (3.3.5-3) ...

# exit
logout
exited from VE 101
#
</pre>

{{ Note|In the LiveCD environment, you may have to increase '''shmpages''' resource limit/barrier for the VE (read the next section) or you will run out of "disk space" when trying to install software }}

== Resource limiting ==
The very important feature of VE is that you can limit it by resources: CPU, memory, disk space.
It is also performed via vzctl. For example to set '''shmpages''' (shared memory pages) barrier:limit you
should give this command:
<pre>
vzctl set 101 --shmpages 16384:16384 --save
</pre>
This will give VE 101 64MB of '''shmpages''' (one page equals 4Kb on i386: 4Kb * 16384 = 64Mb)

Current usage values and limits of memory-related resources can be viewed through
<code>/proc/bc/VEID/resources</code> file:
<pre>
# cat /proc/bc/101/resources # or /proc/user_beancounters on 2.6.9 kernels
kmemsize 628209 976969 2752512 2936012 0
lockedpages 0 0 32 32 0
privvmpages 5238 6885 49152 53575 0
shmpages 5012 5014 8192 8192 0
numproc 3 11 65 65 0
physpages 5084 6020 0 2147483647 0
vmguarpages 0 0 6144 2147483647 0
oomguarpages 5084 6020 6144 2147483647 0
numtcpsock 0 2 80 80 0
numflock 1 5 100 110 0
numpty 0 1 16 16 0
numsiginfo 0 6 256 256 0
tcpsndbuf 0 4440 319488 524288 0
tcprcvbuf 0 42180 319488 524288 0
othersockbuf 2220 6660 132096 336896 0
dgramrcvbuf 0 2220 132096 132096 0
numothersock 1 6 80 80 0
dcachesize 0 0 1048576 1097728 0
numfile 106 339 2048 2048 0
numiptent 10 10 128 128 0
#
</pre>
First column is resource name, second is current usage, third is peak usage, forth and fifth are barrier and limit, and last column is fail counter.

Note that if you have nonzero values in the last column, it means that this VE
experienced a resource shortage. This is very common reason why some application fail to
work in a VE. In this case you should increase limits/barriers accordingly; see
[[resource shortage]] for more info.

== Stopping/removing VE ==
Well, let's stop VE and destroy it:
<pre>
# vzctl stop 101
Stopping VE ...
VE was stopped
VE is unmounted
# vzctl destroy 101
Destroying VE private area: /var/lib/vz/private/101
VE private area was destroyed
#
</pre>

== Links ==
That's all you need to start playing with OpenVZ. Additional information can be found in man page on vzctl and at http://wiki.openvz.org/ .

If you experience some difficulties, contact us via http://forum.openvz.org/ . Templates and other tools are available from http://download.openvz.org/ .

Getting started with OpenVZ live CD

2007-07-06T06:50:52Z

Vass: /* VE creation */ centos case added

This article is written for OpenVZ LiveCD and assumes that the reader only starts using OpenVZ. ([[Download live CD]])

== Introduction ==
So, as you probably know, OpenVZ allows the user to create [[VE]]s, or Virtual Environments, which seem very much
like real computers. Real computer can run various distributions: Debian, Gentoo, Red Hat and Novell products, etc.
In the same way, a VE can be based on various [[OS template|OS (Operating System) templates]]. On the LiveCD only few minimal OS templates are installed because of disk space limit. Each VE is identified by its number -- a '''VEID'''.

== VE creation ==
So, how to create a VE with VEID of 101 based on Debian template? Very easy. Just type the following commands in your
terminal (you must be root):
<pre>
# vzctl create 101 --ostemplate debian-3.1-i386-minimal
Creating VE private area (debian-3.1-i386-minimal)
Performing postcreate actions
VE private area was created
</pre>

'''vzctl''' is the tool that manages VEs. Look in <tt>/vz/template/cache/</tt> (CentOS LiveCD)
or in <tt>/var/lib/vz/template/cache/</tt> (KNOPPIX LiveCD) directories for other OS templates available on LiveCD:
<pre>
# ls -1 /var/lib/vz/template/cache/
centos-4-i386-minimal.tar.gz
debian-3.1-i386-minimal.tar.gz
fedora-core-5-i386-minimal.tar.gz
</pre>

== List of VEs ==
You can get the list of all created VEs on '''HN''' (Hardware Node) using '''vzlist''' command:
<pre>
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 - stopped - -
</pre>

As you see, VE 101 is in stopped state now.

== Starting VE ==
Let's start it:
<pre>
# vzctl start 101
Starting VE ...
VE is mounted
Setting CPU units: 1000
VE start in progress...
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 5 running -
</pre>

o Note: In the LiveCD environment, you may have to increase SHMPAGES for the VE or you will run out of "disk space" when trying to install software. You can do this "on the fly" by issuing:

<pre>
' vzctl set 101 --shmpages $((8192*8)):$((8192*8)) --save '
</pre>

o This will give VE 101 64MB of shmpages; you may wish to give it more if you're planning on doing more than just basic openvz testing.

o You can verify the change from the host by issuing:

<pre>
' vzctl exec 101 cat /proc/user_beancounters '
</pre>

== Executing commands in VE ==
From the "vzlist" command you see that 5 processes are running inside VE 101. (The "NPROC" field indicates the number of Processes, or PIDs, that are active in the VE -- not the number of Processors, or CPUs.) Being on usual [[hardware node]] you can use <code>ps</code> command to identify those, and the same command can be used here. The only difference is that this command should be called inside VE.

In order to perform any command inside VE `vzctl exec` is used:
<pre>
# vzctl exec 101 ps
PID TTY TIME CMD
1 ? 00:00:00 init
7672 ? 00:00:00 rc
7674 ? 00:00:00 S10sysklogd
7677 ? 00:00:00 syslogd
7678 ? 00:00:00 syslogd
7683 ? 00:00:00 ps
</pre>

== Entering VE ==
Any self-respected OS provides a shell for the user. This is how you can get the VE's shell:
<pre>
# vzctl enter 101
entered into VE 101
#
</pre>

In this shell you can do almost all you can do on the real HN. For example create a new user:
<pre>
# useradd new-user
# passwd new-user
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
# mkdir /home/new-user
# chown new-user /home/new-user/
# su new-user
$ cd ~
$ pwd
/home/new-user
exit
#
</pre>

In order to exit from VEs shell, just type exit:
<pre>
# exit
logout
exited from VE 101
#
</pre>

== Setting up VE networking ==
Let's set up networking in VE.

<pre>
# echo 1 > /proc/sys/net/ipv4/ip_forward
# ifconfig venet0 up
# vzctl set 101 --ipadd 10.1.1.1 --save
Adding IP address(es): 10.1.1.1
Saved parameters for VE 1
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 4 running 10.1.1.1 -
</pre>

Now your [[Hardware Node]] can ping VE and VE can ping HN:
<pre>
# ping 10.1.1.1
PING 10.1.1.1 (10.1.1.1) 56(84) bytes of data.
64 bytes from 10.1.1.1: icmp_seq=1 ttl=64 time=3.80 ms

--- 10.1.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.804/3.804/3.804/0.000 ms
#
# vzctl exec 101 ping 192.168.0.244
PING 192.168.0.244 (192.168.0.244) 56(84) bytes of data.
64 bytes from 192.168.0.244: icmp_seq=1 ttl=64 time=0.508 ms

#
</pre>

However, it is not possible to ping other computers in the network: for it we need to
set up NAT (Network Address Translation) and set the nameserver.

Assume that you've set up network on HN (for example via DHCP) and the IP address
of your node is 192.168.0.244 and nameserver IP address is 192.168.1.1.
<pre>
# iptables -t nat -A POSTROUTING -s 10.1.1.1 -o eth0 -j SNAT --to 192.168.0.244
# vzctl set 101 --nameserver 192.168.1.1 --save
File resolv.conf was modified
Saved parameters for VE 101
# vzctl exec 101 ping google.com
PING google.com (64.233.167.99) 56(84) bytes of data.
64 bytes from py-in-f99.google.com (64.233.167.99): icmp_seq=1 ttl=241 time=23.0 ms
</pre>

== Installing software inside VE ==
I guess you've noted that there is not so many packages in VE. It is because minimal template was used.
But of course, you can install any software in VE by yourself. For example, in Debian usual <tt>apt-get</tt> tool can be used.

Now, for example, we can install gcc inside VE 101 for developing purposes:
<pre>
# vzctl enter 101
entered into VE 101
#
# apt-get install gcc
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
binutils cpp cpp-3.3 gcc-3.3
Suggested packages:
binutils-doc cpp-doc make manpages-dev autoconf automake libtool flex bison gdb gcc-doc gcc-3.3-doc
Recommended packages:
libc-dev libc6-dev
The following NEW packages will be installed:
binutils cpp cpp-3.3 gcc gcc-3.3
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 5220kB of archives.
After unpacking 13.6MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.freenet.de stable/main binutils 2.15-6 [2221kB]
Get:2 http://ftp.freenet.de stable/main cpp-3.3 1:3.3.5-13 [1393kB]
Get:3 http://ftp.freenet.de stable/main cpp 4:3.3.5-3 [29.6kB]
Get:4 http://ftp.freenet.de stable/main gcc-3.3 1:3.3.5-13 [1570kB]
Get:5 http://ftp.freenet.de stable/main gcc 4:3.3.5-3 [4906B]
Fetched 5220kB in 10s (507kB/s)
Selecting previously deselected package binutils.
(Reading database ... 7436 files and directories currently installed.)
Unpacking binutils (from .../binutils_2.15-6_i386.deb) ...
Selecting previously deselected package cpp-3.3.
Unpacking cpp-3.3 (from .../cpp-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package cpp.
Unpacking cpp (from .../cpp_4%3a3.3.5-3_i386.deb) ...
Selecting previously deselected package gcc-3.3.
Unpacking gcc-3.3 (from .../gcc-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package gcc.
Unpacking gcc (from .../gcc_4%3a3.3.5-3_i386.deb) ...
Setting up binutils (2.15-6) ...

Setting up cpp-3.3 (3.3.5-13) ...
Setting up cpp (3.3.5-3) ...
Setting up gcc-3.3 (3.3.5-13) ...
Setting up gcc (3.3.5-3) ...

# exit
logout
exited from VE 101
#
</pre>

== Resource limiting ==
The very important feature of VE is that you can limit it by resources: CPU, memory, disk space.
It is also performed via vzctl. Current usage values and limits of memory-related resources can be viewed through
<code>/proc/bc/VEID/resources</code> file:
<pre>
# cat /proc/bc/101/resources
kmemsize 628209 976969 2752512 2936012 0
lockedpages 0 0 32 32 0
privvmpages 5238 6885 49152 53575 0
shmpages 5012 5014 8192 8192 0
numproc 3 11 65 65 0
physpages 5084 6020 0 2147483647 0
vmguarpages 0 0 6144 2147483647 0
oomguarpages 5084 6020 6144 2147483647 0
numtcpsock 0 2 80 80 0
numflock 1 5 100 110 0
numpty 0 1 16 16 0
numsiginfo 0 6 256 256 0
tcpsndbuf 0 4440 319488 524288 0
tcprcvbuf 0 42180 319488 524288 0
othersockbuf 2220 6660 132096 336896 0
dgramrcvbuf 0 2220 132096 132096 0
numothersock 1 6 80 80 0
dcachesize 0 0 1048576 1097728 0
numfile 106 339 2048 2048 0
numiptent 10 10 128 128 0
#
</pre>
First column is resource name, second is current usage, third is peak usage, forth and fifth are barrier and limit, and last column is fail counter.

Note that if you have nonzero values in the last column, it means that this VE
experienced a resource shortage. This is very common reason why some application fail to
work in a VE. In this case you should increase limits/barriers accordingly; see
[[resource shortage]] for more info.

== Stopping/removing VE ==
Well, let's stop VE and destroy it:
<pre>
# vzctl stop 101
Stopping VE ...
VE was stopped
VE is unmounted
# vzctl destroy 101
Destroying VE private area: /var/lib/vz/private/101
VE private area was destroyed
#
</pre>

== Links ==
That's all you need to start playing with OpenVZ. Additional information can be found in man page on vzctl and at http://wiki.openvz.org/ .

If you experience some difficulties, contact us via http://forum.openvz.org/ . Templates and other tools are available from http://download.openvz.org/ .

Download/live CD

2007-07-06T06:39:23Z

Vass: /* Changelog */ fix

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-based CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt> or <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to <tt>/vz/private/</tt> for CentOS-based CD and to <tt>/var/lib/vz/private</tt> for KNOPPIX-based CD.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}.
You can work around it by mounting some filesystem (<tt>ext2</tt>, <tt>ext3</tt>, <tt>tmpfs</tt>) to <tt>/var/lib/vz/private</tt> directory.
On CentOS-based LiveCD <tt>tmpfs</tt> is mounted on this directory automatically during the booting.

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:38:52Z

Vass: /* Limitations */ small fixes, workaround for checkpointing

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-based CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt> or <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to <tt>/vz/private/</tt> for CentOS-based CD and to <tt>/var/lib/vz/private</tt> for KNOPPIX-based CD.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}.
You can work around it by mounting some filesystem (<tt>ext2</tt>, <tt>ext3</tt>, <tt>tmpfs</tt>) to <tt>/var/lib/vz/private</tt> directory.
On CentOS-based LiveCD <tt>tmpfs</tt> is mounted on this directory automatically during the booting.

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:18:18Z

Vass: /* Booting */ missprint

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-based CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt> or <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:17:09Z

Vass: /* Downloading */ formatting

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files|This page]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-base CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt>, <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:15:01Z

Vass: /* Downloading */ CentOS 4.4 LiveCD added.

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.md5|size=}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=CentOS-4.4-OpenVZ-i386-LiveCD.iso.asc|size=}}

{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-base CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt>, <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:07:24Z

Vass: /* Centos LiveCD contents */ formatting

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== CentOS LiveCD contents ==
This CD is based on CentOS 4.4 Live CD and contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-base CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt>, <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Download/live CD

2007-07-06T06:06:25Z

Vass: /* Centos LiveCD contents */ 2.6.20 kernel added, vzctl updated, formatting

Here you can find OpenVZ Live CD images. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

We offer two live CDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4.

== Centos LiveCD contents ==
This CD contains the following OpenVZ packages:
* '''kernels''':
** original CentOS 4.4 kernel without virtualization (2.6.9-42.livecd.c4)
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
** 2.6.20-ovz007.1
* '''tools''':
** vzctl 3.0.17
** vzquota 3.0.9
* '''template tools''':
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* '''template metadatas''':
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* '''precreated templates''':
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.) in order
to free disk space for OpenVZ packages and templates.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-base CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt>, <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-05T13:49:09Z

Vass: /* Peculiarities of building 2.6.18-rhel5 */ squshfs 3.2

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there, but you should use <tt>3.2</tt>! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* FIXME

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-05T11:26:44Z

Vass: /* Install all these finally */ vzmigrate fix added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* FIXME

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>.
But when disk quota is off <tt>vzmigrate</tt> fails! ;) This is the [http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #512].
So, don't forget to apply the patch attached to this bug report to <tt>vzmigrate</tt>.

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-05T07:50:13Z

Vass: /* Peculiarities of building 2.6.18-rhel5 */ unionfs build

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there! :)
* While building <tt>unionfs</tt> some small fixes: <tt>i_blksize</tt> field is missing in <tt>inode</tt> structure, <tt>ino_t</tt> should be changed to <tt>u64</tt> in some functions.

==== Peculiarities of building 2.6.20 ====
* FIXME

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Download/live CD

2007-07-04T14:17:42Z

Vass: CentOS 4.4. version added

Here you can find OpenVZ Live CD images. We have to LiveCDs at the moment. The first one is based on Knoppix 5.1.1 and the second one is based on Centos 4.4. Live CD is a great way to test drive the OpenVZ technology without a need to actually install it on your machine.

== Centos LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernels:
** original CentOS 4.4 kernel without virtualization
** 2.6.9-023stab044.4
** 2.6.18-028stab035
** 2.6.18-028stab035-rhel5
* vzctl 3.0.16
* vzquota 3.0.9
* template tools:
** vzyum-2.4.0-11
** vzrpm44-4.4.1-22.5
** vzrpm43-4.3.3-7
** vzpkg-2.7.0-18
* template metadatas:
** vztmpl-centos-4-2.0-2
** vztmpl-fedora-core-3-2.0-2
** vztmpl-fedora-core-4-2.0-2
** vztmpl-fedora-core-5-2.0-2
* several precreated templates:
** centos-4-i386-minimal
** fedora-core-5-i386-minimal
** debian-3.1-i386-minimal

The number of original CentOS packages were removed (<tt>openoffice</tt>, <tt>gimp</tt>, etc.)
to free space for OpenVZ packages.

== Knoppix LiveCD contents ==
This CD contains the following OpenVZ packages:
* kernel 2.6.18-028stab027
* vzctl 3.0.16
* vzquota 3.0.9
plus the number of precreated templates:
* Debian 3.1 minimal
* CentOS 4 minimal
* Fedora Core 5 minimal

It also contains all the software from the original Knoppix 5.1.1 CD, excluding:
* OpenOffice
* GIMP
* Frozen Bubble
The above packages were taken out to make some room for OpenVZ packages.

== Downloading ==

The precreated ISO image is available directly from [http://download.openvz.org/livecd/ download.openvz.org/livecd/] or any of the [[download mirrors]].

{{KernelDownloadTableHead}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso|size=683 MB}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.md5|size=77 B}}
{{DownloadTableRow|path=http://download.openvz.org/livecd|file=KNOPPIX_V5.1.1-OPENVZ-CD-2007-01-04-EN.iso.asc|size=189 B}}
{{KernelDownloadTableTail}}

Note that .md5 and .asc files does not need to be copied to the CD-R; they are here to check the authentity of the iso file. [[Package_signatures#Checking_files]] has some info on this topic.

== Burning ==

After downloading the image, you have to burn it to the actual media, i.e. a CD-R or CD-RW disk. The process depends on the actual OS and CD burning software that you have, and the exact details are out of scope of this article.

If you don't know how to burn the ISO image to CD-R, try [http://www.knoppix.net/wiki/Downloading_FAQ#Q:_I_have_downloaded_the_ISO_file._How_do_I_burn_the_ISO.3F_How_is_the_ISO_supposed_to_be_burned.3F this document].

== Booting ==
Just insert the fresh CD into your CD-ROM and reboot. In some cases you have to modify your BIOS settings in order to enable booting from a CD.
If you use CentOS-base CD you'll be able to choose the kernel to boot: <tt>ovz9rh</tt>, <tt>ovz18</tt>, <tt>ovz18rh</tt>, <tt>ovz20</tt>, <tt>linux</tt> (original CentOS kernel).

== Using ==
Upon the successful boot, a browser window with some helful hints will appear to help you start using OpenVZ. The latest version of that document is also available on this wiki: [[Getting started with OpenVZ live CD]].

== Limitations ==
Since this is a live CD, everything is created in RAM, i.e. is not persistent. If you will create a number of VEs, they will not be available after the reboot.

A number of [[VE]]s that you can create using this live CD greatly depends on the amount of RAM your machine has, since the system uses RAM instead of a hard disk. On a 1 gigabyte RAM machine you can create about 5 VEs. To work around this, you have to have a disk partition mounted to /var/lib/vz.

OpenVZ disk quota does not work on LiveCD at the moment due to {{Bug|558}}, so <code>df</code> in [[VE]] shows ludicrous values if quota is on.
Because of it on CentOS 4.4 LiveCD disk quota is switched off by default.

Checkpointing (and consequently live migration) doesn't work on KNOPPIX LiveCD at the moment due to {{Bug|606}}. But it '''works''' on CentOS-based LiveCD!

== Changelog ==
* 7 May 2007, initial version based on Knoppix 5.1.1
* 5 July 2007, second version based on Centos 4.4

== External links ==
* [http://knoppix.com/ knoppix.com], the home of Knoppix distribution
* [http://centos.org/ centos.org], the home of CentOS distribution

Getting started with OpenVZ live CD

2007-07-04T13:42:56Z

Vass: remove knoppix mentioning: we have now one more LiveCD based on Centos 4.4

This article is written for OpenVZ LiveCD and assumes that the reader only starts using OpenVZ. ([[Download live CD]])

== Introduction ==
So, as you probably know, OpenVZ allows the user to create [[VE]]s, or Virtual Environments, which seem very much
like real computers. Real computer can run various distributions: Debian, Gentoo, Red Hat and Novell products, etc.
In the same way, a VE can be based on various [[OS template|OS (Operating System) templates]]. On the LiveCD only few minimal OS templates are installed because of disk space limit. Each VE is identified by its number -- a '''VEID'''.

== VE creation ==
So, how to create a VE with VEID of 101 based on Debian template? Very easy. Just type the following commands in your
terminal (you must be root):
<pre>
# vzctl create 101 --ostemplate debian-3.1-i386-minimal
Creating VE private area (debian-3.1-i386-minimal)
Performing postcreate actions
VE private area was created
</pre>

'''vzctl''' is the tool that manages VEs. Look in <tt>/var/lib/vz/template/cache/</tt> directory for other OS templates available on LiveCD:
<pre>
# ls -1 /var/lib/vz/template/cache/
centos-4-i386-minimal.tar.gz
debian-3.1-i386-minimal.tar.gz
fedora-core-5-i386-minimal.tar.gz
</pre>

== List of VEs ==
You can get the list of all created VEs on '''HN''' (Hardware Node) using '''vzlist''' command:
<pre>
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 - stopped - -
</pre>

As you see, VE 101 is in stopped state now.

== Starting VE ==
Let's start it:
<pre>
# vzctl start 101
Starting VE ...
VE is mounted
Setting CPU units: 1000
VE start in progress...
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 5 running -
</pre>

o Note: In the LiveCD environment, you may have to increase SHMPAGES for the VE or you will run out of "disk space" when trying to install software. You can do this "on the fly" by issuing:

<pre>
' vzctl set 101 --shmpages $((8192*8)):$((8192*8)) --save '
</pre>

o This will give VE 101 64MB of shmpages; you may wish to give it more if you're planning on doing more than just basic openvz testing.

o You can verify the change from the host by issuing:

<pre>
' vzctl exec 101 cat /proc/user_beancounters '
</pre>

== Executing commands in VE ==
From the "vzlist" command you see that 5 processes are running inside VE 101. (The "NPROC" field indicates the number of Processes, or PIDs, that are active in the VE -- not the number of Processors, or CPUs.) Being on usual [[hardware node]] you can use <code>ps</code> command to identify those, and the same command can be used here. The only difference is that this command should be called inside VE.

In order to perform any command inside VE `vzctl exec` is used:
<pre>
# vzctl exec 101 ps
PID TTY TIME CMD
1 ? 00:00:00 init
7672 ? 00:00:00 rc
7674 ? 00:00:00 S10sysklogd
7677 ? 00:00:00 syslogd
7678 ? 00:00:00 syslogd
7683 ? 00:00:00 ps
</pre>

== Entering VE ==
Any self-respected OS provides a shell for the user. This is how you can get the VE's shell:
<pre>
# vzctl enter 101
entered into VE 101
#
</pre>

In this shell you can do almost all you can do on the real HN. For example create a new user:
<pre>
# useradd new-user
# passwd new-user
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
# mkdir /home/new-user
# chown new-user /home/new-user/
# su new-user
$ cd ~
$ pwd
/home/new-user
exit
#
</pre>

In order to exit from VEs shell, just type exit:
<pre>
# exit
logout
exited from VE 101
#
</pre>

== Setting up VE networking ==
Let's set up networking in VE.

<pre>
# echo 1 > /proc/sys/net/ipv4/ip_forward
# ifconfig venet0 up
# vzctl set 101 --ipadd 10.1.1.1 --save
Adding IP address(es): 10.1.1.1
Saved parameters for VE 1
# vzlist -a
VEID NPROC STATUS IP_ADDR HOSTNAME
101 4 running 10.1.1.1 -
</pre>

Now your [[Hardware Node]] can ping VE and VE can ping HN:
<pre>
# ping 10.1.1.1
PING 10.1.1.1 (10.1.1.1) 56(84) bytes of data.
64 bytes from 10.1.1.1: icmp_seq=1 ttl=64 time=3.80 ms

--- 10.1.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.804/3.804/3.804/0.000 ms
#
# vzctl exec 101 ping 192.168.0.244
PING 192.168.0.244 (192.168.0.244) 56(84) bytes of data.
64 bytes from 192.168.0.244: icmp_seq=1 ttl=64 time=0.508 ms

#
</pre>

However, it is not possible to ping other computers in the network: for it we need to
set up NAT (Network Address Translation) and set the nameserver.

Assume that you've set up network on HN (for example via DHCP) and the IP address
of your node is 192.168.0.244 and nameserver IP address is 192.168.1.1.
<pre>
# iptables -t nat -A POSTROUTING -s 10.1.1.1 -o eth0 -j SNAT --to 192.168.0.244
# vzctl set 101 --nameserver 192.168.1.1 --save
File resolv.conf was modified
Saved parameters for VE 101
# vzctl exec 101 ping google.com
PING google.com (64.233.167.99) 56(84) bytes of data.
64 bytes from py-in-f99.google.com (64.233.167.99): icmp_seq=1 ttl=241 time=23.0 ms
</pre>

== Installing software inside VE ==
I guess you've noted that there is not so many packages in VE. It is because minimal template was used.
But of course, you can install any software in VE by yourself. For example, in Debian usual <tt>apt-get</tt> tool can be used.

Now, for example, we can install gcc inside VE 101 for developing purposes:
<pre>
# vzctl enter 101
entered into VE 101
#
# apt-get install gcc
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
binutils cpp cpp-3.3 gcc-3.3
Suggested packages:
binutils-doc cpp-doc make manpages-dev autoconf automake libtool flex bison gdb gcc-doc gcc-3.3-doc
Recommended packages:
libc-dev libc6-dev
The following NEW packages will be installed:
binutils cpp cpp-3.3 gcc gcc-3.3
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 5220kB of archives.
After unpacking 13.6MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.freenet.de stable/main binutils 2.15-6 [2221kB]
Get:2 http://ftp.freenet.de stable/main cpp-3.3 1:3.3.5-13 [1393kB]
Get:3 http://ftp.freenet.de stable/main cpp 4:3.3.5-3 [29.6kB]
Get:4 http://ftp.freenet.de stable/main gcc-3.3 1:3.3.5-13 [1570kB]
Get:5 http://ftp.freenet.de stable/main gcc 4:3.3.5-3 [4906B]
Fetched 5220kB in 10s (507kB/s)
Selecting previously deselected package binutils.
(Reading database ... 7436 files and directories currently installed.)
Unpacking binutils (from .../binutils_2.15-6_i386.deb) ...
Selecting previously deselected package cpp-3.3.
Unpacking cpp-3.3 (from .../cpp-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package cpp.
Unpacking cpp (from .../cpp_4%3a3.3.5-3_i386.deb) ...
Selecting previously deselected package gcc-3.3.
Unpacking gcc-3.3 (from .../gcc-3.3_1%3a3.3.5-13_i386.deb) ...
Selecting previously deselected package gcc.
Unpacking gcc (from .../gcc_4%3a3.3.5-3_i386.deb) ...
Setting up binutils (2.15-6) ...

Setting up cpp-3.3 (3.3.5-13) ...
Setting up cpp (3.3.5-3) ...
Setting up gcc-3.3 (3.3.5-13) ...
Setting up gcc (3.3.5-3) ...

# exit
logout
exited from VE 101
#
</pre>

== Resource limiting ==
The very important feature of VE is that you can limit it by resources: CPU, memory, disk space.
It is also performed via vzctl. Current usage values and limits of memory-related resources can be viewed through
<code>/proc/bc/VEID/resources</code> file:
<pre>
# cat /proc/bc/101/resources
kmemsize 628209 976969 2752512 2936012 0
lockedpages 0 0 32 32 0
privvmpages 5238 6885 49152 53575 0
shmpages 5012 5014 8192 8192 0
numproc 3 11 65 65 0
physpages 5084 6020 0 2147483647 0
vmguarpages 0 0 6144 2147483647 0
oomguarpages 5084 6020 6144 2147483647 0
numtcpsock 0 2 80 80 0
numflock 1 5 100 110 0
numpty 0 1 16 16 0
numsiginfo 0 6 256 256 0
tcpsndbuf 0 4440 319488 524288 0
tcprcvbuf 0 42180 319488 524288 0
othersockbuf 2220 6660 132096 336896 0
dgramrcvbuf 0 2220 132096 132096 0
numothersock 1 6 80 80 0
dcachesize 0 0 1048576 1097728 0
numfile 106 339 2048 2048 0
numiptent 10 10 128 128 0
#
</pre>
First column is resource name, second is current usage, third is peak usage, forth and fifth are barrier and limit, and last column is fail counter.

Note that if you have nonzero values in the last column, it means that this VE
experienced a resource shortage. This is very common reason why some application fail to
work in a VE. In this case you should increase limits/barriers accordingly; see
[[resource shortage]] for more info.

== Stopping/removing VE ==
Well, let's stop VE and destroy it:
<pre>
# vzctl stop 101
Stopping VE ...
VE was stopped
VE is unmounted
# vzctl destroy 101
Destroying VE private area: /var/lib/vz/private/101
VE private area was destroyed
#
</pre>

== Links ==
That's all you need to start playing with OpenVZ. Additional information can be found in man page on vzctl and at http://wiki.openvz.org/ .

If you experience some difficulties, contact us via http://forum.openvz.org/ . Templates and other tools are available from http://download.openvz.org/ .

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T13:11:06Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ quite remove

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there! :)
==== Peculiarities of building 2.6.20 ====
* FIXME

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

And I suggest to remove <tt>quite</tt> parameter, to see what happens: it will help greatly if some problem will happen.

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T13:08:53Z

Vass: /* Building OpenVZ kernels */ peculiarities of building added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above. But some important differences in building
kernel of version other then 2.6.18 one is summarized below.
==== Peculiarities of building 2.6.9-rhel4 ====
* You should also include this path (http://git.openvz.org/?p=linux-2.6.18-openvz;a=commit;h=681d0b65dedd54d8c716cef56418ac1a478f94da)
in building process. Hopefully will be fixed in <tt>2.6.9-044stab044.8</tt> kernel.
* Use <tt>unionfs 1.0.14</tt>, but not <tt>unionfs 1.1.5</tt>, as offered at its official site!
* Small problems while building <tt>unionfs</tt>: arguments of <tt>vfs_permissions()</tt>. Just add <tt>NULL</tt>
to appropriate places.
==== Peculiarities of building 2.6.18-rhel5 ====
* Note, <tt>squashfs</tt> patch is already there! :)
==== Peculiarities of building 2.6.20 ====
* FIXME

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T12:45:21Z

Vass: /* Embellishment */ desktop background change

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

One more thing we change is default desktop background. It resides in <tt>usr/share/backgrounds/images/default.png</tt>.
I added OpenVZ logo there.

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T10:32:18Z

Vass: /* Creating ISO */ <tt> added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed <tt>make_iso.sh</tt> script on CD.
Using this script, it is very easy to create iso:
<pre>
# cd ovz-livecd
# ./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T08:45:06Z

Vass: /* Debugging LiveCD */ serial console add

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.
Also it can be useful to redirect kernel output to serial console:
<pre>
console=ttyS0,115200 console=tty
</pre>

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T08:23:45Z

Vass: /* Embellishment */ information about splash.cfg and splash.lss added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===
If you want to release LiveCD to other people some "embelishments" should be added.

First, edit <tt>splash.cfg</tt>, <tt>help1.txt</tt>, <tt>help2.txt</tt> and <tt>help3.txt</tt>
files (that are in <tt>~/workbench/ovz-livecd/boot/help</tt> directory) to contain appropriate information.
These messages appear on the LiveCD initial boot: they help to choose kernel and kernel options.

Also, you can note, that CentOS logo appears at this moment. This logo resides in <tt>ovz-livecd/boot/splash.lss</tt>
file. I added OpenVZ logo there. The following tools will help you: <tt>lss16toppm</tt>, <tt>ppmtolss16</tt> and
of course <tt>gimp</tt> :)

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-04T08:12:01Z

Vass: /* Technical details */ embelishment section added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Embellishment ===

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T14:56:59Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ ramdisk_size change

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=102400 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

Also '''don't forget''' to change <tt>ramdisk_size</tt> to 102400 for all kernels including Centos 4.4 default!

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T13:39:14Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ increase initrd size

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patches to <tt>liblinuxlive</tt> file. Initrd image that goes with Centos 4.4 is
of size 40Mb. OpenVZ LiveCD will contain several kernels, so initrd should have several sets of
kernel modules (for each kernel) and 40Mb isn't enough in this case. I created an initrd of size 100Mb.
Don't forget to update <tt>ramdisk_size</tt> in <tt>isolinux.cfg</tt> file later!

Creating "big" initrd with original content:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/initrd-old.gz
# cd initrd.d/
# mkdir mount-old
# gunzip initrd-old.gz
# mount -o loop initrd-old mount-old/
# dd if=/dev/zero of=initrd bs=1048576 count=100
# mkdir mount
# mount -o loop initrd mount/
# cp -a mount-old/* mount/
# umount mount-old/
# rm -rf mount-old initrd-old
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T10:48:51Z

Vass: /* Install all these finally */ switch off disk quota by default

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
At the moment vzquota doesn't support <tt>tmpfs</tt> ([http://bugzilla.openvz.org/show_bug.cgi?id=558 Bug #558]).
We're working on this problem, but for now '''switching the disk quota off''' in <tt>/etc/vz/vz.conf</tt>: <tt>DISK_QUOTA=no</tt>

'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T10:41:22Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ tmpfs on /vz/private added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
As mentioned earlier we want to mount <tt>tmpfs</tt> on /vz/private automatically. Just apply this patch to
<tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig-fstab 2007-07-02 17:50:34.000000000 +0400
+++ mount/liblinuxlive 2007-07-03 12:33:30.000000000 +0400
@@ -414,6 +414,7 @@ activate_fstab()
echo "tmpfs / tmpfs defaults 0 0" >$FSTAB
echo "devpts /dev/pts devpts gid=5,mode=620 0 0" >>$FSTAB
echo "proc /proc proc defaults 0 0" >>$FSTAB
+ echo "tmpfs /vz/private tmpfs defaults 0 0" >>$FSTAB

# now done by fstab-sync
# list_cdrom_devices | while read DEVICE; do
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T10:08:05Z

Vass: /* Install all these finally */ /vz/private creation for tmpfs on it later

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
Here we have to make one step to workaround [[Checkpointing and live migration]] problem. The thing is that at the moment
chekpointing isn't supported on <tt>unionfs</tt> filesystem ([http://bugzilla.openvz.org/show_bug.cgi?id=606 Bug #606]).
But we want of course checkpoiting to work on LiveCD. If we mount <tt>tmpfs</tt> on <tt>/vz/private/</tt> on '''booted''' LiveCD,
then all newly created VEs reside on <tt>tmpfs</tt> and checkpointing works. Moreover we do not introduce any additional memory usage,
because even without <tt>tmpfs</tt> all VEs reside in RAM (they contain new/updated files for <tt>unionfs</tt>).

Create <tt>/vz/private</tt> directory, later, while modifying initrd image we will add the line that will mount <tt>tmpfs</tt>
on this directory:
<pre>
mkdir root/vz/private/
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T07:36:23Z

Vass: /* Building OpenVZ kernels */ path fixing

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# cd /usr/src/redhat/SPECS/
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# cd ~/workbench/
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T07:18:29Z

Vass: /* Creating booted LiveCD root filesystem */ compatibility to old squashfs added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able to mount <tt>squashfs</tt>) and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools.
Here is '''an important''' note. We want, our LiveCD to be able to boot original Centos 4.4 kernel. This original kernel doesn't support
<tt>squashfs</tt> of version <tt>3</tt>. Consequently we use <tt>sqashfs2.2-r2</tt> for <tt>mksquasfs</tt> tool, but for <tt>unsquashfs</tt> tool and later for compiling fresh OpenVZ kernels with <tt>squashfs</tt> support we use <tt>sqashfs3.2-r2</tt> sources/patches, because they are back compatible with <tt>sqashfs v2</tt>.
<pre>
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs2.2-r2.tar.gz
# tar xzf squashfs2.2-r2.tar.gz
# cd squashfs2.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -o mksquashfs
# cp mksquashfs /usr/bin/

# cd ~/workbench/
# wget http://belnet.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-03T06:31:50Z

Vass: /* Creating ISO */ fix missprint

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this script, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T15:23:47Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ short names in isolinux.cfg

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD. It is important to give '''short name''' (not greater
than 8 characters) to kernel image, otherwise <tt>isolinux</tt> will not see it.
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/ovz18
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-07-02 19:10:51.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz18
+kernel boot/ovz18
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest

</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T14:16:42Z

Vass: /* Vmlinux, initrd and isolinux.cfg */ cdrom modules loading patch added

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD:
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/
</pre>
Modify initrd image. More precisely, we need to add OpenVZ kernel modules there
and apply small patch to <tt>liblinuxlive</tt> file.
Mount initrd:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
</pre>
Copy modules to initrd:
<pre>
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
</pre>
initrd image, that goes with Centos 4.4 LiveCD doesn't load cdrom kernel modules.
It works well for CentOS kernel, because cdrom support is built-in there.
But OpenVZ kernel supports cdrom (usual ide cdrom) via kernel module.
Consequetnly we need to apply the following patch to <tt>liblinuxlive</tt> file:
<pre>
--- liblinuxlive.orig 2006-08-19 08:53:11.000000000 +0400
+++ liblinuxlive 2007-07-02 17:50:34.000000000 +0400
@@ -337,6 +337,9 @@ modprobe_essential_modules()
{
echolog "starting loop device support"
modprobe_module loop max_loop=32
+ echolog "starting cdrom support"
+ modprobe_module cdrom
+ modprobe_module ide-cd
echolog "starting cdrom filesystem support"
modprobe_module isofs
echolog "starting squashfs support"
</pre>
Finishing actions with initrd:
<pre>
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>
Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-06-29 18:22:29.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz-2.6.18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz-2.6.18
+kernel boot/vmlinuz-2.6.18-ovz028stab035.1-smp
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest
</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T14:05:26Z

Vass: /* Install all these finally */ removing ~/.bash_history file

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
While being in chrooted environment bash history was written to <tt>~/.bash_history</tt> file.
Clean it:
<pre>
# rm root/root/.bash_history
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD:
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/
</pre>
Modify initrd image. More precisely, we just need to add OpenVZ kernel modules there:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>

Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-06-29 18:22:29.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz-2.6.18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz-2.6.18
+kernel boot/vmlinuz-2.6.18-ovz028stab035.1-smp
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest
</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T13:49:15Z

Vass: /* Install all these finally */ removal list updated

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt>, <tt>gnome-games</tt>,
<tt>tetex</tt>, <tt>latex</tt>, <tt>k3b</tt> and printing related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD:
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/
</pre>
Modify initrd image. More precisely, we just need to add OpenVZ kernel modules there:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>

Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-06-29 18:22:29.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz-2.6.18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz-2.6.18
+kernel boot/vmlinuz-2.6.18-ovz028stab035.1-smp
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest
</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T13:28:58Z

Vass: /* Install all these finally */ rpm -e fix

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt> related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex tetex tetex-afm \
tetex-dvips tetex-fonts tetex-xdvi
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD:
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/
</pre>
Modify initrd image. More precisely, we just need to add OpenVZ kernel modules there:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>

Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-06-29 18:22:29.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz-2.6.18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz-2.6.18
+kernel boot/vmlinuz-2.6.18-ovz028stab035.1-smp
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest
</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]

Creating OpenVZ LiveCD based on Centos 4.4

2007-07-02T13:27:36Z

Vass: /* Install all these finally */ remove gimp package from LiveCD

This article is a step by step guide explaining how to add OpenVZ functionality to Centos 4.4 LiveCD.
As a result, you will have a LiveCD, that can boot in various OpenVZ kernels (rhel4-2.6.9, 2.6.18,
rhel5-2.6.18, 2.6.20) and in original CentOS kernel also. After booting from the CD, you will be able
to create [[VE]]s, start them, enter them - i.e. perform [[Getting started with OpenVZ live CD | common OpenVZ activity]].

CentOS 5 LiveCD is already announced and as soon as it will be available we'll update the article.

== General actions ==
It is quite easy to add OpenVZ functionality to existent LiveCD. The most difficult work is already done
by the original LiveCD creators. But it requires pretty much hand job. Probably some day some good guy will
automatize this process. ;)

Actually you need to accomplish the following general actions - everything else is only technical details:
* Get a copy of CentOS 4.4 LiveCD image
* Use <tt>unsquashfs</tt> tool to extract the contents of *.mo files, located on the image
* Construct '''booted''' LiveCD root filesystem from extracted files
* Create OpenVZ kernel RPM-packages with <tt>squashfs</tt> support
* Compile <tt>unionfs</tt> kernel modules against OpenVZ kernels
* Get OpenVZ tools RPM-packages: vzctl, vzquota, vzpkg, etc. Get template metadatas and precreated templates
* Remove some RPM-packages from '''booted''' LiveCD root filesystem (conflicting with OpenVZ kernel packages and redundant packages - to free some disk space)
* Install OpenVZ kernels, tools and templates in '''booted''' LiveCD root filesystem
* Using <tt>mksquashfs</tt> tool compress '''booted''' CD root filesystem
* Modify initrd image: add there kernel modules for OpenVZ kernels
* Introduce small modifications to isolinux configuration file
* Create new iso image

== Technical details ==
This part contains detailed description of LiveCD creation. I've removed the output of some
commands in order to simplify the reading.

=== Creating booted LiveCD root filesystem ===
Create a directory, that will be our "workbench":
<pre>
# cd ~
# mkdir workbench
# cd workbench
</pre>

First of all we need some raw material - CentOS 4.4 LiveCD image:
<pre>
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/CentOS-4.4-i386-LiveCD.iso
# wget http://mirror.nsc.liu.se/CentOS/4.4/isos/i386/md5sum.livecd
# md5sum --check md5sum.livecd
CentOS-4.4-i386-LiveCD.iso: OK
</pre>

Copy the contents of CD to the directory, where we will form the OpenVZ LiveCD filesystem.
I named this directory "ovz-livecd":
<pre>
# mkdir ovz-livecd
# mkdir mount
# mount -o loop -t iso9660 CentOS-4.4-i386-LiveCD.iso mount/
# cp -a mount/* ovz-livecd/
# umount mount
# rm -rf mount
</pre>

You can see, that <tt>ovz-livecd/base/</tt> directory owns several *.mo files - images of squashfs filesystem,
and each image holds one directory of '''booted''' LiveCD root filesystem: <tt>bin</tt>, <tt>usr</tt> and so on:
<pre>
# ls -1 ovz-livecd/base
bin.mo etc.mo lib.mo root.mo sbin.mo usr.mo var.mo
</pre>

In order to decompress these images we need <tt>unsquashfs</tt> tool. <tt>squashfs</tt> source archive contains
patches for various kernel versions (to be able mount <tt>squashfs</tt>), and user-space tools for compressing usual filesytem into <tt>squashfs</tt> filesystem and for decompressing it back. Well, we don't need kernel patches right now (but we will need them later), we only want to install <tt>squashfs</tt> user-space tools:
<pre>
# wget wget http://switch.dl.sourceforge.net/sourceforge/squashfs/squashfs3.2-r2.tar.gz
# tar xzf squashfs3.2-r2.tar.gz
# cd squashfs3.2-r2/squashfs-tools/
# make
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc unsquashfs.o -lz -o unsquashfs
# cp mksquashfs /usr/bin/
# cp unsquashfs /usr/bin/
# cd ~/workbench
</pre>
Well, having the intruments in hand, why not extract the images? Create a directory, called <tt>livecd-root</tt>,
that will store the root filesystem of '''booted''' LiveCD. I.e. it will contain <tt>/bin</tt>, <tt>/sbin</tt>,
<tt>/etc</tt> and other directories, which you will see after booting the LiveCD. Unfortunately <tt>unsquashfs</tt>
tool is not very convenient, and I found only this way to use it:
<pre>
# mkdir livecd-root

# unsquashfs ovz-livecd/base/bin.mo
# mv squashfs-root/bin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/etc.mo
# mv squashfs-root/etc/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/lib.mo
# mv squashfs-root/lib/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/root.mo
# mv squashfs-root/root/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/sbin.mo
# mv squashfs-root/sbin/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/usr.mo # this action takes some time, be patient ;)
# mv squashfs-root/usr/ livecd-root/
# rm -rf squashfs-root

# unsquashfs ovz-livecd/base/var.mo
# mv squashfs-root/var/ livecd-root/
# rm -rf squashfs-root
</pre>
Yet we have nearly complete root filesystem of '''booted''' LiveCD in <tt>livecd-root</tt> directory. However some things
are missing, and I guess you know what are they: <tt>/dev</tt>, <tt>/proc</tt>, <tt>/sys</tt>, <tt>/tmp</tt> and <tt>/boot</tt> directories. We need them, because some tools (<tt>rpm</tt>, <tt>bash</tt>, ...) will not work without them in chrooted environment later.
<pre>
# mkdir root/dev
# cp -a /dev/* root/dev
# mkdir root/proc
# mount -t proc proc root/proc/
# mkdir root/sys
# mount -t sysfs sysfs root/sys/
# mkdir root/tmp
# mkdir root/boot
</pre>

That's it. We have '''booted''' LiveCD root filesystem in <tt>livecd-root</tt> directory.
Now we can <tt>chroot</tt> in this directory and install all needed packages. But what
packages do we need?
* OpenVZ kernels
* OpenVZ tools: vzctl, vzquota, vzpkg, vzrpm43, vzrpm44, vzyum
* OpenVZ template metadatas: centos-4, fc-3, fc4 and fc-5
* Some OpenVZ precreated templates
* Some CentOS tool updates: sqlite package
Tools and templates related packages can be simply downloaded and installed.
But it is not so easy with kernel. Usual OpenVZ kernel won't work: it doesn't
support <tt>squashfs</tt>. The next chapter explains how to create RPM-package with
OpenVZ kernel, that supports <tt>squashfs</tt>

=== Building OpenVZ kernels ===
Let's consider, how to build <tt>2.6.18</tt> OpenVZ kernel RPM-package with <tt>squashfs</tt> support.
First download and install OpenVZ source RPM:
<pre>
# wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-2.6.18-ovz028stab035.1.src.rpm
# rpm -ivh kernel-2.6.18-ovz028stab035.1.src.rpm
</pre>
<tt>squashfs</tt> sources contain the patch against <tt>2.6.18</tt>. Let's copy this patch to RPM build directory:
<pre>
# cp squashfs3.2-r2/kernel-patches/linux-2.6.18/squashfs3.2-patch /usr/src/redhat/SOURCES/
</pre>
Now we need to modify the spec-file (<tt>/usr/src/redhat/SPECS/kernel-ovz.spec</tt>) in order to add <tt>squashfs</tt> patch to build process. Below are the changes you should introduce in diff format.
The meaning of changes is transparent: firstly, do not build UP, enterprise and debug kernels (our LiveCD will be
based on SMP kernel) and secondly, apply <tt>squashfs</tt> patch before compilation.
<pre>
--- kernel-ovz.spec.orig 2007-06-09 11:40:15.000000000 +0400
+++ kernel-ovz.spec 2007-06-28 17:42:41.000000000 +0400
@@ -5,13 +5,13 @@ Summary: The Linux kernel (the core of t

# normal/debug/debuginfo
%define buildnormal 1
-%define builddebug 1
+%define builddebug 0
%define builddebuginfo 0

# kernels we want to build
-%define buildup 1
+%define buildup 0
%define buildsmp 1
-%define buildenterprise 1
+%define buildenterprise 0
%define buildentnosplit 0

# versions
@@ -161,6 +161,9 @@ Patch2104: linux-2.6.18-drbd-0.7.22-0.8.
Patch2105: linux-2.6.18-drbd-8.0.0-8.0.2.patch
Patch2106: linux-2.6.18-drbd-8.0.2-8.0.3.patch

+# squashfs3.2-patch
+Patch2107: squashfs3.2-patch
+
# ----------------------------------------------------------------------------

BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
@@ -318,6 +321,8 @@ cd linux-%{kversion}
%patch2105 -p1
%patch2106 -p1

+%patch2107 -p1
+
# ----------------------------------------------------------------------------

mkdir configs
</pre>
We need one more tiny hack in order RPM-build process to walk smoothly.
There are config files for various OpenVZ kernel flavours
in <tt>/usr/src/redhat/SOURCES</tt> directory:
<pre>
# ls /usr/src/redhat/SOURCES/*.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-ppc64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-enterprise.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-i686-smp.config.ovz /usr/src/redhat/SOURCES/kernel-2.6.18-x86_64-smp.config.ovz
/usr/src/redhat/SOURCES/kernel-2.6.18-ia64.config.ovz
</pre>
'''All''' these files should contain information about <tt>squashfs</tt>.
In fact, only two additional lines are necessary:
<pre>
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
</pre>
I did it with the following command:
<pre>
# cd /usr/src/redhat/SOURCES/
# echo "CONFIG_SQUASHFS=m
> CONFIG_SQUASHFS_EMBEDDED=n" | tee -a *.config.ovz
CONFIG_SQUASHFS=m
CONFIG_SQUASHFS_EMBEDDED=n
# cd ~/workbench/
</pre>
After all efforts, building the RPM-package is very easy:
<pre>
# rpmbuild --target=i686 -ba kernel-ovz.spec
</pre>
If everything goes smoothly, in several tens of minutes depending on the power of your node you willl have OpenVZ kernel RPM-package with <tt>squashfs</tt> support. Let's copy it to our local directory <tt>to_booted_livecd/kernels/</tt>:
<pre>
# ls /usr/src/redhat/RPMS/i686/
kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# mkdir -p to_booted_livecd/kernels
# cp /usr/src/redhat/RPMS/i686/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm to_booted_livecd/kernels/
</pre>
OpenVZ LiveCD supports several kernels. So, the actions mentioned above should be accomplished to all kernels,
that you need on LiveCD. In this article I skip the description of building RPMS for all kernels. All description
below are based on <tt>028stab035</tt> kernel, that has been just build above.

=== Unionfs kernel module ===
In most present-day LiveCDs <tt>unionfs</tt> filesystem is used. This filesystem "can appear to merge the contents of several directories, while keeping their physical content separate". For example, after the command
<pre>
# mount -t unionfs -o dirs=/tmp/changes=rw:/cdrom=ro unionfs /union
</pre>
you will see merged contents of <tt>/cdrom</tt> and <tt>/tmp/changes</tt> directories in <tt>/union</tt> directory.
At that, <tt>cdrom</tt> will be read-only. I.e. if you will write to some file in <tt>/union</tt> directory,
that resides in <tt>/cdrom</tt> in fact, this modified file will be automatically moved to <tt>/tmp/changes</tt>.
Such useful property is logically used in LiveCDs: <tt>unionfs</tt> merges cdrom contents (that is read-only)
and temporary in-memory filesystem.
In order to add support of <tt>unionfs</tt> filesystem to OpenVZ kernel we need to compile <tt>unionfs</tt> kernel
module against used kernel. Download the sources of unionfs:
<pre>
# wget ftp://ftp.filesystems.org/pub/unionfs/unionfs-1.x/unionfs-1.4.tar.gz
# tar xzf unionfs-1.4.tar.gz
</pre>
In previous chapter we have created an RPM-package with OpenVZ kernel. We should install it: because
this packages contents kernel headers, required for building kernel modules.
<pre>
# rpm -ivh to_booted_livecd/kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
Preparing... ########################################### [100%]
1:kernel-smp ########################################### [100%]
</pre>
Now compile <tt>unionfs</tt>:
<pre>
# cd unionfs-1.4
# echo 'LINUXSRC=/lib/modules/2.6.18-ovz028stab035.1-smp/build/
> TOPINC=-I$(LINUXSRC)/include
> MODDIR=/lib/modules/2.6.18-ovz028stab035.1-smp/' > fistdev.mk
# make
# ls unionfs.ko
unionfs.ko
# cd ..
</pre>
Later we will place <tt>unionfs.ko</tt> to appropriate place in '''booted''' LiveCD filesystem.
At the moment copy it to local <tt>to_booted_livecd/unionfs-modules/2.6.18/</tt> directory:
<pre>
# mkdir -p to_booted_livecd/unionfs-modules/2.6.18/
# cp unionfs-1.4/unionfs.ko to_booted_livecd/unionfs-modules/2.6.18/
</pre>
I repeat once again: if you want to use several kernels on LiveCD - the operation above should
be repeated for each wishful kernel.

=== OpenVZ tools and templates. CentOS updates ===
We have now the kernel RPM-package. But we also need OpenVZ tools RPM-packages.
It is much easier with them: just download these packages from http://openvz.org:
<pre>
# mkdir to_booted_livecd/tools
# cd to_booted_livecd/tools/

# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzctl/3.0.16/vzctl-lib-3.0.16-1.i386.rpm
# wget http://download.openvz.org/utils/vzquota/3.0.9/vzquota-3.0.9-1.i386.rpm
# wget http://download.openvz.org/template/utils/vzpkg/2.7.0-18/vzpkg-2.7.0-18.noarch.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm43/current/vzrpm43-python-4.3.3-7_nonptl.6.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzrpm44/current/vzrpm44-python-4.4.1-22.5.i386.rpm
# wget http://download.openvz.org/template/utils/vzyum/current/vzyum-2.4.0-11.noarch.rpm

# mkdir ../template-metadatas
# cd ../template-metadatas

# wget http://download.openvz.org/template/metadata/centos-4/vztmpl-centos-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-3/vztmpl-fedora-core-3-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-4/vztmpl-fedora-core-4-2.0-2.i386.rpm
# wget http://download.openvz.org/template/metadata/fedora-core-5/vztmpl-fedora-core-5-2.0-2.i386.rpm

# mkdir ../precreated
# cd ../precreated

# wget http://download.openvz.org/template/precreated/centos-4-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/debian-3.1-i386-minimal.tar.gz
# wget http://download.openvz.org/template/precreated/fedora-core-5-i386-minimal.tar.gz

# cd ~/workbench/
</pre>
As you understand, you can add other precreated templates to LiveCD. The main thing here is that usual CD is
not rubber and precreated templates are rather big! ;)

Centos 4.4 LiveCD has a serios problem: <tt>sqlite</tt> package is broken and consequently
<tt>yum</tt> doesn't work. In order to fix this problem we need updated <tt>sqlite</tt> package:
<pre>
# mkdir to_booted_livecd/updates
# cd to_booted_livecd/updates
# wget http://centos.arcticnetwork.ca/4.5/os/i386/CentOS/RPMS/sqlite-3.3.6-2.i386.rpm
# cd ~/workbench/
</pre>

=== Install all these finally ===
We have all components that will be on '''booted''' LiveCD ready. Let's install them
in <tt>chrooted</tt> environment. But before uninstall <tt>lsctp-tools</tt> package,
because these tools need kernel version prior to 2.6.10. Then update <tt>sqlite</tt>
package. Also we need to remove some other packages in order to free some disk space:
I've removed <tt>openoffice.org</tt>, <tt>gimp</tt>, <tt>dia</tt> related packages:
<pre>
# cp -r to_booted_livecd/ root/tmp/
# chroot root/

# rpm -e lksctp-tools-1.0.2-6.4E.1.i386

# rpm -e openoffice.org-libs openoffice.org
# rpm -e gimp-data-extras xsane-gimp gimp-print-plugin gimp-print-utils
# rpm -e gimp gimp-print gimp-help gimp-gap
# rpm -e dia
# rpm -e planner
# rpm -e gnome-games
# rpm -e tetex-latex jadetex xmltex passivetex
# rpm -e tetex
# rpm -e tetex-afm tetex-dvips-2.0.2-22.EL4.7 tetex-fonts-2.0.2-22.EL4.7 tetex-xdvi-2.0.2-22.EL4.7
# rpm -e k3b
# rpm -e foomatic Omni Omni-foomatic system-config-printer \
system-config-printer-gui hal-cups-utils desktop-printing

# cd /tmp/to_booted_livecd/

# rpm -Uvh updates/sqlite-3.3.6-2.i386.rpm

# rpm -ivh kernels/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
# rpm -ivh tools/*.rpm
# rpm -ivh template-metadatas/*.rpm
</pre>

Copy precreated templates to appropriate place:
<pre>
# cp precreated/* /vz/template/cache/
</pre>
Add <tt>unionfs.ko</tt> to other kernel modules and run <tt>depmod</tt>:
<pre>
# mkdir /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs
# cp unionfs-modules/2.6.18/unionfs.ko /lib/modules/2.6.18-ovz028stab035.1-smp/kernel/fs/unionfs/
# depmod 2.6.18-ovz028stab035.1-smp
# exit
exit
</pre>
'''Booted''' LiveCD filesystem is ready, congratulations!

=== Compressing '''booted''' LiveCD filesystem ===
Since <tt>mksquashfs</tt> tool is already installed we can simply use it to create
<tt>*.mo</tt> files:
<pre>
# rm -f ovz-livecd/base/*.mo
# mksquashfs root/lib/ ovz-livecd/base/bin.mo -keep-as-directory
# mksquashfs root/etc/ ovz-livecd/base/etc.mo -keep-as-directory
# mksquashfs root/lib/ ovz-livecd/base/lib.mo -keep-as-directory
# mksquashfs root/root/ ovz-livecd/base/root.mo -keep-as-directory
# mksquashfs root/sbin/ ovz-livecd/base/sbin.mo -keep-as-directory
# mksquashfs root/usr/ ovz-livecd/base/usr.mo -keep-as-directory # it can take long...
# mksquashfs root/var/ ovz-livecd/base/var.mo -keep-as-directory
# mksquashfs root/vz/ ovz-livecd/base/vz.mo -keep-as-directory
</pre>

=== Vmlinux, initrd and isolinux.cfg ===
The next step is to force <tt>isolinux</tt> (cdrom boot loader) to boot OpenVZ kernel.
First, copy kernel image to OpenVZ CD:
<pre>
# cp root/boot/vmlinuz-2.6.18-ovz028stab035.1-smp ovz-livecd/boot/
</pre>
Modify initrd image. More precisely, we just need to add OpenVZ kernel modules there:
<pre>
# mkdir initrd.d
# cp ovz-livecd/boot/initrd.gz initrd.d/
# cd initrd.d/
# mkdir mount
# gunzip initrd.gz
# mount -o loop initrd mount/
# cp -r ../root/lib/modules/2.6.18-ovz028stab035.1-smp/ mount/lib/modules/
# rm -rf mount/lib/modules/2.6.18-ovz028stab035.1-smp/build # we don't neet it on initrd image
# sync
# umount mount/
# gzip initrd
# /bin/cp initrd.gz ../ovz-livecd/boot/initrd.gz
# cd ..
</pre>

Now we should modify <tt>ovz-livecd/isolinux.cfg</tt> file, that controls <tt>isolinux</tt> boot loader.
We inform boot loader, that new kernel was added by introducing the following changes (in diff format):
<pre>
--- ovz-livecd/isolinux.cfg.orig 2006-08-19 08:53:09.000000000 +0400
+++ ovz-livecd/isolinux.cfg 2007-06-29 18:22:29.000000000 +0400
@@ -1,5 +1,5 @@
display boot/splash.cfg
-default linux
+default ovz-2.6.18
prompt 1
timeout 300
F1 boot/help1.txt
@@ -11,5 +11,9 @@ label linux
kernel boot/vmlinuz
append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet

+label ovz-2.6.18
+kernel boot/vmlinuz-2.6.18-ovz028stab035.1-smp
+append max_loop=32 vga=791 initrd=boot/initrd.gz init=linuxrc load_ramdisk=1 prompt_ramdisk=0 ramdisk_size=40000 root=/dev/ram0 rw rhgb quiet
+
label memtest
kernel boot/memtest
</pre>

=== Creating ISO ===
Centos 4.4 LiveCD creators were very gentle and have placed make_iso.sh script on CD.
Using this sctipt, it is very easy to create iso:
<pre>
./make_iso.sh ../CentOS-4.4-i386-OpenVZ-LiveCD.iso
</pre>

== Debugging LiveCD ==
If you have trouble with created OpenVZ LiveCD, boot the kernel with
<pre>
vga=0 debug
</pre>
command line options. It will help you to diagnose the problem more precisely.

== External Links ==
* [http://centos.org Official CentOS community website]
* [http://kernel.org Linux kernel website]
* [http://redhat.com RedHat Inc. website]
* [http://squashfs.sourceforge.net/ squashfs website]
* [http://www.filesystems.org/project-unionfs.html uninonfs official homepage]