6,534
edits
Changes
part 1
This articles tries to summarize why ploop is needed, and why is it a better technology.
== Before ploop ==
First of all, a few facts about the pre-ploop era technologies and their
limitations.
As you are probably aware, a container file system was just a directory
on the host, which a new container was chroot()-ed into. Although it
seems like a good and natural idea, there are a number of limitations.
<ol>
<li>Since containers are living on one same file system, they all
share common properties of that file system (it's type, block size,
and other options). That means we can not configure the above properties
on a per-container basis.</li>
<li>One such property that deserves a special item in this list is
file system journal. While journal is a good thing to have, because
it helps to maintain file system integrity and improve reboot times
(by eliminating fsck in many cases), it is also a bottleneck for
containers. If one container will fill up in-memory journal (with
lots of small operations leading to file metadata updates, e.g.
file truncates), all the other containers I/O will block waiting
for the journal to be written to disk. In some extreme cases we saw
up to 15 seconds of such blockage.</li>
<li>There is no such thing as a per-directory disk quota for Linux,
so in order to limit containers disk space we had to develop one,
it's called vzquota.</li>
<li>When doing a live migration without some sort of shared storage
(like NAS or SAN), we sync the files to a destination system using
rsync, which does the exact copy of all files, except that their
i-node numbers on disk will change. If there are some apps that rely
on files' i-node numbers being constant (which is normally the case),
those apps are not surviving the migration</li>
<li>Finally, a container backup or snapshot is harder to do because
there is a lot of small files that need to be copied.</li>
</ol>
== Before ploop ==
First of all, a few facts about the pre-ploop era technologies and their
limitations.
As you are probably aware, a container file system was just a directory
on the host, which a new container was chroot()-ed into. Although it
seems like a good and natural idea, there are a number of limitations.
<ol>
<li>Since containers are living on one same file system, they all
share common properties of that file system (it's type, block size,
and other options). That means we can not configure the above properties
on a per-container basis.</li>
<li>One such property that deserves a special item in this list is
file system journal. While journal is a good thing to have, because
it helps to maintain file system integrity and improve reboot times
(by eliminating fsck in many cases), it is also a bottleneck for
containers. If one container will fill up in-memory journal (with
lots of small operations leading to file metadata updates, e.g.
file truncates), all the other containers I/O will block waiting
for the journal to be written to disk. In some extreme cases we saw
up to 15 seconds of such blockage.</li>
<li>There is no such thing as a per-directory disk quota for Linux,
so in order to limit containers disk space we had to develop one,
it's called vzquota.</li>
<li>When doing a live migration without some sort of shared storage
(like NAS or SAN), we sync the files to a destination system using
rsync, which does the exact copy of all files, except that their
i-node numbers on disk will change. If there are some apps that rely
on files' i-node numbers being constant (which is normally the case),
those apps are not surviving the migration</li>
<li>Finally, a container backup or snapshot is harder to do because
there is a lot of small files that need to be copied.</li>
</ol>