Difference between revisions of "Ploop/readme"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(Marked this version for translation)
(rm translate tags)
Line 1: Line 1:
<translate>
 
<!--T:1-->
 
 
This document explains how to use user-space <code>ploop</code> utility for typical
 
This document explains how to use user-space <code>ploop</code> utility for typical
 
use-cases.
 
use-cases.
  
<!--T:2-->
 
 
{{Warning|The commands below are low-level stuff. It's better to use vzctl which has all the features in place.}}
 
{{Warning|The commands below are low-level stuff. It's better to use vzctl which has all the features in place.}}
  
== Getting help == <!--T:3-->
+
== Getting help ==
  
<!--T:4-->
 
 
All user-space ploop management operations are available via "<code>ploop</code>"
 
All user-space ploop management operations are available via "<code>ploop</code>"
 
utility. Run it w/o args to get help:
 
utility. Run it w/o args to get help:
  
  <!--T:5-->
+
  # ploop
# ploop
 
  
<!--T:6-->
 
 
Run with a cmd as the only arg to get cmd-specific help, e.g.:
 
Run with a cmd as the only arg to get cmd-specific help, e.g.:
  
  <!--T:7-->
+
  # ploop init
# ploop init
+
 
 +
== Load modules ==
  
== Load modules == <!--T:8-->
 
  
<!--T:9-->
 
 
Further ploop commands assume that all necessary modules are loaded:
 
Further ploop commands assume that all necessary modules are loaded:
  
  <!--T:10-->
+
  # modprobe ploop
# modprobe ploop
 
 
  # modprobe pfmt_ploop1
 
  # modprobe pfmt_ploop1
 
  # modprobe pfmt_raw
 
  # modprobe pfmt_raw
 
  # modprobe pio_direct
 
  # modprobe pio_direct
  
<!--T:11-->
+
 
 
It's not always required to load both <code>pfmt_ploop1</code> (support of ploop1 format) and
 
It's not always required to load both <code>pfmt_ploop1</code> (support of ploop1 format) and
 
<code>pfmt_raw</code> (support of raw format). If we're going to use ploop1 image file,
 
<code>pfmt_raw</code> (support of raw format). If we're going to use ploop1 image file,
Line 39: Line 31:
 
plans to snapshot it, <code>pfmt_ploop1</code> can be omitted.
 
plans to snapshot it, <code>pfmt_ploop1</code> can be omitted.
  
== Initialize image file == <!--T:12-->
+
== Initialize image file ==
 +
 
  
<!--T:13-->
 
 
In-kernel ploop operates on image files of "<code>raw</code>" or "<code>ploop1</code>" format.
 
In-kernel ploop operates on image files of "<code>raw</code>" or "<code>ploop1</code>" format.
 
An image file should be created and initialized (entirely in user-space)
 
An image file should be created and initialized (entirely in user-space)
 
before asking in-kernel ploop to start using it.
 
before asking in-kernel ploop to start using it.
  
<!--T:14-->
 
 
To just create and initialize a ploop image file with a GPT partition table and an ext4 filesystem inside:
 
To just create and initialize a ploop image file with a GPT partition table and an ext4 filesystem inside:
  
  <!--T:15-->
+
  # ploop init -s 1g -t ext4 /ploop.image
# ploop init -s 1g -t ext4 /ploop.image
 
  
<!--T:16-->
 
 
where <code>/ploop.image</code> is full path to new image file and 1g is block-device size
 
where <code>/ploop.image</code> is full path to new image file and 1g is block-device size
 
equal to 1GB. This command succeed only if the file <code>/ploop.image</code> didn't
 
equal to 1GB. This command succeed only if the file <code>/ploop.image</code> didn't
 
exist at the time of running "<code>ploop init</code>".
 
exist at the time of running "<code>ploop init</code>".
  
== Mount == <!--T:17-->
+
== Mount ==
  
<!--T:18-->
 
 
Assuming that previous steps were done, the following command is used to "mount"
 
Assuming that previous steps were done, the following command is used to "mount"
 
ploop device over image file.
 
ploop device over image file.
  
  <!--T:19-->
+
  # ploop mount /ploop.image
# ploop mount /ploop.image
 
  
<!--T:20-->
 
 
Since this point, <code>/dev/ploopXXXX</code> is operable. One can read/write any data from/to
 
Since this point, <code>/dev/ploopXXXX</code> is operable. One can read/write any data from/to
 
it (e.g. with "dd"), manipulate partition table on it (with <code>parted</code>, since ploop uses GUID Partition Table, or GPT), format it with <code>mkfs.ext4</code> and mount it on some mount-point. In the other words, since
 
it (e.g. with "dd"), manipulate partition table on it (with <code>parted</code>, since ploop uses GUID Partition Table, or GPT), format it with <code>mkfs.ext4</code> and mount it on some mount-point. In the other words, since
 
now <code>/dev/ploop0</code> can be used as any other ordinary block device.
 
now <code>/dev/ploop0</code> can be used as any other ordinary block device.
  
== Snapshot == <!--T:21-->
+
== Snapshot ==
  
<!--T:22-->
 
 
Let <code>/dev/ploop0</code> be a ploop device running over <code>/ploop.image</code> (i.e. step 5 was
 
Let <code>/dev/ploop0</code> be a ploop device running over <code>/ploop.image</code> (i.e. step 5 was
 
done) and <code>/ploop-delta.image</code> - some full path to non-existent file. Then the
 
done) and <code>/ploop-delta.image</code> - some full path to non-existent file. Then the
 
command:
 
command:
  
  <!--T:23-->
+
  # ploop snapshot -d /dev/ploop0 /ploop-delta.image
# ploop snapshot -d /dev/ploop0 /ploop-delta.image
 
  
<!--T:24-->
 
 
will create empty "ploop1" image file and register it in kernel ploop
 
will create empty "ploop1" image file and register it in kernel ploop
 
forming "snapshotted" configuration <code>top_delta → base_delta</code> where <code>base_delta</code>
 
forming "snapshotted" configuration <code>top_delta → base_delta</code> where <code>base_delta</code>
 
is <code>/ploop.image</code> and <code>top_delta</code> is <code>/ploop-delta.image</code>.
 
is <code>/ploop.image</code> and <code>top_delta</code> is <code>/ploop-delta.image</code>.
  
<!--T:25-->
 
 
Since now, all i/o targeted at <code>/dev/ploop0</code> will change only <code>top_delta</code>.
 
Since now, all i/o targeted at <code>/dev/ploop0</code> will change only <code>top_delta</code>.
 
Actually, while performing snapshot operation in kernel, ploop re-open
 
Actually, while performing snapshot operation in kernel, ploop re-open
Line 92: Line 74:
 
for example, to backup <code>base_delta</code>.
 
for example, to backup <code>base_delta</code>.
  
<!--T:26-->
 
 
It's allowable to snapshot snapshotted configuration. Following example above,
 
It's allowable to snapshot snapshotted configuration. Following example above,
 
the command:
 
the command:
  
  <!--T:27-->
+
  # ploop snapshot -d /dev/ploop0 /ploop-delta1.image
# ploop snapshot -d /dev/ploop0 /ploop-delta1.image
 
  
<!--T:28-->
 
 
will form <code>top_delta → delta → base_delta</code> configuration where:
 
will form <code>top_delta → delta → base_delta</code> configuration where:
  
  <!--T:29-->
+
  top_delta := /ploop-delta1.image
top_delta := /ploop-delta1.image
 
 
  delta := /ploop-delta.image
 
  delta := /ploop-delta.image
 
  base_delta := /ploop.image.
 
  base_delta := /ploop.image.
  
<!--T:30-->
 
 
All deltas in snapshotted configuration are enumerated in kernel ploop in
 
All deltas in snapshotted configuration are enumerated in kernel ploop in
 
the natural order starting from 0 for <code>base_delta</code>:
 
the natural order starting from 0 for <code>base_delta</code>:
  
  <!--T:31-->
+
  base_delta' number is 0
base_delta' number is 0
 
 
  delta' number is 1
 
  delta' number is 1
 
  top_delta' number is 2.
 
  top_delta' number is 2.
  
<!--T:32-->
 
 
This knowledge is useful for online merge below.
 
This knowledge is useful for online merge below.
  
== Merge == <!--T:33-->
+
== Merge ==
  
<!--T:34-->
 
 
Merge operation implies copying all new data from an upper delta to a lower
 
Merge operation implies copying all new data from an upper delta to a lower
 
delta. In simplest case of delta2-->delta1 configuration, merge will copy
 
delta. In simplest case of delta2-->delta1 configuration, merge will copy
Line 127: Line 101:
 
merge will copy new data from <deltaN, ..., delta2> to delta1.
 
merge will copy new data from <deltaN, ..., delta2> to delta1.
  
<!--T:35-->
 
 
There are two types of merge: offline and online. "offline" means that we have
 
There are two types of merge: offline and online. "offline" means that we have
 
a bunch of stand-alone image files w/o kernel ploop running over them.
 
a bunch of stand-alone image files w/o kernel ploop running over them.
Line 134: Line 107:
 
device or freeze upper-layer apps).
 
device or freeze upper-layer apps).
  
=== Offline merge === <!--T:36-->
+
=== Offline merge ===
  
<!--T:37-->
 
 
ploop has no heruistic about determining raw/ploop1 format of image file. So,
 
ploop has no heruistic about determining raw/ploop1 format of image file. So,
 
in case of offline merge, user should specify the format of base_delta
 
in case of offline merge, user should specify the format of base_delta
 
explicitly. If it's ploop1, merge command looks like:
 
explicitly. If it's ploop1, merge command looks like:
  
  <!--T:38-->
+
  # ploop merge /ploop-delta.image /ploop.image
# ploop merge /ploop-delta.image /ploop.image
 
  
<!--T:39-->
 
 
This will merge /ploop-delta.image into /ploop.image. More than one source
 
This will merge /ploop-delta.image into /ploop.image. More than one source
 
delta can be specified:
 
delta can be specified:
  
  <!--T:40-->
+
  # ploop merge /ploop-d2.image /ploop-d1.image /ploop-d.image /ploop.image
# ploop merge /ploop-d2.image /ploop-d1.image /ploop-d.image /ploop.image
+
 
  
<!--T:41-->
 
 
This will merge /ploop-d2.image, /ploop-d1.image and /ploop-d.image into
 
This will merge /ploop-d2.image, /ploop-d1.image and /ploop-d.image into
 
/ploop.image.
 
/ploop.image.
  
<!--T:42-->
+
 
 
When merge completed, source deltas can be deleted because all data that
 
When merge completed, source deltas can be deleted because all data that
 
was present in them has been copied to destination delta.
 
was present in them has been copied to destination delta.
  
<!--T:43-->
+
 
 
For raw format, "-f raw" should be added as option. E.g.:
 
For raw format, "-f raw" should be added as option. E.g.:
  
  <!--T:44-->
+
  # ploop merge -f raw /ploop-d1.image /ploop.image
# ploop merge -f raw /ploop-d1.image /ploop.image
 
  
=== Online merge === <!--T:45-->
+
=== Online merge ===
  
<!--T:46-->
 
 
In this case user should only specify ploop device and a range of deltas in the
 
In this case user should only specify ploop device and a range of deltas in the
 
form of LEVEL1..LEVEL2 where LEVEL1 and LEVEL2 should be non-negative integers
 
form of LEVEL1..LEVEL2 where LEVEL1 and LEVEL2 should be non-negative integers
Line 174: Line 141:
 
deregistered from kernel ploop and can be deleted by user.
 
deregistered from kernel ploop and can be deleted by user.
  
<!--T:47-->
 
 
Let's consider simple snapshotted configuration as example:
 
Let's consider simple snapshotted configuration as example:
  
  <!--T:48-->
+
  # ploop mount -f ploop1 -d /dev/ploop0 /ploop.image
# ploop mount -f ploop1 -d /dev/ploop0 /ploop.image
 
 
  # ploop snapshot -d /dev/ploop0 /ploop-delta.image
 
  # ploop snapshot -d /dev/ploop0 /ploop-delta.image
 
  # ploop snapshot -d /dev/ploop0 /ploop-delta1.image
 
  # ploop snapshot -d /dev/ploop0 /ploop-delta1.image
  
<!--T:49-->
 
 
In this configuration the command:
 
In this configuration the command:
  
  <!--T:50-->
+
  # ploop merge -d /dev/ploop0 -l 0..2
# ploop merge -d /dev/ploop0 -l 0..2
 
  
<!--T:51-->
 
 
will merge /ploop-delta1.image and /ploop-delta.image into /ploop.image. Here
 
will merge /ploop-delta1.image and /ploop-delta.image into /ploop.image. Here
 
/ploop-delta1.image and /ploop-delta.image are source deltas and can be
 
/ploop-delta1.image and /ploop-delta.image are source deltas and can be
 
deleted.
 
deleted.
  
<!--T:52-->
 
 
Alternatively, the command:
 
Alternatively, the command:
  
  <!--T:53-->
+
  # ploop merge -d /dev/ploop0 -l 0..1
# ploop merge -d /dev/ploop0 -l 0..1
 
  
<!--T:54-->
 
 
will merge /ploop-delta.image into /ploop.image. Here /ploop-delta.image is
 
will merge /ploop-delta.image into /ploop.image. Here /ploop-delta.image is
 
source delta and can be deleted.
 
source delta and can be deleted.
  
<!--T:55-->
 
 
The last example is:
 
The last example is:
  
  <!--T:56-->
+
  # ploop merge -d /dev/ploop0 -l 1..2
# ploop merge -d /dev/ploop0 -l 1..2
 
  
<!--T:57-->
 
 
It will merge /ploop-delta1.image into /ploop-delta.image. Here
 
It will merge /ploop-delta1.image into /ploop-delta.image. Here
 
/ploop-delta1.image is source delta and can be deleted.
 
/ploop-delta1.image is source delta and can be deleted.
  
== Migration support == <!--T:58-->
+
== Migration support ==
  
<!--T:59-->
 
 
Assuming that /dev/ploop0 is ploop device running over /ploop.image,
 
Assuming that /dev/ploop0 is ploop device running over /ploop.image,
 
/ploop1.image is the path to non-existent file and external_stop is some
 
/ploop1.image is the path to non-existent file and external_stop is some
 
executable script or binary, the following command:
 
executable script or binary, the following command:
  
  <!--T:60-->
+
  # ploop copy -s /dev/ploop0 -d /ploop1.image -F external_stop
# ploop copy -s /dev/ploop0 -d /ploop1.image -F external_stop
 
  
<!--T:61-->
 
 
will copy /home/ploop.image to /home/ploop1.image iteratively. external_stop
 
will copy /home/ploop.image to /home/ploop1.image iteratively. external_stop
 
should be an utility that completely freeze all i/o targeted at ploop device.
 
should be an utility that completely freeze all i/o targeted at ploop device.
 
For instance, container freeze.
 
For instance, container freeze.
  
<!--T:62-->
 
 
From user view, the command above should be equivalent to "external_stop;
 
From user view, the command above should be equivalent to "external_stop;
 
cp /home/ploop.image /home/ploop1.image". The benefit of "ploop copy" is
 
cp /home/ploop.image /home/ploop1.image". The benefit of "ploop copy" is
Line 235: Line 187:
 
amount of data after external_stop.
 
amount of data after external_stop.
  
<!--T:63-->
 
 
It's also possible to split copy operation in two nodes:
 
It's also possible to split copy operation in two nodes:
  
  <!--T:64-->
+
  node01# ploop copy -d /ploop1.image
node01# ploop copy -d /ploop1.image
 
 
  node02# ploop copy -s /dev/ploop0 -F external_stop
 
  node02# ploop copy -s /dev/ploop0 -F external_stop
  
<!--T:65-->
 
 
In this case "ploop copy" on node02 will write data to standard output
 
In this case "ploop copy" on node02 will write data to standard output
 
in some special binary format and "ploop copy" on node01 will read data
 
in some special binary format and "ploop copy" on node01 will read data
from stndard input, parse that special format and store data in
+
from standard input, parse that special format and store data in
 
/home/ploop1.image ("-s" stands for "source", "-d" stands for "destination").
 
/home/ploop1.image ("-s" stands for "source", "-d" stands for "destination").
 
This should work if standard output of node02 and standard input of node01
 
This should work if standard output of node02 and standard input of node01
 
are bound via pipe or socket connection.
 
are bound via pipe or socket connection.
  
== Grow ploop device == <!--T:66-->
+
== Grow ploop device ==
  
<!--T:67-->
+
"ploop grow" command is to extend image file as necessary (offline or online)
"ploop grow" command is to extend image file as neccessary (offline or online)
 
 
and propagate new block-device size to linux kernel (in online case).
 
and propagate new block-device size to linux kernel (in online case).
  
=== Offline grow === <!--T:68-->
+
=== Offline grow ===
  
<!--T:69-->
 
 
To grow image file offline, user should specify its format explicitly. The
 
To grow image file offline, user should specify its format explicitly. The
 
default is "ploop1" format:
 
default is "ploop1" format:
  
  <!--T:70-->
+
  # ploop grow -s 32g /ploop.image
# ploop grow -s 32g /ploop.image
 
  
<!--T:71-->
 
 
will re-arrange "ploop1" image file /ploop.image to become 32GB size long.
 
will re-arrange "ploop1" image file /ploop.image to become 32GB size long.
  
<!--T:72-->
 
 
For "raw" format, the command:
 
For "raw" format, the command:
  
  <!--T:73-->
+
  # ploop grow -s 32g -f raw /ploop.image
# ploop grow -s 32g -f raw /ploop.image
 
  
<!--T:74-->
 
 
will do the same.
 
will do the same.
  
=== Online grow === <!--T:75-->
+
=== Online grow ===
  
<!--T:76-->
 
 
Assuming that /dev/ploop0 is runnung ploop device, the command:
 
Assuming that /dev/ploop0 is runnung ploop device, the command:
  
  <!--T:77-->
+
  # ploop grow -s 32g -d /dev/ploop0
# ploop grow -s 32g -d /dev/ploop0
 
  
<!--T:78-->
 
 
will re-arrange underlying image file, update internal kernel ploop structures
 
will re-arrange underlying image file, update internal kernel ploop structures
 
and propagate changes to linux kernel making ploop device 32GB size long.
 
and propagate changes to linux kernel making ploop device 32GB size long.
  
<!--T:79-->
 
 
If user had ext4 formatted and mounted on /dev/ploop0, ext4 fs can be extended
 
If user had ext4 formatted and mounted on /dev/ploop0, ext4 fs can be extended
 
online (when "ploop grow" completed):
 
online (when "ploop grow" completed):
  
  <!--T:80-->
+
  # resize2fs /dev/ploop0 32g
# resize2fs /dev/ploop0 32g
 
  
== Ballooning == <!--T:81-->
+
== Ballooning ==
  
<!--T:82-->
 
 
ploop doesn't support pure shrinking block-device size due to lack of online
 
ploop doesn't support pure shrinking block-device size due to lack of online
 
shrink support in ext4. As a workaround, "ballooning" technique is proposed.
 
shrink support in ext4. As a workaround, "ballooning" technique is proposed.
Line 307: Line 243:
 
tail of image file.
 
tail of image file.
  
<!--T:83-->
 
 
Desired outcome is image file of smaller size. However, it's quite possible
 
Desired outcome is image file of smaller size. However, it's quite possible
 
that inflated balloon file will span only blocks that were never touched
 
that inflated balloon file will span only blocks that were never touched
Line 313: Line 248:
 
case nothing will be relocated and nothing truncated.
 
case nothing will be relocated and nothing truncated.
  
<!--T:84-->
 
 
So, if balloon operation succeeded, it's only guaranteed that user of ploop
 
So, if balloon operation succeeded, it's only guaranteed that user of ploop
 
device won't be able to consume more space than initial block device size
 
device won't be able to consume more space than initial block device size
Line 320: Line 254:
 
balloon operation will result in significant truncate of image file.
 
balloon operation will result in significant truncate of image file.
  
<!--T:85-->
 
 
To enable ballooning, ext4 residing on ploop device should be mounted
 
To enable ballooning, ext4 residing on ploop device should be mounted
 
with special "balloon_ino" option:
 
with special "balloon_ino" option:
  
  <!--T:86-->
+
  # mount -t ext4 -o balloon_ino=12 /dev/ploop0 /mnt_ploop
# mount -t ext4 -o balloon_ino=12 /dev/ploop0 /mnt_ploop
 
  
<!--T:87-->
 
 
where 12 is inode number of balloon file as reported by "ls -i".
 
where 12 is inode number of balloon file as reported by "ls -i".
 
(it's assumed that initially, e.g. while constructing container,
 
(it's assumed that initially, e.g. while constructing container,
 
someone mounted ext4 on ploop device w/o balloon_ino option, then created
 
someone mounted ext4 on ploop device w/o balloon_ino option, then created
 
empty balloon file there, found out its inode number and saved it for the
 
empty balloon file there, found out its inode number and saved it for the
future use)
+
future use).
  
<!--T:88-->
 
 
Currently, only online ballooning is supported. The following command performs
 
Currently, only online ballooning is supported. The following command performs
 
this operation:
 
this operation:
  
  <!--T:89-->
+
  # ploop balloon change -s 1g -d /dev/ploop0 -m /mnt_ploop
# ploop balloon change -s 1g -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:90-->
 
 
where 1g is desired new size of balloon file, /dev/ploop0 is ploop block
 
where 1g is desired new size of balloon file, /dev/ploop0 is ploop block
 
device, /mnt_ploop is mount-point where ext4 residing on /dev/ploop0 is
 
device, /mnt_ploop is mount-point where ext4 residing on /dev/ploop0 is
 
mounted to.
 
mounted to.
  
<!--T:91-->
 
 
If balloon file was empty, the command above simply inflates it to become
 
If balloon file was empty, the command above simply inflates it to become
 
1GB size. If it was non-empty but smaller than 1GB, that command extends it
 
1GB size. If it was non-empty but smaller than 1GB, that command extends it
Line 353: Line 280:
 
command does nothing.
 
command does nothing.
  
<!--T:92-->
 
 
Along with "change" sub-command, "ploop balloon" supports a few auxiliary ones:
 
Along with "change" sub-command, "ploop balloon" supports a few auxiliary ones:
  
  <!--T:93-->
+
  # ploop balloon show -m /mnt_ploop
# ploop balloon show -m /mnt_ploop
 
  
<!--T:94-->
 
 
will show current ploop balloon size.
 
will show current ploop balloon size.
  
  <!--T:95-->
+
  # ploop balloon status -d /dev/ploop0 -m /mnt_ploop
# ploop balloon status -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:96-->
+
will report current in-kernel status of maintenance like "merge in progress",
will report current in-kernel status of maintainance like "merge in progress",
 
 
"grow in progress", "ballooning started", etc. This is useful because on the
 
"grow in progress", "ballooning started", etc. This is useful because on the
 
one hand balloon operation can't be performed while merge or grow is in
 
one hand balloon operation can't be performed while merge or grow is in
Line 372: Line 294:
 
someone before its completion.
 
someone before its completion.
  
  <!--T:97-->
+
  # ploop balloon clear -d /dev/ploop0 -m /mnt_ploop
# ploop balloon clear -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:98-->
+
will flush stale in-kernel "BALLOON" state of maintenance. This is useful if
will flush stale in-kernel "BALLOON" state of maintainance. This is useful if
 
 
previous "ploop balloon" died early leaving in-kernel ploop locked.
 
previous "ploop balloon" died early leaving in-kernel ploop locked.
  
  <!--T:99-->
+
  # ploop balloon complete -d /dev/ploop0 -m /mnt_ploop
# ploop balloon complete -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:100-->
 
 
will complete previously interrupted balloon operation. An expectation is that
 
will complete previously interrupted balloon operation. An expectation is that
 
user monitors exit status of ploop commands he/she runs in some way. If
 
user monitors exit status of ploop commands he/she runs in some way. If
 
user issued "ploop balloon change" and it was killed in the middle, the user
 
user issued "ploop balloon change" and it was killed in the middle, the user
knows that it didn't complete with zero exit status. Then user shoud inquire
+
knows that it didn't complete with zero exit status. Then user should inquire
current maintainance state with "ploop balloon status" command, and, if it
+
current maintenance state with "ploop balloon status" command, and, if it
 
reported "FBLOAD" or "RELOC", the user should use "ploop balloon complete"
 
reported "FBLOAD" or "RELOC", the user should use "ploop balloon complete"
before proceeding with any other maintainance operations (shanphsot, merge,
+
before proceeding with any other maintenance operations (snapshot, merge,
 
grow, balloon).
 
grow, balloon).
  
  <!--T:101-->
+
  # ploop balloon check -d /dev/ploop0 -m /mnt_ploop
# ploop balloon check -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:102-->
 
 
will check whether existent balloon file was properly processed. This is useful
 
will check whether existent balloon file was properly processed. This is useful
 
if previous "ploop balloon" was interrupted, but "ploop balloon status"
 
if previous "ploop balloon" was interrupted, but "ploop balloon status"
reports "OFF" or "BALLOON" maintainance state. In this case it's possible
+
reports "OFF" or "BALLOON" maintenance state. In this case it's possible
 
that balloon file was inflated but no further processing happened.
 
that balloon file was inflated but no further processing happened.
  
<!--T:103-->
 
 
"ploop balloon check" reports total number of free blocks in existent balloon
 
"ploop balloon check" reports total number of free blocks in existent balloon
 
file. If it's not zero, the user should use the following command to repair
 
file. If it's not zero, the user should use the following command to repair
 
balloon:
 
balloon:
  
  <!--T:104-->
+
  # ploop balloon repair -d /dev/ploop0 -m /mnt_ploop
# ploop balloon repair -d /dev/ploop0 -m /mnt_ploop
 
  
<!--T:105-->
 
 
This command does essentially the same as "ploop balloon change" but w/o
 
This command does essentially the same as "ploop balloon change" but w/o
 
inflating balloon.
 
inflating balloon.
  
== See also == <!--T:106-->
+
== See also ==
  
<!--T:107-->
 
 
* [[Ploop]]
 
* [[Ploop]]
 
* {{Man|ploop|8}}
 
* {{Man|ploop|8}}
</translate>
 
  
 
[[Category: Storage]]
 
[[Category: Storage]]

Revision as of 15:17, 10 March 2021

This document explains how to use user-space ploop utility for typical use-cases.

Warning.svg Warning: The commands below are low-level stuff. It's better to use vzctl which has all the features in place.

Getting help

All user-space ploop management operations are available via "ploop" utility. Run it w/o args to get help:

# ploop

Run with a cmd as the only arg to get cmd-specific help, e.g.:

# ploop init

Load modules

Further ploop commands assume that all necessary modules are loaded:

# modprobe ploop
# modprobe pfmt_ploop1
# modprobe pfmt_raw
# modprobe pio_direct


It's not always required to load both pfmt_ploop1 (support of ploop1 format) and pfmt_raw (support of raw format). If we're going to use ploop1 image file, loading pfmt_raw can be omitted. If we're going to use raw image file and have no plans to snapshot it, pfmt_ploop1 can be omitted.

Initialize image file

In-kernel ploop operates on image files of "raw" or "ploop1" format. An image file should be created and initialized (entirely in user-space) before asking in-kernel ploop to start using it.

To just create and initialize a ploop image file with a GPT partition table and an ext4 filesystem inside:

# ploop init -s 1g -t ext4 /ploop.image

where /ploop.image is full path to new image file and 1g is block-device size equal to 1GB. This command succeed only if the file /ploop.image didn't exist at the time of running "ploop init".

Mount

Assuming that previous steps were done, the following command is used to "mount" ploop device over image file.

# ploop mount /ploop.image

Since this point, /dev/ploopXXXX is operable. One can read/write any data from/to it (e.g. with "dd"), manipulate partition table on it (with parted, since ploop uses GUID Partition Table, or GPT), format it with mkfs.ext4 and mount it on some mount-point. In the other words, since now /dev/ploop0 can be used as any other ordinary block device.

Snapshot

Let /dev/ploop0 be a ploop device running over /ploop.image (i.e. step 5 was done) and /ploop-delta.image - some full path to non-existent file. Then the command:

# ploop snapshot -d /dev/ploop0 /ploop-delta.image

will create empty "ploop1" image file and register it in kernel ploop forming "snapshotted" configuration top_delta → base_delta where base_delta is /ploop.image and top_delta is /ploop-delta.image.

Since now, all i/o targeted at /dev/ploop0 will change only top_delta. Actually, while performing snapshot operation in kernel, ploop re-open base_delta read-only. So, when "ploop snapshot" is completed, it's quite safe, for example, to backup base_delta.

It's allowable to snapshot snapshotted configuration. Following example above, the command:

# ploop snapshot -d /dev/ploop0 /ploop-delta1.image

will form top_delta → delta → base_delta configuration where:

top_delta := /ploop-delta1.image
delta := /ploop-delta.image
base_delta := /ploop.image.

All deltas in snapshotted configuration are enumerated in kernel ploop in the natural order starting from 0 for base_delta:

base_delta' number is 0
delta' number is 1
top_delta' number is 2.

This knowledge is useful for online merge below.

Merge

Merge operation implies copying all new data from an upper delta to a lower delta. In simplest case of delta2-->delta1 configuration, merge will copy new data from delta2 to delta1. In case of deltaN-->...-->delta1 configuration. merge will copy new data from <deltaN, ..., delta2> to delta1.

There are two types of merge: offline and online. "offline" means that we have a bunch of stand-alone image files w/o kernel ploop running over them. "online" means that we have kernel ploop running over them. It's OK to perform online merge concurrently with other i/o activity (i.e. no need to stop ploop device or freeze upper-layer apps).

Offline merge

ploop has no heruistic about determining raw/ploop1 format of image file. So, in case of offline merge, user should specify the format of base_delta explicitly. If it's ploop1, merge command looks like:

# ploop merge /ploop-delta.image /ploop.image

This will merge /ploop-delta.image into /ploop.image. More than one source delta can be specified:

# ploop merge /ploop-d2.image /ploop-d1.image /ploop-d.image /ploop.image


This will merge /ploop-d2.image, /ploop-d1.image and /ploop-d.image into /ploop.image.


When merge completed, source deltas can be deleted because all data that was present in them has been copied to destination delta.


For raw format, "-f raw" should be added as option. E.g.:

# ploop merge -f raw /ploop-d1.image /ploop.image

Online merge

In this case user should only specify ploop device and a range of deltas in the form of LEVEL1..LEVEL2 where LEVEL1 and LEVEL2 should be non-negative integers corresponding to in-kernel delta enumeration (see the end of 5th section above) and LEVEL1 must be lesser than LEVEL2. When merge completed, source deltas are deregistered from kernel ploop and can be deleted by user.

Let's consider simple snapshotted configuration as example:

# ploop mount -f ploop1 -d /dev/ploop0 /ploop.image
# ploop snapshot -d /dev/ploop0 /ploop-delta.image
# ploop snapshot -d /dev/ploop0 /ploop-delta1.image

In this configuration the command:

# ploop merge -d /dev/ploop0 -l 0..2

will merge /ploop-delta1.image and /ploop-delta.image into /ploop.image. Here /ploop-delta1.image and /ploop-delta.image are source deltas and can be deleted.

Alternatively, the command:

# ploop merge -d /dev/ploop0 -l 0..1

will merge /ploop-delta.image into /ploop.image. Here /ploop-delta.image is source delta and can be deleted.

The last example is:

# ploop merge -d /dev/ploop0 -l 1..2

It will merge /ploop-delta1.image into /ploop-delta.image. Here /ploop-delta1.image is source delta and can be deleted.

Migration support

Assuming that /dev/ploop0 is ploop device running over /ploop.image, /ploop1.image is the path to non-existent file and external_stop is some executable script or binary, the following command:

# ploop copy -s /dev/ploop0 -d /ploop1.image -F external_stop

will copy /home/ploop.image to /home/ploop1.image iteratively. external_stop should be an utility that completely freeze all i/o targeted at ploop device. For instance, container freeze.

From user view, the command above should be equivalent to "external_stop; cp /home/ploop.image /home/ploop1.image". The benefit of "ploop copy" is minimizing duration of freezed state: it will try to copy the most part of data before calling external_stop and only some (hopefully small) amount of data after external_stop.

It's also possible to split copy operation in two nodes:

node01# ploop copy -d /ploop1.image
node02# ploop copy -s /dev/ploop0 -F external_stop

In this case "ploop copy" on node02 will write data to standard output in some special binary format and "ploop copy" on node01 will read data from standard input, parse that special format and store data in /home/ploop1.image ("-s" stands for "source", "-d" stands for "destination"). This should work if standard output of node02 and standard input of node01 are bound via pipe or socket connection.

Grow ploop device

"ploop grow" command is to extend image file as necessary (offline or online) and propagate new block-device size to linux kernel (in online case).

Offline grow

To grow image file offline, user should specify its format explicitly. The default is "ploop1" format:

# ploop grow -s 32g /ploop.image

will re-arrange "ploop1" image file /ploop.image to become 32GB size long.

For "raw" format, the command:

# ploop grow -s 32g -f raw /ploop.image

will do the same.

Online grow

Assuming that /dev/ploop0 is runnung ploop device, the command:

# ploop grow -s 32g -d /dev/ploop0

will re-arrange underlying image file, update internal kernel ploop structures and propagate changes to linux kernel making ploop device 32GB size long.

If user had ext4 formatted and mounted on /dev/ploop0, ext4 fs can be extended online (when "ploop grow" completed):

# resize2fs /dev/ploop0 32g

Ballooning

ploop doesn't support pure shrinking block-device size due to lack of online shrink support in ext4. As a workaround, "ballooning" technique is proposed. Ballooning operation consists of inflating special balloon file in user-space (the file will be invisible for ordinary users, e.g. inside container), loading fiemap info of inflated balloon to kernel, relocating blocks of image file from the tail to the space specified by fiemap info and truncating tail of image file.

Desired outcome is image file of smaller size. However, it's quite possible that inflated balloon file will span only blocks that were never touched before. They will look as "not allocated" space from kernel ploop view. In this case nothing will be relocated and nothing truncated.

So, if balloon operation succeeded, it's only guaranteed that user of ploop device won't be able to consume more space than initial block device size minus size of inflated balloon. On the other hand, if user of block device used a lot of space on it, then freed significant part of used space, balloon operation will result in significant truncate of image file.

To enable ballooning, ext4 residing on ploop device should be mounted with special "balloon_ino" option:

# mount -t ext4 -o balloon_ino=12 /dev/ploop0 /mnt_ploop

where 12 is inode number of balloon file as reported by "ls -i". (it's assumed that initially, e.g. while constructing container, someone mounted ext4 on ploop device w/o balloon_ino option, then created empty balloon file there, found out its inode number and saved it for the future use).

Currently, only online ballooning is supported. The following command performs this operation:

# ploop balloon change -s 1g -d /dev/ploop0 -m /mnt_ploop

where 1g is desired new size of balloon file, /dev/ploop0 is ploop block device, /mnt_ploop is mount-point where ext4 residing on /dev/ploop0 is mounted to.

If balloon file was empty, the command above simply inflates it to become 1GB size. If it was non-empty but smaller than 1GB, that command extends it to given size (1GB). If it was non-empty but larger that 1GB, that command truncates it down to given size. If it was exactly 1GB size, the command does nothing.

Along with "change" sub-command, "ploop balloon" supports a few auxiliary ones:

# ploop balloon show -m /mnt_ploop

will show current ploop balloon size.

# ploop balloon status -d /dev/ploop0 -m /mnt_ploop

will report current in-kernel status of maintenance like "merge in progress", "grow in progress", "ballooning started", etc. This is useful because on the one hand balloon operation can't be performed while merge or grow is in progress, and on the other hand previous "ploop balloon" could be killed by someone before its completion.

# ploop balloon clear -d /dev/ploop0 -m /mnt_ploop

will flush stale in-kernel "BALLOON" state of maintenance. This is useful if previous "ploop balloon" died early leaving in-kernel ploop locked.

# ploop balloon complete -d /dev/ploop0 -m /mnt_ploop

will complete previously interrupted balloon operation. An expectation is that user monitors exit status of ploop commands he/she runs in some way. If user issued "ploop balloon change" and it was killed in the middle, the user knows that it didn't complete with zero exit status. Then user should inquire current maintenance state with "ploop balloon status" command, and, if it reported "FBLOAD" or "RELOC", the user should use "ploop balloon complete" before proceeding with any other maintenance operations (snapshot, merge, grow, balloon).

# ploop balloon check -d /dev/ploop0 -m /mnt_ploop

will check whether existent balloon file was properly processed. This is useful if previous "ploop balloon" was interrupted, but "ploop balloon status" reports "OFF" or "BALLOON" maintenance state. In this case it's possible that balloon file was inflated but no further processing happened.

"ploop balloon check" reports total number of free blocks in existent balloon file. If it's not zero, the user should use the following command to repair balloon:

# ploop balloon repair -d /dev/ploop0 -m /mnt_ploop

This command does essentially the same as "ploop balloon change" but w/o inflating balloon.

See also