Difference between revisions of "Vzmigrate filesystem aware"
(→Other issues) |
|||
| (19 intermediate revisions by 2 users not shown) | |||
| Line 2: | Line 2: | ||
Reading thoroughly quick installation documentation, it says "It is recommended to use a separate partition for container's private directories (by default /vz/private/<veid>)". As much as I searched the net, I have not found anything about it. | Reading thoroughly quick installation documentation, it says "It is recommended to use a separate partition for container's private directories (by default /vz/private/<veid>)". As much as I searched the net, I have not found anything about it. | ||
| − | This is | + | This is something I thought about long ago, but I considered difficult to do in current |
| − | implementation. Now things | + | implementation. Now things have changed. I have received directives in my job to have each container in separated filesystems insulated from the rest of containers. |
| − | Actually vzmigrate does not | + | Actually vzmigrate does not take into account this issue. vzmigrate assume data are available when migration occurs and it does not know about filesystems neither mounted filesystems at all. |
| − | Because of this matters, this issue I had put | + | Because of this matters, this issue I had put in his moment off until new order has got back to the scene. |
| − | Openvz has its own mechanisms to perform actions when starting a container. Migration knows about container state before migrating so after migrating data it can | + | Openvz has its own mechanisms to perform actions when starting a container. Migration knows about container state before migrating so after migrating data it can return the container to the previous state. |
When considering migration that take into account insulated filesystems, it involve situations not considered by standard migration. By example, migrating a stopped container with its own filesystem, probably it will not have available its files because openvz supposedly will have scripts to unmount filesystem when container is stopped. | When considering migration that take into account insulated filesystems, it involve situations not considered by standard migration. By example, migrating a stopped container with its own filesystem, probably it will not have available its files because openvz supposedly will have scripts to unmount filesystem when container is stopped. | ||
| Line 24: | Line 24: | ||
To get Openvz mount the filesystem before starting the container, '''vps.premount''' can have: | To get Openvz mount the filesystem before starting the container, '''vps.premount''' can have: | ||
<pre> | <pre> | ||
| + | #!/bin/bash | ||
source ${VE_CONFFILE} | source ${VE_CONFFILE} | ||
[ -d ${VE_PRIVATE} ] || mkdir ${VE_PRIVATE} | [ -d ${VE_PRIVATE} ] || mkdir ${VE_PRIVATE} | ||
| Line 29: | Line 30: | ||
</pre> | </pre> | ||
| − | Similarly to get the filesystem unmounted after the container is stopped, we can have in vps.postumount: | + | Similarly to get the filesystem unmounted after the container is stopped, we can have in '''vps.postumount''': |
<pre> | <pre> | ||
#!/bin/bash | #!/bin/bash | ||
| Line 43: | Line 44: | ||
this optional device can be set with configuration parameter '''VE_DUMP_DEVICE'''. This parameter has only meaning if we share between HN the device over which container is set up. | this optional device can be set with configuration parameter '''VE_DUMP_DEVICE'''. This parameter has only meaning if we share between HN the device over which container is set up. | ||
| − | == Migrations depending on | + | == Migrations depending on context == |
| − | [[ | + | [[Image:Vzmigrate_a.jpeg|400px]] |
| − | Migration in this case, as you would expect, | + | Migration in this case, as you would expect, is the same as always. |
| Line 53: | Line 54: | ||
[[Image:Vzmigrate_b.jpeg|400px]] | [[Image:Vzmigrate_b.jpeg|400px]] | ||
<pre> | <pre> | ||
| − | vzmigrate --dst-device / | + | vzmigrate --dst-device /dev/sdg1 HN_target 123 |
</pre> | </pre> | ||
| − | [[ | + | [[Image:Vzmigrate_c.jpeg|400px]] |
Remove VE_DEVICE parameter from 123.conf and operate the same as shared-shared | Remove VE_DEVICE parameter from 123.conf and operate the same as shared-shared | ||
| − | |||
| − | + | [[Image:Vzmigrate_d1.jpeg|400px]] | |
| − | [[ | ||
<pre> | <pre> | ||
vzmigrate --dst-device /dev/sdm1 HN_target 123 | vzmigrate --dst-device /dev/sdm1 HN_target 123 | ||
| Line 73: | Line 72: | ||
| − | [[ | + | [[Image:Vzmigrate_d2.jpeg|400px]] |
<pre> | <pre> | ||
vzmigrate HN_target 123 | vzmigrate HN_target 123 | ||
| Line 84: | Line 83: | ||
== Undo actions == | == Undo actions == | ||
| − | The issue has | + | The issue has required a lot hard work for modifying vzmigrate to track undo actions when a error arises in the code. I felt free to reorganise undo actions in the function described on the following diagram. |
| − | [[ | + | [[Image:Vzmigrate_undo.jpeg|250px|center]] |
== Other issues == | == Other issues == | ||
| Line 96: | Line 95: | ||
mount /dev/sda1 /vz/private/123 | mount /dev/sda1 /vz/private/123 | ||
mv /var/tmp/123/* /vz/private/123 | mv /var/tmp/123/* /vz/private/123 | ||
| − | # reflect | + | # reflect these changes in /etc/vz/conf/123.conf |
| + | ... | ||
| + | VE_PRIVATE="/vz/private/$VEID" | ||
| + | VE_DEVICE="/dev/sda1" | ||
| + | ... | ||
</pre> | </pre> | ||
| + | |||
| + | == Links to modified vzmigrate == | ||
| + | ftp://ftp.uma.es/pub/Linux/openvz/ | ||
[[Category:HOWTO]] | [[Category:HOWTO]] | ||
Latest revision as of 11:24, 8 June 2012
Contents
Motivation[edit]
Reading thoroughly quick installation documentation, it says "It is recommended to use a separate partition for container's private directories (by default /vz/private/<veid>)". As much as I searched the net, I have not found anything about it.
This is something I thought about long ago, but I considered difficult to do in current implementation. Now things have changed. I have received directives in my job to have each container in separated filesystems insulated from the rest of containers.
Actually vzmigrate does not take into account this issue. vzmigrate assume data are available when migration occurs and it does not know about filesystems neither mounted filesystems at all.
Because of this matters, this issue I had put in his moment off until new order has got back to the scene.
Openvz has its own mechanisms to perform actions when starting a container. Migration knows about container state before migrating so after migrating data it can return the container to the previous state.
When considering migration that take into account insulated filesystems, it involve situations not considered by standard migration. By example, migrating a stopped container with its own filesystem, probably it will not have available its files because openvz supposedly will have scripts to unmount filesystem when container is stopped.
I have modified migration script to adapt to these circumstances. I have named as shared those containers that share filesystem with others containers (legacy) and those with their own filesystem as insulated. With this in mind, there are four migration scenario.
- shared -> shared
- shared -> insulated
- insulated -> shared
- insulated -> insulated
To get Openvz to know a container is mounted on its own filesystem, optional parameter VE_DEVICE is defined in configuration file.
To get Openvz mount the filesystem before starting the container, vps.premount can have:
#!/bin/bash
source ${VE_CONFFILE}
[ -d ${VE_PRIVATE} ] || mkdir ${VE_PRIVATE}
[ -n "$VE_DEVICE" ] && mount ${VE_DEVICE} ${VE_PRIVATE} || echo -n " "
Similarly to get the filesystem unmounted after the container is stopped, we can have in vps.postumount:
#!/bin/bash
source ${VE_CONFFILE}
if [ -n "$VE_DEVICE" ]; then
umount ${VE_PRIVATE} 2> /dev/null || echo -n ""
fi
This configuration along the use of a SAN, gives interesting advantages when migrating containers that last so long at sincronization phase. If the same device is presented in both HN, we can skip sincronization phase. Migration only have to unmount filesystem in source HN and mount the filesystem in target HN avoiding this way the copy of data.
If we want to improve and minimize migration time, we could consider sharing another device for dump/undump container and share this way the file between HN. Using this optional device can be set with configuration parameter VE_DUMP_DEVICE. This parameter has only meaning if we share between HN the device over which container is set up.
Migrations depending on context[edit]
Migration in this case, as you would expect, is the same as always.
vzmigrate --dst-device /dev/sdg1 HN_target 123
Remove VE_DEVICE parameter from 123.conf and operate the same as shared-shared
vzmigrate --dst-device /dev/sdm1 HN_target 123
vzmigrate HN_target 123
If you use a shared dump filesystem between HN.
vzmigrate --dump-device /dev/VG/dump --online HN_target 123
Undo actions[edit]
The issue has required a lot hard work for modifying vzmigrate to track undo actions when a error arises in the code. I felt free to reorganise undo actions in the function described on the following diagram.
Other issues[edit]
Creating a container over a insulated filesystem is not possible straightly because of vzctl complains if the the device is previously mounted under VE_PRIVATE. To overcome this:
vzctl create 123 --private /var/tmp/123 mkdir /vz/private/123; mount /dev/sda1 /vz/private/123 mv /var/tmp/123/* /vz/private/123 # reflect these changes in /etc/vz/conf/123.conf ... VE_PRIVATE="/vz/private/$VEID" VE_DEVICE="/dev/sda1" ...
