Difference between revisions of "Checkpointing and live migration"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(categorized)
 
(8 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
CPT is an extension to OpenVZ kernel which allows to save full state of a running VE and to restore it later on the same or on a different host in a way transparent for running applications and network connections. This technique has several applications, the most important being live (zero-downtime) migration of VEs and taking an instant snapshot of a running VE for later resume, i. e. CheckPoinTing.
+
<translate>
 +
<!--T:1-->
 +
CPT is an extension to the OpenVZ kernel which can save the full state of a running VE and to restore it later on the same or on a different host in a way transparent to running applications and network connections. This technique has several applications, the most important being live (zero-downtime) migration of VEs and taking an instant snapshot of a running VE for later resume, i.e. CheckPointing.
  
−
Before CPT, it was only possible to migrate a VE through shutdown and subsequent reboot. The procedure not only introduces quite a long downtime of network services, it is not transparent for clients using the VE, making impossible migration, when clients runs some tasks which are not tolerant to shutdowns.
+
<!--T:2-->
 +
Before CPT, it was only possible to migrate a VE through a shutdown and subsequent reboot. The procedure not only introduces quite a long downtime for network services, it is not transparent for clients using the VE, making migration impossible when clients run some tasks which are not tolerant to shutdowns.
  
−
Comparing to this old scheme, CPT allows to migrate a VE in a way, essentially invisible both for users of this VE and for external clients, using network services located inside VE. It still introduces a short delay in service, required for actual checkpoint/restore of the processes, but this delay is indistinguishable from a short interruption of network connectivity.
+
<!--T:3-->
 +
Compared with this old scheme, CPT allows migration of a VE in a way which is essentially invisible both for users of this VE and for external clients using network services located inside the VE. It still introduces a short delay in service, required for actual checkpoint/restore of the processes, but this delay is indistinguishable from a short interruption of network connectivity.
  
−
== Online migration ==
+
<!--T:4-->
 +
{{Note|In OpenVZ 7, CPT is replaced by our sub-project [http://criu.org CRIU]}}
  
−
There is special utility vzmigrate in OpenVZ distribution intended to support VE migration. With it help one can perform live (a.k.a. online) migration, i.&nbsp;e. for migration VE “freezes” for some time, and after migration it continues to work as though nothing has happened. Online migration can be performed by
+
 
 +
== Online migration == <!--T:5-->
 +
 
 +
<!--T:6-->
 +
There is a special utility vzmigrate in the OpenVZ distribution intended to support VE migration. With its help one can perform live (a.k.a. online) migration, i.e. during migration the VE “freezes” for some time, and after migration it continues to work as though nothing had happened. Online migration can be performed with:
 
<pre>vzmigrate --online <host> VEID</pre>
 
<pre>vzmigrate --online <host> VEID</pre>
−
command. During online migration all VE private data saved to an image file, which is transferred to target host.
 
  
−
In order for vzmigrate to work without asking for a password, ssh public keys from the source host should be placed to destination host's <code>authorized_keys</code>. In other words, command <code>ssh root@host</code> should not ask you for a password. See [[ssh keys]] for more info.
+
<!--T:7-->
 +
During online migration all VE private data saved to an image file, which is transferred to the target host.
 +
 
 +
<!--T:8-->
 +
In order for vzmigrate to work without asking for a password, ssh public keys from the source host should be placed in the destination host's <code>/root/.ssh/authorized_keys</code> file. In other words, command <code>ssh root@host</code> should not ask you for a password. See [[ssh keys]] for more info.
  
−
== Manual Checkpoint and Restore Functions ==
+
== Manual Checkpoint and Restore Functions == <!--T:9-->
  
−
<code>vzmigrate</code> is not strictly required to perform online migration. <code>vzctl</code> utility, accompanied with some file system backup tools, provides enough of power to do all the tasks.
+
<!--T:10-->
 +
<code>vzmigrate</code> is not strictly required to perform online migration. The <code>vzctl</code> utility, accompanied with some file system backup tools, provides enough power to do all the tasks.
  
−
VE can be checkpointed with command:
+
<!--T:11-->
 +
A VE can be checkpointed with:
 
<pre>vzctl chkpnt VEID --dumpfile <path></pre>
 
<pre>vzctl chkpnt VEID --dumpfile <path></pre>
−
This command saves all the state of running VE to dump file and stops the VE. If the option <code>--dumpfile</code> is not set, <code>vzctl</code> uses default path <code>/vz/dump/Dump.VEID</code>.  
+
This command saves all the state of a running VE to the dump file and stops the VE. If the option <code>--dumpfile</code> is not set, <code>vzctl</code> uses a default path <code>/vz/dump/Dump.VEID</code>.  
  
−
After this it is possible to restore the VE exactly in the same state executing:
+
<!--T:12-->
 +
After this it is possible to restore the VE to the same state executing:
 
<pre>vzctl restore VEID --dumpfile <path></pre>
 
<pre>vzctl restore VEID --dumpfile <path></pre>
−
If dump file and file system is transferred to another HW node, the same command can restore VE there with the same success.
+
If the dump file and file system is transferred to another HW node, the same command can restore the VE there with the same success.
  
−
It is critical requirement that file system at the moment of restore must be identical to the file system at the moment of checkpointing. If this requirement is not held, depending on severity of changes process of restoration can be aborted or the processes inside VE can see this as an external corruption of open files. When VE is restored on the same node where it was checkpointed, it is enough not to touch file system accessible by the VE. When VE is transferred to another node it is necessary to synchronize VE file system before restore. <code>vzctl</code> does not provide this functionality and external tools (f.e. <code>rsync</code>) are required.
+
<!--T:13-->
 +
It is a critical requirement that file system at the moment of restore must be identical to the file system at the moment of checkpointing. If this requirement is not met, depending on the severity of changes, the process of restoration can be aborted or the processes inside a VE can see this as an external corruption of open files. When a VE is restored on the same node where it was checkpointed, it is enough to not touch the file system accessible by the VE. When a VE is transferred to another node it is necessary to synchronize the VE file system before restore. <code>vzctl</code> does not provide this functionality and external tools (i.e. <code>rsync</code>) are required.
  
−
== Step-by-step Checkpoint and Restore ==
+
== Step-by-step Checkpoint and Restore == <!--T:14-->
  
−
Process of checkpointing can be performed by stages. It consists of three steps.
+
<!--T:15-->
 +
The process of checkpointing can be performed in stages. It consists of three steps.
  
−
First step – suspending VE. At this stage CPT moves all the processes to special beforehand known state and stops VE network interfaces. This stage can be done by
+
<!--T:16-->
 +
First step – suspend the VE. At this stage CPT moves all the processes to a special beforehand known state and stops VE network interfaces. This stage can be done with:
 
<pre>vzctl chkpnt VEID --suspend</pre>
 
<pre>vzctl chkpnt VEID --suspend</pre>
−
command. Second step – dumping VE. At this phase CPT saves state of processes and global state of VE to image file. All the process private data need to be saved: address space, register set, opened files/pipes/sockets, System V IPC structures, current working directory, signal handlers, timers, terminal settings, user identities (uid, gid, etc), process identities (pid, pgrp, sid, etc), rlimit and other data. This stage can be done by
+
 
 +
 
 +
<!--T:17-->
 +
Second step – dumping VE. At this stage CPT saves the state of processes and global state of VE to an image file. All the process private data needs to be saved: address space, register set, opened files/pipes/sockets, System V IPC structures, current working directory, signal handlers, timers, terminal settings, user identities (uid, gid, etc), process identities (pid, pgrp, sid, etc), rlimit and other data. This stage can be done with:
 
<pre>vzctl chkpnt VEID --dump --dumpfile <path></pre>
 
<pre>vzctl chkpnt VEID --dump --dumpfile <path></pre>
−
command. Third step – killing or resuming processes. If the migration succeeds VE can be stopped with the command:
+
 
 +
 
 +
<!--T:18-->
 +
Third step – killing or resuming processes. If the migration succeeds the VE can be stopped with the command:
 
<pre>vzctl chkpnt VEID --kill</pre>
 
<pre>vzctl chkpnt VEID --kill</pre>
−
If migration failed by some reason or if the goal was taking a snapshot of VE state for later restore, CPT can resume VE with:
+
If migration failed for some reason or if the goal was taking a snapshot of the VE state for later restore, CPT can resume the VE with:
 
<pre>vzctl chkpnt VEID --resume</pre>
 
<pre>vzctl chkpnt VEID --resume</pre>
  
−
Process of restoring consists of two steps. The first step is to restore processes and to leave them in a special frozen state. After this step processes are ready to continue execution, however, in some cases CPT has to do some operations after process is woken up, therefore CPT sets process return point to function in our module. This stage can be done by
+
<!--T:19-->
 +
The process of restoring consists of two steps.
 +
 
 +
<!--T:20-->
 +
The first step is to restore processes and to leave them in a special frozen state. After this step processes are ready to continue execution, however, in some cases CPT has to do some operations after a process is woken up, therefore CPT sets process return point to function in our module. This stage can be done with:
 
<pre>vzctl restore VEID --undump --dumpfile <path></pre>
 
<pre>vzctl restore VEID --undump --dumpfile <path></pre>
−
command. Second step – waking up processes or killing them if restore process failed. After CPT wakes up process, it performs necessary operations in our function and continues execution. This stages can be done by
+
 
−
<pre>vzctl restore VEID --resume
+
<!--T:21-->
−
vzctl restore VEID --kill</pre>
+
Second step – waking up processes or killing them if the restore process failed. After CPT wakes up process, it performs necessary operations in our function and continues execution. This stage can be done with:
−
commands.
+
<pre>vzctl restore VEID --resume</pre>
 +
or
 +
<pre>vzctl restore VEID --kill</pre>
 +
 
 +
== See also == <!--T:22-->
 +
 
 +
<!--T:23-->
 +
* http://criu.org/
 +
</translate>
  
 
[[Category: Technology]]
 
[[Category: Technology]]
 
[[Category: Concepts]]
 
[[Category: Concepts]]

Latest revision as of 17:34, 29 January 2018

<translate> CPT is an extension to the OpenVZ kernel which can save the full state of a running VE and to restore it later on the same or on a different host in a way transparent to running applications and network connections. This technique has several applications, the most important being live (zero-downtime) migration of VEs and taking an instant snapshot of a running VE for later resume, i.e. CheckPointing.

Before CPT, it was only possible to migrate a VE through a shutdown and subsequent reboot. The procedure not only introduces quite a long downtime for network services, it is not transparent for clients using the VE, making migration impossible when clients run some tasks which are not tolerant to shutdowns.

Compared with this old scheme, CPT allows migration of a VE in a way which is essentially invisible both for users of this VE and for external clients using network services located inside the VE. It still introduces a short delay in service, required for actual checkpoint/restore of the processes, but this delay is indistinguishable from a short interruption of network connectivity.

Yellowpin.svg Note: In OpenVZ 7, CPT is replaced by our sub-project CRIU


Online migration[edit]

There is a special utility vzmigrate in the OpenVZ distribution intended to support VE migration. With its help one can perform live (a.k.a. online) migration, i.e. during migration the VE “freezes” for some time, and after migration it continues to work as though nothing had happened. Online migration can be performed with: 

vzmigrate --online <host> VEID

During online migration all VE private data saved to an image file, which is transferred to the target host.

In order for vzmigrate to work without asking for a password, ssh public keys from the source host should be placed in the destination host's /root/.ssh/authorized_keys file. In other words, command ssh root@host should not ask you for a password. See ssh keys for more info.

Manual Checkpoint and Restore Functions[edit]

vzmigrate is not strictly required to perform online migration. The vzctl utility, accompanied with some file system backup tools, provides enough power to do all the tasks.

A VE can be checkpointed with: 

vzctl chkpnt VEID --dumpfile <path>

This command saves all the state of a running VE to the dump file and stops the VE. If the option --dumpfile is not set, vzctl uses a default path /vz/dump/Dump.VEID.

After this it is possible to restore the VE to the same state executing: 

vzctl restore VEID --dumpfile <path>

If the dump file and file system is transferred to another HW node, the same command can restore the VE there with the same success.

It is a critical requirement that file system at the moment of restore must be identical to the file system at the moment of checkpointing. If this requirement is not met, depending on the severity of changes, the process of restoration can be aborted or the processes inside a VE can see this as an external corruption of open files. When a VE is restored on the same node where it was checkpointed, it is enough to not touch the file system accessible by the VE. When a VE is transferred to another node it is necessary to synchronize the VE file system before restore. vzctl does not provide this functionality and external tools (i.e. rsync) are required.

Step-by-step Checkpoint and Restore[edit]

The process of checkpointing can be performed in stages. It consists of three steps.

First step – suspend the VE. At this stage CPT moves all the processes to a special beforehand known state and stops VE network interfaces. This stage can be done with: 

vzctl chkpnt VEID --suspend


Second step – dumping VE. At this stage CPT saves the state of processes and global state of VE to an image file. All the process private data needs to be saved: address space, register set, opened files/pipes/sockets, System V IPC structures, current working directory, signal handlers, timers, terminal settings, user identities (uid, gid, etc), process identities (pid, pgrp, sid, etc), rlimit and other data. This stage can be done with: 

vzctl chkpnt VEID --dump --dumpfile <path>


Third step – killing or resuming processes. If the migration succeeds the VE can be stopped with the command: 

vzctl chkpnt VEID --kill

If migration failed for some reason or if the goal was taking a snapshot of the VE state for later restore, CPT can resume the VE with: 

vzctl chkpnt VEID --resume

The process of restoring consists of two steps.

The first step is to restore processes and to leave them in a special frozen state. After this step processes are ready to continue execution, however, in some cases CPT has to do some operations after a process is woken up, therefore CPT sets process return point to function in our module. This stage can be done with: 

vzctl restore VEID --undump --dumpfile <path>

Second step – waking up processes or killing them if the restore process failed. After CPT wakes up process, it performs necessary operations in our function and continues execution. This stage can be done with: 

vzctl restore VEID --resume

or 

vzctl restore VEID --kill

See also[edit]

</translate>

Retrieved from "https://wiki.openvz.org/index.php?title=Checkpointing_and_live_migration&oldid=22879"