User Guide/Operations on Containers
|Warning: This User's Guide is still in development|
This chapter describes how to perform day-to-day operations on separate Containers taken in their wholeness.
|Note: We assume that you have successfully installed, configured, and deployed your OpenVZ system. In case you have not, please turn to the Installation Guide providing detailed information on all these operations.|
 Creating New Container
This section guides you through the process of creating a Container. We assume that you have successfully installed OpenVZ and prepared at least one OS template. If there are no OS templates prepared for the Container creation, turn to the Templates Management Guide first.
 Before You Begin
Before you start creating a Container, you should:
- Check that the Hardware Node is visible on your network. You should be able to connect to/from other hosts. Otherwise, your Containers will not be accessible from other servers.
- Check that you have at least one IP address per Container and the addresses belong to the same network as the Hardware Node or routing to the Containers has been set up via the Hardware Node.
To create a new Container, you have to:
- choose the new Container ID;
- choose the OS template to use for the Container;
- create the Container itself.
 Choosing Container ID
Every Container has a numeric ID, also known as Container ID, associated with it. The ID is a 32-bit integer number beginning with zero and unique for a given Hardware Node. When choosing an ID for your Container, please follow the simple guidelines below:
- ID 0 is used for the Hardware Node itself. You cannot and should not try to create a Container with ID 0.
- The OpenVZ software reserves the IDs ranging from 0 to 100. Though OpenVZ uses only ID 0, future versions might use additional Container IDs for internal needs. To facilitate upgrading, please do not create Containers with IDs below 101.
The only strict requirement for a Container ID is to be unique for a particular Hardware Node. However, if you are going to have several computers running OpenVZ, we recommend assigning different Container ID ranges to them. For example, on Hardware Node 1 you create Containers within the range of IDs from 101 to 1000; on Hardware Node 2 you use the range from 1001 to 2000, and so on. This approach makes it easier to remember on which Hardware Node a Container has been created, and eliminates the possibility of Container ID conflicts when a Container migrates from one Hardware Node to another.
Another approach to assigning Container IDs is to follow some pattern of Container IP addresses. Thus, for example, if you have a subnet with the 10.0.x.x address range, you may want to assign the 17015 ID to the Container with the 10.0.17.15 IP address, the 39108 ID to the Container with the 10.0.39.108 IP address, and so on. This makes it much easier to run a number of OpenVZ utilities eliminating the necessity to check up the Container IP address by its ID and similar tasks. You can also think of your own patterns for assigning Container IDs depending on the configuration of your network and your specific needs.
Before you decide on a new Container ID, you may want to make sure that no Container with this ID has yet been created on the Hardware Node. The easiest way to check whether the Container with the given ID exists is to issue the following command:
# vzlist -a 101 Container not found
This output shows that Container 101 does not exist on the particular Hardware Node; otherwise it would be present in the list.
 Choosing OS Template
Before starting to create a Container, you shall decide on which OS template your Container will be based. There might be several OS templates installed on the Hardware Node and prepared for the Container creation; use the following command to find out what OS templates are available on your system:
# ls /vz/template/cache/ centos-4-x86.tar.gz fedora-7-x86.tar.gz suse-10.3-x86.tar.gz centos-4-x86_64.tar.gz fedora-7-x86_64.tar.gz suse-10.3-x86_64.tar.gz centos-5-x86.tar.gz fedora-8-x86.tar.gz ubuntu-7.10-x86.tar.gz centos-5-x86_64.tar.gz fedora-8-x86_64.tar.gz ubuntu-7.10-x86_64.tar.gz debian-3.1-x86.tar.gz fedora-9-x86.tar.gz ubuntu-8.04-x86.tar.gz debian-4.0-x86.tar.gz fedora-9-x86_64.tar.gz ubuntu-8.04-x86_64.tar.gz debian-4.0-x86_64.tar.gz
| Note: You have to remove the |
 Creating Container
After the Container ID and the installed OS template have been chosen, you can create the Container private area with the
vzctl create command. The private area is the directory containing the actual files of the given Container; it is usually residing in
/vz/private/CTID/. The private area is mounted to the
/vz/root/CTID directory on the Hardware Node and provides Container users with a complete Linux file system tree.
vzctl create command requires only the Container ID and the name of the OS template as arguments; however, in order to avoid setting all the Container resource control parameters after creating the private area, you can specify a sample configuration to be used for your new Container. The sample configuration files are residing in the
/etc/vz/conf directory and have names with the following mask:
ve-configname.conf-sample. The most commonly used sample is the
ve-basic.conf-sample file; this sample file has resource control parameters suitable for most Containers.
Thus, for example, you can create a new Container by typing the following string:
# vzctl create 101 --ostemplate centos-5-x86 -–config basic Creating container private area (centos-5-x86) Performing postcreate actions Container private area was created
In this case, the OpenVZ software will create a Container with ID 101, the private area based on the
centos-5-x86 OS template, and configuration parameters taken from the
ve-basic.conf-sample sample configuration file.
If you specify neither an OS template nor a sample configuration,
vzctl will try to take the corresponding values from the global OpenVZ configuration file (
/etc/vz/vz.conf). So you can set the default values in this file using your favorite text file editor, for example:
and do without specifying these parameters each time you create a new Container.
Now you can create a Container with ID 101 with the following command:
# vzctl create 101 Creating container private area (centos-5-x86) Performing postcreate actions Container private area was created
In principle, now you are ready to start your newly created Container. However, typically you need to set its network IP address, hostname, DNS server address and root password before starting the Container for the first time.
 Configuring Container
Configuring a Container consists of several tasks:
- Setting Container startup parameters;
- Setting Container network parameters;
- Setting Container user passwords;
- Configuring Quality of Service (Service Level) parameters.
For all these tasks, the
vzctl set command is used. Using this command for setting Container startup parameters, network parameters, and user passwords is explained later in this subsection. Service Level Management configuration topics are dwelled upon in the Managing Resources chapter.
 Setting Startup Parameters
vzctl set command allows you to define the
onboot Container startup parameter. Setting this parameter to
yes makes your Container automatically boot at the Hardware Node startup. For example, to enable Container 101 to automatically start on your Hardware Node boot, you can execute the following command:
# vzctl set 101 --onboot yes --save Saved parameters for CT 101
onboot parameter will have effect only on the next Hardware Node startup.
 Setting Network Parameters
In order to be accessible from the network, a Container shall be assigned a correct IP address and hostname; DNS servers shall also be configured. In addition, the SSH daemon shall be running inside the Container. The session below illustrates setting the Container 101 network parameters:
# vzctl set 101 --hostname server101.mydomain.com --save Set hostname: server101.mydomain.com Saved parameters for CT 1010101 # vzctl set 101 --ipadd 10.0.186.1 --save Adding IP address(es): 10.0.186.1 Saved parameters for CT 1010101 # vzctl set 101 --nameserver 192.168.1.165 --save File resolv.conf was modified Saved parameters for CT 1010101
These commands will assign Container 101 the IP address of
10.0.186.1, the hostname of
server101.mydomain.com, and set the DNS server address to
–-save flag instructs vzctl to also save all the parameters set to the Container configuration file.
You can issue the above commands when the Container is running. In this case, if you do not want the applied values to persist, you can omit the
–-save option and the applied values will be valid only until the Container shutdown.
To check whether SSH is running inside the Container, use
vzctl exec, which allows executing any commands in the Container context.
# vzctl start 101 [This command starts Container 101, if it is not started yet] # vzctl exec 101 service sshd status sshd is stopped # vzctl exec 101 service sshd start Starting sshd: [ OK ] # vzctl exec 101 service sshd status sshd (pid 3801) is running...
The above example assumes that Container 101 is created on the CentOS 5 template. For other OS templates, please consult the corresponding OS documentation.
For more information on running commands inside a Container from the Hardware Node, see the #Running Commands in Container subsection.
 Setting root Password for Container
Setting the root user password is necessary for connecting to a Container via SSH. By default, the root account is locked in a newly created Container, and you cannot log in. In order to log in to the Container, it is necessary to create a user account inside the Container and set a password for this account, or unlock the root account. The easiest way of doing it is to run:
# vzctl set 101 --userpasswd root:test
In this example, we set the root password for Container 101 to "test", and you can log in to the Container via SSH as root and administer it in the same way as you administer a standalone Linux server: install additional software, add users, set up services, and so on. The password will be set inside the Container in the
/etc/shadow file in an encrypted form and will not be stored in the Container configuration file. Therefore, if you forget the password, you have to reset it. Note that
--userpasswd ignores the
--save switch, the password is anyway persistently set for the given Container.
While you can create users and set passwords for them using the
vzctl exec or
vzctl set commands, it is suggested that you delegate user management to the Container administrator advising him/her of the Container root account password.
 Starting, Stopping, Restarting, and Querying Status of Container
When a Container is created, it may be started up and shut down like an ordinary server. To start Container 101, use the following command:
# vzctl start 101 Starting container ... Container is mounted Adding IP address(es): 10.0.186.1 Setting CPU units: 1000 Configure meminfo: 65536 Set hostname: server101.mydomain.com File resolv.conf was modified Container start in progress...
To check the status of a Container, use the
vzctl status command:
# vzctl status 101 CTID 101 exist mounted running
Its output shows the following information:
- Whether the Container private area exists;
- Whether this private area is mounted;
- Whether the Container is running.
In our case,
vzctl reports that Container 101 exists, its private area is mounted, and the Container is running. Alternatively, you can make use of the
# vzlist 101 CTID NPROC STATUS IP_ADDR HOSTNAME 101 10 running 10.0.186.1 server101.mydomain.com
Still another way of getting the Container status is checking the
/proc/vz/veinfo file. This file lists all the Containers currently running on the Hardware Node. Each line presents a running Container in the <CT_ID> <reserved> <number_of_processes> <IP_address> ... format:
# cat /proc/vz/veinfo 101 0 10 10.0.186.1 0 0 79
This output shows that Container 101 is running, there are 10 running processes inside the Container, and its IP address is 10.0.186.1. The second line corresponds to the Container with ID 0, which is the Hardware Node itself.
The following command is used to stop a Container:
# vzctl stop 101 Stopping container ... Container was stopped Container is unmounted # vzctl status 101 CTID 101 exist unmounted down
vzctl has a two-minute timeout for the Container shutdown scripts to be executed. If the Container is not stopped in two minutes, the system forcibly kills all the processes in the Container. The Container will be stopped in any case, even if it is seriously damaged. To avoid waiting for two minutes in case of a Container that is known to be corrupt, you may use the
# vzctl stop 101 --fast Stopping container ... Container was stopped Container is unmounted
Make sure that you do not use the
--fast switch with healthy Containers, unless necessary, as the forcible killing of Container processes may be potentially dangerous.
vzctl start and
vzctl stop commands initiate the normal Linux OS startup or shutdown sequences inside the Container. In case of a Red Hat-like distribution, System V initialization scripts will be executed just like on an ordinary server. You can customize startup scripts inside the Container as needed.
To restart a Container, you may as well use the
vzctl restart command:
# vzctl restart 101 Restarting container Stopping container ... Container was stopped Container is unmounted Starting container ... Container is mounted Adding IP address(es): 10.0.186.1 Setting CPU units: 1000 Configure meminfo: 65536 Set hostname: server101.mydomain.com File resolv.conf was modified Container start in progress...
|Note: You can also use Container names to start, stop, and restart the corresponding Containers. For detailed information on Container names, please turn to the #Setting Name for Container section.|
 Listing Containers
Very often you may want to get an overview of the Containers existing on the given Hardware Node and to get additional information about them — their IP addresses, hostnames, current resource consumption, etc. In the most general case, you may get a list of all Containers by issuing the following command:
# vzlist -a CTID NPROC STATUS IP_ADDR HOSTNAME 101 10 running 10.101.66.101 server101.mydomain.com 102 - stopped 10.101.66.102 server102.mydomain.com 103 5 running 10.101.66.103 server103.mydomain.com
-a switch tells the
vzlist utility to output both running and stopped Containers. By default, only running Containers are shown. The default columns inform you of the Container IDs, the number of running processes inside Containers, their status, IP addresses, and hostnames. This output may be customized as desired by using
vzlist command line switches. For example:
# vzlist -o ctid,diskinodes.s -s diskinodes.s CTID DQINODES.S 101 400000 103 200000
This shows only running Containers with the information about their IDs and soft limit on disk inodes (see the Managing Resources chapter for more information), with the list sorted by this soft limit. The full list of the
vzlist command line switches and output and sorting options is available in the vzlist(8) man page.
 Setting Name for Container
You can assign an arbitrary name to your Container and use it, along with the Container ID, to refer to the Container while performing this or that Container-related operation on the Hardware Node. For example, you can start or stop a Container by specifying the Container name instead of its ID.
You can assign names to your Containers using the
--name option of the
vzctl set command. For example, to set the
computer1 name for Container 101, you should execute the following command:
# vzctl set 101 --name computer1 --save Name computer1 assigned Saved parameters for Container 101
You can also set a name for Container 101 by editing its configuration file. In this case you should proceed as follows:
1. Open the configuration file of Container 101 (
/etc/vz/conf/101.conf) for editing and add the following string to the file:
2. In the
/etc/vz/names directory on the Hardware Node, create a symbolic link with the name of
computer1 pointing to the Container configuration file. For example:
# ln --symbolic /etc/vz/conf/101.conf /etc/vz/names/computer1
When specifying names for Containers, please keep in mind the following:
- Names may contain the following symbols:
9, underscores (
_), dashes (
-), spaces, the symbols from the ASCII character table with their code in the 128–255 range, and all the national alphabets included in the Unicode code space.
- Container names cannot consist of digits only; otherwise, there would be no way to distinguish them from Container IDs.
- If it contains one or more spaces, the Container name should be put in single or double quotes, or the spaces have to be escaped by preceding them with a backslashes (
After the name has been successfully assigned to Container 101, you can start using it instead of ID 101 to perform Container-related operations on the Node. For example:
- You can stop Container 101 with the following command:
# vzctl stop computer1 Stopping container ... Container was stopped Container is unmounted
- You can start Container 101 anew by issuing the following command:
# vzctl start computer1 Starting container ... ...
You can find out what name is assigned to Container 101 in one of the following ways:
- Using the
# vzlist -o name 101 NAME computer1
- Checking the
NAMEparameter in the Container configuration file (
/etc/vz/conf/101.conf). For example:
# grep NAME= /etc/vz/conf/101.conf NAME="computer1"
- Checking which symlink in the
/etc/vz/names/directory links to Container configuration file. The file name of the symlink is the name for Container. For example:
# ls -l /etc/vz/names/ | grep /101.conf lrwxrwxrwx 1 root root 21 Jan 16 20:18 computer1 -> /etc/vz/conf/101.conf
 Storing Extended Information on Container
|Note: This feature is available since vzctl-3.0.23.|
Sometimes, it may be difficult to remember the information on certain Containers. The probability of this increases together with the number of Containers and with the time elapsed since the Container creation. The OpenVZ software allows you to set the description of any Container on the Hardware Node and view it later on, if required. The description can be any text containing any Container-related information; for example, you can include the following in the Container description:
- the owner of the Container;
- the purpose of the Container;
- the summary description of the Container;
Let us assume that you are asked to create a Container for a Mr. Johnson who is going to use it for hosting the MySQL server. So, you create Container 101 and, after that, execute the following command on the Hardware Node:
# vzctl set 101 --description "Container 101 > owner - Mr. Johnson > purpose - hosting the MySQL server" --save Saved parameters for CT 101
This command saves the following information related to the Container: its ID, owner, and the purpose of its creation. At any time, you can display this information by issuing the following command:
# vzlist -o description 101 DESCRIPTION Container 101 owner - Mr. Johnson purpose - hosting the MySQL server
You can also view the Container description by checking the
DESCRIPTION parameter of the Container configuration file (
/etc/vz/conf/101.conf). However, the data stored in this file are more suitable for parsing by the
vzlist command rather than for viewing by a human since all symbols in the
DESCRIPTION field except the alphanumerical ones ('a-z', 'A-Z', and '0-9'), underscores ('_'), and dots ('.') are transformed to the corresponding hex character code.
While working with Container descriptions, please keep in mind the following:
- You can use any symbols you like in the Container description (new lines, dashes, underscores, spaces, etc.).
- If the Container description contains one or more spaces or line breaks (as in the example above), it should be put in single or double quotes.
- As distinct from a Container name, a Container description cannot be used for performing Container-related operations (e.g. for starting or stopping a Container) and is meant for reference purposes only.
 Migrating Container
The OpenVZ Hardware Node is the system with higher availability requirements in comparison with a typical Linux system. If you are running your company mail server, file server, and web server in different Containers on one and the same Hardware Node, then shutting it down for hardware upgrade will make all these services unavailable at once. To facilitate hardware upgrades and load balancing between several Hardware Nodes, the OpenVZ software provides you with the ability to migrate Containers from one physical box to another.
Migrating Containers is possible if OpenVZ is installed on two or more Hardware Nodes, so you are able to move a Container to another Node. Migration may be necessary if a Hardware Node is undergoing a planned maintenance or in certain other cases.
 Standard (offline) migration
The standard migration procedure allows you to move both stopped and running Containers. Migrating a stopped Container includes copying all Container private files from one Node to another and does not differ from copying a number of files from one server to another over the network. In its turn, the migration procedure of a running Container is a bit more complicated and may be described as follows:
- After initiating the migration process, all Container private data are copied to the Destination Node. During this time, the Container on the Source Node continues running.
- The Container on the Source Node is stopped.
- The Container private data copied to the Destination Node are compared with those on the Source Node and, if any files were changed during the first migration step, they are copied to the Destination Node again and rewrite the outdated versions.
- The Container on the Destination Node is started.
WARNING: By default, after the migration process is completed, the Container private area and configuration file are deleted on the Source Node! However, if you wish the Container private area on the Source Node to not be removed after the successful Container migration, you can override the default
vzmigrate behavior by using the
–r no switch.
There is a short downtime needed to stop the Container on the Source Node, copy the Container private data changes to the Destination Node, and start the Container on the Destination Node. However, this time is very short and does not usually exceed one minute.
The following session moves Container 101 from the current Hardware Node to a new one named ts7.mydomain.com:
# vzmigrate ts7.mydomain.com 101 Starting migration of container 101 on ts7.mydomain.com Preparing remote node Initializing remote quota Syncing private Syncing 2nd level quota Turning quota off Cleanup
|Note: For the command to be successful, a direct SSH connection (on port 22) should be allowed between the Source and Destination Nodes.|
 Zero-downtime (online) migration
OpenVZ allows you to migrate your Containers from one Hardware Node to another with zero downtime. The zero downtime migration technology has the following main advantages as compared with the standard one:
- The process of migrating a Container to another Node is transparent for you and the Container applications and network connections, i.e., on the Source and Destination Nodes, no modifications of system characteristics and operational procedures inside the Container are performed.
- The Container migration time is greatly reduced. In fact, the migration eliminates the service outage or interruption for Container end users.
- The Container is restored on the Destination Node in the same state as it was at the beginning of the migration.
- You can move the Containers running a number of applications which you do not want to be rebooted during the migration for some reason or another.
| Note: Zero-downtime migration cannot be performed on Containers having one or several opened sessions established with the |
Before performing zero-downtime migration, it is recommended to synchronize the system time on the Source and Destination Nodes, e.g. by means of NTP (http://www.ntp.org). The reason for this recommendation is that some processes running in the Container might rely on the system time being monotonic and thus might behave unpredictably if they see an abrupt step forward or backward in the time once they find themselves on the new Node with different system clock parameters.
To migrate a Container by using the zero downtime migration technology, you should pass the
--online option to the
vzmigrate utility. In this case a Container is 'dumped' at the beginning of the migration, i.e. all Container private data including the state of all running processes are saved to an image file. This image file is then transferred to the Destination Node where it is 'undumped'.
For example, you can migrate Container 101 from the current Hardware Node to the Destination Node named my_node.com by executing the following command:
# vzmigrate --online my_node.com 101
 Deleting Container
You can delete a Container that is not needed anymore with the
vzctl destroy CTID command. This command removes the Container private area completely and renames the Container configuration file and action scripts by appending the
.destroyed suffix to them.
A running Container cannot be destroyed with the
vzctl destroy command. The example below illustrates destroying Container 101:
# vzctl destroy 101 Container is currently running. Stop it first. # vzctl stop 101 Stopping container ... Container was stopped Container is unmounted # vzctl destroy 101 Destroying container private area: /vz/private/101 Container private area was destroyed # ls /etc/vz/conf/101.* /etc/vz/conf/101.conf.destroyed # vzctl status 101 CTID 102 deleted unmounted down
If you do not need the backup copy of the Container configuration files (with the .destroyed suffix), you may delete them manually.
 Disabling Container
There may appear situations when you wish to forbid Container owners to use their Containers. For example, it may happen in case the Container owner uses it for unallowed purposes: intruding into computers of other users, participating in DoS attacks, etc.
In such cases, the OpenVZ software allows you to disable a Container, thus, making it impossible to start the Container once it was stopped. For example, you can execute the following command to disable Container 101 residing on your Hardware Node:
# vzctl set 101 --disabled yes --save
| Note: This option makes no sense without the |
After the Container stopping, the Container user will not be able to start it again until you enable this Container again by passing the
--disabled no option to
vzctl set. You can also use the
--force option to start any disabled Container. For example:
# vzctl start 103 Container start disabled # vzctl start 103 --force Starting container ... Container is mounted Adding IP address(es): 192.168.16.3 Setting CPU units: 1000 Configure meminfo: 65536 Container start in progress...
 Suspending Container
OpenVZ allows you to suspend any running Container on the Hardware Node by saving its current state to a special dump file. Later on, you can resume the Container and get it in the same state the Container was at the time of its suspending.
In OpenVZ-based systems, you can use the
vzctl chkpnt command to save the current state of a Container. For example, you can issue the following command to suspend Container 101:
# vzctl chkpnt 101 Setting up checkpoint... suspend... dump... kill... Container is unmounted Checkpointing completed succesfully
During the command execution, the
/vz/dump/Dump.101 file containing the entire state of Container 101 is created and the Container itself is stopped.
Note: You can set another directory to store dump files for your Containers by changing the value of the DUMPDIR parameter in the OpenVZ global file. Detailed information on the OpenVZ global file and the parameters you can specify in it is provided in the vz(5).
At any time, you can resume Container 101 by executing the following command:
# vzctl restore 101 Restoring container ... Starting container ... Container is mounted undump... Adding IP address(es): 192.168.16.3 Setting CPU units: 1000 Configure meminfo: 65536 resume... Container start in progress... Restoring completed succesfully
The Container state is restored from the
/vz/dump/Dump.101 file on the Node. Upon the restoration completion, any applications that were running inside Container 101 at the time of its suspending will be running and the information content will be the same as it was when the Container was suspended.
While working with dump files, please keep in mind the following:
- You can restore the Container dump file on the Source Node, i.e. on the Node where this Container was running before its dumping, or transfer the dump file to another Node and restore it there.
|Note: Before restoring a Container from its dump file, please make sure that the file system on the Destination Node is identical to that at the moment of the Container dumping; otherwise, the Container restoration may fail.|
- You cannot change settings of the suspended Container.
 Running Commands in Container
Usually, a Container administrator logs in to the Container via network and executes any commands in the Container as on any other Linux box. However, you might need to execute commands inside Containers bypassing the normal login sequence. This can be helpful if:
- You do not know the Container login information, and you need to run some diagnosis commands inside the Container in order to verify that it is operational.
- Network access is absent for a Container. For example, the Container administrator might have accidentally applied incorrect firewalling rules or stopped the SSH daemon.
The OpenVZ software allows you to execute commands in a Container in these cases. Use the
vzctl exec command for running a command inside the Container with the given ID. The session below illustrates the situation when the SSH daemon is not started:
# vzctl exec 103 /etc/init.d/sshd status openssh-daemon is stopped # vzctl exec 103 /etc/init.d/sshd start Starting sshd: [ OK ] # vzctl exec 103 /etc/init.d/sshd status openssh-daemon (pid 9899) is running...
Now Container users can log in to the Container via SSH (assuming that networking and firewall are not misconfigured).
When executing commands inside a Container from shell scripts, use the
vzctl exec2 command. It has the same syntax as
vzctl exec but returns the exit code of the command being executed instead of the exit code of
vzctl itself. You can check the exit code to find out whether the command has completed successfully.
If you wish to execute a command in all running Containers, you can use the following script:
# for CT in $(vzlist -H -o ctid); do echo "== CT $CT =="; vzctl exec $CT command; done
command is the command to be executed in all the running Containers. For example:
# for CT in $(vzlist -H -o ctid); do echo "== CT $CT =="; vzctl exec $CT uptime; done == CT 103 == 15:17:19 up 13 min, 0 users, load average: 0.00, 0.00, 0.00 == CT 123123123 == 15:17:19 up 22:00, 0 users, load average: 0.00, 0.00, 0.00