Difference between revisions of "UBC systemwide configuration"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(added <references/>)
(link to vzmemcheck)
 
(2 intermediate revisions by 2 users not shown)
Line 17: Line 17:
 
The best way to make sure that the configuration of the whole system
 
The best way to make sure that the configuration of the whole system
 
is valid is to run periodic automatic checks, based on the formulae described
 
is valid is to run periodic automatic checks, based on the formulae described
below.
+
below. {{Man|vzmemcheck|8}} utility can be helpful in calculations.
  
 
== Resource utilization and commitment level ==
 
== Resource utilization and commitment level ==
Line 27: Line 27:
 
containers at the given time.
 
containers at the given time.
 
In general, low utilization values mean that the system is under-utilized.
 
In general, low utilization values mean that the system is under-utilized.
Often, it means that the system is capable of supporting more Virtual
+
Often, it means that the system is capable of supporting more containers
Environment if the existing containers continue to maintain
+
if the existing containers continue to maintain
 
the same load and resource consumption level.
 
the same load and resource consumption level.
 
High utilization values (in general, more than <code>1</code>)
 
High utilization values (in general, more than <code>1</code>)
Line 42: Line 42:
 
overcommitted.
 
overcommitted.
 
If the system runs a lot of containers,
 
If the system runs a lot of containers,
it is usually acceptable to have some overcommitment, because it is unlikely that all Virtual
+
it is usually acceptable to have some overcommitment, because it is unlikely that all containers will request resources at the same time.
Environments will request resources at the same time.
 
 
However, higher commitment levels (as discussed below for each resource
 
However, higher commitment levels (as discussed below for each resource
 
individually) will cause containerss to experience failures to
 
individually) will cause containerss to experience failures to
 
allocate and use the resources promised to them.
 
allocate and use the resources promised to them.
  
== “Low memory” (x86 specific) ==
+
== “Low memory” (x86_32 specific) ==
 
Because of specifics of architecture of Intel's x86 processors, the RAM of the
 
Because of specifics of architecture of Intel's x86 processors, the RAM of the
 
computer can't be used uniformly.
 
computer can't be used uniformly.
Line 197: Line 196:
 
The normal commitment level is about <math>0.8</math>—<math>1</math>.
 
The normal commitment level is about <math>0.8</math>—<math>1</math>.
  
Commitment levels more than <math>1</math> means that the Virtual Environmens are
+
Commitment levels more than <math>1</math> means that the containers are
 
guaranteed more memory than the system has.
 
guaranteed more memory than the system has.
 
Such overcommitment is strongly not recommended, because in that case
 
Such overcommitment is strongly not recommended, because in that case
Line 205: Line 204:
  
 
It is better to guarantee containers less and have less commitment
 
It is better to guarantee containers less and have less commitment
levels than to accidently overcommit the system by memory plus swap.
+
levels than to accidentally overcommit the system by memory plus swap.
 
If the system has spare memory and swap, containers will
 
If the system has spare memory and swap, containers will
 
transparently be able to use the memory and swap above their guarantees.
 
transparently be able to use the memory and swap above their guarantees.

Latest revision as of 12:11, 6 June 2011

User Beancounters
Definition
/proc/user_beancounters
/proc/bc/
General information
Units of measurement
VSwap
Parameters description
Primary parameters
numproc, numtcpsock, numothersock, vmguarpages
Secondary parameters
kmemsize, tcpsndbuf, tcprcvbuf, othersockbuf, dgramrcvbuf, oomguarpages, privvmpages
Auxiliary parameters
lockedpages, shmpages, physpages, numfile, numflock, numpty, numsiginfo, dcachesize, numiptent, swappages
Internals
User pages accounting
RSS fractions accounting
On-demand accounting
UBC consistency
Consistency formulae
System-wide configuration
vzubc(8)
Configuration examples
Basic
Derived
Intermediate configurations
Tables
List of parameters
Parameter properties
Consistency
Config examples

The UBC consistency check article discussed validation of UBC configuration for a single container. This article discusses checks that the UBC configuration of the whole system is valid.

Configurations where resources allowed for containers exceed system capacity[1] are not valid and dangerous from stability point of view. They may result in abnormal termination of the applications, bad responsiveness of the system and, sometimes, system hangs. Whereas the configuration validation discussed in UBC consistency check addressed application functionality, the validation considered in this section is aimed at security and stability of the whole system.

The best way to make sure that the configuration of the whole system is valid is to run periodic automatic checks, based on the formulae described below. vzmemcheck(8) utility can be helpful in calculations.

Resource utilization and commitment level[edit]

Several resources of the whole system (such as RAM) are discussed below in terms of utilization and commitment level.

Utilization shows the amount of resources consumed by all containers at the given time. In general, low utilization values mean that the system is under-utilized. Often, it means that the system is capable of supporting more containers if the existing containers continue to maintain the same load and resource consumption level. High utilization values (in general, more than 1) mean the the system is overloaded and the service level of the containers is degraded.

Commitment level shows how much resources are “promised” to the existing containers. Low commitment levels mean that the system is capable of supporting more containers. Commitment levels more than 1 mean that the containers are promised more resources than the system has, and in this case the system is said to be overcommitted. If the system runs a lot of containers, it is usually acceptable to have some overcommitment, because it is unlikely that all containers will request resources at the same time. However, higher commitment levels (as discussed below for each resource individually) will cause containerss to experience failures to allocate and use the resources promised to them.

“Low memory” (x86_32 specific)[edit]

Because of specifics of architecture of Intel's x86 processors, the RAM of the computer can't be used uniformly. The most important memory area is so called “low memory”, a part of memory residing at lower addresses and directly accessible by the kernel. For current Linux kernels, the size of low memory area is 832MB (or, if the computer has less RAM than 832MB, the size of the RAM).

Note that 64-bit kernels (i.e. x86_64, ia64 etc.) can access all memory directly and are not using “low memory” area.

Utilization[edit]

The lower bound estimation of low memory utilization is

where

Utilization of low memory below is normal. Utilization above is not safe, and utilization above is dangerous is very likely to cause bad system responsiveness, application stalls for seconds or more and termination of some applications.

Commitment level[edit]

The commitment level can be computed as

Commitment levels below are normal. Levels between and are usually acceptable for systems with about 100 containers. Systems with more containers may have the commitment level increased, to about for 400 containers. Higher commitment levels for this resource are not recommended, because the consequences of exceeding the low memory capacities are severe and affect the whole system and all the containers.

Total RAM[edit]

This subsection discusses usage of the whole RAM and its utilization. Usage of swap space and the sum of used RAM and swap space are discussed below in subsection Memory and swap space.

Current version of OpenVZ can't guarantee availability of certain amount of memory (in opposite to the sum of memory and swap space), so the commitment level is not applicable to the total RAM. Such guarantees will be implemented in future versions.

Utilization[edit]

The amount of RAM consumed by all containers can be computed as

The difference between memory usage shown by free(1) or /proc/meminfo and the total amount of RAM consumed by containers is the memory used by system daemons and different caches.

The memory utilization can be computed as

Utilization levels from to are normal. Lower utilization means that the system is under-utilized, and, if other system resources and their commitment levels permit, the system can host more containers. By the nature of the accounting of physpages and other parameters, total RAM utilization can't be bigger than .

Memory and swap space[edit]

The main resource of the computer determining the amount of memory applications can use is the sum of RAM and swap sizes. If the total size of the used memory exceeds the RAM size, Linux kernel moves some data to swap and and loads it back when the application needs it. More frequently used data tends to stay in RAM, less frequently used data spends more time in swap.

Swap-in and swap-out activity reduces the system performance to some extent. However, if this activity is not excessive, the performance decrease is not very noticeable. On the other hand, the benefits of using swap space are quite big, allowing to increase the number of containers in the system about 2 times.

Swap space is essential for handling system load bursts. System with enough swap just slows down at high load bursts, whereas the system without swap reacts to high load bursts by refusing memory allocations (causing applications to refuse to accept clients or terminate) and by direct killing of some applications. Additionally, the presence of swap space helps the system to better balance memory and move data between “low memory” and the rest of the RAM.

In all OpenVZ installations it is strongly recommended to have swap of size not less than the RAM size.

Also, it is not recommended to create swap space of the size of more than 4 times RAM size because of performance degradation related to swap-in and swap-out activity. That is, the system should be configured so that The optimal configuration is when swap size is twice more than the RAM size.

Utilization[edit]

The normal utilization of memory plus swap ranges between and

Lower utilization means that the system memory is under-utilized at the moment of checking the utilization. Higher utilization is likely to cause gradual performance degradation because of swap-in and swap-out activity and is a sign of overloading of the system.

Commitment level[edit]

The normal commitment level is about .

Commitment levels more than means that the containers are guaranteed more memory than the system has. Such overcommitment is strongly not recommended, because in that case if all the memory is consumed, random applications, including the ones belonging to the host system, may be killed and the system may become inaccessible by ssh(1) and lose other important functionality.

It is better to guarantee containers less and have less commitment levels than to accidentally overcommit the system by memory plus swap. If the system has spare memory and swap, containers will transparently be able to use the memory and swap above their guarantees. Guarantees given to containers should not be big, and it is normal if memory and swap usage for some containers stays above their guarantee. It is also normal to give guarantees only to containers with preferred service. But administrators should not guarantee container more than the system actually has.

Allocated memory[edit]

This subsection considers standard memory allocations made by applications in the container. The allocations for each container are controlled by two parameters: vmguarpages and privvmpages, discussed in sections FIXME.

Allocated memory is a more “virtual” system resource than the RAM or RAM plus swap size. Applications can allocate memory but start to use it only later, and the amount of system's free memory will decrease only at the moment of the use. The sum of allocated memory size of all containers is an estimation of how much physical memory will be used when (and if) all applications claim the allocated memory.

Utilization[edit]

This utilization level is the ratio of the amount of allocated memory to the capacity of the system.

Low utilization level means that the system can support more containers, if other resources permit. High utilization levels may, but doesn't necessarily mean that the system is overloaded. As it was explaned above, not all applications use all the allocated memory, so this utilization level may exceed .

Computing this utilization level is useful for comparing it with the commitment level and the level of memory allocation restrictions, discussed below, to configure memory allocation restrictions for the container.

Commitment level[edit]

Allocation guarantee commitment level

is the ratio of the memory space guaranteed to be available for allocations to the capacity of the system. Similarly to the commitment level of memory plus swap space (as discussed in subsection Memory and swap space), this level should be kept below . If the level is above , it significantly increases the chances of applications to be killed instead of be notified in case the system experiences a memory shortage.

It's better to provide lower guarantees than accidently guarantee more than the system has, because containers are allowed to allocated memory above their guarantee if the system is not tight on memory. It is also normal to give guarantees only to containers with preferred service.

Limiting memory allocations[edit]

In addition to providing allocation guarantees, it is possible to impose restrictions on the amount of memory allocated by containers.

If a system has multiple containers, it is important to make sure that for each container

If this condition is not satisfied, a single container may easily cause an excessive swap-out and very bad performance of the whole system. Usually, for each container privvmpages limitations are to values much less than the size of the RAM.

The resource control parameters should be configured in a way, so that in case of memory shortage applications are given chance to notice the shortage and exit gracefully, instead of being terminated by the kernel. For this purpose, it is recommended to maintain reasonable total level of memory allocation restrictions, computed as

This number shows how much memory applications are allowed to allocate in comparison with the capacity of the system.

In practice, a lot of applications do not use the memory very efficiently and, sometimes, allocated memory will never be used later. For example, Apache Web server at start time it allocates about 20–30%% more memory that it will ever use. Some multi-threaded applications are especially bad at using their memory, and their rate of allocated to used memory may happen to be 1000%.

The bigger the level of memory allocation restrictions, the more chances are that applications will be killed instead of getting an error on next memory allocation in case the system experiences memory shortage. The levels ranging in can be considered acceptable. Administrators can find experimentally the optimal setting for their load, basing on the frequency of messages “Out of Memory: killing process” in system logs, saved by klogd(8) and syslogd(8). However, for stability-critical applications, it's better to keep the level not exceeding .

  1. More precisely, configurations with excessive overcommitment, as explained below.