Open main menu

OpenVZ Virtuozzo Containers Wiki β

Editing UBC systemwide configuration

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision Your text
Line 1: Line 1:
 
{{UBC toc}}
 
{{UBC toc}}
  
The [[UBC consistency check]] article discussed validation of [[UBC]] configuration for a single container. This article discusses checks that the UBC configuration of the
+
The [[UBC consistency check]] article discussed validation of a resource control configuration
 +
for a single Virtual Environment. This article discusses checks that the configuration of the
 
'''whole''' system is '''valid'''.
 
'''whole''' system is '''valid'''.
  
Configurations where resources allowed for containers exceed system
+
Configurations where resources allowed for Virtual Environments exceed system
 
capacity<ref>More precisely, configurations with excessive
 
capacity<ref>More precisely, configurations with excessive
 
overcommitment, as explained below.</ref> are not valid
 
overcommitment, as explained below.</ref> are not valid
Line 17: Line 18:
 
The best way to make sure that the configuration of the whole system
 
The best way to make sure that the configuration of the whole system
 
is valid is to run periodic automatic checks, based on the formulae described
 
is valid is to run periodic automatic checks, based on the formulae described
below. {{Man|vzmemcheck|8}} utility can be helpful in calculations.
+
below.
  
 
== Resource utilization and commitment level ==
 
== Resource utilization and commitment level ==
Line 25: Line 26:
  
 
'''Utilization''' shows the amount of resources consumed by all
 
'''Utilization''' shows the amount of resources consumed by all
containers at the given time.
+
Virtual Environments at the given time.
 
In general, low utilization values mean that the system is under-utilized.
 
In general, low utilization values mean that the system is under-utilized.
Often, it means that the system is capable of supporting more containers
+
Often, it means that the system is capable of supporting more Virtual
if the existing containers continue to maintain
+
Environment if the existing VEs continue to maintain
 
the same load and resource consumption level.
 
the same load and resource consumption level.
 
High utilization values (in general, more than <code>1</code>)
 
High utilization values (in general, more than <code>1</code>)
 
mean the the system is overloaded and the service level
 
mean the the system is overloaded and the service level
of the containers is degraded.
+
of the Virtual Environments is degraded.
  
 
'''Commitment level''' shows how much resources are “promised” to the existing
 
'''Commitment level''' shows how much resources are “promised” to the existing
containers. Low commitment levels mean that the system
+
Virtual Environments. Low commitment levels mean that the system
is capable of supporting more containers.
+
is capable of supporting more Virtual Environments.
 
Commitment levels more than <code>1</code> mean that the
 
Commitment levels more than <code>1</code> mean that the
containers are promised more resources
+
Virtual Environments are promised more resources
 
than the system has, and in this case the system is said to be
 
than the system has, and in this case the system is said to be
 
overcommitted.
 
overcommitted.
If the system runs a lot of containers,
+
If the system runs a lot of Virtual Environments,
it is usually acceptable to have some overcommitment, because it is unlikely that all containers will request resources at the same time.
+
it is usually acceptable to have some overcommitment, because it is unlikely that all Virtual
 +
Environments will request resources at the same time.
 
However, higher commitment levels (as discussed below for each resource
 
However, higher commitment levels (as discussed below for each resource
individually) will cause containerss to experience failures to
+
individually) will cause Virtual Environmentss to experience failures to
 
allocate and use the resources promised to them.
 
allocate and use the resources promised to them.
  
== “Low memory” (x86_32 specific) ==
+
== “Low memory” ==
Because of specifics of architecture of Intel's x86 processors, the RAM of the
+
Because of specifics of architecture of Intel's processors, the RAM of the
 
computer can't be used uniformly.
 
computer can't be used uniformly.
 
The most important memory area is so called “low memory”, a part of memory
 
The most important memory area is so called “low memory”, a part of memory
Line 55: Line 57:
 
(or, if the computer has less RAM than 832MB, the size of the RAM).
 
(or, if the computer has less RAM than 832MB, the size of the RAM).
  
Note that 64-bit kernels (i.e. x86_64, ia64 etc.) can access all memory
 
directly and are not using “low memory” area.
 
 
=== Utilization ===
 
=== Utilization ===
 
The lower bound estimation of low memory utilization is
 
The lower bound estimation of low memory utilization is
  
 
<math>
 
<math>
\frac{\displaystyle\sum_{all\ containers}
+
\frac{\displaystyle\sum_{all\ VE}
 
         (kmemsize_{cur}+allsocketbuf_{cur})}
 
         (kmemsize_{cur}+allsocketbuf_{cur})}
 
     {0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm,
 
     {0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm,
Line 81: Line 81:
  
 
<math>
 
<math>
\frac{\displaystyle\sum_{all\ containers}
+
\frac{\displaystyle\sum_{all\ VE}
 
         (kmemsize_{lim}+allsocketbuf_{lim})}
 
         (kmemsize_{lim}+allsocketbuf_{lim})}
 
     {0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm.
 
     {0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm.
Line 88: Line 88:
 
Commitment levels below <math>1</math> are normal.
 
Commitment levels below <math>1</math> are normal.
 
Levels between <math>1</math> and <math>1.2</math> are usually acceptable for systems with about
 
Levels between <math>1</math> and <math>1.2</math> are usually acceptable for systems with about
100 containers.
+
100 Virtual Environments.
Systems with more containers may have the commitment level
+
Systems with more Virtual Environments may have the commitment level
increased, to about <math>1.5</math>—<math>2</math> for 400 containers.
+
increased, to about <math>1.5</math>—<math>2</math> for 400 VEs.
 
Higher commitment levels for this resource are not recommended,
 
Higher commitment levels for this resource are not recommended,
 
because the consequences of exceeding the low memory capacities
 
because the consequences of exceeding the low memory capacities
are severe and affect the whole system and all the containers.
+
are severe and affect the whole system and all the Virtual Environments.
  
 
== Total RAM ==
 
== Total RAM ==
Line 107: Line 107:
 
=== Utilization ===
 
=== Utilization ===
  
The amount of RAM consumed by all containers can be computed as
+
The amount of RAM consumed by all Virtual Environments can be computed as
  
 
<math>
 
<math>
\sum_{all\ containers}
+
\sum_{all\ VE}
 
     (physpages_{cur}\cdot4096+kmemsize_{cur}+allsocketbuf_{cur})\rm.
 
     (physpages_{cur}\cdot4096+kmemsize_{cur}+allsocketbuf_{cur})\rm.
 
</math>
 
</math>
Line 116: Line 116:
 
The difference between memory usage shown by <code>free(1)</code>
 
The difference between memory usage shown by <code>free(1)</code>
 
or <code>/proc/meminfo</code> and the total amount of RAM consumed by
 
or <code>/proc/meminfo</code> and the total amount of RAM consumed by
containers is the memory used by system daemons
+
Virtual Environments is the memory used by system daemons
 
and different caches.
 
and different caches.
  
Line 122: Line 122:
  
 
<math>
 
<math>
\frac{\displaystyle\sum_{all\ containers}
+
\frac{\displaystyle\sum_{all\ VE}
 
         (physpages_{cur}\cdot4096+
 
         (physpages_{cur}\cdot4096+
 
         kmemsize_{cur}+allsocketbuf_{cur})}
 
         kmemsize_{cur}+allsocketbuf_{cur})}
Line 131: Line 131:
 
Lower utilization means that the system is under-utilized, and,
 
Lower utilization means that the system is under-utilized, and,
 
if other system resources and their commitment levels permit,
 
if other system resources and their commitment levels permit,
the system can host more containers.
+
the system can host more Virtual Environments.
 
By the nature of the accounting of <code>physpages</code> and other parameters,
 
By the nature of the accounting of <code>physpages</code> and other parameters,
 
total RAM utilization can't be bigger than <math>1</math>.
 
total RAM utilization can't be bigger than <math>1</math>.
Line 147: Line 147:
 
However, if this activity is not excessive, the performance decrease is not
 
However, if this activity is not excessive, the performance decrease is not
 
very noticeable.  On the other hand, the benefits of using swap space are
 
very noticeable.  On the other hand, the benefits of using swap space are
quite big, allowing to increase the number of containers in the
+
quite big, allowing to increase the number of Virtual Private Servers in the
 
system about 2 times.
 
system about 2 times.
  
Line 170: Line 170:
 
=== Utilization ===
 
=== Utilization ===
 
<math>
 
<math>
\frac{\displaystyle\sum_{all\ containers}
+
\frac{\displaystyle\sum_{all\ VE}
 
         (oomguarpages_{cur}\cdot4096+
 
         (oomguarpages_{cur}\cdot4096+
 
         kmemsize_{cur}+allsocketbuf_{cur})}
 
         kmemsize_{cur}+allsocketbuf_{cur})}
Line 177: Line 177:
  
 
The normal utilization of memory plus swap ranges between
 
The normal utilization of memory plus swap ranges between
<math>\frac{RAM\ size}{RAM\ size + swap\ size}</math>
+
<math>\frac{RAM\ size}{RAM\ size + swap\ size}
and
+
{\rm \quad and\quad}
<math>\frac{RAM\ size + \frac12swap\ size}{RAM\ size + swap\ size}.</math>
+
\frac{RAM\ size + \frac12swap\ size}{RAM\ size + swap\ size}.</math>
  
 
Lower utilization means that the system memory is under-utilized
 
Lower utilization means that the system memory is under-utilized
Line 188: Line 188:
 
=== Commitment level ===
 
=== Commitment level ===
 
<math>
 
<math>
\frac{\displaystyle\sum_{all\ containers}
+
\frac{\displaystyle\sum_{all\ VE}
 
         (oomguarpages_{bar}\cdot4096+
 
         (oomguarpages_{bar}\cdot4096+
 
         kmemsize_{lim}+allsocketbuf_{lim})}
 
         kmemsize_{lim}+allsocketbuf_{lim})}
Line 196: Line 196:
 
The normal commitment level is about <math>0.8</math>—<math>1</math>.
 
The normal commitment level is about <math>0.8</math>—<math>1</math>.
  
Commitment levels more than <math>1</math> means that the containers are
+
Commitment levels more than <math>1</math> means that the Virtual Environmens are
 
guaranteed more memory than the system has.
 
guaranteed more memory than the system has.
 
Such overcommitment is strongly not recommended, because in that case
 
Such overcommitment is strongly not recommended, because in that case
Line 203: Line 203:
 
inaccessible by <code>ssh(1)</code> and lose other important functionality.
 
inaccessible by <code>ssh(1)</code> and lose other important functionality.
  
It is better to guarantee containers less and have less commitment
+
It is better to guarantee Virtual Environments less and have less commitment
levels than to accidentally overcommit the system by memory plus swap.
+
levels than to accidently overcommit the system by memory plus swap.
If the system has spare memory and swap, containers will
+
If the system has spare memory and swap, Virtual Environments will
 
transparently be able to use the memory and swap above their guarantees.
 
transparently be able to use the memory and swap above their guarantees.
Guarantees given to containers should not be big, and it is normal
+
Guarantees given to Virtual Environments should not be big, and it is normal
if memory and swap usage for some containers stays above their
+
if memory and swap usage for some VEs stays above their
 
guarantee.
 
guarantee.
It is also normal to give guarantees only to containers with
+
It is also normal to give guarantees only to Virtual Environments with
 
preferred service.
 
preferred service.
But administrators should not guarantee container more
+
But administrators should not guarantee Virtual Environment more
 
than the system actually has.
 
than the system actually has.
 
== Allocated memory ==
 
 
This subsection considers standard memory allocations made by
 
applications in the container.
 
The allocations for each container are controlled by
 
two parameters: <code>vmguarpages</code> and <code>privvmpages</code>,
 
discussed in sections FIXME.
 
 
Allocated memory is a more “virtual” system resource than the RAM or RAM
 
plus swap size.
 
Applications can allocate memory but start to use it only later, and
 
the amount of system's free memory will decrease only at the moment of the
 
use.
 
The sum of allocated memory size of all containers is an estimation
 
of how much physical memory will be used when (and if) all applications
 
claim the allocated memory.
 
 
=== Utilization ===
 
<math>
 
\frac{\displaystyle\sum_{all\ containers}
 
        (privvmpages_{cur}\cdot4096+
 
        kmemsize_{cur}+allsocketbuf_{cur})}
 
    {RAM\ size + swap\ size}\rm.
 
</math>
 
 
This utilization level is the ratio of the amount of allocated memory
 
to the capacity of the system.
 
 
Low utilization level means that the system can support more
 
containers, if other resources permit.
 
High utilization levels may, but doesn't necessarily mean that
 
the system is overloaded.
 
As it was explaned above, not all applications use all the allocated memory,
 
so this utilization level may exceed <math>1</math>.
 
 
Computing this utilization level is useful for comparing
 
it with the commitment level and the level of memory allocation restrictions,
 
discussed below, to configure memory allocation restrictions
 
for the container.
 
 
=== Commitment level ===
 
Allocation guarantee commitment level
 
 
<math>
 
\frac{\displaystyle\sum_{all\ containers}
 
        (vmguarpages_{bar}\cdot4096+
 
        kmemsize_{lim}+allsocketbuf_{lim})}
 
    {RAM\ size + swap\ size}
 
</math>
 
 
is the ratio of the memory space guaranteed to be available for
 
allocations to the capacity of the system.
 
Similarly to the commitment level of memory plus swap space (as discussed
 
in subsection [[{{PAGENAME}}#Memory and swap space|Memory and swap space]]),
 
this level should be kept below <math>1</math>.
 
If the level is above <math>1</math>, it significantly increases the chances of
 
applications to be killed instead of be notified in case the system
 
experiences a memory shortage.
 
 
It's better to provide lower guarantees than accidently guarantee more than
 
the system has, because containers are allowed to allocated memory
 
above their guarantee if the system is not tight on memory.
 
It is also normal to give guarantees only to containers with
 
preferred service.
 
 
== Limiting memory allocations ==
 
In addition to providing allocation guarantees, it is possible
 
to impose restrictions on the amount of memory allocated by
 
containers.
 
 
If a system has multiple containers,
 
it is important to make sure that for each container
 
 
<math>privvmpages_{lim}\cdot4096 \le 0.6\cdot {RAM\ size}\rm.</math>
 
 
If this condition is not satisfied, a single container may easily
 
cause an excessive swap-out and very bad performance of the whole system.
 
Usually, for each container <code>privvmpages</code> limitations are
 
to values much less than the size of the RAM.
 
 
The resource control parameters should be configured in a way,
 
so that in case of memory shortage applications are given chance
 
to notice the shortage and exit gracefully, instead of being terminated
 
by the kernel.
 
For this purpose, it is recommended to maintain reasonable
 
total level of memory allocation restrictions, computed as
 
 
<math>
 
\frac{\displaystyle\sum_{all\ containers}
 
        (privvmpages_{lim}\cdot4096+
 
        kmemsize_{lim}+allsocketbuf_{lim})}
 
    {RAM\ size + swap\ size}\rm.
 
</math>
 
 
This number shows how much memory applications are allowed to allocate
 
in comparison with the capacity of the system.
 
 
In practice, a lot of applications do not use the memory very efficiently
 
and, sometimes, allocated memory will never be used later.
 
For example, Apache Web server at start time it allocates about
 
20–30%% more memory that it will ever use.
 
Some multi-threaded applications are especially bad at using their memory,
 
and their rate of allocated to used memory may happen to be 1000%.
 
 
The bigger the level of memory allocation restrictions,
 
the more chances are that applications will be killed instead
 
of getting an error on next memory allocation in case
 
the system experiences memory shortage.
 
The levels ranging in <math>1.5</math>–<math>4</math> can be considered acceptable.
 
Administrators can find experimentally the optimal setting for their load,
 
basing on the frequency of messages “Out of Memory: killing process”
 
in system logs, saved by <code>klogd(8)</code> and <code>syslogd(8)</code>.
 
However, for stability-critical applications, it's better to keep
 
the level not exceeding <math>1</math>.
 
 
<references/>
 

Please note that all contributions to OpenVZ Virtuozzo Containers Wiki may be edited, altered, or removed by other contributors. If you don't want your writing to be edited mercilessly, then don't submit it here.
If you are going to add external links to an article, read the External links policy first!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)

Templates used on this page: