6,534
edits
Changes
created (work in progress)
{{UBC toc}}
The [[UBC consistency check]] article discussed validation of a resource control configuration
for a single Virtual Environment. This article discusses checks that the configuration of the
'''whole''' system is '''valid'''.
Configurations where resources allowed for Virtual Environments exceed system
capacity<ref>More precisely, configurations with excessive
overcommitment, as explained below.</ref> are not valid
and dangerous from stability point of view.
They may result in abnormal termination of the applications,
bad responsiveness of the system and, sometimes, system hangs.
Whereas the configuration validation discussed in [[UBC consistency check]]
addressed application functionality,
the validation considered in this section is aimed at security and stability
of the whole system.
The best way to make sure that the configuration of the whole system
is valid is to run periodic automatic checks, based on the formulae described
below.
== Resource utilization and commitment level ==
Several resources of the whole system (such as RAM) are discussed below
in terms of '''utilization''' and '''commitment level'''.
'''Utilization''' shows the amount of resources consumed by all
Virtual Environments at the given time.
In general, low utilization values mean that the system is under-utilized.
Often, it means that the system is capable of supporting more Virtual
Environment if the existing VEs continue to maintain
the same load and resource consumption level.
High utilization values (in general, more than <code>1</code>)
mean the the system is overloaded and the service level
of the Virtual Environments is degraded.
'''Commitment level''' shows how much resources are “promised” to the existing
Virtual Environments. Low commitment levels mean that the system
is capable of supporting more Virtual Environments.
Commitment levels more than <code>1</code> mean that the
Virtual Environments are promised more resources
than the system has, and in this case the system is said to be
overcommitted.
If the system runs a lot of Virtual Environments,
it is usually acceptable to have some overcommitment, because it is unlikely that all Virtual
Environments will request resources at the same time.
However, higher commitment levels (as discussed below for each resource
individually) will cause Virtual Environmentss to experience failures to
allocate and use the resources promised to them.
== “Low memory” ==
Because of specifics of architecture of Intel's processors, the RAM of the
computer can't be used uniformly.
The most important memory area is so called “low memory”, a part of memory
residing at lower addresses and directly accessible by the kernel.
For current Linux kernels, the size of low memory area is 832MB
(or, if the computer has less RAM than 832MB, the size of the RAM).
=== Utilization ===
The lower bound estimation of low memory utilization is
<math>
\frac{\displaystyle\sum_{all\ VE}
(kmemsize_{cur}+allsocketbuf_{cur})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm,
</math>
where
<math>
allsocketbuf=tcprcvbuf+tcpsndbuf+dgramrcvbuf+othersockbuf\rm.
</math>
Utilization of low memory below <math>1</math> is normal.
Utilization above <math>1</math> is not safe, and utilization above <math>2</math>
is dangerous is very likely to cause bad system responsiveness,
application stalls for seconds or more and termination of some applications.
=== Commitment level ===
The commitment level can be computed as
<math>
\frac{\displaystyle\sum_{all\ VE}
(kmemsize_{lim}+allsocketbuf_{lim})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm.
</math>
Commitment levels below <math>1</math> are normal.
Levels between <math>1</math> and <math>1.2</math> are usually acceptable for systems with about
100 Virtual Environments.
Systems with more Virtual Environments may have the commitment level
increased, to about <math>1.5</math>—<math>2</math> for 400 VEs.
Higher commitment levels for this resource are not recommended,
because the consequences of exceeding the low memory capacities
are severe and affect the whole system and all the Virtual Environments.
== Total RAM ==
This subsection discusses usage of the whole RAM and its utilization.
Usage of swap space and the sum of used RAM and swap space are discussed
below in subsection [[{{PAGENAME}}#Memory and swap space|Memory and swap space]].
Current version of OpenVZ can't guarantee availability of certain
amount of memory (in opposite to the sum of memory and swap space),
so the commitment level is not applicable to the total RAM.
Such guarantees will be implemented in future versions.
=== Utilization ===
The amount of RAM consumed by all Virtual Environments can be computed as
<math>
\sum_{all\ VE}
(physpages_{cur}\cdot4096+kmemsize_{cur}+allsocketbuf_{cur})\rm.
</math>
The difference between memory usage shown by <code>free(1)</code>
or <code>/proc/meminfo</code> and the total amount of RAM consumed by
Virtual Environments is the memory used by system daemons
and different caches.
The memory utilization can be computed as
<math>
\frac{\displaystyle\sum_{all\ VE}
(physpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size}\rm.
</math>
Utilization levels from <math>0.8</math> to <math>1</math> are normal.
Lower utilization means that the system is under-utilized, and,
if other system resources and their commitment levels permit,
the system can host more Virtual Environments.
By the nature of the accounting of <code>physpages</code> and other parameters,
total RAM utilization can't be bigger than <math>1</math>.
== Memory and swap space ==
The main resource of the computer determining the amount of memory
applications can use is the sum of RAM and swap sizes.
If the total size of the used memory exceeds the RAM size,
Linux kernel moves some data to swap and and loads it back when the
application needs it.
More frequently used data tends to stay in RAM, less frequently used
data spends more time in swap.
Swap-in and swap-out activity reduces the system performance to some extent.
However, if this activity is not excessive, the performance decrease is not
very noticeable. On the other hand, the benefits of using swap space are
quite big, allowing to increase the number of Virtual Private Servers in the
system about 2 times.
Swap space is essential for handling system load bursts.
System with enough swap just slows down at high load bursts,
whereas the system without swap reacts to high load bursts by refusing
memory allocations (causing applications to refuse to accept clients or
terminate) and by direct killing of some applications.
Additionally, the presence of swap space helps the system to better
balance memory and move data between “low memory” and the rest of the RAM.
In all OpenVZ installations it is strongly recommended
to have swap of size not less than the RAM size.
Also, it is not recommended to create swap space of the size
of more than 4 times RAM size because of performance degradation
related to swap-in and swap-out activity.
That is, the system should be configured so that
<math>RAM\ size \le swap\ size \le 4\cdot RAM\ size\rm.</math>
The optimal configuration is when swap size is twice more than the RAM size.
=== Utilization ===
<math>
\frac{\displaystyle\sum_{all\ VE}
(oomguarpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size + swap\ size}\rm.
</math>
The normal utilization of memory plus swap ranges between
<math>\frac{RAM\ size}{RAM\ size + swap\ size}
{\rm \quad and\quad}
\frac{RAM\ size + \frac12swap\ size}{RAM\ size + swap\ size}.</math>
Lower utilization means that the system memory is under-utilized
at the moment of checking the utilization.
Higher utilization is likely to cause gradual performance degradation because
of swap-in and swap-out activity and is a sign of overloading of the system.
=== Commitment level ===
<math>
\frac{\displaystyle\sum_{all\ VE}
(oomguarpages_{bar}\cdot4096+
kmemsize_{lim}+allsocketbuf_{lim})}
{RAM\ size + swap\ size}\rm.
</math>
The normal commitment level is about <math>0.8</math>—<math>1</math>.
Commitment levels more than <math>1</math> means that the Virtual Environmens are
guaranteed more memory than the system has.
Such overcommitment is strongly not recommended, because in that case
if all the memory is consumed, random applications, including the ones
belonging to the host system, may be killed and the system may become
inaccessible by <code>ssh(1)</code> and lose other important functionality.
It is better to guarantee Virtual Environments less and have less commitment
levels than to accidently overcommit the system by memory plus swap.
If the system has spare memory and swap, Virtual Environments will
transparently be able to use the memory and swap above their guarantees.
Guarantees given to Virtual Environments should not be big, and it is normal
if memory and swap usage for some VEs stays above their
guarantee.
It is also normal to give guarantees only to Virtual Environments with
preferred service.
But administrators should not guarantee Virtual Environment more
than the system actually has.
The [[UBC consistency check]] article discussed validation of a resource control configuration
for a single Virtual Environment. This article discusses checks that the configuration of the
'''whole''' system is '''valid'''.
Configurations where resources allowed for Virtual Environments exceed system
capacity<ref>More precisely, configurations with excessive
overcommitment, as explained below.</ref> are not valid
and dangerous from stability point of view.
They may result in abnormal termination of the applications,
bad responsiveness of the system and, sometimes, system hangs.
Whereas the configuration validation discussed in [[UBC consistency check]]
addressed application functionality,
the validation considered in this section is aimed at security and stability
of the whole system.
The best way to make sure that the configuration of the whole system
is valid is to run periodic automatic checks, based on the formulae described
below.
== Resource utilization and commitment level ==
Several resources of the whole system (such as RAM) are discussed below
in terms of '''utilization''' and '''commitment level'''.
'''Utilization''' shows the amount of resources consumed by all
Virtual Environments at the given time.
In general, low utilization values mean that the system is under-utilized.
Often, it means that the system is capable of supporting more Virtual
Environment if the existing VEs continue to maintain
the same load and resource consumption level.
High utilization values (in general, more than <code>1</code>)
mean the the system is overloaded and the service level
of the Virtual Environments is degraded.
'''Commitment level''' shows how much resources are “promised” to the existing
Virtual Environments. Low commitment levels mean that the system
is capable of supporting more Virtual Environments.
Commitment levels more than <code>1</code> mean that the
Virtual Environments are promised more resources
than the system has, and in this case the system is said to be
overcommitted.
If the system runs a lot of Virtual Environments,
it is usually acceptable to have some overcommitment, because it is unlikely that all Virtual
Environments will request resources at the same time.
However, higher commitment levels (as discussed below for each resource
individually) will cause Virtual Environmentss to experience failures to
allocate and use the resources promised to them.
== “Low memory” ==
Because of specifics of architecture of Intel's processors, the RAM of the
computer can't be used uniformly.
The most important memory area is so called “low memory”, a part of memory
residing at lower addresses and directly accessible by the kernel.
For current Linux kernels, the size of low memory area is 832MB
(or, if the computer has less RAM than 832MB, the size of the RAM).
=== Utilization ===
The lower bound estimation of low memory utilization is
<math>
\frac{\displaystyle\sum_{all\ VE}
(kmemsize_{cur}+allsocketbuf_{cur})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm,
</math>
where
<math>
allsocketbuf=tcprcvbuf+tcpsndbuf+dgramrcvbuf+othersockbuf\rm.
</math>
Utilization of low memory below <math>1</math> is normal.
Utilization above <math>1</math> is not safe, and utilization above <math>2</math>
is dangerous is very likely to cause bad system responsiveness,
application stalls for seconds or more and termination of some applications.
=== Commitment level ===
The commitment level can be computed as
<math>
\frac{\displaystyle\sum_{all\ VE}
(kmemsize_{lim}+allsocketbuf_{lim})}
{0.4\cdot\min(RAM\ size, {\rm 832MB})}\rm.
</math>
Commitment levels below <math>1</math> are normal.
Levels between <math>1</math> and <math>1.2</math> are usually acceptable for systems with about
100 Virtual Environments.
Systems with more Virtual Environments may have the commitment level
increased, to about <math>1.5</math>—<math>2</math> for 400 VEs.
Higher commitment levels for this resource are not recommended,
because the consequences of exceeding the low memory capacities
are severe and affect the whole system and all the Virtual Environments.
== Total RAM ==
This subsection discusses usage of the whole RAM and its utilization.
Usage of swap space and the sum of used RAM and swap space are discussed
below in subsection [[{{PAGENAME}}#Memory and swap space|Memory and swap space]].
Current version of OpenVZ can't guarantee availability of certain
amount of memory (in opposite to the sum of memory and swap space),
so the commitment level is not applicable to the total RAM.
Such guarantees will be implemented in future versions.
=== Utilization ===
The amount of RAM consumed by all Virtual Environments can be computed as
<math>
\sum_{all\ VE}
(physpages_{cur}\cdot4096+kmemsize_{cur}+allsocketbuf_{cur})\rm.
</math>
The difference between memory usage shown by <code>free(1)</code>
or <code>/proc/meminfo</code> and the total amount of RAM consumed by
Virtual Environments is the memory used by system daemons
and different caches.
The memory utilization can be computed as
<math>
\frac{\displaystyle\sum_{all\ VE}
(physpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size}\rm.
</math>
Utilization levels from <math>0.8</math> to <math>1</math> are normal.
Lower utilization means that the system is under-utilized, and,
if other system resources and their commitment levels permit,
the system can host more Virtual Environments.
By the nature of the accounting of <code>physpages</code> and other parameters,
total RAM utilization can't be bigger than <math>1</math>.
== Memory and swap space ==
The main resource of the computer determining the amount of memory
applications can use is the sum of RAM and swap sizes.
If the total size of the used memory exceeds the RAM size,
Linux kernel moves some data to swap and and loads it back when the
application needs it.
More frequently used data tends to stay in RAM, less frequently used
data spends more time in swap.
Swap-in and swap-out activity reduces the system performance to some extent.
However, if this activity is not excessive, the performance decrease is not
very noticeable. On the other hand, the benefits of using swap space are
quite big, allowing to increase the number of Virtual Private Servers in the
system about 2 times.
Swap space is essential for handling system load bursts.
System with enough swap just slows down at high load bursts,
whereas the system without swap reacts to high load bursts by refusing
memory allocations (causing applications to refuse to accept clients or
terminate) and by direct killing of some applications.
Additionally, the presence of swap space helps the system to better
balance memory and move data between “low memory” and the rest of the RAM.
In all OpenVZ installations it is strongly recommended
to have swap of size not less than the RAM size.
Also, it is not recommended to create swap space of the size
of more than 4 times RAM size because of performance degradation
related to swap-in and swap-out activity.
That is, the system should be configured so that
<math>RAM\ size \le swap\ size \le 4\cdot RAM\ size\rm.</math>
The optimal configuration is when swap size is twice more than the RAM size.
=== Utilization ===
<math>
\frac{\displaystyle\sum_{all\ VE}
(oomguarpages_{cur}\cdot4096+
kmemsize_{cur}+allsocketbuf_{cur})}
{RAM\ size + swap\ size}\rm.
</math>
The normal utilization of memory plus swap ranges between
<math>\frac{RAM\ size}{RAM\ size + swap\ size}
{\rm \quad and\quad}
\frac{RAM\ size + \frac12swap\ size}{RAM\ size + swap\ size}.</math>
Lower utilization means that the system memory is under-utilized
at the moment of checking the utilization.
Higher utilization is likely to cause gradual performance degradation because
of swap-in and swap-out activity and is a sign of overloading of the system.
=== Commitment level ===
<math>
\frac{\displaystyle\sum_{all\ VE}
(oomguarpages_{bar}\cdot4096+
kmemsize_{lim}+allsocketbuf_{lim})}
{RAM\ size + swap\ size}\rm.
</math>
The normal commitment level is about <math>0.8</math>—<math>1</math>.
Commitment levels more than <math>1</math> means that the Virtual Environmens are
guaranteed more memory than the system has.
Such overcommitment is strongly not recommended, because in that case
if all the memory is consumed, random applications, including the ones
belonging to the host system, may be killed and the system may become
inaccessible by <code>ssh(1)</code> and lose other important functionality.
It is better to guarantee Virtual Environments less and have less commitment
levels than to accidently overcommit the system by memory plus swap.
If the system has spare memory and swap, Virtual Environments will
transparently be able to use the memory and swap above their guarantees.
Guarantees given to Virtual Environments should not be big, and it is normal
if memory and swap usage for some VEs stays above their
guarantee.
It is also normal to give guarantees only to Virtual Environments with
preferred service.
But administrators should not guarantee Virtual Environment more
than the system actually has.
