Fairsched API

There are several parameters tuning which the OpenVZ administrator can change the CPU usage of a VE. Usually one sets these parameters with the aid of the vzctl utility. But sometimes it can be useful to modify these parameters directly, without using vzctl. For example, it can be required by developers who want to implement an in-place solution to control a particular virtualization system based on OpenVZ. That is why the Fairsched API is introduced.

The Fairsched API is the set of system calls that can be used (and that are, in fact, used in the vzctl utility) for altering various parameters of the CPU Fair scheduler, such as cpuunits, cpulimit, cpus, and cpumask. The article is targeted at describing the API.

CPU weight

Synopsis:

int fairsched_chwt(unsigned int id, unsigned int wght);

Description:

The CPU weight defines how much CPU time a VE gets. The smaller the weight is, the more CPU time the container gets. The CPU weight is a relative parameter. That means that multiplying the weight of all running VEs by the same value does not change the CPU time distribution. The minimal value of the weight is 1, and the maximal value is FSCHWEIGHT_MAX (65535). The weight of a newly created VE is set to 500 by default.

The fairsched_chwt() system call sets the CPU weight of a VE. id is the id of the VE. wght is the desired weight.

Return value:

On success, fairsched_chwt() returns 0. On error, -1 is returned, and errno is set appropriately.

Errors:

• EINVAL: id=0.
• EINVAL: the value of wght is out of the allowed range.
• ENOENT: the VE does not exist.
• EPERM: the caller does not have permissions to modify the CPU parameters of the VE.

Notes:

It is possible to change the CPU weight of the host system. The host system's id equals INT_MAX (2147483647).

The cpuunits option of the vzctl utility is not the same thing as the CPU weight described above, but they are closely connected by the simple equation:

cpuweight = 500000 / cpuunits

CPU limit

Synopsis:

int fairsched_rate(unsigned int id, int op, unsigned int rate);

Description:

The CPU limit is the upper bound to the CPU usage for a VE. The limit is set for all CPUs the VE can execute on. That means the maximal value of the CPU limit (i.e. no limit) is the maximal limit on 1 CPU, which equals 1024, multiplied by NUM_CPUS, the number of CPUs this VM can execute on. The CPU limit is an absolute parameter in the sense that altering the limit of one VE does not affect the limits of other VEs.

The fairsched_rate() system call is used to set, get, or drop (i.e. set to the maximum) CPU limit of a VE. id is the id of the VE. op can be one of the following:

• FAIRSCHED_SET_RATE to set the limit; in this case rate is the limit value to set.
• FAIRSCHED_DROP_RATE to drop the limit; rate is not used.
• FAIRSCHED_GET_RATE to get the current value of the limit; the function returns the limit value, rate is not used.

Return value:

On success, fairsched_rate() returns the current value of the CPU limit if op is FAIRSCHED_SET_RATE or FAIRSCHED_GET_RATE, or 0 if op is FAIRSCHED_DROP_RATE. On error, -1 is returned, and errno is set appropriately.

Errors:

• EINVAL: id=0.
• EINVAL: op is invalid.
• EINVAL: op=FAIRSCHED_SET_RATE, and rate<=0.
• ENOENT: the VE does not exist.
• ENODATA: op=FAIRSCHED_GET_RATE, and the limit is not set (or was dropped).
• EPERM: the caller does not have permissions to modify the CPU parameters of the VE.

Notes:

It is possible to limit the CPU usage of the host system. The host system's id equals INT_MAX (2147483647).

CPU limits are available only in rhel5-based and rhel6-based kernels, and they behave a bit differently in them in respect of distributing CPU time among CPUs the VE can execute on.

In the rhel5 kernel the limit has a container-wide meaning. For example, if there are 2 CPUs available for a container, and the limit is set to 1024 (of 2048), the container's CPU usage can be 100/0%, or 50/50%, or any other values whose sum does not exceed 100%.

In the rhel6 kernel the limit is divided between the CPUs proportionally, and a busy CPU cannot borrow time from an idle one, i.e. with a 2 CPUs container and 1024 (of 2048) limit set the usage of each CPU cannot exceed 50% in any case.

As a result, the minimal value for the limit is 1 in the rhel5 kernel and 1 * NUM_CPUS in the rhel6 kernel. If the given limit is out of the allowed range it is automatically clamped to the range.

Number of CPUs

Synopsis:

int fairsched_vcpus(unsigned int id, unsigned int vcpus);

Description:

The fairsched_vcpus() system call sets the number of virtual CPUs available in a VE. id is the id of the VE. vcpus is the number of virtual CPUs to set for the VE. If vcpus=0, the call attempts to set the number of CPUs in the VE to the number of online CPUs.

Return value:

On success, fairsched_vcpus() returns 0. On error, -1 is returned, and errno is set appropriately.

Errors:

• EINVAL: id=0.
• ENOENT: the VE does not exist.
• EPERM: the caller does not have permissions to modify the CPU parameters of the VE.

Notes:

The parameter have not been implemented in the rhel6 kernel yet.

CPU affinity mask

Synopsis:

int fairsched_cpumask(unsigned int id, unsigned int len, unsigned long *mask)

Description:

A VE's CPU affinity mask determines the set of CPUs on which it is eligible to run. A CPU affinity mask is represented by an array of long's so that a VE is allowed to execute on the n-th CPU if and only if the (n%sizeof(long))-th bit of the (n/sizeof(long))-th element of the array is set.

The fairsched_cpumask() system call sets the CPU affinity mask for a VE. id is the id of the VE. mask is the mask, and len is the size of the mask, in bytes.

Return value:

On success, fairsched_cpumask() returns 0. On error, -1 is returned, and errno is set appropriately.

Errors:

• EBUSY: the VE has a child cpuset which is not a subset of the mask.
• EINVAL: id=0.
• EINVAL: the mask intersects with a cpuset, and either that cpuset or the VE's cpuset is marked exclusive.
• ENOENT: the VE does not exist.
• ENOMEM: there is no memory to copy the mask.
• ENOSPC: the mask has no active CPUs set.
• EPERM: the caller does not have permissions to modify the CPU parameters of the VE.

Notes:

The parameter is available since the rhel6 kernel.