OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Performance tuning

2007-09-19T11:39:11Z

Alexandr Andreev:

This page describes how to improve performance of OpenVZ system.

= HW node environment tuning =

== Disable unnecessary services ==

Disable all default services that you do not need to use and then reboot your host.

For example, the <code>audit</code> daemon can significantly decrease performance of linux kernel system calls (up to ~20%) even if you do not use any audit rules, or even if you just stopped this service without host reboot!

To setup default services, use <code>chkconfig</code> or <code>ntsysv</code> in RedHat, or <code>rc-update</code> in Gentoo, <code>update-rc.dv</code> on Debian

== Shell scripts performance improvement ==

To improve performance of small shell scripts, which spends a lot of time starting the shell binary itself (like the shell scripts test from the [http://www.tux.org/pub/tux/niemi/unixbench/ unixbench] package), you can set your <code>LANG</code> environment variable to <code>"C"</code>.

To see current settings, type

# locale

If you want to change it only for the current shell session, do:

# export LANG=C

If you want to change the default value, modify the <code>/etc/sysconfig/i18n</code> file.

If your default <code>LANG</code> environment variable was set to something like <code>en_US.UTF-8</code>, you can reduce shell (bash) startup time up to ~15% with <code>LANG=C</code>.

= Virtual Environment tuning =

== CPU distribution inside VE on SMP hosts ==

* If the total number of VE's in your host is more than CPUs number, and there are many '''threads''' running inside each VE it is better to give just a single VCPU to each VE.
In this case thread memory locality will significantly reduce overhead on SMP memory coherence and overall performance can be increased up to ~50-100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== Network checksumming ==
('''TODO''')

== Shell scripts performance improvement ==

Please note, that on VE creation the default <code>LANG</code> value will be the same as in the HW node. So you can tune it in node (see [[#Shell scripts performance improvement]] above), or set it in VE the same way.

The second important thing is the locale cache. On <code>rpm</code> based distributions, usually it is created by the <code>glibc-common-XXX.rpm</code> post install script and it can be up to 50 MBytes on some distributions. So on some VE templates it can be missed to save disk space. But you can always create it inside VE later by the following command (you must be the root user):

# build-locale-archive

And again, in some cases shell (bash) startup time can be reduced up to ~15%.

Performance tuning

2007-09-19T11:34:44Z

Alexandr Andreev:

This page describes how to improve performance of OpenVZ system.

= HW node environment tuning =

== Disable unnecessary services ==

Disable all default services that you do not need to use and then reboot your host.

For example, the <code>audit</code> daemon can significantly decrease performance of linux kernel system calls (up to ~20%) even if you do not use any audit rules, or even if you just stopped this service without host reboot!

To setup default services, use <code>chkconfig</code> or <code>ntsysv</code> in RedHat, or <code>rc-update</code> in Gentoo, <code>update-rc.dv</code> on Debian

== Shell scripts performance improvement ==

To improve performance of small shell scripts, which spends a lot of time starting the shell binary itself (like the shell scripts test from the [http://www.tux.org/pub/tux/niemi/unixbench/ unixbench] package), you can set your <code>LANG</code> environment variable to <code>"C"</code>.

To see current settings, type

# locale

If you want to change it only for the current shell session, do:

# export LANG=C

If you want to change the default value, modify the <code>/etc/sysconfig/i18n</code> file.

If your default <code>LANG</code> environment variable was set to something like <code>en_US.UTF-8</code>, you can reduce shell (bash) startup time up to ~15% with <code>LANG=C</code>.

= Virtual Environment tuning =

== CPU distribution inside VE on SMP hosts ==

* If the total number of VE's in your host is more than CPUs number, and there are many '''threads''' running inside each VE it is better to give just a single VCPU to each VE.
In this case thread memory locality will significantly reduce overhead on SMP memory coherence and overall performance can be increased up to ~50-100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== Network checksumming ==
('''TODO''')

== Shell scripts performance improvement ==

Please note, that on VE creation the default <code>LANG</code> value will be the same as in the HW node. So you can tune it in node (see [[#Shell scripts performance improvement]] above), or set it in VE the same way.

The second important thing is the locale cache. By default it is created by glibc rpm post install script and it can be up to 50 MBytes on some distributions. So on some VE templates it can be missed to save disk space. But you can always create it inside VE after VE creation by the following command (you must be the root user):

# build-locale-archive

And again, in some cases shell (bash) startup time can be reduced up to ~15%.

Performance tuning

2007-09-19T11:34:07Z

Alexandr Andreev:

This page describes how to improve performance of OpenVZ system.

= HW node environment tuning =

== Disable unnecessary services ==

Disable all default services that you do not need to use and then reboot your host.

For example, the <code>audit</code> daemon can significantly decrease performance of linux kernel system calls (up to ~20%) even if you do not use any audit rules, or even if you just stopped this service without host reboot!

To setup default services, use <code>chkconfig</code> or <code>ntsysv</code> in RedHat, or <code>rc-update</code> in Gentoo, <code>update-rc.dv</code> on Debian

== Shell scripts performance improvement ==

To improve performance of small shell scripts, which spends a lot of time starting the shell binary itself (like the shell scripts test from the [http://www.tux.org/pub/tux/niemi/unixbench/ unixbench] package), you can set your <code>LANG</code> environment variable to <code>"C"</code>.

To see current settings, type

# locale

If you want to change it only for the current shell session, do:

# export LANG=C

If you want to change the default value, modify the <code>/etc/sysconfig/i18n</code> file.

If your default <code>LANG</code> environment variable was set to something like <code>en_US.UTF-8</code>, you can reduce shell (bash) startup time up to ~15% with <code>LANG=C</code>.

= Virtual Environment tuning =

== CPU distribution inside VE on SMP hosts ==

* If the total number of VE's in your host is more than CPUs number, and there are many '''threads''' running inside each VE it is better to give just a single VCPU to each VE.
In this case thread memory locality will significantly reduce overhead on SMP memory coherence and overall performance can be increased up to ~50-100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== Network checksumming ==
('''TODO''')

== Shell scripts performance improvement ==

Please note, that on VE creation the default <code>LANG</code> value will be the same as in the HW node. So you can tune it in node (see [[#Shell scripts performance improvement] aaa]), or set it in VE the same way.

The second important thing is the locale cache. By default it is created by glibc rpm post install script and it can be up to 50 MBytes on some distributions. So on some VE templates it can be missed to save disk space. But you can always create it inside VE after VE creation by the following command (you must be the root user):

# build-locale-archive

And again, in some cases shell (bash) startup time can be reduced up to ~15%.

Performance tuning

2007-08-03T15:49:35Z

Alexandr Andreev:

Performance tuning

2007-08-03T15:27:14Z

Alexandr Andreev: Performance measurement moved to OpenVZ performance tuning

This page describes how to do correct performance measurements on OpenVZ system.

= Test conditions =

== kernels with different versions ==
* If you want to compare performance of the kernel on different hosts, or measure OpenVZ performance overhead, it's strongly recommended to compare the same kernel version with similar .config file and on the same linux distribution.

* If you compare kernels with different versions, please check all .config options that are differ, especially _DEBUG_ options. For example, on unixbench pipe throughput test on 2.6.18 kernel, disabled CONFIG_DEBUG_HIGHMEM option will increase performance up to 20%

== running services ==
* Before performing test measurement, you should disable '''all''' default services in your runlevel and then '''reboot''' your host.

It is not enough to just stop services, because some services, like <code>audit</code> will affect performance even if the daemon is already stopped. For some unixbench tests it can be ~20% overhead if <code>auditd</code> was started once on a host before reboot.

On RedHat distributions use <code>chkconfig</code> or <code>ntsysv</code> utility to disable default services. (<code>rc-update</code> in Gentoo, <code>update-rc.dv</code> for Debian)

== filesystem tests ==
* If you perform filesystem tests, please keep in mind filesystem type, block size, mount options and so on.

For example, ext3 filesystem performance highly depends on journal type and mount options.

* Also please always note/report IO-scheduler type. Different IO-schedulers can highly affect your tests results (up to 30%).

If your kernel support different IO-schedulers, you can get/set the type here:

# cat /sys/block/hda/queue/scheduler
noop anticipatory deadline [cfq]
# echo noop > /sys/block/hda/queue/scheduler

== network isolation ==
* You should disable local network/internet connection if your tests doesn't require it.

== CPU distribution inside VE on SMP hosts ==
* If the number of VE's in your host is more than CPUs number, and there are many tasks/tests running inside each VE, and tasks are scheduled quite often, it's better to give just one CPU for each VE. In this case the VirtualCPU-scheduler performance overhead can be significantly decreased, and performance can increase up to 100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== network performance ==
* please do not use file transferring utilities to test the network performance, because the bottleneck of these tests is usually file system performance - not TCP/IP stack

== network checksumming ==
'''TODO'''

Performance measurement

2007-08-03T15:27:14Z

Alexandr Andreev: Performance measurement moved to OpenVZ performance tuning

#REDIRECT [[OpenVZ performance tuning]]

Performance tuning

2007-08-03T12:32:24Z

Alexandr Andreev: /* CPU distribution inside VE on SMP hosts */

Performance tuning

2007-08-03T12:30:13Z

Alexandr Andreev: /* kernels with different versions */

This page describes how to do correct performance measurements on OpenVZ system.

= Test conditions =

== kernels with different versions ==
* If you want to compare performance of the kernel on different hosts, or measure OpenVZ performance overhead, it's strongly recommended to compare the same kernel version with similar .config file and on the same linux distribution.

* If you compare kernels with different versions, please check all .config options that are differ, especially _DEBUG_ options. For example, on unixbench pipe throughput test on 2.6.18 kernel, disabled CONFIG_DEBUG_HIGHMEM option will increase performance up to 20%

== running services ==
* Before performing test measurement, you should disable '''all''' default services in your runlevel and then '''reboot''' your host.

It is not enough to just stop services, because some services, like <code>audit</code> will affect performance even if the daemon is already stopped. For some unixbench tests it can be ~20% overhead if <code>auditd</code> was started once on a host before reboot.

On RedHat distributions use <code>chkconfig</code> or <code>ntsysv</code> utility to disable default services. (<code>rc-update</code> in Gentoo, <code>update-rc.dv</code> for Debian)

== filesystem tests ==
* If you perform filesystem tests, please keep in mind filesystem type, block size, mount options and so on.

For example, ext3 filesystem performance highly depends on journal type and mount options.

* Also please always note/report IO-scheduler type. Different IO-schedulers can highly affect your tests results (up to 30%).

If your kernel support different IO-schedulers, you can get/set the type here:

# cat /sys/block/hda/queue/scheduler
noop anticipatory deadline [cfq]
# echo noop > /sys/block/hda/queue/scheduler

== network isolation ==
* You should disable local network/internet connection if your tests doesn't require it.

== CPU distribution inside VE on SMP hosts ==
* If the number of VE's in your host is more than CPUs number, and there are many tasks/tests running inside each VE, and that tasks are scheduled quite often, it's better to give just one CPU for each VE. In this case the VirtualCPU-scheduler performance overhead can be significantly decreased, and performance can increase up to 100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== network performance ==
* please do not use file transferring utilities to test the network performance, because the bottleneck of these tests is usually file system performance - not TCP/IP stack

== network checksumming ==
'''TODO'''

Performance tuning

2007-08-03T12:29:03Z

Alexandr Andreev: New page: This page describes how to do correct performance measurements on OpenVZ system. = Test conditions = == kernels with different versions == * If you want to compare performance of the ker...

This page describes how to do correct performance measurements on OpenVZ system.

= Test conditions =

== kernels with different versions ==
* If you want to compare performance of the kernel on different hosts, or if you want to measure OpenVZ performance overhead, it's strongly recommended to compare the same kernel version with similar .config file and maybe with the same linux distribution.

* If you compare kernels with different versions, please check all .config options that are differ, especially _DEBUG_ options. For example, on unixbench pipe throughput test on 2.6.18 kernel, disabled CONFIG_DEBUG_HIGHMEM option will increase performance up to 20%

== running services ==
* Before performing test measurement, you should disable '''all''' default services in your runlevel and then '''reboot''' your host.

It is not enough to just stop services, because some services, like <code>audit</code> will affect performance even if the daemon is already stopped. For some unixbench tests it can be ~20% overhead if <code>auditd</code> was started once on a host before reboot.

On RedHat distributions use <code>chkconfig</code> or <code>ntsysv</code> utility to disable default services. (<code>rc-update</code> in Gentoo, <code>update-rc.dv</code> for Debian)

== filesystem tests ==
* If you perform filesystem tests, please keep in mind filesystem type, block size, mount options and so on.

For example, ext3 filesystem performance highly depends on journal type and mount options.

* Also please always note/report IO-scheduler type. Different IO-schedulers can highly affect your tests results (up to 30%).

If your kernel support different IO-schedulers, you can get/set the type here:

# cat /sys/block/hda/queue/scheduler
noop anticipatory deadline [cfq]
# echo noop > /sys/block/hda/queue/scheduler

== network isolation ==
* You should disable local network/internet connection if your tests doesn't require it.

== CPU distribution inside VE on SMP hosts ==
* If the number of VE's in your host is more than CPUs number, and there are many tasks/tests running inside each VE, and that tasks are scheduled quite often, it's better to give just one CPU for each VE. In this case the VirtualCPU-scheduler performance overhead can be significantly decreased, and performance can increase up to 100%!

To set the number of CPUs available inside VE use:

# vzctl set $VEID --cpus N

== network performance ==
* please do not use file transferring utilities to test the network performance, because the bottleneck of these tests is usually file system performance - not TCP/IP stack

== network checksumming ==
'''TODO'''