OpenVZ Virtuozzo Containers Wiki - User contributions [en]

KSM (kernel same-page merging)

2016-10-10T04:35:05Z

Corradofiore: Corrected typo (KSM should be written in lowercase)

<translate>

KSM is a memory-saving de-duplication feature, developed by Red Hat. It first appeared in the Linux kernel version 2.6.32.


KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').


Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead == 
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).


The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>


It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node == 
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y


The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>


For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt


To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run


You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.


Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers == 
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:


:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''


Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.


Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.


:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''


:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''


Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:


[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

(If you get an error like "No package yum-downloadonly available", just ignore it and proceed. Most likely, your Yum installation includes that plugin already).

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''


We can now install just the ksm_preload RPM:


[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications == 
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so


E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''
'''export LD_PRELOAD'''

''mode=$1 # start or stop''
''...''


Then (re)start your Percona Server as usual.

== How to check efficiency of KSM == 
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/ksm/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.


To see all the KSM parameters, issue the following command on the HN:
<nowiki>grep -H '' /sys/kernel/mm/ksm/*</nowiki>


On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning == 
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):


[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs

== Caveats == 
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning, effectively trading CPU utilization for less chances of disk swapping, and then relax them after 10 mins or so. Another possibility is to place your swap onto an SSD drive.

== References == 


<references/>

== External links == 


* [[wikipedia:Kernel same-page merging]]
</translate>

[[Category: HOWTO]]

KSM (kernel same-page merging)

2016-10-10T04:34:11Z

Corradofiore: Added note in case yum-downloadonly plugin is not found

<translate>

KSM is a memory-saving de-duplication feature, developed by Red Hat. It first appeared in the Linux kernel version 2.6.32.


KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').


Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead == 
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).


The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>


It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node == 
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y


The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>


For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt


To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run


You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.


Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers == 
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:


:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''


Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.


Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.


:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''


:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''


Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:


[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

(If you get an error like "No package yum-downloadonly available", just ignore it and proceed. Most likely, your Yum installation includes that plugin already).

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''


We can now install just the ksm_preload RPM:


[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications == 
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so


E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''
'''export LD_PRELOAD'''

''mode=$1 # start or stop''
''...''


Then (re)start your Percona Server as usual.

== How to check efficiency of KSM == 
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.


To see all the KSM parameters, issue the following command on the HN:
<nowiki>grep -H '' /sys/kernel/mm/ksm/*</nowiki>


On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning == 
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):


[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs

== Caveats == 
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning, effectively trading CPU utilization for less chances of disk swapping, and then relax them after 10 mins or so. Another possibility is to place your swap onto an SSD drive.

== References == 


<references/>

== External links == 


* [[wikipedia:Kernel same-page merging]]
</translate>

[[Category: HOWTO]]

Setup OpenStack with Virtuozzo 7

2015-12-14T04:50:47Z

Corradofiore: Shortened the EPEL repo install command; no need to specify the full URL as it's a core package in CentOS

{{Virtuozzo}}

This howto describes steps by step installation of OpenStack devstack with [[Virtuozzo]] 7.

Execute the following commands within you Virtuozzo 7 host as root:

Install RDO repo:

$ yum install -y https://rdoproject.org/repos/rdo-release.rpm

Install EPEL repo:

$ yum install -y epel-release

Install required packages:
$ yum install -y git patch redhat-lsb-core euca2ools mysql-connector-python scsi-target-utils

Reinstall http and mod_wsgi:

$ yum remove -y httpd httpd-tools mod_wsgi
$ yum install -y httpd

Enable IP connection tracking for CT0:

$ echo -ne "options nf_conntrack ip_conntrack_disable_ve0=0\n" > /etc/modprobe.d/vz.conf
$ echo -ne "options nf_conntrack ip_conntrack_disable_ve0=0\n" > /etc/modprobe.d/parallels.conf

Then reboot your system:

$ reboot

Create user "stack" and add him to sudoers:

$ adduser stack -d /vz/stack
$ chmod 755 /vz/stack
$ echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

Download and unpack container image:

$ su stack -c "cd ~ && wget http://updates.virtuozzo.com/server/virtuozzo/en_us/odin/7/techpreview-ct/centos7-exe.hds.tar.gz"
$ su stack -c "cd ~ && tar -xzvf centos7-exe.hds.tar.gz"

Clone devstack:

$ su stack -c "cd ~ && git clone git://git.openstack.org/openstack-dev/devstack"

Create local.conf file:

$ cat > ~stack/devstack/local.conf << _EOF
<nowiki>[[local|localrc]]</nowiki>
FORCE=yes
HOST_IP=1.1.1.1
MYSQL_PASSWORD=password
SERVICE_TOKEN=password
SERVICE_PASSWORD=password
ADMIN_PASSWORD=password
LIBVIRT_TYPE=parallels
RABBIT_PASSWORD=password
#Basic services
ENABLED_SERVICES=key,rabbit,mysql,horizon
# Enable Nova services
ENABLED_SERVICES+=,n-api,n-crt,n-cpu,n-cond,n-sch,n-novnc,n-cauth
# Enable Glance services
ENABLED_SERVICES+=,g-api,g-reg
# Enable Cinder services
#ENABLED_SERVICES+=,c-sch,c-api,c-vol
# Enable Heat, to test orchestration
#ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
# Enable Neutron services
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron
# Destination path for installation
DEST=/vz/stack
# Destination for working data
DATA_DIR=/vz/stack/data
# Destination for status files
SERVICE_DIR=/vz/stack/status
LOG_COLOR=False
LOGDAYS=3
LOGFILE=/vz/stack/logs/stack.sh.log
SCREEN_LOGDIR=/vz/stack/logs/screen
ENABLE_METADATA_NETWORK=True
ENABLE_ISOLATED_METADATA=True
IMAGE_URLS="file:///vz/stack/centos7-exe.hds"
_EOF

Change HOST_IP within created ~stack/devstack/local.conf to a valid IP address of your Virtuozzo 7 host.
Change password for OpenStack services to whatever you prefer in ~stack/devstack/local.conf.
Make libvirt daemon accessible via socket for stack user:

$ sed -i s/"#unix_sock_group = \"libvirt\""/"unix_sock_group = \"stack\""/ /etc/libvirt/libvirtd.conf
$ sed -i s/"#unix_sock_ro_perms = \"0777\""/"unix_sock_ro_perms = \"0777\""/ /etc/libvirt/libvirtd.conf
$ sed -i s/"#unix_sock_rw_perms = \"0770\""/"unix_sock_rw_perms = \"0770\""/ /etc/libvirt/libvirtd.conf
$ sed -i s/"#unix_sock_dir = \"\/var\/run\/libvirt\""/"unix_sock_dir = \"\/var\/run\/libvirt\""/ /etc/libvirt/libvirtd.conf
$ sed -i s/"#auth_unix_ro = \"none\""/"auth_unix_ro = \"none\""/ /etc/libvirt/libvirtd.conf
$ sed -i s/"#auth_unix_rw = \"none\""/"auth_unix_rw = \"none\""/ /etc/libvirt/libvirtd.conf

Clone nova repository:

$ su stack -c "cd ~ && git clone git://git.openstack.org/openstack/nova"

Make libvirt 1.2.8 be minimum required version to run with Virtuozzo:

$ sed -i -e "s/MIN_LIBVIRT_PARALLELS_VERSION = (1, 2, 12)/MIN_LIBVIRT_PARALLELS_VERSION = (1, 2, 8)/" ~stack/nova/nova/virt/libvirt/driver.py

Start devstack:

$ su stack -c "~/devstack/stack.sh"

After script finishes, setup your iptables rules to allow incoming http trafic if you want to use OpenStack dashboard. For instance:

$ iptables -D INPUT -j REJECT --reject-with icmp-host-prohibited
$ iptables -A INPUT -p tcp --dport http -j ACCEPT
$ iptables -A INPUT -j REJECT

Here you are!

[[Category: HOWTO]]

Quick installation

2015-12-14T04:39:53Z

Corradofiore: /* Setup on pre-installed Linux distribution */ Added EPEL requirement as per https://bugzilla.openvz.org/show_bug.cgi?id=3275

{{Virtuozzo}}

{{Note|See [[Quick installation]] if you are looking to install the current stable version of OpenVZ.}}

This document briefly describes the steps needed to install Virtuozzo Linux distribution on your machine.

There are a few ways to install Virtuozzo:

=== Bare-metal installation ===

OpenVZ project builds its own Linux distribution with both hypervisor and container virtualization.
It is based on our own Linux distribution, with the additions of [[Download/kernel/rhel7-testing|our custom kernel]], OpenVZ management utilities, [[QEMU]] and Virtuozzo installer. It is highly recommended to use OpenVZ containers and virtual machines with this Virtuozzo installation image. See [[Virtuozzo]].
[http://download.openvz.org/virtuozzo/releases/7.0-beta2/x86_64/iso/ Download] installation ISO image.

=== Using Virtuozzo in the Vagrant box ===

[https://www.vagrantup.com/ Vagrant] is a tool for creating reproducible and portable development environments.
It is easy to run environment with Virtuozzo using Vagrant:

* Download and [https://docs.vagrantup.com/v2/installation/ install Vagrant]
* Download and install [https://www.virtualbox.org/wiki/Downloads Virtualbox], VMware Fusion or VMware Workstation
* Download [https://atlas.hashicorp.com/OpenVZ/boxes/Virtuozzo-7.0 Virtuozzo box]:

$ vagrant init OpenVZ/Virtuozzo-7.0

* Run box:

$ vagrant up --provider virtualbox

and in case of VMware hypervisor:

$ vagrant up --provider vmware_desktop

and in case of Parallels hypervisor:

$ vagrant up --provider parallels

* Attach to console:

$ vagrant ssh

* Use ''vagrant/vagrant'' to login inside box

=== Using Virtuozzo in the Amazon EC2 ===

Follow steps in [[Using Virtuozzo in the Amazon EC2]].

=== Setup on pre-installed Linux distribution ===

{{Note|Pay attention, this installation method currently blocked by broken network after installation - {{OVZ|6454}}.}}

Alternatively, one can install OpenVZ on a pre-installed RPM based Linux distribution.
Supported Linux distributions: Cloud Linux 7.*, CentOS 7.*, Scientific Linux 7.* etc

Follow step-by-step instruction below:

Package ''virtuozzo-release'' will bring meta information and YUM repositories:

# yum localinstall http://download.openvz.org/virtuozzo/releases/7.0/x86_64/os/Packages/v/virtuozzo-release-7.0.0-10.vz7.x86_64.rpm

EPEL is a requisite:

# yum install -y epel-release

Then install mandatory Virtuozzo RPM packages:

# yum install -y prlctl prl-disp-service vzkernel

See OpenVZ [[Packages]] available in various Linux distributions.

=== OpenVZ with upstream Linux kernel ===

See article [[OpenVZ with upstream kernel]] if you want more details about support of upstream kernel.

== Using Virtuozzo ==

Page with [[screencasts]] shows demo with a few Virtuozzo commands. Feel free to add more.

== See also ==

* [https://docs.openvz.org/ Official Virtuozzo documentation]

[[Category: Installation]]
[[Category: HOWTO]]

KSM (kernel same-page merging)

2015-07-21T13:35:13Z

Corradofiore: /* Tuning */ Fixed a typo in the command: there was a ":50" at the end.

KSM is a memory-saving de-duplication feature, developed by Red Hat. It first appeared in the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
<nowiki>grep -H '' /sys/kernel/mm/ksm/*</nowiki>

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning, effectively trading CPU utilization for less chances of disk swapping, and then relax them after 10 mins or so. Another possibility is to place your swap onto an SSD drive.

== References ==

<references/>

== External links ==

* [[wikipedia:Kernel same-page merging]]

[[Category: HOWTO]]

KSM (kernel same-page merging)

2015-07-18T11:29:22Z

Corradofiore: /* Caveats */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
<nowiki>grep -H '' /sys/kernel/mm/ksm/*</nowiki>

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning, effectively trading CPU utilisation for less chances of disk swapping, and then relax them after 10 mins or so. Another possibility is to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:27:09Z

Corradofiore: /* How to check efficiency of KSM */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
<nowiki>grep -H '' /sys/kernel/mm/ksm/*</nowiki>

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:26:17Z

Corradofiore:

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
grep -H '' /sys/kernel/mm/ksm/*

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for <code>/sys/kernel/mm/KSM/sleep_millisecs</code> is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher <code>pages_to_scan</code>, lower <code>sleep_millisecs</code>) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:25:09Z

Corradofiore: /* Enabling memory deduplication libraries in containers */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual <code># yum install ksm_preload</code> command inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
grep -H '' /sys/kernel/mm/ksm/*

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher pages_to_scan, lower sleep_millisecs) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:24:09Z

Corradofiore: /* Enabling KSM on the hardware node */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in <code>/sys/kernel/mm/ksm/</code>, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in <code>/etc/rc.local</code> on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
grep -H '' /sys/kernel/mm/ksm/*

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher pages_to_scan, lower sleep_millisecs) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:08:22Z

Corradofiore: /* Enabling KSM on the hardware node */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/, readable by all but writable only by root. On your hardware node, if KSM is not yet active, you'll see these default values:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in /etc/rc.local on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
grep -H '' /sys/kernel/mm/ksm/*

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher pages_to_scan, lower sleep_millisecs) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:07:03Z

Corradofiore: /* How to check efficiency of KSM */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/, readable by all but writable only by root:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in /etc/rc.local on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

To see all the KSM parameters, issue the following command on the HN:
grep -H '' /sys/kernel/mm/ksm/*

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the same information in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher pages_to_scan, lower sleep_millisecs) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T11:05:46Z

Corradofiore: /* Tuning */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/, readable by all but writable only by root:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in /etc/rc.local on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the saved RAM amount in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

== Caveats ==
The ksmd daemon will take one or two minutes to start deduplicating memory and will require several minutes to reach stable state. During the boot phase your HN could start swapping if you have heavily overcommitted your RAM. You might want to use more aggressive settings (higher pages_to_scan, lower sleep_millisecs) at the beginning and then relax them after 10 mins or so. Others suggest to place your swap onto an SSD drive.

KSM (kernel same-page merging)

2015-07-18T10:38:50Z

Corradofiore: /* Enabling memory deduplication in applications */

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/, readable by all but writable only by root:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in /etc/rc.local on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

== How to check efficiency of KSM ==
To check if KSM is actually reducing memory usage, issue this command on the HN:
[root@HN /]# cat /sys/kernel/mm/KSM/pages_sharing
If the value is greater than 0, you're saving memory. Refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt for more details.

On [https://gist.github.com/wankdanker/1206923 this page] you'll find a simple script which displays the saved RAM amount in MB.

== Tuning ==
On a production machine, you'll want to modify some of the default values. A more sane value for /sys/kernel/mm/KSM/sleep_millisecs is usually between 50 and 250 (YMMV though):

[root@HN ~]# echo 50 > /sys/kernel/mm/ksm/sleep_millisecs:50

KSM (kernel same-page merging)

2015-07-18T08:46:50Z

Corradofiore: Initial version

KSM is a memory-saving de-duplication feature, added by RedHat to the Linux kernel version 2.6.32.

KSM replaces RAM pages of identical content with a single write-protected page, which in turn gets automatically copied to a new one if a process later wants to update its content. This makes the de-duplication mechanism transparent to applications. This strategy is commonly known as COW (''Copy On Write'').

Although KSM was originally developed for use with KVM, it can be used with OpenVZ containers as well.

== Performance and overhead ==
KSM is more effective when the hardware node has plenty of RAM and is running many containers with the same applications (e.g. a HN running 50 containers with Apache, PHP and MySQL).

The amount of RAM that can be saved is in the 30-50% range depending on the application, with applications that have a mixed I/O and CPU footprint (e.g. MySQL and Apache) getting the best results.<ref>[http://www.researchgate.net/publication/220946080_An_Empirical_Study_on_Memory_Sharing_of_Virtual_Machines_for_Server_Consolidation ''An Empirical Study on Memory Sharing of Virtual Machines for Server Consolidation'']</ref>

It is worth noting that KSM and Virtuozzo use totally different strategies for RAM deduplication: KSM provides a constantly running daemon (called ''ksmd'') which scans the HN's memory and merges identical RAM pages gradually over time, whereas Virtuozzo merges requests from different containers to the same physical binaries on disk. In doing so, Virtuozzo incurs no overhead at all, while KSM has a 5-10% CPU overhead depending on its configuration (faster scanning of the HN RAM will require more CPU power).

== Enabling KSM on the hardware node ==
First, you'll want to verify that KSM support is present and enabled in your OpenVZ kernel:
[root@HN ~]# grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/, readable by all but writable only by root:
<nowiki>[root@HN ~]# grep -H '' /sys/kernel/mm/ksm/*
/sys/kernel/mm/ksm/full_scans:0
/sys/kernel/mm/ksm/merge_across_nodes:1
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0
/sys/kernel/mm/ksm/run:0
/sys/kernel/mm/ksm/sleep_millisecs:20</nowiki>

For the meaning of each parameter, please refer to https://www.kernel.org/doc/Documentation/vm/ksm.txt

To start ''ksmd'', issue
[root@HN ~]# echo 1 > /sys/kernel/mm/ksm/run

You can copy the same command in /etc/rc.local on the HN to make it persistent at boot.

Verify that ''ksmd'' is running:
[root@HN ~]# ps aux | grep ksmd
root 264 0.3 0.0 0 0 ? SN Jun14 176:52 [ksmd]
root 989187 0.0 0.0 103252 896 pts/0 S+ 09:30 0:00 grep ksmd

== Enabling memory deduplication libraries in containers ==
In order to have KSM consider a memory page as a candidate for deduplication, the application itself must mark it as mergeable:

:''KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call: int madvise(addr, length, MADV_MERGEABLE).''

Note that most applications do not use madvise() at all: that's why KSM is generally used in conjunction with KVM, which takes care of marking pages as mergeable on behalf of the applications running within each virtual machine.

Luckily, we can override the default behaviour of each application using the '''ksm_preload''' package (http://vleu.net/ksm_preload/) available in CentOS base repo.

:''Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).''

:''This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.''

Do not use the usual "yum install ksm_preload" inside your containers, as it will install an unnecessary stream of dependencies. Assuming your container is running on a recent CentOS 6.x template, issue the following commands instead:

[root@container /]# cd /usr/local/src
[root@container /]# yum install -y yum-downloadonly
''...bunch of output...''

[root@container src]# yum -y --downloadonly --downloaddir=/usr/local/src install ksm_preload
''...bunch of output...''
exiting because --downloadonly specified ''// this is OK''

We can now install just the ksm_preload RPM:

[root@container src]# rpm -i ksm_preload-0.10-3.el6.x86_64.rpm --nodeps

== Enabling memory deduplication in applications ==
In order to make an application take advantage of ksm_preload and use KSM on the HN, add this line into its startup script (assuming your container is running CentOS 6.x x86_64):
LD_PRELOAD=/usr/lib64/libksm_preload.so

E.g., if you want to make Percona Server use KSM, modify its startup script like the following:
[root@container /]# nano /etc/init.d/mysql

''...''
''PATH="/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin"''
''export PATH''

'''LD_PRELOAD=/usr/lib64/libksm_preload.so'''

''mode=$1 # start or stop''
''...''

Then (re)start your Percona Server as usual.

Ploop/Getting started

2014-09-20T14:31:46Z

Corradofiore: /* Requirements */

This article describes how to use container-in-a-file technology aka [[ploop]]

== Requirements ==

The following is required for ploop to work:

# [[Download/kernel/rhel6|RHEL6-based OpenVZ kernel]], i.e. 042stabXXX.
# <code>/vz</code> partition formatted as <code>ext4</code> and mounted in ordered mode, or residing on NFS. No other filesystem is supported.
# ploop library and tools installed (run <code>yum install ploop</code>).

== Making a CT on ploop ==

=== Global configuration ===

In global VZ configuration file [[Man/vz.conf.5|/etc/vz/vz.conf]], set

VE_LAYOUT=ploop

This means that all new containers you will create will have ploop layout by default.

You don't have to set this option if you don't want ploop to be default. Alternatively, you can:

* use <code>--layout</code> option of <code>vzctl create</code> command;
* convert existing container to ploop.

=== Creating a new CT ===

To create a new container on ploop, use:

vzctl create ''CTID'' [--layout ploop] [--diskspace ''nnn''G]

# You can omit <code>--layout</code> option if you have set <code>VE_LAYOUT=ploop</code> in {{Man|vz.conf|5}}
# You can specify initial file system size using <code>--diskspace</code> option (example: 10G for 10 gigabytes)
# If <code>--diskspace</code> is not specified, the size defaults to DISKSPACE limit value in sample CT config used

=== Converting an existing CT ===

1. Set some sane diskspace value (which will be used as a size of newly created ploop image). Here is the example to set diskspace to 10 gigabytes:
vzctl set ''CTID'' --diskspace 10G --save

2. Convert:
vzctl convert ''CTID''

== Resizing a ploop image ==

Ploop image can be resized in both directions (i.e. either shrank or grown). Ploop also supports online/live resize so you don't have to stop a CT to do resize.

To resize a ploop CT image, use

vzctl set ''CTID'' --diskspace ''nnn''G --save

Note that
* resize can safely be performed online (i.e. when container is running)
* there is no need to specify two values for diskspace (unlike simfs, there is no soft and hard quota).
* there is no way to limit or change the number of inodes available for a container (i.e. option '''--diskinodes''' is ignored)

== Showing disk space usage / limits ==

You can use {{Man|vzlist|8}} to see CT disk space usage and limits in a uniform way for any containers (simfs or ploop, started or stopped).

The field '''diskspace''' is showing the current disk space usage, and the fields '''diskspace.s''' and '''diskspace.h''' are showing the filesystem size (or disk space quotas, for simfs case).

The field '''diskinodes''' is showing the current inodes usage, and the fields '''diskinodes.s''' and '''diskinodes.h''' are showing the filesystem size (or disk inodes quotas, for simfs case).

Examples:

vzlist -o smart_name,diskspace.h,diskspace
vzlist -o smart_name,diskinodes.h,diskinodes

On a lower level, same info can be obtained via '''ploop info''' command:

ploop info /vz/private/501/root.hdd/DiskDescriptor.xml

== Using disk quotas inside container ==

If you want to use standard Linux per-user and per-group disk quota inside a container, you have to enable it using

vzctl set ''CTID'' --quotaugidlimit 1000 --save

Note that:
* unlike with simfs, ''--quotaugidlimit'' argument can be any non-zero value;
* enabling or disabling in-container disk quotas requires a container restart, so you can use vzctl's '''--setmode''' option.

See {{Man|vzctl|8}} for more details.

== See also ==

* [[Ploop]]
* [[Ploop/backup]]
* {{Man|ploop|8}}
* {{Man|vz.conf|5}}
* {{Man|vzctl|8}}

[[Category: Storage]]
[[Category: HOWTO]]

Ploop/Getting started

2014-09-20T14:31:22Z

Corradofiore: /* Requirements */

This article describes how to use container-in-a-file technology aka [[ploop]]

== Requirements ==

The following is required for ploop to work:

# [[Download/kernel/rhel6|RHEL6-based OpenVZ kernel]], i.e. 042stabXXX.
# <code>/vz</code> partition formatted as <code>ext4</code> in ordered mode, or residing on NFS. No other filesystem is supported.
# ploop library and tools installed (run <code>yum install ploop</code>).

== Making a CT on ploop ==

=== Global configuration ===

In global VZ configuration file [[Man/vz.conf.5|/etc/vz/vz.conf]], set

VE_LAYOUT=ploop

This means that all new containers you will create will have ploop layout by default.

You don't have to set this option if you don't want ploop to be default. Alternatively, you can:

* use <code>--layout</code> option of <code>vzctl create</code> command;
* convert existing container to ploop.

=== Creating a new CT ===

To create a new container on ploop, use:

vzctl create ''CTID'' [--layout ploop] [--diskspace ''nnn''G]

# You can omit <code>--layout</code> option if you have set <code>VE_LAYOUT=ploop</code> in {{Man|vz.conf|5}}
# You can specify initial file system size using <code>--diskspace</code> option (example: 10G for 10 gigabytes)
# If <code>--diskspace</code> is not specified, the size defaults to DISKSPACE limit value in sample CT config used

=== Converting an existing CT ===

1. Set some sane diskspace value (which will be used as a size of newly created ploop image). Here is the example to set diskspace to 10 gigabytes:
vzctl set ''CTID'' --diskspace 10G --save

2. Convert:
vzctl convert ''CTID''

== Resizing a ploop image ==

Ploop image can be resized in both directions (i.e. either shrank or grown). Ploop also supports online/live resize so you don't have to stop a CT to do resize.

To resize a ploop CT image, use

vzctl set ''CTID'' --diskspace ''nnn''G --save

Note that
* resize can safely be performed online (i.e. when container is running)
* there is no need to specify two values for diskspace (unlike simfs, there is no soft and hard quota).
* there is no way to limit or change the number of inodes available for a container (i.e. option '''--diskinodes''' is ignored)

== Showing disk space usage / limits ==

You can use {{Man|vzlist|8}} to see CT disk space usage and limits in a uniform way for any containers (simfs or ploop, started or stopped).

The field '''diskspace''' is showing the current disk space usage, and the fields '''diskspace.s''' and '''diskspace.h''' are showing the filesystem size (or disk space quotas, for simfs case).

The field '''diskinodes''' is showing the current inodes usage, and the fields '''diskinodes.s''' and '''diskinodes.h''' are showing the filesystem size (or disk inodes quotas, for simfs case).

Examples:

vzlist -o smart_name,diskspace.h,diskspace
vzlist -o smart_name,diskinodes.h,diskinodes

On a lower level, same info can be obtained via '''ploop info''' command:

ploop info /vz/private/501/root.hdd/DiskDescriptor.xml

== Using disk quotas inside container ==

If you want to use standard Linux per-user and per-group disk quota inside a container, you have to enable it using

vzctl set ''CTID'' --quotaugidlimit 1000 --save

Note that:
* unlike with simfs, ''--quotaugidlimit'' argument can be any non-zero value;
* enabling or disabling in-container disk quotas requires a container restart, so you can use vzctl's '''--setmode''' option.

See {{Man|vzctl|8}} for more details.

== See also ==

* [[Ploop]]
* [[Ploop/backup]]
* {{Man|ploop|8}}
* {{Man|vz.conf|5}}
* {{Man|vzctl|8}}

[[Category: Storage]]
[[Category: HOWTO]]

Performance

2012-04-13T07:44:49Z

Corradofiore:

== Benchmark results ==

This page is to collect a list of benchmark results available and demonstrate not only the words about containers technology being more superior from performance point of view compared to hypervisors but also to provide some data proving that.

{| class="wikitable"
|+ Benchmarks
|-
|| '''Benchmark''' || '''Description'''
|-
|| [[/Response Time/]]
|| Microbenchmark demonstrating latency issues of interactive applications in virtualized and loaded systems (netperf RR in various conditions).
|-
|| [[/Network Throughput/]]
|| 10Gbit simple network throughput comparison using netperf test.
|-
|| [[/LAMP/]]
|| Linux Apache+MySql+PHP (LAMP) stack benchmark in multiple simultaneously running virtualization instances.
|-
|| [[/vConsolidate-UP/]]
|| UP configuration of Intel vConsolidate server consolidation benchmark (Java+Apache+MySQL workloads).
|-
|| [[/vConsolidate-SMP/]]
|| SMP configuration of Intel vConsolidate server consolidation benchmark (Java+Apache+MySQL workloads).
|-
|| [[/Microbenchmarks/]]
|| Various microbenchmarks like context switch, system call, etc. Plus Unixbench resuls.
|}

== External sources ==
* [http://www.hpl.hp.com/techreports/2007/HPL-2007-59R1.pdf HP Labs: Performance Evaluation of Virtualization Technologies for Server Consolidation]
* [http://www.ieee-jp.org/section/kansai/chapter/ces/1569177239.pdf A Comparative Study of Open Source Softwares for Virtualization with Streaming Server Applications] - conference proceedings of the 13th IEEE International Symposium on Consumer Electronics (ISCE2009)
* [http://thesai.org/Downloads/Volume2No9/Paper%2020%20-%20The%20Performance%20between%20XEN-HVM,%20XEN-PV%20and%20Open-VZ%20during%20live-migration.pdf The Performance between XEN-HVM, XEN-PV and Open-VZ during live-migration]

Performance

2012-04-13T07:43:58Z

Corradofiore: /* External sources */

Performance

2012-03-18T23:15:18Z

Corradofiore: /* External sources */