WP/Containers density

Containers density. Why containers are so good at it?

What is density?

Density is a characteristic which tells how many containers (CTs) or Virtual Machines (VMs) virtualization technology can run successfully on a given hardware. Typically users also care for quality of service of their software, i.e. how well their services are working on utilized node and how fast they respond to external requests. Typical metrics of quality of service are average response time, 99.9% requests response time, and max response time. When a hardware node is capable to handle the load, these metrics either do not grow much with bigger number of containers, or grow linearly. When a hardware node becomes over utilized, these metrics typically start to degrade exponentially (due to memory swap out).

What makes containers to be perfectly suitable for high density?

• Containers do not reserve memory assigned to them. They exhibit real-time elastic behavior: if a container do not use memory, then it is free for use by other containers. In other words, overcommitment is as natural and easy as on a standalone Linux box, when multiple applications compete for memory resources. As a result, a simple container running ssh, apache, init and cron takes only 10-20Mb of physical RAM. Sure, for efficient caching of apache pages more RAM is needed.

• Containers memory management is system-wide wise. If one container needs more physical RAM (e.g. for apache pages caching) and hardware node have no more memory available, kernel will automatically reclaim last recently used caches of other containers.

• Parallels Virtuozzo Containers product goes a step further by introducing container templates, used as a basis for all containers, and a special copy-on-write filesystem, which makes sure that original template is kept untouched and container gets its own private copy of template file when it tries to modify it. As a result all common files are shared across the containers and present on disk and in memory caches in a single instance. This saves memory, reduces I/O and makes L2/L3 caches work more effective.

Why Virtual Machines are not that good?

Hypervisors (Xen, ESX, KVM) are not that good on high density scenarios. There are multiple reasons for that:

• Memory is basically reserved on the host on VM start. KVM and XEN reserve the whole memory by default and do not allow memory overcommitment. As a result you can't run more then 15 VMs with 1GB RAM on 16Gb box. ESX, being the most advanced hypervisor, uses page sharing and ballooning technologies to introduce memory overcommitment. However, on practice it allows to get only about 2x times overcommitment on the guests of the same type. Our experiments tell that half of the improvement is due to page sharing, and half is due to ballooning.
• Multiple kernels and their system data structures in memory. RAM pages sharing and ballooning help a bit here, too.
• what happens when CT is above it's limit
• what happens when node RAM is exhausted
• plots and examples. Kir had http_load plot in the past. We will have LAMP results as well.