Difference between revisions of "Performance/vConsolidate-SMP"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(Created page with '=== Benchmark Description === This benchmark is designed to measure aggregated server performance in consolidation scenario: where different apps are running at the same time in…')
 
m (Reverted edits by 18.85.56.20 (talk) to last revision by Vlukovnikov)
 
(5 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
=== Benchmark Description ===
 
=== Benchmark Description ===
  
This benchmark is designed to measure aggregated server performance in consolidation scenario: where different apps are running at the same time in separate virtual machines or operating system containers.
+
The goal of Intel vConsolidate benchmark is to measure aggregated server performance in consolidation scenario: when different (and non-related unlike to LAMP benchmark) applications are running on the same box each in its own virtual environment (virtual machine or container).
  
 +
vConsolidate benchmark was developed by Intel in cooperation with other vendors. It runs separate workloads with Java (SPECjbb test), Mail, Web and Database VMs running concurrently. Each set of such VMs is called CSU - Consolidation Stack Unit. Performance metric is a geomean from throughput of each workload type: transactions/sec for Db, requests/sec for Web and java operations/sec for Java. The same type of metric is commonly used in other benchmarks like those from SPEC.
 +
 +
vConsolidate benchmark is very similar to other virtualization specific benchmarks like VMMark from VMWare and SPECvirt, but the latter are more enterprise oriented and generate the load requiring a fast SAN storage and high end hardware. Since we believe containers benefits are even more prominent on commodity hardware we use vConsolidate benchmark to demonstrate that.
  
 
=== Implementation ===
 
=== Implementation ===
  
We use standard industrial benchmark: vConsolidate. It was developed by Intel in cooperation with other vendors. vConsolidate measures performance from Java, Web and Db VMs running concurrently (we excluded Mail VM as it was MS Windows version only). Each set of such three VMs is called CSU - Consolidation Stack Unit.
+
Intel vConsolidate v1.1. We do not run Mail VM as Intel benchmark has Microsoft Windows version only for mail workload.
Performance metric is geomean from throughput of each workload type: transactions/sec for Db, requests/sec for Web and java operations/sec for Java.
 
  
 
=== Testbed Configuration ===
 
=== Testbed Configuration ===
Server: 4xHexCore Intel Xeon (2.66 GHz), 64 GB RAM, HP MSA1500 SAN Storage, 8 SATA (7200 RPM) Disks in RAID0
+
Hardware:
 +
* Server: 4xHexCore Intel Xeon (2.66 GHz), 64 GB RAM, HP MSA1500 SAN Storage, 8 SATA (7200 RPM) Disks in RAID0
 +
* Client: 4xHexCore Intel Xeon (2.136 GHz), 32 GB RAM
 +
* Network: 1Gbit direct server <-> client connection
  
Client: 4xHexCore Intel Xeon (2.136 GHz), 32 GB RAM
+
Platform:
 
+
* Virtualization Software: ESXi4.1up1, XenServer5.6fp1, OpenVZ (RH5) 2.6.18-028stab090.4, OpenVZ (RH6) 2.6.32-042test006.1.x86_64
Network: Gbit direct server<>client connection
+
* Guest OS: Centos 5.5 x86_64
 
 
Virtualization Software: XenServer5.6fp1, OpenVZ (RH5) 2.6.18-028stab085.3, OpenVZ (RH6) 2.6.32-042test006.1.x86_64
 
 
 
Guest OS: Centos 5.5 x86_64
 
  
 
Software and Tunings:
 
Software and Tunings:
 +
* Each VM/CT was configured with 4 vCPUs, 1 GB RAM
 +
* Custom vConsolidate profile was used: 4 load threads for Java workload, 4 load threads for Db workload and 8 threads for Web workload (standard setting)
 
* Firewall was turned off
 
* Firewall was turned off
* All other tunings were left at default values
+
* All other settings were left as defaults
  
 
=== Benchmark Results ===
 
=== Benchmark Results ===
Line 30: Line 33:
 
=== Summary ===
 
=== Summary ===
  
*TODO: write summary
+
OpenVZ demonstrates outstanding performance benefits over hypervisors in case of multi-processor environments and multi-threaded workloads:
 +
* Almost 3x times better overall performance in case of single set of workloads (1 CSU).
 +
* 2.1-2.2x times better overall performance on 5 and 10 CSU.
 +
 
 +
=== TODO ===
 +
* add a link to paper where explaining why containers are great for SMP workloads and CPU overcommit.

Latest revision as of 05:23, 28 November 2011

Benchmark Description[edit]

The goal of Intel vConsolidate benchmark is to measure aggregated server performance in consolidation scenario: when different (and non-related unlike to LAMP benchmark) applications are running on the same box each in its own virtual environment (virtual machine or container).

vConsolidate benchmark was developed by Intel in cooperation with other vendors. It runs separate workloads with Java (SPECjbb test), Mail, Web and Database VMs running concurrently. Each set of such VMs is called CSU - Consolidation Stack Unit. Performance metric is a geomean from throughput of each workload type: transactions/sec for Db, requests/sec for Web and java operations/sec for Java. The same type of metric is commonly used in other benchmarks like those from SPEC.

vConsolidate benchmark is very similar to other virtualization specific benchmarks like VMMark from VMWare and SPECvirt, but the latter are more enterprise oriented and generate the load requiring a fast SAN storage and high end hardware. Since we believe containers benefits are even more prominent on commodity hardware we use vConsolidate benchmark to demonstrate that.

Implementation[edit]

Intel vConsolidate v1.1. We do not run Mail VM as Intel benchmark has Microsoft Windows version only for mail workload.

Testbed Configuration[edit]

Hardware:

  • Server: 4xHexCore Intel Xeon (2.66 GHz), 64 GB RAM, HP MSA1500 SAN Storage, 8 SATA (7200 RPM) Disks in RAID0
  • Client: 4xHexCore Intel Xeon (2.136 GHz), 32 GB RAM
  • Network: 1Gbit direct server <-> client connection

Platform:

  • Virtualization Software: ESXi4.1up1, XenServer5.6fp1, OpenVZ (RH5) 2.6.18-028stab090.4, OpenVZ (RH6) 2.6.32-042test006.1.x86_64
  • Guest OS: Centos 5.5 x86_64

Software and Tunings:

  • Each VM/CT was configured with 4 vCPUs, 1 GB RAM
  • Custom vConsolidate profile was used: 4 load threads for Java workload, 4 load threads for Db workload and 8 threads for Web workload (standard setting)
  • Firewall was turned off
  • All other settings were left as defaults

Benchmark Results[edit]

VConsolidate-SMP.png

Summary[edit]

OpenVZ demonstrates outstanding performance benefits over hypervisors in case of multi-processor environments and multi-threaded workloads:

  • Almost 3x times better overall performance in case of single set of workloads (1 CSU).
  • 2.1-2.2x times better overall performance on 5 and 10 CSU.

TODO[edit]

  • add a link to paper where explaining why containers are great for SMP workloads and CPU overcommit.