Measure Performance with the Industry's First Virtualization Platform Benchmark

VMware VMmark® is a free tool that hardware vendors, virtualization software vendors and other organizations use to measure the performance and scalability of applications running in virtualized environments. Get an accurate measurement of your virtual data center performance with VMmark , the industry's first virtualization platform benchmark for x86-based computers. VMmark enables users to:

  • Measure virtual data center performance accurately and reliably.
  • View and compare the performance of different hardware and virtualization platforms.

For the previous version go to: VMmark Version 1

The Need for a Virtualization Platform Benchmark

State-of-the-art server consolidation typically collects several diverse workloads onto a virtualization platform - a collection of physical servers accessing shared storage and network resources. Traditional single-workload performance and scalability benchmarks for non-virtualized environments were developed with neither virtual machines nor server consolidation in mind. Even previous virtualization benchmarks have not fully captured the complexities of today’s virtualized data centers. VMmark 2.x, the industry’s first multi-server data center virtualization benchmark, addresses this gap by including as part of the benchmark a variety of common platform-level workloads such as live migration of virtual machines, cloning and deploying of virtual machines, and automatic virtual machine load balancing across the data center.

VMmark 2.x: Measure Virtualization Performance with the Industry's First Virtualization Platform Benchmark

VMmark 2.x is the first standard methodology for comparing virtualized platforms. VMmark 2.x generates a realistic measure of virtualization platform performance by incorporating a variety of platform-level workloads such as dynamic virtual machine relocation (vMotion) and dynamic datastore relocation (storage vMotion), in addition to traditional application-level workloads. The benchmark system in VMmark 2.x comprises a series of “sub-tests” that are derived from commonly used load-generation tools and commonly initiated virtualization administration tasks.

The VMmark 2.x benchmark features a tile-based scheme for measuring application performance and provides a consistent methodology that captures both the overall scalability and individual application performance. The total number of tiles that a multi-host platform can accommodate and the performance of each individual workload within the tile determine the overall benchmark score.

New in VMmark 2.5 – the benchmark also allows the measurement of data center power usage.

Tile-Based Virtualization Platform Benchmarking

concept_diagram

VMmark 2.5: Measure Power Usage of Virtualized Data Centers  

Virtualization technologies enable multiple application workloads to be consolidated on to relatively few physical servers. This consolidation can result in significant power savings when compared to deploying each application workload natively on a dedicated physical server. The natural fluctuations in computing demands over time that data centers experience provide additional power saving opportunities.

Many technologies, such as VMware Distributed Power Management (DPM) and various CPU-based power management features, exist to further reduce server power consumption as power demands vary over time. In addition, new and emerging technologies, such as flash-based storage, can have power demands that are markedly different than traditional solutions.

IT architects might wish to consider trade-offs in performance and power consumption when designing data centers. VMmark 2.5 allows the measurement of power usage while running its full application and infrastructure workloads across a multi-host cluster by providing for three test types:

  • Performance only.
  • Performance with Server Power.
  • Performance with Server and Storage Power.

The power measurement capability in VMmark 2.5 utilizes the SPEC®™ PTDaemon (Power Temperature Daemon). The PTDaemon provides a straightforward and reliable building block with support for the many power analyzers that have passed the SPEC Power Analyzer Acceptance Test.

Note, however, that although VMmark uses the SPEC PTDaemon, VMmark results are not SPEC metrics and cannot in any manner be compared to SPEC metrics.

Performance Blog

Learn about performance of virtualized environments at the VROOM! Blog

Read Now