VMmark is a free tool that hardware vendors, virtualization software vendors and other organizations use to measure the performance and scalability of applications running in virtualized environments. Get an accurate measurement of your virtual datacenter performance with VMmark® , the industry's first virtualization platform benchmark for x86-based computers. VMmark enables users to:
- Measure virtual datacenter performance accurately and reliably
- View & compare the performance of different hardware and virtualization platforms
For the previous version go to: VMmark Version 1
The Need for a Virtualization Platform Benchmark
State-of-the-art server consolidation typically collects several diverse workloads onto a virtualization platform - a collection of physical servers accessing shared storage and network resources. Traditional single-workload performance and scalability benchmarks for non-virtualized environments were developed with neither virtual machines nor server consolidation in mind. Even previous virtualization benchmarks have not fully captured the complexities of today’s virtualized datacenters. VMmark 2.x, the industry’s first multi-server datacenter virtualization benchmark, addresses this gap by including as part of the benchmark a variety of common platform-level workloads such as live migration of virtual machines, cloning and deploying of virtual machines, and automatic virtual machine load balancing across the datacenter.
VMmark 2.x: Measure Virtualization Performance with the Industry's First Virtualization Platform Benchmark
VMmark 2.x is the first standard methodology for comparing virtualized platforms. VMmark 2.x generates a realistic measure of virtualization platform performance by incorporating a variety of platform-level workloads such as dynamic VM relocation (vMotion) and dynamic datastore relocation (storage vMotion), in addition to traditional application-level workloads. The benchmark system in VMmark 2.x is comprised of a series of “sub-tests” that are derived from commonly used load-generation tools and commonly initiated virtualization administration tasks.
The VMmark 2.x benchmark features a tile-based scheme for measuring application performance and provides a consistent methodology that captures both the overall scalability and individual application performance. The total number of tiles that a multi-host platform can accommodate and the performance of each individual workload within the tile determine the overall benchmark score.
New in VMmark 2.5 – the benchmark also allows the measurement of datacenter power usage.
Tile-Based Virtualization Platform Benchmarking
VMmark 2.5: Measure Power Usage of Virtualized Data Centers
Virtualization technologies enable multiple application workloads to be consolidated on to relatively few physical servers. This consolidation can result in significant power savings when compared to deploying each application workload natively on a dedicated physical server. The natural fluctuations in computing demands over time that datacenters experience provide additional power saving opportunities.
Many technologies, such as VMware Distributed Power Management (DPM) and various CPU-based power management features, exist to further reduce server power consumption as power demands vary over time. In addition, new and emerging technologies, such as flash-based storage, can have power demands that are markedly different than traditional solutions.
IT architects might wish to consider trade-offs in performance and power consumption when designing datacenters. VMmark 2.5 allows the measurement of power usage while running its full application and infrastructure workloads across a multi-host cluster by providing for three test types:
- Performance only
- Performance with Server Power
- Performance with Server and Storage Power
The power measurement capability in VMmark 2.5 utilizes the SPEC®™ PTDaemon (Power Temperature Daemon). The PTDaemon provides a straightforward and reliable building block with support for the many power analyzers that have passed the SPEC Power Analyzer Acceptance Test.
Note, however, that although VMmark uses the SPEC PTDaemon, VMmark results are not SPEC metrics and cannot in any manner be compared to SPEC metrics.