Introduction to the Isolation Benchmark Suite (IBS).

Isolation Benchmark Suite is designed to quantify the degree to which a virtualization system limits the impact of a misbehaving virtual machine on other well-behaving virtual machines running on the same physical machine

This benchmark suite includes six different stress tests -

  • a CPU intensive test
  • a memory intensive test
  • a fork bomb.
  • a disk intensive test
  • two network intensive tests (send and receive)
Unlike other benchmarks suites that typically quantified the overhead of virtualization for one VM compared to a base OS, quantifying performance isolation is challenging because the impact can vary based on the type of "misbehavior". Nevertheless, performance isolation is an essential feature especially in commercial service hosting environments, a key target application of modern virtualization systems, where mutually untrusting VMs run on the same physical machine.

This Isolation Benchmark Suite has been applied to three flavors of virtualization systems --- an example of full virtualization (VMware Workstation), an example of paravirtualization (Xen) and two examples of operating system level virtualization (Solaris Containers and OpenVZ). We have published a paper: "Quantifying the Performance Isolation Properties of Virtualization Systems". The isolation performance results can be found here.

The results, from the Isolation Benchmark Suite, can highlight the difference between different classes of virtualization systems as well as the importance of considering multiple categories of resource consumption when evaluating the performance isolation properties of a virtualization system

Isolation Benchmark Suite Download Resources
  • The submitted version of the paper to the Experimental CS conference is available here: DOWNLOAD
  • The current version of the Isolation Benchmark Suite is available here: DOWNLOAD
  • The current results from running the Isolation Benchmark Suite are available here: DOWNLOAD