Benchmarking virtual machines is very, very tricky...

2008-04-28T12:31:00.000-04:00

Benchmarking virtual machines is very, very tricky for two reasons:

1) The virtual disk may be cached by the host operating system.

The reason your filesystem benchmark performed so well was probably because you were effectively doing it on a RAM disk: your Mac was likely caching reads and writes to the virtual disk in memory.

2) Benchmarks depend on accurate timers.

In the virtual world, time is a flexible concept. Whenever benchmarking something in a virtual machine, make sure to use a stopwatch to see how much actual time progresses when running a test.

You may be surprised to learn that a test thinks it runs for 10 minutes, but it actually takes 11 or 12 minutes of wall-clock time in some virtualized environments.

Comments on digital sanitation engineering: Virtualization is faster than Native on Mac OS X 1...

Benchmarking virtual machines is very, very tricky...