That little article on Battle of Mac Virtualization Products caused quite a stir with Parallels and another web publisher contacting me. I'll follow up with them and report back but in the meantime check this out.
I mentioned last time that on Parallels, the linux filesystem (ext3) seemed quite a bit faster than the native filesystem. They are many ways this could happen and honestly I have neither time not expertise to learn about the finer details of HFS3 vs. ext3. Filesystems are messy.
Not that CPU benchmarks are much better, but I do have a silly python benchmark that tests raw performance. I stole the idea from http://furryland.org/~mikec/bench/. I use it mostly to see if compiler flags make any difference when compiling python from source, or what the performance of a pre-made version of python is. The code is at the end.
Apple OS X 10.5
$ ./benchmark.py test_hash: 2.833932 test_list: 3.647719
Ubuntu 8.0.4 Server 64-Bit in VMWare Fusion
$benchmark.py test_hash: 1.697652 test_list: 2.255775
Uhhhh, say what? VMWare is 33%+ faster than native? Remember this is running the same code, on the same machine. I have no idea if Ubuntu being 64-bit (vs. 32) makes any difference. This code doesn't do very much, so I suspect that the Linux memory allocators must be quite a bit better than Mac OS X.
This is of course a bit synthetic, and I can find other examples of the virtualized linux being a lot slower than native mac, however, it is interesting.
Comments most welcome.
The Lame Test
from timeit import Timer
test_hash = """
for i in xrange(100):
x={}
for j in xrange(1000):
x[j]=i
x[j]
"""
test_list = """
v=['a','b','c','d','e','f','g']
for j in xrange(1000):
v.append(j)
v[j]
"""
t = Timer(stmt=test_hash)
print "test_hash: %f" % t.timeit(100)
t = Timer(stmt=test_list)
print "test_list: %f" % t.timeit(10000)
1 comments:
Benchmarking virtual machines is very, very tricky for two reasons:
1) The virtual disk may be cached by the host operating system.
The reason your filesystem benchmark performed so well was probably because you were effectively doing it on a RAM disk: your Mac was likely caching reads and writes to the virtual disk in memory.
2) Benchmarks depend on accurate timers.
In the virtual world, time is a flexible concept. Whenever benchmarking something in a virtual machine, make sure to use a stopwatch to see how much actual time progresses when running a test.
You may be surprised to learn that a test thinks it runs for 10 minutes, but it actually takes 11 or 12 minutes of wall-clock time in some virtualized environments.
Post a Comment