Over the past few weeks, I have attempted to get FreeBSD running well on a XEN-HVM and XEN-PV LEB without much success. Last night, I finally decided it was time to give BuyVM and KVM a shot. This is more an example of what virtio can do for BSD on KVM.

** All tests were run 5 times with the best and worse results excluded **

Lets take a base after default FreeBSD 9.0 i386 install...

$ dd if=/dev/zero of=test bs=64k count=16k

16384+0 records in
16384+0 records out
1073741824 bytes transferred in 37.775315 secs (28424431 bytes/sec)
1073741824 bytes transferred in 71.264295 secs (15067038 bytes/sec)
1073741824 bytes transferred in 41.564684 secs (25833032 bytes/sec)

^ Painful does not even begin to explain how slow that felt!

/usr/local/bin/wget cachefly.cachefly.net/100mb.test -O /dev/null

2012-05-11 18:20:56 (45.4 MB/s) - `/dev/null' saved [104857600/104857600]

^ I am actually pretty happy with this. Coming from Australia (lucky to get 20Mbps) and a bunch of 100Mbps LEBs, I thought this was great!

Next up I tried to install /usr/ports/emulators/virtio-kmod without much success... It refused to recognise the disk or NIC at boot, I'm pretty sure something was missing which was stopping the driver from loading

Being forced into a corner, I decided to rebuild world and compile a new kernel.
Having done this a few times before, I was able to knock up a heavily optimised make.conf, src.conf and kernel.conf fairly quickly. The result was a kernel that was more than halved in sized, booting ~4 times as quickly and absolutely flying in actual use... I had a feeling that this round of tests would be good!

Post optimised FreeBSD 10 world and kernel

dd if=/dev/zero of=test bs=64k count=16k

1073741824 bytes transferred in 29.275458 secs (36677200 bytes/sec)
1073741824 bytes transferred in 20.908691 secs (51353852 bytes/sec)
1073741824 bytes transferred in 26.776536 secs (40100102 bytes/sec)

/usr/local/bin/wget cachefly.cachefly.net/100mb.test -O /dev/null

2012-05-12 10:36:31 (50.0 MB/s) - /dev/null' saved [104857600/104857600] 2012-05-12 10:36:37 (46.5 MB/s) -/dev/null' saved [104857600/104857600]
2012-05-12 10:36:41 (48.4 MB/s) - `/dev/null' saved [104857600/104857600]

^ Well that is definitely good... Disk performance was up ~ 100% and somehow interface performance was up another ~10%

... It was a hollow victory, as I was aiming for atleast 50Mbps (preferably 100Mbps) speeds on the disk!
That's when I realised that virtio hadn't even been enabled yet. A quick edit to rc.conf and fstab followed by a reboot and...

Post optimised FreeBSD 10 world and kernel with virtio enabled

dd if=/dev/zero of=test bs=64k count=16k

1073741824 bytes transferred in 8.948978 secs (119984852 bytes/sec)
1073741824 bytes transferred in 9.203055 secs (116672323 bytes/sec)
1073741824 bytes transferred in 8.351456 secs (128569414 bytes/sec)

^ JEEBUSSSSSS! My disk performance has gone from ~ 15Mbps to well over 100Mbps. This is what I was looking for!

/usr/local/bin/wget cachefly.cachefly.net/100mb.test -O /dev/null

2012-05-12 10:47:28 (59.5 MB/s) - /dev/null' saved [104857600/104857600] 2012-05-12 10:47:33 (64.6 MB/s) -/dev/null' saved [104857600/104857600]
2012-05-12 10:47:42 (61.6 MB/s) - `/dev/null' saved [104857600/104857600]

^ Oh and another handy little 25% gain on the interface as well.

All up I am pretty excited with BuyVM, KVM, FreeBSD 10 and virtio at the moment!