FreeBSD (12) + KVM bad performance

jordynegen11

Hello,

Currently we're running QEMU/KVM on a CentOS 7 server. We found out that the performance of a FreeBSD VM is significant lower then a Ubuntu VM.

We're talking about 15% less CPU/mem performance and 60% less disk IO performance.

Result of CPU benchmark on FreeBSD VM: (2 CPU cores)

--- proc/mem/performance test single core ---
................................................................
64 rounds~ 1.00 GB -> 171.89 MB/s
--- proc/mem/performance test multi-core ---
................
4 times 64 rounds ~ 4.00 GB -> 364.98 MB/s

Result of CPU benchmark on Ubuntu VM: (2 CPU cores)

--- proc/mem/performance test single core ---
................................................................
64 rounds~ 1.00 GB -> 231.36 MB/s
--- proc/mem/performance test multi-core ---
................
4 times 64 rounds ~ 4.00 GB -> 500.66 MB/s

Result of disk benchmark on FreeBSD VM:

--- disk test ---
Sequential writing
.................................................................................................................................
347.55 MB/s
Random writing
.................................................................................................................................
304.57 MB/s
Sequential reading
.................................................................................................................................
699.01 MB/s
Random reading
.................................................................................................................................
690.14 MB/s

Result of disk benchmark on Ubuntu VM:

--- disk test ---
Sequential writing
.................................................................................................................................
725.89 MB/s
Random writing
.................................................................................................................................
1.247 GB/s
Sequential reading
.................................................................................................................................
2.391 GB/s
Random reading
.................................................................................................................................
2.152 GB/s

Both VMs are located on the same node so they have the same CPU and RAM. All drivers are set to VIRTIO.

Mabe there is someone here that has or had the same problem as us? Is this normal?

I think mabe it's a driver issue in FreeBSD, (for the disk problems) but i'm not sure yet.

help is appreciated

theblackesthat

Try https://www.netbsd.org and see what kind of performance that gives you?

jsg

@theblackesthat said:
Try https://www.netbsd.org and see what kind of performance that gives you?

Uhm, what would that tell him about the problem he described?

angstrom

@theblackesthat said:
Try https://www.netbsd.org and see what kind of performance that gives you?

If the OP wants to use FreeBSD, it's not clear that "try NetBSD" (for that matter, why not: "try OpenBSD") is such useful advice.

@jordynegen11:

As far as I'm aware, VirtIO yields better results with Linux than with *BSD, so I'm not sure that one is able to replicate Linux performance with *BSD on KVM even in the best of cases. But the difference that you observe isn't fully explained by this.

By the way, are you using CPU host-passthrough on both KVMs?

deank

Sending a complaint to EU might help.

angstrom

Also, what file system are you using on FreeBSD? It may be better to try with UFS (UFS2).

jordynegen11

@angstrom

Hi,

Thanks for your answer.

By the way, are you using CPU host-passthrough on both KVMs?

Yes, I enabled host-passthrough on both VMs

Also, what file system are you using on FreeBSD? It may be better to try with UFS (UFS2).

Gonna try that!

angstrom

@jordynegen11 said:

@angstrom

Hi,

Thanks for your answer.

By the way, are you using CPU host-passthrough on both KVMs?

Yes, I enabled host-passthrough on both VMs

Also, what file system are you using on FreeBSD? It may be better to try with UFS (UFS2).

Gonna try that!

By the way, as a general sanity test, it may be good to try the same version of FreeBSD with the same guest settings on a KVM at another provider in order to see whether the results are comparable.

Also, how are you doing the benchmarks? Any chance that they may be Linux-biased in some way?

jordynegen11

angstrom said: the way, as a general sanity test, it may be good to try the same version of FreeBSD with the same guest settings on a KVM at another provider in order to see whether the results are comparable.

Also, how are you doing the benchmarks? Any chance that they may be Linux-biased in some way?

I used 3 diffrent benchmarks special made for FreeBSD and Linux. All the same results.

FiroSolutions

https://wiki.freebsd.org/bhyve

Falzo

@FiroSolutions said:
https://wiki.freebsd.org/bhyve

good luck with your provider tag.

jordynegen11

@Falzo said:

@FiroSolutions said:
https://wiki.freebsd.org/bhyve

good luck with your provider tag.

We don't have any FreeBSD hypervisors and since bhyve only runs on FreeBSD, this is not an option. @FiroSolutions

jsg

@jordynegen11 said:

@Falzo said:

@FiroSolutions said:
https://wiki.freebsd.org/bhyve

good luck with your provider tag.

We don't have any FreeBSD hypervisors and since bhyve only runs on FreeBSD, this is not an option. @FiroSolutions

And you are right. While Bhyve is really nice it also is really young, even still immature.

FHR

2.4GB/s random reads sound awfully like Linux cache. I reckon it's a benchmarking error.