All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
RAID 1 Significant Performance Loss Dedicated Server
integritly
Member
Running some tests on a Hetzner auction server with 2x 512GB NVMe drives in RAID 1 and noticed that when I run bench.sh inside the rescue system (no RAID configured) I get massive IO speed, upwards of 2000 MB/s.
But as soon as I install an OS and RAID 1 goes into effect, that speed drops to ~550-650 MB/s. Which is still plenty fast, but clearly well below what these drives are capable of.
I also noticed something similar with an NForce dedi running a pair of Samsung 850/860 SSDs, where in single drive performance the IO speed would be ~580 MB/s, but in RAID 1 they would be capped at 400 MB/s.
Has anyone noticed something like this before?
In my case, I run a pretty latency-intensive database, so I'll take any additional IO speed gains I can get. So my question is, if software RAID 1 puts such a performance limit on SSD/NVMe drives, would I be better off going with a single drive application and implementing redundancy a different way?
Thanks.
Comments
Are you sure you are actually doing the IO on the disks? AFAIK Hetzner's rescue system runs off the system RAM.
RAID1 isn't really going to give you a ton of a performance boost... if any.
RAID1 is mirroring only. There is no striping. Striping is where you start to see real performance gains.
You might see performance gains with reading data in a RAID1 environment, since the data can be read from either disk. But with writing, the data is written to both disks at the same time.
Although, with that much of a difference, I would echo @saibal comment.
cat /proc/mdstat
See if the raid is still syncing when you're testing.
If it's latency based, then disk throughput speed isn't the right test for such. You should be testing NVMe latency response times for your specific loads - including testing at the same block/page size as your database i.e. MySQL InnoDB tables use 16KB page size by default. And drives fsync performance see https://www.percona.com/blog/2018/07/18/why-consumer-ssd-reviews-are-useless-for-database-performance-use-case/
and https://www.percona.com/blog/2018/02/08/fsync-performance-storage-devices/
You'll always loose IO performance on a raid-1 over a single disk (general rule, from what I remember, is about half IOPS). Raid-0 will give you a performance increase. Raid-5, 6 and 10 will give you a performance increase if you have more disks. I don't remember the exact IOPS calculations off the top of my head, but Google is your friend here.
simply this.
Thank you for those articles, that's going to be a big help for me in determining the best route to go for our db. Looks like we haven't been testing the right way to figure out which drives would grant us the lowest latency, as it's not just a "one size fits all" fast IO = good to go approach.
Looks like the Intel drives have much better fsync performance than their Samsung counterparts, even on the consumer side! Very surprising. And btw, centminmod has been excellent for us as well. Thanks again!
You are benchmarking the ramdisk I/O...
The Dedi we have a Hetzner for NanoKVM does about 1.3GB/sec in Raid 1.
2000MB/sec does sound a bit low for memory speeds usually they are between 4-5GB/sec.
500MB/sec sounds low for NVMe for SSD would be better as expected in Raid 1.