New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Fedora's IO limitations
I have been rebuilding my fedora servers to debian recently and noticed that my debian machines have significantly higher IO performance. At first I thought I was hallucinating but when I rebuilt my second machine, I am sure there is something strange is going on.
Here are the results from rebuilding @Spaceberg 's VPS
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2023-04-23 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Mon Aug 7 05:21:35 PM CEST 2023
100 44480 100 44480 0 0 61340 0 --:--:-- --:--:-- --:--:-- 1561k
Basic System Information:
---------------------------------
Uptime : 6 days, 4 hours, 56 minutes
Processor : AMD EPYC 7402 24-Core Processor
CPU cores : 4 @ 2794.750 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 7.7 GiB
Swap : 0.0 KiB
Disk : 2.0 TiB
Distro : Fedora Linux 38 (Cloud Edition)
Kernel : 6.2.9-300.fc38.x86_64
VM Type : KVM
IPv4/IPv6 : ✔ Online / ✔ Online
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 126.96 MB/s (31.7k) | 119.20 MB/s (1.8k)
Write | 127.29 MB/s (31.8k) | 119.82 MB/s (1.8k)
Total | 254.26 MB/s (63.5k) | 239.03 MB/s (3.7k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 109.87 MB/s (214) | 112.61 MB/s (109)
Write | 115.71 MB/s (225) | 120.11 MB/s (117)
Total | 225.58 MB/s (439) | 232.73 MB/s (226)
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2023-04-23 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Mon Aug 7 05:28:33 PM CEST 2023
100 44480 100 44480 0 0 158k 0 --:--:-- --:--:-- --:--:-- 158k
Basic System Information:
---------------------------------
Uptime : 0 days, 0 hours, 1 minutes
Processor : AMD EPYC 7402 24-Core Processor
CPU cores : 4 @ 2794.750 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 7.7 GiB
Swap : 0.0 KiB
Disk : 1023.9 GiB
Distro : Debian GNU/Linux 12 (bookworm)
Kernel : 6.1.0-9-amd64
VM Type : KVM
IPv4/IPv6 : ✔ Online / ✔ Online
IPv6 Network Information:
---------------------------------
ISP : OVH SAS
ASN : AS16276 OVH SAS
Host : OVH
Location : Strasbourg, Grand Est (GES)
Country : France
fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 224.19 MB/s (56.0k) | 3.28 GB/s (51.3k)
Write | 224.78 MB/s (56.1k) | 3.30 GB/s (51.6k)
Total | 448.97 MB/s (112.2k) | 6.59 GB/s (103.0k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 14.91 GB/s (29.1k) | 15.61 GB/s (15.2k)
Write | 15.70 GB/s (30.6k) | 16.65 GB/s (16.2k)
Total | 30.61 GB/s (59.7k) | 32.26 GB/s (31.5k)
The previous machine that I rebuilt was from @crunchbits which I did not have the comparison IO results but it showed significant IO performance differences.
Anyone have any thoughts about what happened here?
Comments
Doesn't look like the same disk.
Fedora: Disk : 2.0 TiB - I assume it's an HDD slice?
Debian: Disk : 1023.9 GiB - must be an NVMe?
Also, why would you use 6-months support distro on server ? 😕
I tested it out on Hetzner. I guess it is not an OS problem.
its a testing machine
Could be btrfs(Fedora) vs ext4 (Debian) ?
I think fedora defaults to btrfs, btrfs and ext4 have very different IOPS on the same disk
It is due to Debian's I/O scheduler.
The disks have btrfs (fedora) and xfs (debian). I guess you are right.
Better test would be to test on them both using the same filesystem.
Fodora is using BFQ, and Debian is using Deadline. BFQ is good for rotating drives, not for solid state drives
I don't think you can choose the filesystem from the build templates though.
CentOS 9 Stream (XFS)
Ubuntu 22.04 (EXT4)
From the tests I made it seems to be a BTRFS problem but we won't know for sure without swapping the filesystem in the same OS.
I have confirmed it with Hetzner.
Another yabs of fresh install