Give us your best sysctl tweaks

Astro

I recently saw someone post some network tweaks in a thread here. What are some of the things that you throw in there? I'd like to create a nice all-in-one kind of template that I can use across my servers.

NHNHNH000

it's possibel
https://fasterdata.es.net/host-tuning/linux/#toc-anchor-2

JeDaYoshi

Here are my base tweaks. I don't think they're perfect yet, and it's something I still am benchmarking/changing for improvements as time goes by, but it already gives me much better results than the base settings on 1-10Gbps servers. Happy for feedback on these too

net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1

net.core.somaxconn = 8192
net.core.netdev_max_backlog = 5000
net.core.default_qdisc = fq_codel
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_base_mss = 1240
net.ipv4.tcp_mtu_probing = 2
net.ipv4.tcp_fastopen = 3
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_notsent_lowat = 131072
net.ipv4.tcp_shrink_window = 1
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv6.route.max_size = 2147483647
net.core.optmem_max = 65536
net.core.rmem_default = 1310720
net.core.rmem_max = 33554432
net.core.wmem_default = 1310720
net.core.wmem_max = 33554432
net.ipv4.tcp_rmem = 4096 262144 67108864
net.ipv4.tcp_wmem = 4096 262144 67108864
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192
vm.min_free_kbytes = 65536

fs.inotify.max_queued_events = 1048576
fs.inotify.max_user_instances = 1048576
fs.inotify.max_user_watches = 1048576
vm.max_map_count = 1048576
vm.overcommit_memory = 1
vm.swappiness = 1

For the qdisc I use fq_codel by default instead of fq (it is explicitly set because Linux still defaults to pfifo_fast, fq_codel being a default is a systemd thing so it's not there on, as an example, Alpine Linux) because fq is more tuned for TCP workloads - in my case, I usually have mixed TCP/UDP use-cases (mostly real-time multimedia streaming, QUIC, and tunnels/VPNs - all use UDP). I use cake in some servers too (and my home network :P); but depends on use-case.

For congestion control, I use BBRv3 on my servers (compared to the BBRv1 the kernel includes). This would need a custom kernel - for Debian/Ubuntu you can use XanMod as an easy way to have it (plus includes a few other changes).

MSS is 1240 to fit in the default of a 1280 MTU. Depending on the server's use-case I might adjust this to increase it further if it won't be communicating too often with clients, but otherwise I default on the err side for people in lower MTUs (Tailscale for example uses MTU 1280, and this also applies to the "exit node" function a lot of people use. Same thing for those who use Cloudflare's WARP tunnel), since tcp_mtu_probing should increase it as long as the connection stays open.

The rmem/wmem values are what I want to benchmark further and what I might tweak the most often depending on server/clients location and, of course, the bandwidth speed I might have available on said server. But those work fine at the moment for me as a base where latency is in a normal ~100ms range, either for high-bandwidth file serving to another continent, for streaming HD video via UDP, and for my VPNs.

dedipromo

There are some TCP tuning tools floating around in Chinese communities; here is one that I use occasionally (use your browser's translator).

https://omnitt.com/

Apparently there's no universal settings. You'll have to tweak based on your port settings and use cases.

Also found a one-click script though I haven't used myself.

https://github.com/BlackSheep-cry/TCP-Optimization-Tool

muffin

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

Astro

Thanks guys!

Also does anyone know how to use spare RAM to make writing to HDDs faster? As in caching using some ram. RAID0 is not cutting it on some servers.

JeDaYoshi

@muffin said:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

That's depressing, and everytime you do this, a @yoursunny dies.

senos

@Astro said:
Thanks guys!

Also does anyone know how to use spare RAM to make writing to HDDs faster? As in caching using some ram. RAID0 is not cutting it on some servers.

Block RAM Disk

Frameworks

I'm actually using this: https://github.com/k4yt3x/sysctl/blob/master/sysctl.conf

vicaya

I use https://tuned-project.org/ for tweaking host profiles.

senos

@vicaya said:
I use https://tuned-project.org/ for tweaking host profiles.

Same choice

It's installed by default on the Linux distro I'm using

mw

@vicaya said:
I use https://tuned-project.org/ for tweaking host profiles.

this + custom profile per box using throughput-performance as base

spoontie

@muffin said:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

"I NEED NAT! FUCK IPV6"

Mik3y326

@Astro said:
Thanks guys!

Also does anyone know how to use spare RAM to make writing to HDDs faster? As in caching using some ram. RAID0 is not cutting it on some servers.

I did try that, trust me mate, aint worth the effort. Switch to ZFS, mind your CPU and RAM usage, tweak ZFS for your usecase and enjoy.

Kolestor

@muffin said:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

LOL how is that a good tweaking! anyway I hate IPv6

emgh

@muffin said:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

Can recommend

darkimmortal

kernel.unprivileged_userns_clone=0

I’ve lost count of the amount of CVEs having this disabled made me immune to

darkimmortal

@Astro said:
Thanks guys!

Also does anyone know how to use spare RAM to make writing to HDDs faster? As in caching using some ram. RAID0 is not cutting it on some servers.

Play with vm.dirty_ratio and vm.dirty_background_ratio and increase commit time (fs-dependent)

MannDude

I'm actually working on a sysctl.conf configurator for IncogNET. Can either select from a default VPS plans or enter custom values, select the purpose of the server and add some random additional features.

It works, but needs more testing.

On all my personal stuff, I usually swap the kernel for Xanmod and do some various sysctl tweaks.

raindog308

@darkimmortal said: I’ve lost count of the amount of CVEs having this disabled made me immune to

I just googled on this and you're right. Wow.

maverick

@darkimmortal said:
kernel.unprivileged_userns_clone=0

I’ve lost count of the amount of CVEs having this disabled made me immune to

And in 6.15 (yet unreleased) linux kernel that tunable seems to be completely gone:

# sysctl kernel.unprivileged_userns_clone
sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory

I wonder why.

nullnothere

@maverick said: I wonder why.

See: user.max_user_namespaces

cmeerw

@maverick said:

@darkimmortal said:
kernel.unprivileged_userns_clone=0

I’ve lost count of the amount of CVEs having this disabled made me immune to

And in 6.15 (yet unreleased) linux kernel that tunable seems to be completely gone:

# sysctl kernel.unprivileged_userns_clone
sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory

I wonder why.

Because it's a Ubuntu/Debian specific patch that adds that tunable, so you won't find it in a mainline kernel.

maverick

@cmeerw said:

@maverick said:

@darkimmortal said:
kernel.unprivileged_userns_clone=0

I’ve lost count of the amount of CVEs having this disabled made me immune to

And in 6.15 (yet unreleased) linux kernel that tunable seems to be completely gone:

# sysctl kernel.unprivileged_userns_clone
sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory

I wonder why.

Because it's a Ubuntu/Debian specific patch that adds that tunable, so you won't find it in a mainline kernel.

Oh, sh*t... didn't see that coming.

Thanks for the info!

jperkins

Good thread. Gotta admit the only ones I used consistently is to disable ipv6.

DataRecovery

@muffin said:

net.ipv6.conf.all.disable_ipv6 = 1

@jperkins said:
Good thread. Gotta admit the only ones I used consistently is to disable ipv6.

The thread of sanity