New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Problems on Kimsufi / Routing Hickups
I was one of the lucky ones to catch a BF Kimsufi server. Unfortunatly the routing seems to be broken somewhat so i'm asking for help as i've not used Kimsufi / OVH for quite a while:
Let me quote my interfaces:
auto lo
iface lo inet loopback
iface lo inet6 loopback
auto eno1
iface eno1 inet static
address 188.165.192.X/24
broadcast 188.165.192.255
gateway 188.165.192.254
iface eno1 inet6 static
address 2001:41d0:2:8XXX::1/64
gateway 2001:41d0:2:8Xff:ff:ff:ff:ff
System has been setup deboostrapped (clear install). Pings feel SUPER laggy. Until the first pings answer / ipv6 works takes super long until IPv6 starts working. IPv4 is a bit better but still feels "laggy". When i have a permanent ping run in the background it becomes a bit better but nothing like it should be:
root@test:~# time ping heise.de
PING heise.de(redirector.heise.de (2a02:2e0:3fe:1001:302::)) 56 data bytes
64 bytes from redirector.heise.de (2a02:2e0:3fe:1001:302::): icmp_seq=1 ttl=54 time=8.53 ms
64 bytes from redirector.heise.de (2a02:2e0:3fe:1001:302::): icmp_seq=2 ttl=54 time=8.55 ms
64 bytes from redirector.heise.de (2a02:2e0:3fe:1001:302::): icmp_seq=3 ttl=54 time=8.55 ms
^C
--- heise.de ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 3040ms
rtt min/avg/max/mdev = 8.530/8.544/8.552/0.009 ms
real 0m4,480s
user 0m0,000s
sys 0m0,003s
Here is an even better example. 1 ping goes out in 5!! seconds:
root@test:~# time ping heise.de -4
PING (193.99.144.80) 56(84) bytes of data.
64 bytes from redirector.heise.de (193.99.144.80): icmp_seq=1 ttl=243 time=8.69 ms
^C64 bytes from 193.99.144.80: icmp_seq=2 ttl=243 time=8.55 ms
--- ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 8.548/8.620/8.692/0.072 ms
real 0m5,176s
user 0m0,004s
sys 0m0,000s
Comments
There's nothing abnormal in having 8ms between France and Germany.
If the first ping reply takes too long to your taste, try and ping the IP rather than the hostname, you'll save some (milli)seconds waiting for dns resolution.
Nah not really. This is no normal behavior. The problem is not dns related so it makes no difference if i use the IP instead. I was not talking about the RTT, i was talking about until the routing works and the first pings leave the server.
Update: Okay, the problem was on my side actually. I've been using Cloudflare as DNS and that indeed caused massive problems. I have no clue whatsoever. Could anyone of you please check with Cloudflare dns (1.1.1.1) if you experience the same problems?
Problems were solved once i switched back to Quad9 DNS (9.9.9.9)
For what it's worth, from my KS-LE, Cloudflare (1.1.1.1 replies faster to my queries (via 'dig') than 9.9.9.9.
Update 2: The problem was indeed related to DNS. Same problems now with Quad9. I'll try to record a video and maybe that will make it more clear.
And what about if you don't rely on DNS at all for your ping tests? that is, using only the IP of the server you want to ping?
Completely discarding DNS and just pinging IPs (IPv6 and IPv4) is working intended. The question is, why does DNS resolving not work on this host. It works on a SYS server perfectly fine and on all my Hetzner servers aswell. So it must be related to this server.
TCPDump while trying to ping "golem.de". Until the first ping went out it took 6 !! seconds.
@fLoo can you open up a mtr for 5 minutes and post it here? I.e.;
apt install mtr -y
mtr golem.de
Ofcourse. But again, when i try to ping the IP of "golem.de" there is no problem. So its definitly related to my DNS queries beeing blocked by either DNS providers (Quad9, CF..) or throttled by OVH.
Here is your MTR:
what do you have in /etc/resolv.conf ?
Ok, I tried to query 9.9.9.10 from my KS-LE and the first time it tooks several seconds to get an answer.
do you want to try OVH's DNS? 213.186.33.99.
Please do this. Use the 9.9.9.10 DNS. Ping a domain (golem.de). Wait a few seconds. Ping (heise.de). Wait a few seconds. Ping (golem.de) again. You should see a pattern that no matter what the resolv takes ages.
This is limited to OVH, i have no problems from Hetzner using the exact same servers.
OVH DNS works without a problem. Google DNS works aswell. Cloudflare and Quad9 do not.
Shitty peering? reply packets too large?
@ Everyone watching this thread:
If you have a Kimsufi LE server, please set your DNS (resolf.conf) to 9.9.9.10 (Quad9 DNS) and ping "heise.de" (german IT newspaper) for like 30 seconds/times. After 24 times/queries resolving gets stuck and it takes SECONDS to resume. Here is a TCPDump, i'll mark the "lags" with "##":
FYI: "ping" usually fires off a ping once a seconds which means that the pinged domain gets resolved once per second aswell (thats how ping is supposed to work):
Okay so i extended the tests. Its easy to test this.
time ping heise.deIf the issued pings equal to the seconds passed everything is okay.
Not Working:
Working:
If you want to help me and confirm my findings with your Kimsufi-LE you can use this resolf.conf.
Again: This problem only persists on my new Kimsufi server. SoYouStart and Hetzner are fine with all hosts.
Update: Confirming the problem using Rescue and file a ticket. Server is not usable for me.
Out of curiosity, your SYS server on which you said it works perfectly fine, is it located in the same or a different datacenter than the Kimsufi?
Final Result via Rescue Mode:
IPv4: 60 Pings - 59.131 seconds
IPv6: 60 Pings - 71.190 seconds.
Houston, Kimsufi has some serious IPv6 Problems.
Install
unboundand use 127.0.0.1 as your DNS. Sure it is a networking hiccup at OVH (probably some UDP DDoS filter is poorly tuned), but to say that a server is "not usable" because of not being able to use a random third-party DNS service, is going way too far.On a dedicated server of all things, do not rely on others for your resolving (and for your tracking and for your logging of all your requests) -- just run your own.
that is indeed strange. Could be a lot of stuff, @ninzo59 could you have this checked out? i can confirm this happening on my ks-le
9.9.9.9 is probably not the best resolve to use anyway sometime ago when I was debugging a issue for a friend on his server where network seem to hang for a sec or so randomly in the end I traced it done to lookup to 9.9.9.9 would randomly timeout or hang for a few sec after switching to his server to 8.8.8.8 never heard any issues about it again.
As i said. Its not related to 9.9.9.9 anymore. It seemed so but this was until i turned off IPv6 on my KS-LE. Also, same thing happens with rescue image without touching the network config at all. 9.9.9.9 is a very reliable dns resolver we're using in our corporate network for ages now.
This problem turns out to be a routing / ipv6 issue on OVH side. And yes. Server is unusable if IPv6 has such hickups.
Had similar issues with a Kimsufi Box in BHS.
IPv6 pings took super long and my IPv6 connection even dropped randomly.
No problems with SYS in SBG however
Do not see how IPv6 is related here. The first two servers in your
resolv.confare IPv4. It gets to ones further down only in case when the first two fail to respond. And if they fail like that, then that was very much an IPv4 problem, not IPv6.And stop using ping of unrelated host to diagnose DNS issues ffs. "Time until first ping" is exactly where it waits for DNS to resolve (or time out) the reverse DNS record. If there's a high delay at that stage, then it is a DNS issue. And use the proper tools to diagnose DNS, specifically
nslookupanddig.time nslookup heise.de 9.9.9.9and run 50 times or such.Your answer makes no sense whatsoever if you didnt even understand what i've been testing with the ping command after all. Sad that you're trying to act like a professional while you're making a fool out of yourself.
For OVH/SYS/KS IPv6, you should add prefixlength as /56 instead of /64
So going back to your previous post:
That's still a DNS problem, you disagree? I just wonder why not diagnose DNS directly, as opposed to via ping, which only gets delayed due to its DNS queries as a side-effect.
Because it is not a DNS Problem. Querying DNS via IPv6 just revealed the root cause of the problems. IPv6 connectivity. I was using "ping" because it has one advantage for me checking for the root cause here:
Because ping queries a domain for each ping, i could check DNS issues and routing issues at the same time with tcpdump in the background.
On my new KS-LE server, IPv6 just keeps dropping. For example: Restart your KS-LE and issue IPv6 pings against a target. You'll see that it takes ~ 5-10 seconds for OVH routers to build routes so packets start to flow.
@ninzo59 Could you please check? My Ticket: 3444223