Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Is this packet loss from Reliablesite acceptable?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Is this packet loss from Reliablesite acceptable?

Their exact words to me were: We cannot find any major packet loss issue in ping.pe. Please verify it again from your end and if the issue persists, we can check on it with our network team.

Here is a peak time graph: https://imgur.com/a/aggEXFG
Here is an off-peak graph: https://imgur.com/a/cS14zHB

For us, it's pretty much unusable and believe it should be 0% excluding China. Do you think this is acceptable?

Over the past week or so, we've been having some crazy packet loss on one of our Reliablesite servers. It's been back and forth with their support denying there is any loss even when presented of evidence using ping.pe or mtrs (I provided all this in my original ticket). It seems to be congestion related as during peak times it gets worse, however our node has only reached maximum speed of In: 18.14Mbps Out: 93.67Mbps during that week.

Comments

  • tinokuntinokun Member
    edited January 9

    Thanks for contacting (Un)ReliableSite support, your packet loss has been doubled!

  • blorgedblorged Member
    edited January 9

    obviously it's ridiculous for you to demand your cheap hosting provider ensure none of the other links on the Internet - except china, as you graciously excluded - have any packet loss, yes.

    if I was a hosting provider, I would make my customers do a quiz before I let them give me money.

    Thanked by 1ralf
  • show us MTRs too?

  • ralfralf Member

    Maybe read this thread, the answer to most of your questions are in there: https://lowendtalk.com/discussion/201628/providers-in-netherlands-that-dont-use-upstreams-as1299-neither-as174/p1

    TLDR though:

    • ping uses ICMP and many routers will only forward and respond to these if there's nothing better for them to be doing. Any routers that are close to link capacity on any route will probably drop those packets.
    • ping therefore has little bearing on what actual data might do.
    • if you care about point-to-point, set up a wireguard network between those two hosts and run ping or mtr across the wireguard network. This will give you an indication of packet loss for real-world data.

    Final thought, if you're using TCP then it can be extremely sensitive to packet loss in high latency situations. If you're measuring throughput over TCP, you can easily drop from 1Gbps to 100Mbps or worse between Europe and the US with just a few lost packets even though the link as a whole is mostly reliable. There are lots of kernel settings you can change to increase the send buffers to mitigate this problem, but the default settings aren't really ideal for high-latency and high-throughput networks if there's even occasional packet loss.

  • @nanankcornering said:
    show us MTRs too?

    Sure: https://imgur.com/a/35wRUrb

    Another one but text only I took earlier.

    -> 8.8.8.8 (8.8.8.8) 2025-01-09T15:50:56+0000
    Keys: Help Display mode Restart statistics Order of fields quit
    Packets Pings
    Host Loss% Snt Last Avg Best Wrst StDev
    1. redacted 1.7% 117 0.7 1.5 0.2 13.4 2.3
    2. et-0-0-61.cr4-lax2.ip4.gtt.net 0.0% 117 0.4 2.8 0.2 29.0 5.1
    3. as15169.cr4-lax2.ip4.gtt.net 3.4% 117 1.7 2.1 0.3 19.5 3.4
    72.14.194.163
    4. 142.251.226.193 0.9% 117 0.4 0.4 0.3 1.6 0.2
    216.239.57.29
    5. 142.251.60.111 0.9% 117 0.3 0.4 0.3 1.1 0.1
    6. dns.google 3.4% 116 0.3 10.2 0.3 583.8 74.0

    @ralf said:
    Maybe read this thread, the answer to most of your questions are in there: https://lowendtalk.com/discussion/201628/providers-in-netherlands-that-dont-use-upstreams-as1299-neither-as174/p1

    TLDR though:

    • ping uses ICMP and many routers will only forward and respond to these if there's nothing better for them to be doing. Any routers that are close to link capacity on any route will probably drop those packets.
    • ping therefore has little bearing on what actual data might do.
    • if you care about point-to-point, set up a wireguard network between those two hosts and run ping or mtr across the wireguard network. This will give you an indication of packet loss for real-world data.

    Final thought, if you're using TCP then it can be extremely sensitive to packet loss in high latency situations. If you're measuring throughput over TCP, you can easily drop from 1Gbps to 100Mbps or worse between Europe and the US with just a few lost packets even though the link as a whole is mostly reliable. There are lots of kernel settings you can change to increase the send buffers to mitigate this problem, but the default settings aren't really ideal for high-latency and high-throughput networks if there's even occasional packet loss.

    This isn't a configuration issue. We don't have other servers with these issues (including others at RS) and they're all deployed the same way. It's something related to the nic, fiber, switch etc. You don't need kernel optimizations for 100mbps of data.

  • edited January 9

    yeah that is weird. and you said "one of our reliablesite servers", does that mean the others are okay?

    if it is, sounds like a nic/cable issue to me.

    what i'll do is do an MTR/ping test to other servers locally within the /24 block to isolate the issue.

    Thanked by 1yoursunny
  • Yes, other servers in the same location are fine: https://imgur.com/a/AzPAQZS

    No idea if it's the same rack or not. The other week there were total network outages on the affected server, but they said it was due to a network configuration change.

  • ralfralf Member

    @pedrotski said:

    @ralf said:
    Maybe read this thread, the answer to most of your questions are in there: https://lowendtalk.com/discussion/201628/providers-in-netherlands-that-dont-use-upstreams-as1299-neither-as174/p1

    TLDR though:

    • ping uses ICMP and many routers will only forward and respond to these if there's nothing better for them to be doing. Any routers that are close to link capacity on any route will probably drop those packets.
    • ping therefore has little bearing on what actual data might do.
    • if you care about point-to-point, set up a wireguard network between those two hosts and run ping or mtr across the wireguard network. This will give you an indication of packet loss for real-world data.

    Final thought, if you're using TCP then it can be extremely sensitive to packet loss in high latency situations. If you're measuring throughput over TCP, you can easily drop from 1Gbps to 100Mbps or worse between Europe and the US with just a few lost packets even though the link as a whole is mostly reliable. There are lots of kernel settings you can change to increase the send buffers to mitigate this problem, but the default settings aren't really ideal for high-latency and high-throughput networks if there's even occasional packet loss.

    This isn't a configuration issue. We don't have other servers with these issues (including others at RS) and they're all deployed the same way. It's something related to the nic, fiber, switch etc. You don't need kernel optimizations for 100mbps of data.

    The fact that you replied so confidently that it absolutely cannot be a configuration issue after barely enough time to skim read my comment, let alone the thread I suggested you read where basically the exact same questions were asked, suggests to me that you haven't actually even considered trying the thing I suggested or looking to read the advice for the other person in the exact same situation as you.

    I agree, it might be the fault with your NIC, your switch or your cabling. The odds are, though, that it probably isn't.

    And for your information, the stuff I said about TCP and kernel limits is true. It's not "kernel optimisations", it's changing the settings from the defaults that have been the defaults since the times when a couple of bonded-T1s were considered fast. If you have any packet loss at all, it can make a massive difference to your throughput.

    But by all means, feel free to ignore that information. It sounds like you don't really want help, you just want people to agree that it's your provider's fault.

  • edited January 9

    @ralf said: I agree, it might be the fault with your NIC, your switch or your cabling. The odds are, though, that it probably isn't.

    basic icmp towards google's dot8/cloudflare's dot1/quad9 should not result in packet loss.

    yes i agree, (the path) towards those services might show packet loss in mtr, especially cogent/he which limits icmp. even juniper mx/ex doesn't prioritize icmp packets (would show latency jumps)

    if destination shows packet loss, then something is not right on either ends.

  • lukast__lukast__ Member, Megathread Squad

    @nanankcornering said: basic icmp towards google's dot8/cloudflare's dot1/quad9 should not result in packet loss.

    8.8.8.8 does deprioritize/sometimes drop ICMP packets, see https://groups.google.com/g/public-dns-discuss/c/p1o62SJElck/m/w0flYsmqBQAJ:

    While Google does not block ICMP or random UDP to 8.8.8.8 or other Google Public DNS IP addresses, there are rate limits on ICMP error replies from Google networking equipment, and ICMP error replies may be de-prioritized within Google networks and could be more likely to be dropped.

  • edited January 9

    @lukast__ said: 8.8.8.8 does deprioritize/sometimes drop ICMP packets, see https://groups.google.com/g/public-dns-discuss/c/p1o62SJElck/m/w0flYsmqBQAJ:

    it makes sense if you're sending tons of icmp packets within a small period of time.

    had a similar issue with @pedrotski and had a call with my provider. apparently it was an issue with LACP within their core/agg switches, which caused packet loss towards random IPs.

  • I had a similar issue in the past with them, a ticket resulted in getting the faulty nic replaced within 30 minutes and all back to fully operational.

    So definitely contact them. And be nice, if you're a d1ck to them - they will treat you as one.

  • emg88emg88 Member
    edited January 10

    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

  • @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    switch providers

    Thanked by 2yoursunny emg88
  • yoursunnyyoursunny Member, IPv6 Advocate

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

  • @yoursunny said:

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

    how many push-ups for chargeback?

    Thanked by 1yoursunny
  • yoursunnyyoursunny Member, IPv6 Advocate

    @ethanblake87 said:

    @yoursunny said:

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

    how many push-ups for chargeback?

    he injured his shoulder now so no pushup - @cybertech

  • @yoursunny said:

    @ethanblake87 said:

    @yoursunny said:

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

    how many push-ups for chargeback?

    he injured his shoulder now so no pushup - @cybertech

    they injured their shoulder now so no pushup

  • @yoursunny said:

    @ethanblake87 said:

    @yoursunny said:

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

    how many push-ups for chargeback?

    he injured his shoulder now so no pushup - @cybertech

    The end of the world is here, no need to do push-ups anymore.. :(

  • yoursunnyyoursunny Member, IPv6 Advocate

    @cybertech said:

    @yoursunny said:

    @ethanblake87 said:

    @yoursunny said:

    @emg88 said:
    Having this same problem with a different provider. What can you do if a ticket doesn't resolve?

    Chargeback yesterday - @cybertech

    how many push-ups for chargeback?

    he injured his shoulder now so no pushup - @cybertech

    they injured their shoulder now so no pushup

    I am slightly concerned about how well you know about him - @FAT32

    Thanked by 1FAT32
Sign In or Register to comment.