Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


OVH IPv6 downlink rate limiting
New on LowEndTalk? Please Register and read our Community Rules.

OVH IPv6 downlink rate limiting

jordynegen11jordynegen11 Member
edited July 8 in General

Hi everyone,

In this discussion I want to inform you guys about a recent issue I had that turned out to be a global issue at OVH.

I had the wonderful idea to setup some GRE tunnels over IPv6 to avoid mitigation issues on the IPv4 network of OVH. I noticed that the downlink on the IPv6 network was limited somehow. Via IPerf wasn't able to receive more then 100Mb/s from a single IP.

[email protected]:~# iperf3 -c 2a01:4f8:251:XXXX::X -u -b 1000M -R
Connecting to host 2a01:4f8:251:XXXX::X, port 5201
Reverse mode, remote host 2a01:4f8:251:XXXX::X is sending
[ 4] local 2001:41d0:700:XXXX::X port 43539 connected to 2a01:4f8:251:XXXX::X port 5201
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-1.00 sec 29.9 MBytes 250 Mbits/sec 0.089 ms 10979/14801 (74%)
[ 4] 1.00-2.00 sec 11.4 MBytes 95.6 Mbits/sec 0.076 ms 12868/14326 (90%)
[ 4] 2.00-3.00 sec 11.4 MBytes 95.5 Mbits/sec 0.110 ms 12865/14322 (90%)
[ 4] 3.00-4.00 sec 11.4 MBytes 95.5 Mbits/sec 0.105 ms 12868/14325 (90%)
[ 4] 4.00-5.00 sec 11.4 MBytes 95.5 Mbits/sec 0.063 ms 12869/14326 (90%)
[ 4] 5.00-6.00 sec 11.4 MBytes 95.6 Mbits/sec 0.104 ms 12867/14325 (90%)
[ 4] 6.00-7.00 sec 11.4 MBytes 95.5 Mbits/sec 0.075 ms 12871/14328 (90%)
[ 4] 7.00-8.00 sec 11.4 MBytes 95.5 Mbits/sec 0.116 ms 12870/14328 (90%)
[ 4] 8.00-9.00 sec 11.4 MBytes 95.5 Mbits/sec 0.094 ms 12871/14328 (90%)
[ 4] 9.00-10.00 sec 11.4 MBytes 95.5 Mbits/sec 0.085 ms 12872/14329 (90%)


[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec 0.085 ms 126800/143738 (88%)
[ 4] Sent 143738 datagrams
iperf Done.

The first ticket I opened, OVH blamed my configuration somehow but after multiple tickets and months of waiting (support response time is useless) OVH found out that it was caused by a global backbone policy. OVH does not have this limitation on their IPv4 network btw.

OVH is rate limiting their Ipv6 network to prevent attacks. Which make sense but I did pay for a full 3Gb/s on our servers, also on the IPv6 network. So I've read the contract I accepted/signed and OVH is allowed to activate mitigation during an attack, but a 24/7 IPv6 rate-limit is not mitigation of course. Also this limitation is nowhere to be found on their website.

4 weeks have past since that reaction and OVH still refuses to remove this limitation or inform their customers about it. It's still not listed on their website or in a new contract.

This is a warning for you all. If you want to use the OVH IPv6 network, you can better go somewhere else. This what you get if you go cheap I guess..

@OVH_APAC @OVHcloud_james

Thanked by 1Shot2

Comments

  • jackbjackb Member, Provider
    edited July 8

    Are you sure it's because it's ipv6 and not because you're using UDP?

    Even going back about 10 years OVH had rate limited UDP per source address.

  • yoursunnyyoursunny Member, IPv6 Advocate

    10Mbps is more than enough for most applications.
    You have 95Mbps.
    No need to PMS over it.

  • jordynegen11jordynegen11 Member
    edited July 8

    @jackb said:
    Are you sure it's because it's ipv6 and not because you're using UDP?

    Even going back about 10 years OVH has rate limited UDP per source address.

    Thanks but I have the same issues when using TCP. I'm dealing with it for over 6 months now and did all tests possible. OVH made it very clear. Their IPv6 is rate-limited.

  • jordynegen11jordynegen11 Member
    edited July 8

    @yoursunny said:
    10Mbps is more than enough for most applications.
    You have 95Mbps.
    No need to PMS over it.

    Sorry bro B) but not enough for my use case. I even pay extra for the 3 gbit guaranteed bandwidth.

  • skorupionskorupion Member

    CALL THEM FFS. One thing about OVH is that you won't have shit done if you won't call them. Put in 3 bucks worth of credit into skype and call them.

  • jordynegen11jordynegen11 Member
    edited July 8

    @skorupion said:
    CALL THEM FFS. One thing about OVH is that you won't have shit done if you won't call them. Put in 3 bucks worth of credit into skype and call them.

    I think I called them 10 times about this already. Every time they promise me to "answer the ticket today". But it's more like "mabe this week". They also did refuse to redirect me to a manager or someone in charge after waiting for 5 weeks on a response.

    I even pay for premium support btw

  • vovlervovler Member

    Well if you paid you for, you should be able to use it be it IPv4 or IPv6

  • jordynegen11jordynegen11 Member
    edited July 9

    Ok little update here.

    OVH was willing to give me a compensation for the fact IPv6 is rate-limited on all my servers. They offered by a 25% discount on my next bill. I was actually surprised that OVH cared and was ok with it.

    Turns out it was only a 25% discount for 1 server. I have 40 servers at OVH and yes my plan was to use IPv6 on all those servers. So in the end that 6500 euro bill had a "nice" 70!!! euro discount, for all the trouble ofcourse.

    Nevermind, they don't care at all.


    Of course @OVH_APAC was online today and ignored this. :)

  • Shot2Shot2 Member

    Basically, OVH does not care about IPv6. No wonder they blindly rate-limit it, because... who cares.

  • Smith42Smith42 Member

    Honestly if it has taken more than 1 month for your case, don't waste your time any more. Find another host that can cover your use case. I was in the same shit as you at one point and paid 24k euro/month for servers I couldn't properly use for 6 months while they deliberated on my case. Issue is OVH has all these different departments: Sales, Billing, Tech, Network, Abuse, Devops etc. and they don't get along with each other at all. So in your case as well, the Tech team is probably not able to get the Network team to do anything for you at all, so they just went to the Sales team and got a commercial gesture for you, hoping you'd STFU.

    This was my issue with them a while back https://www.lowendtalk.com/discussion/158739/ovh-double-bandwidth-guarantee-no-longer-guaranteed

    I moved 22k euro worth of servers and only pay 14k euro at Hetzner for similar configurations, with overall a more pleasant experience.

    TLDR; Move on from OVH

  • jordynegen11jordynegen11 Member
    edited July 9

    @Smith42 said:
    Honestly if it has taken more than 1 month for your case, don't waste your time any more. Find another host that can cover your use case. I was in the same shit as you at one point and paid 24k euro/month for servers I couldn't properly use for 6 months while they deliberated on my case. Issue is OVH has all these different departments: Sales, Billing, Tech, Network, Abuse, Devops etc. and they don't get along with each other at all. So in your case as well, the Tech team is probably not able to get the Network team to do anything for you at all, so they just went to the Sales team and got a commercial gesture for you, hoping you'd STFU.

    This was my issue with them a while back https://www.lowendtalk.com/discussion/158739/ovh-double-bandwidth-guarantee-no-longer-guaranteed

    I moved 22k euro worth of servers and only pay 14k euro at Hetzner for similar configurations, with overall a more pleasant experience.

    TLDR; Move on from OVH

    I would love to use Hetzner but they just don't have as good ddos protection as OVH does. Also no locations in the US, Singapore and australia. Don't get me wrong, besides their DDOS protection I really enjoy my experience with hetzner. We already have a few servers there.

    I am running OK at OVH. I host for over 10 years now at OVH. The price is really good but the support is the worst I have ever seen in my life.

    Any other healty company would fight to keep customer like me. Especially when breaking contract. Its a real shame they just don't care at OVH, even when you have 40 servers from them.

    @OVH_APAC @ninzo59 I can post a whole story here about the OVH support and my experience in the past, but fact is it's just really really bad. This is not my first issue with your support. Seriously? Breaking contract and offer a 70 euro discount on a 6K bill as "compensation"? I think we all want an explanation or response to that.

    Thanked by 1pike
  • rm_rm_ Member
    edited July 10

    @yoursunny said: 10Mbps is more than enough for most applications.
    You have 95Mbps.
    No need to PMS over it.

    From other posts you seem to be a smart guy, don't force yourself too hard to be funny. Unless you were serious, in which case I am mistaken about #1.

    @jordynegen11 said: Thanks but I have the same issues when using TCP

    I don't see the same limit, just was able to download 600-700 Mbit on IPv6 TCP. Can you cite the actual result of iperf3 -c iperf6.online.net -p 5203 -R? (change 5203 between 5200...5209 if it says busy).

    On UDP yes, only got ~120 Mbit.

    Sure you wouldn't run VPN over TCP, so what remains to check is if the actual GRE is also limited (like UDP), or not (like TCP). Chances are it's not, because UDP is often used to DDoS, but GRE not so much.

    Thanked by 1dosai
  • yoursunnyyoursunny Member, IPv6 Advocate

    @rm_ said:

    @yoursunny said: 10Mbps is more than enough for most applications.
    You have 95Mbps.
    No need to PMS over it.

    From other posts you seem to be a smart guy, don't force yourself too hard to be funny. Unless you were serious, in which case I am mistaken about #1.

    I want to be funny person 😔
    I want troll tag 😈

    @jordynegen11 said: Thanks but I have the same issues when using TCP

    I don't see the same limit, just was able to download 600-700 Mbit on IPv6 TCP. Can you cite the actual result of iperf3 -c iperf6.online.net -p 5203 -R? (change 5203 between 5200...5209 if it says busy).

    On UDP yes, only got ~120 Mbit.

    Evolution Host in Roubaix France, which is in OVH network.
    Same speed on IPv6 and IPv4 UDP.

    [email protected]:~$ iperf3 -c iperf6.online.net -p 5203 -R
    Connecting to host iperf6.online.net, port 5203
    Reverse mode, remote host iperf6.online.net is sending
    [  5] local 2001:41d0:203:a07a:: port 50608 connected to 2001:bc8:1:: port 5203
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.07   sec  99.2 MBytes   777 Mbits/sec                  
    [  5]   1.07-2.08   sec  83.8 MBytes   693 Mbits/sec                  
    [  5]   2.08-3.00   sec  99.2 MBytes   908 Mbits/sec                  
    [  5]   3.00-4.00   sec   112 MBytes   941 Mbits/sec                  
    [  5]   4.00-5.00   sec   125 MBytes  1.05 Gbits/sec                  
    [  5]   5.00-6.00   sec  95.8 MBytes   804 Mbits/sec                  
    [  5]   6.00-7.00   sec   114 MBytes   953 Mbits/sec                  
    [  5]   7.00-8.62   sec   104 MBytes   541 Mbits/sec                  
    [  5]   8.62-9.57   sec  74.5 MBytes   659 Mbits/sec                  
    [  5]   9.57-10.56  sec  96.7 MBytes   819 Mbits/sec                  
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.56  sec  1017 MBytes   808 Mbits/sec  5367             sender
    [  5]   0.00-10.56  sec  1004 MBytes   798 Mbits/sec                  receiver
    
    iperf Done.
    [email protected]:~$ iperf3 -c iperf.online.net -p 5203 -R
    Connecting to host iperf.online.net, port 5203
    Reverse mode, remote host iperf.online.net is sending
    [  5] local 5.135.37.0 port 35422 connected to 62.210.18.0 port 5203
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.06   sec   121 MBytes   958 Mbits/sec                  
    [  5]   1.06-2.08   sec  83.6 MBytes   686 Mbits/sec                  
    [  5]   2.08-3.00   sec   102 MBytes   932 Mbits/sec                  
    [  5]   3.00-4.00   sec   110 MBytes   919 Mbits/sec                  
    [  5]   4.00-5.00   sec   110 MBytes   920 Mbits/sec                  
    [  5]   5.00-6.63   sec   115 MBytes   590 Mbits/sec                  
    [  5]   6.63-7.49   sec  84.3 MBytes   823 Mbits/sec                  
    [  5]   7.49-8.37   sec  78.2 MBytes   745 Mbits/sec                  
    [  5]   8.37-9.35   sec  86.6 MBytes   741 Mbits/sec                  
    [  5]   9.35-10.26  sec  67.5 MBytes   619 Mbits/sec                  
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.26  sec   969 MBytes   792 Mbits/sec  4080             sender
    [  5]   0.00-10.26  sec   957 MBytes   782 Mbits/sec                  receiver
    
    iperf Done.
    
  • jordynegen11jordynegen11 Member
    edited July 10

    Thanks for both your answers @rm_ @yoursunny.

    It seems that OVH has removed the TCP limitation (last time I checked TCP was 6 months ago) but the UDP 100Mb/s limit still exist as @rm_ mentioned. And yes, UDP is a big deal for me.

    For example: an alternatieve like wireguard over IPv6 is also not possible becuase of the UDP limitation, since wireguard uses UDP :smiley:. TCP would give you a bad time.

    I already tested it before but I ran the tests again to show you guys the performance of the GRE protocol over IPV6: (Spoiler: Its even worse)

    Configuration

    Server A (Outside OVH network): 
    sudo ip tunnel add tun333 mode ip6gre local 2a01:4f8:xxxx:xxxx::2 remote 2001:41d0:xxxx:xxxx::2 ttl 255                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             5
    sudo ip addr add 10.50.1.2/30 dev tun333 
    sudo ip link set tun333 up
    
    Server B (OVH server): 
    sudo ip tunnel add tun333 mode ip6gre local 2001:41d0:xxxx:xxxx::2 remote 2a01:4f8:xxxx:xxxx::2 ttl 255                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             5
    sudo ip addr add 10.50.1.1/30 dev tun333 
    sudo ip link set tun333 up
    

    Result TCP packets over GRE (IPv6)

    [email protected]:~# iperf3 -c 10.50.1.2  -R
    Connecting to host 10.50.1.2, port 5201
    Reverse mode, remote host 10.50.1.2 is sending
    [  5] local 10.50.1.1 port 55370 connected to 10.50.1.2 port 5201
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.00   sec   155 KBytes  1.27 Mbits/sec
    [  5]   1.00-2.00   sec   117 KBytes   960 Kbits/sec
    [  5]   2.00-3.00   sec   138 KBytes  1.13 Mbits/sec
    [  5]   3.00-4.00   sec   115 KBytes   938 Kbits/sec
    [  5]   4.00-5.00   sec   143 KBytes  1.17 Mbits/sec
    [  5]   5.00-6.00   sec   145 KBytes  1.18 Mbits/sec
    [  5]   6.00-7.00   sec   104 KBytes   849 Kbits/sec
    [  5]   7.00-8.00   sec   150 KBytes  1.23 Mbits/sec
    [  5]   8.00-9.00   sec   120 KBytes   983 Kbits/sec
    [  5]   9.00-10.00  sec   130 KBytes  1.06 Mbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.01  sec  1.37 MBytes  1.15 Mbits/sec  235             sender
    [  5]   0.00-10.00  sec  1.28 MBytes  1.08 Mbits/sec                  receiver
    
    iperf Done.
    

    Result UDP packets over GRE (IPv6)

    [email protected]:~# iperf3 -c 10.50.1.2 -R -u -b 1000M
    Connecting to host 10.50.1.2, port 5201
    Reverse mode, remote host 10.50.1.2 is sending
    [  5] local 10.50.1.1 port 42975 connected to 10.50.1.2 port 5201
    [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
    [  5]   0.00-26.83  sec  1.36 KBytes   416 bits/sec  101003957172.665 ms  0/1 (0%)
    iperf3: error - control socket has closed unexpectedly
    

    So yeah, if you see this @ninzo59, Your IPv6 rate-limits are useless. Shame on you. Also not mentioned on the OVH website or in the contract.

  • rm_rm_ Member
    edited July 10

    @jordynegen11 said: the performance of the GRE protocol over IPV6

    Yeah, just tested myself, for whatever reason I get 724ms ping over GRE, where the direct ping is 9ms. And also a high packet loss.

    Thanked by 1jordynegen11
  • @rm_ said: I get 724ms ping over GRE, where the direct ping is 9ms. And also a high packet loss.

    Wow, that means it is unusable, I doubt this is permanent otherwise everyone would have been "PMS-ing". Maybe it is some temporary or configuration issue.

  • jordynegen11jordynegen11 Member
    edited July 11

    @Maounique said:

    @rm_ said: I get 724ms ping over GRE, where the direct ping is 9ms. And also a high packet loss.

    Wow, that means it is unusable, I doubt this is permanent otherwise everyone would have been "PMS-ing". Maybe it is some temporary or configuration issue.

    Not everyone. This is only when using GRE over IPv6. I dont think much OVH customers use that, thats why OVH simply does not care.

    Thanked by 1webcraft
  • MaouniqueMaounique Member
    edited July 11

    Hum, so they limit IPv6 and further GRE over IPv6? Because 100 mbps should not result in that ping nor high packet loss if GRE would not be limited separately and really badly (presuming IPv6 is not already saturated 200%).

  • ehhthingehhthing Member

    This is very disappointing because compared to most providers, OVH has a very diverse network.

    It's really difficult to find a network as high quality as OVH's at least in peering and transit.

  • jordynegen11jordynegen11 Member
    edited July 11

    @Maounique said:
    Hum, so they limit IPv6 and further GRE over IPv6? Because 100 mbps should not result in that ping nor high packet loss if GRE would not be limited separately and really badly (presuming IPv6 is not already saturated 200%).

    Yes they limit GRE over IPv6 even more.

    UDP over IPv6 = 100Mbit limit
    UDP over GRE over IPv6 = test will not even finish
    TCP over GRE over IPv6 = 1mbit limit

    I did not test the latency on my end. Will try that when I'm back home.

  • jordynegen11jordynegen11 Member
    edited July 11

    @ehhthing said:
    This is very disappointing because compared to most providers, OVH has a very diverse network.

    It's really difficult to find a network as high quality as OVH's at least in peering and transit.

    Yes we are also very happy about the OVH network when using IPv4 of course, but their IPv6 is just.........

    For such a big provider it's really disappointing. But what is even more disappointing is the support that just don't care at all.

  • @jordynegen11 said: but their IPv6 is just

    IMO, they know they have a big target on their back for the DDoS "community". Nulling IPv6 is a joke, so they had to do something, I just doubt this is the right approach.

  • exception0x876exception0x876 Member, Provider

    @jordynegen11 the 100Mbit/s limit applies to UDP over IPv6 only. There are other protocols that you can use for tunnels, a few of them listed here - https://developers.redhat.com/blog/2019/05/17/an-introduction-to-linux-virtual-interfaces-tunnels

  • jordynegen11jordynegen11 Member
    edited July 11

    @exception0x876 said:
    @jordynegen11 the 100Mbit/s limit applies to UDP over IPv6 only. There are other protocols that you can use for tunnels, a few of them listed here - https://developers.redhat.com/blog/2019/05/17/an-introduction-to-linux-virtual-interfaces-tunnels

    Thanks but as you can see in my previous post, alternatives like IP6GRE are even more limited. I tried almost every protocol possible on IPv6 but OVH's rate limiting is just killing it all.

    I also dont want to use something else then Wireguard or GRE since the overhead on those will be the lowest.

    Yes I could just use IPv4 (and I do now) but their mitigation is causing issues sometimes. Especially when using Wireguard.

  • exception0x876exception0x876 Member, Provider

    @jordynegen11 said:

    @exception0x876 said:
    @jordynegen11 the 100Mbit/s limit applies to UDP over IPv6 only. There are other protocols that you can use for tunnels, a few of them listed here - https://developers.redhat.com/blog/2019/05/17/an-introduction-to-linux-virtual-interfaces-tunnels

    Thanks but as you can see in my previous post, alternatives like IP6GRE are even more limited. I tried almost every protocol possible on IPv6 but OVH's rate limiting is just killing it all.

    Have you tried IPIP6 tunnel? That's what I used and the limitations didn't apply to it. You can find the instructions for setting it up in the article that I linked.

  • OVH_APACOVH_APAC Member, Provider

    Hi Jordy, we regret to hear about your frustration. We will try our best to troubleshoot your issue with the network team from our end. Dropped you a PM.

  • rm_rm_ Member

    @exception0x876 said: Have you tried IPIP6 tunnel?

    Actually this is what I tested with and had the 724ms ping mentioned above.

  • @exception0x876 said:

    @jordynegen11 said:

    @exception0x876 said:
    @jordynegen11 the 100Mbit/s limit applies to UDP over IPv6 only. There are other protocols that you can use for tunnels, a few of them listed here - https://developers.redhat.com/blog/2019/05/17/an-introduction-to-linux-virtual-interfaces-tunnels

    Thanks but as you can see in my previous post, alternatives like IP6GRE are even more limited. I tried almost every protocol possible on IPv6 but OVH's rate limiting is just killing it all.

    Have you tried IPIP6 tunnel? That's what I used and the limitations didn't apply to it. You can find the instructions for setting it up in the article that I linked.

    I am going to test that aswell thanks for the tip.

  • @OVH_APAC said:
    Hi Jordy, we regret to hear about your frustration. We will try our best to troubleshoot your issue with the network team from our end. Dropped you a PM.

    Thanks for you message.

    OVH does already know what the issue is. If you read the first message in this topic you will see the OVH support identified this problem as a result of a backbone policy on your Ipv6 network.

    We will discuss further in PM. Thanks.

Sign In or Register to comment.