New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Comments
Looks like they changed my route from HE to Cogent too
Slightly offtopic but did you/are you moving away from Royale completely or did you get the issues straightened out? @mw
Too early to tell at this stage and we're planning on diversifying fully before making the decision to outright drop them.
All I can say is there has been no effort on their end whatsoever to keep us besides offering a "free server" along with the condition that we need to tell them we will not move away to get it, which we declined as a "free server" doesn't address the core reasons we are considering moving away.
We've also been relegated from having direct access to their CEO via Discord (which we had since the beginning) to having to use their standard ticketing queue system which to us feels like they themselves have lost interest in our relationship.
It's funny, I have a couple servers on their end with 10G ports each, and all of them are currently running over 5Gbps normally.
I also ran a iperf3 test with my other one located at crunchbits and it was also quite good.
From my personal experience, at least their network shouldn't be grossly oversold and the capacity is adequate.
However - things may be different to specific areas.
I'm very curious if you try to diversify and test the speed of your network?
For example, using Speedtest, or pulling some public mirrors, etc. to test the rate issue more holistically.
Or do you have them open a new server and grant them access and you use the same test methods. This will help to reproduce any problems.
Otherwise a lot of times this is like trying here and there like a fly on the wall, but it doesn't help solve the problem.
I've done Speedtest.net, iperf to every public iperf server on Google, iperf to my own 20/40G boxes in EU, HTTP, rclone (SFTP, Google Drive, Dropbox, OneDrive)
I sent them the exact commands used with iperf3 server IPs so they could check 1:1 what they see in their own testing. I'm waiting to hear back about the strange MTU issue because it might be linked to the high TCP retransmissions I'm seeing and if they want access to my server they're welcome, but they haven't asked for that.
I got sent a screenshot saying the switch says MTU 9000
I did test jumbo frames, but anything above 1472 gets fragmented. I asked about encapsulation again, let's hope they shed some more light on that specifically.
EDIT:
Is there a better way to test this than:
ping -M do -s size IP?
Can't believe you never tuned the system as soon as you recieved the server. I always tune sysctl and get superior single thread speeds. Makes a world of difference. Purevoltages network is actually amazing in NY to APAC and EU. Their NY dc is start of the art.
its still performing poorly. run an iperf on your box to scaleway for me? i want to see something
Big shout out to PureVoltage's Jake for being such a team player. They have been extremely willing to help get to the bottom of this.
As it stands, we've tested -everything- and seem to have hit something odd which if fixed we are happy to proceed and full send.
All SSH traffic, irrespective of port or whether using mainline SSH or rclone's SSH seem to be limited to 200-250Mbps
Everything else is fast
Does anyone have any advice for things to look at to understand why this is?
I've already let PureVoltage know about this but while I wait for a response maybe one of you dweebs have seen this before?
I had a few interactions with PureVoltage's team, Jake is indeed a great guy!
So what speeds are you now able to reach, excluding SSH/rsync?
And what's your config like?
I need to tune a few 2x25Gbps servers too.
Can you try disabling a few/all ciphers to test out if there's anything going on in crypto land for SSH? There should be some quick online guides to help you select/disable some ciphers to test it out. Ciphers will impact speed/performance (depending on CPU on both sides) but they may come into play only at >10G, certainly not to limit you to 250Mbps.
I'm also curious on your kernel and the CPU (as well as the distro you're using to test things out).
When you say everything else is fast, do you mean even things like wget/curl for HTTP/S or do you mean only iperf?
Please do post your findings.
SSH/SFTP is a bit of a fun problem. Even if you use the lightest of ciphers, cracking above 150mbps with even 100ms of latency or higher is the limit, I've honestly never been able to get higher. Even with some tuning.
I'll do some digging as this topic interests me as since I live in NZ and the two dedis and the 1 host c vps I work with have latency of about 220ms and higher... Latency is definitely a factor.
I think it all depends on the in-between links.
I've had pretty good consistent throughput (not always of course) reaching 35-40 MB (so roughly 300+ Mbps) with latencies from 150ms to 250ms.
No tuning, standard vanilla SSH (rsync over SSH, Debian).
Latency does impact things but if the links are not particularly congested things are pretty stable.
I'll give that a shot this weekend but in all honesty I don't think it's an issue. Using the standard config I apply to all these servers I'm able to push 30Gbps+ over SSH using rclone without much fuss from Netherlands <-> Germany and have seen 20Gbps on US East <-> Netherlands from another provider here during our testing. It's why we have always used SSH and why the poor performance on PureVoltage is an issue in the first place.
So far I have tested HTTP using stock nginx and SSH, with the former I can hit 10Gbit using aria2c with 4 threads, while with SSH using rclone and 16 transfers with 4 connections per transfer I hit some sort of limit @ 200-250Mbps. With SCP over a single connection it also hits the same limit, leading me to believe whatever the issue is won't be fixed by throwing more concurrents.
I spoke about the speeds in the message above. I'm sure I can get near line rate with more effort too. I posted my sysctl.conf somewhere above in this thread too. Nothing exotic at all hahaha
As mentioned above, I can cook over SSH at well above 10Gbit under ideal conditions even cross continent. It's why we have always used SSH.
We just ran tests with stock SSH over a Wireguard tunnel and performance was great - so we now know for certain the slowness is due to some form of traffic management they're doing. We hope this can be resolved
I'd tend to agree - I've rarely come across any SSH issue on modern CPUs and I assume if you're on 10+Gbps links you're on a fast enough CPU anyway. Nevertheless, it's more to rule out some strange odd thing.
Was this also the case if you changed SSH ports (i.e. without WG)?
So is it the SSH port that is being shaped or is there something that is detecting SSH traffic and shaping that particular stream?
i ran ssh over port 80 and it was the same situation lol
no clue why anyone would ever shape SSH
So they have something that is doing some sort of DPI equivalent for SSH traffic?
Can you also just give a shot at some cipher changes to see if there's anything there that changes your traffic rate. Might give a clue on what is really going on.
Really puzzled.
I decided to test the same scenario with a fresh ubuntu install on a KS-GAME-LE I'm about to put up for sale.
Without WG = 80mbps average over a 1 minute test.
With WG = 20mbps average over a 1 minute test.
I expected it to be lower as I'm adding another layer of encryption ON TOP of the already encrypted traffic.
I don't think I can add much more to this so I will bow out from posting, however I am watching with great interest as to see what the root cause is and resolution.
With WG = 20mbps average over a 1 minute test.
That doesn't seem to be OK. Can you also check with a MTR to see if there's any consistent packet loss? SSH is TCP but WG is UDP so TCP on UDP should only have fragmentation issues but I'm a little surprised at the loss of througput.
Also, what is the latency between the end points you were testing?
224MS and that MTR would be clean(currently reinstalling as I am about to throw it up in Service Transfers).
Test was 10 minutes ago so 0 loss during that time.
My MTR (this is from another dedi that I have, same specs and everything) is very long to BHS but the performance is within what I'd expect. My international transport beyond as55850 (Mercury) get's really funky.
that cant be right. as for nested encapsulation, just run a speedtest (tls) on a vpn (wg) on your phone. you wouldn't expect to get less than 95% of your line rate unless your vpn was bad. same applies here
Too late unfortunately.
It's up for service transfer and I don't want to mess with any of my production dedi's with this.
I may experiment with my RO from Host-C tomorrow and I can double check.
i have a box with host-c too lemme know if u want me to double check ur tests
Did you get to the bottom of this? Any insights?
Very curious on whatever the issue is/was.
We could not and have been refunded by PureVoltage - we're currently in final testing with another provider which has performed as expected
Glad there was a resolution, unfortunate PureVoltage couldn't get to the bottom of it.
I also want to apologize I didn't get back to you when I said I would. Work has been mega busy.