All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Evaluating network quality
Hi, I am wondering if anyone can share a way to methodically determine a vendors network quality? I am hoping for a deterministic method, as opposed to opinions on LET (which are important for sure)
For example is there a tool-based way to understand the relative quality of vendors like FDC vs Psychz using tools without knowing the reputations of carriers like Cogent, etc.
One of my main concerns is packet loss, as I am doing video over UDP, where overprovisioned links may just slow TCP/HLS/HTTP, overprovisioning will mean packet loss for UDP, and not late video segments, but macroblocking of video. (Although late-late TCP video blows)
So, without knowing the reputations of everybody, Datapacket.com, Cogent, FDC, Psychz, etc, etc, is there a good tool or web-tool way to determine what I would get for what I am paying? (network-wise?)
I know I can ping and download the 1000mb files from outside, but I don't know a tool based way to do this to provide a good feedback metric on a particular vendor+datacenter combo.
Thanks in advance.
Cameron
Comments
Determining network quality is quite a tricky thing to do. First of all, quality doesn't necessarily mean you get the best ping or the best latency, but also the stability of the network, so each is best evaluated separately. Similarly, a lower latency doesn't always guarantee better throughput.
We test the network quality on different levels.
To determine the routing and latency, a very useful tool is https://atlas.ripe.net/ which also has a lot of probes on consumer networks. It returns traceroutes / latencies from thousands of probes around the world. This gives you a good idea if traffic is backhauling and how well the routing is configured etc.
However, this means nothing if a link is congested. For example, Cogent has usually quite a good latency, but we all know how it performs. Testing this is a bit tricky though, for us the best tool is https://perfops.net/ (similar to Cedexis) which uses real-user tests to run small benchmarks millions of times per day from all around the world. For methodical testing, this is probably the tool we use most. It also picks up network issues that other tools might not as they work on the user level directly, but of course, monitoring various providers with this isn't really an option.
As more of a homebrew solution, you can rent a few dozen VMs around the world with a provider such as Vultr and download a test file from the tested network. Be careful not to run wget tests on the server itself as you will be testing the download rate and not the upload rates. Usually, download is much less congested than the upload and the results won't really be accurate. You need to be careful to take results with some grains of salt though and possibly test with multiple providers. A Server<->Server connection will usually behave very differently than a Server<->User. Some servers will also have direct peering between them, so always make sure to run some MTRs as to try and figure out what's going on. Running a test on an actual consumer network is what will give you accurate results, so perhaps finding some people around the world to help out would be useful.
Another important thing is that you should always test things during multiple times of the day. Depending on the region, the highest traffic levels usually happen between around 4pm to 9pm and this is the time when most of the congestion happen.
Finally, to test the "quality" of a network, your best bet is to run constant metrics so that you can see how it behaves over a longer period of time. I'd say anything less than at least a few months won't really be representative. Tools such as https://cocopacket.com/ are useful, but they usually aren't cheap. Perhaps a bit easier is to have a look at yearly metrics published for each carrier by Noction for example and then crossing that based on what your provider uses, but I can't really vouch how accurate this is. https://www.noction.com/blog/tier-1-carrier-performance-report-january-2018
Amongst the tools I listed above, I would probably want to highly recommend joining RIPE Atlas. It's free to use if you manage to get a probe from them, but it depends a bit on the region you live in.
In the end, I think looking at a provider's reputation and pricing will tell you a lot about the quality you can expect, but make sure to be able to filter through that. There are people that will rate single-homed Cogent as the best thing ever and people that will rate a really amazing network configuration as crap just because their IP got blocked. Even looking at a company's clients can tell you quite a bit about them.
Also, keep in mind that you might be just coming at a really bad moment, so testing these things always takes time. Lastly, it's important to value how a provider handles an issue and responds to your feedback. Issues happen to everyone, so it's equally important to take into account on how they handle it and how quickly.
Anyway... I hope that was useful.
I wrote such a long post that I don't really feel like re-reading everything and I probably skipped a few things I wanted to say, so apologies if it's a mess
Vouch for RIPE Atlas. If anyone needs credits, ping me.
@BunnySpeed Thanks for such a great in-depth response, so much to learn!
For simple and free...
Network latency, ping, or packet loss I would recommend WinMTR for end user to server, or vice versa.
For server or VPS performance, server to server, I would recommend a simple bash wget script to ping/download test files and output to a text file, from/to different locations, depending on what locations you prefer.
Otherwise, what was said ^ above ^
Wow. excellent nerd-sniping CameronE.
It's a treat to read Original content.