Contabo follow up - and: the *real* Contabo problem
As some may remember I did a couple of Contabo VPS reviews which lead to diverse reactions and allegations ranging from me having got a somehow special VPS to Contabo having high steal and brutally limiting IO.
While the allegations of my benchmark not reflecting the VPS' performance in production were neither (completely) false nor surprising - after all I benchmarked pre-launch and I said so clearly - I did personally exclude the allegations of Contabo tricking me by providing a special test-VPS for me (meaning that they f_cked everybody by letting me benchmark a product "pimped up" for nice results but not available for purchase). But what counts at the end of the day is not my personal view but only cold facts and data.
Again, I had no, zero, nada reason to mistrust Contabo; they've been fair and square from day one and they even told me quite directly that their objective wasn't a nice review on LET but rather to gain more insight for themselves and to spot eventual rough corners.
But I had my own position too and while I'm a friendly person and ready to help most providers, my main focus and my loyalty always is LET and the users there. If a provider can gain some insight along the way or asks me to do some extra work, no problem, but at the end of the day I'm after data for LET - and I told them so.
Regarding the steal, (a) I myself didn't experience significant steal and frankly don't care a lot anyway because "shared" in my opinion and experience is basically just another word for "there will be quite some steal". If one feels to need dedicated resources one should look for a VDS (or whatever term you prefer); with a VPS some steal is to be expected. And (b) pardon me but with a VPS costing €6/mo (VAT incl.) and 4 Epyc vCores, even if the average steal were 25%, that would leave you with 3 vCores "net" and 8 GB (decent) memory, a generous NVMe (50 GB) -or- a very generous SSD (200 GB) and an unusually high traffic volume of 32 TB, so I don't see any basis for complaints even if the "steal" allegations were true.
Regarding the "special test VPS" I already explained that I indeed had a pre-launch VPS. But I don't complain about the beating I got for that. I personally think that it's great if LET gets early access to new products and I would do it again (albeit with an even more conservative approach), but I fully understand the POV of those who demand data based on a VPS in production - and I followed up on that.
As you may remember I PITA'd Contabo into a special arrangement, namely into permitting me to order a VPS with a false name and data in order to get at a VPS just like Joe Everybody - and they agreed. Thank you Contabo and kudos for that!
To really make sure that Contabo had no chance to know I'm behind it I waited about two weeks after having their OK because I played it in paranoid mode; who knows, maybe they would look closely at new orders for a couple of days to spot me. Plus I didn't follow my usual routine but ran far fewer benchmark runs per day so as to not give them a chance to spot me. And I did that for about 3 months ...
Here is the result, first the official test VPS
ProcMem SC [MB/s]: avg 300.9 - min 263.1 (87.5 %), max 321.9 (107.0 %) ProcMem MA [MB/s]: avg 676.1 - min 288.0 (42.6 %), max 903.8 (133.7 %) ProcMem MB [MB/s]: avg 832.5 - min 619.1 (74.4 %), max 947.3 (113.8 %) --- Disk IOps (Sync/Direct) Write seq. [MB/s]: avg 26.99 - min 6.16 (22.8%), max 37.82 (140.1%) IOps : avg 6909.86 - min 1577.30 (22.8%), max 9682.29 (140.1%)
and now the "anonymous" VPS
ProcMem SC [MB/s]: avg 279.6 - min 245.5 (87.8 %), max 297.5 (106.4 %) ProcMem MA [MB/s]: avg 554.2 - min 304.5 (55.0 %), max 789.2 (142.4 %) ProcMem MB [MB/s]: avg 816.4 - min 601.6 (73.7 %), max 926.5 (113.5 %) --- Disk IOps (Sync/Direct) Write seq. [MB/s]: avg 13.98 - min 3.19 (22.8%), max 30.09 (215.3%) IOps : avg 3577.82 - min 816.26 (22.8%), max 7703.09 (215.3%)
(all with vpsbench v. 244a)
So, yes, there is some difference, but before shouting "we got you, Contabo!", read on ...
Part 2 - or the real Contabo problems
As you might have noticed, there is a new vpsbench version 2.50 that also does some test using OpenSSL (AES-128-CBC and RSA-1024 key pair creation) and as always I tested the new version extensively on diverse VPS and dedis ... and noted something strange on my 3 Contabo VPS (DE, US STL, and SGP).
Let me show you:
DE avg min% max% US avg min% max% SG avg min% max% PMSC [MB/s]: 144.6 - 39.0 %, 166.8 % 148.5 - 40.8 %, 160.4 % 146.3 - 39.1 %, 162.8 % PMMA [MB/s]: 626.5 - 31.0 %, 119.3 % 677.0 - 88.6 %, 110.8 % 652.7 - 79.4 %, 117.4 % PMMB [MB/s]: 713.7 - 66.0 %, 109.5 % 732.4 - 90.7 %, 107.8 % 699.1 - 86.3 %, 108.6 % PMAES [MB/s]: 980.5 - 48.9 %, 109.3 % 1057.1 - 82.8 %, 102.5 % 1010.2 - 81.3 %, 107.4 % PMRSA [kp/s]: 67.6 - 83.4 %, 111.9 % 66.6 - 91.0 %, 106.2 % 65.7 - 87.1 %, 105.6 % --- Disk IOps (Sync/Direct) --- WrSeq. [MB/s]: 14.69 - 11.2%, 198.7% 1.97 - 100.0%, 100.0% 1.97 - 100.0%, 100.0% IOps : 3761.72 - 11.2%, 198.7% 505.11 - 99.9%, 100.0% 504.74 - 99.7%, 100.1%
(all v.2.5.0, Sorry in case I f_cked up formatting.)
Looks quite nice at first glance right? Similar enough values and spreads over the 3 locations one would think. But have a closer look!
To show it (hopefully) even more clear what I mean:
--- ALL --- min max ProcMem SC [MB/s]: 56.4 241.1 ProcMem MA [MB/s]: 194.3 750.2 u ProcMem MB [MB/s]: 471.2 789.8 u ProcMem AES [MB/s]: 479.6 1084.6 s ProcMem RSA [kp/s]: 56.4 75.6 Write seq. [MB/s]: 1.65 29.19 IOps : 421.92 7473.65
These are the absolute min and max values of all 3 locations ('u' (US STL) and 's' (SGP) indicate the location if it's not DE). I liked Contabo, a lot, and still do (kind of), but sorry, that's not nice, not even acceptable, and indicates rather poor node management and/or cramming too many VPS onto a node and by the looks of it too different usage patterns (yes, that term does also hint at abusers)
Getting a VPS with a single-core performance between 56 and 241 (a factor of almost 5x), AES, a very important and frequently used algorithm, maybe half the speed, maybe double, and a disk with a performance factor between 5x to 10x, boild down to not buying a VPS but playing lottery, sorry.
And yes, 1 of my locations (DE) is NVMe while the others are SSD, but in fact the above min/max values are from the same device, the DE NVMe.
Plus, note the "perfect" SSD disk results from US STL and SGP: 500 IOPS with about 0.1% spread - or in other words nailed down and a not exactly pleasant - and also nailed down - 2 MB/s in 4K block size, 4 threads, sync. mode.
And, coincidence certainly, today I've got a robot email informing me that next monday some changes will happen on "my" DE node. I'm in suspense what it might be; maybe, just maybe, my NVMe performance will be nailed down, too?
Sorry, Contabo, while I do welcome your attempts to lower disk spread - and it is necessary - I do not welcome the way you do it and the results. They are poor and frankly, unless you find a better way I do not see a chance to recommend your products any more, sorry.