All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Automated Server Benchmarking
serverbear
Member
Hey guys, I'm new here. Been a lurker for quite a while, from Melbourne in Australia 
We've been working (in our spare time) on something for the last few months to help people compare servers in general by performance metrics. I've seen how important benchmarking LEB's are when offers are posted & would love some feedback. I'm really sick of seeing review sites or comparison websites that only show affiliate offers & provide no real value.
Our script does the following:
Grabs general system info (CPUs/RAM/HDD info)
dd - Runs 4 variations of dd for comparison
IOping - 4 variations of IOping (Pings, Seek Test, Sequential & Cached IO)
Network Efficiency - Tests download speed from 9 locations
Tracert to Cachefly
Ping to Cachefly
UnixBench
It's all run from a single command which we append to a particular host & plan. If you personally run the benchmark we generate a report like this: serverbear.com/benchmark/2012/07/22/nhvwlr22zS4yz60O. You can share that or do whatever you like with it, the URL is private by default.
We aggregate some of the key metrics up so people can compare them when searching across plans (Here's VPS plans under $7 as an example: http://serverbear.com/compare/lowendbox), you could overlay SSD onto this report or filter by IO/Unixbench score. What other filters would be useful (i.e. Low End SSD Providers by IOPs).
We'd love some feedback, we're keen to make this as comprehensive as possible plus get as much data as possible, we're toying with the idea of a proper score based on all the data (rather than analyzing it individually). We only released the IO/IOping stuff today so we're a bit light on data.
Disclaimer: We do plan on monetising hosts that do have affiliate programs (we have to make a living), but that won't impact our data. It's filtered by the raw metrics, not by who pays us the most commissions.
Cheers,
Stuart @ ServerBear
Comments
Hm... so basically you guys put everything we usually do in a review and a test (hardware/product-wise) and put it into that.
I like it.
Looks pretty good, going to give it a try. Welcome to LET
You should give credit/acknowledgement to freevps as you've just added a few more tests onto their script
Also, I can understand doing a traceroute for Cachefly (as it's anycast, it helps to know where you're downloading from), but I don't think a ping to an anycast system provides much relevance on the network performance.
Other than that, I do like it and I really like the format/layout of the site.
Yep, we currently support Ubuntu, CentOS & Debian. We automatically install all the packages & run everything.
Would love to know if there's things that are missing or what could be improved (i.e. multiple runs of dd/iops & taking an average), doing tracert's from multiple locations rather than 1. We're talking with GeekBench at the moment to see if we can integrate that plus looking at some of the Phoronix stuff.
Conscious to not blow out the time to complete by hours though
Grumble grumble at the "automatically install all the packages" part, but it's not like they're that many anyway.
In the middle of running a test right now!
ALSO: Site does not play nice in Opera.
Found a bit of a bug: benchmarked a 128MB SecureDragon and somehow it morphed into a Heroku Fugu on the stat page along the way. Interestingly, the email said "Fugu @ SecureDragon"
ALSO: 256MB VPS6 turned into "Intel Core2 Quad Q8200 @ VPS6" via email turned into "Intel Core2 Quad Q8200 @ Burst.net" via stat page.
@serverbear wow, you benchmarked an BuyVM KVM from @Francisco and @Aldryic. Impossibru! (stock is a myth)
@ihatetonyy Yeah sorry about that, a bit of a bug with plans we don't have already added. We've fixed it & will push out tomorrow, Opera is a big of a weird one, we're focusing on FF & Chrome for now
@djvdorp We've had quite a few benchmarks already from existing BuyVM customers, of course we'd love our own boxes too. We're planning on building a parser to let people know when certain hosts are in or out of stock.
Any other constructive feedback is appreciated, we can iterate new features pretty quick so let me know what you'd find useful!
Something like doesbuyvmhavestock.com or lowendstock.com ?
@djvdorp Both those sites look awesome, potentially something similar but at a larger scale across more hosts (it's easy to scale if they all use WHMCS for example), we also want to integrate the idea of there may be data of a similar host with better performance (for the same or lower price).
How many affiliated links did you stick in there? You should disclose which links are affiliated and which ones are not
@zhuanyi We're at less than 1% affiliated at the moment, we're more concerned with getting benchmark data & getting plans populated. Disclosure is pretty important to me, but at the same time getting more valuable data is a bigger priority than signing up to affiliate programs
Agreed, but for the ones you have affiliated links please disclose rather than just masking the link with a quick HTTP redirect.
Otherwise, as far as I am concerned, you could be benchmarking based on the affiliated commission rather than the actual performance
Very much so, though turns out what I'm whining about is just a Bootstrap bug.
Once I pop in the Bootstrap-2.1.0 JS the tabs work beautifully.
I like it. As for affiliate links, I've no problem with a site like this using an affiliate link for every provider. It's a good way to pay the bills without ads, and it doesn't hurt anyone.
Because you are the first one and the only one in LET with an affiliated links in the signature haha
No offence intended, but I like your style where you clearly specify it is an affiliated link and if I like you or if I do not mind give you some commission, then I'd be happy to click on it. On the other hand, if I do not want to do so for any reason, at least I know I should not be clicking on that link.
Earning money is important, earning money the right way, particularly for a review site, is even more important
Affiliate links should be fine as long as they do not create any bias.
I'm curious, whats the reason? You don't lose anything from affiliate links, and you're helping (at no cost to you) a site thats giving you information for free.
Quite frankly speaking, in this kind of situation I really doubt if the reviews/benchmarks are based off the performance of the VPS or the affiliated payout. No offense, but I have seen too many "review" sites which claim Hostgator/FatCow/GoDaddy as the "best" hosting providers out there simply because they offer the best commission payouts.
And just for clarity: I am OK with affiliated links, and I agree no one should work for free for the valuable information they provide. What I am against is not disclosing them and give the readers the option to click on affiliated links or not.
They were upfront from the very first post about affiliate links, and if it's what they believe they need to make money from the free service they're providing, so be it.
Unless every part of the website's operations were completely and impossibly transparent from start to finish, someone could come along and take issue with their form of funding and claim that it could/does result in a bias in their results or editorial content.
Their use of affiliate links is also stated at the bottom of their About Us but it could do better at the bottom of each page, though.
(If one wants to make sure that their results are not being fuckered with, the script also leaves behind an sb-output.log file in the directory you ran it in that has the raw output from each test.)
Sorry if I was not being clear, but this is what I hope to see:
http://providersites/affiliateLinks (Affiliated Links)
http://providersites (non-affiliated Links)
In every single link they send to the providers' website rather than
http://serverbear/itCouldbeAnAffiliateLink
LOL
403 forbidden from iam here
@liam Thanks liam, appreciate it.
@zhuanyi We use the link masking for tracking purposes, I may eventually want to show popular plans per day/week/month. It's a core part of the way the site is built. We may integrate a [a] type symbol beside hosts that are affiliated, I wouldn't say it's high on the priority list atm tho.
@ynzheng You're the second person that's been having that issue, are you using Google DNS by chance?
Anyone got any feedback on features & would like more detail on certain things/like to see stuff included? Plus from a web hosting POV what could we do to help hosts show off their benchmarks scores a bit better? (API, Widgets).
safari in IP4 with 3G connect