New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Stop abusing "dd", use ioping instead?
In the thread http://lowendtalk.com/discussion/19618/simplercloud-1-month-review, @eLohkCalb wrote "Stop abusing "dd", use ioping instead".
So far, I understand that DD will sacrifice a lot of CPU and IO of the host node. One of my providers recommends that I should DD only once or twice a day. My question is can we do ioping every hour?
I am looking for a way to know how fast the IO "hourly" without affecting IO of my "neighbors".
Comments
Well, i'd say stop abusing both. What really matters is "how fast my real world application works". Or in the most common case "how fast my site works". This includes a combination of CPU, IO (if the site is mysql heavy), network, etc.
Why would you want to do this? Is there a reason you need this figure?
Sure you can, it doesn't load the node the same way dd does.
Why not? E.g. in the past I had an issue with CPU speeds on one VPS, since then I keep an eye on it with a brief CPU benchmark in crontab, so I can check if that issue has reappeared. That's just basic monitoring, similar to watching network pings and if there's any packet loss. Of course I made an effort to set it up so that it doesn't affect other users, specifically running for only about 1 second every hour (and not at hour:00 but with an arbitrary offset, to be unaffected by everyone's cronjobs typically set to fire at :00).
Constantly benchmarking the application itself can be trickier than watching CPU/disk/network speeds for any degradations directly.
I think @Mun does this on a VPS at GVH just to see how terrible the IO speed is at parts of the day, and also to see if it gets progressively worse or better.
Edit: Yup, found it.
http://192.3.31.219/dd_historical.txt
Imagine everyone runs that
Though, it should be run once at the host level and graphs shared. All automated. Performance figures, monitoring, suspension, everything transparent. In the ideal world that is.
Because realistically it doesn't mean much. You might check every hour. What if it dies every 1/2 hour or per 26 minutes? You get the idea?
I feel pity for your provider!
It's not that pity, a ioping is not that hard on the disks and finishes fast (depends on the options of course). A simple sendmail cron running every 20 minutes can be a lot heavier.
I've just started monitoring my vpses recently when I observed my app run very slowly. My original purpose is to be able send a graph to my provider so that the ticket is more fact based.
Do you mean it also hits host node performance to run ioping frequently? It is the main question for this thread.
After several days of collection, I saw that io latency and dd results are very highly correlated. If it is safe to run it hourly, I will use it as IO indicator instead of DD.
Both are stupid, in terms usage that is thrown about on LEB, I cringe everytime I see a dd result here.
If you do actually have an application running (other than benchmarks) on your VPS measure things that matter - do you have a baseline? in terms of a web app - the response time of your application (time to first byte, time from first byte to completion, availability of the node), if these show problems or high variability then start looking for the culprit (which may or may not include tools like ioping). If validating a node for production then use tools such as https://www.blitz.io/ to stimulate real load etc..
Stop abusing "dd", use ioping instead". --> Stop doing stupid pointless shit that doesnt provide you with any real insight
For me, it is not pointless to have these benchmarks on VPS environments where we are sharing resources with others. To really evaluate how well our code is written, I think we need both the benchmarks for the our app and the benchmarks of the VPS environment.
Thanks all for your replies from which I can interpret that ioping can be used safely.
If you need guaranteed consistency get a dedi
a vps is still a shared environment and you have to expect variable performance metrics.
Ioping is near pointless on a vps especially on openvz because your hitting the cache in most cases.
Random write and read tests would make more sense. But again if performance consistency is so important that you need to check it hourly, you need a dedi
If you must, just do one ping only and spread it out at to at least several minutes. Sticking the results through any reasonable graphing system that does averaging will show you when a node is becoming overburdened.
Could you please elaborate further on how to do this? Currently I am using "ioping -c 10 ."
Yeah but if every single user on the node was running a DD or ioping test at regular intervals, that would cause some sizable performance issues no? In an effort to "benchmark" you're potentially impacting another users LEGITIMATE disk usage.
@SkylarM that is exactly my concern and main reason for this thread. I am looking for alternatives. I will only do it if there is minimum impact on the neighbors.
I don't think you'll get around that issue in all fairness. You're running something that isn't required to run and operate your website/services on the vps at regular intervals for the sake of benchmarking. I like to run a general rule of "do I need to be running this? No, then lets not run it" when I buy VPS.
ioping -c 1
Sure, it may waver, but when spread out over a large period and aggregated, it's just as good an indicator as c 10, but with far less load.
@tchen I see what you meant, thanks for your reply
Are you sure you realize what exactly "ioping -c 10" does?
Eh was more directed at a dd test than ioping. In general the same idea applies towards it being unnecessary, but yeah not the same as a DD. Should have been more specific.
Here's a crazy idea targeted at no individual. If you have a website, how about you monitor it's load time and latency as well as your iowait and CPU% like everyone else in the world?
I mean honestly, that's what most of us do with our vps anyway and that's how you monitor your applications...by monitoring your applications.
But who are we kidding, dd is king. We all know LEBs are for benchmarking.
Yes sir!
Most of my process time is to do with I/O wait.
@jarland yep, this is how I see it:
Host A...
sequential writes used for almost nothing real world = 1.8 GB/s random reads used for almost everything in the real world = 4 MB/s
Host B...
sequential writes used for almost nothing real world = 70 MB/s random reads used for almost everything in the real world = 340 MB/s
Everyone buys host A anyway... because reasons.
Amen, bro.
Even if you do have a website, I think it's a fair idea to get some disk load statistics across a period of time before moving a website onto a new server.
I generally set a cron every 5 minutes to write 64KB (that's KB, not MB/GB) with dd on a new VPS to see how good the disk latency is over a period of at least one day, before installing anything else.
This test should place practically no additional I/O load on the host node and gives you an idea of what to expect from the disk. (I generally look for what the highest latency is, because, assuming your application needs responsive I/O, that's when it's going to choke)
I'm not saying that it's a replacement for evaluating the performance of your website, but it's much quicker and simpler to set up, and is also rather application agnostic.
Might be relevant http://serverfault.com/q/12679/40825
If you really need a very fast website you cache everything like mad in RAM. No disk will do better than RAM in random access and responsiveness. There are tons of mega ram offerings,with proper cache almost everything will not need a fast storage.
Thats a good point, in the business of scaling websites a general rule of thumb is that your GET pipeline (on hot areas at least) should never touch disk and should do only a minimal amount of processing. This also extends to any server-side analytics which are generally batched to a limit and then background persisted or shipped somewhere else for persistence. I nearly had a heart attack when I witnessed the atrocities Magento was committing (and indeed many other PHP based apps). It is kind of cute when you see all this dd and ioping benchmarks when reality is that few and far between ever required fast persistence.
This is different to what's happening here. People are running it non-stop before and after moving a website.
To be fair the industry (and LEB, and WHT to an extent) has created a lot of this hype, marketing such "super fast SSDs", "SSD performance", "fast raid 10 disk", and then stats/benchmarks etc.. and then joe punter (that collects VPSes for no other reason than a proxy/vpn or tiny wordpress site and could suffice with a 1993 20mb ibm ide drive) wants to feel he's got what he believes he has paid for. Sadly too it seems to have becoming a pissing competition between the kids, my io is faster than your io .. your mum's io sucks