New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
By SPOF you mean if my card failed, I am gonna have a very bad day ??
In my case, the RAID card is actually (LSI MegaRAID SAS 9260-8i). Not sure if it qualifies for "good and relatively modern raid card". Does it??
PS. Thanks Maounique.
It's really useful info
That is one of the newer cards which work well with ssd.
2 things:
1. If you can have so many disks with it, get the regular HDD, the more you have and the more cache, the difference to SSD will shrink. If you really have a very busy database and ram will not cut it and/or you do not need a lot of storage/the price per GB is irrelevant, then, SSD will still matter. For a raid 1 it is a giant waste, though.
2. You MUST use a battery if you enable write-back cache. If you do not enable it, it will be largely useless, if you enable without a battery, you will regret it very much later. Also, the cache seems rather small, i would expect 1 GB for this config.
If you got it second hand and it is a sturdy one in mint condition, you really need that many disks and your app requires north of 200k IOPS, you do not have enough sata ports, they are SATA 2 and your disks SATA 3 and other scenarios, use the controller. If any of the things above is not true (especially you are not sure how many IOPS your app needs but you heard that more is better) you probably do not need it, or, likely, not even SSD.
If I may chime in ...
There is a major misunderstanding out there re. HW vs SW RAID. SW is faster if a) the CPU can perform the RAID ops faster then the HW RAID controller CPU (which was and is almost always the case) and b) if the connection to the disks is as fast or faster than that of the HW RAID controller.
But there's a BUT: The idea of HW RAID was not to be faster than the host CPU and disk connection. It was to offload RAID processing so as to keep the host CPU free for its real job (like doing PHP and serving web content).
And don't forget that it's not just about RAID computation (which is trivial for 0,1,10 and still well acceptable for 5) but also about handling the read write IO (which is an ugly beast).
In summary I'd say:
All those "ZFS can do x,y,z performance vs. HW controller only does a,b,c" is nonsensical because almost always your system is not just a disk controller but rather a web server, DB server, whatever.
If you just love benchmarks too much, at least make reasonable ones that is sth. like "My LAMP server on linux foo x.y does ... with HW controller and ... with ZFS or other SW RAID".
Hth
I run some Intel 320s in H/W RAID 1 via an old Adaptec 2405 and they do great for a medium duty database of about 3.3m queries per hour; https://www.facebook.com/Incero/photos/a.398352700204330.94996.259717217401213/880971508609111/?type=1&theater
I think a lot of people way over-spec the hardware needed for their projects, which is great when you're selling servers, but no so great when you're boot-strapping a project. Sometimes I wonder if the people buying RAID 10 SSD machines for MySQL have even added an Index on their tables or configured memory/caching correctly etc.
Obviously 8+ SSDs in RAID 10 with a beefy LSI card is nice for VPS hosts etc.
Yeah, in 2000. Now, we have 2015 with multiple cores, more cpus, fast sata/sas connections to the cpu and flash memory. Don't waste the money for a hw controller and buy a faster cpu. This will be handle your i/o load better and you can process your webserver faster.
Not really, do we take a look at enterprise storage: netapp and other storage vendors do not use any kind of raid controllers only hbas. Why, easy the software can control it better and add additional redundancy to your data like the 520 bytes blocks with 8 bytes of additional ECC data.
BBU, not need if you have caps and your filesystem controls block and file level. BBU do not protect the cache in your disk.
Sure, all modern filesystems like BTRFS going this way to have control over block devices and your files in your filesystem. It is even better to control the block device and the filesystem in one component than only see the block device like a raid controller does. This helps to secure your data and performance. Or free up space with compression or deduplication.
With all due respect, you seem to not have understood it.
Looking closer (and knowing about it ...) one will see that behind disk IO there is lots of stuff happening. It's not so much the reading or writing per se but the housekeeping and system level things that make it ugly. While on the surface it might look like "Oh well, some bytes are transmitted" in reality there are interrupts (which flush your cores pipeline and cache) and a frenzy action back and forth on the busses, etc.
So HW controllers are in fact, the mechanism that actually make disk IO and RAID what people, looking just at the surface think it is.
So, no buying a faster CPU is usually not the solution.
Again, you didn't really read (or understand) what I wrote. Those are storage providers/vendors. Read again! Because - as I've clearly said - I've talked about the usual situation, so sth. like LAMP. For a storage server it's an entirely different story because there storage is the very application. Accordingly it makes sense to use all the systems performance and capacity for disk IO and RAID. That's what those systems are for, after all.
For a, say, web server, however, I want to use the performance/capacity to generate and serve web pages - disk IO and RAID are just a necessity but not the purpose of such a server.
"BBU"
You simply mix up quite diverse things here. And btw. don't overestimate capacitors ("caps" as you like to call them).
What you say is about as illogical as "use a UPS and you're troublefree". UPS, capacitors (if at all), BBU, and drive caches address completely different problems.
"block devices, btrfs, etc"
Pardon me but that's blabla indicating that your view is superficial and, excuse me, "religious".
I don't care about HW controllers or ZFS or generally software RAID. These are all just mechanisms and approaches that, depending on the scenario, make sense or not. In some cases an OS's software RAID will be just the right thing to do, in other cases ZFS will be the right solution and in yet other cases a HW controller will be the best choice.
As most users here are in one way or the other confronted with web server scenarios it made sense for me to mainly talk towards that sort of use case. And there, indeed, HW controllers often are the best solution.
You disable disk cache when using BBU.
I do agree that some power is going to compute stuff for the raid in a sw scenario, the only issue here is that it is insignificant, especially for a raid 1 scenario which we started from. And, while some cpu is spend for other bus operations and memory transfer (extremely marginal) before going to disks, it is not that much lower in case of a raid controller compared with the "standard" SATA onboard ones. After all, the cpu does not really control writing on any disk since dropping the PIO model, it is the controller which does that, the cpu simply instructs the ram controller to dump the data to the disk controller and then largely forgets about it.
Yes, in case of raids with parity of sorts, the cpu does some computing in the sw scenario more than in the hw one and that is where the most difference is, but even then, the difference is so low in today's CPUs, that they do not matter. Saying it takes from php computing power is true, indeed, but it is like taking buckets from the lake, it simply does not matter.
IMO, for complex large arrays with schemes like raid 6 and 24 disks it is worth to have a hardware controller and even two for a 61 raid to protect for the case of one controller failure even for the simple fact that the mainboard does not have enough sata connectors. But that is a dedicated storage box and a SAN already starts to look feasible, modern ones provide much more than simple RAID.
For a regular usage scenario, not to mention really simple cases like raid 1, sw is more than enough for ssd. There might be other scenarios out there, but i admit my imagination is not very active lately.
Looking closer (and knowing about it ...) one will see that behind disk IO there is lots of stuff happening. It's not so much the reading or writing per se but the housekeeping and system level things that make it ugly. While on the surface it might look like "Oh well, some bytes are transmitted" in reality there are interrupts (which flush your cores pipeline and cache) and a frenzy action back and forth on the busses, etc. So HW controllers are in fact, the mechanism that actually make disk IO and RAID what people, looking just at the surface think it is.
Do you measured the performance loss of a normal raid setup with software? It do not look likes, this will cost under 1% of your total performance. If you need this percent urgent, then you have other problems and should upgrade. So why should i buy a hardware raid card for 700€ when i can speed up my application with the same money serval times. Realy performance for my application is much higher then with the same server for the same money. Correct, make no sense to invest in a hw controller when you want the best performance for your application.
Try it by your own, overload a system mit cpu benchmarks or whatever and then do a storage bench on hw raid and soft raid. Both of them will have the same performance(~3-5%) in standard raid levels.
See, on top.
BBU protect the raid controller cache, caps protect the cache in your disk. Both doing the same, protect your data, but in different positions. You need no BBU if you do not have any raid controller cache, the BBU is in your disk. You only get in trouble (in both scenarios) when you use write-back and the os stores the data in memory and both will lost this data on power loss.
There a no really choice for hw controllers any more. They stuck on the old level since years. Trim, oh not yet..compression not yet, deduplication not yet. A hw controller makes only sense for systems which do not support software raid.
Nothing new, most user don't do it.
@Maounique
Sure, what modern OSs bring along is usually good enough. But then, we're here in a situation where the question came up anyway and so it makes sense to look at the options.
I am, in fact, writing this on a machine with SW RAID1 and I won't complain. But then, on this box I, the human, am the limiting factor.
Sorry for correcting you (or extending on you) in 1 point. No the sata controller doesn't do any magic; it's still the OS that does all the work. The sata controller is mainly a "translator" from PCIe to Sata and managing the low level Sata housekeeping. All the rest, that is virtually everything, is still done by the OS.
And that shows. In my case I lose some 10 or 15 MB/s (as compared to no SW RAID1). That's perfectly fine on a workstation and also good enough for, I guess, 90% of the web servers out there.
But again, I take it that we have this discussion here for the other 10% of cases.
Now, look at modern web servers with a 1Gb link. Filling that link (which rather rarely happens) will equate to something in the range of 500 - 5000 req/s. And that's the decisive measure because, after all that's why we do what we do, to handle requests and pump out responses.
Now, do a test. Pump 1Gb consisting of 500 responses of 200KB each. Then pump 5000 responses of 20KB each. The former won't be a problem, the latter will bring many smaller servers to their knees. Unfortunately the latter is the more frequent (and we're just half way to the 10kreq/s issue ...).
It's not the data volume but the IOs. And not just tofro the disk(s) but also tofro the server (threads or, worse, processes) and tofro the network controller. And usually the DB, too.
And each of those is riddled with interrupts, waiting for call backs, event handling, context switches, empty pipeline and L1 (and often L2) caches, etc.
Having some low level packet handling offloaded to a good network controller and having some low-level IO handling offloaded to a good RAID controller does help and help a lot.
Again, for the average Joe and Mary website or server this is far beyond their reality. HW RAID vs software RAID, SSD vs spindles, etc. doesn't mean a lot for their scenarios. If anything they should be advised to increase RAM and be done with it.
But for, say, a hoster, the story is different. Because being capable of running just, say 25% more sites by adding a relatively modest investment will very often translate to significantly increased income.
And then there are cases where simple scaling ("add more cores!") just doesn't cut it because there are limits (usually in the range of 4 CPUs, about 10 cores each). Sure, you can go SPARC or POWERx but that brutally changes the economy of your project.
@fileMEDIA
I'm afraid that discussion with you leads nowhere. You are set on the OS/software way no matter what.
Well, I don't care. If that is your religion and you're glad with it, do it that way by all means
You see, while you're talking about TRIM and judge solutions by their capability to do that, I just don't care about it. The reason is simple: I've not yet seen many project where SSDs were a decisive factor. Where you say "HW controllers don't support TRIM, they are evil!" I would say "SSDs need special sauce. They are evil!")
Well noted, I'm not at all set against SSDs. They are, for instance, a very nice marketing tool for VPS hosters and I could even see cases where they are actually useful.
But my world is mainly 2 things: Complex projects and security. 500 or 5000 reqs/s are just not my scenario, neither is 500MB/s rather than 200MB/s disk IO.
Let me give you an example: In my world we use multi-layered caches, auto-DB shadows, and optimized TLS (performance and security wise). About the major factor for disks with us is data safety, i.e. the prevention of loss, (almost) no matter the cost. Performance? I don't care that much because with tens of thousands of req/s the decisive factor isn't enhanced disk IO but rather smartly reduced and optimized disk IO. I'm not interested in factors of 2 or 5, I can't afford those. I'm interested in factors of 20 or 50 or 200.
In the end it's simple. Either you have a more or less average scenario, then all those issues don't matter much (you like SSDs? Fine, buy them and use ZFS and feel good) - or you have a complex and/or high performance case, then you'd better really know your stuff profoundly well and design an adequate architecture. Sometimes SSDs might be a piece of the puzzle but usually they aren't.
Friendly greetings
Using a Galera cluster, all this H/W vs. S/W RAID (1, 10) and BBU and SSD onboard cache discourse, while very interesting and educational, is moot I think. In this scenario (Galera cluster), I'm only concerned with the speed of writes to DB. I think 2x 125GB setup in S/W RAID 0 - as opposed to a single 125GB or 250GB - should speed writes to DB on the order of 100%, n'est pas? What I'm asking is: will MariaDB take advantage of this? Any special configuration setup needed?
One other thing: does the above setup make any sense when data is coming in over the network (from a remote host, not generated on localhost)?
That is what the thread was about, mainly, will raid 1 benefit enough to justify the extra risk and investment for a hw controller. The answer is clear, no.
Obviously there are scenarios where it is needed. In a hosting machine for shared hosting with god knows how many people with unoptimized dbs and 0 caching, a raid controller with a HEFTY cache, writeback and BBU, 8+ SSDs will be needed if we are to saturate 1 Gbps port, however, a monster like that will have already at least 12 E5 cores and, given a sensibly built mobo with no bottlenecks in the buses which should be the default for any server, even low end, 8 SSDs in raid 5 sw will probably be tolerable. Yes, the performance wont be that big, but I suspect will be just enough to do in the conditions of the other limitations, such as 1 Gbps port, net latency, cpu limits for each customer which will slow down the abusers, for example.
As I said, there can be scenarios out there where a HW controller for ssd can be needed, but, even for professional use, a sw one should be enough. We are introducing risks with a hw controller, not reducing them, therefore the performance increase must outweight the risks and for this must be absolutely needed, i.e. there is plenty of other resources available and the 8+ SSDs and their IOPS are really the bottlneck. We do not even consider the costs. We must not forget that we are always talking SSD here unless mentioned otherwise, in a 2 spindles scenario even in raid 1, a controller with even a low amount of cache would help, but for SSD that is not at all the case.
Only in theory, in fact there are other things which contribute to latency, however, raid 0 is generally a very bad idea unless for data you do not really care for or you have it replicated elsewhere in real time.
Also, in the whole discussion, most people do not make the difference between small and big writes as well as their number in places. The SSDs do not outperform the HDD by far in sequential writes, but in the number of operations per second and they can read and write in the same time, in theory, while, even theoretically, a HDD cant very often. A RAID 1 will write in the same time the data to both disks taking twice as long as a raid 0, but can, in theory read twice as fast, because will read from one disk a part of the data and from another the other part, so raid 1 is not twice as bad as raid 0, both have their place. but raid 1 is way more required than raid 0.
So, to come to the raid 0 for 2 drives, they might write theoretically twice as fast, but will read twice as slow as the raid1 scenario.
In my experience, databases used in hosting have about 5 times more read IOPS than write ones, and this makes a huge difference in choosing the right kind of raid.
As I said, there are no 2 identical databases and no 2 identical usage scenarios even outside the database scenario.
I do not write this to argue, but to add further precision to our general talk.
@aglodek
Nope, not really. Because SW RAID 0 has to write to two disks, i.e. perform 2 full write cycles (which makes it still faster than single disk/double volume writing but takes a toll which is usually smal but can get ugly on lots of small writes).
That said, with a DB it works somewhat differently anyway.
Short version: Almost certainly you'll be fine anyway. HW vs SW RAID shouldn't be your problem.
As for Maria DB I can't tell you much; I don't use it (for a reason but I'm not in the mood to light up a new religious war).
As for the last part of your question: hard to tell without knowing any parameters. Very generally and losely speaking you's want to roughly match net and disk IO with disk IO having some reserve. So, for a 1Gb link you'd like to have a disk setup that allows you to easily handle min. 120MB/s - which shouldn't be a problem with a decent system and reasonable disk setup.
@Maounique
I widely agree, except for RAID HW and "risk". I see it the other way round. The way I see it, SW introduces risk and HW RAID minimizes it.
As for aglodeks question you braught up a good and valuable point. I not only agree with you but actually suggest he doesn't go RAID 0 but 1 (assuming he wants 2 disks only). From what he says about his setting I'd think that RAID 1, maybe with some more RAM, would be well feasible and more reasonable.
Just curious, what risks does sw raid introduce?
HW, besides the obvious controller failure, also has the problem with proprietary formats (yes, i also had the problem with revisions not being compatible backwards), it can also mess up happily more easily until it is way too late for even a chance of recovery than a sw one because the sw one has the kernel to register anomalies before it is too late, while proprietary chips in the controller run most of the time headless and outside some clear recovery cases, are not able to detect failure.
Yes, a HW controller will detect minor failures and disks slowdowns which will slow down the sw raid until there are smart errors in the logs or the operator notices slowness, even random rebuilds, but, in general, if setup right, should be as safe as a hw one, barring the card failure.
@Maounique
Well, I'm looking at those things very pragmatically. Doing that I can't but notice that belly up or, worse, dying and unreliable, power supplies are among the top culprits in any server farm. And that's where your SW RAIDs get badly fu--ed up while the HW sets keep you save.
As for that ugly, ugly ultra-capitalist greedy habit of manufacturers to change some bits here and there so as to keep you caught the solution is simple for any farm of any not insignificant size: Buy lots of one and the same controller and keep a good reserve.
That said, I'm waiting for some smart Chinese guys to come up with a HW controller that is basically just a cheap SOC with linux on a PCIe card. With such a thing we could have all the advantages of HW RAID without the greed tribute.
If you do writeback mostly in raid 0 scenarios.
Trust me, I had tons of power failures (welcome to romania where power and running water are less reliable than the landline internet, even through trees), it managed to mess up the BIOS (!) of a raid controller but no sw raid ever failed. I use HDDs in raid 10, though, might be way different in raid 5 for example.
@Maounique
We can continue all day long and discuss diverse scenarios. My experience is quite some screwed data sets due to power/power supply problems. Of course, experiences are different.
The main point, however is: I'm not against SW RAID. I'm merely against rather unreflected fanboy blabla (not meaning you) and preaching of linux SW RAID being the paradise and HW RAID not being justified no matter what.
HW RAID is pretty much unnecessary now with only a few exceptions. Another point of failure and additional cost and proprietary. Those are 3 good reasons to get rid of it even if you take a slight performance hit.
Please keep away from dedify.com, the biggest hustlers I have meet.
There server is no worth a cent, have taking screen shot on how they limit my account so I can't delete services or create working services!
DON'T BE A FOOL LIKE ME! KEEP AWAY, DEDIFY.COM IS A HUSTLER COMPANY!