All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Where can I get godtier SSD storage / cloud storage as fast as SSD speeds
I currently have a number of EX42-NVMe servers with Hetzner each with an extra 960GB NVMe drive (20.5 Euro) added to them as the processes I run are mostly I/O bound by my tests and I wanted some very fast swap speed, and extra storage. I'm running out of storage on these and experimented with using an SX62 to host a 40 TB LizardFS system but each EX42 machine could only run about 60% of the processes it could using only local storage before all these processes actually crashed.
I wish I could remember the numbers concretely but I was running iostat at the time and noticed about 40% load, but I didn't check the queue length. I am a silly boy.
I was convinced at the time, and still am really, that this means the processes I am running are I/O bound as there was no difference in swap, what medium that swap is tied to (NVMe), memory (64 GB DDR4), or the CPU (i7 6700) between these tests. That is, running X processes on the machine with the files on those NVMe drive and LizardFS under the same conditions except these processes on the LizardFS mount (HDD 7200rpm, see the SX62) resulted in 0.6X capacity.
So, where can I find a tonne of mountable SSD-esque storage solutions that I could mount and replace for my EX42-NVMe servers?
I should note I don't think NVMe is necessary, nor DDR4. I believe baseline SSD and DDR3 would be fine if the only solutions are different dedicated machines or cloud solutions with more storage and lesser specs. Must have 64 GB of memory and a similar CPU though.
I'm going to experiment with Hetzners BX and NX storage solutions now but I don't think they would cut it only because they are shared connections so throughput wouldn't be guaranteed, who knows.
Help me LowEnd!
Comments
are you heavily budget restricted? getting >1g (or like old infiniband stuff+) connectivity is usually not cheap on dedis to the point where it becomes feasible to start buying hardware or otherwise..
i have not had good luck with moosefs/lizardfs/etc personally
If you're talking about swap doesn't that mean you want more RAM?
They do offer an expensive Optane SSD add-on. That should be faster than the regular nvme stuff.
The networking is not as important, each machine packs on around 1-3 GB a day so spread that over 24 hours to get the minimum connection speed required.
More memory would be nice but memory is far more expensive than storage, so if I can get SSD storage (fast enough for swap) at a good price I can expand storage. It's the storage I am running out of and need to increase, at the moment I see no solution but to buy more EX42 machines and dilute the load across another every period or so.
i am slightly curious what kind of application this is, or more specifically how much you give a shit about budget
if it is a budget bound thing, would it be cheaper to get a lot more nvme (or even sata machines at lower performance each) machines and spread load out even more as opposed to adding a 1TB drive (iirc these are ~35?eur, which is almost the cost of another dedicated server there, correct me if im wrong?)
I run a tonne of network nodes that update state and are required to hold state at all times, this cache grows at a decent rate per day. Grows at a rate of 1-3G in size per machine per day.
I have tested SATA machines and they didn't work according to my tests. I could give them a shot again but every SATA machine I've seen on offer has a lesser CPU (?) and given the first-month setup fees at Hetzner I'd have to fork out just to test a setup. I can see why there are fees, to prevent people spawning up and down like madmen, but doesn't help from a testability standpoint on a tight budget.
I currently add the 960 GB (sorry I accidentally called it 1 TB) which is 20.5 Euro, so it's cheaper than an entire new EX42 but you can only add one drive per machine
https://aws.amazon.com/ebs/features/
Things like this would be cool but io1 is so expensive, $65 for 1 TB.
You can use the Hetzner PX92, it is possible to add 8 drives..
What's god tier mean
@rmrfroot do what he said, fill that dedi with ssds and nvmes
What is the actual application? How much state are you talking about? If you're relying on swap you are probably taking a huge hit. You probably want a database with some sane cache management.
Yeah, it must be some new, super tier that's been introduced only recently.
(Anyway, I suspect that the OP is already gone.)
He went to see a God apparently.
Sounds like syncing multiple blockchains to me.
Yeah I figured it had to be something like that, whatever it means. How much storage is involved? Thanks.
Just don't cross the streams.