New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Looking for high-storage server
Hi, I am looking for a high-storage server to host a full-replica Arweave node. I appreciate any offer or recommendation!
Requirements
CPU: AMD Ryzen/Threadripper/Epyc or any CPU with good RandomX performance
RAM: 16 GB+
SSD: 1x 240GB as a boot drive
HDD: Total 160 TB in either 4 TB (200 MB/s) or 8 TB (400 MB/s) drives.
Network: 1 GBit/s, 150 TB+
Location: any
[i] Storage can also be split across multiple servers
Comments
We have a 4 x 20TB HDD option or a 12 x 20TB HDD option available.
Also, some other options are available here: https://www.interserver.net/dedicated/large-storage-servers.html
Unfortunately the disks are too large for my use case. With 20 TB disks I would have to store 5 partitions per disk requiring a read speed of about 1000 MB/s.
@layer7 @labze
Do you have a bit more info? I'm confused by the HDD speed requirements: is that per disk or per vdev/array (mirrored/raid10, etc)? If it's per disk I'd say to start looking at 4TB TLC SSDs or refurb NVMe offerings. NVMe would definitely be attractive here but almost certainly talking ~$1k/mo server for 160TB of NVMe + EPYC/Ryzen.
With the storage being split note you have, can it be network attached? I.e. a less expensive Ryzen node up front on a private (10G?) VLAN feeding much cheaper dense bulk-storage units?
It's going to be relatively tough to find 40x+ 4TB HDDs into a chassis that is also a Ryzen but if you can split the storage and do separate dedicated physical storage units with your Arweave node on it's own physical box up front I think you'll get multiple providers able to offer you up something competitive.
I've sent you a DM with some options
The whole Arweave blockchain is about 160 TiB at the moment. It's split into partitions 3.6 TB each. One partition can cause up to 200 MB/s of reads and usually one partition is stored on one 4 TB disk. If I store 2 partitions on one 8 TB disk, they cause up to 400 MB/s of reads.
With storage being split I mean that Arweave nodes allow coordinated mining. I'm not entirely sure how it works but they probably synchronize over the (public) network.
Hi,
i think you have a wrong idea about what hardware can physical deliver.
A normal SATA drive, spinning, 7200 RPM will give you 200 MB/s under perfect conditions ( if ever ) -- no matter the size. The maximum you will find in the spec sheets of the vendors are around 260 MB/s maybe there are some with more, but never > 300 MB/s.
And you can imagine yourself where this numbers come from the vendor writes in his own spec sheet of a product.
In real life, you will not see that performance.
That given 4 TB drives with 200 MB/s are the maximum size your setup is allowed to have, and even then no hoster will be able to give you a performance gurantee for your usecase.
And just like @crunchbits already mentioned, for your 160 TB useable space, you will need in 4 TB drives 40 drives.... and we did not even talk about redundancy. Or do you want your 160 TB data to be completely lost if 1 drive fails?
So in short: We talk here about a very big amount of disks, and if its not SSD, you will not even have a guarantee that the performance will work ( except you will do some speed raid setups, requiring even more disks ).
Please give us some idea of your budget or if you really considered all this information that crunchbits and now me too gave you, before there can be serious talk about offers.
You write that you are not entirely sure how it works. Please allow me to suggest that you will ask people who actually run a working setup first to make sure that you will order something that will actually work. Everything else will be just a big money waste with a lot of bad mood for the hoster you choosed and for youself too.
@interservermike Man why I can't open your site? Didi you blocked me or my ISP
are you getting a Cloudflare error message with a rayid? PM me some more info.
as @layer7 explained
the only feasible way to get the IO throughput that you want is to go with 4TB disks
if you are looking for 160TB raw we are talking about 40 disks regardless of disk size
once you go above 36 disks in one server you will have to have some rare servers like server with jbods or one of 60 or 72 disks servers
if you are looking for 160TB usable after raid etc then you will need more disks even
I can get you a solution with all your specs with up to 72x 6tb disks,
if we are looking at 40x 6tb disks (you need 40x 6tb anyway due to io throughput) on such server will cost approx 600$/m (with 2nd gen EPYC servers)
adding more disks should be at 7$/disk/m up to 72 disks
i can only aggree that mb per second is just marketing, and also not very interesting.. tell us more about the use case. it also matters if its read or write, fsync or not..
Hello @leNic
We can offer you:
CPU: 2 х E5-2670V3
RAM: 128GB DDR4
STORAGE: 10×16Tb HDD
DRIVE FOR OS: 250GB SSD
PORT: 10 Gbps
BANDWIDTH: 1Gbps Unlimited
550$ per month
Write us with the link to this thread to get this offer.
Thank you for your reply! You are entirely right. Normally this kind of setup requires setting up a JBOD or some kind of SAN. That's also the route most Arweave miners go. But the Arweave node has a new and experimental feature called "coordinated mining". I wasn't sure how it works because it's not really documented yet. The nodes are effectively doing sharded mining with some minimal communication between them. In the end 5 machines with 8x4TB each would be sufficient and realistic in my opinion.
What should simplify it further is the fact that no redundancy is required at all. All data that is lost can simply be re-downloaded from the blockchain.
Hi,
that sounds like a cool and handy feature.
I would suggest: Get some VPS's and test this setup with some spread nodes.
As soon as you confirmed that its in general working, checkout some budget servers like hetzner or so, maybe they can give you something for 50 EUR or so.
But also here, as long as you stay in the traditional 12 hdd range that will fit into 2U chassis just like your 8x4 TB config, most providers who run their own infrastructure will be able to provide something for you. ( we as layer7 wont, as we dont have such small drives anymore ).
So check in a test setup if this coordinated mining really works, and if yes, then make a new request