New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Cost effective solution to store around 10TB data for a year
A friend of mine have around 10TB of data (mostly web crawled) to be backup up. Please suggest a cost effective solution. Renting out a cheap server and 5 x 2TB HDD's will be around 70-100$ per month I guess. Are there any cheaper solutions? The data transfer rates will be very low, once all data has been moved there. Just FTP access or any remote mountable storage will be fine.
Came across free plans from Chinese companies offering 10TB and even 36TB, but I am looking for a solution, that clearly takes into consideration the use of entire space being purchased. Let me know if anyone has experience with any such providers.
Comments
hubic.com
€5 per month for 10TB from hubic.com looks great. Has anyone tried putting up that much data there?
Bandwidth seems to be limited to 10Mbit. That's gonna take a lot of time to push 10TB.
+1 for hubic I've put a few TB on there before
I believe you can move data a lot faster to hubic from within ovh, you could get a kimsufi for a month or get someone to lend you use of one and pull the data to hubic via the ovh box.
OVH has weekly rentals for "testing"
I have a spare atom that I got about 5 years ago and never used, your welcome to it but iirc its only 100mbit nic
Any specific reason for that "belief"?
Yep, I looked in to hubic myself a while back, I think around 18 months ago but don't quote me on that, when it was considered beta iirc.
I remember reading something about the internal OVH network <> hubic speeds on the OVH forum that indicated you will get better speeds within OVH.
Will have a scout around and see if I can find it.
Would make sense because all of the OVH network servers have very high internal bandwidth. Just make sure it's in one of their FR datacenters.
Right..... forget hubic then
It seems they disabled the limits for a while a long time ago but from what I am reading now some people are struggling to even reach 1mbit ever mind 10..
They should just double the price and make it a usable service I guess, still cheap at twice the price.
First of all unless your clients data is either in China or the West Coast it will take forever to upload. They force you to download their software so you can download your own files again. Also they will never delete your files and use them for their own gains.
Don't bother.
Back then it was not limited to 10 Mbit as it is now. And it doesn't matter where you upload from, it's even smart enough to limit you to 10 Mbit total if you upload from two servers at the same time for example.
I suspect the extra speed 18 months ago won't apply anymore, but it would be interesting to know if anyone can test it. Last year I used to get fast upload speed from within France but outside OVH's network. They've recently tried (are trying) to reposition Hubic to make it less attractive to big data shifters partly by enforcing the 10Mbs limit both ways, unlike some reports I do get ~10Mbs most of the time from FR and almost always 5Mbs from UK (my home connection is only 5Mbs up)
If you're thinking of giving it a go now, bear in mind they're changing the underlying SWIFT storage from multiple copies to a striped parity model to reduce costs and have had a LOT of reliability issues these past few months, it's greatly improved today, but you'll still experience some upload failures on larger files >1GB.
It is by far the cheapest storage option around, provided you research and accept the limitations. There's no FTP or SFTP access unfortunately, on Linux I can recommend HubicFuse which works reliably for me, but needs some care setting up rsync if you don't want to see double uploads or repeated syncing e.g. '--inplace --no-times --size-only --whole-file --no-perms --no-owner --no-group'.
Well there's also Amazon Cloud Drive -- and supposedly not as speed-limited as Hubic. But I'd say it is hard to trust any "unlimited" storage to not kick you out for storing just 2-3-5TB or whatever for "abuse".
https://www.hetzner.de/ot/hosting/storagebox/bx40 at 9,90 € monthly maybe?
comes with some snapshots which I'd use to store data that is probably never needed again and really only to be backupped in such a snapshot instead of a live accessable storage ;-)
depends on your real use case though.
regarding hubic I also remember there have been higher rates at around 18-12 months ago, but I highly doubt that now internal connection would make a difference, as they limited all connections to 10 MBit not long ago. would be interesting to know, though, maybe I am going to test it later on ;-)
Indeed, or for storing files other than 'photos, videos, music, documents' or any perception of commercial use. At least you get what's printed on the tin with Hubic, if you can live with the speed cap.
I had considered Amazon when Hubic seemed to be going down the pan a few months ago, but the unlimited plan wasn't available in the UK then like it is now, thanks for the heads-up.
I have uploaded 8TB to hubic. Upload speed was not limited, but download speed is limited to 10Mbit. Also beware, that there is no convenient way to upload. I used hubicfuse and have some faulty blocks in a few 30gb backup files.
Maybe you can have a look at https://www.online.net/en/c14
I didn't use this service so far, but for 10tb it comes down to a fair price of 20€. Also this seems suitable for backing up data with the need to alway access it.
You didn't notice the "operation" cost?
Interesting, I wonder if that had anything to do with the segmenting that goes on with such large files.
I've not experienced any corrupt files myself, most of the files I retrieve are compressed database backups only about 1-2GB in size, but am minded now to check some other larger files, prompted by your post.
Which reminds me to mention segmenting as another limitation for using rsync with Hubicfuse, it [rsync] gets completely confused with these split files, so you have to ensure you only rsync folder structures with files less than 'segment_above' size.