Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Cheap way to store a few TBs of data - Page 3
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Cheap way to store a few TBs of data

135

Comments

  • tar czvf - / >/dev/null. It's freeeeeeeeee (⌐▨_▨)

  • Hard disk at friend's house, and carry one around also.

  • geekalotgeekalot Member
    edited March 2015

    @linuxthefish said:
    Hard disk at friend's house, and carry one around also.

    ^^^ This, with the following edit:
    Encrypted Hard disk at friend's house, and carry an encrypted one around also.

    Edit: Initially I said "this should be a trustworthy friend" but the friend needs to be able to trust you too as they won't know what is on the disk.

  • deadbeefdeadbeef Member
    edited March 2015

    @geekalot said:
    Edit: Initially I said "this should be a trustworthy friend" but the friend needs to be able to trust you too as they won't know what is on the disk.

    You don't even need to bother with a friend, buy some physical storage space and presto ;)

  • rm_rm_ IPv6 Advocate, Veteran

    deadbeef said: bother with a friend

    That's the spirit. :)

  • @deadbeef said:
    You don't even need to bother with a friend, buy some physical storage space and presto ;)

    Cost of storage space usually >>> cost of renting server/VPS.

  • @MuZo said:
    Not true, click on learn more on the buisness plan: 1tb for business is 25$/month and 100tb are 1400$/month.

    Click on the 'pro' plan instead of the 'business' plan ...

  • MuZoMuZo Member

    @wiffel said: Click on the 'pro' plan instead of the 'business' plan ...

    yes? the pro plan is 1TB, no option to pick 100TB

  • I know you said no NAS but the price might make you consider it. 16TB for £370. You would have to find a company that will accept it and reship outside of the UK though. Has USB and USM so you dont need to connect it to an actual network to make use of it

    Thanked by 2Nyr deadbeef
  • @geekalot said:
    Cost of storage space usually >>> cost of renting server/VPS.

    By physical storage I meant "a box in the bank or other provider that offers such services". Not a server :)

  • @rawrwriter said:
    I know you said no NAS but the price might make you consider it. 16TB for £370. You would have to find a company that will accept it and reship outside of the UK though. Has USB and USM so you dont need to connect it to an actual network to make use of it

    This is very nice!

  • wiffelwiffel Member
    edited March 2015

    MuZo said: ... , no option to pick 100TB

    Ok, I understand now.

  • @deadbeef said:
    By physical storage I meant "a box in the bank or other provider that offers such services". Not a server :)

    @deadbeef, I understood what you meant. Maybe safe deposit box etc might very well be cheaper for this amount of storage.

    Either way, KISS (keep it simple stupid) and patience likely need to be liberally applied to make a cost-effective solution for this. I had the same challenges with keeping redundant copies of high-volume personal data.

    Thanked by 1netomx
  • n0myn0my Member

    Megasync

  • emgemg Veteran
    edited March 2015

    If price is @Nyr's main concern, then it seems that local hard drives are the obvious solution, unless the storage rental is very short term. (If the storage is very short term, then where is the data going after that?) There is a higher up front cost for hard drives, but it won't take long before the cost of renting storage would overtake that - perhaps a few months.

    With local storage, @Nyr also avoid issues with bandwidth. How will @Nyr upload terabytes of data? Even if @Nyr has access to a 100 mbps pipe (say, 10 Megabytes/sec), it would take more than a full day to move each terabyte of data. Can @Nyr find consistent high-speed access while on the move? I doubt it.

    The next question is redundancy. How does @Nyr back up the data to avoid loss? A storage service may offer remote backup (at what cost)? Perhaps @Nyr can take the risk if the remote storage solution is chosen, but if @Nyr is carrying hard drives in a mobile situation, I would want a backup of some kind.

    If @Nyr chooses local storage, then the remaining questions are:

    Which configuration (RAID or not)? RAID is not cheap, and has its own penalties and risks. It does not seem like the right solution for someone on the move who is trying to keep costs at a minimum.

    Which interface (it depends on which interfaces are supported by @Nyr's MacBook)? Does your MacBook support USB 3? If so, then that is the cheapest solution that can handle terabyte sizes with ease. Otherwise, you are faced with choosing between Firewire (800, hopefully) or Thunderbolt (expensive!).

    RECOMMENDED SOLUTION:

    My gut tells me that the simplest, cheapest, best solution is to buy three external drives in the smallest multi-terabyte size that fits @Nyr's needs. External drives in the 2, 3, or 4 Terabyte size are not too expensive.

    You can find very portable bus powered 2 terabyte external drives on Amazon (in the US) for under $100 each. Here is an example: Western Digital My Passport Ultra 2 TB Portable Hard Drive, Black - $93. I gave each of my children the 1 terabyte version of this drive to take with them to college for personal backups (not that they use them, darnit). They are very small and light, and seem reasonably durable. They would be ideal for someone on the move like @Nyr.

    Designate one drive as "Main". Designate the other two drives as "Backup 1" and "Backup 2". Be sure to label each drive so that no chance of mixup might occur. Each night, copy the new data to "Main". Whenever there is time, clone Main to the current backup drive. When an opportunity presents itself, leave the current backup at a friend's place, and start cloning Main to the other backup. Whenever you visit the friend's place, swap backup drives. If you do not have a friend with a drop off point, then you can simplify with only two drives, a Main and a Backup, but the risks are higher if you keep them together on your person as you move around.

    To clone drives on a Mac, I recommend the "Carbon Copy Cloner" program if you can afford it ($40). You can create "Backup Tasks" that make it less likely that you would accidentally overwrite your main drive. SuperDuper! is a less expensive alternative. Since you are on LET and clearly have 1337 admin skillz, you can simply roll your own clone/backups from the command line using rsync (or ditto or asr) for free.

    That's what I do at home. Three drives. Carbon Copy Cloner creates encrypted disk images of the family computers on the "Main" backup whenever convenient (e.g., when the children come home for a break). Carbon Copy Cloner to clone Main to Backup1 or Backup2. When my brother comes to visit from far away, we swap Backup1 and Backup2 and he takes the updated backup to his home. Repeat.

  • @emg said:
    ...

    Yup, that's basically it: physical drives + encryption + periodic sync/backup + multiple locations.

    "Poor man's NAS" at its best

    Thanked by 1netomx
  • emgemg Veteran

    P.S. I looked at Google Apps for Work. The main problem is that the $10 "unlimited storage" plan applies when you have more than 5 users. Otherwise, @Nyr would be limited to 1 terabyte of storage with a single user $10/month plan. The other issue is whether @Nyr has access to sufficient Internet speeds and/or data limits. That's why I proposed the external drive solution.

  • geekalotgeekalot Member
    edited March 2015

    @emg said:
    ...

    @emg, I think your solution is spot on. It is what I currently use for personal data:

    • HP Thin Client running Debian + LUKS
    • Add-in eSATA card (supports 4 external drives)
    • (USB only if run out of eSATA ports. Depending on thin client model, that could be 8 USB ports)
    • External HD in various types of enclosures supporting eSATA, USB & sleeping/spin-down (especially the quick "docking station" ones so you can swap them to other locations/carry just the drives)
    • All this with < 20 watts of power consumption when running (depending on number of drives spinning of course)

    Actually, makes for a good NAS/digital media streaming replacement as well.

    Edit: I have also tried more "conventional" NAS solutions like a hacked DNS-323, but always return to good ole Debian + LUKS on a thin client every time.

    Thanked by 1Nyr
  • FalzoFalzo Member

    probably internet speeds shouldn't be first concern, as @Nyr probably wont move the whole data around every single time. most probably a high percentage of that precious data will only be there for beeing accessed in rare cases..

    so a solution with three external disks involving cloning them to each other will be much more complicated in aspects of time and handling. this becomes even worse if you drop your backup at friends and such, which you then will visit once a week or so, only to hang around for hours cloning your disk? ;-)

    don't get me wrong, nothing against a single external disk for backup purposes. but to have data accessible while moving around, automating a lot of syncing or backup tasks, why one should consider doing this mostly manually offline?

    depending on the data itself and how they should be accessible, I'd recommend something like the mentioned cheap dedicated servers probably with self-hosted seafile or alike.
    to add redundancy some other space to mirror to would be great, but that will raise the pricetag remarkably...

    Thanked by 1Nyr
  • Any idea to store at home with WD My Cloud Mirrorr maybe? It can be set as RAID 1 or 0.

  • I would recommend buying a USB 2.0 2TB HDD and use it for cold storage. Reliable & cheap and you can keep a copy an another USB 2.0 2TB HDD once it gets a few years old.

  • emgemg Veteran
    edited March 2015

    Two Additional Comments:

    1: In my opinion, USB 2.0 is too slow to backup a multi-terabyte drive. It would take days. Considering that USB 3.0 drives are nearly in the same price range, it does not make much sense to buy a USB 2.0 drive, especially if the OP's production Mac is the only one available to do the backups.

    If the OP's MacBook does not support USB 3.0 (older MacBooks do not), then I would think about getting Firewire 800 or Thunderbolt (expensive) drives instead. The problem with that solution is the OP's need for an inexpensive solution.

    2: Today Amazon announced unlimited cloud storage for $60 a year. See:

    https://www.amazon.com/clouddrive/home

    Then the question is whether the OP has access to sufficient Internet throughput speeds (and possibly data limits).

    Thanked by 1deadbeef
  • TinyTunnel_Tom said: I was considering sending a Pi to MSP and a HDD and use that. Would work well

    Sorry for my ignorance, i haven't been around lately, who is MSP?

  • deadbeefdeadbeef Member
    edited March 2015

    @Janevski said:
    2: Today Amazon announced unlimited cloud storage for $60 a year.

    Awesome!!! Have you found any mention about BW limits/cost (i.e. is it like Glacier?)?

    Edit: Just noticed there's a thread about it.

  • NekkiNekki Veteran

    @Janevski said:
    Sorry for my ignorance, i haven't been around lately, who is MSP?

    MyServerPlanet. Relatively new UK-based provider, owner is @MSPNick who seems a nice guy.

    Thanked by 1Janevski
  • emg said: or Thunderbolt (expensive) drives instead

    IIRC if you have TB on a MB(A/PRO) you also HAVE USB 3.0.

  • deadbeefdeadbeef Member
    edited March 2015

    @William said:
    IIRC if you have TB on a MB(A/PRO) you also HAVE USB 3.0.

    Not on early 2011 MBP. Comes with TB and USB 2.0. Unless you mean with some kind of tunneling through TB?

  • mikhomikho Member, Host Rep

    You can also get a 2TB usb-stick for 11-13 usd on aliexpress. Cheapest one I've found so far.

  • @Janevski said:
    Sorry for my ignorance, i haven't been around lately, who is MSP?
    @Nekki said:
    MyServerPlanet. Relatively new UK-based provider, owner is MSPNick who seems a nice guy.

    Thanks! I can surely sort something out for you, shoot us a PM i'll see what I can do.

    Thanked by 1Janevski
  • InfinityInfinity Member, Host Rep

    @mikho said:
    You can also get a 2TB usb-stick for 11-13 usd on aliexpress. Cheapest one I've found so far.

    They're fake though?

Sign In or Register to comment.