Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Backup rely?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Backup rely?

zhuanyizhuanyi Member
edited March 2013 in General

Hello all,

Here is my situation right now:

After my Drobo crashed again 3 weeks ago and I lost my personal backups that has been accumulated in the past 5 years (photos of all trips+all my work and other stuff), I finally ditched the idea of keeping my own backup at home (or use that as the only means of backup at least).

For my home internet, I have a 2mbps uplink home DSL (already upgraded recently from 512kbps), and if I am using that to upload directly to any of my backup servers (one in OVH, one with @Fliphost in NJ, it might take months to upload all my recovered files to the server).

On the other hand, I do have a VPS that is in a datacentre literally 10 minutes walk from my home and uploading there would be much faster than directly uploading to those backup servers. And the best part? The server is on 100mbit shared port which is way faster than what I have at home. Unfortunately I have very limited disk space on that server (5GB to be exact) and a local server with 1TB of storage would definitely cost way more than what I can afford.

So...my idea is, is there any backup tool I can use which would use my local VPS as a backup relay, so that I can upload a file to the local server, the local server then pass it to the backup server and once that is done, the local VPS will automatically delete the file and get another file from my home PC(or we can even do it at a bit by bit level).

Ideally, the tool will have some means to control my home bandwidth so that I don't burn through my 200GB of monthly bandwidth allowances as well. Also, I would prefer my personal data to stay private and not uploaded to another shared medium (dropbox, etc) if possible.

Any idea how it could be done? Or if you have any other suggestion on how to do it, please let me know as well. Thanks a lot! Renting a dedi for a month in my local DC is already out of question unfortunately.

Comments

  • @zhuanyi said: So...my idea is, is there any backup tool I can use which would use my local VPS as a backup relay, so that I can upload a file to the local server, the local server then pass it to the backup server and once that is done, the local VPS will automatically delete the file and get another file from my home PC(or we can even do it at a bit by bit level).

    I don't know of any tools that do that, but it's certainly possible. The thing is, if your home connection is 2Mbps and the server's connection is 100Mbps, the transfer speed will still be limited to 2Mbps, so I don't really see the point in doing this.

  • earlearl Member

    @zhuanyi said: datacentre literally 10 minutes walk from my home

    Which DC?

  • NickkNickk Member
    edited March 2013

    @Bogdacutuu said: the transfer speed will still be limited to 2Mbps, so I don't really see the point in doing this.

    Perhaps OP can get 2Mbps to the DC but not to his backup overseas servers overseas.
    In that case OP, why not just VPN to your local server? That should do the trick if international transfers are not maxing out your upload speed.

  • @earl said: Which DC?

    I'll PM you.

  • @Bogdacutuu said: I don't really see the point in doing this.

    I guess the point is I won't be getting the full 2mbps either way, so I was thinking may be it would be faster by going from home -> local VPS -> remote server route than home -> remote server route directly.

    The other purpose is so that the local VPS will upload to the remote servers and use the upload bandwidth x 2 (or if I decided to add more servers later on, x3, x4 and so on) rather than having my home Internet to do this.

  • KuJoeKuJoe Member, Host Rep
    edited March 2013

    Why not setup an NFS mount on your local VPS? Then just upload to your local VPS like normal and it would act as a middle man for the destination of the NFS mount.

  • @KuJoe said: Why not setup an NFS mount on your local VPS?

    My only concern is the lcaol VPS has only 5GB of HDD size so it has to backup a little, delete a little and backup another little... and I don't want to make the backup and delete part manually since I have close to a TB of data to backup.

  • KuJoeKuJoe Member, Host Rep

    Then use an NFS mount.

  • @KuJoe said: Then use an NFS mount.

    I am not sure if I get what you mean...isn't NFS mount just like another network drive on my PC? Or am I missing something?

  • You're probably better off having 2 backup servers at home, 1 as daily backup, the other ideally runs on Linux that you can just do a rsnapshot where you can keep long term archive in case you need to dig into a file that you edited rather than deleted.

    Remote backups don't make sense when you're bandwidth and data transfer restricted on your home link. The most you can do is upload really important stuff up on your VPS.

  • KuJoeKuJoe Member, Host Rep
    edited March 2013

    @zhuanyi You are correct, an NFS mount is a network drive. If you setup a network from from your OVH server to your local VPS then your local VPS would treat it like a normal folder when it is actually sending the data to your OVH server.

    Here's an example:
    Setup an NFS mount from your local VPS to OVH and create a /home/backup/ directory on your local VPS (point the NFS mount to /home/backup/).
    Copy files from your desktop to your local VPS while writing to the /home/backup/ directory, your desktop will think it's writing to your local VPS while your local VPS is actually writing to your OVH server.

    Desktop -> local VPS -> OVH

  • earlearl Member

    @Kenshin said: You're probably better off having 2 backup servers at home, 1 as daily backup, the other ideally runs on Linux that you can just do a rsnapshot where you can keep long term archive

    This would probably make the most sense! and it would not chew up any of your bandwidth.. even just getting a 2 TB external drive should be fine.

  • @Kenshin said: 2 backup servers at home, 1 as daily backup, the other ideally runs on Linux that you can just do a rsnapshot

    Would be nice if I have a bigger bedroom to squeeze in another server...to be honest the noise generated by Drobo has already made me want to throw it out of the window sometimes.

  • @earl said: 2 TB external drive should be fine.

    Ha, yes, I have never thought about using an external HDD! Thought it was outdated but it seems to be the most viable solution.

  • @zhuanyi said: Would be nice if I have a bigger bedroom to squeeze in another server...to be honest the noise generated by Drobo has already made me want to throw it out of the window sometimes.

    Try looking around the HTPC forums for ideas on how to build a silent PC for cheaps. An atom setup with maybe a 2 drive RAID1 would be OS flexible with quieter fans, yet draw less power and can be built small.

  • earlearl Member

    @zhuanyi said: Ha, yes, I have never thought about using an external HDD! Thought it was outdated but it seems to be the most viable solution.

    I have a couple of WD elements external (I think they come with green drives) they work great and no noise what so ever, if you need something quicker just get one with an esata port.

    you can pick up an external HD 2-3TB for around $120..

  • Buy 3x3tb harddrives, copy your stuff and place 2 of them in your home location somewhere. Put the third one (encrypted) somewhere at a friend and sync those manually half a year or prepare another drive to just swap them. You can have x3, x4 redundancy just by the one time fee of buying another drive and with the help of another friend :D
    Apartment explodes for some reason -> buy a new laptop and get your drive from one of those harddrive-buddies. Faster than your home connection, no caps, no recurring fees.

    "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." - Tanenbaum, Andrew S.

    Back to topic, you can still upload everything through your vps to one of your backup machines and rsync everything to the second box frequently from there.

  • @KuJoe said: Here's an example:

    Setup an NFS mount from your local VPS to OVH and create a /home/backup/ directory on your local VPS (point the NFS mount to /home/backup/).
    Copy files from your desktop to your local VPS while writing to the /home/backup/ directory, your desktop will think it's writing to your local VPS while your local VPS is actually writing to your OVH server.

    Now I see how it might work out...yes, this strategy should work, I can even install Crashplan on the local VPS and have the backup stored in the remote folder! Thanks a lot!

  • zhuanyizhuanyi Member
    edited March 2013

    @Kenshin said: Try looking around the HTPC forums for ideas on how to build a silent PC for cheaps. An atom setup with maybe a 2 drive RAID1 would be OS flexible with quieter fans, yet draw less power and can be built small.

    My Linux box is actually a Tonidoplug, which is really quiet, but the Drobo fan was killing me...

    My honest recommendation, don't get a Drobo, it crashed on me and because it is using some proprietary technology rather than standard RAID, there is almost nothing that would be able to recover the folder structure of the Drobo...

    @earl said: I have a couple of WD elements external (I think they come with green drives) they work great and no noise what so ever,

    Checking RFD for a good deal right now :)

    @chrisp said: Buy 3x3tb harddrives, copy your stuff and place 2 of them in your home location somewhere. Put the third one (encrypted) somewhere at a friend and sync those manually half a year or prepare another drive to just swap them. You can have x3, x4 redundancy just by the one time fee of buying another drive and with the help of another friend :D

    That is really redundant, thanks for the tips

  • flyfly Member

    do you have physical access to your server? if so, just sneakernet that bitch

  • @earl said: I have a couple of WD elements external (I think they come with green drives) they work great and no noise what so ever, if you need something quicker just get one with an esata port.

    These are awful to use standalone, the failure rate is ridiculous. I ran 12 pieces on a RAIDZ2 array, I had to send 6 for RMA after I turned off the machine for about a year (cold storage box) and turned up, did a disk check on each disk. Plenty of corrupted data too. Would never advise standalone use.

    @zhuanyi said: My Linux box is actually a Tonidoplug, which is really quiet, but the Drobo fan was killing me...

    My honest recommendation, don't get a Drobo, it crashed on me and because it is using some proprietary technology rather than standard RAID, there is almost nothing that would be able to recover the folder structure of the Drobo...

    I wanted to say replace the fan on the Drobo, but then again I'm well aware of propriety software causing havoc when I had to do a few data recovery jobs for a friend. Running your own linux box is the way to go.

  • @fly said: do you have physical access to your server?

    Not for my local VPS, I don't think any hosting provider will allow their VPS customer to attach a hard drive on the physical node

  • @Kenshin said: I wanted to say replace the fan on the Drobo, but then again I'm well aware of propriety software causing havoc when I had to do a few data recovery jobs for a friend. Running your own linux box is the way to go.

    I picked up Drobo even before I got myself into the VPS world and at that time Drobo attracted me because it uses thin provision and you can attach the disks with different sizes together, as well as the ability to have hot-swappable disks. I guess it worked well (although sending and receiving data is very slow) until now, when it actually crashes..

  • @zhuanyi said: I guess it worked well (although sending and receiving data is very slow) until now, when it actually crashes..

    Propriety software is a pain, it's also why 14 years into hosting I still refuse to provide MS Exchange, anything fucks up I have no idea what I'm going to tell my clients. On qmail/Exim (cPanel) anything goes wrong I know exactly why (most of the time), know how to fix it as well as how to prevent it.

    And you can never have enough of backups. For my main hosting business I do daily backup on a 12TB RAID10 server in the same DC, mirrored to backup archival servers (24TB RAIDZ2, 14 day+12 month snapshots) in 2 different DCs. Each of the 3 servers has failed before, but never together, so while it's a complete pain in the ass, it's pretty necessary.

  • @Kenshin said: Propriety software is a pain,

    Agreed, I have tried 5 different recovery software and the best I could get are some of the files, however I lost all my folder structures and file names and most of my code are lost as well.

Sign In or Register to comment.