All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Backup Your Backups?
For the past year I've been data hoarding and this obsession has translated into backup paranoia. I now feel the need to backup everything, not just my precious data, and to backup my backups. I don't know whether somewhere there might be an IRC channel or mumble server acting as a support group for this issue.
Anyway, I'm curious what your backup topology looks like. I've got a machine in RAID 10 running bacula, but not sure whether I want to spend quite a bit to get "enough" storage. And I have syncthing running on some machines, although I don't use it for the purpose of backups; that's just a side benefit. Do you backup literally everything, like even your noncritical VPS and shit like that? I want the ultimate backup solution with uber redundancy, or a support group.
Comments
Pray.
I was going to ask if the top keeps spinning in your universe or not, and welp..
For my own use I back up critical data - and depending on how critical it is, I might use multiple backup solutions for same data so even if the backup dies then I'll (most likely) be safe.
For customer data I do backups which are stored on servers with raid 6 (somewhat safe), I do not do backups of backups, simply because the customers doesn't pay enough for it to even make sense - and customers should by all means keep backups of their own data anyway.
I use a raided server as "backup" and really important stuff (which typically isn't that big. In my case mainly some financial/tax stuff, letters, and source code) is burnt onto DVDs. LTO might be more professional but its cost can hardly be justified for a home setup plus I quite like to have critical stuff on a safer medium than magnetic.
In the company it's something similar but with LTO and with millenium media rather the DVD. Plus, of course, a stringent backup regime (incl. having backups at different location).
Are you already on the unhealthy psycho side? I don't think so although I myself don't go that far.
I remember when you could use Google Cache for a backup. :{
When was your last refill ?
None of my projets ever rely on user submissions/uploads and I never handle any type of media files.
Nothing fancy, all the projects files are already stored in bitbucket + local folder synced to drive in addition of monthly backups to an external drive.
Compress and upload the daily logs to drive + downloaded to my laptop daily (which goes into the monthly backup).
For the databases, it depends on its size and importance. I can't really backup a 12gb database daily, same goes for a 20mb db. In addition to the size and importance, the frequency of its updates dictates the frequency of the backup. Still working on this, I'm open to suggestions.
If something goes wrong, I just rebuild using my config files and own wiki, 90 mins and I'm up and running (I'm slow, both ways).
If you're so worried, you should make sure your host(s) is(are) reliable, use their backup + db replication + multiple offsite backups on different locations + cold storage + local backups (and backup those too if you're feeling adventurous) + RAID10 everything + shitload of notifications and monitoring (seems like this will be part of your next obsession if it wasn't the cause of all this).
On top of all that, you should be really organized to keep track of all that (remember to pay your bills on time). All that backup will require a lot of resources on each step, from storage, cpu to bandwidth and time in addition to your local electricity bill and hardware (+ mental health).
Also, are your servers secure ? Are you sure ?
What if something goes wrong somewhere in your setup, how long is it gonna take you to restore everything back to normal ? Also, which backup exactly and from which source ? What if the backup files are compromised ? Will you be running some random tests on all the backups at every step/location ? What if you're not available, can anyone else do it instead of you ? Is everything documented ? Do you also plan on encrypting everything ?
Again, it all depends on what you're handling, its size, importance and updates frequency (also accessibility).
If after all of this you're still worried, get some data insurance + pay someone else to handle the backups, you suck at it.
If you do not deduplicate backups that is your own fault.
Unless you need the performance a RAID10 for backups is rather useless, putting the HDDs in a RAID5 (at 4 disks) or a RAID6 (at 5+) is more useful.
No, why? Rsync does not cost anything aside of BW and a bit CPU - especially if you have AES-NI on both sides. ZFS deduplication costs RAM only on the backup server side. Snapshots in ZFS do not cost anything resource wise (i have a single machine with more than 500k snapshots running fine).
A few minutes, because the backup server cannot write on the nodes so there is no risk of something breaking anywhere aside of there.
How should it be? I checksum the backup after creation and store the value, if it does not match it will not restore, there is no risk of anything being compromised. Snapshots cannot be manipulated after creation (aside of deleting).
ZFS can do encryption internal (BSD/Solaris) or use dm-crypt'ed disks (ZoL).
Why not? You run a local server, live sync it as slave and take incremental snapshots (way A) or snapshot the underlying filesystem (way
.
I backup my backups inside my backups
Sweet, thanks a lot!
Any resources or guides for these ?
Yo dawg, we heard you like backups so we put backups in your backups so you can backup while you backup.
It's a normal MySQL read slave, then depending on DB layout simple mysqldump works to get a file, deduplication is... more... complicated.
download the backup to own PC,upload it to google drive or Amazon or something. Done.
>
Why not directly to google drive/amazon? Unless I mean you want to also keep a copy that is self-hosted on your pc?
I don't think there's a support group for this :P
Here's my backup strategy:
Plus testing the integrity of any of these locations backups at random times.
I also want to introduce a delayed replication node in there at some point in the future.
I've been shoving static backups into Online.net C14 Intensive for a while (0.5 eurocents/GB/mo) but might give OVH Cloud Archive a try at some point. I haven't worried about anything like real time replication for personal stuff so far. I do rsync stuff between servers if it's fresh enough to not want to deal with C14 yet.
If I were to pull my head of of discords arse then I would bring online a lowcost backup service. its designed for backups with high ingress bandwidth. The plans are single disk per customer so if it fails you loose your shit, but then you buy two and mirror them via rsync
Low cost as in $5/TB on offer
That is low, but not as low as it could be.
I only host my own stuff, so nothing is too major. I have a couple of web/email servers, name servers, a dev/test server, a storage server, and some bits and pieces at home. My servers run individual backup scripts to dump necessary stuff like databases. Later, I have a storage VPS that uses rsync to pull those backups and configurations from those servers. Once a month I create an archive tarball of each system backup on the storage server and push it to a cheap reseller cpanel account on a different provider. Occasionally I download those archives to a portable HDD at home.
The push to the cpanel account may seem overkill, but my other services are with one provider, and I've previously had a provider literally disappear overnight. I'm not happy to leave all my stuff to the fortunes and abilities of someone else.
rsync + mysql slave so i have failover server than hourly mysql and daily file backup replicated to 2 or 3 place
The backup of my backups his backup is my backup.
Here's my backup scheme for 2016, it's evolved a lot since then relying on more off-site backups and less local backups.
I only backup mission critical data and usher a prayer for the rest just like @bluesega. Surprisingly enough, it works in more than 80% of the cases.
Sorry but that's unimpressive for storage junkies seeking new pastures. Look at C14, Hetzner Storage Box, the various budget storage VPS's, Kimsufis on their regular promos, or plain old Hetzner auction servers. It gets more interesting if you have some kind of RAID, but even still you're competing both with bigger providers at comparable prices, and with other LET hosts at lower prices. $5/TB non-raid is maybe of more interest to customers of your other plans, for backup and quick access.
Subreddit /DataHoarder
man , you need to see someone...
just backup on multiple location.
backup1 on USA , backup2 on UK, backup3 on DE, backup4 on FR, backup5 on NL, backup6 on blah blah blah.
unless this world comes to its end , it's almost impossible to loose all your backups on all these different location.