New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
After this review, I finally settled on Borg as backup tool, and BorgBase + rsync.net for the storage. Backup and restore speeds are awesome.
I do similar, but I'm still using rsync and a bunch of scripts I wrote years ago. I keep meaning to evaluate borg & similar but never get around to it…
More than one copy though (deduplicating within snapshot sets, but keeping ore than one set). Complete deduplication is fine until there is some corruption then many versions of the file are lost. Though I backup to more than one location so maybe I'm being overly paranoid there!
Everything remote, and some things also on external drives. One of the off-site backups is soft-offline (the source machines can't see connect to it nor vice versa, it pulls data from the intermediate backup they push to, this way anyone somehow managing to get even root access to a main data location can't also affect all the backups from there).
I think borg is awesome (I use it myself), but one thing to be aware of is that as the amount of data you are backing up grows, it can start to consume a lot of RAM on the client for the hash/index cache.
rclone
Have you tried restic and does it also have this problem ?
And mounting an archive and listing/restoring files from it can get very slow as the number of files increases, even with plenty of memory and high performance storage. I'm dealing with a Borg backup of roughly 45M files, and it's painful to mount and browse it.
Never tried it, so no idea. Before borg, I had my own custom solution using rsync and
--link-destto external drives and a backup schedule that involved swapping the external drive with one at work so I always had an off site (away from home) backup. Got fed up of plugging in the drives every time I wanted to kick off a backup, so got a storage server about 4 years ago and decided to try out borg. Worked well enough that I've not tried anything else, although I keep meaning to try out borg2 for its cross-repo deduplication.Just bash scripts
Just does whatever needs doing, dumping the DB be it SQL or Mongo or whatever, getting all files needed, tarballing and uploading to multiple buckets
One bash per project, running in its own dedicated backup container per project
Veeam Backup & Replication Community Edition (https://www.veeam.com/products/free/backup-recovery.html)
It’s a reliable brand, and for my personal use the free edition’s 10-workload limit is more than enough. It’s basically a ‘set and forget’ solution.
Uhm.... I have been comparing Borg and Restic more since I had some time and Restic is consistently faster with both backups and restores to/from the same destinations. So perhaps I will stick with Restic... not sure yet
tar -czvf
If we’re not strictly talking about VPS and including general compute, then part of me wanted to say Arq (https://www.arqbackup.com/) as it’s nice and versatile tool for Mac.
But, I think I’ll go the Backblaze personal setup. It’s my pick less so for the software but the fact the service has a fixed cost that doesn’t go up with consumption, at least as far as I’m aware. My laptops use is fairly moderate given I only have so much storage there, but I also have it running on an UnRaid NAS backing up about 15-17 TB data for a few dollars a month. Any other cloud solution I’ve seen outside of CrashPlan (which throttles speeds as you backup more and therefore is less practical) would cost a magnitude more per a month for me to backup the same volume of data.
Apparently there’s other people out there using the service to back up 50TB of data or whatever for the same $9 USD a month. I don’t expect this to last forever, but while you can use the service to backup 10’s TB for $9, it gets my pick.
For VPS, using Restic however.
Arq is nice but it's ridiculously slow compared to Restic for example. That's why I stopped using it.
I've read that restic uses large amounts of memory, much more than borg. Borg has it's own server-side process, whereas restic doesn't.
I haven't noticed that because I have been using it on personal Macs with lots of memory and servers with 8 GB of RAM minimum for small repositories. But for my personal Macs and servers with larger backups I have a minimum of 32 GB of RAM so I didn't notice any memory related issue, but I have heard about it. I wonder if it was an issue with older versions of Restic?
Actually, this is an important step, whatever backup system you use.
Very few backup systems (I believe ZFS snapshots might be the only exception, but I don't use ZFS) will capture a database in a safe and consistent state. For all my systems with databases, I have a script that runs shortly before the scheduled borg backup, and dumps out to a new file with a date/time in the filename. The script clears out the older backups, so there's only about a week's worth, and these are the things that are included in the borg incremental backup, not the actual live database.
The server-side process doesn't actually use all that much memory for borg. It's heavily weighted on the client.
Actually, another gotcha with borg is that if the client index cache somehow gets corrupted, it can take a very long time to rebuild it before anything happens. This is quite rare though.
Backups are pretty near the top of the list of things that you DO NOT want to set and forget. Test them regularly, if only to keep yourself familiar with the process so that you can repeat them when you're panicking because the backup source is down.
Restic !
Before borg backup runs, my script runs mysqldumps, does a check and analyze on the databases, then it stops mariadb, copies the database files, and restarts mariadb. Borg backs up the copies.
Years ago I used to use mariabackup to do hot backups, but I had too many problems when I needed to restore from those, so now I just shut the database down for a few seconds to copy the files.
In Veeam Backup & Replication, “set and forget” refers to designing your backup jobs so they run reliably and automatically with no ongoing manual intervention. In other words: you configure the job once > it runs on schedule > it maintains retention > it handles cleanup and maintenance by itself. Veeam is known for being stable once configured. Admins often say “set it and forget it” because Veeam jobs don’t require daily babysitting unless something fails.
You could use
mariabackup --backup --target-dir=/path/to/your/backup/without stopping instead of mysqldumpI do the mysqldump without stopping, but I stop it before copying the database files.
I used to use mariabackup to do hot backups without stopping, but ran into issues when I tried to use them (corrupt database, wouldn't start). This was years ago, though. Maybe it was user error, or who knows. But that's why I don't use it anymore.
I did some more testing and Restic is consistently faster than Borg, so I am sticking with Restic.
I totally agree. I test restores every 2-3 weeks
Why you copying database files if you have sqldump file? sqldump already contains ful backup of your DB. 🤷♂️
The copy of the database files is what I'd normally restore. That's quickest.
The sqldumps let me restore individual databases, if need be. Also, an alternate form of backup in case the first one has problems (like before, when I had issues with mariabackup).
Prayer
duplicacy, in a docker container, easy to use and reliable, can restore specific file/folder as I need. Just it, no more, no less, no bloat, trash functions I never use.
Yes, this caught me by surprise the first time I tested Borg on a relatively low-memory VPS with millions of small files. I assumed the client side would be pretty RAM-friendly, and the server (destination) would need to be more beefy, but it was the opposite.