Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Any convenient ways to backup my servers?
New on LowEndTalk? Please Register and read our Community Rules.

Any convenient ways to backup my servers?

FrankRuanFrankRuan Member

I have a bunch of servers and there're different kinds of services on it. I want to back them up.
My plan is to Tarball my program data into a tar file and export my MySQL database. Zstandard compress them and then RSYNC to my backup server.
But that will be really, really complicated and what if my program data is 50% the disk space... The tar file will just BROKE the whole filesystem.

Thanked by 1Halo11

Comments

  • AbdussamadAbdussamad Member
    edited August 7

    don't use tar. just use rsync. use rsnapshot to create incremental backups automatically to save disk space.

    Thanked by 1vyas11
  • ralfralf Member
    edited August 7

    Have a look at borg backup: https://borgbackup.readthedocs.io/en/stable/

    It has a few problems in my mind, but is great for backing up a machine to somewhere else.

    The problems really come when you are backing up a massive dataset as it is uses quite a lot of memory while it's running. It also reads every file every time to calculate a hash, so it can also become quite slow. It also has good aspirations to de-duplicate files, which sounds perfect when you're backing up multiple servers to the same place, except different servers can't share the same repository, so it's only useful for de-duplication on the same machine, which is usually less frequent.

    But I'd still highly recommend it. I've stuck with it in spite of my few reservations as it's better than anything else I tried, and better than my previous hand-rolled solution (which was based on rsync and --link-dest).

    Thanked by 2Erisa i4P1
  • I use UrBackup which has a nice GUI and can backup Linux and Windows machines

  • @Abdussamad said:
    don't use tar. just use rsync. use rsnapshot to create incremental backups automatically to save disk space.

    Oh, rsnapshot is snapshotting the whole filesystem? Heck yeah! Thanks dude

  • ralfralf Member

    Pretty much every tool - tar, rsync, borg, etc, all have options to not cross file system boundaries.

  • AstroAstro Member

    I have heard veeam is great

  • @Abdussamad said: to create incremental backups automatically to save disk space.

    Remember that when you do this, if a file does not change between snapshots it will be one file referenced many times, so a bad sector on physical storage will affect all copies of the file.

    If you only have one backup location, keep more than one snapshot chain on it (the secondary doesn't necessarily have to be as granular as the primary, but I prefer it to be). Better still keep snapshots in two locations.

    @ralf said: better than my previous hand-rolled solution (which was based on rsync and --link-dest).

    I still use hand-rolled solutions, though I've never moved from rsync-to-master-then-“cp -al”-if-changes-detected to using --link-dest (introduced in ~2005?) because I'm old and don't like change! (actually there are some fail scenarios where --link-dest might cause more breakage than my hand-rolled setup, though they are likely both rare & minor, and easy to recover from if detected)

  • edited August 7

    @FrankRuan said: if my program data is 50% the disk space... The tar file will just BROKE the whole filesystem.

    You can pipe the output of mysqldump (and similar backup operations) directly into the compressor, so you don't have this problem as early as 50%. Better still arrange your backup operations to send directly to another machine (or at very least another drive) so the volume being entirely full shouldn't stop your backups from working (though the lack of space will cause other problems, of course). If everything must be on the same drive (you have absolutely no money for a proper backup at all) then use LVM or other partitioning to split it up so the live volume(s) being near full won't break backups (or cause backups to fill the remainder of the space on the live volume(s)).

  • ralfralf Member

    @MeAtExampleDotCom said:
    I still use hand-rolled solutions, though I've never moved from rsync-to-master-then-“cp -al”-if-changes-detected to using --link-dest (introduced in ~2005?) because I'm old and don't like change! (actually there are some fail scenarios where --link-dest might cause more breakage than my hand-rolled setup, though they are likely both rare & minor, and easy to recover from if detected)

    One good thing with borg is that if you move files around, it doesn't break the incremental backups.

    TBH no idea when --link-dest got added, but I've been using it for about 11-12 years. I think I've said somewhere else, I used to have two external drives with encrypted LVM volumes with the same name. My script mounted them and then found the last directory created under that host's subdirectory and passed that as --link-dest, but otherwise did a full incremental backup of the entire machine. I plugged the drive into each machine at home in turn and ran the script. One drive lived at work and the other in the car boot, so whenever I remembered, I took the drive out of the car and ran the backups, and then put it back. Also, when I remembered at work, I'd swap the drives. Worked well for me for about a decade until I started expanding from just one VPS and then looked into using borg to copy between them all instead.

  • @ralf said:

    @MeAtExampleDotCom said:
    I still use hand-rolled solutions, though I've never moved from rsync-to-master-then-“cp -al”-if-changes-detected to using --link-dest (introduced in ~2005?) because I'm old and don't like change! (actually there are some fail scenarios where --link-dest might cause more breakage than my hand-rolled setup, though they are likely both rare & minor, and easy to recover from if detected)

    One good thing with borg is that if you move files around, it doesn't break the incremental backups.

    TBH no idea when --link-dest got added, but I've been using it for about 11-12 years. I think I've said somewhere else, I used to have two external drives with encrypted LVM volumes with the same name. My script mounted them and then found the last directory created under that host's subdirectory and passed that as --link-dest, but otherwise did a full incremental backup of the entire machine. I plugged the drive into each machine at home in turn and ran the script. One drive lived at work and the other in the car boot, so whenever I remembered, I took the drive out of the car and ran the backups, and then put it back. Also, when I remembered at work, I'd swap the drives. Worked well for me for about a decade until I started expanding from just one VPS and then looked into using borg to copy between them all instead.

    Can borg backup to remote locations (i.e. rsync, SFTP)

  • @Astro said:
    I have heard veeam is great

    Guess what, why the fuck is this thing so expensive.

    @bluehairminerboy said:
    I use UrBackup which has a nice GUI and can backup Linux and Windows machines

    Seems good, I'm going for it. The webui is great

  • cpsdcpsd Member
    edited August 8

    I use mysqldump to make daily backups in the same partition of a +100GB database, tar without compress and scp to my backup server.

  • _cece_cece Member

    Did someone tried the Proxmox Backup Server? It’s not only restricted to be used for the Proxmox VE, and offers a probgram for backing up while servers…

  • ralfralf Member

    @FrankRuan said:
    Can borg backup to remote locations (i.e. rsync, SFTP)

    borg usually backs up either to a repository that's mounted in the filesystem or via ssh.

    In the ssh use, you run the borg program on the both ends, and a single ssh user can support multiple repositories using ssh key authentication to determine access control. The approach is better than the filesystem for several reasons, not least that you can choose to make your backups append only so that old data cannot be wiped (if for instance your client got hacked)

    I don't think it's possible to use bare SFTP or rsync directly, but I guess you could mount them using fuse and backup that way. Borg backups are designed to be used in-place though, so you probably don't want to think of it as a replacement for rsync.

Sign In or Register to comment.