Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


SFTP backups and their management software
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

SFTP backups and their management software

k9bangerk9banger Barred
edited November 10 in General

I have been reading some threads on the forum about backups, such as The Backup & Restore thread, Best software: backup and Looking for storage for Borg backups and the impression I get is that some of the utilities don't require any server side software, ie all the backup databases are managed locally.

Which of those utilities have good local GUIs?

The part that has me confused is the deduplication and other optimization features that require access to the backed up data.

For that kind of optimization do they keep local copies they optimize then upload and replace those on the servers or do they have they ability to optimize them remotely using locally stored metadata?

I can't see how SFTP can optimize remotely stored backups unless there are some server side utilities or scripts helping.

I've got some idle storage space with Hetzner and I'm looking for software with a good management interface to put it to good use.

I just checked and realized that it was 1TB. I thought it was 500GB. What a waste.

Comments

  • Any views @vitobotta?

  • @k9banger said: how SFTP can optimize remotely

    SFTP is an extension of SSH

    @k9banger said: have good local GUIs?

    Why focus on GUI, many good tools are cli.

  • Petey_LongPetey_Long Member
    edited November 12

    I personally love rclone (via SFTP) . Multithreaded, can encrypt backups on the fly and it's super easy to get set up and running. After you get it tested out, add it to cron and call it a day. One thing that you may want that it doesn't have is de-dupe.

    GUI wise, I haven't done backups via GUI in a good long while but at my last contracting gig, Veeam was their go-to and that was straight forward to use. Features did include de-duping/compression.

    Thanked by 2yoursunny k9banger
  • @Petey_Long said: Multithreaded, can encrypt backups on the fly

    And it can combine all your storage vps and all your google drives/one drives/nextclouds etc. into a single huge backup volume. People are abusing free cloud tiers with it.

    @Petey_Long said: it doesn't have is de-dupe.

    By name and/or hash
    https://rclone.org/commands/rclone_dedupe/

    Thanked by 2Petey_Long k9banger
  • @egoror said: By name and/or hash

    https://rclone.org/commands/rclone_dedupe/

    Wow...you just made my day.

  • Checkout kopia.io

  • @k9banger said:
    Any views @vitobotta?

    Honestly, the only good GUI I've come across is the Web Edition for Duplicacy. It's well-designed and includes most of the key features from the CLI version (though not all). The issue is that even though Duplicacy has a solid reputation, I've had some problems with it that make me question its reliability. Right now, I'm using Restic CLI without any GUI on both my Mac and several servers.

    @Petey_Long said:
    I personally love rclone (via SFTP) . Multithreaded, can encrypt backups on the fly and it's super easy to get set up and running. After you get it tested out, add it to cron and call it a day. One thing that you may want that it doesn't have is de-dupe.

    GUI wise, I haven't done backups via GUI in a good long while but at my last contracting gig, Veeam was their go-to and that was straight forward to use. Features did include de-duping/compression.

    Rclone is a great and helpful tool, but it's not exactly designed for backups on its own.

    Thanked by 1k9banger
  • @vitobotta said: but it's not exactly designed for backups on its own.

  • @egoror said:

    @vitobotta said: but it's not exactly designed for backups on its own.

    I think a good backup tool should include features like deduplication, compression, encryption, snapshotting, retention management, and integrity checks all bundled together. Rclone doesn't offer all of that.

  • @vitobotta said: Rclone doesn't offer all of that.

    It actually does.

    Thanked by 1umzak
  • @egoror said:

    @vitobotta said: Rclone doesn't offer all of that.

    It actually does.

    Does it have all those features built-in?

  • @vitobotta said: Does it have all those features built-in?

    Yeah, except for creation of file system snapshots, but that's bundled with OS. Why does it matter? It's a lower-level tool.

    @vitobotta said: deduplication

    rclone dedupe

    @vitobotta said: compression

    'compress' remote

    @vitobotta said: encryption

    'crypt' remote

    @vitobotta said: snapshotting

    fs snapshotting tool + rclone

    @vitobotta said: retention management

    support differs from backend to backend, some have full versioning support like Backblaze, some are like "delete backups older than X days"

    @vitobotta said: integrity checks

    rclone check

    Thanked by 1k9banger
  • @egoror said:

    @vitobotta said: Does it have all those features built-in?

    Yeah, except for creation of file system snapshots, but that's bundled with OS. Why does it matter? It's a lower-level tool.

    @vitobotta said: deduplication

    rclone dedupe

    @vitobotta said: compression

    'compress' remote

    @vitobotta said: encryption

    'crypt' remote

    @vitobotta said: snapshotting

    fs snapshotting tool + rclone

    @vitobotta said: retention management

    support differs from backend to backend, some have full versioning support like Backblaze, some are like "delete backups older than X days"

    @vitobotta said: integrity checks

    rclone check

    fs snapshotting tool + rclone is NOT snapshotting done by rclone.

  • @vitobotta said: fs snapshotting tool + rclone is NOT snapshotting done by rclone.

    None of the tools do fs snapshotting themselves and are using os tools behind the scenes most probably.
    You are picking on rclone for no reason.

  • @egoror said:

    @vitobotta said: fs snapshotting tool + rclone is NOT snapshotting done by rclone.

    None of the tools do fs snapshotting themselves and are using os tools behind the scenes most probably.
    You are picking on rclone for no reason.

    First, good backup tools handle snapshotting on their own, so you don't need to rely on file system snapshots. Second, I'm just pointing out some facts and answering questions—I'm not criticizing Rclone. In fact, I use it myself, specifically for syncing object storage buckets.

  • @vitobotta
    This convo is pointless, opinions are already on the table. I'm not here to fight windmills.

  • If gui is a requirement, you should probably look at kopia or Duplicacy.
    If you don't mind paying a bit, Acronis or Veeam gets the job done.

    I have tried them all but I'm still relying on Restic. It's cli but extremely easy to use and has never failed me. There are various webinterfaces for Restic floating around but I've never tried them.

    Thanked by 2k9banger maverick
  • @egoror said:

    @vitobotta said: Does it have all those features built-in?

    Yeah, except for creation of file system snapshots, but that's bundled with OS. Why does it matter? It's a lower-level tool.

    @vitobotta said: deduplication

    rclone dedupe

    @vitobotta said: compression

    'compress' remote

    @vitobotta said: encryption

    'crypt' remote

    @vitobotta said: snapshotting

    fs snapshotting tool + rclone

    @vitobotta said: retention management

    support differs from backend to backend, some have full versioning support like Backblaze, some are like "delete backups older than X days"

    @vitobotta said: integrity checks

    rclone check

    Do the remote commands involve rclone downloading the file and reuploading again, ie like compress and crypt?

    That would be rather inefficient so I take it there are some utilities on the server side for those tasks. Does the rclone installation process specify what these utilities should be?

  • egororegoror Member
    edited November 16

    @k9banger said: Do the remote commands involve rclone downloading the file and reuploading again, ie like compress and crypt?

    "Crypt" and "compress" are remote types that can be layered on top of actual remotes. For example, using sftp+crypt will perform all operations over SFTP, but it will automatically encrypt and decrypt data as you transfer it between the remote server and the client. Also if you mount your remote as a local drive using rclone mount, you will see decrypted files.

    @k9banger said: on the server side

    Nothing is installed on server

Sign In or Register to comment.