New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Or have the users mail one to you. Exporting 10TB is always going to be an "contact us, engineer gets involved, and we bill you hourly if we can get away with it" thing which thankfully won't happen very often.
Circling back on this. I haven't found an option better than restic at the moment. It ticks enough of the boxes.
@ypmLA77zcs and the other restic enjoyers, would you expect to be able to manage restic users, or maybe just 2 logins:
Feedback
Francisco
Applogies in advance if it's a stupid idea or if it has already been mentioned over the previous 2 pages:
Hetzner Storage Box is cheap.
A user can configure a Storage Box sub-account that is limited to one directory in their Storage Box.
Could a user provide that account info so you can configure automated backups to that location on a monthly (or weekly basis - however is reasonable)?
That does allow FTP download so interruptions should not pose a problem (correct me if I'm wrong).
It also allows users to easily move backups out of the path that is accessible to the provider, so no problems on that end would put backups at risk.
MXroute Reseller could be configured via DirectAdmin's backup system to export to Hetzner Storage box and as far as I can tell, that still works fine (for new setups, the option is gone, so you would need to bother tech. support to set it up I believe - not 100% sure, haven't set up new backups recently).
Relja
A user can configure a Storage Box sub-account that is limited to one directory in their Storage Box.
>
'Push to users place' then gets bottlenecked by their remote end. There's always some rate limit that'll exist (how fast the remote end can accept, object limits, etc). It goes poorly and then clogs other backup tasks.
We have a handful of users on the shared side that have their own jetbackup tasks and half of them cause huge log jams for everything else, even our own internal nightly backups, because their service gets rate limited, or their remote SFTP/SCP box is in iowait hell.
There's also the chance that we can't do incremental backups with their backend (again, rate limits, missing features, etc), meaning we end up burning tons of bandwidth (and the run time) for that task.
With restic we'd treat those backups as the 'remote backups' we currently do with
rsync, so we aren't using anymore bandwidth/resources than we already were.Some users don't want 'all' of their data either, they'll want a subset. Restic would let them cherry pick.
Francisco
Hetzner storage boxes support SSH/SFTP and work fine with restic. And I don't think there is any S3 provider with cheaper storage prices than Hetzner's 2.5€/TB with unlimited ingress/egress (+ fast speeds to a VPS in the same DC).
You could run a DB replica on a Hetzner server and use that for backing up to Hetzner storage boxes, just an idea.
I get 2Gbps mounted via sshfs on my cheapo VPS:
My point being is doing 'push' sucks. Its a huge burden on us to maintain support and quirks. Oh this provider uses an older version or some funny port setup or requires some weird SSH key options. If you want to setup a cron on Hetzner to pull down from the redis dump, all the more power to you.
We're not charging anything for access to this. At the moment we could very well tell people to pound sand, use imapsync + rclone. We're trying to find a happy middle ground
Francisco
Ah, I thought you were talking about restic 'push' before and that you wanted the user to "own" the backups.
But yeah, they can just do that on their end if they want via cron. Keeping the backups within your own infra is definitely simpler and more reliable.
...and hetzner offers a supreme bf deal at the moment!!11 @Hetzner_OL
where can i get 1TB hetzner stoage for 2,5€/TB/month?
ok... 2,xx starts with bx21 package.
math genius over here
My dumb question @Francisco so forgive me if it is, but can't you just allow something 'simple' like ftp for them to download so that most users can automate downloading their own backups and maybe do it where all the data even if its separate files are in a single directory (sub directories in there are fine too) so it can be recursively done by whatever ftp client they are using? Now the reason I ask this is not to circumvent ssl r whatever other encryption mechanisms are out there but I have had really slow speeds trying to use sftp to download stuff before (not your servers, general stqtement) and I am not sure where the bottleneck was and for that reason I didnt suggest it. I might have missed what you needed from the original post so forgive me if I did. Maybe you meant sftp because sync supports that incremental update feature?
Uber Drive SSD's
I'm joining the conversation late, but... no.
Any sane user will want the data encrypting before shipping, so you need to set that up on a per-customer basis, agree the type of encryption, the key, etc. Then you've got the cost of the drive, the shipping, and the man-hours to write and verify the data. Then you've got the time it takes to securely wipe the data when (if) you get the disk back. Then you need inventory tracking so that you know how many times each disk has been written and shipped.
If you have to ship a disk, the customer keeps it and you charge accordingly.
Honestly when this thread first started we weren't running ZFS, so giving 'snapshots' wasn't really possible. We've since then change our storage configuration 2 - 3 times and finally have ZFS snapshots.
Technically we could just extend our current FTP to instead expose the ZFS daily snapshots folders, and people then pull whatever.
Francisco
Nice! BTW I really like the service thus far, working well, thanks for bringing this out!
Glad you like it
thanks for the ideas.
Always open to feedback.
Francisco
IIRC the initial thought was for users to somehow get access to existing restic repositories which you're creating for your own backups anyway.
Re: "wanting to give access to a client (only available to non resellers)" - who would be a non-reseller's client? I'm for the simplest setup that allows someone to access the repo(s) for disaster recovery purposes - ie one master login (separate repo / domain) would be enough as far as I'm concerned...
More importantly - how do you suggest the restic password is created / distributed?
How would this be secured in transit? FTP doesn't hold a candle to restic for this IMHO...
We aren't using restic yet. At the moment we do 5 minute snapshots along with a pile of hourly & daily backups via ZFS. Every night we push all data to another datacenter as our 'remote' via
rsync.If we moved to restic for users, then we'd abandon the
rsyncand just use those same repositories for our own remote backups.With SFTP a user would login and pull backups on their own, however they want. We'd expose the ZFS daily snapshots. Auth would most likely be a separate database of users, with clients being able to add/remove FTP users to their repos (so they can have their own 'master' login and then give their client access or whatever).
SFTP is nice since there's no CPU overhead in offering it, we're just exposing data we're already hoarding. We don't need restic trimming backups out or things like that, ZFS does that already.
It also (mostly) solves Akash's issue with normie users not being able easily access the data.
Francisco
I thought you said early on that data is encrypted at rest - is that still the case? If so - would decryption happen on-the-fly when accessing the snapshots, then re-encrypted for ssh tunnelling? How efficient would this be?
And I'm not sure ZFS can replace a proper backup strategy, but I'm far from an expert...
ZFS encrypts at rest. You still have to mount/decrypt to access. That's the story with everything. ZFS does it with minimal overhead.
Where is SSH tunneling coming from? You'd be going to the data via SFTP.
ZFS handles the local stores for faster restores when needed (someone gets popped, rogue staffer deleting their data, etc). As I said though, we have a full backup taken of the entire node and sent to another datacenter.
Literally what every backup solution does.
Francisco
I thought the whole point of a managed provider was that the customer doesn't need to get involved in admin minutiae such as regular backups. Because if I'm outsourcing my email delivery then I'm probably not competent enough to be with full backups anyway.
That being said, if I did want access to such a backup it would likely be for compliance/legal reasons. In that case, I would think a read-only searchable archive would be more use than a binary blob that I then have find someway of accessing.
If I was big enough to absolutely require some sort of off-site regular 10TB backup, I would probably suggest you write it to LTO then ship it to somewhere like Iron Mountain.
...
...
Perhaps "tunnelling" was wrongly worded, I was merely referring to the fact that sftp is running on top of ssh
This is because users are asking for it. Some users already use imapsync to pull down emails just fine.
Francisco
...
Problem with imapsync is (quote from its page): "Imapsync can't backup nor restore email messages to or from a local directory. Imapsync works only with IMAP accounts, which always belong to some IMAP server."
I'd very much prefer I get a backup copy stored on a filesystem
You’re still a bit hooped though. Smartermail stores things in its own proprietary format, not as flat eml files on a disk.
We have asked them for either a tool to dump the files, or the file structure so we can try to build such a tool ourselves, and opensource it if allowed.
There is some paid products for dumping/extracting grp files.
Francisco
So this is a non issue. They actually supply a free tool for dumping GRP -> HDR/EML files.
http://www.smartertools.com/downloads/utilities/groupfileemailextractor.zip
I'd need to test if it works on .NET on Linux, it might. I did ask for the source just incase so we could tweak/recompile it.
So I guess the only concern here is how to provide access to the data
I've asked our dev. Failing that, restic was the original plan anyway.
Francisco