Storage providers with hourly/minute billing?

fragpic

I have to make some changes to a few servers/VMs which might corrupt the data attached to them. So, I'm looking for a way to take a temporary backup (around 5Tb total) of the data until the changes are done and validated.
I don't expect it to take more than 2hrs total, so is there any provider or service that supports sftp/rsync etc. and offers minute/hourly billing?

seriesn

S3 storage ?

fragpic

@seriesn Does it support rsync/sftp? I'm not sure how to send and retrieve data from S3-like storage. Also, atleast for AWS, S3 seems to be billed monthly.

Erisa

A lot of S3 providers will advertise their monthly price but instead bill it based on hours (Or sometimes seconds) consumed. Amazon is one of these providers.

For something S3 but a little on the cheaper end I enjoy Scaleway Object Storage (FR/NL/Poland) or Backblaze B2(US/NL). Both bill based on hourly usage, you get your invoice monthly of course.

One thing I will say is watch out for egress(Download from them to you) charges. Both examples given have them and theyre consumed as a resource so downloading your 5TB if it comes to that will cost you a fair bit. Note also that Scaleway does not charge egress to their instances and even encourages the use of one as a proxy.

As for shifting files with it, no you can't use rsync or sftp. I usually use rclone but there are others like an official S3 tool that can do simple push/pull/sync jobs. Can't find a link to the one I used right now though.

As for SSH compatible, er. I honestly have no idea. The only ones that I would recommend (Hetzner Storage Box and rsync.net) are most definitely not billed based on hours. S3 is sort of the standard for that territory unfortunately.

Tejy

Deploy SFTPGO on any of your VPSes, and create an user based on a S3 bucket.
The cheapest S3 providers are Wasabi (but there a min amount which will be billed).
Otherwise, Scaleway could be a good idea! (to avoid bandwidth fees, I advise you to deploy SFTPGO on any Scaleway instance, as traffic isn't billed when internal).
It will allow you to get an FTP space based on a S3 bucket (and you will pay only for what you're using).

Erisa

Had no idea SFTPGo existed, that sounds tremendously useful for my Scaleway S3 usage.
Thanks @Tejy!

fragpic

@Erisa

S3 sounds good but transfer charges are quite high which negates the purpose of hourly billing

Thanks, @Tejy.
Per my understanding, SFTPGo is pretty much like rclone correct?
As long as rclone supports the S3 provider, there is no difference between the two right?

NetDynamics24

We will offer you this for free. 5 TB FTP/SFTP storage for a few hours.

perennate

AWS could be an option since the ingest charge is free. But yeah if you end up needing to download the backup it might cost 0.09*5000 = $450.

Have you checked https://www.hetzner.com/cloud ?

Hakim

Scaleway object storage+instance. No data transfer fee.

cazrz

@fragpic said:
@Erisa

S3 sounds good but transfer charges are quite high which negates the purpose of hourly billing

Thanks, @Tejy.
Per my understanding, SFTPGo is pretty much like rclone correct?
As long as rclone supports the S3 provider, there is no difference between the two right?

I think SFTPGo and rclone are different. Rclone is a client whereas SFTPGo is a server.

fragpic

@NetDynamics24
Thanks for the offer. I'll probably PM you this one time but I'm also looking for a long term solution that I can use whenever I have to do this.(Probably once every 6months)

@perennate

This is a real possibility so egress will definitely screw me over.

@Hakim
This seems to be the best option for now. Per my napkin math, it should cost around a $1.

marvel

If you just need 5 TB temporary I can offer you a weekly plan for €20/week incl transfer which you can prepay and rent on and off.

Let me know if you're interested.

NetDynamics24

@fragpic said:
Thanks for the offer. I'll probably PM you this one time but I'm also looking for a long term solution that I can use whenever I have to do this.(Probably once every 6months)

I will consider to create weekly or daily plans for such cases. But it would only be $5 per week or $1 per day?!

Tejy

@fragpic said:
@Erisa

S3 sounds good but transfer charges are quite high which negates the purpose of hourly billing

Thanks, @Tejy.
Per my understanding, SFTPGo is pretty much like rclone correct?
As long as rclone supports the S3 provider, there is no difference between the two right?

That's why, I advise you to deploy a Scaleway cloud instance, even the cheapest, because traffic between their instances, and S3 buckets are free!
And yes, SFTPGo will create a FTP/SFTP/DAV/SSH server powered by an S3 bucket!

fragpic

@NetDynamics24 said: $1 per day?!

This would be perfect. Also PMed.

@Tejy
This is my backup plan since it requires some setup. Thanks for this solution.

NetDynamics24

@fragpic said:

@NetDynamics24 said: $1 per day?!

This would be perfect. Also PMed.

That's correct. You can try it free of charge for 2 days.