New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Isn't it regular egress fees?
So $1 to store, but $90/TB if you ever want to restore...
Retrieval fees are different for S3 tiers. Deep Glacier is the colest of tiers. Downloading files cost more than any other tier, but storing them is cheaper than any other tier.
Price is also highly dependent not only on tier but also on DC location.
It's meant to be used either for complience data that needs to be stored but is rare to ever be downloaded, or, for backups of backups of backups.
The data is still as secure in terms of redundancy as I understand it.
@webontop Where did you see that limit? I can't find it.
I have 500GB storage and 4TB like BW used.. and request like 30M
where should I go ? all Object storage either charging hugh for BW or Request.
Terrahost S3 doesnt charge for BW but 1TB/$25
Yeah for this use case it's probably the cheapest and safest option (cheaper would imply finding a super cheap storage VPS, not very safe and/or reliable in the long run + not maintenance-free). Gonna experiment with it, thanks for mentioning this (relatively) new S3 storage class.
In case you need to retrieve the files, it rarely takes more than 12 hours as I understand it, is that right?
object storage & cheap = not compatible. Average price for not awful object storage without hidden fees around 10 usd / TB. With hidden fees - starting from 5 usd / TB.
Below 5usd/TB - crap. Always. No exceptions.
The only option for you - HDDs & dedi server with your own gbit/s ports.
For example: https://www.hetzner.com/dedicated-rootserver/matrix-sx
4 x 16 (real 14.5TB) = ~ 58TB of disk space.
58 TB disk space * 5 usd/mo = 290 usd / mo (minimum) required if you will use any object storage. While the dedi server & a lot of space costs ~100 usd /mo.
i.e. 3x times cheaper.
This is the cheapest option at all.
The problem - abstraction above the hardware & easy management & abilities to easily push/pull files from the server or easy to integrate into working apps / sites / services.
And the guy above already said about https://github.com/minio/minio and tools like that.
Do not waste money for jumping from one to another object storage. You wont find anything good below 10 usd / TB.
https://github.com/seaweedfs/seaweedfs#introduction
Thanks for this. I tried to search for minio replacement, and seaweedfs seems promising
Just a question, why did you want to replace minio?
Yeah, that’s how I understand it too, but I just started using it, so no clue really.
Nonsense.
S3 Deep Glacier runs at less then 1/10th of what you consider the lowest possible price for good storage and is among the most redundant options avaliable, much more so than a dedicated with RAID, the data safety isn’t even comparable.
And don’t come crying about retrieval or bandwidth fees, the whole point of Deep Glacier is that it’s like an insurance, a place that 100 % has your data if everything else turns to shit.
A backup plan shouldn’t involve regurarely going to Deep Glacier to get files.
Deep Glacier has a huge clientele of companies who have actually calculated what’s cheapest considering data safetely, and found Deep Glacier to be that, at 1/10th of your suggested minimum.
Here’s a Reddit thread with some calculations where you surely could learn a thing or two: https://www.reddit.com/r/DataHoarder/comments/10uh8l3/aws_glacier_deep_archive_is_far_superior_to/
Classic. Sorry, if i was younger i will argue each your point.
But i do not see any sense anymore. But answer to you, that this is theory shit.
While i have real experience & practice. And this is my own experience & my own attempts. Yes, i can be wrong. But your message out of context at all, or you did not read topic at all. Please re-read, and you will understand, that your answer is nonsense.
Correct me if I'm wrong but it looks good on paper but you should also consider Erasure Code thingy which if applied for most redundancy, should reduce your available storage space by 50%. Then your pricing will be about the same as those $5-$6 per TB providers. Plus they're most likely using SSDs/NVMes while you're using HDDs with this setup. At that point you can save the headache of managing your own storage for a dollar or two more.
Hey man, can you give an example for $2/TB or similar? I'm interested as I'm looking for a way to store my data and I'm comfortable with building my own system using MinIO or something similar.
The SX series of servers from Hetzner is not the most cheapest in terms of $/TB. For example on their auction they have 10x10TB for 100EUR. If you do two disks worth of redundancy, that comes around to 1.33 EUR per TB. Even if you do 50% worth of parity information at the file-level that's still only 2EUR per TB. You could buy two of these servers in different locations for more redundancy if you want, and will finally be the same price as $5/TB.
Also which providers are doing $5-6 per TB on SSD/NVMe? iDrive e2 seems to do $10/TB on their object storage SSD tier.
I must've calculated wrong. With a rough calculation I thought it'd cost about $4 per TB using Hetzner's dedis with HDDs.
Aren't those providers using SSDs? I thought everyone were using SSDs like Wasabi, Backblaze, Storadera, Scaleway etc.
That's probably true with their non-auction servers.
Backblaze uses SSD only for boot: https://www.backblaze.com/blog/ssd-edition-2022-drive-stats-review/
They are among the most competitive for $/TB on object storage, so I don't think anyone else is doing pure SSD storage for a similar price.
So SSD only for the main system? I wonder if Scaleway is also using HDDs because I'm using Scaleway's Object Storage well over 3 years in their NL location, uploading ~500GB of data every month (my triweekly backups) (and usually downloading about 100GB per month) and I've never seen anything that fast. It is extremely fast, I must say. I'm only paying about 5 Euros for this. Though I'm uploading these data from a Hetzner dedicated server (AX61). Also I proxied their Object Storage with a 5 Euros Cloud server from same location in their network so the download is basically free but it's not as fast as directly downloading from their S3.
No I don't think so. They would specify that if that was the case. And I suppose they would be more expensive.
I'm not a legal expert, but the AGPL license requires derivative works to adopt the AGPL license as well. If I build a SaaS using Minio, I'm concerned it might be considered a "derivative work," even though I don't modify Minio's source code and only use it as a dependency for my app.
ChatGPT argues that in this case, my software and Minio can be seen as separate works. However, to err on the side of caution, if there's a production-ready alternative with a more permissive license like Apache and still easy to maintain, I'd choose that option.
This is the chatgpt response
If you are using an unmodified version of Minio under the AGPL license as a separate component and not integrating it directly into your software's code, you are not required to license your software under the AGPL. In this scenario, Minio acts as a standalone application that interacts with your software through its API, and the two can be considered separate works.
However, if you integrate Minio's source code directly into your software or create a derivative work by modifying Minio, then your software would be subject to the AGPL license. In such cases, you would need to release your software under the AGPL license and provide access to its source code.
If you wish to avoid the AGPL license requirements for your software, you can:
Ensure that Minio is used as a separate, standalone application, communicating with your software only through its API.
Look for alternative object storage solutions that are distributed under a more permissive license, such as the Apache License 2.0 or the MIT License, which do not impose the same copyleft requirements as the AGPL.
It is important to carefully review the terms and conditions of the software licenses involved in your project to ensure compliance and avoid potential legal issues.
Scaleway glacier storage is $2/TB with no transaction fees and no cost to restore to hot storage. The best I've found for my use case of just backing up personal stuff.
This list is great!!
Obviously it's "theory shit" because we're litterally talking about comparing the price of S3 compatible storage, you except me to explain that with bananas and apples, or maybe with push-ups?
It's not bad but just note:
I'm backing up a small CloudPanel installation (and storing a few PDF files) to R2 and have been doing so for 6 days.
These are my stats:
So if they count it the same, I'd be 26x over the free tier limit in 6 days.
Note that I've been performing "clones" several times per hour simply for testing, so your usage could be MUCH less, but even if your usage is much less it could be limiting.
I've just switched from cloning every 10 minutes (I've changed several times to analyze the process and the different S3 providers) to settle for exactly which providers I want to use and how (who gets the top spot and who's the backup and who's the backup of the backup) to cloning every 3 hours now so I hope for a drastic turn in the number of operations to the better.
My project includes giving users access to files temporarily and specifically in an automated fashion, and having played around with R2, S3 and a local Swedish provider based on CEPH, I have to say the way Cloudflare integrates with R2 Workers is really really good.
In a few minutes I created a worker that can generate access keys to be used in URL parameters to access specific links on the R2 custom domain, a worker that accepts/verifies these tokens on a per URL (and therefore per file basis) and a custom domain with WAF rules to only allow my server IP to be able to access the "code generating" worker.
b2?
Great! And how about Backblaze b2?
Yeah it's a good option and cheaper than aws deep glacier if you need to restore, and their setup seems to be rock solid even though data is hosted in one DC only (while with S3 deep glacier, "data is redundantly stored across multiple Availability Zones that are physically separated within an AWS Region").
I mean it depends.
If you do need to restore, but:
1. It’s very very rarely
2. And/or you’d then only need to restore some smaller specific files
S3 could still be way cheaper because of 1/2 storage price.
It’s very hard to generalize, no AWS class and no provider will be the cheapest for everyone.
Although Scaleway don’t seem to have any operation fees (such as reading/writing fees like R2 etc)?
Have you tried CloudFlare R2? It seems they have not clarified about the bandwidth speed, the only information I got from someone on Reddit is 200 Mbit/s. This is quite slow with me. Please update if they are faster now.
I currently do not think their egress is really free and unlimited.