New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
today is hot ... r u wearing any pants?
Do you already have a client lined up that committed to using 100TB of storage (based on your calculations)? If not, I would just start with small with a scalable/flexible S3 provider. The usual suspects (Backblaze etc) have already been mentioned in this topic.
Also, I’m really curious how the math would work out if you opt to rent a single storage box in terms of throughput. What is the expected amount of concurrent users for example? Is 1Gbps still sufficient then or do you need to look at 10Gbps? Imho this doesn’t scale.
This is unobtainium.
You want bulk storage at bulk storage pricing, but you also want the performance to upload a lot, if i understood correctly maybe as much as 100x ?
This is beyond semi-cold storage solution in that case.
Going for dedis etc. for low cost is then the right solution.
Otherwise, you'll have to pay dearly.
Free lunches still don't exist, someone somewhere has to be for what's being utilized.
Our combo recommendation would be caching / frontend: https://pulsedmedia.com/minidedi-dedicated-servers-finland.php
and bulk storage: https://pulsedmedia.com/storage-boxes.php
You'll get well below 5$/TB pricing, but will have to do quite a bit of sysadmin work.
Supposedly someone should explain to him what vc is. He tries to cut corners everywhere, but I'd bet that wouldn't be a problem for someone else's money.
B2 with caching on a fleet of MD machines might actually be one of the most cost effective solutions with minimal administration, especially if those 2.5Gbit uplinks come along. Easy to start cheap, easy to grow.
That's curious. Except B2 storage alone is 6$/TB, why not just use our storage boxes? You could even have 2x for mirroring for the same price.
tbh, a lot of great advances are exactly because someone wanted to cut corners and costs, one just have to realize what is possible in terms of first principles. Getting something for nothing is not.
Engineering for lower cost however is getting something for very little -> ie. use dedis, frontends / backend storage, add a rinse of sysadmin to get fraction of the cost per capacity.
The more capacity, the less the relative overhead is.
There you go and have a flywheel of ever shrinking costs.
Because with B2 you don't need to worry about provisioning more boxes, you can spread your data across 2 locations, you likely don't need to worry about the shared nature of the storage boxes and the fact that I've not managed to max 1Gbit between a storage box and a MD machine.
The storage boxes are great for storing large files, backups etc but I wouldn't use them in a production environment.
But yes, if you want to go absolute minimum cost without the outlay of large dedis and as much (but still more) sysadmin time, a storage box would likely work well enough.
Good starting point would be:
2160p, AV1 10bit, 10Mbps
1080p, AV1 10bit, 3Mbps
1080p, H264, 6Mbps
720p, AV1 10bit, 1Mbps
720p, H264 2Mbps
480p, AV1 10bit, 0.5Mbps
480p, H264 1Mbps
360p, H264 0.5Mbps
AV1 = SVT-AV1 preset 4-6
H264 = x264 slow
With very bad connection (less than 1Mbps) you deliver 480p to most devices apart from older iPhones which don't have AV1 support in Safari.
With 3G speeds or for people that need to save mobile data you deliver 720p for most devices.
With ADSL/congested 4G speeds you deliver 1080p to most devices.
For people that can see 2160p (TVs, PC etc.) you deliver 2160p AV1 10bit. You don't deliver H.264 at 4K because it makes no sense.
For audio you can have seperate audio tracks with HE-AAC v1 48Kbps stereo and AAC-LC 128Kbps stereo.
These bitrates are starting point, with CQ and restricted bitrate you can actually go a lot lower with bitrate while maintaining good picture quality.
If you combine all of these bitrates you'll get ~24Mbps.
TV series like Friends has 22 minutes per average.
24 * 60 * 22 / 8 / 1000 = 3.96GB for one episode.
24 episodes will require 95GB of space. This is one whole series.
This will cost you $0.6/mo to host on Backblaze B2.
100TB is completely crazy number, if you really will host that much you will make milion dollars from sales.
And this is exactly why you get object storage, so you don't need to care if you need 10TB, 100TB or 200TB.
With object storage you pay for what you use and you don't have limits. You can use 1TB and pay for 1TB, you can use 69TB and pay for 69TB.
If you will host 10TB it will cost you $60. If you really want to save costs then migrate data off object storage when you have 100TB of these videos and you can fill up 100TB server.
It makes no sense to futureproof rented dedicated server for which you pay monthly, you can just migrate it in one day with no extra costs. It's not like you are buying this hardware and then need to sell it or whatever
If these bitrates seem too low for you then take a look how well AV1 behaves at ridiculously low bitrates: https://bitmovin.com/demos/stream-test?format=dash&manifest=https://cdn.bitmovin.com/website-assets/tos/stream.mpd
(set resolution/bitrate in player settings)
Bitrates I mentioned are really "starting point", you can go lower for that in most content.
And remember
It's not a good idea to go with high bitrates for VoD platform, it dissatisfies 60% of users (mobile data, congested network, offline playback, buffering, way slower seeking, increased heat output) just to satisfy 1-5% of users. If you want to still do it you can add 20Mbps 4K HEVC 10bit stream (AV1 looks bad at high bitrates, H264 wastes too much bits).
@AXYZE
Thanks mate. Awesome answer. You explained lot of things to me. I felt like read a nice tutorial
Well, yeah. But especially on a smol scale/operation it's hard to have requirements this low. Reading a bit into that topic, he woulda seen. It's not like he'd need to raise the budget massively if he - what it seems like - a SaaS-esque solution and not e.g. a colo'ed storage server with some sort of nvme/ssd cache.