New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Lol
For a large scale mirror operation, you're better off centralizing storage at some place like Hetzner Storage boxes : 3.99 Euro/TB-month for the 10TB plans.
(https://www.hetzner.de/hosting/produktmatrix/storagebox-produktmatrix)
Use multiple VPS with multiple threads to 'stream' the output to these central 'SAN's via a FUSE mount. Definitely have a RDBMS in there somewhere to keep records. (postgres!)
Split up crawl and storage to keep your project somewhat resilient.
It might take much longer than you think with big sites like Pornhub especially if they are employing intelligent traffic throttling for their viewer streams.
Porn industry is always at cutting edge of technology .
I thought of this as more of a datahoarder operation than a mirror. Thus the idea of owning all the hard drives and keeping them at home instead of online, once they are filled. Maybe you even want to think about Ultrium tape or the modern equivalent if there is one.
Isn't that some sort of semi-offline storage with no instant access? As the OP said
So there will need to be a website, presumably with real-time playback access to files.
"Unlimited" Hetzner - https://wiki.hetzner.de/index.php/Traffic/en
If the traffic limit is exceeded then the bandwidth will be restricted to 10 Mbit/s.
Depends on where the friends are etc. I mean there are always pornhub's own servers unless something happens. I always thought the concept of someone having 100s of TB of porn was a joke though. I knew a guy with 1TB and even that seemed like it would get hard to tell apart after a while.
Sounds exactly like PornHub. Teach them how to enter the website into a browser and you're set .
My suggestion: Download the videos you like and leave the rest on PornHub. They're not going anywhere. There's no need to waste hundreds of TB's on @Nekki's crazy hentai tentacle pony porn.
Thaaaat's debatable, recently the largest anime/hentai tracker nyaa.se went down, luckily some old DB dumps have surfaced and people are already building replacements; another tracker, bakabt.me, went private-only. Also YouTube channels get closed down and deleted entirely for silly copyright shit all the time. Stuff does get taken down from the Internet every day, never count on anything always staying there.
@vpsman22 use the SYS ARM nodes, the ones I tested had 500Mbits down (Capped to 10MB upload)
How did you manage to do it?