All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Downloads web site, how would you host/CDN it?
Hello,
I have a small stupid web page where I host Android ROMs which makes about 100 downloads per day for 30GB/day = 1TB/m of BW on average. Nothing serious or resources consuming.
Now I'm thinking of expanding creating a full featured web site.
My problem is: how do I host it in a way that I load balance the resources for the first months without wasting a fortune?
At the beginning it will a very niche website so I would expect to get 1000 downloads per day for various months, that translates to 100TB/m and that's a lot even for medium VPS. Of course this could lower especially at the very beginning.
I'm looking for a solution that, when the site is on experimental stages, I spend the least possible and then I can upgrade to more resources. I know I described basically a cloud service but I think they cost too much for me now and I'd like to play around with custom configurations 
I thought of these options:
1) buy 3 VPS's from fliphost (3 x 50GB HDD Unmetered 100Mbit for $18/y each) for storage and use them in round robin solution (mirroring or just host one ROM copy in one server) + a front end VPS that hosts the actual website with database.
Advantages: cheap price for storage/BW, I suppose I get a LAN to transfer files between local VPS's
Disadvantages: no multiple locations, i'm not sure if the cpu is enough considering the price
2) cloud service, I checked digital ocean and vultrm, they're not very storage-oriented I get huge amount of ram and cpu I don't need so I don't think they're worth the money for this project. Vultr has the storage plans but the BW is low.
Advantages: I suppose I get a LAN to transfer files, multiple locations, easy way to upgrade in seconds
Disadvantages: I pay for RAM and SSD I don't need, high prices for the usage
3) big dedicated server and run all in there. I'm not sure if this a good idea but I could get a lowend dedicated server like online.net or kimsufi, considering the huge amount of RAM/HDD and BW I could host everything there.
Advantages: All in one place, good prices for what you get
Disadvantages: All in one place, no redundancy, the website could slowdown the "mirror" function or vice versa. Difficult to upgrade or add locations.
So what do you think? Could you give me an advice on how you would do it at the beginning for the least money (not the just the provider but how you would actually organize things). I'm also willing to switch provider/solution one time when the site gets big (if) but the second time must be definitive so I that I can upgrade accordingly without changing everything.
Also it would great if these services won't count inbound traffic!
Thank you!
Comments
100dl = 1tb/month
1000dl = 100tb/month ?
EDIT: I was on my cell phone so couldn't explain a lot.
If you estimate 10TB a month bandwidth, you can always get yourself a 10 euro box from online-net. Slicebox, with a $27/month deal, is also a candidate for unmetered 100mbps.
And you also have the choice to purchase a cheap box from
GVHas a mirror in case you have some trouble with your main server. But as far as I can tell, online-net or kimsufi are quite stable and have a superior uptime.Vultr has storage plans
First I saw this I would recommend using a full cloud like http://www.ovh.ie/cloud/storage/ but after doing some maths it would get very expensive in terms of bandwidth.
So my idea now is using a dedicated server or two on backstage, with all the roms, and a vps with the site and database (if it's more or less static, you can even cache it and run it on lowend). But even that, if you are looking in using 100TB of bandwidth every month, man, that's some big thing, I think you may should consider a custom plan and paying a little (much) more.
I wish you the best luck doing this!
I don't know how much disk space you're looking for but SeFlow's Cloud One plans come with a ridiculous amount of bandwidth for the price (€3 for 100 Mbps unmetered, €7 for 300, €28 for 500):
http://www.domflow.it/oneplans.php
And that's on a true cloud with two-datacenter redundancy, too.
YGM
I did something like this for video downloads (podcast). I use Route53 latency-based queries and have 1 Western US, 1 Eastern US, and 1 EU VPS setup - which the user will be sent to based on their location.
So you can do 2-3 cheap 128MB VPS's (if they have enough storage/bandwidth), and then setup nginx/lighttpd and rsync the web directory between all of them. All setup as the same domain/subdomain, and then use a DNS service with latency or GEO-based queries. So when a user hits the download link it'll find the closest server and download from there.
This was the best way I found to give the best experience and do it all on the cheap. You can just keep adding more boxes as needed - as this is usually cheaper than adding additional bandwidth to a plan.
Get a CDN with push/FTP access. Your going to spend the same amount either way.
That would be a disastrous choice. GVH is fine if you need a low reliability box for torrenting or storage. But I would never recommend it as something to use a backup option or reserve. Their uptime is terrible, and IO fluctuations may hit upto 6MBps.
Here is what I am working on right now. varnish + nginx + btsync on the cdn nodes, add files from home with btsync client, and wrap it all up with cloudflare and pointing a subdomain on a short url to all the ip's. Cloudflare does geoDNS, and hits the closest server you have with content. If one dies, build another node, copy key and config, and change ip in cloudflare, pretty resilient with some monitoring.
Sorry i meant to write 10TB/m
and it's still a lot. Anyway thanks everyone for the suggestions. What I really don't get is: why do companies sell the "storage" plans with 2GB+ of RAM? Shouldn't 256MB/512MB be enough to serve static files with nginx even if I have hundreds of concurrent transfers? Maybe I'm missing something here?
>
I can see you read my post.
Wow I wasn't aware of that, really insteresting. What doesn't it mean "You have 100Mbps uplink port without traffic restriction." for all plans?
Our cloud system offers the same with our plans, 100Mbps Un-metered for all plans.
@sandro: Just out of my own interest how do you publish those? I mean just hosting those files doesn't get you traffic. Do you post the links in Forums or something? Most mods have their own mirrors already and you can't contribute there.
http://hosteasy.eu - their dedicated servers are very affordable if you need alot of bandwidth + storage, the 500Gbps DDoS protection is a nice extra feature.
I let developers create their own accounts so that they can upload ROMs and publish the links themselves
You call €50/m affordable?
I'd start with a cheap dedicated server.
For the features & specifications that hosteasy.eu offers yes, it is a really good deal. If I were in your shoes i'd definitly consider getting a dedicated server there, no need to mess around with 999999 vps using roundrobin etc just everything on 1 single powerful machine that is protected against DDoS attacks and is upgradable whenever necessary.
If you cannot afford it then I recommend that you state your max budget, I haven't seen you post that anywhere. It will help us offering you deals that suit you more + it shows us if your expectations are realistic.
@sandro
http://www.seflow.net/dedicati/
20TB bandwidth included, its not 100TB but do you really need that much?
Few Kimsufi's in ca and fr should be more than enough for a small downloads site, but it depends where most of your users are from.
I don't have a specific budget but paying that much is not suitable for the first months. The seflow plan is really good!! I would like to upgrade to those dedicated servers when the site grows. So I think I'm going with just 2-3 VPS's for now (no need to waste €20/m ATM) but as I said I can't find storage plans that have high BW and high storage but very low RAM/CPU which I don't need! Prices all gets high because of the RAM.
It would be great if I could find something like Storage50 or Storage100 from fliphost but in Europe as a secondary location.
Any idea?
Kimsufi, and a few ads to pay for your websiteDone.. start with 2, only 10 eur a month. that's never more than 2 or 3 reliable vps'es 500gb in canada and 500gb in france should be plenty considering a android rom is only 300-400mb. (~32TB b/w a month/dedi)
Yeah but Kimsufi is a race to get, imagine to even get 2 different locations
But you do have a point for 500GB!
it is a race, but it is worth it especially for such traffic intense projects. especially now that Oles announced to upgrade Kimsufis to 1Gbps
Is there a trick to get them? I tried that tracking service but it's flawed, I got no email for today restock for instance.
KS-2 is in stock right now http://www.kimsufi.com/en/ - for another 10-20 mins maybe
Tell me, is it really worth it? i've seen alot people complaining about bad support and VERY long delivery delays + proof processing. Would you recommend a kimsufi server to anyone? so far i've dropped all french dedi providers because of my bad experience with online.net - I've gone with providers like seflow.it, datashack.net, gameservers.com, hosteasy.eu just to avoid using any french providers. Will kimsufi.com be smooth sailing if you use correct information while signing up or do they still ask ID and everything delaying the whole transaction?
@Mark_R The stock was gone 7 mins after I posted that, haha.
I've had zero issues. KS-3 in Canada, KS-1 & KS-2 in France. Setup time was about 2-3 minutes before I got into the admin, and then an install takes maybe 10-15 mins and you get an email with the root login.
I use legit info, have paid via both Paypal & credit card. Zero issues. I've pushed & pulled my full 100mbps multiple times between various providers while moving 80-200GB backups. So a steady 11-ish MB/s for hours.
I've been paying VAT since they haven't verified my info that I'm not in the EU yet, so that's a bit of a pain, but it's still a pretty good deal with the 20% VAT. I'm happy with them, but I've only been using the Kimsufi line for maybe.. 2.5 months now.
It just depends. The KS line for me has always been about bandwidth & half-decent dedicated resources. I can totally abuse the box and push 100mbps consistently, that's all I really care about. Not running any production stuff on here.
Exactly this:
I would never ever use OVH for any productional stuff because of the support and because things happened in the past (like when they decided they'd cap your traffic no matter how much dedis you have to 200? mbit for all of them combined), that show how unprofessional and random they behave. On the other hand for a heavily abusing ressources the KS servers are just perfect
When those plans first launched there were only the starter plans and they all had 100Mbps connections. They probably forgot to change that note when they changed the plans.
EDIT:
Oh hey it looks like they've modified it since your last post to just say "* Bandwidth is flat rate." so there's your answer:
http://www.domflow.it/oneplans.php