Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Need advice for growing site.
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Need advice for growing site.

MoonshotMoonshot Member
edited February 2016 in Help

Site Type:
Online testing platform (Spring/Java Application) for schools.

Previous Config

Currently were are on a VPS configured with 8GB/4 Core/OpenVZ/SSD-Cached with 320GB storage, running CentOS 6, Tomcat 7 with nginx in front (static deliveries), and MySQL (cached with EhCache). All on one machine.

The application is built pretty well, and is not stupid when it comes to abusing the database.
(Results are cached into EhCache, all writes are in batches, all best practices are followed), however, the database itself was never tuned beyond absolute basics (due to time and budget constraints).

So far, we've never felt the need to benchmark our VPS for the simple reason that we had only 37 schools signed up, totaling to roughly 39,000 students. Tests delivered to students are infrequent, but more importantly: Multiple schools barely administer tests on the same day, more so lesser during the same hour.

And yet, we've begun to hit bottlenecks.

  • Bandwidth (1 TB) ran out on many occasions and we had to buy additional bandwidth.
  • Concurrent connections were dropped many times, when 5 schools from the same location administered tests to their student bodies on the same day. (About 3500 students online at the same time, same hours as well)
  • Storage is turning out to be a major problem since we decided to host rich media contents (lots of pictures used for tests, audio and video at times, but rarely)
  • Many schools have started delivering weekly tests, sometimes multiple times a week

  • The problem (a good one) is, our application has started gaining traction and we have about 100+ schools in the pipeline, and sales will likely convert all (they're very aggressive).

And we're posed with many questions and options:

Option 1:
Each school as a sub-domain, but each sub-domain serviced by an independent VPS (same provider)

However, this means:

  • we lose the single point of failure luxury which we currently enjoy, and have to manage installs across each VPS. Everything single point, now gets multiplied by the number of schools we manage - no nightmarish management of software, system upgrades, issues etc.

  • we lose cross school telemetry data
    (strictly non-student, non-school data, authorized by schools) is lost - e.g. total tests delivered, total students tested, etc.), unless we write distributed code (expensive!)

  • although not sky high, we incur additional costs per vps which will be passed on to schools with some additional overhead for provisioning, monitoring, maintenance etc.

  • we have to find a REALLY AWESOME, time tested, battle tested, future proof provider with whom we can partner, but they won't be cheap at all due to those attributes, and we don't want to do this across multiple providers.

  • but as a pro, we don't have to infinitely scale our single virtual machine (or can this be done?)

Option 2:
Per hour services, like the AWS

  • Honestly, we are traditionalists in the sense we don't understand most of it. So we don't know if we could end up losing a lot of money.

Option 3:
A solid dedicated server configured well

  • We can currently afford about $500 to $750 per month ($1k if we really wanted to) for this if needed, but, is it a better idea than a distributed approach (option 1) or an elastic cloud (2)?

  • Single point, so all the luxury we currently enjoy stays as is :)

  • May hit limits again in the near future.

We don't want to lose the luxury of having to manage a single machine and all the ease of gathering telemetry from a single source, but, we're hitting limits now, and maybe a single dedicated will not be enough in the future.

We want to build a system which can handle at least 25,000 concurrent test takers with ease, not one that starts to choke near about 3500 logged in students.

Your comments, criticism and suggestions are welcome. (And also questions, if any).

Your recommendation:
  1. Go with a:33 votes
    1. A VPS Cluster with dynamic provisioning
      21.21%
    2. A killer dedicated server, custom configured
      66.67%
    3. Multiple VPS-es (1 provisioned per school)
        6.06%
    4. AWS, Azure or similar elastic stuff
        6.06%
«1

Comments

  • edited February 2016

    What I would reccomend is either dedicated server or VPS cluster.

    For both, you can easily scale out simply by buying more VPSes/servers, and load balancing the configuration. You will have to check to check the usage policies of any VPSes you buy as most providers with a lower cost do not allow constant usage of CPU and will require you to keep load under a certain number. You can even do a mixture of VPS/Dedicated if you want. I generally find that storage nodes use much less resources, so you can keep them on VPSes, and use the dedicated servers as computing nodes

    For the storage:

    Storage can be sharded if you have multiple servers to allow for HA. Most storage systems such as gluster/ceph/xtreemfs/etc should allow sharding for large files while preserving HA so that you can use storage more effectively without running out.

    However, please please please do not use AWS. If you calculate the pricing, you will find that it is indeed extremely expensive.

    Thanked by 2Junkless MikePT
  • @ALinuxNinja said:
    What I would reccomend is either dedicated server or VPS cluster.

    For both, you can easily scale out simply by buying more VPSes/servers, and load balancing the configuration. You will have to check to check the usage policies of any VPSes you buy as most providers with a lower cost do not allow constant usage of CPU and will require you to keep load under a certain number.

    For the storage:

    Storage can be sharded if you have multiple servers to allow for HA. Most storage systems such as gluster/ceph/xtreemfs/etc should allow sharding for large files while preserving HA so that you can use storage more effectively without running out.

    However, please please please do not use AWS. If you calculate the pricing, you will find that it is indeed extremely expensive.

    +1
    I thought the same about AWS after doing my math. It's basically a new hot-shot on the team who keeps talking about "droplets" on Digital Ocean and all this elastic stuff with great enthusiasm. Not to demean young talent and their enthusiasm, but I thought that it can become prohibitively expensive beyond a certain point.

    Thanked by 1vimalware
  • Go on dedicated route with HA in mind.

  • @tsanten said:
    Go on dedicated route with HA in mind.

    Not sure if too ambitious a target, but will a dedicated server be able to handle 25,000 concurrent connections with ease (HTTP sessions, pooled database connections etc), or will that require a very expensive dedicated server well beyond our ideal budget of about $1k?

  • @Moonshot said:
    I thought the same about AWS after doing my math. It's basically a new hot-shot on the team who keeps talking about "droplets" on Digital Ocean and all this elastic stuff with great enthusiasm. Not to demean young talent and their enthusiasm, but I thought that it can become prohibitively expensive beyond a certain point.

    There is no much need for elasticity if you do planning, as you already know how much users there are, how much users there will be on at a time, and what resources are needed.

  • edited February 2016

    @Moonshot said:
    Not sure if too ambitious a target, but will a dedicated server be able to handle 25,000 concurrent connections with ease (HTTP sessions, pooled database connections etc), or will that require a very expensive dedicated server well beyond our ideal budget of about $1k?

    What's the CPU you have right now?

    cat /proc/cpuinfo

    You can do some estimates based on benchmark, and current CPU/Memory usage/users to find what you need.

    Also, are these schools all over the world, or in one general location?

  • Don't forget that you are on OVZ as of now. Dedicated will give more than a linear growth in performance.

  • MoonshotMoonshot Member
    edited February 2016

    @ALinuxNinja said:
    Also, are these schools all over the world, or in one general location?

    1. Don't need to run that, I remember plan says Xeon E3-1220v1, whatever that is...
    2. Mostly US, some UK (5), some European locations (3)

    Similar trend for upcoming schools, most are US based.

  • GM2015GM2015 Member
    edited February 2016

    I don't know where you're located, but look at this:

    http://calculator.s3.amazonaws.com/index.html

    I've looked at your specs.

    1 instance:

    100% cpu util. linux on m4.2xlarge ondemand pricing $350/m

    1 volume:

    general ssd storage 320gb/0 snapshots

    0 extra elastic ip/all those options

    data transfer out 800gb(estimate)

    data transfer in 400gb(estimate)

    estimate of your monthly bill: $450/m.

    Leaseweb is said to be a good company, but the mess they've done with Lowendspirit's Singapore location makes me wonder what makes them so great. Plus, they're obsessed with calling customers.

  • @GM2015 said:
    I don't know where you're located, but look at this:
    ...

    • 320 GB won't be enough, we're looking at 1TB+
    • 1.2TB in-out transfer? No way, we're already exceeding our 1TB bandwidth frequently. Needs to be a minimum of 2TB, more if possible.
  • edited February 2016

    @Moonshot said:
    Similar trend for upcoming schools, most are US based.

    You can probably get a few soyoustart servers, https://www.soyoustart.com/us/essential-servers/.

    For cloud, see https://www.ovh.com/us/cloud/instances/ram.xml

    The cloud includes HA block storage, so likely more reliable than your average dedicated server. E5-2620 is a tad weaker than the current processor you are using according to benchmarks.

  • GM2015GM2015 Member
    edited February 2016

    I've given you a link to a price calculator and just listed what I estimated you currently have.

    I'd buy instead 4-5 dedicated servers(which are still easily manageable) with at least 2 providers. You can then assign $80-$100 per server which would get you a really good server with assigned bandwidth, whereas at amazon aws you 9 cent/GB. That's $92.16 per 1024GB. Insane.

    Most providers seem to offer at least 5-20TB bandwidth on servers above $20/m, which are included in the price.

    AWS is unmanaged, so you get no support at all, unless you buy in for insane prices.

    There's no gain on aws other than vendor-lock in and scalability possibilities which small businesses don't really need or can find a lot better options.

    Moonshot said: 320 GB won't be enough, we're looking at 1TB+

    1.2TB in-out transfer? No way, we're already exceeding our 1TB bandwidth frequently. Needs to be a minimum of 2TB, more if possible.

  • tsantentsanten Member
    edited February 2016

    Your problem is this: we have about 100+ schools in the pipeline.

    You don't have the luxury to play with vps's go dedicated (more than one) with 1 Gbps port and real quick if your conversion is high.

  • @GM2015 said:
    I've given you a link to a price calculator and just listed what I estimated you currently have.

    I'd buy instead 4-5 dedicated servers(which are still easily manageable) with at least 2 providers. You can then assign $80-$100 per server which would get you a really good server with assigned bandwidth, whereas at amazon aws you 9 cent/GB. That's $92.16 per 1024GB. Insane.

    Most providers seem to offer at least 5-20TB bandwidth on servers above $20/m.

    Ah sorry, my bad - I thought you were recommending AWS :) Indeed, and our current provider provides us a measly 1TB in+out.

    @ALinuxNinja said:
    ...

    • soyoustart looks interesting. Do they fit the bill when it comes to maturity, time in business, reputation and future proofing / business continuity? Never heard of them before (which is of course, more likely my stupidity than their fame).

    • I recently heard of datashack (datashack.net). What about these guys? Some of their servers are solid yet ridiculously cheap, but then, free lunches do scare me a lot.

  • I will say, get a few E3 servers, set them up with a load balancer. This way, you will get a lot of bandwidth as well as solid parallel processing power, which you need from your description.

  • edited February 2016

    @Moonshot said:

    Soyoustart is operated by OVH, they have been here for a extremely long time, and are reliable. Includes DDOS protection.

    Datashack is only double homed to Cogent+HE, and have network downtimes every once in a while. Not reccomended.

  • I'd recommend a powerful dedicated servers. If it is currently unstable, as Junkless suggested, a load balancer across a few servers sounds like a good idea.

  • MoonshotMoonshot Member
    edited February 2016

    I'm scared, if I'm being completely honest. Pretty sure I should be going for something smaller and less expensive, but just picked this one from OVH for an analysis:

    Can someone break this down for me? (704/month).

    • 16 cores, 32 threads
    • 512 GB of RAM (how is this possible? larger than an our current HDD?)
    • 2 x 4TB of SAS (do they mean 8TB of storage or some RAID involved)?
    • SAS I can understand (15k drives, right?), but what is SOFT?
    • vRack 10Gbps - so that's the port speed, but how much bandwidth?

    I'm not sure if they mean that they will actually provision this rack solely for us, or we will be on a similar rack with others utilizing the resources as well, but it says dedicated.

    Sorry if I'm sounding completely stupid, but we're only just moving out of our proverbial garage. Compared to our current modest setup, this is like going from a typical second hand 10-year old Honda to an all new 2016 Audi or BMW, I don't know.

  • @Moonshot said:
    I'm scared, if I'm being completely honest. Pretty sure I should be going for something smaller and less expensive, but just picked this one from OVH for an analysis:

    Can someone break this down for me? (704/month).

    • 16 cores, 32 threads
    • 512 GB of RAM (how is this possible? larger than an our current HDD?)
    • 2 x 4TB of SAS (do they mean 8TB of storage or some RAID involved)?
    • SAS I can understand (15k drives, right?), but what is SOFT?
    • vRack 10Gbps - so that's the port speed, but how much bandwidth?

    I'm not sure if they mean that they will actually provision this rack solely for us, or we will be on a similar rack with others utilizing the resources as well, but it says dedicated.

    Sorry if I'm sounding completely stupid, but we're only just moving out of our proverbial garage. Compared to our current modest setup, this is like going from a typical second hand 10-year old Honda to an all new 2016 Audi or BMW, I don't know.

    I think this is a bit overkill, myself.

  • WSCallumWSCallum Member
    edited February 2016

    @Moonshot said:
    I'm scared, if I'm being completely honest. Pretty sure I should be going for something smaller and less expensive, but just picked this one from OVH for an analysis:

    Can someone break this down for me? (704/month).

    • 16 cores, 32 threads
    • 512 GB of RAM (how is this possible? larger than an our current HDD?)
    • 2 x 4TB of SAS (do they mean 8TB of storage or some RAID involved)?
    • SAS I can understand (15k drives, right?), but what is SOFT?
    • vRack 10Gbps - so that's the port speed, but how much bandwidth?

    I'm not sure if they mean that they will actually provision this rack solely for us, or we will be on a similar rack with others utilizing the resources as well, but it says dedicated.

    Sorry if I'm sounding completely stupid, but we're only just moving out of our proverbial garage. Compared to our current modest setup, this is like going from a typical second hand 10-year old Honda to an all new 2016 Audi or BMW, I don't know.

    Read more about OVH vRack here: https://www.ovh.co.uk/solutions/vrack/

    As others have suggested load balancing would be a good way to go, and OVH would be a good choice for this, take a look at their 'IP Load Balancing' and purchase some of the smaller yet powerful servers, and load balance between them, that'll improve reliability (especially if you have 2 or 3 servers across different datancenters) and maintain performance, as their IP load balancing will direct traffic to the least utilized server, and best of all you can easily add more servers. Definitely worth considering for this type of project.

    In terms of the disks, you'd want at least RAID 1, but if you're going with HDDs on the customized HDD range, have at least 4 HDDs and RAID 10, or go for the pre defined options and choose the 4 x 800GB SSDs, that'll provide good performance and decent storage capacity.

  • Go with 2 dedicated (Intel Xeon D D-1540) same cpu power of that monster.

  • the ram is over kill but the disk is too slow.
    4 x 800GB SSD in RAID10 would be much better.

  • ProfforgProfforg Member
    edited February 2016

    Not clear are you looking for free advice or you accept any offers, so i'll just reply:

    I think you should start with some optimization on current VPS. That's crazy to just migrate to random hardware like your's choosen 700$ ovh server. It may be not much better than your current VPS if bottleneck is not in resources, it may be java application or something else. It needs investigation.

    If you are fine to work with sysadmin - feel free to pm me for further discussion. I can review your current situation, perform fast optimization on current server and pre-think possible solutions.

    Thanked by 1vimalware
  • Mh.

    I'm working at a software house and we play with mugh higher numbers in terms of data and concurrect access. We also heavily utilize external services and APIs, which also generates wait and load. Per customer we have from 100.000 to millions of datasets which are all linked, updated / operated on etc.

    I guess you should focus more on the Database and concepts behind the software. We run pretty well on AWS. Initially we started with two instances (HA) and one load balancer and ended up with four instances and two load balancers. Good thing is, that every aspect is easily scalable and we can focus on our customers and the software. Not sure about the exact specs, but they were are and still are below your current vps.

  • RolterRolter Member
    edited February 2016

    @Moonshot said:
    Not sure if too ambitious a target, but will a dedicated server be able to handle 25,000 concurrent connections with ease (HTTP sessions, pooled database connections etc), or will that require a very expensive dedicated server well beyond our ideal budget of about $1k?

    If optimised and caching what can be cached , you can go very far with a SSD Dedicated server for way below $1k.

    If you are using a database (which i guess you are) , setup a different server for DB alone - a dedicated server perhaps (make it SSD for faster querying) , setup redis for object caching for caching volatile data.

    Also , do not buy one powerful dedicated server for all your taskts , spread them - more like multiple VPS's but with dedicated resources and no neighbours .

  • Tomcat 7 defaults to BIO. You may get better performance switching to NIO for large number of connections.

    One service you may want to look at is Google App Engine https://cloud.google.com/appengine/ which is able to run servlets so should be fairly easy to port your app to. It automatically scales the number of instances depending on the demand, provides cloud storage for your static data. The main difficulty will be porting over to the NoSQL datastore.

  • WolfWolf Member
    edited February 2016

    @Moonshot

    Thank you for finally adding some more details.

    The best solution highly depends on your software and concepts.

    You should check which limits get hit (more precise and technical please...) and then find out which way your software scales more efficiently. You do not have to make the multi- server solution visible to the client / customer as your software could deal with that. Prepare for easy deploy and installation scripts / iso- files, which you just have to execute and also find a way to maintain multiple servers. (Puppet or stuff like this).

    Another concept would be to just go the load-balancer - way, but depending on your software that might be a little tough. As I quite struggle to understand what exactly you are doing / providing...

    I'd do the following:

    • Further optimize the DB and find potential missing indexes etc.

    • Split DB, Website and Test-runs to different servers

    • Measure the load and especially the spikes caused

    • Check which way your software CAN scale and whats most efficient.
      (More cores or rather more bang, SSD Disks or just a better network?)

    • Reduce the network load and traffic if possible. Minimize images, reduce connection amount and so on. Maybe a CDN could help to save some traffic?

    4 Cores / 8GB for ~3.500 concurrent users.

    Does that mean 8 Cores and 16GB RAM could serve ~7.000 users?

    So for 28.000 users you would require ~ 32 Cores and 64GB RAM.

    Sounds odd? Yeah, it is. As this calculation means your application scales 1:1 which would be odd, as the DB and everything else would get more and more load. High chance your app does not work like this.

    As mentioned before, find out how your app scales and what resources are really required. Dont be a fool and just buy any fat box, without even properly utilizing the resources and knowing your software. It wont guarantee to prevent further problems and is ruining your profit.

    Single point also means a single point of failure, which will then cause problems with the customers, waiting for you to fix it.

  • IkoulaIkoula Member, Host Rep

    Hello,

    If it fits your needs you could go for a load balancer on a cloud service with NAT.

    You could create multiple vm (clones) with static content replicated on all vm as web front servers and one database vm all on a private network.

    Then you will create NAT rules, so front servers can be reached but not database (it is more secure).

    When you will do a minor update (like content update) you will be able to push it to the other vm by syncing the files.

    When you will have to make a major update like OS ones etc... you will save time by doing it on one vm and then deploying X vm from that one, you can also automate that job if there is an API.

    Basically you will have to maintain two vm (web front+DB), so you will nearly keep the benefits of the actual config.

  • vimalwarevimalware Member
    edited February 2016

    Hire a sysadmin.
    A little capex spend will help you keep your opex minimal.

    Sometimes I feel 'the cloud' is brilliant scheme to extract maximum VC dollars from all-developer VC-funded startups. #cynic

  • Go with a dedicated server and hire a sysadmin.

    If you really want to do it yourself, get a dedicated server and create OpenVZ templates with your pre-installed software.

Sign In or Register to comment.