New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
How much RAM, hardware power needed (ssd, port speed, bw, etc)?
Hi Guyz,
How much ram/hardware needed for a 5-10k (even more) concurrent connections (estimate), retrieving dynamic content in MySQL querying 200-300k records. Added to that, there's also a cronjob running.
What is the best optimized web server for this kind of scenario?
Current setup is :
ubuntu 12.04 + lighttpd + php(fastcgi) + mysql + openvz(128MB,256vswap,800mhz)
Still in testing mode so i am using small vps. Want to upgrade before going live.
Target : mostly Mobile users
Thank you in advance. Appreciate any suggestion. PS (in budget server) 
Comments
depends on your app i guess
You will likely need something not to low end just based on the number of concurrent connections. It would need to be true visualization for sure and not OpenVZ. Those kind of numbers may well be considered network abuse by some.
If you are doing simple selects on an indexed column 30k records are not a lot.
@cleonard Yes, im using indexes to make querying faster, but any idea of ram needed?
How big is the database size wise (e.g. in MB)? If you get enough ram to cover the size of the database + indexes, you should be fine as it will be able to cache it in memory. Also, if those users are doing updates / writes, look at getting an SSD based VPS so you're not killed by IOwait.
It doesn't need to fit in RAM, MySQL already has an awesome caching of actively accessed parts of your database, more RAM is better, but you don't need to fit the DB.
You probably would want a VPS with enough RAM to keep MySQL alive and healthy, 512MB - 1GB should do. SSD is a must due to concurrent operations. Might want to grab some more RAM seeing you run PHP.
Thanks, I might consider SSD, and yeah PHP takes more CPU/RAM, my cronjob script runs at 60-80% cpu load but that's 800Mhz and 100-120mb ram in my current vps.
Concurrent users not yet included in that.
Thanks @hughesey, Current db size now is in 600MB, and will get bigger everyday. Users, only write one time in registration, mostly the cronjob scripts do all the writings, and users only do reads in database.
If your connections are truly concurrent, you need one system process to service each concurrent connection. Are you telling me you're running 5-10k PHP processes on a 128MB,256vswap OVZ container?
@rajprakash Not yet at the moment, im just estimating and assuming after the app goes live. so i can tell what/which vps should i buy for the upgrade, without purchasing overkill resources.
Btw, what do you mean by "one system process to service"?
Each program running on a Linux system is a called process. When your web server starts up, it starts with a particular number of helper processes (you can set how many you want it to start with by default, btw). If you only had one process running, and 2 concurrent connections came in, the one process would have to service the first connection, and then sequentially go service the second connection. While the first connection is being serviced, the second is queued and waiting or it can be rejected due to lack of resources (HTTP error code 503, typically).
If you had two processes running, and 2 concurrent connections came in, the two connections could be serviced in parallel. Basically, the maximum number of concurrent incoming connections that you can service in parallel is equal to the number of helper processes you allow your web server and helper PHP processes you set. The issue is, when your helper processes are not doing anything, they are still consuming resources while waiting for an incoming connection.
So assuming your PHP process takes up 2MB of RAM while idle, and you have 5-10k processes running, you'll need 10-20kMB of RAM to have them sit there. Statistically speaking though, your site may NEVER reach 10-20k concurrent processes. They may be staggered slightly, even if they are just milliseconds apart. So you can get away with using far less actual helper processes.
This is why balancing resources, analyzing your actual concurrency requirements, and recording statistical data about your systems, makes server administration such an art form.
@rajprakash Thanks a lot for that beefy explanation. Even though i am not server admin,or should i say not knowledgable in server(linux), i'm a developer actually, i can still understand what you mean, however i still really need to have server admin
You should be able to scale your vps up and down quickly with no interruptions. So start where you feel comfortable.. And if you need to you can always upgrade/downgrade quickly.
Are most of these queries SELECT statements or are you performing hundreds of thousands of UPDATEs and INSERTs as well? If it's the former, how many tables are you typically joining in one query and how many rows are on those tables?
i suggest just get a vps right away with reasonable amount of ram and good config. then test your site, slowly generate more and more traffic. and then you will feel how much you need.
Almost is SELECT queries, searching in almost 300k records as of now. UPDATE queries maybe 15-20k records per day.
I'm not joining other tables in most of my queries.. just straightforward, 1 table per queries.
Ei master, what could be the best optimized setup for a web server like this.?
Current setup :
Maybe ?
i love nginx + phpfpm, but that's because i'm more familiar with it. not because i think it is the best setup.
regardless, what i mean by optimized is configuration for each software. For VPS, I will start with 1GB RAM with SSD.
For your app, even though you mentioned it is dynamic, caching can help a lot if contents don't change much. For example, blogs and wikis are seldom updated.
If most of the queries are simple SELECTs without table joins then you could probably get away with quite a low-end box--especially if you implemented memcached to cache some of the SELECT statement query results in memory.