New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Might work on quite big scale, around 400K CONCURRENT users
We are waiting for a clarification from a customer but the initial enquiry talks about 400K concurrent users 
I have worked on systems with billions of requests per day before (in advertising) but 400K users active at the same time is MASSIVE.
Does anyone here have experience with this kind of scale?
Thanked by 1greentea
Comments
Concurrent users doing what out of curiosity? Just loads of incoming requests, do they require lots of processing in the background, do you need to keep 400k concurrent websocket connections live?
I hope for your sake that you can take advantage of caching
Waiting for a clarification from the customer but concurrent normally would mean "active at the same time", so generating load. And yes, we keep a persistent websocket connection per client. We do a lot of caching, but not as much as we could do, so this is something we would need to address in this exceptional case
Ah man, good luck with 400k websockets, if you own the client it's not so bad but on my current project we just own the server side and users connect with their own clients. Having to deal with slow consumers and backpressure is painful.
So far the max we have peaked was 30K, which is already a lot
I designed a nice system for websockets and it can scale quite a lot, but I haven't tested it with 400K 
Load balancing and plan plan and plan. I highly suggest to charge 3x usual rate for this job. System architecture which involves proper load balancing is very complicated to create.
Idk load balancing on the api side isn't too bad. Load balancing websockets is way harder and then it's the databases and stuff.
Scaling databases can be a real nightmare, in my experience at least.
@vitobotta I'd be interested to hear about your websocket setup. Currently I'm only dealing with an average 5k concurrent sockets but it's due to increase 100x
Is it some sort of a game which use sockets? I mean it is very specific... Discord pioneered high usage of sockets. Search for their infra abstracts.
Mine is finance market data, really high throughput. Been looking at slacks blog posts about it but I'll check out discord too
We have an app where Approx. 50-60K concurrent users with each users sending atleast 3 requests per second to server for the data. Everything is selfhosted from servers to databases.
We are migrating (Planning) to WebSockets particularly SignalR but the internet connectivity is not very good where our users are .
Right now generating approx. 3B Req per day.
Edit : Selfhosted mean Self managed.
What type of is this? A game or some specific function?
>
Semi Gambling/Trading app.
can you please give what stack/os are you using?
>
Its Consist of many Parts . Main DB is SQLServer , 2nd is MongoDB
1st Cache is Memory Cache and 2nd Is Redis.
We Also use Redis streams , RabbitMQ ,
Using MongoDB for Reads . Elastic Search for Raw Application Logs.
CloudFlare on Top .
Based On .Net Core , C#
For web servers few are on windows and IIS and few services are on Docker Debian with Nginx.
Using Mix of Hetnzer and OVH.
Few Ax161 at hetzner with 10Gbit private switch with some cloud server .
@karanchoo thanks a lot for the info above.
i appreciate it.
There's probably lots to learn from how Netflix tunes their BSD boxes for high concurrent connections.
People don't realize, you need to tune the fuck out of any OS to get high performance. The defaults are pretty much terrible for that and meant to cover most people under typical use cases.
What do you use to hydrate read model? I am considering CQRS for a project with EventStoreDB for write and Materialize for read model but I am not yet settled on how to keep them in sync.
how much monthly cost you for such server and services?
Can't imagine how Amazon handles concurrent traffic?
Netflix deals with a scenario where there is high throughput and few requests with sequential/cached access. That is pretty easy to handle simply because you are limited by the bandwidth. Netflix customers, while looking for a series to watch, are using databases and similar things but that is a very small part of the time they are connected.
You missed the part where Netflix also does on net caching with any isp that will, spreading all that load out. They have units pushing 400gbit on FreeBSD or some crazy crap like that.
Francisco
It depends on the types of operations. For searching, ElasticSearch is well enough; for OLTP, TiDB is great; for inference, more GPUs and taking the advantage of clients. Technically, putting them together is not so hard in 2023, but what is actually difficult is keeping everything stable and surviving from network outage, hardware failure and human errors in operation. Just one or two persons may be able to build the system, but I don't think they can maintain it.
If you are talking about web apps thats not too hard. I did something for a client with 500k+ registered users and ~35M pageviews per day.
Master load balancing server * 1
Mariadb galera cluster
2 Master server cost ~$250 each. Other server cost less than $120 each. All server are with NVME SSDs.
You can move cachable files to CDN but it maybe quite expansive.
How many concurrent though?
65k Peak without problem. Never test more than that.
Where was the bottleneck? I guess galera was pita.
Anyone tried Caprover for app scaling and managing clusters of servers.
https://caprover.com/docs/app-scaling-and-cluster.html#run-multiple-servers
Would really like if you can share your experience.
They put out detailed reports, which often sounds like they were done with AMD sales support, since they trumpet the Epyc performance gains...
They get every bit of performance they can.
Worked on an MMORPG which supported about 100K online users at peak about 10 yrs back.
For the application, we deployed a bunch of load-balancers and front-end servers to persist connections and handle routing between users and internal services.