New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
What's the most lightweight server side language for mysql server?
Let's say I have hundred of thousands of users submitting pool / vote at the same time
What's the most lightweight server side language to accomplish this?
I am talking about backend, front end is already covered, static page / cache /cdn.
I am looking for saving the users vote.
This discussion has been closed.
Comments
I would say Node.js or Golang would fit this purpose well.
nginx + lua
Thank you. Probably gonna go with nodejs.
Rust
With that update rate the db itself will be a bottleneck. You might have to use UDF's (user-defined functions) in mysql, which are written in C. Alternatively, consider something like Redis instead of mysql.
Im surprised someone mentioned Rust.
If no one mentions PHP, there's still hope for humanity.
Heres a 20 line Golang code that accepts POST to /vote, and saves to a var.
https://play.golang.org/p/zq4vW4RRubb
As said above, don't use MySQL. Try Redis if the db fits in memory, or something like RocksDB if it doesn't.
You are better of using redis or memcached
The latter can be horizontaly scalled as well
Node js seems to be a clear choice.
THIS.
Yes, basically use any non-relational database that is fast. Using key-value pair can ensure the uniqueness of the records too (using username / user ID as key)
C
nope, that not 20 lines
it's import another library. I can run one liner with completed server 
php -S localhost:9090 /data/server.
Portuguese.
pool / vote?
Google Forms.
This is a legit answers. I have already considering it. But google forms lack of callbacks.
Is there a way to use callback when the user succeed submitting the form ?
I have already succeed integrating google forms to my website. It works as expected, except I don't know how to catch the callback.
I'm also curious about that, I've never tried it.
But I've seen someone succeed in doing it. If I'm right you are Indonesian, then try checking the Pakuan university's email registration form.
Forth or Lua.
I have tried it. People have no idea that I am using Google drive. Basically I am copying the HTML and using Ajax post to submit the form to the Google server. While it works , sometimes problem arise. There is no way to tell whether the form successfully submitted or something wrong. E.g. time out. This results in some users doesn't get their form submitted properly. They just close the browser assuming they have already submitted it.
In such a situation (100k+ req/s) I wouldn't play with google (or other) web forms at all but do the whole process myself. Reasons:(a) problems (like what you described), (b) hand over is too tricky and too slow.
I don't see the language as the big problem. Lua (have a look at a lua/terra combo!), nodejs (w/libuv), Go, and some others should do the job. The problem is the DB - but based on what you told us about that (and some things like how you recognize users) - basically nothing - we can't provide good advice beyond (frankly mindless) "useRedis","useXYZ". But I assume that looking up users, checking for multiple votes, and the DB (or table serialization, or ...) will be the trouble spots.
As another matter, if you're taking 1000s of new connections per second and it's an https page, your TLS stack may be a bottleneck. Normally one should prefer using https for everything, but in this case maybe regular http is ok. Alternatively, make sure to set up your stack for a fast cipher suite, ECDH/ECDSA or whatever.
You might need multiple hosts and a load balancer. 100k requests/sec is really a heck of a lot.
Also, if you're just counting votes, you don't need a database at all (not even redis) and you are probably better off using in-process memory, perhaps using something like OpenResty if you like nginx and lua.
If you want a very fast back end for a multicore server, look at seastar.io, but you have to be pretty hardcore to use it.
An in-memory data-structure like Redis queue or a lightweight message queue (beanstalkd?) would let you implement pub-sub between multiple front-end servers and a single source of truth (DB) .
The choice of language is less important than the design of the system.
Maybe PHP on the web workers(producer) , golang on the backend (consumer) ?
The correct answer is to hire jsg of course.
Further optimizations (v2)
Conclusion: The main problem you should concentrate on is that no classic RDBMS (MySQL, Postgres) is going to handle that, so don't save directly to RDBMS.
Use Fortran, don't be a nodejs millennial.
Pre-forktran, even.
True but smartly choosing the language can make async IO and some other things easier (or more cumbersome).
Maybe the voting form can even be a static web page. That would be the easiest and fastest solution. PHP btw. (or Python) won't cut it once one has that kind of amount of req/s.
Thanks so much but that's bad advice. I can do projects like that but I hate them and would touch that kind of project only for a very very good client.
If you also go for low memory, I would suggest picking rqlite, which is a sqlite database written in GOLANG which has replica build in.
It does have a cli and http api, which are easy to use
You're better off without any IPC happening on every single request: that's why you want in-process storage. Since it's for voting, all you have to do is add up the per-process totals once in a while, through Redis or whatever.
Crystal fits ( https://crystal-lang.org/ ). Its like Go for the people that can not stand "if err != nil"
Rust is really overkill for web development and i advice against it unless you have some serious C/C++ background. Both Rust and Crystal use LLVM in the background to compile down AOT.
Crystal is nice in the sense that people with Ruby experience can get going fairly fast with the language. For people with a PHP or JS background, it may take some time but its not a hard language. A simple webform will probably take you a few hours to write.
Your looking at processing 30k/s requests per second ( http, https is different issue ) per core with Crystal. So simply set up reuse_port with multiple instances. In regards to speed, Go is by default slower with its default HTTP but if you use fasthttp, then its faster.
Now the real blocker is going to be your database writes. My advice is to simply have a memory pool and write the results every time your pool hits 1000 requests. Write to database in one move. Single insert, serialized object or json. Do NOT write every form submitting to the database when you get it because it will kill your speed.
PHP is good for fast development but to handle 100.000req/x seconds. No ... The way PHP works it can not handle that traffic ( unless you want to balance a few hundred of servers ). Let alone the memory need for every PHP instance lol. Apache is also a bad choice when it comes for this workload.
Go, Crystal, etc are much better suited for this work because of the way they work. You can get away with PHP IF you run it for instance combined with Swoole. That can handle the load but that it a whole different game.
The real issue is the writes / database anyway. And potentially HTTPS with this amount of connections.
OpenResty (NGINX + LuaJIT) with MySQL plugin (https://github.com/openresty/lua-resty-mysql).
You know, there's a button for that.
I think i've told before that i gather all of my thanks at the end of each month and i buy sanitary napkins with them. I just love the sense of extended safety having them provides. Like, nothing can possibly go wrong and cause great emotional distress.
For write speeds, I would recommend an LSM based DB (like RocksDB) instead of B Tree or B+ trees (like InnoDB, SQLite) due to lower write amplification (fewer disk writes per DB update).
Maybe something like http://myrocks.io/ (Comparison: https://github.com/facebook/mysql-5.6/wiki/MyRocks-advantages-over-InnoDB).
Alternately, use SQLite in WAL mode (https://sqlite.org/wal.html)