All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Raymii.org got reddited, lwn-ed and news.ycomb-ed, on a 128MB LEB, with stats!
So, I have this article on my website about small Linux / ARM pc's like the Raspberry Pi. Heres a link to it
Now on sunday the 15th I submitted that to the http://reddit.com/r/linux reddit. To see my visits go up from about a hundred a day to twenty-thousand... I was shocked, happy ad hoped my little RAM host vps (128MB ram, 128mb burst, 25gb disk, 350GB bw, lighttpd) would hold. And it did, it was like, is this all ya got?
Now note that my website normally has about 100 visitors a day, as the Piwik stats say:
And this is how the spike looks in Piwik:
Awesome right? Same period as above, visitors / hour:
Now the Munin stats, how did the server hold up? Stats by week, from HostedMunin
Load average
Memory Usage
Lighttpd traffic
CPU Usage
Firewall
HTTP Loadtime
Where did it came from?
Here's a screenshot of the referrer list from Piwik:
Now, the server held up pretty well, all of the uptime monitors did not report any failures. But I had some spare time, so I utilized some vps' I have, and it now runs in a round-robin DNS, webroot on a GlusterFS replicated filesystem, lighttpd + php5 + SSL, automated git deploy cluster-thingie. You can notice via this on the bottom of every page: Cluster Host: 81.30.145.69.
Now, the next thing is to get hit again, and see how it holds up in the semi-cluster setup.
What do you guys/gals think about it?
Here's a link to all the images, since LET does not get my markdown links to images:
Comments
Impressive
Do you know how the loading time held up?
Will that sync the files you put in one server's root folder to every other server in the cluster?
btw you can use html img tags to link your images.
Thanks for the tip on html code. Load time is added, also from the munin graphs. While the raid, it was night here in the Netherlands, and I'm still doing a lot of night shifts, I was working then.
The glusterfs server runs on all the webservers, a volume on /gluster-raymii. (and on 3 other vps's which are not frontend webservers). The glusterfs client runs on the webservers and my git machine and my local boxes, and mounts the glusterfs volume on /var/www/glusterroot. Lighttpd's webroot is that, and the git deploy also shits there after a master branch push. The volume is replicated, meaning almost the same thing as RAID 1, only via software, TCP/IP and FUSE, all the files are acessible everywhere from the clients. If I wanted I could also mount it via NFS or SMB... If server 1 creates a file, server 2 to 8 can edit it and it is the same on all the servers... Very awesome.
Ya I'm curious about that too. Do you use PHP-FPM?
If the volume is replicated, (not striped) and the clients mount the volume as the webroot then yes, everything is on all the servers. It is not synced (the server has the files, you access them via the gluster mount, so it is synced but not synced, quite complex, it is real-time up-to-date, with things like file locking and such, via xattrs.) Does require FUSE support on the openvz vps.
Great share. That should add to a lot of people's confidence in their choice of a low end system. Solid content too, wish I had time to sit and read the whole article. Actually just surprised I have Internet here, though I did have to hang my iPhone in the window with a clothes hanger, but that's another story.
This. Sometime i want to test how a site hosted on lowendbox will handle reddit+newsyc effect, but i don't have any idea what site to host that can make people interested.
Congratulations!
Considering it's lighttpd, I'd suspect it's a fastCGI setup. You'd have to confirm that with @Raymii though.
EDIT: Also, @Raymii, congratulations
apt-get install lighttpd php5-cgi nano /etc/php5/cgi/php.ini uncomment cgi.fix_pathinfo=1 lighttpd-enable-mod fastcgi lighttpd-enable-mod fastcgi-phpSo, I think it is a fastcgi setup @joepie91 but I thought php-fpm also was using fastcgi? @NanoG6
@Raymii, could you post your lighttpd.conf file by chance? I'd be interested to see your config for that kind of traffic.
@KuJoe here's my config: http://is.gd/XeRSzN or if you hate short urls http://www.0bin.net/paste/8cd81f6be590a25c02b13e3957a0b42b579650c4#3pL4jjVEGqhqyRyPxgbcf9HqvhzAfyvgpENRbM3ZqAc=
I think this is what you're most interested in:
server.max-keep-alive-requests = 20 server.max-keep-alive-idle = 15 server.max-read-idle = 15 server.max-write-idle = 15 server.max-worker = 1 connection.kbytes-per-second = 1024 server.kbytes-per-second = 4096 evasive.max-conns-per-ip = 250 server.max-request-size = 512 server.range-requests = "disable" server.follow-symlink = "disable" server.force-lowercase-filenames = "disable"I didn't see anything about PHP, do you have a separate config file for that?
The only change I make to the php config file (/etc/php5/cgi/php.ini) is uncomment the fix_cgipathinfo line.
Actually I meant the fastcgi configs. I've been tweaking mine seeing if I notice any improvement but so far I've only found settings that cause instability. I was going to resort back to the default settings but I'm interested to see your settings with that kind of traffic.
I don't know if I changed those, and if I did, it has been a long time ago, the vps is almost 2 years now
Here's the /etc/lighttpd/conf-enabled/10-fastcgi.conf: http://0bin.net/paste/18513ef911b4e2aee37909b50e3950ca8f82b3a5#DVXsIS5jO5JHrfYdQBQ2v/sbYyynoj/1iS45GnUIqRw=
Yes, but php-fpm uses a separate process manager whereas php-fastcgi uses the built-in process manager in lighttpd (which, in my experience, gives better results).
Very nice!
Nice, congrats. Keep up the good work
I did not know that about php-fpm, thanks. And as I saw, this setup holds quite well under some load. I don't know how cryto holds, or yunnicc, is that the same setup?
@maxexcloo @Asim thanks
@Raymii Any GlusterFS performance issue on high load ?
Thanks for the positive feedback! We're happy you're happy.
That's quite an interesting idea clustering multiple vps's together like that.
Hopefully we'll see an article on your website about how you did that :-)
I've now changed it to a rsync script, now via cron but it will go in the git post-receive hook deploy thing. I have three lower-end nodes which were starting to swap because of gluster. Was eating up almost 40mb of ram, and if you have 64, then lighttpd gives you the finger...
@hien well since I've been hit, I haven't been hit again so I would not be able to tell. Visits are now up stable 600/1000 a day instead of 100
@ramnet Here you go: https://raymii.org/cms/p_Gluster_webroot_cluster (Or, if the cluster node is not up 2 date yet http://let.raymii.org/cms/p_Gluster_webroot_cluster )
http://news.ycombinator.com/item?id=4274860
This makes me swell with lighttpd pride. Thanks for sharing your configs, and congrats on all the traffic! I've taken notes for my low end configs.
Pretty impressive! The most traffic I've ever seen was around 8k hits in 24 hours for a video I mirrored on reddit, but all the load was absorbed by my dedi's Varnish cache — apache saw maybe two requests in all (although since then I've reduced the caching time by quite a bit — 24hr to 1hr).