All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Monitoring solution
I guess this must have come up many times, but I'm trying to understand a general view about server monitoring and I'd like to ask for you help in this.
As far as I've researched there are 3 different kind of monitoring:
Internal monitoring:
Here servers are sending regular updates to a central node, which stores this data and displays it in a control-panel like interface.
Examples:- Nagios
- Cacti
- Observium
- Zabbix
- OMD - The Open Monitoring Distribution http://omdistro.org/
- FAN - http://www.fullyautomatednagios.org/
- NodeQuery - LET project, https://nodequery.com/
Extrernal monitoring:
Some external service is regularly pinging your server to check if it's up and running and functional.
Examples:- Pingdom
- Uptimerobot
- NodePing
Webserver log analysis
Leaving access log enabled and using a script to analyse this from time to time (say daily).
As I'm looking for nginx analysing the only one I found is:- Awstats
My questions:
- Can you recommend / extend the list with easy to use and user-friendly solutions? I'm perfectly happy with hosted solutions and it's not a problem even if they cost a little bit of money.
- Which ones do you recommend for each task? Is Awstats the only one for nginx log analysing?
Comments
Nagios and Zabbix can do both internal and external monitoring (to some extent; at least I know they do ping). Nagios, Zabbix, and Cacti are generally harder to setup, you'll need to read lots of documentation and won't be able to just install and figure things out; Zabbix does have a powerful web interface to add hosts and such though. Observium is pretty easy setup and has a very nice web UI.
For 3, IMO you're better off using JavaScript / image based solution like Piwik.
Pingdom, Uptimerobot, NodePing are all very easy to setup and use. monit is also good for external monitoring, has ping and HTTP and other checks; but no distributed monitoring support.
Edit: here is a whole list of various kinds of monitoring scripts: http://lowendtalk.com/discussion/17958/2013-review-server-status-monitor-script-tool-from-let ; they have very different features, each one designed for a different purpose.
We've been using serverDensity for internal and external and it is working well.
we have been using PRTG systems since a couple of months, and it is fairly hard to understand first time, but when you get a hang of it it is easy plus it has tons of features and different kinds of sensors
Zabbix is awesome. A bit deep in the setup/get to know it arena arena.
Thanks! My problem with piwik is that I'd need to monitor a static file hosting (a linux repo), thus no JS solutions are possible. I'd actually need to parse the nginx log files. Are there any alternatives for this other then awstats?
Some updates: tried serverdensity (15 day trial), after half an our of navigating in a huge mess of overcomplicated settings, I've given up. So no serverdensity.
NodeQuery - I really really like it, but the JS based graphing makes it impossible to see the fine resolution of the data. For example I had a load spike at 8 AM, but I have no idea if it was 5 seconds or half an hour, as on their graphs it's simply not visible.
Try the script from this guy http://namhuy.net/2849/tracking-centos-server-with-bash-script.html I have not tried myself though.
Piwik actually supports a log analytics approach, maybe that's what you are looking for?
I am glad you like our service. The reason we offer only a small resolution of the data is simply our goal to provide a cheaper service with data that is sufficient for small scale server monitoring. Right now we keep detailed reports for the past 24 hours which is however not visible to the user. I will think about possibilities to display those data on demand.
Another service you might want to look at is New Relic. They provide a much higher resolution on their paid plans - quite pricey though.
@Joe_NQ, after having a look at a few alternative, I more and more like your service! The biggest two reasons are:
1. Super simple installation with no resource-hog background service, just a bash script (just don't forget to add --no-check-certificate to wget)
2. The web interface is super super clean! Seriously, all your competition has a way too cluttered interface, which is really hard to understand.
The only missing point for us is high resolution data viewing. For example it'd be really important to see if an alert is triggered by a 1 minute CPU peak or a 30 minute one!
A few ideas I have in mind:
1. Have high res javascript for the last 24 or possibly 48 hours. Paid plans can be a good option in this, for example I'd gladly pay to have 168 hour high resolution data. Also viewing the data in table / csv file would be really nice.
2. Generate high-res images for each day and host them on S3. It'd cost you pennies and would render fast, as no JS/database would be involved.
The problem with New Relic is the insane pricing they ask for the pro plan:
"$199 PER MO. PER HOST"
I mean seriously? Probably they target Rackspace customers...
I'm using the free Lite plan, only feature I'm missing is multiple users.
24 hour data rentention... I understand that they have to differentiate between free and pro, but it's ridiculous that it jumps from $0 to $199 for such features like 24+ hour data retention.
try elastictrace.com, it will be most extensive enterprise monitoring solution.
cloudstats.me maybe?
i'm using nodequery.com
for VPS monitoring please checkout vpsmt.com, it can monitor and manage solusvm / buyvm powered vps
for external monitoring, I use Uptimerobot. It's free
If I don't remember wrong, @NodePing said they were developing a monitor agent for gathering more information
We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.
I use observium running from another provider's VPS to monitor services. Works well, and using http://www.observium.org/wiki/Application/nginx, we can monitor connection stats on Nginx. Not useful for detailed log analysis, but it points us in the right direction.
thanks for the mention means a lot. we been working really hard on a release that will include very customization alerting.
Checkout monitorama, lots of good practices and ideas.
We used gangalia to monitor the system statistics, graphite for the metrics, and zabbix for pager, splunk for log search and some BI reports, which is quite expensive though.
My ideal solution is to use ELK stack, but using haka instead of logstash.
Nodequery has the nicest interface I have seen, also the agent is nice and smiple, i have tried a few others, and resource usage is too high.
Everything has the same usage notification setting, so if I want to know when cpu usage is high, I have to take the percentage for memory, and storage space too.
If I have 3 storage devices, I cant set a usage alert for one of them, it is the total usage for the pool of them.
Detailed usage reports would be nice, even if it was a csv file, that I had to graph myself.
same here
I use LibreNMS, a actively developed fork of Observium, with alerts. http://github.com/librenms/librenms
Funny enough yes.
If you have a Rackspace customer ID (don't even have to have active services) you can get 7 days retention via the standard account https://newrelic.com/rackspace
It's different than the lite 24-hour plan and used to fit between Lite and Pro.
P.S. They also have one for AWS customers http://newrelic.com/aws, one for loggly, cloud bees, etc etc. Just google up any SaaS/PaaS you're already signed up as and there's probably a partnership program from New Relic.
You can look datadoghq.com
count me... always want to subscribe for long term use. just need a little push.
don't really need too fancy functionality. just being as good (or better) as longview from Linode will do.
Thank you, I am glad you like it. You will soon be able to at least specify a custom threshold for either system load, RAM or disk usage. Alerts for individual disks are currently not planned but I'll add it to our list of possible features.
At least being able to pull disk space out of the warnings would be great.