Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Service watchdog script - input wanted!
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Service watchdog script - input wanted!

littleguylittleguy Member
edited July 2012 in General

This is a general-purpose monitoring script for any service. It will check if the service is running and restart it if it is not via cron. It can also email you when service restarts occur so you can investigate the cause.

Would love to have your input and suggestions on how to improve it.

Example usage in Crontab:
*/5 * * * * sh /root/watchdog.sh apache2
(Checks if apache is running every five minutes and restart it if it's down.)

*/5 * * * * sh /root/watchdog.sh mysqld mysql
("Loose" check for mysqld every 5 minutes, second parameter is the name of the service to restart, in case the application and service names differ.)

More examples are available at the top of the shell script.

Download:
https://dl.dropbox.com/u/2758854/watchdog.sh

Source:
https://dl.dropbox.com/u/2758854/watchdog.sh.txt

Requirements:
pgrep
service

Comments

  • gsrdgrdghdgsrdgrdghd Member
    edited July 2012

    @littleguy said: (Checks if apache is running every five minutes and restart it if it's down.)

    You could also just do */5 * * * * /etc/init.d/apache start ;)

  • @littleguy said: Would love to have your input and suggestions on how to improve it.

    Dispatch an email to the admin when a restart is necessary.

    Apache or MySQL or other services should never just 'stop' on a stable server. If they do there's a problem that needs investigating. Simply restarting the service is a temporary fix, not the solution. I'm not criticizing your script :) just saying that notification of a 'stopped' service is important.

    Thanked by 2Taz djvdorp
  • dannixdannix Member

    You should check monit

    Thanked by 1marrco
  • littleguylittleguy Member
    edited July 2012

    @sleddog said: Dispatch an email to the admin when a restart is necessary.

    That's a great idea, I have updated the script, you can now optionally add an email as a fourth parameter, if the service needs to be restarted an email will be sent to that address.

    Example:
    */5 * * * * sh /root/watchdog.sh mysqld mysql [email protected]

    @dannix said: You should check monit

    Looks like a great monitoring system! My script is somewhere between monit and the one-liner of gsrdgrdghd. :D

  • Suggestion for a 5th parameter:

    • Memory usage. While checking if it's running, make sure it's under the alloted MB of RAM. (e.g. Ensure Apache is under 1GB)

    Try to cram as much as you can into 100 lines :)

  • @telephone said: Suggestion for a 5th parameter:

    • Memory usage.

    I think that would be non-trivial (adding memory usage over multiple threads, memory usage from external scripts such as php5-fpm, php-cgi etc)

    But as a different, separate script, this could be an interesting idea!

    @telephone said: Try to cram as much as you can into 100 lines :)

    Haha, that's an interesting philosophy. :)

  • marrcomarrco Member

    @dannix said: You should check monit

    this

  • @littleguy

    This is a general-purpose monitoring script for any service. It will check if the service is running and restart it if it is not via cron. It can also email you when service restarts occur so you can investigate the cause.

    take a look at restard ;)

    btw, that's what i use to spawn max concurrent process
    (usage ./maxspawn.sh "matching_ps_process" 2 "/a/kind/of/concurrentscript/filename_matching_ps_process.sh")

    #!/bin/sh
    PSMATCH="$1"
    MAXPROC="$2"
    SPAWNCMD="$3"
    
    CURRENT=`ps awjx | grep -F "${PSMATCH}" | grep -v grep -c -m "$MAXPROC"`
    
    if [ "$CURRENT" = "$MAXPROC" ]; then
            echo "Max concurrent process, exit without running";
            exit;
    fi
    
    echo "$CURRENT concurrent process running"
    
    `"${SPAWNCMD}" &`
    

    no security consideration at the monent: if you are smart enough you can start a runloop ;)

Sign In or Register to comment.