High CPU usage - Alert script?

Comments

• Nomad Member

  April 2015 edited April 2015

  Monit

  Edit: Didn't see the 32mb part. Maybe you'ld require just a shell script that runs every minute.
• Mridul Member

  April 2015

  use NodeQuery.
• dcc Member, Host Rep

  April 2015 edited April 2015

  Here you go:

  loadavg=$(cat /proc/loadavg | awk '{print $1}' )
  tmp_loadavg=$(echo "$loadavg * 100" | bc -l | cut -d'.' -f1)
  
  if [ $tmp_loadavg -ge 90 ]; then
      echo "Load average is greater than 0.9" | sendmail "[email protected]"
  fi
  

  Make sure you have bc in your system and call this, say, once a minute.
  One might ask - why am I multiplying the number by 100? Well it is because bash cannot compare floats so that is one dirty way to solve that

  EDIT: one thing to note, this way you are not really monitoring "cpu usage". Well, you are, but only "sort of". A bunch of other stuff will matter too, like i/o latency etc.

  Thanked by 2Chuck karjaj
• telephone Member

  April 2015 edited April 2015

  dcc said: Make sure you have bc in your system and call this, say, once a minute.

  You may also want a file based log, so you're not bombarded by emails every minute you're over 90%.

  You could write the date/time to file, and check if the last alert was sent within X minutes from the last email.
• belinik Member

  April 2015 edited April 2015

  delete
• linuxthefish Member

  April 2015

  Yeah nodequery is good
• JohnRoe Member

  April 2015

  Im using nodequery
• J1021 Member

  April 2015

  @linuxthefish said:
  Yeah nodequery is good

  I doubt it'll be ideal on a system with 32MB RAM.
• TinyTunnel_Tom Member

  April 2015

  @Mun has a status script that monitors load. On my NanoVZ it reboots if load reaches 0.95 and that's why ryan loves me :P
• PacketPunks Member

  April 2015

  You'll want to look into solutions like NodeWatch.
• VPSnet Member

  April 2015 edited April 2015

  you can get load:

  15min=`uptime | awk '{print $(NF)/1}'`
  5min=`uptime | awk '{print $(NF-1)/1}'`
  1min=`uptime | awk '{print $(NF-2)/1}'`
  

  you can get cpu usage:

  cpu=`top -bn2 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%\id.*/\1/" | sed "s/.*, *\([0-9.]*\) \id.*/\1/" | tail -n1 | awk '{print 100-$1}'`
  

  so all script will look like this:

  #!/bin/bash
  #15mins load aler
  a15min="4"
  #5mins load alert
  a5min="3"
  #1min load aler
  a1min="2"
  #cpu alert
  acpu="70"
  #email
  email="[email protected]"
  #ip
  ip=`hostname -i`
  #how many time in seconds sleep between checks
  wait="500"
  
  #loop forever
  while [ 1 ]; do
  #15min load average
  load15=`uptime |  awk '{print $(NF)/1}' | cut -d . -f1`
  #5min load averge
  load5=`uptime | awk '{print $(NF-1)/1}' | cut -d . -f1`
  #1min load average
  load1=`uptime | awk '{print $(NF-2)/1}' | cut -d . -f1`
  #cpu usage
  cpu=`top -bn2 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%\id.*/\1/" | sed "s/.*, *\([0-9.]*\) \id.*/\1/" | tail -n1 | awk '{print 100-$1}' | cut -d . -f1`
  
  ##checking if higher then alert
  #load15
  if [ "$load15" -gt "$aload15" ]; then
  echo "server $ip 15min load is greater than $a15min witch is $load15" > /tmp/load15
  $message="/tmp/load15"
  $subject="Server $ip 15 mins load is $load15"
  mail -s "$subject" "$email" < $message
  rm /tmp/load15
  fi
  #load5
  if [ "$load5" -gt "$aload5" ]; then
  echo "Server $ip 5min load is greater than $a5min witch is $load5" > /tmp/load5
  $message="/tmp/load5"
  $subject="Server $ip 5 mins load is $load5"
  mail -s "$subject" "$email" < $message
  rm /tmp/load5
  fi
  #load1
  if [ "$load1" -gt "$aload1" ]; then
  echo "Server $ip 1min load is greater than $a1min witch is $load1" > /tmp/load1
  $message="/tmp/load1"
  $subject="Server $ip 1 mins load is $load1"
  mail -s "$subject" "$email" < $message
  rm /tmp/load1
  fi
  #cpu
  if [ "$cpu" -gt "$acpu" ]; then
  echo "Server $ip CPU usage is greater than $acpu witch is $cpu" > /tmp/cpu
  $message="/tmp/cpu"
  $subject="Server $ip CPU usage is $cpu"
  mail -s "$subject" "$email" < $message
  rm /tmp/cpu
  fi
  #sleep some time
  sleep $wait
  done
  

  Thanked by 1Chuck
• Mun Member

  April 2015

  @TinyTunnel_Tom said:
  Mun has a status script that monitors load. On my NanoVZ it reboots if load reaches 0.95 and that's why ryan loves me :P

  https://github.com/Munroenet/serverstatus

  Thanked by 1Chuck