Skip to content

Sending SMS notifications of md device failure

I just wrote a script to send sms from a unix machine and I thought it would be a good idea to add an sms notification to mdadm. Therefore I wrote this script, called handle-md-event.sh:

#!/bin/bash
 
# Add 
# PROGRAM /usr/local/sbin/handle-md-event.sh
# To mdadm.conf
 
event="$1"
device="$2"
related="$3"
# Don't use the FQDN, because on machines with misconfigured DNS, it can take a long time to retrieve it and result in an error
hostname=`hostname`
 
mailto="root"
 
if [ -z "$related" ]; then
  related="none specified"
fi
 
if [ `echo "$event"|grep -E -i "^rebuild[0-9]{2}$"` ]; then
  event="$event% done"
  percentage_notice="true"
fi
 
message="mdadm on $hostname reports an event with device: $device: $event. Related devices: $related."
 
# Don't sms on Rebuild20, Rebuild40, Rebuild60 events.
# And check if /proc/mdstat actually contains an [U_] pattern, so that you only get SMSes on failures and not just random events.
if [ "$percentage_notice" != "true" ] && [ -n "`grep '\[[^]]*_[^]]*\]' /proc/mdstat`" ]; then
  send-sms.sh -m "$message"
fi
 
message="$message \n\nBecause there is/was a bug in the kernel, the normal routine checkarray function also reports Rebuildxxxxxxx, as opposed to check or something. Therefore, This message is probably just causded by the periodic check of the array, but to be sure, here is /proc/mdstat for you to check whether there is a drive failure: \n\n`cat /proc/mdstat`"
echo -e "$message"|mail -s "Mdadm on $hostname reports event $event on device $device" $mailto

In /etc/mdadm.conf you need to add the following line:

PROGRAM /usr/local/sbin/handle-md-event.sh

If you already have a handler defined, you could write a wrapper script that does both.


    3 Comments ( Add comment / trackback )

    1. (permalink)
      Comment by halfgaar
      On December 12, 2009 at 15:25

      Apparently, the -P option to grep is not supported on some machines. I bet you have to have perl installed. I therefore use -E instead.

    2. (permalink)
      Comment by halfgaar
      On December 12, 2009 at 16:01

      Never mind my last comment. I refactored the code, so it doesn’t use grep anymore.

    3. (permalink)
      Comment by halfgaar
      On January 3, 2010 at 13:30

      I rewrote the script and it does use grep…