Smokes your problems, coughs fresh air.

Sending SMS notifications of md device failure

I just wrote a script to send sms from a unix machine and I thought it would be a good idea to add an sms notification to mdadm. Therefore I wrote this script, called handle-md-event.sh:

#!/bin/bash
 
# Add 
# PROGRAM /usr/local/sbin/handle-md-event.sh
# To mdadm.conf
 
event="$1"
device="$2"
related="$3"
# Don't use the FQDN, because on machines with misconfigured DNS, it can take a long time to retrieve it and result in an error
hostname=`hostname`
 
mailto="root"
  [ -z "$related" ];
  related="none specified"
  [  "$event"|grep -E -i "^rebuild[0-9]{2}$"` ];
  event="$event% done"
  percentage_notice="true"
 
message="mdadm on $hostname reports an event with device: $device: $event. Related devices: $related."
 
# Don't sms on Rebuild20, Rebuild40, Rebuild60 events.
# And check if /proc/mdstat actually contains an [U_] pattern, so that you only get SMSes on failures and not just random events.
[ "$percentage_notice" != "true" ] && [ -n "`grep '\[[^]]*_[^]]*\]' /proc/mdstat`" ];
  send-sms.sh -m "$message"
 
message="$message \n\nBecause there is/was a bug in the kernel, the normal routine checkarray function also reports Rebuildxxxxxxx, as opposed to check or something. Therefore, This message is probably just causded by the periodic check of the array, but to be sure, here is /proc/mdstat for you to check whether there is a drive failure: \n\n`cat /proc/mdstat`" -e "$message"|mail -s "Mdadm on $hostname reports event $event on device $device" $mailto

In /etc/mdadm.conf you need to add the following line:

PROGRAM /usr/local/sbin/handle-md-event.sh

If you already have a handler defined, you could write a wrapper script that does both.

3 Comments

  1. halfgaar

    Apparently, the -P option to grep is not supported on some machines. I bet you have to have perl installed. I therefore use -E instead.

  2. halfgaar

    Never mind my last comment. I refactored the code, so it doesn’t use grep anymore.

  3. halfgaar

    I rewrote the script and it does use grep…

© 2022 BigSmoke

Theme by Anders NorenUp ↑