Smokes your problems, coughs fresh air.

Author: halfgaar (Page 22 of 26)

Halfgaar is Wiebe. Wiebe is a contributing author on this weblog. He also has a lot of stuff (such as long, in-depth articles) on his personal website.

Wiebe's day job is as a senior software developer and system administrator at YTEC.

In his free time, he built the free, open-source FlashMQ software. Together with Jeroen and Rowan, he is now building a managed MQTT hosting business around his open masterpiece.

Preventing NameVirtualHost *:80 has no VirtualHosts

I often get the “NameVirtualHost *:80 has no VirtualHosts” on my webservers. It seems that every version of every distro has different requirements. My existing Debian config was fine, until I upgraded to Lenny. With fiddling, I determined the solution:

In /etc/apache2/ports.conf, put (among other things):

NameVirtualHost *
Listen 80
 
<IfModule mod_ssl.c>
    # SSL name based virtual hosts are not yet supported, therefore no
    # NameVirtualHost statement here
    Listen 443
</IfModule>

Then you can define virtual hosts:

<VirtualHost *>
    ServerName bla
    ServerAlias hoho
</VirtualHost>

The key is to not define *:80 for the namevirtualhost and virtual hosts. Instead, use the listen directive to define the port.

Sending SMS notifications of md device failure

I just wrote a script to send sms from a unix machine and I thought it would be a good idea to add an sms notification to mdadm. Therefore I wrote this script, called handle-md-event.sh:

#!/bin/bash
 
# Add 
# PROGRAM /usr/local/sbin/handle-md-event.sh
# To mdadm.conf
 
event="$1"
device="$2"
related="$3"
# Don't use the FQDN, because on machines with misconfigured DNS, it can take a long time to retrieve it and result in an error
hostname=`hostname`
 
mailto="root"
  [ -z "$related" ];
  related="none specified"
  [  "$event"|grep -E -i "^rebuild[0-9]{2}$"` ];
  event="$event% done"
  percentage_notice="true"
 
message="mdadm on $hostname reports an event with device: $device: $event. Related devices: $related."
 
# Don't sms on Rebuild20, Rebuild40, Rebuild60 events.
# And check if /proc/mdstat actually contains an [U_] pattern, so that you only get SMSes on failures and not just random events.
[ "$percentage_notice" != "true" ] && [ -n "`grep '\[[^]]*_[^]]*\]' /proc/mdstat`" ];
  send-sms.sh -m "$message"
 
message="$message \n\nBecause there is/was a bug in the kernel, the normal routine checkarray function also reports Rebuildxxxxxxx, as opposed to check or something. Therefore, This message is probably just causded by the periodic check of the array, but to be sure, here is /proc/mdstat for you to check whether there is a drive failure: \n\n`cat /proc/mdstat`" -e "$message"|mail -s "Mdadm on $hostname reports event $event on device $device" $mailto

In /etc/mdadm.conf you need to add the following line:

PROGRAM /usr/local/sbin/handle-md-event.sh

If you already have a handler defined, you could write a wrapper script that does both.

Bash script for sending SMS using Mollie

I signed up for a Mollie account so that I can send SMS’s from my machines. To do that, I needed a bash script, so I wrote one:

#! /bin/bash
#
# Script to send an SMS email notifcation to Mollie's HTTP gateways
# Based on the SMS2Email script which can also be found on nagiosexchange.org
# Reworked by Dennis Storm - Brainstorm ICT
# Again reworked by Wiebe Cazemier (wiebe@halfgaar.net), to include proper failure checks
#
#################################################################################
 
config_file="/etc/send-sms.conf"
log_file="/var/log/send-sms.log"
  [ -L "$log_file" ];
  "cannot continue, $log_file is a symlink."
  2
 
touch "$log_file" > /dev/null 2>&1
 
sender="SMSScript"
  [ -f "$config_file" ];
  $config_file
 
curl_location=`which curl 2> /dev/null`
  [ -z "$curl_location" ];
  "Curl command not found. Install that." >&2
  1
 
logfail()
{
  message="$1"
  message="Failure: [`date`]: $message" 
 
  $message
 
  [ -w "$log_file" ];
    "$message" >> "$log_file"
 
 
  1
}
 
# Show usage if necessary
[ $# -eq 0 ]; then
    "Usage: $0 -s [sender] -n [numbers] -m [message] -u [username] -p [password]";
    "";
    "[numbers]  = SMS numbers (if multiple, enclose in quotes) to send message to.";
    "[message]  = Text of message you want to send";
    "[username] = Username assocated with Mollie  account";
    "[sender]   = Sender"
    "[password] = MD5 HTTP API Password assocated with Mollie account";
    ""
    "-d         = Dry run, pretend success."
    ""
    "The numbers, sender, username and password options are optional and";
    "override the account credentials defined in $config_file.";
    ""
    "A log file $log_file will be kept if it is writable to the user."
    "";
    1;
 
# Get command line arguments
[ "$1" != "" ] ;
    $1
   
  -n)
      # Get the SMS numbers that we should send message to
      numbers="$2";
      2;
      ;;
  -m)
      # Get the message we should send
      message="$2";
      2;
      ;;
  -s)
      # Get the sender to show in the SMS
      sender="$2";
      2;
      ;;
  -u)
      # Get the username
      username="$2";
      2;
      ;;
  -p)
      # Get the password
      password="$2";
      2;
      ;;
  -d)
      dry_run="dry_run";
      1;
      ;;
  *)
      "Unknown option: $1"
      1;
      ;;
   
  [ -z "$username" ];
  logfail "No username specified or found in $config_file."
  [ -z "$password" ];
  logfail "No password specified or found in $config_file."
  [ -z "$numbers" ];
  logfail "No numbers specified or found in $config_file."
 
message_length= -n "$message"|wc -c`
sender_length= -n "$sender"|wc -c`
  [ "$message_length" -gt "160" ];
  logfail "SMS message is longer than 160 chars ($message_length). That is not allowed." >&2
  [ "$sender_length" -gt "11" ];
  logfail "Sender is longer than 11 chars ($sender_length). That is not allowed." >&2
 
# We haven't sent the message yet
message_sent_ok=0;
 
# The API supports sending to a comma seperated list, but that doesn't seem
# to work well. Therefore, I space seperate them and just call the API for
# each number.
number $numbers; 
  [ ! "$dry_run" ];
    RESPONSE=`curl -s -d username="$username" -d md5_password="$password" -d originator="$sender" -d recipients="$number" -d message="$message" http://www.mollie.nl/xml/sms/`
 
    RESPONSE="<success>true</success>"
 
 
  # Curl was able to post okay...
  [ "$?" -eq "0" ];
    success_line_true= "$RESPONSE"|grep -i "<success>true</success>"`
 
    [ -z "$success_line_true" ];
      logfail "$message to $number. Response was: $RESPONSE"
   
 
    logfail "curl return an error while trying to contact mollie to send an SMS." >&2
 
 
  [ -w "$log_file" ];
    "Success: [`date`]: $dry_run Sent '$message' to $number" >> $log_file
 
  "Success sending sms(es)."

It has the following config file in /etc/notify_sms.conf:

# Default values. Can be overriden with command line arguments
username="halfgaar"
password="" # MD5sum of HTTP API password.
sender="Melk"
numbers="+316xxxxxxxx" # You can set multiple numbers by space seperating them. 

LDAP search filter for Thunderbird and Zimbra

When configuring an LDAP addressbook, one thing that has given me a lot of trouble, is the LDAP filter. Here are two I use.

For thunderbird:

(|(cn=%v*)(mail=%v*)(sn=%v*)(displayName=%v*)(givenName=%v*))

For Zimbra:

(|(cn=%s*)(mail=%s*)(sn=%s*)(displayName=%s*)(givenName=%s*))

In Zimbra, I had to configure our ruby-ldapserver to never return more than 50 results, because in the configuration panel, it runs a test based on a search query which goes fine, but it does another query with the %s literally repeated. This causes the SQL condition that is generated to be %s% and that gives a whole lot of results, hanging Zimbra. It is beyond me why Zimbra runs this second query, but I guess it’s a bug.

Debian network bridge setup for kvm/qemu

When you run virtual machines, the most convenient network setup is having a bridge between the virtual machine and your normal ethernet network. In Debian, I use the following config in /etc/network/interfaces to have such a bridge:

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
 
# The loopback interface
auto lo
iface lo inet loopback
 
# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
auto eth1
iface eth1 inet manual
 
auto br0
iface br0 inet static
    address 192.168.1.101
    network 192.168.1.0
    broadcast 192.168.1.255
    netmask 255.255.255.0
    gateway 192.168.1.100
    bridge_ports eth1
    bridge_fd 9
    bridge_hello 2
    bridge_maxage 12
    bridge_stp off

In this setup, the br0 gets the IP address that eth1 had before. Intuitively, I would have created a bridge which would connect eth1 and new interfaces, but when I did that, I could not get it to work.

I then used a command similar to this to install the virtual machine:

virt-install --connect qemu:///system -n vm10 -r 512 --vcpus=2 -f ~/vm10.qcow2 -s 12 -c ~/debian-500-amd64-netinst.iso --vnc --noautoconsole --os linux --os-variant debianLenny --accelerate --network=bridge:br0 --hvm

You can adjust where necessary. What’s important, is the –network=bridge:br0. This makes sure that kvm is run with:

kvm [more options] -net nic,macaddr=54:52:00:52:1c:7c,vlan=0,model=virtio -net tap,fd=7,script=,vlan=0,ifname=vnet0

The network setup looks like:

br0       Link encap:Ethernet  HWaddr 00:15:17:23:83:67
          inet addr:192.168.1.101  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::215:17ff:fe23:8367/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1866633 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1678602 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:892540330 (851.1 MiB)  TX bytes:971936093 (926.9 MiB)
 
eth1      Link encap:Ethernet  HWaddr 00:15:17:23:83:67
          inet6 addr: fe80::215:17ff:fe23:8367/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13371212 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12703312 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:4234365756 (3.9 GiB)  TX bytes:201738186 (192.3 MiB)
          Memory:88180000-881a0000
 
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2672817 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2672817 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:493599313 (470.7 MiB)  TX bytes:493599313 (470.7 MiB)
 
vnet0     Link encap:Ethernet  HWaddr 00:ff:2d:a0:76:34
          inet6 addr: fe80::2ff:2dff:fea0:7634/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:159072 errors:0 dropped:0 overruns:0 frame:0
          TX packets:310906 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:10860687 (10.3 MiB)  TX bytes:465526222 (443.9 MiB)

source for all this.

Making sure virtual machine guests shut down properly on a Debian host

I just installed libvirt and installed a vritual machine with virt-install. I also used virsh to configure it to autostart on machine start. This way, it integrates quite nicely with the host operating system.

However, I found out that libvirt provides no way to properly shut the machines down when the host goes down. It is completely beyond me that they didn’t implement this. When you shut down libvirtd, the guests just die…

Googling yielded almost nothing, except a bug report from 17 Mar 2009 with a request for this. The report still wasn’t handled, but the guy said that he wrote a crude script to do it. I mailed him to ask for it, and here it is:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          kvm_domains
# Required-Start:    $remote_fs
# Required-Stop:     $remote_fs
# Default-Start:     
# Default-Stop:      0 6
# Short-Description: Shutdown kvm domains
# Description:       Shutdown kvm domains on reboot/poweroff.
#                    Forcefully kill instances that still run after a
#                    given timeout.
### END INIT INFO
 
# Author: Michael Biebl <biebl@teco.edu>
#
 
# Do NOT "set -e"
 
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="KVM domains"
VIRSH=/usr/bin/virsh
SCRIPTNAME=/etc/init.d/kvm_domains
TIMEOUT=60
 
# Exit if the package is not installed
[ -x "$VIRSH" ] || 0
 
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
 
#
# Function that stops the daemon/service
#
do_stop()
{
	domains=$($VIRSH -q list | awk '/running/ {print $2}')
 [ -z "$domains" ] ;
		log_progress_msg "none"
	 0
 
 d $domains ;
		$VIRSH -q shutdown $d > /dev/null
		log_progress_msg $d
 
 
	i=0
 [ $i -lt $TIMEOUT ] ;
	 $VIRSH -q list | grep -q running ;
			i=$(($i+1))
			log_progress_msg "."
			sleep 1
	
		 0
	
 
	# forcefully kill the remaining kvm instances
	killall -9 kvm
 1 
}
 
 "$1"
  start)
	;;
  stop)
	log_begin_msg "Stopping $DESC:"
	do_stop
 "$?"
		0) log_end_msg 0 ;;
		1) log_end_msg 1 ;;
 
	;;
  restart|force-reload)
	;;
  *)
	#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
 "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
 3
	;;
 
:

It depends on text output by virsh, so it’s not the most robust, but it works.

Run this to put in it in the proper runlevels (the 10 on my system makes sure it executes before kvm and libvirt):

update-rc.d kvm_domains stop 10 0 6 .

Adding a recycle bin to a samba share

People are stupid and remove things accidentally. You can partly mitigate that by adding recycle bin functionality to a samba share.

[Our documents]
  comment = Central storage for all our documents
  path = /home/smb
  guest ok = yes
  writeable = yes
  browseable = yes
  force group = samba
  create mask = 0660
  directory mask = 0770
 
  ; recycler
  vfs object = recycle:recycle
  recycle:subdir_mode = 0770
  recycle:repository = .recycle ; directory name
  recycle:keeptree = Yes
  recycle:touch = Yes
  recycle:versions = No
  recycle:maxsize = 100000000 ; 100 metric million bytes 

The action is hooked on the delete action. Pressing shift-delete won’t prevent files from being moved to the recycler, which is good.

You can then make a cron job which throws away old files:

# Every day at 6am
0 6 * * * root    find /home/smb/.recycle/  f -mtime +5 -delete > /dev/null

Permissions on a samba share

When you mount a samba share without unix extensions enabled, you can set a GID, UID and permissions (on the client machine, at mount time) so you can adjust it to let non-root users use it.

Mounting FAT works this way as well. But there is a big difference. Where new files on a FAT file system are created according to the permissions you set at mount-time, new files on a SAMBA share have their permissions determined by the umask. However, when you unmount and remount the share, the permissions are changed to what they were set to at mount time.

This is very annoying behavior, because when you have files that belong to root:smbusers and you copy a file, it still belongs to root:smbusers, but when your umask is 0022, it will no longer be group writable and it has become a read only file.

I think this is a bug in the SMBFS/CIFS file system driver.

Samba rounds filesizes off to whole MB’s

I noticed when I did ‘du -hs’ on a sambamount, I got a disk usage that was unrealistically high. I did some research, and it appears that Samba rounds off file sizes to whole MB units, to optimize for windows clients:

And btw, why is samba rounding the minimum size up to 1MB ?

An optimisation for Windows clients. If we do this they allocate actually run faster against a Samba server (based on tests done by a NAS vendor).

Bash quoting

I’m always confused by bash’s quoting. I hope to put all my quote wisdom in this post and invoke other’s quote wisdom in the comments. I’ll give some examples of what I mean.

Let’s say you have a file with a space: “bla bla.txt”. If I were to ls that file, I would do:

ls 'bla bla.txt'

This works. However, when I want to do this from a variable (in a script) and do:

command="ls 'bla bla.txt'"
$command

The result is:

ls: cannot access 'bla: No such file or directory
ls: cannot access bla.txt': No such file or directory

You can solve this by using eval:

command="ls 'bla bla.txt'"
eval $command

This gives:

bla bla.txt

Some time ago, I suggested this as answer on somebodies question at userfriendly, to which somebody else said that using eval actually makes things worse:

That’s actually worse. . . as the quoting gets re-parsed (remember, ‘eval’ means “take arguments as shell input”), which means that single quotes in the name break it, horribly, and names with spaces get even _worse_.

Another example: let’s say you have two files:

-rw-r----- 1 halfgaar halfgaar 0 2009-10-18 16:51 bla's bla"s.txt
-rw-r----- 1 halfgaar halfgaar 0 2009-10-18 16:52 normal.txt

I’m gonna run this command on it: find . -mindepth 1 -exec ls ‘{}’ \;. When executed without eval, it says this:

find: missing argument to `-exec' 

With eval, it says:

./normal.txt
./bla's bla"s.txt

Eval seems to be what I need, so what is wrong with using it? Also, shouldn’t that double quote be a problem? If someone can give a situation where that poses problems, I’m all ears.

« Older posts Newer posts »

© 2024 BigSmoke

Theme by Anders NorenUp ↑