Short and improved version of this.

To allow SSH TCP connections, but after more than 20 NEWs in 20s, drop:

iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -I INPUT -p tcp --dport 22 -m recent --rcheck --seconds 20 --hitcount 20 --name sshbadguys --rsource -j DROP
iptables -I INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name sshbadguys --rsource

To check current list (from the man page):

cat /proc/net/xt_recent/sshbadguys

Test with:

#!/bin/bash
ip="1.2.3.4"
port="22"
for i in {1..100}
do
  echo "attempt $i"
  # do nothing just connect and exit
  echo "exit" | nc ${ip} ${port};
done