Mailscanner cut out on me, without errors in the log. It was only after turning on debug (which prevents backgrounding and it then only processes one batch) that it showed me.

I needed to change the first line of Mailscanner in /usr/sbin/Mailscanner:

#!/usr/bin/perl -I/usr/share/MailScanner/ -U

Source.

Of course, this is far from optimal and it bugs me that Debian didn’t fix this yet, because it’s an old issue.