Banning Bots

They’re everywhere, but there’s fewer of them than you’d think. Software bots, we’re talking about. Specifically bad bots that webmasters need to ban.

We’ve written our own analytics for our Apache logs and when we find hits that are trying to obtain login credentials, post spam comments, or other undesirable behaviours we schedule them for blocking.

We try to avoid false-positives by first manually checking the source IP against online databases such as Stop Forum Spam, which is a good website and gives a pretty good indication of whether the IP should be banned.

If it should, we inject the IP address into our outer firewall with iptables directly and silently drop all future requests from that IP.

Here’s our current list. We haven’t refined this with iptables-compatible CIDR addressing yet, and there’s also a couple or three class C’s that can and should be blocked, but we’ll get around to doing that once the list has grown a bit and we’ve developed a MySQL schema to manage them.

Webmasters and security professionals are welcome to take a copy of our list and use it as they see fit. We restrict this to webmasters and security professionals because they have been trained. We do not advise others to copy information from unknown sources and paste it into their IT security systems.

This entry was posted in General IT and tagged , , . Bookmark the permalink.