Movable Type and Spam = Bad Mike Minton

Managing Movable Type can be an extremely frustrating experience. Today Ryan Hadley and I spent a good chunk of the day trying to keep a rather large MT deployment from being overrun with SPAM. The fun began with our Nagios Monitor alerting us to the number of processes on the primary blog server.

A quick review showed us that the mt-comment.cgi processes were plentiful and persistent. This typically signifies that we are under a comment spam attack. Usually it is a matter of identifying the most blantant remote hosts and blocking them via firewall. In this case though, the comments were coming from an apparent bot net as they were well distributed. During the hours leading up to the alerts we averaged between 500-1000 comments an hour from a multitude of IP addresses. Many of the IP blocks were from Russia and almost all of the comments referenced Viagra or Xanax.

There does not seem to be a mechanism for me as the blog Administrator to enforce policy regarding comments. This particular installation has 200+ individual blogs and someone (usually those that don’t have to deal with the effects of SPAM other than to complain when their blog is non-responsive) frequently believes that ‘Allow anonymous, no email, immediate approval’ is the American way. Unfortunately, the American way seems to be leaving a comment like ‘Nice Post’ with a link to their Viagra site for SEO purposes. Multiply by 40,000.

Unfortunately the mt-comments.cgi process can consume system resources rather quickly as the database gets backed up querying the comment and permission table. The dreaded ‘copying to tmp’ table is an often repeated state of MySQL queries. At this point the database is an IO bound slug.

Stopping the current activity is fairly straightforward and usually the Apache logs can quickly point at the open door or doors. On a large blog though there many be many open doors and it becomes a matter of cat and mouse. I use the following SQL query to find which blogs are wide open:

We don’t currently have anything on WordPress nearly to the scale of our Movable Type deployments but we have had good success with Askimet to date. It seems as though it is time to investigate Askimet for Movable Type to replace the rather anemic built in Spam Filter.

Comment spam is a tough problem where the desires of the content authors to allow free, open, and anonymous participation can conflict with the responsibility of IT to provide a responsive and available blog. My personal preference is to allow known users to continue commenting without approval while putting first time comment posters into moderation. How are the big blogs like TechCrunch managing this on a daily basis?

• 1 Comment
• Posted In Engineering, QA, IT