Comment spam is a problem for all blogs. To what extent it’s a problem depends entirely on your blogs popularity, whether you have run a spam blocker like Akismet, and what you think constitutes spam. Regardless how much comment spam you have to endure, it’s very likely that some of the spam bots crawling your site are based on the popular Perl module libwww.

Donncha wrote an excellent post back on October 10, 2007 that offered a quick fix for stopping the Perl libwww based bots. The recommendation to stop these bots from crawling your site is very simple to implement; however, before taking the suggested action I recommend you examine your server logs to make sure you’re getting such visitors.

Unfortunately, Perl libwww based bots are not the only crawlers visiting your site, but Donncha’s post does point that out and offers a link to another post with more useful htaccess edit ideas.

Share, Bookmark, or Email this post

If you liked this post, subscribe to TechTraction's RSS feed or TechTraction's email feed

Filed under: Friday Finds