Latest Attempt to Curb Comment Spam
I don’t get a lot of “real” comments on my site, but I certainly get more than enough spam comments. If you run a blog, then you know what I mean. Comment spam is an annoying comment that contain links to adult content, replica rolex watches, ring tones or other unrelated material. Recently the total amount of comment spam appearing on my site has become so large I was finding it difficult to scan the spam comments to extract the few false positives caught by my comment spam filter, Akismet (comment spam filter plug-in for WordPress). It became clear I needed something else to help Akismet fight off the spam, but what?
Where does the comment spam come from?
I believe that nearly all comment spam (fringe or otherwise blatant) is driven by a spam bot. If you’re not familiar with the term, a bot is essentially a robot program that automatically surfs the net and blindly inserts meaningless comments into a blog. These bots have become very sophisticated but still have difficulty working around a captcha.
How a captcha can help reduce comment spam
A captcha is that little annoying image that requires the visitor to retype what they see. Nearly anywhere there is a web site registration, there is usually a captcha. Bots don’t deal well with captchas because they can’t easily “read” the contents of an image — for that, “real” human eyes are required. And while not a 100% reliable, captcha’s do help reduce a significant amount of bot inserted comment spam.
I’ve held off on using a captcha because I wanted to encourage comments from real visitors. Any sort of extra step for writing a comment might be discourage visitors from participating in the discussion. Unfortunately, the volume of comment spam has become so large that I had no choice and I installed reCAPTCHA (a WordPress ready plug-in that makes it easy to set up a captcha on my site).
Good, but not perfect, results thus far
So far the captcha has made a serious impact on the total amount of comment spam appearing on my site. Where I used to see approximately 50 spam comments a day, now I only see about 10. This reduction has made it much easier to scan the 10 comments and de-spam any false positives, which by the way are also very few (maybe somewhere around 1 in 50 might be a false positive).
While I’m pleased with the results, I’m still somewhat baffled by how the bots are getting around the captcha. I’ve heard that some bots actually link back to the captcha on a different site, then they entice visitors at that site to type in what they see in order to view additional adult content. I must admit, it’s a clever work round to get real humans to do the dirty, and difficult, work of a bot. Unfortunately, I have no proof this is how the bot’s are defeating the captcha but I do know they are getting around it somehow.
Bots seem to target the same post
In an effort to understand how to better combat the spam bots, I’ve started to look at which posts they are placing their comments. Most of the bots are hitting the same post over and over. Therefore I’ve started to close the comments on any of those posts that are a couple months old. I know there is a WordPress plug-in to help automate this process, and I plan to look into that more when time allows. In the meantime, however, I’m manually closing comments as needed. Hopefully this won’t discourage visitors from leaving comments on posts where the comments are still open. But for now, my battle against comment spam continues. If anyone has any suggestions to help my fight, please let me know.
Share, Bookmark, or Email this post
If you liked this post, subscribe to TechTraction's RSS feed or TechTraction's email feed
Filed under: Blogging Related
[...] online community. The best part is … it’s all 100% free! Check them out here: Join Hey Nielsen! Latest Attempt to Curb Comment Spam saved by 1 others rkoheel bookmarked on 01/07/08 | [...]
You can try Bad Behavior. I have noticed that tendency to target certain posts and have closed comments on certain posts but I don’t like that option. There is also Math Comment Spam Protection which I like because normal captcha are difficult for people with visual disabilities (but I really don’t like that it deletes the comment if a person gets the answer wrong - instead of keeping it and giving them another chance).
Thanks for the suggestion John. I’ll definitely take a look at those additional options. And I agree, I’m not a big fan of closing comments in order to prevent spam. Unfortunately, it’s been the only additional solution I have at the moment.
Thanks again for the comment.
this is why we built defensio.com… go take a look, and consider giving it a try!
Thanks for the tip Matt.
Hey Matt (hope you’re monitoring the discussion thread for this post), I was wondering…does defensio support the idea of a “black list?” I was reading the FAQs and didn’t see anything along these lines. What I’m looking for is a customized “black list” that I create. When something matches the “black list” then it instantly gets deleted. My previously probably was that lots of spam was getting flagged correctly, but the quarantine was getting so large that it was becoming impossible to scan for legitimate comments. I wanted something that would just delete something that matched my “black list.” Can defensio do something like that?
No Defensio doesn’t do a blacklist. We feel that the best solution — for consistency and performance — is to handle end-to-end filtering on the server side. Notice that we provide full transparency with your performance statistics, which means we stand by the performance of our algorithms. We’re now running around 99.7% accuracy overall — quite a bit high for really busy blogs… so really, a blacklist should not be necessary. Beyond performance, we think it’s the added convenience of easily identifying false positives (when they do happen) that really makes the service compelling.
Why don’t you give it a try for a few weeks and see how it works out for you?
(
Thanks for the response Matt. I just might give Defensio a try; however, the reason I wanted a Black List is that I wanted some spam to just get removed so my spam queue was easier to scan for any false positives. My previous problem was that I was getting overrun with spam that scanning for false positives in the spam queue was nearly impossible. Since I kept seeing the same spam getting caught over and over I just wanted to create a Black List so that once spam on the Black List was caught it would simply get removed which would make my spam queue easier to scan before hitting the delete button.
I promise you, with Defensio your spam queue will be *MUCH* easier to deal with, seeing it’s sorted by “spaminess”. Only the low-spammy comments (ie those most likely to be false positives) bubble up to the top… Further, you can actually hide “obvious” spam from view (above some threshold).
Do try it for a couple weeks, and let me know what you think.
meaningless comments that are not relevant are (at best) annoying , and at worst dangerous to readers computers (as they can be connected to virus sites) Defensio will probably work for you!
Bret,
The answer is probably simple. I had the same thing. I used a captch plugin to stop the bots and it cut the numbers down, but some where still getting around it. These tended to target one or two posts. The answer:
The remaining bot spam is NOT comment spam. It is TRACKBACK spam.
I use the the Simple Trackback Validation plugin to stop these.
You can choose what to do with the trackback spam - at first I opted to put them in Askimet for a few weeks, then as there were no false positives, I decided just to nuke them.
Anyway, I’ve gone from hundreds of spam (in Askimet) each week down to a handful - maybe 1 or 2 per week, plus 3 or 4 for each post (from the scrapers).
Thanks for sharing your spam experience Stephen. I’m going to take a look at that trackback validator plugin and see if it helps my situation.
I have been visiting this site a lot lately, so i thought it is a good idea to show my appreciation with a comment.
Thanks,
Jim Mirkalami