Spammers reacted by altering their process so they can activate accounts. (I as well as other board owners have seen a dramatic increase in use of gmail accounts for this, so clearly Google’s registration process has been cracked and automated as well.) Like many board owners, I would like to have a “clean” database. But it wasn’t a huge imposition to get spammer registrations. If they never posted, they were not a contributing member of my board but at least they weren’t getting in the way. I had a MOD that prevented board members from entering a web site until they had a minimum number of posts on my board, so at least I didn’t get a member database sprinkled with unsavory web links. There are also MODs available that prevent zero-post users from showing up, and for pruning inactive or zero-post users after some specific period of time. All of these were okay in their day, but are not as effective anymore.

I’ve posted many times about my Checkbox Challenge code. It has served very well in protecting my blogs, several phpBB boards, and even my comment forms from spammers. However I am starting to see some issues, and that bothers me. Why? Because the new spam seems to be coming from humans rather than bots. I don’t know how we can combat that. Spammers seem to be quite creative with their posting strategies as well.

Spammer Posting Strategies

I’ve seen many different types of spam posts. There are streams of sentences that look like they were copied from a recent news article with random links thrown in for spam. There are more creative folks that put reasonable looking text and then make the punctuation marks links. There are folks that post highly useful text like, “This post was great, it answered all of my questions.” and then create a fake signature with spammer links. There are folks that post spam and format it to match the background color of the board style. There are folks that enter a normal spam-free post and come back days (or weeks) later to edit the post to include spam links.

<sigh>

What is a board owner to do?

There is no Internet governing board where we can report this type of activity. It’s rather pointless (at least most of the time) to track down users by IP address as the spammers are either using proxies or zombie computers, or their in some foreign country that could care less if your small board was defaced by someone using a computer under their jurisdiction.

Spammer Hammer

One of the nicer features of phpBB3 is the ability for a moderator to be able to clean up all of the posts from a specific user in one step. The posts can be deleted or moved to a specified forum. (I prefer the move option, as I can preserve the evidence in the cases where I do decide to try to take some sort of action.) There is a separate step to ban the user that often occurs just before (or just after) the posts are removed. I generally would do the ban first to keep the user from further posting, and then do the clean-up work.

I wrote a MOD for my own boards that I called the phpBB Doctor Spammer Hammer. It is unfortunately getting used more because of the human element. The “hammer” takes the following steps:

• Deletes any session records that belong to the user, effectively logging them off.
• Marks their account inactive, preventing them from logging back on.
• Updates their registration “activation key” so that they can’t request a resend of the activation email. That way they can’t reactivate their account.
• Any topic started by the spammer is moved to a hidden forum. This includes any posts from legitimate users, as most of them are probably just “ooh, this is spam” types of responses. Nothing of value there.
• Any posts in topics that were not started by the spammer are also moved to a hidden forum. This catches any post replies in existing topics.

The Spammer Hammer has several safeguards built in. First, you cannot hammer someone with more than a certain number of posts. If you’re a spammer, we’ll figure it out before you reach 200 posts, so anyone above that threshold (just as an example) is immune. Board moderators and administrator accounts cannot be hammered. And a log is made of each hammered account so I know who took the action and when. I started to write an “undo” function, but the complexity of the code increased dramatically and in my opinion there should never be a need to undo the action.

Conclusion

Conclusion? That’s optimistic, I guess. The story is far from concluded. As the spammers continue to get more creative the escalation will continue. As a board owner I do my best to keep spammers from getting in. If (when) they get in, I have systems in place to clean them up quickly and easily and (most importantly) completely. That’s about the best I can do at this point.

I started to include some statistics on spammer posts and registrations as a percentage of valuable traffic. But the truth is that with the Checkbox Challenge in place my boards continue to be relatively protected. I get a few spammers at most a month, and I am getting 25-35 new user registrations every day on my most active board. So I decided to skip it for this post. I also thought about calculating the average response time for my moderator team. I would take the date and time for the initial spam post and compare it to the application date/time for the Spammer Hammer and see how long they take. We rarely have spammers that last more than a few hours, and in many cases it’s minutes. I have a great moderator team.

Yes, that is an actual CAPTCHA image that I was presented with. If anyone can figure out what that one is supposed to be saying, you have better eyes than I do.

These challenges are designed – in theory – to make it harder for automated processes or “bots” to use a service by requiring something like human perception or intelligence to solve a test. The full name is Completely Automated Public Turing test to tell Computers and Humans Apart. What is a Turing Test? Wikipedia says:

The Turing test is a proposal for a test of a machine’s ability to demonstrate intelligence. It proceeds as follows: a human judge engages in a natural language conversation with one human and one machine, each of which tries to appear human. All participants are placed in isolated locations. If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test. In order to test the machine’s intelligence rather than its ability to render words into audio, the conversation is limited to a text-only channel such as a computer keyboard and screen.

The general concept is that the test or challenge is designed to weed out computer bots from real humans. The problem is bots are often better at solving problems than humans are, and even if they aren’t, they have a lot more patience.

As a board owner, there is a fine line to walk here. I want my users to be able to register. I don’t want bots to be able to register. Anything that makes it harder for bots is also likely to make it harder for users. When the scales tip to where the inconvenience to my potential new users outweighs the bot protection then I have a problem. In my opinion, some CAPTCHA techniques tip the scale in that direction, especially some of the more complex image challenges. I’m going to save talking about image CAPTCHAs for another post and focus on alternate methods. I am going to pick three tests and try to propose how easy they are for humans to solve, and how susceptible I think they are to bots. Those methods are question/answer, picture or “kitten auth” method, and my own checkbox challenge.

Question / Answer

This technique was introduced during the phpBB2 days and is much easier to manage with phpBB3 since a board owner can set up custom registration fields. The basic premise is this: the board owner sets up a question on the registration page that requires an answer. The answer could be provided in the form of a drop-down list or other input control, or alternatively it could be an open text field that requires the user to enter the answer manually. The question can be related to the primary subject matter for the board or it could be a general knowledge question like what is 2 + 2 or what color is the sky. In any case, the question is supposed to be easily answered by a human and impossible to answer for a bot. Let’s look at some examples.

Finite Result Set

If the question is presented with a set of options, either via a drop down, radio grouping, or some other interface element, it reduces the risk that a human will fail the test. It also improves the success rate for bots. Let me present a simple example. The form below presents a question and a set of options.

As a typical human I should not have any trouble answering the question. I specifically left out “black” as a color choice, because some people might consider the sky at night and make that choice. I left out white (confusion with clouds) and some other colors for the same reason. My goal is to get the user to select “Blue” as the proper answer to this question. I would guess that 99.9% of humans would be able to pass this test.

Another advantage to this particular question is there’s no regional or subject-matter bias. No matter where you are on the planet, as long as you can read English you should be able to identify with this question and select the proper answer.

This challenge also fairs well for the visually impaired. A screen-scraper will be able to present this challenge and the user should be able to solve it with the information available. While the number of visually impaired people as a percentage of total users of the Internet is certainly quite small, it’s nice to consider their needs.

As a board owner I could provide a question that is more specific to my audience. Suppose that my board audience is made up of electrical engineers. I might present them with a series of color codes and ask them to identify the resistor rating. If my board audience is made up of fans of a particular music artist I might ask them to identify the first hit song for that artist. Knitters could get a question about yarn. Car enthusiasts could get a question about engine technologies. The popularity of this solution is partially based on the fact that the question can be as hard or easy as you want. As a result, the number of options are essentially infinite.

Bots versus Question / Answer Challenge

So far the question / answer challenge seems to do okay at allowing humans to register. How will it do as a bot preventative?

In my opinion, as it has been presented so far, it has a number of issues. The first issue is that there is a finite list of choices to make. The list has six entries: Red, Orange, Yellow, Green, Blue, and Purple. Most unsophisticated bots will pick the first option so it’s important that “Blue” (the correct answer) is not the first on the list. As I have to allow for a user to read the form incorrectly at least once, I should not block or ban the registration after the first failure. In fact I might allow two or three registration attempts before taking any action. With six answers and assuming a bot is smart enough to make different selections as it goes through the form, there is a 50% chance that a bot can “learn” or “guess” the right answer on the first series of registration attempts. Since there are only six answers (and the answer set does not change each time) there is a 100% chance that the bot will be able to register if six attempts are allowed.

What about a different interface choice?

This doesn’t solve the problem; it’s just a different interface choice (radio instead of drop-down control).

How about an open text box?

This interface choice presents an interesting dilemma. On the one hand, a bot can’t use brute force to solve this challenge. There are no options given so the answer must be determined by some other means. Problem solved, yes?

No. Believe it or not, I have read some articles that suggest bots are sophisticated enough to plug unknown questions into a search engine and get the answer that way. When I plug the question “What color is the sky” into Google, the top three results all mention the word “color” and the word “blue” in close proximity. A reasonably sophisticated bot could figure this out. If this particular technique (question / answer with open input field) were to become widely used on the Internet, I have no doubt that bots would very soon be able to handle this challenge as well as (or perhaps better than) humans.

Earlier I suggested that humans should be able to solve the drop-down challenge nearly 100% of the time, certainly with two attempts. With the open text field that percentage would almost certainly drop. Let me examine that a bit further.

Validating Input

Here are some answers that I would anticipate coming into my form if I used the open text box version of the question / answer challenge.

Q: What color is the sky?
blue
Blue
BLUE
Bleu
blu
BLU

… and so on with the variations. Hm. Do I see a problem here? When left on their own, users are going to provide a wide variety of answers that probably should be allowed but won’t be under a strict comparison to the expected answer Blue. I would expect variations in case, in spelling, and perhaps even answers with extra spaces or entire sentences like “The sky is blue.” Once the input becomes open for anything, then anything is what I expect to get. How can I certify these answers (all of which are reasonably correct) and allow the user to register? Ironically if a bot is able to get the answer correct, they will most certainly provide the expected spelling of “blue” rather than one of the variations shown above.

Fortunately there is a simple function that I can use to help solve most of these challenges. That function is called soundex() and I will detail it next.

Introducing The soundex() Function

Whether I see the word “blue” or “blu” or even “bleu” the sound of the word is the same. That’s what the soundex() function does; it returns a code that is supposed to designate the sound aspects of the word rather than the literal word. First I will check the soundex() result for the required answer:

mysql> select soundex('blue');
+-----------------+
| soundex('blue') |
+-----------------+
| B400 |
+-----------------+

Next I will check the results for some of the variations shown.

mysql> select soundex('bleu');
+-----------------+
| soundex('bleu') |
+-----------------+
| B400 |
+-----------------+

mysql> select soundex('blueu');
+------------------+
| soundex('blueu') |
+------------------+
| B400 |
+------------------+

mysql> select soundex('blu e');
+------------------+
| soundex('blu e') |
+------------------+
| B400 |
+------------------+

mysql> select soundex('black');
+------------------+
| soundex('black') |
+------------------+
| B420 |
+------------------+

Notice in every case except for the obviously wrong answer “black” I get the code B400. I won’t go into details of how this code is derived (there is a Wiki link at the end of the post if you want those details). I will make the observation that all of the spellings – both correct and “close enough” to correct – return the same code.

Let me review how I got to this point. A question / answer challenge with a finite list of possible answers is susceptible to brute-force solving by bots or humans. The same challenge with an open text box for the answer is much harder for bots to solve, but it also reduces the success rate for humans due to input variations. I am proposing that the soundex() function could be used to reduce the number of humans rejected because of minor spelling variations, and I do believe it would work.

It also helps the bots.

If I am expecting a single word answer and the user enters “The sky is blue” instead, I still have options. I can programmatically split up the phrase into component words and then apply the soundex() function to each one. As long as the expect word “blue” is in the phrase, I can decide to let the registration attempt succeed.

Other Styles of Questions

I have seen people propose that simple math problems are a good question. There is only one answer, right? Well, it depends.

Q: What is 2 + 2?
4
four
for
22

All of these are potential answers. And this doesn’t help against bots; try plugging 2+2= into the Google search form and see what you get.

I have seen people suggest that the question be embedded within an image like this:

This doesn’t really help either. Bots have already demonstrated a high degree of success against CAPTCHA images so putting our question into an image rather than text doesn’t really buy much. It also reduces or eliminates the ability of a visually impaired user to solve the challenge.

Summary

There are several benefits and issues with the question / answer style of CAPTCHA.

Advantages:

1. With a finite list it is very easy for the user to interact with
2. With a finite list it is very easy to validate
3. The question can be tailored to the board audience
4. The question and related answers can be maintained via a simple administrative page
5. The technique does not penalize visually impared users

Disadvantages:

1. With a finite list this technique is suceptable to brute-force attacks
2. Sophisticated bots might use search engines to solve the answer
3. Use of a text field instead of a list control provides more protection from bots but requires more complex code and impacts the user experience

The number of advantages does outweigh the disadvantages. There are quite a few fans of this technique. There are several MODs for phpBB2 that provide this feature, and phpBB3 essentially has it built-in with the custom registration fields option. I consider this to be a preferable option to the image CAPTCHA techniques that are much more prevalent today.

Next time I want to talk about the “kitten auth” technique. I hope to have that post ready soon but have been fairly busy in real life lately so please be patient if it takes a bit longer.

Related Links

• Wiki on the soundex() function

I have mentioned more than once my frustration with the amount of spam / spam attempts coming from gmail.com email addresses. It’s clear that I can’t ban them (even if I wanted to) because of the number of legitimate users. But what if I made them go through an extra step? Would it help?

I did not spend much time thinking through this yet. That means there are likely to be holes or room for improvement. But suppose the registration process flow chart went something like this:

The “new steps” could be anything. I’m not convinced that it’s worth the effort. Most of the spammers that get through lately are clearly humans rather than bots. Anything a normal human can pass, a spammer human can also pass. The Checkbox Challenge has remained relatively effective against bots.

Oh well. It’s probably a dumb idea.

Our users have posted a total of 385789 articles
We have 43968 registered users

And when I logged in, I had 33 unread PMs as well.

Bots have been busy. I intend to go back and see what additional patterns I can get from the data. In light of one of my recent posts about gmail being the most abused email domain, here are some stats that speak for themselves. These are the top ten email domains in use on my honey pot board:

+-----------------+----------+
| email_domain    |  users   |
+-----------------+----------+
| gmail.com       |    11323 |
| mail.ru         |     6034 |
| meltmail.com    |     1179 |
| gawab.com       |      859 |
| getciallis.info |      855 |
| spambox.us      |      479 |
| serpdomains.com |      449 |
| atlantaclubs.cn |      282 |
| coolgwen.cn     |      274 |
| coolsanta.cn    |      255 |
+-----------------+----------+

It’s not just users either, here are the email address domains associated with posts since January 1 of 2009 on my honey pot board:

+--------------+-------------+
| email_domain | total_posts |
+--------------+-------------+
| gmail.com    |      169065 |
| other        |      161505 |
+--------------+-------------+

There was one particular user that was responsible for over 40,000 posts. (Yes, they used a gmail account.) Here is their posting history.

+------------+-------------+
| post_month | total_posts |
+------------+-------------+
| 2009-03    |        1042 |
| 2009-04    |        5770 |
| 2009-05    |        8544 |
| 2009-06    |        7201 |
| 2009-07    |        8009 |
| 2009-08    |        9747 |
| 2009-09    |        1778 |
+------------+-------------+

That user has posted from 840 different IP addresses. All of them are reportedly assigned to:

person:          Remiga Alexander
address:         JSC UKRTELECOM
address:         18, Shevchenko blvd
address:         Ukraine, Kiev
phone:           +380 (44) 230-9024

That’s all I have for now. I want to do more time analysis as well as go back and see if altering the flood control had any impact on posting frequency. At first glance it doesn’t seem like it.

hotmail, yahoo, gmail… any free email account is subject to abuse. Spammers are using the fact that board owners are, as you are, reluctant to ban gmail outright because it does have so many legitimate users.

Having said that, I decided it was time to go back and work through some numbers. Instead of guessing how bad the problem is, I wanted to get actual statistics to back up my claims. Anyone can say anything they want. Having numbers makes the claims more substantial. And graphs. Pictures are always good. The data used for this post is available as an Excel file for anyone to download and review (link at the end of the post). Here’s the summary:

Google: Your gmail system is borked. Fix it or risk it becoming irrelevant.

Logging Registration Attempts

I have written more than a few posts about my simple Checkbox Challenge MOD. I use it for board registrations as well as comment forms. For this post I am going to concentrate only on registration attempts at my largest phpBB board. I will use registration attempts from January of 2008 through June of 2009 (eighteen months).

For the first step, I ran some preliminary queries to identify the top five domains used. There are plenty of obvious spammer domains out there but that isn’t the point of this post. I know that mail.ru and gawab.com are the source of a lot of spam already. I also can recognize that domains like nastyteengirl.info and onlineovernightpharmacy.com are probably not legitimate. The point I want to drive home is how bad things are for mainstream domains, and for gmail.com specifically. In order to do that I want to focus only on the domains that are the source of higher volumes of registration attempts.

The top five domains and the total registration attempts are shown here.

Domain          Total Attempts   % of Total
gmail.com                12909          61%
yahoo.com                 2968          14%
mail.ru                   2704          13%
hotmail.com               1606           8%
aol.com                    843           4%

Notice that gmail is not only number one; it is in that position by a really large margin. No other email domain comes even close. My first piece of evidence clearly shows that gmail is a popular domain. It is so popular that if I were to consider banning or blocking it, I might lose 61% of my new members. But wait, is that really true? How many of those registration attempts were successful, and how many were blocked as bots?

Checkbox Challenge Data Collection Process

My Checkbox Challenge code presents a user with a standard registration form as well as a series of checkboxes. The user is instructed to click on only the marked checkbox in order to prove they are human. The development is well documented in other posts on my blog, so I won’t go into great detail here. Suffice it to say that bots seem to either ignore all of the checkboxes because they don’t expect them to be there, or they attempt to be smart and mark all of the checkboxes since they know they’re on the form. There are some humans that have issues with the system and might take multiple attempts to get through the screen but those situations are not very common, and for the sake of this post I will assume they don’t exist. Every attempt is logged, and it is that table that I am using for source material for this block post.

I listed the top five domains above. For the rest of this post I am going to drop mail.ru because most board owners know it’s a standard domain used by spammers. I am also going to drop aol.com because at 4% of the total registrations it’s not that relevant. That leaves me with three remaining domains to focus on: gmail.com, yahoo.com, and hotmail.com. (If you’re wondering who is in position six, it was gawab.com, which is another notorious spammer domain.)

Who’s Your Bot?

Any registration attempt is a potential board member. The concept behind most any anti-spam measure is to allow real people through and block bots. I have already established that gmail is by far the number one source of registration attempts. The next step is to evaluate how many of those attempts are desirable new users, and how many are bots. To do that, I retrieved the last 18 full months of data and determined the percentage of successful versus failed registrations. Here are those numbers for the three domains I have decided to focus on for this post.

Total           Success Failed  % Success
gmail.com          5644   7265      43.7%
yahoo.com          2372    596      79.9%
hotmail.com        1384    222      86.2%

Now we start to see the real problem. Both yahoo and hotmail have approximately eighty percent success rates. That means that eight out of ten registration attempts from those domains are expected to be legitimate and valuable users. With gmail over half of the registration attempts fail and therefore are presumed to be bots. Not only is gmail the number one source for registration attempts, it is the worst source in terms of the human to bot ratio.

Is Google Doing Anything To Help?

Given that these numbers start in January of 2008, the next question I want to answer is whether the problem is getting better or worse. I have to believe that Google is aware of the issues that they’re facing. Are they doing anything to help?

Here are the gmail numbers broken down by month.

Log Month        Domain         Success Fail
 2008-01        gmail.com           297   79
 2008-02        gmail.com           260   42
 2008-03        gmail.com           320   94
 2008-04        gmail.com           293  107
 2008-05        gmail.com           290   65
 2008-06        gmail.com           286  139
 2008-07        gmail.com           395  147
 2008-08        gmail.com           346  380
 2008-09        gmail.com           316  398
 2008-10        gmail.com           283  561
 2008-11        gmail.com           316  367
 2008-12        gmail.com           254  484
 2009-01        gmail.com           291  898
 2009-02        gmail.com           343  510
 2009-03        gmail.com           346  808
 2009-04        gmail.com           330  981
 2009-05        gmail.com           291  614
 2009-06        gmail.com           387  591

Here are a few things that I find interesting about these numbers. First, for the past 18 months I have averaged 313 new members (successful registrations) from gmail. That number is remarkably consistent, as shown by this graph. The blue line shows the raw data, and the orange line shows the trend.

Here is the graph for failed registration attempts from gmail.

In this case the red line represents the data and the black line is the trend. The trend is not my friend in this case. Pay careful attention to the scale of those two graphs. While they are presented as the same size (approximately 400 pixels square) the top graph (successes) has a maximum scale of 450 while the bottom graph (failures) goes all the way up to 1200. Here’s a combined graph without trend lines that will help drive that point home.

The data does not look good for Google. Sometime back in 2008 (it looks like August for me) the number of valid registrations and bot registrations were about the same. Prior to that date, bot registrations were in the minority. After that date the bot usage of gmail.com has clearly soared. In February of 2009 (2009-02 on the graph) there was a dip in bot usage, at least on my board. Was it a result of something Google did? If it was, it clearly was not very successful in the longer term as bot usage popped right back up in the following months.

Here’s another chart that shows the value of gmail to me as a board owner. This is a percentage column chart so it ignores the overall numbers and instead presents the data as percentages.

Just how significant is this? Back at the beginning of this post I noted that for the past 18 months the average success rate for a registration attempt from a gmail.com email address was 43.7%. If I recalculate the value for the past six months it drops to 31.1%. That’s not good. Is it fair to pick on Google? During the same time that the success ratio for gmail has dropped from 43.7% to 31.3% (a difference of 12.6%) yahoo has dropped 2.4% and hotmail has dropped 3.1%. In other words, all of the top three domains have seen the ratio of legitimate registrations to bots drop, but the ratio for gmail has dropped four times as much as the other two.

What Can I Do About gmail.com?

New board members are important. Without new members a community will start to get stagnant, and a stagnant community typically doesn’t thrive. As I mentioned earlier, I get an average of over 300 new members a month from gmail.com alone. For the past 18 months I have averaged 751 new members each month, and 314 or 42% of those are from gmail.com email addresses. If I were to consider banning gmail.com that’s a large chunk of my community that would disappear. I don’t think that’s a realistic action to take.

What Should I Do About gmail.com?

I think that Google should be held responsible. I can take individual steps that impact my board… Google can (and should) take steps that will protect everyone on the Internet. Am I overstating the problem? I really don’t think so. All of the numbers I have used for this post came from registration attempts on my largest (and most active) phpBB board. Here are some other numbers to chew on. All of these have been filtered to show only log entries with gmail.com email addresses.

Site Comment Form
Total attempts: 10,441
Total rejected: 10,381
Bot percent: 99.4%

Another phpBB Board
Total attempts: 2,767
Total rejected: 2,723
Bot percent: 98.4%

Still Another phpBB Board
Total attempts: 1,859
Total rejected: 1,843
Bot percent: 99.1%

What conclusion do I draw from these numbers? I submit that the problem is even worse that it appears based on the details I provided in this post! The numbers I used come from an extremely active board. Registration bots don’t pay too much attention to how many legitimate users are already registered on a board. The only goal of a bot is to find a board and register. For a smaller board this means the problem is even worse. My big board didn’t start out big. In the early days we got about 10-20 new registrations each month. Today I get more than that in one day. Because I get so many new legitimate users, it can actually mask just how bad the gmail problem really is. If you are a smaller board owner, having thousands of bogus gmail registrations can be extremely frustrating. If I didn’t have something in place that was – at least for now – somewhat effective in blocking these bogus attempts, I would very seriously have to consider blocking gmail accounts.

The problem is not new. While researching to see if I was the only one impacted by this (of course I am not) I found a post that shows how bots break the gmail CAPTCHA, and the post was from February of 2008. As we have long discussed on phpbb.com there are also services that will put real people to work breaking confirmation codes. I linked a few articles at the end of this post, and most of them are over a year old. The situation hasn’t improved since then either. If anything it has become much worse.

Google, are you listening? It’s time to fix this.

Related Links

• Raw Data used in this post in Microsoft Excel format
• Breaking Google’s CAPTCHA
• Breaking Google CAPTCHAs for $3 a Day

Apparently not.

I have seen a new style of spam coming in on another blog that I have. Based on past experience, I normally expect the spam to include links to various sites that I have no interest in. These sites will normally promote things like products I don’t want (or need).

Lately, however, I have been getting spam comments that include links to “linked in” or other social networking sites. What’s the point of that? <sigh> The comments include anything along these lines (these are actual spam comments)

After reading through the article, I just feel that I really need more information on the topic. Can you suggest some resources ?

The style of writing is quite familiar . Have you written guest posts for other bloggers?

The topic is quite hot in the net right now. What do you pay the most attention to while choosing what to write about?

My friend on Facebook shared this link with me and I’m not dissapointed that I came here.

… and many more like this. The good news is that the comments were held in the moderation queue. The bad news is that these comments were all made on a blog that is protected by the checkbox challenge code that I use here. I have plans to go out and analyze the server logs to see if the comments were made by a human or a bot, based on time spent on the various pages.

It didn’t seem to work.

I checked the post stats a few minutes ago, and while the posting did drop on the days around where I first changed the flood interval, it has also dropped like that previously. So I can’t determine whether this was a natural lull in bot activity or as a result of the flood interval change. I realize today that I should have added some code that tracked how many times the flood interval warning was issued, and I did not do that, so I really don’t have any way to analyze my data or justify my conclusions. I’m a little bit disappointed in myself for not thinking about that until now.

Here is a chart of the posting activity. I have marked the point where I changed the flood interval. As you can probably see, it’s not really possible to draw any conclusions about the effectiveness of this technique based on the data I have collected.

To be very clear, I don’t think that increasing the flood control time limit is a valid anti-spam measure anyway. It does exactly what you don’t want to do as a board owner… it makes your valuable real users alter their own behavior. You want to make your board experience a good one, otherwise people will find somewhere else to go.

At the time of this post there are 5 registered users and 7 guest users online, there are 45,556 total posts and 7,670 total users registered on my honey pot.

PM Spammers

It turned out that the real answer was much more benign… it was the notification of new private messages popping up. Normally I deactivate (or remove) the PM system from my boards, but since this is supposed to be a standard phpBB2 install I left it in place. The PM spamming started on October 10^th it seems. However, the initial attempts did not include the board administrator account. After the initial success there was another round on November 16^th, the 19^th, and again several days later. Altogether I have 329 spam PMs on the board now.

The PMs are from four different users from six different IP addresses. I checked and there are really only four locations associated with the IP address information: Moscow, The Ukraine, The Netherlands, and another hotbed of spammer activity, the state of Illinois. Someone with a Comcast high-speed internet connection is a zombie, it seems. I left the PM system enabled for now, just to see how far this goes.

Flood Interval Update

The real reason I logged in to the board today was I changed the posting flood interval. If you’re not familiar with it, the “flood” is a time limit for consecutive posts from a single user. It is designed to prevent a user from overwhelming your board with frequent posts. The default setting is fifteen seconds. Based on my analysis, bots seem to be programmed to run every 30-45 seconds. So I set the flood interval to 60 seconds earlier today.

It will be interesting to see how the bots react.

Checkbox Challenge Update

In other news…

I noticed on one of my regular (but fairly dormant) boards that there was a user named “vitamary” registered recently. I saw here on the phpBB Doctor blog (which is linked on the other site I mentioned) several spam comments caught by Akismet from a user vmary82@gmail.com. Both the board and this blog have a variation of the Checkbox Challenge in place, and both have been victims of VitaMary.

I also saw another interesting blog comment that was not caught by Akismet but was in my approval queue. The complete context of the post was the single word “test” and the email address related to the comment was gmail. I posted recently about the abuses coming from gmail, so the two of these items combined made me just a bit suspicious. I looked up the IP address associated with the comment… and it was from Russia.

To be brutally honest here, I started to post and release the Checkbox Challenge MOD at phpbb.com but stopped. Why? Because I was being selfish. I wanted to keep the technique all to myself. If the technique became popular enough to attract the attention of the bot writers, then I would have to do something different. Now I believe I have at least some preliminary indications that someone, somewhere, has taken an interest in my little bits of code and is trying to make their bot just a bit smarter.

Oh, well, I can always fall back to a suggestion from the web comic at xkcd.com: