A spammer registers on a board. They might not do anything for a while. Then they try to post something that looks legitimate, using generic language that could be appropriate anywhere. Stuff like:

You make some good points, please keep posting

I find your arguments compelling, can you link your sources?

Thanks, it helped me

None of those add anything to the discussion, but they’re not really spam. What happens next? The spammer goes quiet for a few weeks, hoping that the topics they have posted in will fade from the front page. Then they carefully go back in and edit their post. They might change the text of the post itself, or they might add a signature that wasn’t there before. They are relying on the fact that phpBB (and other boards as well) do not bump a post back to the front page if something is edited, only if new content is added.

Very frustrating.

So far I have not come up with a programmatic solution to the problem. I am working on code that will capture the edit history of a post and allow board moderators to revert to an original version, so that at least would let me prove how the spammer added their content after the fact. That doesn’t solve the problem, it just provides an audit trail should I decide to try to take action against the spammer.

A frequent suggestion at this point might be something along the lines of preventing someone from posting URLs or links until they reach a certain level of post. That doesn’t help either, as the spammers often have five or ten posts under their belt before they come back and edit. Plus it impacts the legitimate new users that come on board with questions that require links. It’s not my favorite concept.

So today what my moderator team does is a manual process. When we get a suspected spammer, they will do a web search for either their username, their email address, or both. If they find the same username on hundreds of different boards that’s a good indication they’re a spammer, especially if the user is recently registered on all of them. They can also pull up posts from the user on these other boards. If they look similar to what they’re posting on our board, that’s another indication. All of these steps are used to decide whether to preemptively ban the spammer before they spam, or decide to wait.

It’s all a manual process for now. So while I’ve been away from phpBB2 for a while because of other demands on my time, this has never really been far from my mind. I just haven’t come up with an idea that can be implemented in code versus a manual process.

Guess I should check in with the BB Protection folks, and see what they’re up to at this point.

Spammers reacted by altering their process so they can activate accounts. (I as well as other board owners have seen a dramatic increase in use of gmail accounts for this, so clearly Google’s registration process has been cracked and automated as well.) Like many board owners, I would like to have a “clean” database. But it wasn’t a huge imposition to get spammer registrations. If they never posted, they were not a contributing member of my board but at least they weren’t getting in the way. I had a MOD that prevented board members from entering a web site until they had a minimum number of posts on my board, so at least I didn’t get a member database sprinkled with unsavory web links. There are also MODs available that prevent zero-post users from showing up, and for pruning inactive or zero-post users after some specific period of time. All of these were okay in their day, but are not as effective anymore.

I’ve posted many times about my Checkbox Challenge code. It has served very well in protecting my blogs, several phpBB boards, and even my comment forms from spammers. However I am starting to see some issues, and that bothers me. Why? Because the new spam seems to be coming from humans rather than bots. I don’t know how we can combat that. Spammers seem to be quite creative with their posting strategies as well.

Spammer Posting Strategies

I’ve seen many different types of spam posts. There are streams of sentences that look like they were copied from a recent news article with random links thrown in for spam. There are more creative folks that put reasonable looking text and then make the punctuation marks links. There are folks that post highly useful text like, “This post was great, it answered all of my questions.” and then create a fake signature with spammer links. There are folks that post spam and format it to match the background color of the board style. There are folks that enter a normal spam-free post and come back days (or weeks) later to edit the post to include spam links.

<sigh>

What is a board owner to do?

There is no Internet governing board where we can report this type of activity. It’s rather pointless (at least most of the time) to track down users by IP address as the spammers are either using proxies or zombie computers, or their in some foreign country that could care less if your small board was defaced by someone using a computer under their jurisdiction.

Spammer Hammer

One of the nicer features of phpBB3 is the ability for a moderator to be able to clean up all of the posts from a specific user in one step. The posts can be deleted or moved to a specified forum. (I prefer the move option, as I can preserve the evidence in the cases where I do decide to try to take some sort of action.) There is a separate step to ban the user that often occurs just before (or just after) the posts are removed. I generally would do the ban first to keep the user from further posting, and then do the clean-up work.

I wrote a MOD for my own boards that I called the phpBB Doctor Spammer Hammer. It is unfortunately getting used more because of the human element. The “hammer” takes the following steps:

• Deletes any session records that belong to the user, effectively logging them off.
• Marks their account inactive, preventing them from logging back on.
• Updates their registration “activation key” so that they can’t request a resend of the activation email. That way they can’t reactivate their account.
• Any topic started by the spammer is moved to a hidden forum. This includes any posts from legitimate users, as most of them are probably just “ooh, this is spam” types of responses. Nothing of value there.
• Any posts in topics that were not started by the spammer are also moved to a hidden forum. This catches any post replies in existing topics.

The Spammer Hammer has several safeguards built in. First, you cannot hammer someone with more than a certain number of posts. If you’re a spammer, we’ll figure it out before you reach 200 posts, so anyone above that threshold (just as an example) is immune. Board moderators and administrator accounts cannot be hammered. And a log is made of each hammered account so I know who took the action and when. I started to write an “undo” function, but the complexity of the code increased dramatically and in my opinion there should never be a need to undo the action.

Conclusion

Conclusion? That’s optimistic, I guess. The story is far from concluded. As the spammers continue to get more creative the escalation will continue. As a board owner I do my best to keep spammers from getting in. If (when) they get in, I have systems in place to clean them up quickly and easily and (most importantly) completely. That’s about the best I can do at this point.

I started to include some statistics on spammer posts and registrations as a percentage of valuable traffic. But the truth is that with the Checkbox Challenge in place my boards continue to be relatively protected. I get a few spammers at most a month, and I am getting 25-35 new user registrations every day on my most active board. So I decided to skip it for this post. I also thought about calculating the average response time for my moderator team. I would take the date and time for the initial spam post and compare it to the application date/time for the Spammer Hammer and see how long they take. We rarely have spammers that last more than a few hours, and in many cases it’s minutes. I have a great moderator team.

What is the difference between recognition and reputation?

After mulling it over for a while, I decided that there is a very distinct difference. I believe that “recognition” is something that can be provided by technology and that “reputation” is something awarded by other board members. I would like to provide a formal definition of each word and then try to describe why I think that. After that I would like to discuss the impact of both on an online community.

What Is Recognition?

When I first heard the phrase I said to myself that recognition is what the board software provides each member. It can be as basic as a post count or something more sophisticated such as a rank. It’s a reward of some kind for participating on the board, and as such is associated with a specific event or point in time. One definition that I found reads as follows:

To give a token of thanks for (a service rendered, etc.)

Isn’t that what board software packages do today? Every time a post is made in a “valuable” forum (one with post counts turned on) the poster is recognized by incrementing their post count. As a post count increases the poster may be awarded further recognition by attaining various rank titles. Some boards grant additional privileges like access to formerly hidden forums, the ability to upload or change an avatar, or even earn “cash” or “points” that can be used at a community store. Some years ago I wrote a MOD for phpBB2 that recognizes years of membership by awarding a star (or other icon) on a member profile.

But my point is these are all system-generated awards. They are driven by specific events, occur at a specific point in time, and are the results of code execution. How is this different from reputation?

What Is Reputation?

Here is what I consider the most appropriate definition of reputation for this topic that I found:

The general estimation in which a person is held by the public.

How is this different from recognition? Reputation is not awarded by technology but by other community members. This could be formally acknowledged by the board software (by systems like “karma” or “points” or “kudos”) or it could just be informal. In either case, I suggest that reputation is not something that occurs at a point in time but instead is earned over longer periods. For that reason, I would add the following words to the definition listed above: “… based on behavior over time.”

On a default phpBB2 board there are no formal systems for reputation. There is a variety of karma or points MODs. I have a couple of different MODs that I wrote in this area myself. One allows people to increase the value (reputation) of a topic by awarding it points. The act of awarding the point is “recognition” while the overall point total adds to the topic “reputation.” See how that works? My search process uses topic points to weight search results so that more valuable topics bubble to the top.

Why didn’t I write a user point MOD? It’s quite simple. To my way of thinking, users will build their own reputation by participating on the board. There is no need to create a way for board members to formally rate other users; that will happen naturally. People don’t come to my board looking for users, they come looking for information… and information is contained in topics.

Ultimately I did write a user “medals” MOD and we have used it to award “member of the year” status for the last three years. For what it’s worth, I am planning on canceling that program from this year forward due to very low participation in the voting.

Recognition, Reputation, and Board Management

Since recognition, at least according to my way of thinking, is system generated, it can be abused. People can post nonsense posts just to increase their post count. (That’s one reason why we have MODs for phpBB2 and a standard feature in phpBB3 for no-post-count forums.) People may falsely assume that users with the highest post counts are the most knowledgeable about the subject. That is quite frequently not the case. Anything that is subject to abuse can make more work for a moderator team.

Reputation, on the other hand, is hard to fake. I have seen it on every board I have participated in to some degree. The community doesn’t take too long to figure out who is providing value and who is fluff. The problem for board owners is to figure out how to formally recognize reputation without opening the process to abuse. I don’t have a perfect answer for that, but I do have some thoughts.

Which I will save for my next post on this subject.

Related Links

• Annual Stars MOD for phpBB2

Definitely need to try to make it.

This year I used a suggestion from my wife… it’s evil. I will post what I did after today is over.

How about you? Any fun tricks to share?

It happens.

I have posted a lot about the Adsense program over the past years, specifically related to advertising on phpBB boards. I currently do not use Adsense, but I did for many years. However I never relied on the revenues from that source to keep my board running. If I had, I might have ended up like this case study:

Warning to Webmasters: It can happen to you

The link contains a case study about Soccerpulse, a web site with over 100,000 members that closed up shop because their Adsense revenues declined and they could no longer afford to run the site.

You can earn more money by:

• Generating more traffic
• Generating more clicks
• Getting higher-paying advertisements

I’ll talk about each of these at a high-level in this post.

Adsense Recap

Adsense is the front-end of the Adwords advertising program. If someone wants to advertise their site(s) they can open an Adwords account and bid on certain keywords. (They can also target specific sites, but I’ll talk more about that later.) Google Adsense program participants are assigned a unique publisher ID and given a bit of javascript to place on their site. When a page is rendered, the javascript sends the content to Google who quickly scans it to identify keywords, then matches keyword advertisers with the content. The return value is a string of HTML that contains various advertising links.

And this all happens really, really, fast.

As a publisher, I can only control certain things. I cannot control what advertisers bid on my keywords, nor can I select ads from specific advertisers for my site. So how can I control my income?

Simply put, I can’t. But there are a few things I can do to try to improve my earnings.

Generating More Traffic

There are plenty of topics on phpbb.com about generating more traffic. I won’t try to cover that here since this topic is about advertising. Instead I want to talk about the impact of increased traffic on advertising revenue.

The first thing a new Adsense publisher has to know is that most ads in the program are paid by click (CPC) and not by impression (CPM). This is certainly in favor of the advertisers which makes it easy to understand why most opt for this configuration. Because of that, it might not be immediately obvious why generating more traffic could generate more income. It comes down to statistics.

Because of the Google Terms of Service (TOS) I can’t share what my click statistics are. So for this post I will make up some numbers. Suppose that I see over the past few months that I get 1 clicks for every 10,000 page views. Suppose that I have also noticed that ratio seems consistent as traffic goes up. It stands to reason that if I can project 5 clicks for 50,000 page views then I can get 50 clicks for 500,000 page views. If each click is worth $1 then it’s easy to see how my income goes up as clicks go up.

From what I have read (and experienced) there are several typical ways to increase earnings from this program. You can try to generate more traffic. You can try to generate more clicks. And you can try to generate higher-paying clicks. There are plenty of suggestions on this forum or the Adsense blog on how to do these things, but I’ll offer a high-level overview. One of your challenges (as I see it) is that the number of sites dealing with games / onling gaming is HUGE and therefore you’re up against a lot of competition.

To generate more traffic you need to get more people to come to your sites. Sounds simple, but it’s not. Whether you do this via advertising, search engine optimization, or some other means of promoting your site is up to you. You also need to make sure that you do what you can to retain your existing visitors while gathering new traffic or you simply end up churning (keeping the same amount of traffic but from different users). From a marketing perspective, most folks will tell you it costs less to keep an existing customer than to get a new one, so try not to go after new traffic at the expense of existing traffic.

Getting More Clicks

There are lots of illegal ways to generate more clicks but also legal ways. Try different ad placements or different color schemes for your existing ads. It works best if you set up different channels so you can track what techniques are successful. And don’t stick with the new style just because something is successful for a short period of time. You need to continue to monitor the relative performance of your different placements and colors. Sometimes a change in style will generate short-term spike in user interest but fall back to the same or even lower levels afterwards. I experienced that myself. I did something as simple as changing the color of the border. Ad clicks initially spiked, but a few months later they dropped. And they continued to drop even below the level of activity that I had before! Ultimately went back to my original style of ads and activity picked back up again.

Getting Paid More Per Click

Finally, generating higher-paying clicks. As mentioned above, Adsense is the front end for the Adwords program. Adwords allows advertisers to bid on various keywords. Because it’s a bid process (meaning Google doesn’t set prices) the advertisers are in control. If you have sites already set up, your content is set, and the keywords related to that content are likewise set. In other words, your rate of return for clicks based on your content are set by the Adwords bidders, and there’s really nothing you can do to alter that. There are some folks that will suggest that your site needs to be about topic X, Y, or Z in order to generate big returns on clicks, but to be honest unless you have expertise or interest in those topics it’s hard to get interested in building a site around those areas. At least it is for me. I would stick with your current sites unless you have other interests and then you can consider branching out to new areas.

There is an option provided by Google where you can tag certain content areas on your site and ignore others. For example, if you have a discussion board (forum) or blog, you would want to tag your content and ignore the template text (like menus and so on). That way your keyword density goes up and you might get better ads. But you really can’t determine how much those ads pay.

Once your site gets big enough it may draw the attention of specific advertisers. If you can do that, it really helps. Those advertisers will bid on your site, specifically, rather than content-driven keywords. Some of those ads will be paid on an impression (page view) basis rather than clicks. I always like it when this happens, because my site does millions of page views on a monthly basis. You can find out if this is happening by going to the advanced report tab if your adsense account. Choose Adsense for Content, then down near the bottom of the parameters you will see an option to “Show data by…”. When you select Individual Ad you will get an additional checkbox that allows you to break out income by contextual or placement.

Contextual ads are there because of keywords. Placement ads are there because the advertiser told Google to specifically target your site. Placement ads might be paid per click, but they may also be paid per page views. From what I have read, you can’t expect this to happen unless your site content is highly unique or you have enough traffic. So we’re back to generating more board traffic at this point. I don’t have any specific knowledge, but 500-700 impressions a day probably isn’t going to get you any placement ads. I first started noticing this because I had days with revenues and no clicks, so I looked into it further. That was back when I was generating 10-15K impressions a day, and I do more than that now.

Summary

So, to summarize:

If you can increase traffic then one would expect your clicks and therefore your income to increase.
If you can generate more clicks from existing traffic, your income should also increase.
You could try different content to see if the clicks pay out at a higher rate, but then you’re starting over with a brand new site.

But it’s really hard to generate more income per click on existing content because that’s controlled by the Adwords bidders.

Finally, you have to be aware of the fact that Google does not publish their algorithms for determining ads that are displayed on your site, nor do they publish how much money they retain for each click on your site. Either or both of these things could change and have a dramatic impact on your earnings from the program. There really isn’t much that you can do in this area, other than be sure not to rely on Google adsense income for your house payment.

What I have seen over the years is my traffic is going up every year. That is a success for me on the first point. My clicks as a percentage of page impressions have actually gone down, so you could say I have failed on the second point. However, even here there are things outside of my control that I believe are affecting my click rates. It could be blamed on the increased use of Firefox which has several adblockers built in, or on increased “ad blindness” due to the fact that Google ads are everywhere now.

In short there is no magic button that you can press to increase your earnings from this program. You can coast along with what you have now, or you can try to make improvements in some of these areas.

First I added some javascript to the page (but only for that topic) that made the Reply and Quote buttons move away from the mouse. That made it impossible to click on the button, but you could still tab to the buttons and invoke the required code. Yesterday I switched the normal images for the buttons with the spacer.gif and sized it to zero by zero pixels, essentially making the button invisible. I also altered the tab index to -1 which according to a few sites I read makes the button disappear from the tab sequence.

Of course there are still several ways for folks to post in the topic. That’s sort of the point, to see how long it takes folks to figure out how to work around the challenges I have put in place. For example in the first version someone could disable javascript and the buttons would no longer move, giving them another way to click the button rather than using the tab key.

To continue the fun, I am looking for suggestions for other ways to challenge folks, and keep them from posting in that one topic. The key is there has to be some sort of loophole, as I’m not trying to completely lock folks out.

Any ideas?

At the time I didn’t have a good explanation (or even a theory) as to why this was happening. As luck would have it, I’m now getting zero dollars from Adsense for Search and am still waiting for a response from Google regarding the matter. While trying to determine what the issue could be, I have been reading the support forums for Adsense. I saw a post that suggested why my search revenues were rising.

Adsense Drivers

Content ads are driven by – naturally enough – the content on my web pages. Who is responsible for the content? Normally that would be me. In the case of a web site that includes a phpBB board, however, the content is created by anyone who participates in a topic. So any board member is creating content for my site.

But are those board members potential customers? Is the content they create suitable for advertising? In many cases the answer is no.

Board Customers Versus Board Members

But what about a board “customer” rather than a board member? I have statistics that show that about 80% of the visitors to my site are guests. They’re not providing any content at all, they’re consuming the content that is already there. They’re looking for something, whether it’s the answer to a question or something else. And how do these customers find what they’re looking for?

They search.

And if they use the Google search widget at the bottom of my page, then Google is quite aware of the key words they entered during the process. Those key words are words that the customer is interested in right now and that’s why they’re searching for them. Let me use a quick example… suppose I ran a discussion board related to a particular type of car. There might be lots of posts about performance tires on my board, so when someone reads a topic about these products it would make sense for Google to place ads about tires on the page. However, the person reading the page might not be interested in buying tires at the moment… he might just be asking a question about the performance characteristics or something else. But when someone searches the board for information about the tires, it’s assumed they have a specific interest… perhaps they’re looking for reviews from consumers before they buy tires. The search results page becomes more valuable real estate because the person is actively looking for tire information and not just reading about it for who knows what reason.

This makes those keywords more valuable than words that a board member happens to put on the page. And since they’re more valuable, they’re willing to pay more. Because they’re paying more, I am earning more. It all makes sense.

Conclusion

Despite this revelation, I still don’t have Google search active on my site. Until I can understand why all of my revenues are all of a sudden being retained for “search costs” I will leave it off. But I like the explanation and decided to share it here as a follow up post.

See you next year.