In the first post in this series I showed some data from my phpbb2 honey pot board that has been collecting spammers for several months now. One of the most interesting observations (as far as I am concerned) is the posting frequency. The posting bot would log on, post, wait 25 seconds, post a second time, wait 25 seconds, post a third post, and then log off for several hours. This behavior would repeat throughout the day with the same user account coming in from different IP addresses around the Earth.

I suggested that this behavior was an indication of “zombie computers” and since today is Halloween it seems a good time to finish the topic.

Zombie Computers

Zombies are computers that have been compromised in some fashion. It the “good old days” a hacker might try to write a virus that would disrupt or even destroy your data. Those viruses made themselves obvious when you tried to reboot your computer and found out your data or even perhaps your entire operating system was gone. The thrill of destroying individual computers was then replaced by the next phase of virus attacks like macro viruses that were responsible for sending out hundreds or thousands of emails to all your friends and business associates. Attacking web servers directly or indirectly became another favorite pasttime.

Today viruses (and I am using the term in a generic sense here) have gone stealth. Instead of drawing attention to themselves they hope you never even notice they’re on your computer. Why?

SETI Strategy

A few years ago someone realized that there were a lot of computers sitting around doing not much of anything for extended periods of time. They wrote a simple application that would connect to a server and download a data packet of information collected by SETI and use your computer idle time to process the data. The process was extended into background applications that were searching for a cure for cancer, processing genetic coded data, and all sorts of other things.

The basic concept here is that there is a huge distributed network doing a lot of processing but in little bits at a time. Volunteers were okay with this as long as they never really felt like their computer was getting slower.

Spammers noticed this. The problem is, while people would willingly download and run something that was looking for signs of extraterrestrial intelligence, they weren’t as excited about using the background clock cycles to send out spam.

Braaaiiiinnnnssss

In horror movies it seems that it is always the blonde that says, “Let’s split up and look, it will go faster.” Bad idea. Clicking that link that promises to free your computer of spyware might just be doing the exact opposite. I am far from a security expert, so I won’t try to describe all of the various ways that you can get infected. Suffice it to say that lots of computers are infected. By some estimates, over half a million computers are currently infected and the number grows daily.

I read one article recently that suggested that spammers were even getting pickier. Their zombie process would first test the speed of the available internet connection. If the connection wasn’t fast enough, they would move on and look for a better option.

But zombies need brains, and that’s the interesting part of how this works. Once a computer is infected, at some point it will wake up and “phone home” either by logging in to an IRC channel or perhaps connecting to a specific web site. Before the connection is established the zombie doesn’t know what to do. By connecting to the spammer server it can download a database of email addresses to spam, or bulletin boards to post on.

Stay in the Shadows

The same strategy that allows Seti@Home to use background clock cycles without being noticed also allows the zombie processes to run undetected. Rather than spin up a CPU-intensive application that causes the computer to slow down in a noticable fashion, the process will lie quiet except for brief periods of activity. At various points throughout the day it will wake up, spam a few boards, and then go quiet again. By doing this the zombie processes hope to remain undetected.

Does this behavior ring any bells? Let me repost some of the data I shared last time:

| 5995fddf  | 2008-08-25 00:10:47 |
| 5995fddf  | 2008-08-25 00:11:12 | 25 seconds
| 5995fddf  | 2008-08-25 00:11:37 | 25 seconds
| 5995fddf  | 2008-08-25 05:14:38 |
| 5995fddf  | 2008-08-25 05:15:05 | 27 seconds
| 5995fddf  | 2008-08-25 05:15:30 | 25 seconds
| 5995fddf  | 2008-08-25 18:28:24 |
| 5995fddf  | 2008-08-25 18:28:51 | 27 seconds
| 5995fddf  | 2008-08-25 18:29:16 | 25 seconds
| 5995fddf  | 2008-08-25 22:58:15 |
| 5995fddf  | 2008-08-25 22:58:40 | 25 seconds
| 5995fddf  | 2008-08-25 22:59:04 | 24 seconds

What I showed was only part of the story. It clearly shows that one zombie computer from Germany is logging in to my board and posting batches of 3 posts at a time. In the initial attack there was one user account posting. They posted 94 times on my board and then I decided to play with him a bit. I disabled and banned that particular account. Here’s where it gets really interesting…

Adaptive Behavior

I banned the account but not the IP address. The account had successfully posted a batch of three posts at 8:08, then repeated again at 9:20, then again at 18:00, and finally at 20:33. They had posted in a similar fashion from August 22 all the way through to August 27 when I disabled the account.

On September 9^th they were back, and with a completely new behavior. At that point their usernames became a simple numeric value. I say “usernames” (plural) because the new behavior was apparently designed to thwart my banning practice… they would register a username, post three times, and that was it. The next time they came back the registered a brand new username and repeated the process. Here is what that data looks like:

+----------+---------------------+---------------------+
| username | reg_date            | post_time           |
+----------+---------------------+---------------------+
| 88199920 | 2008-09-09 12:05:21 | 2008-09-09 12:06:07 |
| 88199920 | 2008-09-09 12:05:21 | 2008-09-09 12:06:32 | 25 seconds
| 88199920 | 2008-09-09 12:05:21 | 2008-09-09 12:06:56 | 24 seconds
| 80811315 | 2008-09-09 12:56:37 | 2008-09-09 13:01:19 |                  new user 51 minutes
| 80811315 | 2008-09-09 12:56:37 | 2008-09-09 13:01:44 | 25 seconds
| 80811315 | 2008-09-09 12:56:37 | 2008-09-09 13:02:08 | 24 seconds
| 69297749 | 2008-09-09 14:35:20 | 2008-09-09 14:40:36 |                  new user 99 minutes
| 69297749 | 2008-09-09 14:35:20 | 2008-09-09 14:41:00 | 24 seconds
| 69297749 | 2008-09-09 14:35:20 | 2008-09-09 14:41:24 | 24 seconds
| 29788522 | 2008-09-09 15:24:48 | 2008-09-09 15:27:01 |                  new user 49 minutes
| 29788522 | 2008-09-09 15:24:48 | 2008-09-09 15:27:26 | 25 seconds
| 29788522 | 2008-09-09 15:24:48 | 2008-09-09 15:27:50 | 24 seconds
| 80072273 | 2008-09-09 16:12:52 | 2008-09-09 16:20:49 |                  new user 48 minutes
| 80072273 | 2008-09-09 16:12:52 | 2008-09-09 16:21:14 | 25 seconds
| 80072273 | 2008-09-09 16:12:52 | 2008-09-09 16:21:38 | 24 seconds
| 36003012 | 2008-09-09 17:02:09 | 2008-09-09 17:07:10 |                  new user 49 minutes
| 36003012 | 2008-09-09 17:02:09 | 2008-09-09 17:07:35 | 25 seconds
| 36003012 | 2008-09-09 17:02:09 | 2008-09-09 17:07:58 | 23 seconds
| 45029267 | 2008-09-09 17:50:00 | 2008-09-09 18:01:25 |                  new user 48 minutes
| 45029267 | 2008-09-09 17:50:00 | 2008-09-09 18:01:52 | 27 seconds
| 45029267 | 2008-09-09 17:50:00 | 2008-09-09 18:02:16 | 24 seconds
| 13234111 | 2008-09-09 18:35:33 | 2008-09-09 18:41:11 |                  new user 46 minutes
| 13234111 | 2008-09-09 18:35:33 | 2008-09-09 18:41:36 | 25 seconds
| 13234111 | 2008-09-09 18:35:33 | 2008-09-09 18:41:59 | 23 seconds

… and it goes on from there. This is one of the reasons why I have so many users registered from the same IP address. Each username was a “throw-away” attempt to get into my board, post their spam, and then move on without relying on a single username anymore. Once they got rolling, they registered a new user approximately every 49 minutes. The one time they didn’t register a new user in 45-50 minutes they came back 99 minutes later, which is essentially double the interval. It still fits the pattern, they just skipped a time slot for some reason.

Bots attack. I react. Bots change their behavior. When the zombie checks in with the zombie master they might even be getting code updates. Heck, it works for Microsoft, why can’t zombies get auto-updates too?

Conclusion

This is barely scratching the surface of zombie / bot behavior. The wiki links (see below) provide some interesting reading. I found a bunch of articles with some basic google search terms. What I found most interesting was how well the data I am collecting fits the exact behaviors that I expected to see.

And this is how I think my Panama spammer was able to get to the UK from Panama in less than half a day. Once the user account got registered it was sent across the globe via IRC or some other protocol. Spamming via background distributed networks.

What can be done about this? The experts don’t know, and I can hardly claim to have the silver bullet. (I am mixing horror movies, I know, but it’s Halloween, so give me a break. ) Services like bbProtection hope to protect your board. Tweaks like the Checkbox Challenge can help for a while. But ultimately the zombies are still out there, so the problem isn’t gone, it’s just hiding. It should be clear that this is not a phpBB problem. It’s an Internet problem.

But now it’s time to take my kids trick-or-treating in the neighborhood. Happy Halloween, and watch out for zombies.

Related Links

• Wikipedia on Zombie Computers
• How Stuff Works: Zombie Computers
• Seti@Home