First come, first served.

Pet Names Don’t Make Good Passwords

People often struggle with remembering passwords. Yes, there are programs that can help you with that, but are they really that much more secure than writing things down on a piece of paper? Here’s one take from one of my favorite web comic authors:

The bottom line is that a password is supposed to protect an account from unauthorized access. It’s not supposed to prevent authorized access, but for infrequently used resources it can do that as well.

Password Storage

One indicator of concern to me is whether a web site can send you your existing password when you forget it. That means they’re likely using an encryption process rather than a hashing algorithm to store passwords. And that means anyone with access to the decryption key can read every password in the system. phpBB2 uses a hashing algorithm which means as a board owner / administrator I can safely say to any of my users that I cannot tell them what their password is. I can assign them a new one, and of course there is a self-service option available to phpBB2 users that will do just that, but I cannot tell them what their existing password is. In a default phpBB2 installation a user password is hashed and stored as a 32 character string. Similar passwords will generate very dissimilar hash strings, so there is no way for someone to easily guess what a password might be simply based on the results of the hash. For example, here are a few similar input values and the resulting MD5 hash outputs:

test     098f6bcd4621d373cade4e832627b4f6
Test     0cbc6611f5540bd0809a388dc95a615b
tset     751ec45015a704a39dc403001c963e97
test1    5a105e8b9d40e1329780d62ea2265d8a

Despite the similarity of the input values they all have very different hash values. That means that similar hash values are going to have very different input values as well. Also, because a hash is generated by a one-way function there is no way to recreate the input value based on the hash. (There are some cases where two different input values could generate the same output hash; that’s allowed.)

If that’s the case, then how can I tell if a user has entered the correct password when they log in? It’s actually really simple. When a user logs in, phpBB takes the password provided on the form, hashes it, and then compares the resulting hash value to the hash stored in the database. As long as the input values are the same (entering “test” as my password will always generate 098f6bcd4621d373cade4e832627b4f6 as the hash value) then the password matches and the user is logged in.

How, then, were hackers able to decipher the passwords stored in the phpBB.com database if there is no way to reverse a hash process? They didn’t. Instead they used a lookup table (also called a rainbow table) to match up known hash values with their source value. Imagine taking a dictionary and running every word in it through the hashing process. When you’re done, you have a list of hashes and their source. By matching password hash values against the list of known entries the hackers were able to figure out what a number of phpBB.com user passwords were.

To help defeat this sort of process, phpBB3 now uses a salted hash which provides even more security. I imagine someone has back-ported this to phpBB2 as a MOD but I have not had time to look for one yet. But the fact is if a secure password is used (a combination of words and numbers, or as the cartoon above suggests several words run together) the odds of the password appearing in a rainbow table are extremely slim, and even the simple hashing algorithm used in phpBB2 is essentially safe. Even changing the case of one letter helps, as shown in these two hash examples from earlier:

test     098f6bcd4621d373cade4e832627b4f6
Test     0cbc6611f5540bd0809a388dc95a615b

Changing the “t” to a “T” changes the hash. It might be even better to change something other than the first letter, for example changing the “e” to an “E” as in tEst. That’s a very easy word to remember and it’s not likely to show up on any rainbow table, unless a hacker wants to run every possible combination of upper and lower case letters for every possible dictionary word. Ultimately it would be better to use a combination of letters (mixed case), numbers, and even symbols where allowed.

So how important is it that phpBB3 uses a salted algorithm for passwords? If people were smart about what passwords they use then it really would not matter so much. But as shown in some of the related links at the end of this post, people are not very smart about the passwords they pick. In that case, the salting process is quite beneficial.

Social Engineering

Which brings me to my final issue for this post: people can be fond of certain passwords and often use (reuse) them on more than one site. If you use the same password for a phpBB board that you use for your banking system, then change it. Change it now. For one thing, most phpBB board owners do not use a secure protocol (such as HTTPS rather than HTTP) for their sites. That means the password is exposed during the transmission of the form data. Ultimately no matter how a password is stored or how complex the password might be, if people can be tricked into giving up their passwords by phishing emails, web site plugins, and other techniques, that’s a problem. If a hacker manages to grab the password for one site, they may then be able to use it on other sites. I have made a practice of using a unique password on every service I sign up for in order to prevent this for a very long time. But once again, xkcd.com says it better:

Related Links

• phpBB Password Analysis
• 500 Worst Passwords

with one of several holiday logos:

And of course this:

During the American Thanksgiving week we used this:

Over at the phpBB2 Refugees site we’ve swapped our normal refugee tent for one with a more holiday feel for the last several weeks:

Right now this is a manual process, which means I have to remember to edit the php code to invoke the new logo (or alternately use ftp to upload a replacement logo with the same name over the current one). Manual process? for Dave?

I’m setting up a database table with dates and alternate logo names, and will set up a cron job that every night at midnight checks to see if holiday logos are needed for the following day or not. On BOB we rotate among several logos, so there could be more than one logo at any given time. The logo information is then written to a cache file where it will be used for the following 24 hours.

Easy as pie.

Holiday pie.

The problem is, it accomplishes nothing.

Private messages are often a hot topic for board owners, probably because of the privacy implications of the name “private” message. As most board owners probably know, private messages are not truly private. Anyone with database access can read the private message text. Anyone with access to a backup SQL dump can do the same. But who has this type of access, and what can be done to prevent it?

Board Owner Access

In many cases, only one person owns and manages a web site that includes a phpBB board. In this case, that person is likely to have the administrator passwords for the phpBB board, the SQL database password, and ftp or even shell access to the server itself. Even if that person does not start out knowing the SQL database password, they can get it easily enough by downloading the config.php file and getting it from there. Now suppose that private messages are encrypted as suggested in the topic listed above. Is the data safe?

No, I’m afraid not. The data in the database is encrypted, but by definition the information has to be able to be unscrambled for the PM recipient to be able to see it. All the board owner has to do is download the php code to obtain the encryption key and then use it to decrypt the data. It turns out it’s not the fact that the data is encrypted or not, because the board owner has access to the data and the tools required to decrypt it. The board owner can still read private messages, it just takes longer.

Encryption Versus Hashing

Passwords are hashed, not encrypted. This means that even though the board owner can see what the hash string is in the field in the database, there is no way to de-hash the data. By definition a hashing algorithm is one-way. But if the private message text is hashed, then there would be no way to get the original text back! That’s why as a board owner I can change a password to something new, but I cannot tell you what your original password was.

The point is, hashing is secure. My password is relatively safe. Encryption by definition has to be reversible, and there is really no way to eliminate all avenues of accessing that information as long as access to the raw data is possible.

It’s About Trust Not Technology

Even in a more complex environment where there is more than one person with access to and permissions to manage a web site there is only so much that can be done to compartmentalize the issue. Someone, at least one person, is going to have access to the server. Even if it takes two or more people (one to get into the database, the other to decrypt the data) private messages are still not so private.

It all comes down to trust. If I don’t want someone to read what I wrote, I don’t write it down. I have to trust that a board administrator is not going to go about reading private messages, or that they’re not going to edit my post to make it look like I’m saying something that I didn’t, or that they’re not going to install a key-logger that captures my password as it’s entered on the login screen, or that they’re not going to try to … well, it goes on from there. Adding encryption to private messages doesn’t fix the issue.

And here’s one more loophole. I mentioned above that password information is hashed and therefore I cannot log in as “you” because I cannot determine your password. But I can do this:

• Log in to the SQL database and retrieve the hash for my password
• Also retrieve the hash for your password and save it
• Update your account so that your hash value is equal to mine. No I “know” your password because it’s the same as mine.
• I log in and do whatever I want to do as “you” because I have effectively stolen your identity
• When done, I reset your password hash back to the original value so you can log in again, and you have no idea that anything has happened

Not very nice, but certainly possible. You just have to trust me not to do these things.

It’s for this and other reasons I have removed the PM feature from the boards that I manage. By removing the illusion of privacy implied by the name “private message” I don’t have to deal with this.

I used the same function to update my canned messages MOD as well.

Oh, and I updated the color picker just a bit. The original color picker used values of 00, 40, 80, and two more to create an array of colors that was 5×5x5. I changed it to the old “Netscape Safe” color palette and used 00, 33, 66, 99, CC, and FF and created a 6×6x6 grid instead.

I have the new posting form activated on two different boards for the moment. As long as I don’t find too many issues I will write it up as a MOD.

I think I’ve managed to come up with something that definitely helps solve the first two scenarios and as a bonus helps the spammer problem as well. I call this my “Cross Post / Double Post” MOD, and it’s being tested on my beta board now.

The MOD design has so far turned out to be fairly simple. I tie into the flood control process and retrieve the post text for the last three posts by the user. From there I take the current post text and compare it to the prior posts. The first check is a straight equality check, meaning I check for the exact same post text. This will catch the “copy/paste” folks with very little overhead. If the post text is not identical, then next I use a function called similar_text(). (similar text reference at php.net) This function takes three arguments. The first two are the two strings to compare, and the third is a variable to store the results of the comparison, which is a number from 0 to 100. The result code should essentially be treated as a percentage. If the two posts are 95% similar then I check to see if the original post already in the database is in the same forum as the new post being attempted. If the forums are the same, then a “Double post” exception is triggered. If the forums are different, then a “Cross post” exception is triggered instead.

The number of posts (3) and percentage of similarity (95) are both controlled via the board configuration screen, so it’s quite flexible. Setting the percentage threshold to zero (0) is the same as turning the comparison process off.

This MOD is being tested on my “beta release” board right now. The first version of the MOD did not use the similar_text() function mentioned above. I attempted to use the soundex() function instead. However it seemed that the soundex() function did not look at enough text, so posts that were clearly different were still being reported as being the same. Switching functions solved that issue.

I’m now wondering if I need to deal with setting different threshold values for different forums. I hate to do that, as it drastically increases the complexity of the code. But for example there are many forum “games” that people play in an “off topic” type of forum. Some of those games look very repetitive, and would potentially trigger the CP/DP exception handling. Then again, the current logic looks across all forums, so as long as the person is active in more areas than just the off-topic games area it might be okay. I don’t want this feature to get in the way of normal use, but I do want to help out the moderator team by capturing / rejecting double post and cross post events.

Stay tuned for details as we start user testing this week.

1. Full-Text Search
   I created a full-text index on the post subject and text over a year ago to see if maintaining that index would cause any performance issues. I’m happy to say that I have not seen any challenges from inserts / updates with this index in place. I’m going to be altering the search screen to allow the full syntax offered by MySQL on this type of index and hope to release that in a few months. Some of the challenges I have not yet decided how to solve are things like limiting forums – either by security or user preference – and other criteria that can be entered on the standard search screen.
2. Capture Post Revisions
   I’ve also added some code to capture post revisions. We’ve had a couple of folks that come back to our board and edit their post, removing all of the text and leaving only something like “…” instead. This destroys the continuity of the topic, and as a result we’re going to now track post revisions by capturing the post text history. If needed a moderator will be able to review and then restore a prior post, and ultimately lock that post from further editing. As with the full text search I have done fairly extensive testing on how this is implemented in order to ensure that performance does not suffer, and I’ll have a few blog posts about that process. This MOD is completed and I expect to roll it out onto the main board in a few weeks. (FWIW, I first talked about this post several years ago, and am just now finally getting it completed.)
3. Moderator Posts
   I’ve added a new field to the post table that allows a moderator to designate whether it’s a moderator post or a user post. For example, moderators can certainly participate in a normal board conversation as a regular person. But they may also add posts in their role as a moderator. This new feature will format those posts differently so they stand out, will automatically remove the “personal” aspects of a post such as signatures, and does not increment a moderator post count for this type of post. It is intended to be a way for moderators to be able to separate out their moderator posts from their board participation posts. This MOD is also completed and expected to be released shortly.
4. Including External Content
   I’ve added some cron jobs that parse RSS feeds from several blogs owned by board members. Their blog posts are automatically set up as part of their signature (as “Latest Blog Posts”) and updated once an hour. For bloggers that our community wants to recognize, this is a great way for them to get additional exposure without having to manually update their signature every time they publish a new blog post. This part of the MOD is already in use on our board. Only board admins can currently enter blogger information, as we want to go through a review process and certify blogs rather than allowing just anybody to link to an external site. This was done by altering the administrator user edit form and leaving the regular user profile form alone.

   As an extension to this, I’m also pulling in the content from the blog post and storing that in a hidden forum. As the blog posts are added to the forum they are obviously added to the full-text index because they’re part of the same table. I am also adding these posts to the standard phpBB2 search tables at the same time. That way if someone searches for term “X” and that’s found in an external blog post, they’ll see a link in their search results. The blog address is stored on the topic table and a different icon is used to show the user that they’re leaving our board and heading to an external site. I have all of the main work done; the last requirement is altering search.php so that it offers the ability to include / exclude external content and then react to that setting accordingly. I hope to get this completed in the next few weeks.

5. Social Media Profile Links
   I’ve added Facebook, Twitter, and LinkedIn fields to user profiles. These are displayed along with the other profile links, using smaller 18×18 pixel logos. I’m planning on going back and redoing the other profile links to use the same form factor but that part hasn’t been done yet. Here are the images I’ve made, using logos or other material provided by each service provider.

One thing that many of these MODs have in common is my concern for performance. We’re over 750K posts now, and still running extremely well on a server that is hosting several dozen sites, although none of them as active as our big board. Every time I touch the code performance is a primary goal. Another MOD that I’ve been planning is to port the phpBB3 posting form back to phpBB2 since it does a better job of supporting modern browsers as well as proving some additional formatting features. I haven’t even started on that yet, but I think it would be good. Now that I’ve personally switched to Chrome as my standard browser I’m noticing some interesting quirks.

So that’s what I’ve been up to for the past few months.

That’s specifically aimed at human-powered paid-to-comment spam. I would rather already have excellent-quality comments than the next quantity of comments.tour headphones Sadly, I’m nonetheless getting an awful lot of spam comments (what’s up, Akismet?), so I think it’s time to install some additional defense layers.

The words “tour headphones” were a link, of course. Subtle, it was not. But I found it extremely ironic and ultimately amusing that the comment itself talked about spam. If you pick a few phrases from that comment you’ll find the exact same thing on other blogs / boards as well, or at least I did when I searched.

I’ve decided to contact the headphone manufacturer directly and let them know that I will never buy their products. Ever. Might not change anything, but it will make me feel better.

Oh, and I added specific code to my anti-spam process to look for this particular type of link.

I have to admit that I find this to be a far more intriguing idea than a mobile template. The folks behind Tapatalk offer a free API that would allow developers to extend the app to different forum systems. It would be interesting to see if anyone is currently working on phpBB2.

Tapatalk

I’m registering on their “Forum owner” area and will see what things look like.

Things have been really busy in real life, which I suppose goes without saying. I have two boys that are growing up and going through cub scouts and sports and school and everything else.

Yet, here I am. I am back to reconsider whether I need to upgrade from phpBB2 to phpBB3 for my main board (which as I write this is very close to collecting it’s 700,000th post, and is running almost one million page views a week). I am back to see if I can figure out how to integrate FB “likes” into my board, and how to allow folks to add Linked-In to their profile. That last one, at least, should be easy.

Things have been humming along quite nicely, although the spam frequency has started to go up. Seems that my Checkbox Challenge is either less of a deterrent, or human spamming is on the rise. For that, I guess I need to look at my server logs and see how long those folks are taking to get through the registration process. Fortunately there is still no answer for the Spammer Hammer, which makes it easy for my moderator team to quickly and easily eradicate all traces of the spammer from the board.

And gmail? Is it still the number one source of spam attempts? It would be interesting to check. I have not looked at those statistics in months (years?).

And I missed Libertyvasion. I had intended to go, but all of a sudden August was here and I had not made any plans, so I stayed home. I have watched some of the sessions posted to Youtube, and it did look fun. I will hopefully be ready for the next one. Anyone know when / where that will be?