I am very excited that phpBB user Brainy has been working on an updated template engine for phpBB2. However, at the moment I can’t get it to work completely. Last night, when I was up late working on other things already anyway, I decided to finally take a look at the template options from the /contrib folder from a standard phpBB2 installation. I was glad I did.
I ran three trials. The first trial included Categories Hierarchy, eXtreme Styles, and the file cache code from the /contrib folder. I called it “phpbb-cache” for clarity. On the first trial I ran a single session with a two second refresh. On the second trial I ran 12 sessions with a five second refresh. On the last trial (the “load test”) I ran 24 sessions with a seven second refresh. The last one is really the most interesting, since it is extremely rare that my largest board has only a single active session.
The reason I ran these trials is I am rolling out a huge list of updates next week, and I really want to get off of eXtreme Styles. The two main reasons for that are (1) based on my testing eXtreme Styles does not scale well under load, and (2) an exploit was recently discovered in this code so I would rather drop it even though I have fixed the known exploit.
All trials were run using the same process and server that I have outlined before.
What Am I Looking For
I look at four numbers on my testing. First, I capture the specific time that it takes to parse the page using the pparse() function. But as has been pointed out elsewhere, that is not the complete picture. A template engine might parse well but do other things inefficiently. So I also track the page generation time, starting in common.php and ending in page_tail.php. That is the second number, and is probably the most effective at measuring total template throughput. Third, I track the database time (SQL time) for each page. In theory, they should be nearly identical for all template engines, since the database doesn’t change from one run to the next. Finally, I calculate the standard deviation of the total page times. What I am looking for here is consistency. The template engine with the lowest standard deviation is the most consistent, and therefore the better performer under system load.
I also try to ensure that I have a fair sample of values. In this sample I have over 500 refreshes for each template engine for each page being tested. (I am only testing pages that can be refreshed, for obvious reasons. I don’t want to have to write a script to testing posting or other input screens.)
Here are the results, sorted by page name and then by average page time.
|Page||Template Engine||Average pparse()||Average Page||Average SQL||Std Dev of Average Page|
I know, that’s a lot of numbers. However, my first observation is that phpbb-cache is the top-rated template for three of the four pages in the test. My second observation is that it is also the top-rated for three of the four standard deviation values. From these numbers I conclude that the basic file cache template engine from the contrib folder does a better job under load than either of the other two template engines that I tested.
I think that it’s ironic that the eXtreme Styles MOD benefited from the best SQL performance on the index page, yet still came in last place for total page generation time. I am not sure why the SQL times range as much as they do; I expected more consistency.
For the next step I added up the four average times, divided it by four and called it an “Expected Page” time. Here are the results:
|Template Engine||Expected Page Time|
As a reminder, these numbers were generated by having 24 concurrent sessions refreshing four pages (index, viewforum, viewtopic, and memberlist) one right after the other. This number is a general indication of how efficient each template engine is across the entire board. As you can see, the phpbb caching template did quite well.
Since I have a major upgrade planned for this weekend, and I want to get off of eXtreme Styles, I am going to switch over to the template_file_cache.php engine from the /contrib folder. I am using it for several reasons. First, it works very well. Second, I have not been able to get my adjusted categories hierarchy template engine to work with the attachment MOD, so I cannot consider it. The same issue is present (at least so far) with the new Speedy Template from Brainy.
Oh, and if you are wondering… the phpbb-cache template came in first place on every page on the single session test, and came in second place on every page (ch was first) on the 12 session load test.