March 8, 2008Testing Template Engines in phpBB2 Part III: Data Collection Complete
I have completed the data collection phase of the template testing. I have run 10 different scenarios all on my isolated development server. A couple of scenarios are going to be tossed from the final results due to bugs in code or collection anomolies.
So here is what I have to evaluate:
| Trial Number | Description | Rows |
|---|---|---|
| 1 | Single concurrent session 2 second meta refresh Template Engines Tested:
|
13,620 |
| 2 | Single concurrent session 2 second meta refresh Template Engines Tested:
|
19,052 |
| 3 | 12 concurrent sessions 5 second meta refresh Template Engines Tested:
|
18,812 |
| 4 | 22 concurrent session 5 second meta refresh Template Engines Tested:
|
19,308 |
| 5 | Single concurrent session 5 second meta refresh Template Engines Tested:
|
23,441 |
| 6 | Bugged code invalidated results, ignored | N/A |
| 7 | Single concurrent session 2 second meta refresh Template Engines Tested:
|
16,709 |
| 8 | 12 concurrent sessions 5 second meta refresh Template Engines Tested: Same as above |
17,832 |
| 9 | 22 concurrent sessions 5 second meta refresh Template Engines Tested: Same as above |
19,241 |
| 10 | 35 concurrent sessions 7 second meta refresh Template Engines Tested: Same as above |
21,111 |
Session 1 did not include all four template engines and will not be used for the final analysis. Sessions 2 through 5 used a prototype version of the Speedy Template code and will not be used for the final analysis. Session 6 included bugged code from the Speedy Template prototype and will not be used.
Session 7, 8, 9, and 10 are where I will focus my attention at this point. What I plan to do is take each template + page combination and remove the top “n” and bottom “n” rows from the collection in order to leave exactly 1,000 page refreshes per each. By eliminating the top and bottom outlier records I hope to eliminate any advantage or disadvantage due to a brief server load. With four template engines and four pages and four data collection sessions that means I should end up with exactly 64,000 rows of data to work with. That even fits nicely within the range of my older version of Microsoft Excel. 