Our test suites include the three benchmarks we used last week to demonstrate Safari 4, including HowToCreate's CSS rendering test (since the rendering times can vary, we averaged the times for five renders for each browser), developer Sean Patrick Kane's octathlon which renders a cumulative speed after a battery of eight tests, and the Web Standards Project's Acid3 compliance test, which renders its score as a percentage.
To these, we added one more suite: the SunSpider JavaScript Benchmark, a pentathlon of real-world simulations that include elements like AES, MD5, and SHA1 cryptography puzzles; ray-tracing formulas; and handling of JavaScript Object Notation (JSON) expressions. With this and the Celtic Kane battery, we'll be able to find out where any browser excels and where it fails. Each of our four suites produced a cumulative index score, which we then counted toward 25% of the final score.
The final index scores for our tests indicate one thing conclusively: The performance differences between all the major Windows-based Web browsers, compared to one another, are clearly perceptible. No test browser is close to any other; in everyday use, the differences in JavaScript rendering speed are enough to be seen with the naked eye.
And there is a clear leader in that department: Apple Safari 4.
Microsoft can say that its IE8 candidate has more than double the performance of its predecessor, and it can probably smile at that. IE8 RC1's index score is 2.19. However, the latest Safari 4 beta's score under the same conditions is 14.39, blowing away anything Microsoft produces. We expected more from the Opera 10 alpha, given what that browser's faithful followers told us. But with an index score of 5.38, Opera's performance right now is bested by Firefox 3.1 Beta 3, which finished with a 7.85. But Mozilla will have to settle for the bronze for now. Google Chrome is the one vying for the lead, with a second-place score of 11.62.
What were the browsers' strong points? Safari 4 scored well in the HowToCreate box rendering experiment. In the Celtic Kane battery, Safari shined with regular expressions, handling the Document Object Model, and at parsing RegEx expressions -- on that one part alone, Safari scored a 13.94 versus IE7. Its AJAX parser also proved strong. But the SunSpider benchmark was able to highlight some even brighter points for Safari, including just the way it manages control of scripts. In that heat, it scored a 90.94 versus IE7. And scoring 100% on the Acid3 certainly helps.
Chrome has clear strengths and weaknesses. It was not the fastest at the HowToCreate rendering experiment, scoring only 4.73 against IE7. Its 79% score on the Acid3 set it back. But it managed to outperform even Safari 4 in the SunSpider suite, with a 33.36 overall versus 32.81 for Safari. The control flow heat alone produced a 148.37 for Chrome versus IE7, which is testament to how well that browser's unique system architecture pays off.
The browser that will be Firefox 3.5 has some ground to make up. It actually performed rather poorly with the HowToCreate rendering test, actually loading the test slower than IE7 (0.93) and producing a rendering score of only 1.83. Its 93% score on the Acid3 is the best we've ever seen from a Mozilla browser. However, if this were a math quiz, Firefox 3.1 Beta 3 would win hands down, and here is where we're seeing that "orders of magnitude" difference we were promised for TraceMonkey: In the math heat of SunSpider, Firefox pulled off a 17.24, compared to 8.97 for Chrome and 6.93 for Safari.
Since last December, Opera has waved its flag high atop the mountain that is the 100% Acid3 score, and we saw it again yesterday. Opera did win the day on the HowToCreate rendering test, scoring a 5.03 -- the highest of the test browsers, although that score was offset by its slower loading speed. But Opera couldn't keep up in the octathlon and pentathlon, scoring only 3.05 and 6.67, respectively -- the lowest scores for any non-Microsoft browser.
Internet Explorer 8 was never expected to be a breakthrough performer. What we did not expect was a rendering score that is actually slower than IE7 (0.83). IE8 put in an appreciable 5.05 on the SunSpider suite and a less-than-appreciable 1.67 on the Celtic Kane suite. But scoring merely 20% on the Acid3, these days, may not be forgivable.
During the era of the so-called "browser wars," the only real matchup was between Netscape Navigator and Microsoft Internet Explorer. And that wasn't even really a battle on equal terms, but rather one between performance and functionality versus convenience. IE won that battle. So the fact that we are actually comparing Web browsers' performance on an equal scale today is symbolic of the fact that this market, for the very first time in its history, may be becoming truly competitive.
No comments:
Post a Comment