Firefox 3 Performance Gets a Boost, Opera Still Champ

Thanks Azmodan for this submission in Back Page News, from CyberNet on its cross browser tests... I've compiled the results of the SunSpider javascript Benchmark test for each of the different browsers. All of the tests below were performed on the same Windows machine, and the Firefox 3 nightly builds definitely came out on top. Here are the results sorted from best to worst (each one is hyperlinked to the full stats):

  1. Firefox 3 Nightly (PGO Optimized): 7263.8ms
  2. Firefox 3 Nightly (02/25/2008 build): 8219.4ms
  3. Opera 9.5.9807 Beta: 10824.0ms
  4. Firefox 3 Beta 3: 16080.6ms
  5. Safari 3.0.4 Beta: 18012.6ms
  6. Firefox 2.0.0.12: 29376.4ms
  7. Internet Explorer 7: 72375.0ms

Very interesting, however I haven't compiled the latest nightly build to check it's performance. My results elaborate that Opera champs over Firefox 3 Beta 3. What about you guys? Also, tell us what you're using by voting for your browser!

Link: CyberNet Results + Full Article

Report a problem with article
Previous Story

Apple To Host iPhone SDK Launch March 6

Next Story

Yahoo Launches Buzz

93 Comments

Commenting is disabled on this article.

remember that the results of this kind of benchmark is difficult to assess... a lot of thigns can happen, including extra caching because Firefox (for example) could have detected repetitive calls and optimize the calls. In the end, if this kind of thing happen, it would certinaley not mean that Firefox is faster than IE for instance. Benchmarks != real world scenarios. Although, it seems obvious that somethign could be done with IE7 's performances, we can't even be sure that the tests are not biased in the first place or if some browsers are optimised to perform better in specific tests (like graphic card drviers are when they detect benchkmarking is in progress). Actually, it could even be the opposite: IE may be optimised for common things you find on the web and Firefox may be optimised in general which would give better results for things like cryptography and decompression tests (something you rarely see on the web, if ever).

All I know is that using IE7 to go on Neowin is as fast on my system as using Firefox, at least, from a human point of view. So I have a hard time believing the other browsers are suppsoed to be 5-10 times more efficient in JS.

For 2008030206 the test takes only 3241.8ms to complete (and that's with the browser running in a sandbox.) Very nice.

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 2396.0ms +/- 0.5%
--------------------------------------------

Firefox 3.0b4pre nightly 2008030106

Full Results

firefox 3 is awful. i presume that i will be sticking with v2.

i know it's still beta but i don't feel that they will make it better with the final release.

I have Firefox 2.0.0.12 and IE7 and they have the same result.

RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 10959.8ms +/- 2.0%

Am I doing something wrong?

I think the mistake that is made is saying "partial JS test = browser performance". The fact that webkit smashes all the other browsers in this test does not mean it's good for daily use.

Yeah but still, webkit is a lot faster than Safari and maybe more stable to me. It's a memory hog though, especially on macs I think...

On my PC, Opera has always been fastest browser. Tried firefox but didn't like it. IE with IE7pro is the alternative when some pages seem broken in Opera. But damn, IE7 is really slow to respond and heavy.

Test results on my C2Q Q6600 machine (from is Firefox 2.0.0.12, to is Opera 9.5 build 9807):

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           2.12x as fast     13006.6ms +/- 0.7%   6140.4ms +/- 1.4%     significant

=============================================================================

  3d:                  3.44x as fast      1871.6ms +/- 1.7%    544.2ms +/- 3.0%     significant

    cube:              3.26x as fast       531.2ms +/- 3.6%    162.8ms +/- 6.4%     significant

    morph:             4.89x as fast      1024.6ms +/- 1.1%    209.6ms +/- 5.1%     significant

    raytrace:          1.84x as fast       315.8ms +/- 2.9%    171.8ms +/- 0.3%     significant

  access:              1.77x as fast      1337.6ms +/- 1.3%    756.2ms +/- 1.5%     significant

    binary-trees:      2.62x as fast       131.4ms +/- 22.3%     50.2ms +/- 17.7%     significant

    fannkuch:          1.16x as fast       412.6ms +/- 2.6%    356.2ms +/- 2.4%     significant

    nbody:             3.13x as fast       606.4ms +/- 2.7%    193.6ms +/- 5.5%     significant

    nsieve:            1.20x as fast       187.2ms +/- 0.3%    156.2ms +/- 0.4%     significant

  bitops:              3.97x as fast      3115.4ms +/- 1.4%    784.0ms +/- 2.8%     significant

    3bit-bits-in-byte: 3.58x as fast       234.0ms +/- 0.0%     65.4ms +/- 24.8%     significant

    bits-in-byte:      2.22x as fast       222.0ms +/- 4.1%     99.8ms +/- 10.5%     significant

    bitwise-and:       5.58x as fast      2353.2ms +/- 2.2%    422.0ms +/- 3.3%     significant

    nsieve-bits:       1.56x as fast       306.2ms +/- 3.4%    196.8ms +/- 5.4%     significant

  controlflow:         1.33x as fast        90.8ms +/- 9.1%     68.4ms +/- 15.9%     significant

    recursive:         1.33x as fast        90.8ms +/- 9.1%     68.4ms +/- 15.9%     significant

  crypto:              1.96x as fast       600.4ms +/- 1.7%    306.2ms +/- 3.4%     significant

    aes:               1.36x as fast       190.8ms +/- 4.5%    140.8ms +/- 0.4%     significant

    md5:               2.61x as fast       203.2ms +/- 6.7%     78.0ms +/- 0.0%     significant

    sha1:              2.36x as fast       206.4ms +/- 4.3%     87.4ms +/- 12.2%     significant

  date:                2.95x as fast      2006.8ms +/- 2.3%    681.4ms +/- 1.6%     significant

    format-tofte:      2.61x as fast       619.0ms +/- 1.7%    237.4ms +/- 3.7%     significant

    format-xparb:      3.13x as fast      1387.8ms +/- 3.2%    444.0ms +/- 2.3%     significant

  math:                2.49x as fast      1165.4ms +/- 2.7%    468.8ms +/- 5.0%     significant

    cordic:            2.71x as fast       550.0ms +/- 3.8%    203.2ms +/- 6.7%     significant

    partial-sums:      2.16x as fast       384.4ms +/- 2.8%    178.2ms +/- 5.9%     significant

    spectral-norm:     2.64x as fast       231.0ms +/- 3.6%     87.4ms +/- 12.2%     significant

  regexp:              *1.11x as slow*     674.8ms +/- 2.4%    747.0ms +/- 1.1%     significant

    dna:               *1.11x as slow*     674.8ms +/- 2.4%    747.0ms +/- 1.1%     significant

  string:              1.20x as fast      2143.8ms +/- 1.2%   1784.2ms +/- 1.4%     significant

    base64:            4.14x as fast       490.6ms +/- 3.6%    118.6ms +/- 9.2%     significant

    fasta:             1.16x as fast       362.6ms +/- 2.4%    312.6ms +/- 4.4%     significant

    tagcloud:          1.30x as fast       368.6ms +/- 4.8%    284.4ms +/- 3.1%     significant

    unpack-code:       *1.47x as slow*     628.0ms +/- 4.5%    925.2ms +/- 1.0%     significant

    validate-input:    2.05x as fast       294.0ms +/- 10.0%    143.4ms +/- 6.1%     significant



I'd use Opera more if it didn't occasionally prevent me from typing into the RTE post boxes on the forums, and if they'd make Netvibes work properly as ever since the 9.5 builds were released the page reader fails to work. The one good thing coming out of these snapshot builds are the UI updates, namely ones done to the native skin. In addition the past two or so weeklies now offer the tab closing functionality many Firefox to Opera switchers were asking for.

I notice Opera has very agressive caching-- it will do cute things like, on some forums which login redirects back to the home page, it will retrieve the cached version, thus defeating the purpose of logging in

(Darken said @ #31.1)
Because these Webpages (Websites) are coded by a dick-heads. :P

Not much you can do though, can you? And don't go judging a book by it's cover.

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 6750.0ms +/- 0.8%
--------------------------------------------

Opera 9.5 Build 9807

(adrianarrakis said @ #28)
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 6130.2ms +/- 0.4%

Using opera 9.5


What is the point of this post? The test result is dependent on how fast your computer is. You can't just post one result and expect it to mean anything - you'd have to test a load of different browsers/builds on your same computer to get results that can be compared to each other..

(Neo Razgriz said @ #27)
What is PGO Optimized?
Profile Guided Optimization. Basically, it runs the code to see what functions get used the most and optimizes the final binary around that data.

(ThePitt said @ #26)
and how much memory uses? 2? 3GB?.
Download a nightly and see for yourself. In case you haven't been following Fx3 development, memory usage is an area which has also been undergoing improvement.

IT uses less memory for the core browser engine, BUT it still has the major suckign memory bug that has been there since 0.1 days.

Why not? These beta and nightly builds shows up the latest development on the browsers. Internet Explorer 8 passed the Acid2 test, and there's not a public test out yet.

I'm not saying that the speed increases aren't legitimate but I am saying that what is released as a beta or a nightly is often quite a different beast than what makes it to final and is released to the masses as such.

These builds may very well be optimized and efficient but the bugs and flaws which remain by the very nature of them being betas could very well translate into some concessions for mass-support.

In other words; when Joe Average hits "update" in his browser of choice it's that final build which truly counts to the vast majority of users in terms of speed, efficiency, and optimization. Anything else is simply a soon-to-be-replaced build which shouldn't count in benchmarks as it isn't what most users will be using.


============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 6332.8ms +/- 1.1%

Fx B3

(KenAF2 said @ #23)
Cryton,

How about a link for everyone?


Not from me; I don't want to link everyone to an untested nightly build that's just had a new feature turned on and may or may not be stable and may or may not cause data-loss or have other unforeseen problems.

Nightly testers who are aware of the risks know where to get such builds, but I wouldn't want to be responsible for anyone hosing their profile with a nightly build I link to.

(KenAF2 said @ #22)
You can download the top performing build (#1 in Neowin post) here:

http://people.mozilla.com/~tmielczarek/fir...en-US.win32.zip

It's more than twice as fast as Firefox 3.0b3, 30% faster than Opera, and ten times faster than IE 7.0.


No, you don't want that - It's a PGO test build from 2008.02.25. It contains unstable code that can lead to dataloss (sqlite and mozstorage are PGO optimized and there's bugs in this).

You want the latest windows firefox nightly trunk build, which has PGO enabled, and at least one other javascript optimization since tmielczarek's test build.

(Azmodan said @ #21.1)

Isn't Camino just Firefox ported to MacOSX with Aqua? :blink:

Yes, but it's faster than Firefox on the mac because it utilizes Cocoa in OS X.

(EXO242 said @ #21.2)
Yes, but it's faster than Firefox on the mac because it utilizes Cocoa in OS X.
So does Firefox 3....since the early alphas...

(RyanVM said @ #21.3)
So does Firefox 3....since the early alphas...

I didn't know that. Thanks for the info though.
I haven't tried it yet.

(RyanVM said @ #21.3)
So does Firefox 3....since the early alphas...
Not quite.

Firefox now uses Native Cocoa page widgets. so, that buttons and web forms look like OSX now.

Gecko still renders Firefox's toolbars inside the window and such, whereas Camino uses Cocoa for that and Gecko only for the page layout.

[edit]OK, I'm wrong about this. Gecko now uses the Cocoa API to draw everything, including native Aqua page widgets. Firefox them uses Gecko to draw the API..
My understanding of Mac app development is minuscule, so I'll shut up now. :P

This is great stuff, I like Firefox alright, but it was slow....

Now if only it wasn't so ugly in Vista... it isn't final yet, but that design just isn't right. I'll just have to wait and see how IE8 is.

(z0phi3l said @ #20.2)

Does anyone really CARE about IE8?

Didn't think so


I think so... they are called web developers...

I think it is great news for Microsoft to make IE8 standards compliant, since it's usually a pain to get a site to work in IE when every other browser displays the site perfectly as it is.

I'm doing the benchmark thing.. is it just me or the thing is just cycling through the tests again and again and again?
Fine I just read what the test does, lol

And is that really big ****? OK, depending on your machine power (unless you have Quad Pent with all that empowering stuff), wait 20 secs or more for FF to open (with only three extensions installed)... Opera takes just two secs to open here - and what's more, Sandboxed! :P

(TRC said @ #15.1)

That's total nonsense, unless you're talking about launching it on a 486.

Or on a Red Hat box (fitted with C2Ds) over a networked user profile. Firefox does take that long to show its window. I installed the Opera weekly builds and they take roughly 7-10 seconds to display (but much longer if the browser crashed last time it ran).

I like FireFox and looking forward to FF3 although I do find the address bar a bit slow sometimes like a web page won't load but it does in IE6 but if I go through Google on FF2 the link loads quicker dunno if it's a bug I'm on XP X64 with NoScript and GreaseMonkey.

Just to note, the PGO optimization is not unofficial, it is a default optimization since two days ago in nightly builds And also will be in beta 4 which the code froze for last night.

Actually it might get backed out for beta 4 because it introduced some rendering regressions. But it will certainly make the cut for final release.

(Cryton said @ #13.1)
Actually it might get backed out for beta 4 because it introduced some rendering regressions. But it will certainly make the cut for final release.
Yeah I thought about that after posting and just decided to leave the post alone instead of ranting on about that it may be but it caused regressions, not sure what else it could be screwing up since it was disable for a few things like the databases and some other stuff, etc.

I got these results:

Latest Firefox 3 Beta 4 Nightly: 2491.2ms +/- 1.5%
Opera: 5368.4ms +/- 1.6%
IE7: 15636.4ms +/- 5.6%

I tried switching to opera, but it didn't feel the same, don't understand why the buttons are all different and the fonts are strange on some sites...

David.

(bangbang023 said @ #9.1)
True, but look at Neowin. It requires a lot of javascript, especially on the front page.

It shouldn't need any of that crap... at least in the ultra lean version (both accessible and fast) that a lot of people wished.

(neufuse said @ #9)
How often do people do massive computations or looping statements in java script anyways? :suspicious:
Consider that much of the browser interface runs on javascript and more will be in the future. Overall UI snappiness has been one very pleasant side effect of the javascript optimizations

(RyanVM said @ #9.3)
Consider that much of the browser interface runs on javascript and more will be in the future. Overall UI snappiness has been one very pleasant side effect of the javascript optimizations :)

not for all browsers it doesn't. in fact I'm pretty sure we're only talking one browser who does that

(HawkMan said @ #9.4)
not for all browsers it doesn't. in fact I'm pretty sure we're only talking one browser who does that :)
True, but it does matter for the browser we're actually talking about having the big speedups with...:rolleyes:

Well, unfortunately I never chose which browser I wanted based on how fast it goes. I first moved to Firefox for the security and got hooked by the features.

(Dakkaroth said @ #7)
Well, unfortunately I never chose which browser I wanted based on how fast it goes. I first moved to Firefox for the security and got hooked by the features.

I'm hooked on FF as well. I tried Opera, just didn't like the feel. No switching for me! :P

That's about the worst reason in the world to use Firefox. Other than that memory leak which it still has, Opera is far more secure than Firefox.

Find these kind of tests, especially testing betas and nightly builds very inaccurate. It's all about YOUR OWN machine and how you use it. Personally and although I don't use EITHER of these browsers, Opera will kick Firefox's a** anyday of the week in ANY category except how many addon you need to use it!!

You can't really count nightly builds or an unofficially optimized nightly build as being full time products. So much can change night to night that it's not an accurate representation and is best used only for reference purposes.

(bangbang023 said @ #5.1)
You can't really count nightly builds or an unofficially optimized nightly build as being full time products. So much can change night to night that it's not an accurate representation and is best used only for reference purposes.
It is not unofficial, it is a default optimization since two days ago

(xiphi said @ #1)
How is Opera still champ if Firefox 3 comes out on top?

The last sentence was my personal opinion. I did the test with Opera and Firefox 3 Beta 3, and Opera won by speed. However, the author of the article claims that Firefox 3 Nightly build (Pre Beta 4) is faster than any other browser in that list.

I guess the news title is missleading.

Azmodan's original post makes it quite obvious that the last sentence is his opinion. But Steven Parker's quotation of the CyberNet results directly followed by Azmodan opinion confuses the issue, and when combined with a stupid article title, confusion reigns supreme!

What's kind of amusing is that I get an ad for a "Flock" web browser that is apparently "designed for Today's Web" just to the right of this article :P

-Spenser

I'm wondering that myself....

Steven Parker? It's your title - explain it (especially since you haven't tested the build discussed in the article.)

(Neobond said @ #2.1)
You guessed it! :D

Riiiight. However, the flaw in your argument is that the version of Opera listed is beta as well.

OK, Opera (even though it is also a beta) is still the main stream fastest browser, it is faster than Firefox 3 Beta 3 isn't it? Therfore it is still Champion of the world

Davebo: So is Firefox 3 Beta 3 which is SLOWER!

(Neobond said @ #2.4)
OK, Opera (even though it is also a beta) is still the main stream fastest browser, it is faster than Firefox 3 Beta 3 isn't it? Therfore it is still Champion of the world </echo>

Davebo: So is Firefox 3 Beta 3 which is SLOWER!

You could at least point out that the improvements are in Firefox 3 beta 4, which IS(or will be when it's released in the coming days) the fastest of the lot.

Neobond, you might consider adding "javascript" to the title somewhere, since this has nothing to do with HTML or css or mathML or any other important benchmark.

I mean as huge of a fan as I am of Fx, Krestal probably still kicks Gecko's ass in most of the benchmarks.

(shakey_snake said @ #2.7)
Neobond, you might consider adding "javascript" to the title somewhere, since this has nothing to do with HTML or css or mathML or any other important benchmark.

I mean as huge of a fan as I am of Fx, Krestal probably still kicks Gecko's ass in most of the benchmarks.

I thought they dumped Gecko for a different rendering engine in FF3?

(Kushan said @ #2.8)
I thought they dumped Gecko for a different rendering engine in FF3?
Uh, no. Gecko has had some massive upgrades, but it's still humming along at version 1.9 (Gecko 1.8.1 is what Fx2 is running).