JavaScript Engine Benchmark Test Results

Allan Feid — Sat, 15 Nov 2008 00:54:00 +0000

All the rage seems to be who has the fastest JavaScript engine these days. Chrome has it's V8 engine, Firefox with their TraceMonkey, and there is also the WebKit project. What I wanted to do was get some hard data to compare all these engines along with older browsers.

In an earlier post, I showed the difference between the stable release of Firefox vs Firefox 3.1 Beta 1 w/ TraceMonkey enabled using the SunSpider benchmark. Google offers their own benchmark testing as well, so I figured why not try them both out?

The Results

After running the Google tests it became aparent these tests were not cross-browser balanced. This test went through a series of 5 different tests and the higher the score the better. Take a look at this chart illustrating the results:

IE 8: 30
Firefox 3.04: 156
Firefox 3.1b1 (w/ TraceMonkey): 49
Chrome: 1659
Safari (Win): 132
Opera 9: 172
Safari w/ WebKit: 1259

As you can see these benchmark tests are specifically for what Chrome's V8 engine is working on. In fact one of the tests completely fail (returning a score of 1) when run in Firefox's TraceMonkey engine. The only engine that came close to Chrome was WebKit on Safari. I first planned to only run these tests but after looking at the results it became clear this is not a good comparison of JavaScript rendering power.

This lead me to use the SunSpider benchmark which is much more balanced and more of a "real world" test. These results are much more accurate in my opinion and are based on the time to complete things rather than who can get the biggest score. Here's a chart of the results:

Now you can see a much clearer picture in how the engines stack up. TraceMonkey is almost a full eighth of a second faster than the V8 engine. That eighth of a second gives the Firefox engine a 1.31% gain in performance over Google's engine. Both of these engines are still in beta releases of software and will be interesting to see what happens in the future.

All I have to say is that I've never had a problem with Gmail load times until I started using TraceMonkey. Now whenever I'm on a slower JavaScript engine it feels very sluggish. I am thoroughly impressed by what the TraceMonkey team has come up with and most importantly this will cause rougher competition among browsers and bring full scale web apps up to par with desktop applications. It will definitely be an interesting next couple of years to see how this all plays out.

If you're skeptical about these results head over Brendan's Roadmap Updates where he runs similar tests and goes into much more detail to show TraceMonkey being 4x faster than V8. He also uses the latest nightly build to perform the tests where you will see even better results from the TraceMonkey engine. These two engines are no where near being final releases so who knows which one will outperform in the long run.

For WebKit, it does seem to have the best results and outperform all other engines, however, it is unlikely this will really be used in any browser. This engine is more of a proof of concept (and yes it works wonderfully) since Safari is only really used on Mac OS X.

Testing Environment

Testing was done in a virtual machine (VMware Workstation 6.5) running Windows XP SP2 with Ubuntu 8.10 as the host. I loaded each browser and ran the tests all individually and compiled the information using Google Docs. If you have any questions feel free to comment or drop me an email.

Firefox 3.1 Beta 1 First Impressions

Allan Feid — Wed, 15 Oct 2008 22:07:17 +0000

Firefox has just released it's newest beta. In this beta is their new javascript engine called TraceMonkey which is, however, disabled by default. It's easy to enable though by toggling a simple option in about:config. Look for "javascript.options.jit.content" and set it to true.

I've been playing around with the newest beta and I must say the new javascript engine is incredibly fast. I haven't used chrome, but wow I am very impressed. When using gmail my messages seemingly instantly load from the inbox. There's no 1 or 2 seconds to wait for the javascript to load as with the old engine.

To further test this, I decided I'd run SunSpider JavaScript Benchmark in both the 3.1 Beta with TraceMonkey enabled and 3.0.3 to see if there's a real improvement here. After running the tests I was even more convinced that TraceMonkey is going to definitely make webapps much more popular. In short, total time to complete test for 3.1 Beta using TraceMonkey was 2499.8ms while 3.0.3 finished at 4297.0ms. That's an astounishing 1.7 second (1797.2ms) difference. With almost 2 seconds of difference you can see how big of a difference that would make on a site relying heavily on JavaScript such as Gmail. These tests show TraceMonkey as being 1.72x faster at rendering javascript than the old engine. Mozilla has said TraceMonkey has plenty of room for improvement so things will only get better. Currently it is said TraceMonkey is faster than Google's Chrome but I cannot verify this for myself with no Linux version of the browser out yet.

Some Tips

Some quick useful tips to use the latest beta without losing all your addons. Download Firefox 3.1 Beta and just save it in a seperate folder so you can easily switch from 3.1 Beta to 3.0.* with no problems (NOTE: This probably will not work with Windows or Mac OSX versions since it will just over write the existing installation).

Enable all your addons: visit "about:config" right click and choose New -> Boolean and enter "extensions.checkCompatibility" and set it to false. This doesn't check your extensions to see if they are compatible and should work fine in most cases.

I already covered this above but just for simple reference, enable TraceMonkey by setting "javascript.options.jit.content" to true.

Check out the new Ctrl+Tab feature. Open 3 or more tabs and switch between with ctrl+tab and you'll see a new 3D visualization like you see in most operating systems with alt+tab.

Allan Feid - TraceMonkey

JavaScript Engine Benchmark Test Results

Tags:

Firefox 3.1 Beta 1 First Impressions

Tags: