Sputnik: ECMAScript 3 conformance test suite

Today, The Chromium Blog has officially released their ECMAScript 3 conformance test suite in a form that is more friendly to test in your browser. The test contains over 5,000 tests (currently 5,246) and continues to grow!

The Chromium Blog has also posted some initial results among the top web browsers for Windows (emphasis is mine).

In this example, when running Sputnik on a Windows machine, we saw the following results: Opera 10.50: 78 failures, Safari 4: 159 failures, Chrome 4: 218 failures, Firefox 3.6: 259 failures and Internet Explorer 8: 463 failures.

An experimental plot to illustrate how the latest stable browsers compare.

Putting that into terms of 100% conformance rates: Opera 10.50: 98.5% successful, Safari 4: 97.0% successful, Chrome 4: 95.9% successful, Firefox 3.6: 95.1% successful and Internet Explorer 8: 91.2% successful.

Running the test myself in the latest Opera 10.50 snapshot for Linux (Build 6242) I’m seeing a solid 77, proof that Opera 10.50 is progressing still!

As explained in their post, the goal of this test is not related to Javascript performance in terms of speed, but in terms of conformance to the spec. Ideally all browsers would be in the center of the bullseye, meaning they all conform and behave (nearly) identically.

The Sputnik tests have been released as an open source project, so if you’re interested in providing conformance test cases to improve the future web..now is a perfect chance to get involved. 😉

To run the test yourself or learn more about it, visit: http://sputnik.googlelabs.com/

Leave a Reply