We created the Octane Benchmark Suite to measure how JavaScript engines perform on tasks that matter to users as well as motivate our own performance work on the V8 JavaScript engine. Today we’re releasing version 2.0 of the suite including a new focus on reducing latency, new benchmarks that target use cases like asm.js and updates to existing benchmarks.

Traditional benchmarks simply measure how quickly JavaScript can be executed. Octane 2.0 additionally measures latency, which is another important aspect of performance that is concerned with the smoothness of execution. In modern JavaScript engines like V8, latency comes from two main sources: compiling JavaScript to machine instructions so it can run faster, and garbage collecting memory that is no longer used. These tasks are computationally intensive and if they run for too long can be visible to users as small hiccups and freezes of JavaScript programs. We've added a modified version of our Mandreel and Splay benchmarks to measure how well JavaScript engines can minimize these pauses.

We’ve also added two new performance tests that target important use cases. One new test is based on the Typescript compiler from Microsoft, a heavy and complex JavaScript application. Its execution stresses raw execution speed, code parsing and the memory subsystem. We've also included the zlib benchmark, an asm.js test from the Mozilla Emscripten suite. Both new benchmarks help measure an engine’s ability to deal with heavy and complex workloads, which are increasingly common in today's web applications.

Finally, we fixed three existing benchmarks to help ensure that they measure what they were intended to:
  • Regexp: verify that regexp calculations give the correct results.
  • GameBoy Emulator: code that was supposed to run in strict mode now actually runs in strict mode.
  • CodeLoad: make sure the code loaded is different on every iteration.
Octane 2.0 represents one more step in our continuing quest to deliver the best possible performance for users. You can run Octane 2.0 in your browser or read the documentation for an in-depth look at the new benchmarks.

Posted by Hannes Payer, Software Engineer and Latency Buster