July 6, 2011

Telemetry in Firefox

One of the big challenges in developing a great Web browser is figuring out where the browser's performance gets in the way of people using the browser. As Nicholas Nethercote notes, you lose more when you're slow than you gain when fast. That means that a top priority for Mozilla is to not just work on increasing overall performance, but to really hone in on the areas where we're slow and make sure we speed them up.

Unfortunately, the traditional tools for measuring performance, various industry standard benchmarks like SunSpider or our own "ts" start-up time tests, don't really do a very good job of measuring when we're slow. We can make lots of improvements to our already quite fast JavaScript engine and we can tune for our startup tests but those don't necessarily translate into real-world and user-perceived performance gains because they're not necessarily getting at those places where performance is bad enough to interrupt the user's flow.

What we really need to measure in order to tackle our slowest code is real-world usage and that requires an entirely different way of thinking about performance and a new set of measurement technologies from what we've grown accustomed to over the last decade or so. It's no longer good enough to optimize for the various performance tests and benchmarks available to us; we must optimize for the performance issues our users are experiencing in the real world. This is where browser telemetry comes in.

Telemetry in Firefox is a new technology that allows our developers to build performance tests right into Firefox itself and to have those tests running while users are interacting with Firefox and the Web. These little tests, also called probes, should go completely unnoticed when you're using Firefox but they make note of the time it takes to accomplish certain tasks like establishing a network connection or freeing system memory. If people opt in to sending this anonymous data to Mozilla, like tens of millions do with crash reports already, then our developers can isolate places where Firefox is slow or uses too much memory and can start to remedy them.

Telemetry makes its debut in the Aurora channel this week and we're encouraging all of our Aurora people to send the anonymous performance and resources data to Mozilla so that we can work on the problems out on the real Web rather than just the things that synthetic tests can identify.

You can read more about the development of Mozilla's telemetry infrastructure in this series of blog posts by telemetry, startup performance, and static analysis tools engineer, Taras Glek:

Firefox Telemetry
Telemetry Updates
Developers: How To Submit Telemetry Data
Telemetry is on in Nightly Firefox builds
Telemetry Status

This is just one of several new efforts at Mozilla to address performance and memory usage issues. Starting with the Aurora update, users will really begin experiencing the many Firefox performance improvements that result from telemetry data and other initiatives designed to make sure Firefox is the fastest, leanest, and most powerful browser in the world.

Posted by asa at 7:11 PM