Zach's Blog

That approach only works, of course, if you have thousands of users testing daily builds. It is amusing, though, how many people inside traditional software companies assume that moz-style QA just 'must' be a failure, even when it does have thousands of users, when their own processes miss so much.

The approach doesn't really work so well even with thousands of users... We've had regressions that took years to spot, simply because people either didn't run into them, or didn't notice when they did.

I'd take 20 full-time testers who know what they're doing over thousands of users.

I think well-organized community testing knows its blind spots and plans accordingly, just as any professional organization does. If moz has had regressions go on that long then maybe someone needs to take a look at why that is and how the community could be encouraged to cover those spots as well.

The community won't test it if it isn't part of a normal day, whatever that happens to be. People have too many other things to do in life than to 1) download a new nightly each day or build from trunk, 2) run specific tests designed to exercise new code or diagnose difficult-to-find bugs, *and* 3) give up any real semblance of normal, day-to-day browsing (because it takes a solid time commitment that's difficult for an unpaid person who doesn't actually work for the Mozilla Foundation or another interested company to make).