January 26, 2008

How you can help prevent [insert QA team member name here] from losing his/her mind, while all the while making Firefox 3 better

There's a voluminous amount of QA work remaining for Firefox 3--both for its beta(s?) and its 3.0 release; even though I'm immersed in it daily, I still find it hard to not only prioritize, but even to capture what work remains. I'm going to make a feeble attempt to encapsulate said work, but I'm quite positive I'll fail in some way :-P Still, here goes...

How can you help?

1. Write or clean up Litmus testcases. Although the current setup of Litmus requires Admin status to edit (add/delete) testcases, you can still write them if they're not clearly stated enough in the bug. In tandem with an "in-litmus?" flag, it can be very helpful to ensure test coverage.


List of requested testcases for Firefox 3.

In the query above, I've excised "Open" bugs simply because that *should* mean the feature/bug fix isn't yet implemented; that doesn't mean, of course, that test coverage around it isn't needed; it's just a prioritization. (Semi-colons ftw!)

Clean up:

Most-unclear testcases

Recently unclear testcases

Ready to start? Create an account!

2. Run Litmus testcases. It should be self-explanatory, but we're looking for users running nightly builds of Firefox 3 to help us track its level of quality and feature implementation. By running *any* accurate test and submitting your valuable feedback, you help us get a clearer picture of when and if something has regressed, or, in general, how stable a given feature set is (based on its tests, though, so coverage varies by each area's level of test-completeness.)

Recent test failures--there are more such links on the footer of Litmus.

3. Verify Resolved FIXED Firefox 3 bugs.

4. Flag bugs as needing Litmus testcases. In each Firefox 3 bug, there should be a flag called "in-litmus" with four states: 1) empty, 2) ?, 3) -, and 4) +.

If you think a fixed bug (including those in the "VERIFIED" state) needs a manual testcase, select "?" from the "in-litmus" dropdown. Feel free to sprinkle liberally, but please exercise common sense; in addition to bugmail spam (which is all fine and good), for those of us whom run queries and write testcases, the time spent determining whether a testcase is indeed needed should be a no-brainer.

That said, the current list we have is by no means comprehensive, and, although it takes time to parse through the current list of testcases covered and match them with the request, it's well worth it to have more comprehensive coverage.

5. Stop by #qa on irc://irc.mozilla.org if you have further questions; we'd love to have folks helping out with any of these areas!


Posted by stephend at January 26, 2008 4:07 AM
Post a comment