August 3, 2009

How I break down a site for testing, part 1

I've often wanted to write a blog post or a wiki page explaining the process by which I look at a site (or in this case, a single-page mockup), and start to break down the functionality/elements that I'll be testing.

Let's look at the Creative Collective example, below:

First things first: the first thing I notice is that the site stores and recognizes usernames (or email addresses, depending on how it authenticates its users). I'd immediately expect that the "Join Us" link leads us to an account creation/registration page, wherein users fill in a combination of their real name, username, and/or email address. (Other fields like a short bio, etc. might exist too.)

On an account-creation page, the usual test candidates are:

* username/password length
** Does it accept more than the stated length limitation?
** How about something like 250 characters?
** How about "special characters", such as ":/?#[]@!$&'()*+,;=.<>"?
** Does it accept empty or only-space-character usernames?
* unique username (try to register two that are identical)
* empty password

Of course, those are just the individual fields; we also need to ensure that users enter all the required information upfront, and if they don't, that we don't lose what they'd typed, and even more than helpfully pointing a user to the field with the problem, we should tell them how they can correct it and move on.

And, that's just registration -- we want to be equally helpful to the user on login: which fields are required, what their minimum lengths are, etc. To test, I would:

* try valid username or email address (whichever the site uses) but invalid password
* try valid username/email address without a password
* try invalid username/email address but valid password (etc.)
* test the forgotten-password feature

Login and account-creation pages also should be served through HTTPS, so ensure that the links start and stay on https:// -- we don't want to send users' data "over the wire" in plain text -- that could potentially be read by others. Additionally, we should test that the fields properly escape data (encode it so that invalid/evil input is made safe); we should also test for XSS (cheat sheet here).

After testing this, I'd move on to search functionality. In the mockup above, we have "Search for" (images) * [search textfield]. Let's break it down: we know from "images" that one datatype will be images, but not how our search terms might match that category; because it's a selector, we can also assume there will be other datatypes -- maybe one for "text", and/or "images and text". For text, are we searching on perhaps tags, titles, or authors (each of which could be in the selector)? And whenever we think of search, there are a bunch of use-cases to consider:

* when you click on the pre-filled "search" text, does it disappear, to be replaced by your entered text?
* what should the empty (i.e. default) search experience be? (Should the user be able to search without entering something? Typically, yes.)
* what about substrings? e.g. "paint" in "painting" -- would it match "painting"?
* do misspelled words trigger a helpful "did you mean?" suggestion (a la Google)
* when you change the "Search for" pulldown, does the textfield clear?
* do your criteria get reset when reloading the page?
* try pasting something huge like the text of the U.S. Constitution -- does the server have to process it in its entirety, and take forever? Does it balk? Throw up an error page, with, perhaps, any internal server-config values/messages? Are there layout issues on the search-results page from such huge a string?
* again, also try XSS here
* try, of course, regular searches :-)

So far, I've really only touched on four components of this site--nay, this page--so you can quickly see how complex (and fun) website testing is.

I'll come back to finish this page towards the end of this week or early next week, so stay tuned!

Feedback/questions welcome at

As always, Mozilla's Web QA team can be found at

Posted by stephend at August 3, 2009 9:08 PM
Post a comment