User Testing Tips

Owned by Allison Bloodworth
Last updated: Oct 25, 2010Version comment
5 min read

Planning a Test

Start testing early. User testing is most useful if done early and often in the design process. Different techniques can be used at different stages of design. In the early stages when ideas and concepts are drawn on whiteboards or paper, teams can use the paper prototyping method. This involves everything from putting hand drawings and post-it notes to wireframes created in a graphics program in front of users. These paper prototypes are and then manipulating them as if they are on the computer as the test participant indicates where he/she would click or type to accomplish specific tasks. Later on in the design cycle continue to test on progressively interactive prototypes or the production system.

Bring developers into the loop early. Let developers know your design and user testing schedule beforehand in order to make them aware of when another design iteration will take place, give them the opportunity to participate in the testing sessions if they are able, and ensure that your prototype or next version of the product or service with all the features you intend to test will be ready.

Three to five users are enough. Unless you have multiple distinct groups of users of a system with very different needs or behaviors (e.g., a system where staff are interacting with an administrative interface while students are interacting with the user interface), getting just three to five users to participate in a usability test will generate very useful data. In the case of multiple groups of users, it's helpful to have three to five users from each distinct group. If your resources allow and by the fifth test you're still learning lots of new things from each user, you may want to continue testing additional users until you realize you're no longer discovering much new information. However, if you have the time and resources to run more than five tests, it's often more helpful to test successive iterations of the product with small numbers of users than it is to test one iteration with a large number.

Keep it short. Participating in a usability test is hard work that takes a lot of concentration (for both the participant and the facilitators). Therefore, limiting a test to no more than an hour (30 minutes is better) usually makes sense.

Share the load. It's much easier if you can split the burden of running a test between two people, one responsible for taking notes, and one responsible for facilitating the test. Trying to both facilitate and take notes can be difficult or stressful. However, if your resources are limited, it is possible to run user tests with just one person being both note taker and facilitator, especially if the parts of the prototype you are testing aren't very complicated.

Involve others. Observing user tests can be a very compelling way for members of your team to get a real understanding of what users are going through. If other people on your team are interested in observing a user test, you may want to arrange for them to view a recording of a session--or perhaps even watch it live from another room as the test takes place. If you must have an observer in the room, it is best to limit it to one other person as test participants are likely to become uncomfortable if there are too many people there. Alternatively, if your team members have the time to learn to facilitate a test you can consider having them actually participate in the session, for example as a note-taker.

Facilitating a Test

Decide whether you'd like to ask the participant to think aloud. Asking a user to think out loud as he or she proceeds through the test can help reveal confusing portions of the interface as well as give you a sense of what features, wording, or help the user is looking for when they do get stuck. This is especially helpful as you are doing formative testing – meaning making initial design decisions. If you do decide to ask the user to think aloud, it's OK to remind the user to continue to do so if they become quiet, but try not to interrupt their train of thought — wait for a natural pause.

When doing summative testing, which means testing the efficiency and efficacy of an interface using measurements such as a users' time on task, it may be best not to ask the user to think aloud as they perform the task. In this case, it may be more appropriate to discuss tasks with the participant after they have completed them.

Help the participant to speak freely. When performing a user test, it it important to guard against the participants' natural tendency to try to please you (as well as to not offend you). Because participants will often try to give you the answer you want to hear, you shouldn't give away what you personally think about it. You want the participant to be comfortable enough to be honest with you during the test and give you as much information as they can about their experience--without being influenced by your feelings or agenda. Following the facilitation rules below will help in this area:

  • Distance yourself from the product.
  • Don't react.
  • Don't interrupt.
  • Don't draw attention to specific issues that you care about.
  • Don't ask leading questions.
  • Don't lead with your note-taking (meaning only taking notes when the participant does something interesting).

Don't help. The idea of a usability test is to see where the user would run into difficulty with a system if they were using it on their own, and to get the user to express his or her own opinions. If you help, you're (obviously) not going to get this data. Nonverbal cues (smiles, sighs, glances, etc.) can be just as "helpful" as explicit instructions — try to remain as neutral as possible as the user performs each task.

Test the system, not the user. It's important to make clear to the user that you're testing the system, not the user, and that it's a flaw in the system if they get stuck — they're not failing. In fact, finding the places where they get stuck is exactly why you're doing this test! A corollary to this rule is to not let the user flounder. If the user has gone down a path where they're obviously stuck, let them try for a little bit to see if they get unstuck, but don't make them struggle needlessly. Just make a note of the confusing point and move on to the next task.

Capture themes, not every action. If you are acting as the note taker for a usability test, don't worry about recording every action and utterance from the user. As you do successive user tests, themes and obvious pain points will emerge very quickly. It is often helpful, however, to use selected, direct quotes from the user in order to illustrate these pain points to other members of your team.

Thank your participants. At the conclusion of the test, be sure each test participant understands the value of the contribution to the project. A small thank you gift (some chocolates or a small gift certificate) never goes amiss, and is often helpful in recruiting participants.