User Testing Protocol

Owned by Barbara Glover
Last updated: Mar 16, 2009 by Allison BloodworthVersion comment
3 min read

User testing tests the usability of a component by putting it in front of several users individually to see if they can use the component successfully. A user testing protocol describes the objective(s), methodology, and organization of a user test - essentially how to administer a user test.

These guidelines provide a template for a user testing protocol, which may then be tailored to specific protocol for user testing of components. Scenarios, tasks, post-questionnaires, and post-test questions must be defined on a per-test-set basis.

Greet user and explain user testing procedure

The over-arching goal of the user greeting is to make the person feel comfortable. Making the person comfortable produces more accurate test results. In order to make the person comfortable:

  • Be at ease yourself (practice beforehand).
  • Introduce yourself and any other observers or note takers.
  • Establish an understanding of the test purpose and goal.
  • Explain user testing procedure and that you are testing the system and not them.
  • Set expectations for the test length and scope.
  • Provide an escape route (that they can stop at any time).
  • Ask them if they have any questions.
  • Convey a positive and enthusiastic attitude.

Greeting script

Sample greeting script for use in user testing.

Hi [user's name]. I'm [your name(s)] with the [project name]. The [project name] is [project description]. Today we are looking for ways to improve the user experience of [name of item being tested ]. This is a test of the component; we are not testing you. If you find something difficult to use, chances are that others will as well, so your feedback helps everyone. This test of the component is simply a means of evaluating the component's design and to discover any issues we need to address.

If you feel uncomfortable you can stop at any time during the study.

Please speak all your thoughts aloud as you go through the tasks. This helps us better understand why you are making certain choices. The study will take about [XX] minutes. We will answer any questions you have at the end of the study.

First we'll need you to sign this Consent Form. [If doing video or audio recording you will want to mention this ]. Information gathered today will only be used for research purposes, and will be kept secure. Published results of this study will be compiled with other participants and any specific references to participants will be done anonymously. Your privacy is our priority.

Do you have any questions?

Let's get started!

Example: Fluid Project

Hi [user's name]. I'm Bill with the Fluid Project. The Fluid Project is an open, collaborative project to improve the user experience of community source software. Today we are looking for ways to improve the user experience of the Video Player component. This is a test of the component; we are not testing you. If you find something difficult to use, chances are that others will as well, so your feedback helps everyone. This test of the component is simply a means of evaluating the component's design and to discover any issues we need to address.

If you feel uncomfortable you can stop at any time during the study.

Please speak all your thoughts aloud as you go through the tasks. This helps us better understand why you are making certain choices. The study will take about 30 minutes. We will answer any questions you have at the end of the study.

First we'll need you to sign this Consent Form. With your consent, we would also like to do a video recording of your facial expressions to further assist our analysis. Information gathered today will only be used for research purposes, and will be kept secure. Published results of this study will be compiled with other participants and any specific references to participants will be done anonymously. Your privacy is our priority.

Do you have any questions?

Let's get started!

Have user sign consent form

A consent form is usually necessary to ensure that the user affirms (in writing) willingness to participate in the study. Have the user sign the consent form, and give address if an honorarium has to be mailed.

Ask if user has any questions

Before starting the study, pause and let the user ask questions. Respond as needed to ensure that the user is at ease and ready to participate.

Have user complete the demographic questionnaire

Have the user fill out the demographic questionnaire. If you would like to build rapport with the user, you can also ask the questions verbally, perhaps beginning with an ice breaker question about some of the demographic information you want to collect anyway, to help put the user at ease.

Conduct the test

  • Have the user sit down in front of the computer (or paper prototype).
  • Remind the user that you would like them to 'think aloud' so that you can understand what they are thinking about their experience.
  • Read the user the scenario for context.
  • Read the user the task to complete. If the tasks are complex, you may want to put a sheet of paper in front of the user with each task written out (as your goal is not to test the user's ability to remember the task).
  • Record the user's actions, comments, questions, and body language.

Offering help during the test

Try not to offer help too soon. Let the user make multiple attempts at completing the task.

If they ask for help reply with:

  • "What do you think you should do?"
  • "What do you think that would do?"
  • "What do you think that means?"

Record responses, and note if you needed to provide help in order for the user to complete the task.

Have user complete a Post-test Questionnaire (optional)

In order to gather more information after the test is complete, have the user complete a short questionnaire. This is an optional step. Post-test questionnaires often use questions with Likert scale responses.

Post-test Questionnaire example question

How easy or difficult was it for you to...?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

Post-test questions (optional)

In order to gather more information after the test is complete, have the user answer reflective questions. This is an optional step. Be sure this step is after they've completed the post-test questionnaires to not bias their answers.

Post-test question examples

  1. How easy or difficult was it for you to...?
  2. Did you notice a visual cue for...?
    • What do you think it is for?
    • How helpful was it to you?
  3. How did you know to...?
  4. Was there anything confusing about...?
    • If so, what?
    • (If confusing) Are there any changes you would suggest to make it less confusing?

Conclude the test

  1. Let the user ask any questions and debrief.
  2. Let the user know how they can find out about changes made to the component as a result of the user testing data.
  3. Thank the user for participating.
  4. Give the user an honorarium if you have one.
    • If the honorarium will be mailed let the user know when to expect to receive it.
    • Give the user a contact number in the case the honorarium doesn't arrive.
On this Page