OSDPL User Testing - Round 1 Protocol

Supporting Materials:

Greeting Script

User Testing Greeting Script

Hi [user's name]. I'm [your name(s)] with the [project name]. The [project name] is [project description]. Today we are looking for ways to improve the user experience of [name of item being tested ]. This is a test of the component; we are not testing you. If you find something difficult to use, chances are that others will as well, so your feedback helps everyone. This test of the component is simply a means of evaluating the component's design and to discover any issues we need to address.

If you feel uncomfortable you can stop at any time during the study.

Please speak all your thoughts aloud as you go through the tasks. This helps us better understand why you are making certain choices. The study will take about [XX] minutes. We will answer any questions you have at the end of the study.

First we'll need you to sign this Consent Form. [If doing video or audio recording you will want to mention this ]. Information gathered today will only be used for research purposes, and will be kept secure. Published results of this study will be compiled with other participants and any specific references to participants will be done anonymously. Your privacy is our priority.

Do you have any questions?

Let's get started!

User Testing Demographic Questionnaire

Interview

Perform short interview to determine user's role in their organization and familiarity with design patterns. Answer any questions they have about design patterns at this time.

Tasks

Task 1:

Based on the initial interview about the user's experience with design patterns and designing user interfaces, ask them to either:

a) Locate a design pattern in the OSDPL which would help you solve a current or past question you've had about designing a user interface.

-OR-

b) Imagine you're designing an image management application, like iPhoto or Flickr. You and have a gallery view of images and would like to allow users to change the order of the images while remaining in the gallery. Find a pattern which will help you with this design.

Task 2:

Please write a very short design pattern for the library using your own experience as a source. It can be any sort of design pattern, from a pattern you've seen in web or application user interfaces to a pattern for designing something the physical world, such as a door knob. Though we would like you to put some thought into the pattern, the content definitely doesn't need to be perfect. Unless you'd like us to keep it in the library, we will delete it after we complete this usability testing session. (If the user seems really unsure of what to do, let them know they are welcome to look through the patterns or other resources in the library for inspiration and guidance. But do not do this immediately as we'd like to see if folks do this themselves first.)

Task 3:

Make a small change to your pattern.

Task 4:

We are testing two different versions of the pattern data entry form, and will now change it to the other version. Change form. Would you please follow approximately the same process you did when you entered the pattern in the first form, in this new version of the form?

[Talk with user at this point about what version of the form they prefer.]

Task 5:

Please go back to the front page of the library. (Let user return to front page, then continue.) Now, imagine you are coming back to the OSDPL a week later with some corrections or additions you'd like to make to the pattern after thinking about the feedback you received on it from other members of the OSDPL community. Make those corrections or additions to the pattern.

Task 6:

Find the version of the pattern you just wrote in the library and make sure it looks right.

Notes for Test Coordinator

Offering help during the test

Don't offer help; let the user attempt to perform the task themselves. If they ask for help reply with:

"What do you think you/that would do?"
"What do you think that means?"

You want to observe whether users are able to:

navigate within the library
find patterns
find other needed information
create a pattern
successfully navigate between the tabs of the pattern data entry form
correct any errors that are reported
upload images

OSDPL Post-test Questionnaire

Have the user fill this out on their own before asking them the post-test questions below.

Post-test questions (ask these verbally after the user has filled out the questionnaire)

1. Why did/didn't you explore the various links in the navigation of the OSDPL? Could you tell me what you think the links in the left navigation mean? How could we make these pages or the link names of this navigation more helpful to you? [Bring out cards for possible card sort if user is interested.]

2. What tasks would you expect to perform on the OSDPL? How helpful was the general organization and navigation of the pattern library in supporting you in performing these tasks? What could we improve to make it easier?

3. Do the design patterns on the site have the type of information you need? What information could we add, subtract, reorganize, etc. to make it easier for you to use the patterns?

4. Did you have all the information you needed at the right time to perform your tasks? What could we change to make it easier?

5. Could you tell us which version of the pattern data entry form you preferred and why?

6. What was your overall impression of the pattern creation and editing process? How can we improve it?