OSDPL User Testing - Round 1 Protocol
Reference: User Testing Protocol
Supporting Materials:
Greeting Script
User Testing Demographic Questionnaire
Interview
Perform short interview to determine user's role in their organization and familiarity with design patterns. Answer any questions they have about design patterns at this time.
Tasks
Task 1:
Based on the initial interview about the user's experience with design patterns and designing user interfaces, ask them to either:
a) Locate a design pattern in the OSDPL which would help you solve a current or past question you've had about designing a user interface.
-OR-
b) Imagine you're designing an image management application, like iPhoto or Flickr. You and have a gallery view of images and would like to allow users to change the order of the images while remaining in the gallery. Find a pattern which will help you with this design.
Task 2:
Please write a very short design pattern for the library using your own experience as a source. It can be any sort of design pattern, from a pattern you've seen in web or application user interfaces to a pattern for designing something the physical world, such as a door knob. Though we would like you to put some thought into the pattern, the content definitely doesn't need to be perfect. Unless you'd like us to keep it in the library, we will delete it after we complete this usability testing session. (If the user seems really unsure of what to do, let them know they are welcome to look through the patterns or other resources in the library for inspiration and guidance. But do not do this immediately as we'd like to see if folks do this themselves first.)
Task 3:
Make a small change to your pattern.
Task 4:
We are testing two different versions of the pattern data entry form, and will now change it to the other version. Change form. Would you please follow approximately the same process you did when you entered the pattern in the first form, in this new version of the form?
[Talk with user at this point about what version of the form they prefer.]
Task 5:
Please go back to the front page of the library. (Let user return to front page, then continue.) Now, imagine you are coming back to the OSDPL a week later with some corrections or additions you'd like to make to the pattern after thinking about the feedback you received on it from other members of the OSDPL community. Make those corrections or additions to the pattern.
Task 6:
Find the version of the pattern you just wrote in the library and make sure it looks right.
Notes for Test Coordinator
Offering help during the test
Don't offer help; let the user attempt to perform the task themselves. If they ask for help reply with:
"What do you think you/that would do?"
"What do you think that means?"
You want to observe whether users are able to:
navigate within the library
find patterns
find other needed information
create a pattern
successfully navigate between the tabs of the pattern data entry form
correct any errors that are reported
upload images
OSDPL Post-test Questionnaire
Have the user fill this out on their own before asking them the post-test questions below.
Post-test questions (ask these verbally after the user has filled out the questionnaire)
1. Why did/didn't you explore the various links in the navigation of the OSDPL? Could you tell me what you think the links in the left navigation mean? How could we make these pages or the link names of this navigation more helpful to you? [Bring out cards for possible card sort if user is interested.]
2. What tasks would you expect to perform on the OSDPL? How helpful was the general organization and navigation of the pattern library in supporting you in performing these tasks? What could we improve to make it easier?
3. Do the design patterns on the site have the type of information you need? What information could we add, subtract, reorganize, etc. to make it easier for you to use the patterns?
4. Did you have all the information you needed at the right time to perform your tasks? What could we change to make it easier?
5. Could you tell us which version of the pattern data entry form you preferred and why?
6. What was your overall impression of the pattern creation and editing process? How can we improve it?