Documentation for a historical release of Infusion: 1.3
Please view the Infusion Documentation site for the latest documentation.
If you're looking for Fluid Project coordination, design, communication, etc, try the Fluid Project Wiki.

Layout Reorderer User Testing - Round 3 Protocol

Owned by Daphne Ogle
Last updated: Mar 19, 2009 by Allison BloodworthVersion comment
4 min read

Round 3 will be a retest of Round 2 with performance enhancements

Round 2 user testing proved difficult because of some performance issues with the component. Many performance enhancements have been made to the Reorderer and we would like to retest with the same protocol we used in Round 2.

Reference:  User Testing Protocol

Supporting Materials:

Greeting script

Error rendering macro 'excerpt-include' : No link could be created for 'User Testing Greeting Script'.

User Testing Demographic Questionnaire

Scenarios

Student

Your university has a portal that you frequent regularly for your classes and submitting assignments, as well as being informed of campus news, announcements, and events. There is an Announcements box on the Welcome tab (page) of the portal that has information you need to see for things like registration schedules, class cancellations, and assignment submission deadlines. On your computer, however, the announcements box is always at the bottom of the page and you have to scroll down to see it. Last week you missed an important announcement because you forgot to scroll down and check the announcements.

Tasks

  1. Change the placement of the announcements box on the Welcome tab.
  2. Change the placement of other boxes on the Welcome tab.
  3. (Note to Test Coordinator: Only ask if user has not yet encountered a locked box.) On the Welcome tab, move (or say drag, if they have not yet used drag-and-drop) the announcements box as far to the upper left as you can.
  4. (Note to Test Coordinator: Only ask if user has not dragged & dropped a box. Explain how to drag-and-drop if necessary.) Drag the announcements box to a new position on the Welcome tab.

Tasks (Blind user)

  1. Change the placement of the announcements box on the Welcome page so that it is third from the top.
  2. Change the placement of the xxxx box so that it follows the announcements box.
  3. Move the announcements box so that it is the first box on the page. (Note to Facilitator: Participant will find that the first box in the portal is locked and cannot be moved. Observe whether they understand: 1) that the box is locked, and, if so, 2) how to unlock the box and complete the task. If they do not understand why they cannot move the announcements box to the topmost position, explain to them that it is locked. Then ask them to attempt to unlock the box, and move the announcements box in its place.
  4. I'm not sure we need this additional task.

Instructor

Your university has a portal that you frequent regularly for managing the classes you teach and distributing assignments, as well as being informed of campus news, announcements, and events. There is an announcements box on the Home page of the portal that has information you need to see for things like faculty and staff meetings. On your computer, however, the announcements box is always at the bottom of the page and you have to scroll down to see it. Last week you missed an important announcement because you forgot to scroll down and check the announcements.

Tasks

  1. Change the placement of the announcements box on the Welcome tab.
  2. Change the placement of other boxes on the Welcome tab.
  3. (Note to Test Coordinator: Only ask if user has not yet encountered a locked box.) On the Welcome tab, move (or say drag, if they have not yet used drag-and-drop) the announcements as far to the upper left as you can.
  4. (Note to Test Coordinator: Only ask if user has not dragged & dropped a box. Explain how to drag-and-drop if necessary.) Drag the announcements box to a new position on the Welcome tab.

Tasks (Blind user)

  1. See above. 

Notes for Test Coordinator

Offering help during the test

Don't offer help; let the user attempt to perform the task themselves. If they ask for help reply with:

  • "What do you think you/that would do?"
  • "What do you think that means?"

You want to observe whether the user has trouble:

  • recognizing they can move a portlet on the portal home page
  • moving a portlet to a specific location
  • differentiating between locked and unlocked portlets

Post-test Questionnaire

Note: Have users fill out this questionnaire on paper.  

How easy or difficult was it for you to realize you could move boxes in the portal (by dragging a box) from one location to another (dropping it in a new location)?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

How easy or difficult was it for you to actually move a box in the portal to a specific location?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

Post-test questions (ask these verbally after the user has filled out the questionnaire)

1. Did you notice a visual cue for picking up one of these boxes? How helpful was it to you? (looking for cursor change to hand)

2. Did you notice a visual cue for dropping (and placing) a box? How helpful was it to you? (looking for green box drop target)

3. Further questions about drop target, which can be skipped if answered in question #2:

  • How did you know where the box you were moving would land?
  • Did you notice a green outline box? What do you think it is for?
  • Was there anything confusing about this box? If so, what?
  • (If confusing) Are there any changes you would suggest to make it less confusing?

4. Did you see a visual cue that helped you determine whether a box could be picked up or not? How helpful was it to you? (looking for lock icon to show the portlet is locked.)

Blind User

  1. Did JAWS do a good job indicating to you the names of the boxes?
  2. Did you understand how to move a box from one location to another?
  3. Did you understand where to place the box?
  4. Did you understand if a box was locked, what that meant, and how to unlock it?
  5. Do you have any suggestions for making any of the actions easier to do?