Layout Reorderer User Testing - Round 1 Results

Summary

People are having trouble realizing they can move portlets at all.
Locked Portlets:
- How should we convey to user that a portlet is locked? Cursor change is not enough.
- Where should a "lock" icon be placed? Upper left corner of titlebar is not a good location.
- Tooltip text is helpful but first must see the icon. Could move the icon to the middle of title bar or to the right corner.
- Tool tip/hover text saying "section cannot be moved" could be if they hover anywhere on title bar.
- Could remove title bar from locked portlets as Shaw-han has suggested.
Drop Target Indicators:
- Green bar is too small and not being noticed enough.
- Maybe make it thicker and with an arrow indicating where portlet will go.

Demographics

User Number	Gender	Age Range	Student Classification	Major/Area of Study	Web Savvy	Type of Computer

User Number	Gender	Age Range	Student Classification	Major/Area of Study	Web Savvy	Type of Computer
User 1	Female	41-50 years	Other	Science	medium	PC (toshiba)
User 2	Female	25-30 years	Part-time	Science (psychology)	low to medium	PC
User 3	Female	25-30 years	Full-time	Education	medium to high	PC laptop
User 4	Female	25-30 years	Full-time	Science	low to medium	PC (HP laptop)

Interaction Notes

User Number	Previous LMS Experience	Moving Portlets	Drag and Drop	Locked Portlets

User Number	Previous LMS Experience	Moving Portlets	Drag and Drop	Locked Portlets
User 1	None	• At first, the user didn't understand the concept of moving portlets; wasn't sure how to reorganize the content. • The user tried clicking the links to find a way to reorganize the content. • Wanted to move portlets into the left hand navigation bar.	• Didn't fully understand the drop target. • Felt that placing the portlet over another portlet's dark blue header, would allow the portlet to be dropped. • While in the discussion section of the test, the user noticed the green drop target. • Wanted to be able to drop a portlet in open space (i.e. somewhere it may appear after dropping), currently can only drop by hovering over another portlet. • The user also had trouble droping items, as the page didn't scroll while dragging an item out of the visible portion of the viewport (up/down).	• The user quickly realized that the locked portlet wasn't moveable • The user saw that the pointer didn't change, and that when she tried to drag it, the text was selected. • She didn't notice the lock icon.
User 2	Has used Blackboard	• The concept of moving portlets is a bit confusing (the user wasn't sure what to move). • If not asked to do so, the user would not have thought to move the portlets. • Would have maximized/minimized portlets. • When asked to move the portlets, dragging was the user's first instinct. • As a personal approach to customization, the user would like to move commonly used portlets to the top. (Liked the Google search portlet).	• Was able to easily drag portlets • Seemed to recognize drop spots • Understood the cursor change	• Figured the locked portlet didn't move, but not sure why. • Saw the lock, but said it wasn't obvious • Felt the lock could relate to the content, as opposed to the lack of movement of the portlet. • The user suggested changing the frame or the colour of the locked portlet to make it stand out more.
User 3	Blackboard	immediately recognized the drag affordance (window style boxes) and started trying to drag the title bars expected to be able to minimize and maximize	found the resizing and shape changing of portlets as she was dragging disorienting drop target affordance was not noticible enough. "i didn't know where things would go if i dropped". the drop target affordance can be covered up by the drag avatar making it hard to see. suggested a larger bar in a more prominent color	did not see the locked icon (this was the new placement, on the left side of the text) did not hover and see the tooltip it "looked like" it could be moved but after trying several times she realized that it was not possible. suggested that the position of the icon was too subtle.
User 4	Blackboard	Concept of moving portlets was confusing. Initially wanted to maximize or minimize portlets. Moved mouse around and noticed cursor change but didn't recognize that she could move the portlet Thought we could have buttons along the top of the portal for each portlet to click to maximize it Suggested hover text appearing to say a portlet can be moved.	Dragged a portlet straight across horizontally and did not even see the green bar. Often dragged outside the portal area and thought something would happen. Expected the portlets to shift as she moved a portlet. When she finally noticed drop target affordance she felt it was too small and should be a different colour. Suggested it should flash/blink.	did not see the locked icon (placement on left side of text) did not hover and see the tooltip suggested we have a tooltip over the title bar for locked portlet that it cannot be moved she did try and move the locked portlet suggesting the cursor not changing did not help her to realize she could not move the portlet she tried to drag other portlets above or straight into the locked portlet and was frustrated when the portlet was not moved.

Post-test Questionnaire Responses

Question	User 1	User 2	User 3	User 4

Question	User 1	User 2	User 3	User 4
How easy or difficult was it for you to realize you could move sections of the portal?	Easy	Easy	Easy	Difficult
How easy or difficult was it for you to move a section of the portal to a specific location?	Difficult	Easy	Difficult	Difficult
How easy or difficult was it for you to distinguish between sections of the portal that could be moved and those that were locked in place?	Easy	Difficult	Neutral	Difficult