Layout Reorderer User Testing - Round 2 Results

Summary

1 User was never able to drag and drop a portlet
The dragging metaphor breaks down if there are performance issues (which may be an issue over slow internet connections as well). Firefox 2.0 has much longer lag time that Firefox 3.0. The lag time made it really difficult to drag and drop. 1 user was never sure it was even allowed because the portlet kept jumping back to place when she didn't drag it far enough.
Ghosting to show dragging was a well understood cue, but this metaphor broke down in a big way when performance was slow (jerky movement didn't translate).
Green drop target box confused users. 2 users weren't sure what it was supposed to indicate (firefox 2.0), but thought it had to do with where the portlet was picked up from (this may have been due to the slow movement when dragging and drop target not showing up quickly).
Long scrolling while dragging was a problem. 1 user had to get out of her seat in order to keep scrolling to try to drag a portlet from the bottom to the top.
Most users had trouble distinguishing locked portlets from movable portlets

1 user said she really likes the idea of personalizing her spaces.
1 user thought the icon in front of portlet title was the indicator that you could drag
1 user noticed the 'you can't drag here' message but was confused about when and why she couldn't. Because of earlier problems she took it as another indication that you couldn't really drag and drop.

Demographics

User Number	Location	Gender	Age	Role	Tech-savvy	Do you own a personal computer?
User 1 (RL)	Berkeley	Female	51-60	Staff	N/A	Mac
User 2 (JH)	Berkeley	Male	35-40	Staff	high	Mac laptop
User 3	Toronto	Female	35-40	Student	Low	HP Desktop
User 4	Toronto	Male	25-30	Grad Student	Low	Dell Laptop
User 5	Toronto	Male	25-30	Student	High	mac laptop
User 6	Toronto	Male	19-24	Student	High	IBM Laptop
User 7 (MS)	Berkeley	Male	35-40	Staff	medium	Customized PC

Do you do any of the following and if so how often?

User	Checking email	Instant message	Shop online	online banking	Internet research	Take class online	Social networking
User 1 (RL)	all the time	hardly ever	a few times a month	all the time	all the time	hardly ever	hardly ever
User 2 (JH)	all the time	all the time	a few times a week	a few times a week	all the time	hardly ever	all the time
User 3	a few times a week	a few times a week	never	never	hardly ever	never	a few times a month
User 4	all the time	hardly ever	never	a few times a week	all the time	never	never
User 5	all the time	hardly ever	a few times a month	a few times a month	all the time	hardly ever	hardly ever
User 6	all the time	all the time	hardly ever	a few times a week	a few times a week	a few times a month	a few times a week
User 7 (MS)	all the time	hardly ever	a few times a month	never	N/A (Never)	a few times a month	a few times a week

Do you upload files on the web and if so how often?

User	Pictures	Media	Documents to course site	Documents to social networking site	Attach a file in email
User 1 (RL)	Hardly ever	A few times a week	A few times a month	Hardly ever	All the time
User 2 (JH)	a few times a month	a few times a month	all the time	a few times a week	all the time
User 3	hardly ever	hardly ever	hardly ever	hardly ever	hardly ever
User 4	hardly ever	never	all the time	a few times a week	all the time
User 5	a few times a month	a few times a week	a few times a week	hardly ever	all the time
User 6	hardly ever	a few times a week	a few times a week	a few times a month	a few times a week
User 7 (MS)	a few times a month	a few times a month	N/A (never)	hardly ever	all the time

Interaction Notes

User Number	Previous LMS Experience	Task 1 - Move a particular box/portlet	Task 2 - Change the placement of other boxes/portlets	Task 3 - Deal with a Locked box	Task 4 (if not done already) - drag and drop a portlet	General comments
User 1		Took a minute to find announcements What I would do is click & drag but I don't think you can She accidentally moved announcements to the left so when she went to move to the top there was a lot of whitespace before the next portlet. She tried to drag to the right but the portlet kept popping back to the left. This threw the rest of the test off. "I would have taken it to top but it just doesn't go there"	User was never able to drag and drop. She was successful with moving announcements to the top by "canceling" (removing) portlets. I've done this with yahoo. I'll just start canceling the ones I don't want until I get them up there Wasn't seeing the green box move to the drop target - slowness, couldn't drop in right bottom of column & lots of scrolling then can't lose sight of green box.	She noticed the you can't drag her message but never noticed the lock icon. She wasn't sure why she couldn't drop there and took it as an indication that she couldn't drop it at all.	Finally told her she could drag and drop and asked her to try again. This is when she was scrolling forever and had to get out of her seat to scroll far enough.	"I would have taken it to top but it just doesn't go there" Lag time was also throwing off the test. It's just pretty clunky moving things around Long scrolling was a problem trying to drag things from the bottom all the way to the top - she could barely reach her arms up long enough. "The way I do things is click around "I'm clicker". She blamed the misbehavior on the mouse but it was the slow, jerkiness of the system. While filling out the questionnaire she said, I think the mouse wasn't work well or it would have been easy as she started to mark easy. I encouraged her to take the mouse factor as is and assume it was working the way it should - that it was not the problem. She still answered "easy".
User 2 (JH)	Moodle, bSpace	first clicked Change Layout but quickly realized that this was for column layout moused around screen and noticed that the pointer changed to a hand over the pane header for each portlet, then easily dragged the Announcements portlet to the correct position.	easily started to make changes	noticed that the Welcome portlet was locked by the locked icon on the portlet also noticed the rollover text	n/a	expects things like this to configurable from his experience with social networking sites and Moodle. Very comfortable with drag and drop the change in cursor was the key indicator for this user
User 3		When asked how do you move a box? Answer: no idea. "you mean click on something and move it?" clicked the X to move the box clicked the window icon to drag eventually saw a green box and figured how to drag a portlet	moved portal vertically on right column, drop marker showed on left column but bounced back tried to drag portal down to empty vertical space below the portlet. Surprised when it doesn't stick. Why are you moving it there? "Because it's empty space"	did not notice the help text help text sometimes flashes quickly tried multiple times to drag portlet, not sure why it wasn't moving	n/a	user was dragged the icons thinking that was how to accomplish the task. warning text when dragging over locked portlet not obvious and missed by user.
User Number	Previous LMS Experience	Task 1 - Move a particular box/portlet	Task 2 - Change the placement of other boxes/portlets	Task 3 - Deal with a Locked box	Task 4 (if not done already) - drag and drop a portlet	General comments
User 4		instead of repositioning, he says he would scroll down to view instead of moving it. not sure how to reposition a box. mouse exploring, but not noticing the change in mouse cursor clicked on the announcement box and it took him to the announcement page. multiple times could not notice the change in mouse icon to a "grab" hand