Layout Reorderer User Testing - Round 2 Results
Summary
- 1 User was never able to drag and drop a portlet
- The dragging metaphor breaks down if there are performance issues (which may be an issue over slow internet connections as well). Firefox 2.0 has much longer lag time that Firefox 3.0. The lag time made it really difficult to drag and drop. 1 user was never sure it was even allowed because the portlet kept jumping back to place when she didn't drag it far enough.
- Ghosting to show dragging was a well understood cue, but this metaphor broke down in a big way when performance was slow (jerky movement didn't translate).
- Green drop target box confused users. 2 users weren't sure what it was supposed to indicate (firefox 2.0), but thought it had to do with where the portlet was picked up from (this may have been due to the slow movement when dragging and drop target not showing up quickly).
- Long scrolling while dragging was a problem. 1 user had to get out of her seat in order to keep scrolling to try to drag a portlet from the bottom to the top.
- Most users had trouble distinguishing locked portlets from movable portlets
- 1 user said she really likes the idea of personalizing her spaces.
- 1 user thought the icon in front of portlet title was the indicator that you could drag
- 1 user noticed the 'you can't drag here' message but was confused about when and why she couldn't. Because of earlier problems she took it as another indication that you couldn't really drag and drop.
Demographics
User Number |
Location |
Gender |
Age |
Role |
Tech-savvy |
Do you own a personal computer? |
---|---|---|---|---|---|---|
User 1 (RL) |
Berkeley |
Female |
51-60 |
Staff |
N/A |
Mac |
User 2 (JH) |
Berkeley |
Male |
35-40 |
Staff |
high |
Mac laptop |
User 3 |
Toronto |
Female |
35-40 |
Student |
Low |
HP Desktop |
User 4 |
Toronto |
Male |
25-30 |
Grad Student |
Low |
Dell Laptop |
User 5 |
Toronto |
Male |
25-30 |
Student |
High |
mac laptop |
User 6 |
Toronto |
Male |
19-24 |
Student |
High |
IBM Laptop |
User 7 (MS) |
Berkeley |
Male |
35-40 |
Staff |
medium |
Customized PC |
Do you do any of the following and if so how often?
User |
Checking email |
Instant message |
Shop online |
online banking |
Internet research |
Take class online |
Social networking |
---|---|---|---|---|---|---|---|
User 1 (RL) |
all the time |
hardly ever |
a few times a month |
all the time |
all the time |
hardly ever |
hardly ever |
User 2 (JH) |
all the time |
all the time |
a few times a week |
a few times a week |
all the time |
hardly ever |
all the time |
User 3 |
a few times a week |
a few times a week |
never |
never |
hardly ever |
never |
a few times a month |
User 4 |
all the time |
hardly ever |
never |
a few times a week |
all the time |
never |
never |
User 5 |
all the time |
hardly ever |
a few times a month |
a few times a month |
all the time |
hardly ever |
hardly ever |
User 6 |
all the time |
all the time |
hardly ever |
a few times a week |
a few times a week |
a few times a month |
a few times a week |
User 7 (MS) |
all the time |
hardly ever |
a few times a month |
never |
N/A (Never) |
a few times a month |
a few times a week |
Do you upload files on the web and if so how often?
User |
Pictures |
Media |
Documents to course site |
Documents to social networking site |
Attach a file in email |
---|---|---|---|---|---|
User 1 (RL) |
Hardly ever |
A few times a week |
A few times a month |
Hardly ever |
All the time |
User 2 (JH) |
a few times a month |
a few times a month |
all the time |
a few times a week |
all the time |
User 3 |
hardly ever |
hardly ever |
hardly ever |
hardly ever |
hardly ever |
User 4 |
hardly ever |
never |
all the time |
a few times a week |
all the time |
User 5 |
a few times a month |
a few times a week |
a few times a week |
hardly ever |
all the time |
User 6 |
hardly ever |
a few times a week |
a few times a week |
a few times a month |
a few times a week |
User 7 (MS) |
a few times a month |
a few times a month |
N/A (never) |
hardly ever |
all the time |
Interaction Notes
User Number |
Previous LMS Experience |
Task 1 - Move a particular box/portlet |
Task 2 - Change the placement of other boxes/portlets |
Task 3 - Deal with a Locked box |
Task 4 (if not done already) - drag and drop a portlet |
General comments |
---|---|---|---|---|---|---|
User 1 |
|
|
|
|
|
|
User 2 (JH) |
Moodle, bSpace |
|
|
|
n/a |
|
User 3 |
|
|
|
|
n/a |
|
User Number |
Previous LMS Experience |
Task 1 - Move a particular box/portlet |
Task 2 - Change the placement of other boxes/portlets |
Task 3 - Deal with a Locked box |
Task 4 (if not done already) - drag and drop a portlet |
General comments |
User 4 |
|
|
|
|
n/a |
|
User 5 |
|
|
|
|
n/a |
|
User 6 |
|
|
|
|
n/a |
|
User 7 |
|
|
|
|
n/a |
|
Post-test Questionnaire Responses
Question |
User 1 (RL) |
User 2 (JH) |
User 3 |
User 4 |
User 5 |
User 6 |
User 7 (MS) |
---|---|---|---|---|---|---|---|
How easy or difficult was it for you to realize you could move boxes in the portal by dragging a box from one location and dropping it in a new location? |
very easy |
very easy |
very easy |
neutral |
very easy |
easy |
neutral |
How easy or difficult was it for you to actually move a box in the portal (by dragging and dropping) to a specific location? |
easy |
very easy |
easy |
very easy |
very easy |
easy |
difficult |
How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place? |
neutral |
easy |
neutral |
neutral |
very easy |
difficult |
neutral (but it was difficult to realize this at first) |
Potential Design Improvements (based on testing)
- (DO)Performance improvements
- (DO)You can't drop here message should say something about why you can't drop here that allows the user to know why. Needs to be concise.
- (AB) Gary and I had come up with: "This box cannot be moved here. The box currently located here is locked."
- (DO)The green box could look different when it appears where you picked up the portlet than it does when it shows you where it will be dragged?
- (DO)Allow dragging into anyplace in the column and drop it underneath the last portlet in that column.
- (AB) Update the Layout Preview design pattern to explain the difficulties of using it with large portlets, and give guidelines for when it should not be used. Also talk about performance requirements and the difficulties of dragging large portlets across the page 'fold' (e.g. perhaps drag and drop may not be appropriate to use with large portlets which must be dragged across a fold.)
- (AB) Allow dragging and dropping into the region above or below of a column of portlets (e.g. don't require that the user get right next to the portlet above it to drop. This may mean dragging into an empty region at the bottom of a column, or dragging on top of the Welcome message at the top.).
- (AB) "This box cannot be moved here" warning message should appear if it appears to the user that they are dragging on top of a locked portlet, even if user initially drags the top of the portlet above an area which is not droppable (e.g. the orange "Welcome" text in the testing prototype).
- (AB) uPortal advice: there are so many links to "Customize" the portal and users aren't sure what they do--try to consolidate them. Perhaps explain how portlets can also be dragged and dropped on the Customize page, as well as how this relates the activities that are performed on this page.
- Portlet title bar should indicate somehow it can be moved. (Already has other icons that indicate functionality).
- Warning message not obvious. Could be hard to see against 2 different portlets.