Layout Reorderer User Testing - Round 3 Results
Summary
- 4 of 4 users were able to drag and drop portlets. The one user who took a longer time to realize she could do this (and she did eventually do it) was confused by the task.
- 1 user suggested that when mousing over a movable portlet it might be helpful for it to say "click & drag to move."
- 4 of 4 users understood what a locked portlet was
- Performance was markedly improved
Demographics
User Number |
Location |
Gender |
Age |
Role |
Tech-savvy |
Do you own a personal computer? |
---|---|---|---|---|---|---|
User 1 (BF) |
Berkeley |
Female |
41-50 |
Staff |
Medium |
Yes, Macbook |
User 2 (AL) |
Berkeley |
Male |
19-24 |
Student |
Medium-high |
Yes, Toshiba laptop (PC) |
User 3 (JV) |
Berkeley |
Female |
19-24 |
Student |
Medium |
laptop, windows XP |
User 4 (DB) |
Berkeley |
Female |
51-60 |
Staff |
Medium-low |
Yes, a PC |
User 5 |
|
|
|
|
|
|
User 6 |
|
|
|
|
|
|
User 7 |
|
|
|
|
|
|
Do you do any of the following and if so how often?
User |
Checking email |
Instant message |
Shop online |
Online banking |
Internet research |
Take class online |
Social networking |
---|---|---|---|---|---|---|---|
User 1 |
All the time |
All the time |
All the time |
All the time |
N/A |
A few times a month (webinar) |
No answer |
User 2 |
All the time |
A few times a month |
A few times a month |
A few times a month |
A few times a week |
Hardly ever |
A few times a week |
User 3 |
All the time |
A few times a month |
Hardly ever |
A few times a month |
A few times a week |
Never |
A few times a week |
User 4 |
All the time |
Hardly ever |
A few times a month |
All the time |
A few times a month |
A few times a month |
A few times a week |
User 4 |
|
|
|
|
|
|
|
User 5 |
|
|
|
|
|
|
|
User 6 |
|
|
|
|
|
|
|
User 7 |
|
|
|
|
|
|
|
Do you upload files on the web and if so how often?
User |
Pictures |
Media |
Docs to course site |
Docs to soc. ntwkg |
File to email |
---|---|---|---|---|---|
User 1 |
Hardly Ever |
Never |
Never |
Hardly Ever |
All the time |
User 2 |
A few times a month |
Hardly Ever |
Never |
Hardly Ever |
A few times a week |
User 3 |
A few times a week |
A few times a month |
Hardly ever |
Never |
All the time |
User 4 |
A few times a week |
Hardly ever |
Never |
Hardly Ever |
All the time |
User 5 |
|
|
|
|
|
User 6 |
|
|
|
|
|
User 7 |
|
|
|
|
|
Interaction Notes
User Number |
Previous LMS Experience |
Task 1 - Move a particular box/portlet |
Task 2 - Change the placement of other boxes/portlets |
Task 3 - Deal with a Locked box |
Task 4 (if not done already) - drag and drop a portlet |
General comments |
---|---|---|---|---|---|---|
User 1 |
|
The Announcements portlet wasn't there as it should have been, so we had the user move the Weather portlet. User seems to have thought we wanted her to move it to another tab, as she said "I never customize the tabs." Daphne had to explain that we wanted her to "move that box" to make it clear what we wanted. She had no problem dragging and dropping the box, but still didn't think that's what we wanted. She didn't understand at first that when you moved a box to a new spot, the box underneath it moved down--she was looking for this box for a bit and couldn't find it. |
She had no trouble dragging and dropping portlets, but she still wanted to interact with the tabs. She tried "Add content" and "Add tab." She said "I would put my bookmarks under a tab; I wouldn't put them up front." |
She dragged the portlet way up to the upper left and read the pop-up message. Then she said, "What's considered a locked box?" She realizes "Welcome" is locked when she sees the lock icon and quickly realizes she can't move the Bookmarks portlet up any higher because of it. |
She had no problem moving the image lightbox down. |
|
User 2 |
|
Scrolls down to Weather and drags and drops it with no problem. Says it's similar to the Facebook interface. |
Tries to drag Weather up. Moves Bookmarks to the left. Moves Image Gallery to the right and there's a little weirdness when he tries to move it to a space where there is no portlet - he can't drop it. |
Tries to drag there two times, then quickly sees that it's locked. Tries to click on the word "Welcome" in the portlet header to unlock it. |
Easily performs this task. |
|
User 3 |
|
Easily drags and drops the portlet right away. Tries to put it on a locked portlet and fails so moves it to upper right. Says right away, "I didn't see this lock so I tried to put this on top of it. Now the lock makes sense." |
Calendar drags a little jerkily as she moves it up, but she still has no problem moving it. |
She'd already done this in task #1 so we didn't ask again. |
She moves Weather down across/below a somewhat bit portlet with no problem. |
|
User 4 |
|
Initially asks, "is that considered a folder?" Then says, "no, it's an object on the page. Shouldn't there be something that tells me I can move things?" She sees "Page Layout." Then she says, "Can I move it (the portlet)? I *can* move it maybe?" Sees the red box and drops it and says, "I can!" Realizes she can and moves the portlet pretty quickly from start to finish. |
Easily drags Calendar between Weather and Bookmarks portlets. |
User tries to drag the portlet to the upper-left. Says, "it won't change with the Welcome so that's the top-most left." |
User easily moves the Weather portlet to the left. |
Thought it might be helpful when mousing over something for it to say "click & drag to move." |
User 5 |
|
|
|
|
|
|
User 6 |
|
|
|
|
|
|
User 7 |
|
|
|
|
|
|
Post-test Questionnaire Responses
Question |
User 1 |
User 2 |
User 3 |
User 4 |
User 5 |
User 6 |
User 7 |
---|---|---|---|---|---|---|---|
How easy or difficult was it for you to realize you could move boxes in the portal by dragging a box from one location and dropping it in a new location? |
Neutral |
Very easy |
Very easy |
Difficult |
|
|
|
How easy or difficult was it for you to actually move a box in the portal (by dragging and dropping) to a specific location? |
Very easy |
Easy |
Very easy |
Very easy |
|
|
|
How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place? |
Neutral |
Easy |
Very easy |
Easy |
|
|
|
Post-test Question Responses (verbally asked)
Question |
User 1 |
User 2 |
User 3 |
User 4 |
User 5 |
User 6 |
User 7 |
User 8 |
User 9 (DS) |
User 10 (KB) |
---|---|---|---|---|---|---|---|---|---|---|
1) Did you notice a visual cue for picking up one of these boxes? How helpful was it to you? (looking for cursor change to hand) |
Didn't notice a visual cue. Did see borders on a box come up when she started to drag and drop. |
He says he didn't notice a visual cue because "you said pick up the box" which told him he could do that. Says the bars on the top of the portlets are modeled "kind of like Windows." Also, the bar at the bottom of the portlet gives it sort of a 3-D feel, which makes him think he can move it. |
"I just assumed you could (drag and drop) because you can do that on Google things." She didn't notice a visual indicator. |
First things she saw were "Customize" or "Page Layout," like in MyYahoo! and a couple other places and thought that's what we wanted. (Though she says maybe MyYahoo! will allow you to drag and drop, too -- she doesn't know.) |
|
|
|
|
|
|
2) Did you notice a visual cue for dropping (and placing) a box? How helpful was it to you? (looking for green box drop target) |
She noticed the shadow of the box when the image is moving with you. Didn't realize it would move things down, however. |
The translucent view was a clue. I've seen that at deviantart.com. After thinking about it, he says, "I guess what helps with placement is the pink box." "It's sort of like solitaire--you pick up a card and move it." |
"Yeah, I like the red boxes. It makes it very clear." |
I should have read that Welcome is locked, I see it's locked now. I just moved it elsewhere and everything shifted around it. I felt comfortable about placement...after I tried to move it where it wouldn't go, I understood where things would go. |
|
|
|
|
|
|
3) Further questions about drop target, which can be skipped if answered in question #2:
|
It was confusing because I wasn't sure what the task was. It would have been helpful to tell her she was "customizing a web page for my own information... "I didn't realize that's what we were doing." I'm not sure how you'd add tabs. |
|
|
|
|
|
|
|
|
|
4) Did you see a visual cue that helped you determine whether a box could be picked up or not? How helpful was it to you? (looking for lock icon to show the portlet is locked.) |
|
The "combination lock thing." |
"It made sense; the box is locked. I can't move it." Said she saw nothing else on the screen (the message) that told her she couldn't move it. |
When I let it go and it went down below Welcome I was nervous (at the beginning of the test) so I didn't notice the lock. |
|
|
|
|
|
|
Potential Design Improvements (based on testing)
- There still seems to be a slight issue if you try to drag a portlet into whitespace way below other portlets--it can't be dropped until you drag it close to the bottom portlet on a column. You should be able to drop anywhere in that whitespace.