User Number | Previous LMS Experience | Task 1 - Move a particular box/portlet
| Task 2 - Change the placement of other boxes/portlets
| Task 3 - Deal with a Locked box
| Task 4 (if not done already) - drag and drop a portlet
| General comments |
|---|
User 1 |
| - Took a minute to find announcements
- What I would do is click & drag but I don't think you can
- She accidentally moved announcements to the left so when she went to move to the top there was a lot of whitespace before the next portlet. She tried to drag to the right but the portlet kept popping back to the left.
- This threw the rest of the test off.
- "I would have taken it to top but it just doesn't go there"
| - User was never able to drag and drop. She was successful with moving announcements to the top by "canceling" (removing) portlets.
- I've done this with yahoo.
- I'll just start canceling the ones I don't want until I get them up there
- Wasn't seeing the green box move to the drop target - slowness, couldn't drop in right bottom of column & lots of scrolling then can't lose sight of green box.
| - She noticed the you can't drag her message but never noticed the lock icon.
- She wasn't sure why she couldn't drop there and took it as an indication that she couldn't drop it at all.
| - Finally told her she could drag and drop and asked her to try again. This is when she was scrolling forever and had to get out of her seat to scroll far enough.
| - "I would have taken it to top but it just doesn't go there"
| - User was never able to drag and drop. She was successful with moving announcements to the top by "canceling" (removing) portlets.
- I've done this with yahoo.
- I'll just start canceling the ones I don't want until I get them up there
- Wasn't seeing the green box move to the drop target - slowness, couldn't drop in right bottom of column & lots of scrolling then can't lose sight of green box.
| - She noticed the you can't drag her message but never noticed the lock icon.
- She wasn't sure why she couldn't drop there and took it as an indication that she couldn't drop it at all.
| - Finally told her she could drag and drop and asked her to try again. This is when she was scrolling forever and had to get out of her seat to scroll far enough.
| - "I would have taken it to top but it just doesn't go there"
- Lag time was also throwing off the test. It's just pretty clunky moving things around
- Long scrolling was a problem trying to drag things from the bottom all the way to the top - she could barely reach her arms up long enough.
- "The way I do things is click around "I'm clicker".
- She blamed the misbehavior on the mouse but it was the slow, jerkiness of the system.
- While filling out the questionnaire she said, I think the mouse wasn't work well or it would have been easy as she started to mark easy. I encouraged her to take the mouse factor as is and assume it was working the way it should - that it was not the problem. She still answered "easy".
|
User 2 (JH) | Moodle, bSpace
| - first clicked Change Layout but quickly realized that this was for column layout
- moused around screen and noticed that the pointer changed to a hand over the pane header for each portlet, then easily dragged the Announcements portlet to the correct position.
| - easily started to make changes
| - noticed that the Welcome portlet was locked by the locked icon on the portlet
- also noticed the rollover text
| n/a | - expects things like this to configurable from his experience with social networking sites and Moodle.
- Very comfortable with drag and drop
- the change in cursor was the key indicator for this user
|
User 3 | - When asked how do you move a box? Answer: no idea. "you mean click on something and move it?"
- clicked the X to move the box
- clicked the window icon to drag
- eventually saw a green box and figured how to drag a portlet
| - moved portal vertically on right column, drop marker showed on left column but bounced back
- tried to drag portal down to empty vertical space below the portlet.
- Surprised when it doesn't stick.
- Why are you moving it there? "Because it's empty space"
| - did not notice the help text
- help text sometimes flashes quickly
- tried multiple times to drag portlet, not sure why it wasn't moving
| n/a | - user was dragged the icons thinking that was how to accomplish the task.
- warning text when dragging over locked portlet not obvious and missed by user.
| User Number | Previous LMS Experience | Task 1 - Move a particular box/portlet
| Task 2 - Change the placement of other boxes/portlets
| Task 3 - Deal with a Locked box
| Task 4 (if not done already) - drag and drop a portlet
| General comments |
|---|
User 4 | - instead of repositioning, he says he would scroll down to view instead of moving it.
- not sure how to reposition a box. mouse exploring, but not noticing the change in mouse cursor
- clicked on the announcement box and it took him to the announcement page.
- multiple times could not notice the change in mouse icon to a "grab" hand
- Said he is usually not expecting this kind of behaviour
| - had no problems moving a portal up and down column once he figured it out.
| - did not notice warning several times
- thinks the portal did not stick because the portlet was too big. despite seeing a green box below the locked portlet.
- not sure why the portlet is "locked". "So you can't place anything above this?"
| n/a | - kept looking at the icons on the top bar, but none of them made sense to move an object
- would like to see an icon to show that it is movable.
| User 5 | - dragged and dropped "there you go. cool!" * dragged and dropped a bunch of other ones
- "this one's locked i guess"
| - went to welcome page, came back, everything was on the left column
tries to move the announcement to the top of right column, unsuccessful
| - "locked i guess. it wouldn't go there"
(because of the lock, and because it told me it's locked)
| n/a | - Upon visiting the page he immediately began giving feedback on the site's layout and design.
- "I wouldn't have the weather here, because it's not as important"
- scrolled up and down
- "would switch the announcement and weather"
| User 6 | - hovers over announcements, grab and move up.
| - moved other boxes, no problems.
- clicked on the locked box, but then quickly released it.
- he was asked "what did you just do there?" "nothing". Then asked "Why didn't you move it?" "Because it's where I like it." (Didn't discover it was locked)
| - grabbed and moved on top of the locked portlet, moving didn't work.
- made sense to him because other sites have ads that can't be moved (the box looks like an ad, I guess. -JH)
| n/a | User 7
(see attached Word version of results with screenshots) | | - User says he would like an anchor he could jump down to at the bottom of the page to see Announcements (e.g. maybe an "On this page" list at the top)
- User tries drag and drop and the performance is poor---it's hard to see where the portlet is going because it jumps around a lot. He says, "You can't move it yourself."
- After more effort, user moves Announcements to the top right corner. At times he has trouble seeing the drop target as it gets lost when dragging over a large portlet. (Note: Much of the users' problem following the portlets' movements I think was tied to performance problems.)
| User has less trouble moving portlets left to right, but more trouble moving them up and down, in part because it's difficult to see the drop target among the large portlets. User comments that he's unsure whether the green box represents where the portlet is coming from or where it's going.User has trouble moving portlets up and down "across" the page fold because the interaction is so slow, and the page doesn't scroll properly when he tries to move the portlet across the "fold."
- Bug: User tries to drag the Announcements portlet from the top of the right column directly over the orange "Welcome" and doesn't receive any message (he should receive one saying that the portlet below is locked). When he drops the portlet and it goes back to where it was he says, "I guess I cannot move it" but does not appear to understand why.
| n/a | - In general this user had a lots of problems understanding how portlets were moving around due to performance issues. Tested on FF2 on Mac OS 10.5.
- Visual cue for picking up a box?
- We are looking for the cursor change to hand here, but the user talks about the lock.
- "I didn't pay attention." Then he looks around a bit and notices the lock. He says, "I wonder if there's a tutorial" and looks in Customize. He sees the interface for moving portlets around without drag and drop, and at a quick glance seems to think this is a tutorial rather than another interface for moving portlets. He says, "it gives you all the clues about what you can do."
- Visual cue for dropping a box? (looking for green drop target)
- "I didn't notice anything, but I now I guess the color shows this."
- User commented that it was hard to see the visual cue for where the box would go.
- Commented at one point on the size of the avatar, saying "is this the right size?" (Note: Currently the avatar size doesn't match the portlet size.)
He said it was clear at the bottom of the page (basically when he was moving into blank space) where things would move, but harder to tell when moving a portlet to the top through other portlets.Lag time was also throwing off the test. It's just pretty clunky moving things around- Long scrolling was a problem trying to drag things from the bottom all the way to the top - she could barely reach her arms up long enough.
- "The way I do things is click around "I'm clicker".
- She blamed the misbehavior on the mouse but it was the slow, jerkiness of the system.
- While filling out the questionnaire she said, I think the mouse wasn't work well or it would have been easy as she started to mark easy. I encouraged her to take the mouse factor as is and assume it was working the way it should - that it was not the problem. She still answered "easy".
|
User 2 (JH) | Moodle, bSpace
| - first clicked Change Layout but quickly realized that this was for column layout
- moused around screen and noticed that the pointer changed to a hand over the pane header for each portlet, then easily dragged the Announcements portlet to the correct position.
| - easily started to make changes
| - noticed that the Welcome portlet was locked by the locked icon on the portlet
- also noticed the rollover text
| n/a | - expects things like this to configurable from his experience with social networking sites and Moodle.
- Very comfortable with drag and drop
- the change in cursor was the key indicator for this user
|
User 3 |
| - When asked how do you move a box? Answer: no idea. "you mean click on something and move it?"
- clicked the X to move the box
- clicked the window icon to drag
- eventually saw a green box and figured how to drag a portlet
| - moved portal vertically on right column, drop marker showed on left column but bounced back
- tried to drag portal down to empty vertical space below the portlet.
- Surprised when it doesn't stick.
- Why are you moving it there? "Because it's empty space"
| - did not notice the help text
- help text sometimes flashes quickly
- tried multiple times to drag portlet, not sure why it wasn't moving
| n/a | - user was dragged the icons thinking that was how to accomplish the task.
- warning text when dragging over locked portlet not obvious and missed by user.
|
User Number | Previous LMS Experience | Task 1 - Move a particular box/portlet
| Task 2 - Change the placement of other boxes/portlets
| Task 3 - Deal with a Locked box
| Task 4 (if not done already) - drag and drop a portlet
| General comments |
|---|
User 4 |
| - instead of repositioning, he says he would scroll down to view instead of moving it.
- not sure how to reposition a box. mouse exploring, but not noticing the change in mouse cursor
- clicked on the announcement box and it took him to the announcement page.
- multiple times could not notice the change in mouse icon to a "grab" hand
- Said he is usually not expecting this kind of behaviour
| - had no problems moving a portal up and down column once he figured it out.
| - did not notice warning several times
- thinks the portal did not stick because the portlet was too big. despite seeing a green box below the locked portlet.
- not sure why the portlet is "locked". "So you can't place anything above this?"
| n/a | - kept looking at the icons on the top bar, but none of them made sense to move an object
- would like to see an icon to show that it is movable.
|
User 5 |
| - dragged and dropped "there you go. cool!" * dragged and dropped a bunch of other ones
- "this one's locked i guess"
| - went to welcome page, came back, everything was on the left column
tries to move the announcement to the top of right column, unsuccessful
| - "locked i guess. it wouldn't go there"
(because of the lock, and because it told me it's locked)
| n/a | - Upon visiting the page he immediately began giving feedback on the site's layout and design.
- "I wouldn't have the weather here, because it's not as important"
- scrolled up and down
- "would switch the announcement and weather"
|
User 6 |
| - hovers over announcements, grab and move up.
| - moved other boxes, no problems.
- clicked on the locked box, but then quickly released it.
- he was asked "what did you just do there?" "nothing". Then asked "Why didn't you move it?" "Because it's where I like it." (Didn't discover it was locked)
| - grabbed and moved on top of the locked portlet, moving didn't work.
- made sense to him because other sites have ads that can't be moved (the box looks like an ad, I guess. -JH)
| n/a |
|
User 7
(see attached Word version of results with screenshots) | | - User says he would like an anchor he could jump down to at the bottom of the page to see Announcements (e.g. maybe an "On this page" list at the top)
- User tries drag and drop and the performance is poor---it's hard to see where the portlet is going because it jumps around a lot. He says, "You can't move it yourself."
- After more effort, user moves Announcements to the top right corner. At times he has trouble seeing the drop target as it gets lost when dragging over a large portlet. (Note: Much of the users' problem following the portlets' movements I think was tied to performance problems.)
| - User has less trouble moving portlets left to right, but more trouble moving them up and down, in part because it's difficult to see the drop target among the large portlets. User comments that he's unsure whether the green box represents where the portlet is coming from or where it's going.
- User has trouble moving portlets up and down "across" the page fold because the interaction is so slow, and the page doesn't scroll properly when he tries to move the portlet across the "fold."
| - Bug: User tries to drag the Announcements portlet from the top of the right column directly over the orange "Welcome" and doesn't receive any message (he should receive one saying that the portlet below is locked). When he drops the portlet and it goes back to where it was he says, "I guess I cannot move it" but does not appear to understand why.
| n/a | - In general this user had a lots of problems understanding how portlets were moving around due to performance issues. Tested on FF2 on Mac OS 10.5.
- Visual cue for picking up a box?
- We are looking for the cursor change to hand here, but the user talks about the lock.
- "I didn't pay attention." Then he looks around a bit and notices the lock. He says, "I wonder if there's a tutorial" and looks in Customize. He sees the interface for moving portlets around without drag and drop, and at a quick glance seems to think this is a tutorial rather than another interface for moving portlets. He says, "it gives you all the clues about what you can do."
- Visual cue for dropping a box? (looking for green drop target)
- "I didn't notice anything, but I now I guess the color shows this."
- User commented that it was hard to see the visual cue for where the box would go.
- Commented at one point on the size of the avatar, saying "is this the right size?" (Note: Currently the avatar size doesn't match the portlet size.)
- He said it was clear at the bottom of the page (basically when he was moving into blank space) where things would move, but harder to tell when moving a portlet to the top through other portlets.
|