Summary
| Excerpt |
|---|
|
- 1 user said she really likes the idea of personalizing her spaces.
- 1 user thought the icon in front of portlet title was the indicator that you could drag
- 1 user noticed the 'you can't drag here' message but was confused about when and why she couldn't. Because of earlier problems she took it as another indication that you couldn't really drag and drop.
Demographics
User Number | Location | Gender | Age | Role | Tech-savvy | Do you own a personal computer? |
|---|---|---|---|---|---|---|
User 1 (RL) | Berkeley | Female | 51-60 | Staff | N/A | Mac |
User 2 (JH) | Berkeley | Male | 35-40 | Staff | high | Mac laptop |
User 3 | Toronto | Female | 35-40 | Student | Low | HP Desktop |
User 4 | Toronto | Male | 25-30 | Grad Student | Low | Dell Laptop |
User 5 | Toronto | Male | 25-30 | Student | High | mac laptop |
User 6 | Toronto | Male | 19-24 | Student | High | IBM Laptop |
User 7 (MS) | Berkeley | Male | 35-40 | Staff | medium | Customized PC |
Do you do any of the following and if so how often?
User | Checking email | Instant message | Shop online | online banking | Internet research | Take class online | Social networking |
|---|---|---|---|---|---|---|---|
User 1 (RL) | all the time | hardly ever | a few times a month | all the time | all the time | hardly ever | hardly ever |
User 2 (JH) | all the time | all the time | a few times a week | a few times a week | all the time | hardly ever | all the time |
User 3 | a few times a week | a few times a week | never | never | hardly ever | never | a few times a month |
User 4 | all the time | hardly ever | never | a few times a week | all the time | never | never |
User 5 | all the time | hardly ever | a few times a month | a few times a month | all the time | hardly ever | hardly ever |
User 6 | all the time | all the time | hardly ever | a few times a week | a few times a week | a few times a month | a few times a week |
User 7 (MS) | all the time | hardly ever | a few times a month | never | N/A (Never) | a few times a month | a few times a week |
Do you upload files on the web and if so how often?
User | Pictures | Media | Documents to course site | Documents to social networking site | Attach a file in email |
|---|---|---|---|---|---|
User 1 (RL) | Hardly ever | A few times a week | A few times a month | Hardly ever | All the time |
User 2 (JH) | a few times a month | a few times a month | all the time | a few times a week | all the time |
User 3 | hardly ever | hardly ever | hardly ever | hardly ever | hardly ever |
User 4 | hardly ever | never | all the time | a few times a week | all the time |
User 5 | a few times a month | a few times a week | a few times a week | hardly ever | all the time |
User 6 | hardly ever | a few times a week | a few times a week | a few times a month | a few times a week |
User 7 (MS) | a few times a month | a few times a month | N/A (never) | hardly ever | all the time |
Questionnaire Responses
Question | User 1 (RL) | User 2 (JH) | User 3 | User 4 | User 5 | User 6 | User 7 (MS) |
|---|---|---|---|---|---|---|---|
How easy or difficult was it for you to realize you could move boxes in the portal by dragging a box from one location and dropping it in a new location? | very easy | very easy | very easy | neutral | very easy | easy | neutral |
How easy or difficult was it for you to actually move a box in the portal (by dragging and dropping) to a specific location? | easy | very easy | easy | very easy | very easy | easy | difficult |
How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place? | neutral | easy | neutral | neutral | very easy | difficult | neutral (but it was difficult to realize this at first) |
Interaction Notes
...
User Number
...
Previous LMS Experience
...
Task 1 - Move a particular box/portlet
...
Task 2 - Change the placement of other boxes/portlets
...
Task 3 - Deal with a Locked box
...
Task 4 (if not done already) - drag and drop a portlet
...
General comments
...
User 1
...
- Took a minute to find announcements
- What I would do is click & drag but I don't think you can
- She accidentally moved announcements to the left so when she went to move to the top there was a lot of whitespace before the next portlet. She tried to drag to the right but the portlet kept popping back to the left.
- This threw the rest of the test off.
- "I would have taken it to top but it just doesn't go there"
...
- User was never able to drag and drop. She was successful with moving announcements to the top by "canceling" (removing) portlets.
- I've done this with yahoo.
- I'll just start canceling the ones I don't want until I get them up there
- Wasn't seeing the green box move to the drop target - slowness, couldn't drop in right bottom of column & lots of scrolling then can't lose sight of green box.
...
- She noticed the you can't drag her message but never noticed the lock icon.
- She wasn't sure why she couldn't drop there and took it as an indication that she couldn't drop it at all.
...
- Finally told her she could drag and drop and asked her to try again. This is when she was scrolling forever and had to get out of her seat to scroll far enough.
...
- "I would have taken it to top but it just doesn't go there"
- Lag time was also throwing off the test. It's just pretty clunky moving things around
- Long scrolling was a problem trying to drag things from the bottom all the way to the top - she could barely reach her arms up long enough.
- "The way I do things is click around "I'm clicker".
- She blamed the misbehavior on the mouse but it was the slow, jerkiness of the system.
- While filling out the questionnaire she said, I think the mouse wasn't work well or it would have been easy as she started to mark easy. I encouraged her to take the mouse factor as is and assume it was working the way it should - that it was not the problem. She still answered "easy".
...
User 2 (JH)
...
Moodle, bSpace
...
- first clicked Change Layout but quickly realized that this was for column layout
- moused around screen and noticed that the pointer changed to a hand over the pane header for each portlet, then easily dragged the Announcements portlet to the correct position.
...
- easily started to make changes
...
- noticed that the Welcome portlet was locked by the locked icon on the portlet
- also noticed the rollover text
...
n/a
...
- expects things like this to configurable from his experience with social networking sites and Moodle.
- Very comfortable with drag and drop
- the change in cursor was the key indicator for this user
...
User 3
...
- When asked how do you move a box? Answer: no idea. "you mean click on something and move it?"
- clicked the X to move the box
- clicked the window icon to drag
- eventually saw a green box and figured how to drag a portlet
...
- moved portal vertically on right column, drop marker showed on left column but bounced back
- tried to drag portal down to empty vertical space below the portlet.
- Surprised when it doesn't stick.
- Why are you moving it there? "Because it's empty space"
...
- did not notice the help text
- help text sometimes flashes quickly
- tried multiple times to drag portlet, not sure why it wasn't moving
...
n/a
...
- user was dragged the icons thinking that was how to accomplish the task.
- warning text when dragging over locked portlet not obvious and missed by user.
...
User Number
...
Previous LMS Experience
...
Task 1 - Move a particular box/portlet
...
Task 2 - Change the placement of other boxes/portlets
...
Task 3 - Deal with a Locked box
...
Task 4 (if not done already) - drag and drop a portlet
...
General comments
...
User 4
...
- instead of repositioning, he says he would scroll down to view instead of moving it.
- not sure how to reposition a box. mouse exploring, but not noticing the change in mouse cursor
- clicked on the announcement box and it took him to the announcement page.
- multiple times could not notice the change in mouse icon to a "grab" hand
- Said he is usually not expecting this kind of behaviour
...
- had no problems moving a portal up and down column once he figured it out.
...
- did not notice warning several times
- thinks the portal did not stick because the portlet was too big. despite seeing a green box below the locked portlet.
- not sure why the portlet is "locked". "So you can't place anything above this?"
...
n/a
...
Interaction Notes
User Number | Previous LMS Experience | Task 1 - Move a particular box/portlet | Task 2 - Change the placement of other boxes/portlets | Task 3 - Deal with a Locked box | Task 4 (if not done already) - drag and drop a portlet | General comments |
|---|---|---|---|---|---|---|
User 1 |
|
|
|
|
| |
User 2 (JH) | Moodle, bSpace |
|
|
| n/a |
|
User 3 |
|
|
| n/a |
| |
User Number | Previous LMS Experience | Task 1 - Move a particular box/portlet | Task 2 - Change the placement of other boxes/portlets | Task 3 - Deal with a Locked box | Task 4 (if not done already) - drag and drop a portlet | General comments |
User 4 |
|
|
| n/a |
| |
User 5 |
|
|
| n/a |
| |
User 6 |
|
|
| n/a | ||
User 7 |
|
|
|
| n/a |
|
Post-test Questionnaire Responses
Question | User 1 (RL) | User 2 (JH) | User 3 | User 4 | User 5 | User 6 | User 7 (MS) |
|---|---|---|---|---|---|---|---|
How easy or difficult was it for you to realize you could move boxes in the portal by dragging a box from one location and dropping it in a new location? | very easy | very easy | very easy | neutral | very easy | easy | neutral |
How easy or difficult was it for you to actually move a box in the portal (by dragging and dropping) to a specific location? | easy | very easy | easy | very easy | very easy | easy | difficult |
How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place? | neutral | easy | neutral | neutral | very easy | difficult | neutral (but it was difficult to realize this at first) |
Potential Design Improvements (based on testing)
- (DO)Performance improvements
- (DO)You can't drop here message should say something about why you can't drop here that allows the user to know why. Needs to be concise.
- (AB) Gary and I had come up with: "This box cannot be moved here. The box currently located here is locked."
- (DO)The green box could look different when it appears where you picked up the portlet than it does when it shows you where it will be dragged?
- (DO)Allow dragging into anyplace in the column and drop it underneath the last portlet in that column.
- (AB) Update the Layout Preview design pattern to explain the difficulties of using it with large portlets, and give guidelines for when it should not be used. Also talk about performance requirements and the difficulties of dragging large portlets across the page 'fold' (e.g. perhaps drag and drop may not be appropriate to use with large portlets which must be dragged across a fold.)
- (AB) Allow dragging and dropping into the region above or below of a column of portlets (e.g. don't require that the user get right next to the portlet above it to drop. This may mean dragging into an empty region at the bottom of a column, or dragging on top of the Welcome message at the top.).
- (AB) "This box cannot be moved here" warning message should appear if it appears to the user that they are dragging on top of a locked portlet, even if user initially drags the top of the portlet above an area which is not droppable (e.g. the orange "Welcome" text in the testing prototype).
- (AB) uPortal advice: there are so many links to "Customize" the portal and users aren't sure what they do--try to consolidate them. Perhaps explain how portlets can also be dragged and dropped on the Customize page, as well as how this relates the activities that are performed on this page.
- Portlet title bar should indicate somehow it can be moved. (Already has other icons that indicate functionality).
- Warning message not obvious. Could be hard to see against 2 different portlets.