Uploader User Testing - Round 1 Results

Summary

The user testing on the Uploader was carried out in a "quick-and-dirty" fashion. Ten users (5 in Toronto, 5 in Berkeley) were asked to spend less than 30 minutes with us to complete 4 upload tasks. Below are the results thus far.

Toronto:

All users were able to successfully upload files
3 of 4 users asked if selecting multiple files was possible before being asked to add multiple files. Given the selected users were rather tech-savvy, this demonstrates the users' high expectations.
All users acknowledged the list of selected files and proceeded to click on Upload

Berkeley:

All users were "wowed" by being able to browse and select multiple files at once. Even though they didn't expect it, all quickly selected multiple files.
While files were uploading, 4 of 4 users chose "browse collections" to return to the image gallery thumbnail view. They expected the upload to continue "behind the scenes". There was no feedback that the upload would stop (but that's what happens). This happened on one to several tasks for each user.
One user mentioned it would be nice to see thumbnails in the upload queue. He said the thumbnail could even get bigger as the user mouses over it.
3 of 4 users said they would not stop the upload in progress to remove files from the queue. They would wait until all files were uploaded and then remove them from the Gallery. One user did mention that if she was doing it on her own she probably would have played around with the pause.
4 out of 4 users didn't immediately notice when the files were done uploading. For a couple it took prompting for them to realize. A couple users mentioned that it was pretty subtle. They expected to be taken back to where they started from (browse images & collections) when the upload was complete.
- Increase the visibility of the "File Uploaded" message. How about "Upload Complete" in bold? There is no reason to grey it out. Users immediately think greyed out information is irrelevant.
The one user that used "remove" in the upload queue said it is too subtle. She had trouble seeing that the file was actually gone (files name with default camera numbers). She ended up removing extra files because she didn't think it was removing.

Demographics

The users were selected at random by the tester rather than through a formal recruiting process. Below table shows the demographics and previous experiences of each user we spoke to. We felt that such criteria as areas of study has little relevance to users' behaviour with file uploader, so we made conscious decision to leave them out.

Demographic Background

User	Gender	Age	Role	Type of Computer

User	Gender	Age	Role	Type of Computer
UT User 1	Female	25-30	Grad Student	Windows laptop
UT User 2	Male	31-35	Grad Student	Windows laptop
UT User 3	Male	25-30	Grad Student	Windows laptop
UT User 4	Female	19-24	Student	Windows desktop
UCB User 1	Male	25-30	Staff	Mac laptop
UCB User 2	Female	35-40	Staff	Mac
UCB User 3	Female	41-50	Staff	PC
UCB User 4	Female	31-35	Staff	PC at work; no home computer

Do you do any of the following online and if so how often?

User	Email	IM	Shop Online	Online Banking	Research Online	Online Classes	Social Networking

User	Email	IM	Shop Online	Online Banking	Research Online	Online Classes	Social Networking
UT User 1	All the time	A few/week	A few/week	A few/week	All the time	A few/month	All the time
UT User 2	All the time	All the time	A few/month	A few/month	All the time	Hardly ever	A few/week
UT User 3	All the time	All the time	A few/month	A few/week	All the time	Hardly ever	A few/week
UT User 4	All the time	All the time	A few/month	A few/month	All the time	Hardly ever	A few/week
UCB User 1	All the time	A few/week	A few/week	Never	All the time	Hardly ever	Hardly ever
UCB User 2	All the time	All the time	A few/month	A few/week	All the time	Hardly Ever	Hardly Ever
UCB User 3	All the time	All the time	A few/month	Never	A few/month	Never	A few/month
UCB User 4	All the time	All the time	A few/moth	Never	Never	Never	Hardly Ever

Do you upload files on the web and if so how often?

User	Pictures	Media	Documents to course sites	Documents to social networking sites	attachment in email

User	Pictures	Media	Documents to course sites	Documents to social networking sites	attachment in email
UT User 1	A few/month	Hardly ever	A few/month	Hardly ever	All the time
UT User 2	A few/month	A few/month	A few/month	A few/month	All the time
UT User 3	A few/week	A few/week	A few/week	Hardly ever	All the time
UT User 4	A few/month	Never	Never	Never	A few/week
UCB User 1	A few/month	A few/month	A few/month	A few/month	A few/month
UCB User 2	A few/month	A few/month	A few/month	Hardly ever	All the time
UCB User 3	All the time	All the time	All the time	A few/month	All the time
UCB User 4	A few/month	Never	A few/week	Hardly ever	All the time


Legend
All the time - A few/week - A few/month - Hardly ever - Never

Interaction Notes

Setup:

These tests were done in "hallway" style. We grabbed people community areas and asked them to participate for a few minutes.
All tests done on a Mac.
Removed emphasis placed on the key convention for selecting multiple files, since the users were PC users
Added a question at the end of Task 4 "Once you're done adding all the files you want, what would you do next?" to see how they exit the File Uploader

Although the PC users were asked to perform the tasks on Erin or Daphne's Macbook, they had little problems completing the tasks. If they were obviously looking for the ctrl key to choose non-consecutive multiples, they were prompted that on a Mac the apple key is the parallel. We assume they would normally be completing this task on their own computer and we were not testing their knowledge of the keyboard shortcuts.

Buttons the users clicked on are notated in [fluid:brackets].

Task 1: Upload a file from the Photography101 folder on your desktop.

Optimal Path: Add Images > Browse > Select file > Upload > Browse Images & Collections

Task 2: Upload the 4 most recent files in the Photography101 folder.

Optimal Path: Add Images > Browse > Select files > Upload > Browse Images & Collections

Task 3: Upload all files in Photography101. <while upload progress is going> You realized that IMG_9409.jpg and IMG_9407.jpg are the wrong files and you didn't intend to upload those to the Photography class site. What would you do?

Optimal Path: Add Images > Browse > Select file(s) from desktop > Upload > Pause > Remove both files > Resume Upload > Browse Images & Collections

Task 4: If you wanted to add a couple files from Photography101 and some files from your desktop, how would you go about doing that?

Optimal Path: Add Images > Browse > Select files from Photography 101 folder > Add more files > Select file(s) from desktop > Upload > Browse Images & Collections

User	Task 1	Task 2	Task 3	Task 4	General Observations

User	Task 1	Task 2	Task 3	Task 4
UT User 1	- [fluid:Browse] - Double-clicked on file - [fluid:Upload] - "It's weird it doesn't show me the files uploaded"	- [fluid:Shift] to select multiple files - "Nice that it tells me the progress" - "This is pretty self-explanatory"	- [fluid:Pause] - Sees that the files she wants removed have already been uploaded, "Here I can't do anything I guess."	- To exit, "I would click on the breadcrumb" - Expects to go back to the "main page" to "reorder, add descriptions, pick cover photo, send it to people"
UT User 2	- [fluid:Browse] - Picked the folder - Picked an image - [fluid:Select] - acknowledges "Ready to Upload" - acknowledges "file uploaded"	- "files are already sorted by dates modified" - [fluid:Shift] to select multiple - Four files are "ready" - [fluid:Upload]	- "If it's just two wrong files, I would wait till the upload is completed. If it's more than half of the files that are wrong, I would pause and do the selection again." - Looked for the two wrong files mentioned and said "I can't select these files" (expected to be able to remove even the already-uploaded files from this screen) - Would go to "Browse Image Collection" to check to see where these files are	- "Remove is right here! for the previous task, I would've used this remove button." - Feels that it should support two different kinds of remove: 1) remove from the "intention list" 2) if they are uploaded already "remove from collection" - "It's surprising the rows can't be clicked on" referring to the rows in the table
UT User 3	- [fluid:Browse] - "So you can upload all of these?" to which I answered Yes, and the user skipped Task 1	- [fluid:Shift] to select multiple - [fluid:Upload]	- [fluid:Pause] - [fluid:Remove] files he doesn't want - [fluid:Upload] the rest	- Would click on "Browse Images" - "Would be nice to see what the images look like on the actual site, like thumbnails of images, so I can see exactly what I've uploaded and remove some if needed"