Mobile user testing results, Participant 4 (Engage 0.3)

I. Reality testing

P refers to participant. M refers to moderator. Quotes are paraphrased, not verbatim. Side notes/observations in round brackets. Initial thoughts in square brackets.

P: "I had no idea what idea the software would be doing."
P knows the photographs well (is the photo archivist at the museum)
Notes object code label, and notes the object code entry option on the screen
Enters the code for an object
Looks at device, looks at object, P: "Looks about right"
Upon looking closer, notes that they're actually a bit different (object is a photograph, but photo in the artifact view is a negative plate)
Saw 'Collect' option in artifact view, tapped on it to "see what happens"
P not scrolling on the artifact view
P: "Performance is a bit slow"
Sees numbers on other labels (not object code labels), tried tapping it in, but didn't work
Wanted information for an object that we didn't have a direct code for
Browsed through Exhibitions screen
Found full catalogue, looking for thing
P: "Good, but not as good as having a number to tap on screen and find out more information."
P: "Wanted to find that thermometer; if there's a search facility, that would be nice, that way I can enter in the name if I knew it"
P: "I don't see it on the list; is the list complete?"
P: "Lots of other neat stuff, but not that one"
P: "Let's try one of the view alls, in case the thermometer is in there---nope, no sign of it"
Would be nice if there was a navigation bar at the bottom instead of going all the way back to the top, or a thing at the bottom saying 'go back to top'
Saw a cap and some other objects in the space, trying to find it in the device (using default full catalogue)---can't find them in the device
P: "[Theme] sections in the device don't seem to match up with the sections in the actual space" [if we're going to use section names, and have it available in the in-museum experience, we'd better make sure the physical and virtual content maps really well to each other]
Sees stove in the space, remembers seeing stove in the full catalogue. P: "Let's see if that stove is in here [the device]"
P: "Let's see if it picks up the right one... Yep, picked up the right one."
P: "Going to try adding a note, just to see if it works."
Correctly tapped on text field to bring up keyboard
Tapped DONE, then SUBMIT
Did not look to see if self's comment showed up under the comment list
P: "Unclear whether these comments are personal comments, or whether it'd be publicly visible."
P: "It is a little slow. It's still working on it."--M: "How do you know it's still working on it?"--P: "There's a little 'I'm loading' symbol at the top."
Makes occasional mistaps (i.e., intends to tap one thing, but ends up tapping another)
P sees a painting in the space, looking for it on device (painting does not have an object code)
P: "Is there a way of looking for items by type of item? If I know what I'm looking for is a painting, maybe there's a way to look for it without looking for it up and down the list."
P: "Doesn't seem to be a way to look for a painting when you know it's a painting; you'd have to get the whole list out. Not what I expected."
Full catalogue not entirely intuitive---only see a few artifacts within a section, and thinks that's all there is
P: "The 'view all' thing should be a little louder"
On a painting:
P: "Tapping on the image doesn't bring it up bigger, and zooming doesn't seem to do anything."
P: "Already have the thing in front of me, but if I wanted to see a detail of it, it'd be nice if I could click on it and zoom or something like that."
On performance: P: "It seems to do a two-step thing---I click on it, and it loads, and then does something, and then loads again. It's a bit disconcerting."
On seeing a media badge icon in the list (incorrectly as an audio icon, but really is a video):
P: "This one has sound---can I pull out my earphones and listen to it?" — M: "There are built-in speakers." — P: "Yes, but they're very tiny sounding."
Goes ahead and tries to listen to it
Taps on the list item, and then immediately puts device to ear---does not think to look at screen
With no audio coming out, P notes not hearing anything, and looks at screen, pushes volume buttons up and tries putting to ear again
Presses back and tries tapping on the actual audio symbol
Expected audio to start playing when tapping on item, simultaneous with being able to view artifact
Tried this for multiple artifacts, also with code entry
P: "Have to go right back to the beginning to enter one of these numbers" (repeated multiple times)
P: "... in fact, right back to the beginning... probably to the home page... right back to the entrance of the page" (some annoyance detected in voice?)
P: "If we wanted to type in the number a little earlier, it might be nice to put the option in a little earlier"
(used back button instead of 'Home' button)
Was in artifact view after object code:
P: "Need to work my way back into Exhibitions again because to get out of the object code thing you need to go all the way back to the beginning. Some exit, halfway along would be a good idea." [did not use 'Home' button; also, Exhibitions and Object Code Entry modes were designed to be separate experiences, but clearly this isn't the case with actual users]
P: "Excellent idea to have the object image and description on the screen because it might be difficult to see details on the object, or if the lighting is bad, the label"
M: "What do you think the collecting option did?" — P: "I would think it would put the information and image into a separate little group that you should be able to recall later. What you would do with that is up to you, maybe for student papers or research. Hopefully you'll be able to look at it after the exhibition, maybe with some saving technique, maybe access these things from the McCord website afterward, maybe download the image to use in a presentation."

II. Task-based testing

P refers to participant. M refers to moderator. Quotes are paraphrased, not verbatim. Side notes/observations in round brackets. Initial thoughts in square brackets.

On playing a video, P knew to flip the device's orientation to get a bigger landscape video
P notes the interface: controls, scrub, length of video
P interrupted the video, didn't appear to want to watch the rest of the video, tapped 'Done'
P notes that none of the related artifacts are from the video (re: curling stone video)
P saw a picture of something in the video, wanted to know what object that was, and found no identifiable way of finding it [our design relies on the content-provider to put in appropriate items under related artifacts and has no explicit affordance for discovering objects in the video in the device]
On having to enter more object codes, P notes wanting access to the object code entry screen from the home of an exhibition and elsewhere---not just from the home screen
On entering an object code, P asks what to do if made a mistake on the second digit entry? (automatic submit)
On video of hockey cup: really nice, likes how it shows things you couldn't see otherwise
P: "It's not clear whether the guestbook is for the object or for the entire exhibition" [not sure how it could be for an object?]
P notes that there's no confirmation that a guestbook entry is added

III. Interview notes

Bullet numbers correspond to the interview question number.

Easy? Easy to use. The response was slow. Some of the flow didn't quite work well, for example, comments: you would tap show comments, but you weren't aware that they were below the screen [fold].
How long to understand? Pretty fast. Couple of things could be modified though---e.g., a way to get back to the section you were in a little faster (e.g., object code entry). It's almost easier to go back to home and restart the exhibition.
Problems? Not really, other than the ones mentioned. Potentially a really neat idea, really like it. Does need to be tightened up a bit, and maybe work a bit faster. The vertical vs. horizontal orientation of video was a bit irritating though (video had a black trim on the left and right sides, and showed up as small in both horizontal and vertical modes).
Improvements? Speed. An exit here and there, to get to main screens. There's a home one, but when you're at the bottom of the screen you need to scroll all the way back to the top. Some way of getting out of what you're doing, now.
Additional features? A link to the internet, especially the McCord website. A way of telling the McCord something right away, commenting directly to the staff (e.g., if you're interested in donating something, or something's wrong with one of the labels). Otherwise, seems pretty good already.
Discoveries? Yes, about the foot warmer---where it'd be used, etc. Liked the who/what/where/when extended description.
Museums & mobile? Great, think it's natural. Helpful, gives more information than what's in the label, more context of where it fit in its universe. You could probably add far more information, an extra section for really in-depth information [multi-tiered information detail... interesting]. Could add sponsorships and ads too.
Best video? Foot warmer one, because it was horizontally oriented and gave more detail in the horizontal orientation. The hockey cup one didn't fill the vertical aspect ratio.
N/A.
Overall impression? Like it, definitely. Needs improvement, but it's a good start. And so many people are carrying around these mobile devices now.
Revisit when it's done? Yeah, like to try out, see if it's faster and see what improvements you've made to it. Think it could be applied to other exhibitions too.