(Floe) UI Options user testing results (March 2012)
This page gathers the results of UI Options user testing, performed in Toronto (Canada) and Barcelona (Spain) in March 2012.
To complete this information, see also:
- UI options user testing protocol (February 2012)
- UI Options user testing report (April 2012)
Barcelona, Spain
Barcelona, user 1
Demographics profile
- Spanish-speaking female retired accountant, 65+
- Comfortable with technology
- Reads on the screen daily
- Always uses glasses
- With temporary mobility problems, from broken arm
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
Yes, when the text is too small and the lines appear too tight.
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
Yes, I change the screen resolution (from Windows Settings)
Is there anything that you’d like to be able to change on websites, but can't?
I'd like to have bigger text without loosing the margins of the page (refers to the fact that when the screen resolutions is lower, not only the text appears bigger but also the whole page, so she sometimes has to use the horizontal scroll)
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
She starts by reading carefully the page (the introduction of the example page contains instructions) to know what to do. After about 5 seconds thinking, she clicks on the "Display Preferences (with preview)" button.
She plays with the text size and line spacing options, and comments "I'd like to have higher contrast".
She carefully reads the options one by one and then discovers the color and contrast: "Better black on white".
Then she reads "Show table of contents" and comments: "Don't need it". For all the three last options, reads carefully the instructions below the option.
Then reads "Emphasize links" and selects it: "Yes, because I usually don't notice the links". Doesn't select the "Make inputs larger".
When asked about returning to the page with the changes done, she doubts about what to do. Reads the options and doubts about their meaning: "Says "Save an Apply. What does this mean?". When asked about her interpretation, explains correctly the function of this option.
She clicks on "Save and Apply", but doesn't know how to return to the site. Clicks on the browser's left arrow (doesn't see the "Return to site" option, maybe because it isn't much visible in black & white).
After ending the undirected testing, the moderator asks about the "Show display preferences" option at the top of the page (the expandable panel). She hadn't noticed it before. When she expands it, comments: "Oh, I like much more this. I can see the page at the same time that I change things".
When asked about returning to the original design, immediately clicks on the "Reset all" options (in the top panel).
When asked about closing the options, immediately clicks on "hide".
Post-test interview
How would you describe your experience using UI Options overall?
Very pleasant. All web pages should have something like this.
What did you find confusing or frustrating using UI Options?
Nothing, only that I havent noticed the top option (the expandable panel)
Would you add any option or functionality?
None.
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, and I prefer the top options (refers to the expandable panel) than the options in another page.
What might we do to improve the application?
Make the top option more visible.
Barcelona, user 2
Demographics profile
- Spanish-speaking male retired administrative, 65+
- Comfortable with technology
- Reads documents on the screen daily
- Always uses glasses
- No motor problems
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
I don't get used to reading on the screen, I prefer printing the contents and read them on paper. It's easier.
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
When the text is brief I read it directly on the screen. Then I expand it with the "Ctrl+" option.
Is there anything that you’d like to be able to change on websites, but can't?
(things for some seconds) No.
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
He starts by reading carefully the page (the introduction of the example page contains instructions) to know what to do. He spends a lot of time thinking. Then says "I don't know what to do". When asked by the moderator about if he can imagine some way to change the appearance of the page, he points to the "Show display preferences" option at the top of the page (the expandable panel) and says "Let's see what says here" and clicks on it. He looks quite confused.
When the panel expands, he starts playing with the text options. He understands their function clearly.
After playing with the text options, he stops. Doesn't see the other tabs. When asked if he things that there are more options to configure, he doubts and see the tabs. Doubts and clicks on "Links and buttons".
Reads "Emphasize links" and says "I don't know what is the meaning of "links" ".
He says that doesn't understand the meaning of "Make inputs larger" (because of "inputs" word).
Then sees the "Layout and navigation" tab and clicks on it. Doesn't understand the meaning of "Table of contents". The moderator asks him to click on it and when he sees the result, comments: "It highlights the sections in the contents, isn't it?". Asked about if it would be useful for him, says: "Well, you can go more directly to the header", but doesn't look very convinced.
When asked about closing the options, clicks on "Reset all" and loses all the changes.
Post-test interview
How would you describe your experience using UI Options overall?
Seems practical, interesting (doesn't look really convinced)
What did you find confusing or frustrating using UI Options?
Nothing.
Would you add any option or functionality?
None.
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, it's comfortable. But I prefer the options in another page (after ending the test, the moderator guides him to the "Display preferences" options, to let him know the other possibilities).
What might we do to improve the application?
It's ok.
Barcelona, user 3
Demographics profile
- Spanish-speaking female designer, 18-25
- Very comfortable with technology
- Reads documents on the screen daily
- Doesn't use glasses or other custom devices
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
Yes, when the columns in the page are too wide and the text is small, it's really unpleasant.
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
When the columns are too wide, I use to zoom out the screen to make it smaller and read more easily. Also, the screen brightness wears me out.
Is there anything that you’d like to be able to change on websites, but can't?
Maybe being able to change the page layout (selecting the documents columns), but I had never raised before.
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
She starts by looking at the whole page and comments "I don't know what to do..." Continues scanning and after some seconds sees the options: "Oh, I was looking at the center of the page, where the content is, and didn't see the options that appear at the margins" (uses high resolution, so the options appear away from the contents).
Clicks on "Display Preferences with preview". She starts playing with the options. When reaches "Color & Contrast" says: "This is interesting, because of the tiring brightness of the screen. Also, in this design the text has low contrast with the background". Selects "White on black".
Keeps playing and trying the options. Comments: "The preview window is too small".
Reads the "Show table contents" options and its instructions and looks confused. Selects this options but the preview window doesn't show anything new. She scrolls the preview until she sees the table and asks: "This seems to be the sitemap, isn't it?" She's not really sure.
Selects "Emphasize links" and says: "This can be really useful". After selecting "Make inputs larger" says: "Well, I don't see much difference between the original and the larger inputs". In both cases, she's read the instructions below the options.
When she ends playing with the options, she clicks directly on the "Return to site" option, loosing all the changes. She looks surprised, because she thought that changes would remain.
After ending the undirected testing, the moderator asks her about the "Display Preferences without preview" option. She comments: "I don't think this can be useful, without preview". Once in this screen, she's asked about returning to the site maintaining the changes. Doubts for some seconds, but finally clicks on "Save and Apply".
Back on the page, she discovers the "Show display preferences" option at the top of the page (the expandable panel): "- Oh, is this a shortcut?" She sees that it contain all the options and can see the changes directly on the page: "I like much more this option, because I can see all the changes and you don't have to use the preview window". "-Didn't you see this option before?", "- Yes, I had seen it but I clicked first on these (the options at the margin) because they are much more visible, with the blue background and the arrows".
Post-test interview
How would you describe your experience using UI Options overall?
I like it. I had seen something similar in typographic websites (refers to foundries where the user can change the text setting to check the fonts before purchasing them), but this has a different application, can be very useful.
What did you find confusing or frustrating using UI Options?
Nothing. Only that when you select another Color & Contrast, the change is too drastic.
Would you add any option or functionality?
None.
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, I think that the top option (the expandable panel) is the most comfortable.
What might we do to improve the application?
In the "Layout & Navigation" tab, I thought that I would be able to change the columns or something in the page design. The word "layout" and the icon led me to think that.
The option at the top could automatically drag when you scroll the page, to keep it visible all the time.
The preview window is too small.
Barcelona, user 4
Demographics profile
- Spanish-speaking female audiovisual technician, 26-35
- Very comfortable with technology
- Reads documents on the screen daily, but if she has a chance, prefers to print them.
- Doesn't use glasses or other custom devices
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
No.
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
No.
Is there anything that you’d like to be able to change on websites, but can't?
No, just an option to add a searcher to search a word in the current page.
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
She starts by looking at the whole page, and immediately sees the options. Clicks on the "Show display preferences" option at the top of the page (the expandable panel). Plays with the options. "This would be great for my grandparents".
When sees the option to change Colour & Contrast, comments: "Yes, in some web pages, the text can not be read because of the background color". She appreciates having this option.
After exploring all the options in Text and Display, she clicks on "Hide".
The moderator asks her about the "Display Preferences" options at the right. Clicks on "with preview" and comments: "I prefer the other one" (the expandable panel).
In the "DP with preview" screen, she notices the Layout and Navigation and Links and Buttons options (she had not noticed them in the top panel). She understands well the meaning of "Show Table of Contents". Doesn't see clearly what happens with the options of "Links and Buttons".
She clicks on "Save and Apply" and looks confused: she expected to be returned automatically to the site. It takes some seconds to see the "Return to site" option.
When back to the site, she notices that the appearance of links and inputs has changed, and remembers the options.
The moderator asks her about returning to the original design, and she clicks on the top option. She's unable to see the "Reset all" option because the page is a bit scrolled, and she doesn't realize that there are some items above the upper edge. Then she selects the "DP with preview" option, and clicks directly in "Reset".
Post-test interview
How would you describe your experience using UI Options overall?
It can be useful.
What did you find confusing or frustrating using UI Options?
Nothing, only that I couldn't find the option to return to site.
Would you add any option or functionality?
A searcher to search words inside the page.
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, I prefer the top option (the panel).
What might we do to improve the application?
Maybe adding more Text Styles.
Barcelona, user 5
Demographics profile
- Spanish-speaking male management technician, 36-45
- Comfortable with technology
- Reads documents on the screen daily
- Uses glasses always, and is colorblind.
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
Yes, sometimes the color combination (I'm colorblind), and pages where the contrast makes reading difficult... like this one (points to the page used in the test as sample).
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
No, rarely I make zoom in using the keyboard options.
Is there anything that you’d like to be able to change on websites, but can't?
No.
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
He starts looking at the page, and comments that the text has not enough contrast with the background, and the paragraphs text is too small.
When asked about customizing it, he clicks on the "DP with preview" option (at the right). Plays with the options.
After going through all the options, clicks on "Save and Apply". She expected to be returned to the page (looks confused). Asks himself: "Maybe Reset?" Finally, he goes back using the browser option.
Now he clicks on "DP without preview". He prefers this option ("I can check the changes later"), but if he were working to print, he'd prefer the "with preview" ("To be sure of what I have before printing").
Clicks on "Save and Apply" and waits. Comments: "It doesn't apply?". He's confused for not seeing the results in the page.
Finally he sees the "Return to site" option, but it seems strange to him to have to explicitly select this option to go back to the page.
The moderator asks him about the top option (expandable panel). Then he discovers this option (he didn't realize before), and comments that he prefers this than any other, because you can find the exact configuration.
When asked about returning to the original design, he closes the top panel (doesn't see the "Reset all" option), clicks on "DP with preview" and clicks on "Reset".
Post-test interview
How would you describe your experience using UI Options overall?
Really useful, we need this because not everybody is standard.
What did you find confusing or frustrating using UI Options?
I haven't noticed the top option (to show the panel) until you have asked me.
Would you add any option or functionality?
It's ok... maybe that the user can hide images. Sometimes the page is full of images, and it's annoying to read.
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, I prefer the top option (the panel).
What might we do to improve the application?
Nothing.
Barcelona, user 6
Demographics profile
- Spanish-speaking male 3D project manager, 36-45
- Very comfortable with technology
- Reads documents on the screen daily
- Uses glasses always.
Pre-test interview
When you're using the web, do you ever find webpages that are difficult to read or use? If so, in what way?
No, just the ones that display ads in the middle of the screen (the ones that overlay the contents).
Do you ever have to fiddle with options or settings to make webpages easier to read or use? If so, which ones?
Just when I read Ars Technica (http://arstechnica.com/), I customize color and font size (using the Customize options in the navigation bar).
Is there anything that you’d like to be able to change on websites, but can't?
No.
User testing
The user has performed the undirected testing. Only at the end, the moderator has asked about performing some directed tasks.
He starts looking at the page, and immediately clicks on the top option (the expandable panel). Plays with the options and clicks on the tabs. He has rapidly explored all the options. He understands clearly the meaning of each option, and the results in the page confirm his inferences.
He finally clicks on the "Hide" option to leave the full page visible.
When asked about returning to the original page, shows again the display preferences (top panel) and clicks directly on "Reset all".
When the moderator asks him to take a look at the right options, he clicks and examines the configuration pages. Finally comments: "It's the same in the three places, but I prefer the first one (top panel). I don't see the sense of the "without preview" option".
Post-test interview
How would you describe your experience using UI Options overall?
Easy, good.
What did you find confusing or frustrating using UI Options?
No
Would you add any option or functionality?
No
Do you think that the integration of UI Options with the document is comfortable enough?
Yes, I prefer the top one.
What might we do to improve the application?
Maybe the user should be able to sort or gather his more used options, or maybe customize his own shortcuts.