See Also:
(Floe) UI Options user testing protocol (September 2012)
(Floe) UI Options user testing results (November 2012)
(Floe) UI Options Design High Fidelity, C.1

Summary of Results

Design issues found in testing

Basic preferences & Advanced preferences for both versions

• Discoverability of UIO, often users would see the magnifying glass icon first as a way to increase size.
  • Is there a more appropriate icon/label for UIO? (many users looks at magnifying glass first)
  • ensure the UIO button visibility in any context (e.g. increasing it's contrast, adding some shadow, colour...)

• Type and contrast issues, users found to difficult to read preset labels that are disabled and identify the headings
  • increase contrast where necessary, avoid italics when possible, and ensure text styles are all distinct

• Table of Contents added by Simplify preset could be distracting
  • take out from initial preset (available on customize)
  • or have Table of Contents initially collapsed (user can expand if needed)

• Insufficient indication of scrollable panel
  • add multiple indicators that there is more than one page of preferences, eg. dot timeline, arrows, etc. (like in the mobile version)
  • make scrolling arrow larger

• Clarity of labels and descriptions, various issues throughout different users
  • review consistency and clarity of terms and wording
  • improve the legibility of the descriptive text (e.g. increase its contrast)
  • use descriptions to clarify function in case of complex options (e.g. in 'text to speech')
  • add more detailed instructions with a tooltips under a '?' icon

• Customize Further is not visible for most users
  • when available, display always the 'customize futher' option, not highlighted. highlight only on hover

• Search Field is not visible for some users
  • improve Search field visibility (increase contrast between the field and the background, and/or locate it at the right side of the panel)

• Some users fear loosing all their preferences when clicking on 'reset all preferences'
  • include some indication to clarify that this action can be undone
  • is this an issue?
• A user (using a magnifying glass) didn't see the 'reset all preferences' option
  • ensure that the option is visible at low vision conditions

• Some users expect to find 'type style' as one of the basic options
  • add 'type style' after 'type size'

• Options that are not directly related to the items in the page are confusing
  • control options awareness, so they are visible only if related items are available in the page (e.g., 'show captions' should appear only if the page contains video or audio sequences)
  • what if there are captions elsewhere on the site? Maybe preference curation can be determined by the implementer?

• A user expected to find some help
  • Question if some contextual helf or guidance is needed for general or specific features

Top panel specific issues

• When comparing side and top panels, all users prefer top panel

Side panel specific issues

• Shifting the page over to the side is frustrating for all users
  • would having side panel overlap content be more ideal?

• All users prefer top panel
  • should side panel be kept?
  • think about having a responsive version keeping top panel characteristics

• Users don't see the scrolling bar
  • make scrolling bar more visible

Browser extension

• Some users are not sure about Browser Extension function from the description
  • use simple and brief wording for the description
  • ensure that main features of the browser extension become apparent
  • take user to another page with more detailed descriptions and instructions about the browser extension

• In case the user doesn't see the scrolling bar, he won't never be able to get b.e. 
  • could the b.e. option be located in another place?
  • maybe not an issue if scrollbar is emphasized

• Some users are not aware of the b.e. button added in the browser bar
  • ensure the button is visible
  • after installing the button, provide a clear visual feedback (e.g. a balloon pointing to the button and providing some brief indication)
  • provide instructions from download page

• Users have difficulty navigation through b.e.
  • review features needed
  • review wording/terms for clarity
  • improve organization

• Options at the bottom (save as default, share,...) are not visible enough
  • move options at the top or closest to the other items in the b.e. panel
  • or move options to chrome icon drop-down?

• For customized themes, having general options at the top bar is confusing
  • in customized themes, get rid of general options (cmmon options, all options, most popular, search) 
  • in customized themes, include a button to add a new preference at the end of the currently available (so the user can add a new one to the previously customized)

• Theme tabs labels are not visible enough
  • increase visibility of tabs by enlarging and/or separating them from the browser's top bar.

• Some users don't know how to close the b.e. panel
  • increase b.e. button visibility
  • include a tooltip
  • include an alternative way to close the panel? (e.g., an option to collapse the panel in its bottom side and have it animate push up)

• Relationship between 'common options' and 'all options' is not clear
  • modify wording and clarify relationship between all and common options
  • question if 'common options' is needed

• In b.e. menu, options are confusing for some users
  • get rid of the 'show preferences' option? (the user can click directly the theme that he wants to edit)
  • get rid of or change the aspect of 'off'. Get rid of it if the user is able to enable/disable a theme directly from the menu. Change its styling to make it different from the list of themes available.

Mobile

• Although all users clearly see the Display Preferences button, but it's uncertain on overloaded designs with other button/images
  • ensure the d.p. button is visually detached from the page beneath
  • could the d.p. button be shown as a bar at the bottom of the page?

• Some users consider the initial instructions as repetitive
  • question if the arrow and the points at the left are enough
  • get rid of the instructions and add bouncing animation on first opening

• Some users consider the instructions as counter-intuitive
  • enhance the consistency between visual clues (instructions if kept and arrow) and the movement expected to go through options
  • ensure that the movement to go through options is consistent with the transition that makes the panel appear

• Gesture needed to go through options is not always clear, and it's not consistent between users
  • give visual clues that make more intuitive the gesture expected (e.g. the options can appear initially with an effect of displacement bottom up)
  • allow the user roll one or another sense, to go forward or backward through the options 

• Instructions about gestural alternatives are confusing for some users
  • clarify wording, or replace textual instructions by icons or simple iconic animations
  • or remove instructions not to distract user, have feature only available through accidental discovery

• Shaking the screen for changing contrast is not intuitive
  • find another gesture for this option
  • is a gesture needed for this option

• Closing the display preferences can be confusing
  • clarify the 'close' function of the d.p. button when the preferences are open (e.g. by adding a small down arrow)
  • have display preferences close when user clicks anywhere on the content screen (when they try scrolling back through the content, display preferences closes)