User Interface Options QA Test Plan - Full No Preview

User Interface Options QA Test Plan - Full: No Preview

Environments

BrowserVersion
ChromeLatest Stable Release
FirefoxLatest Stable Release
MS EdgeLatest Stable Release
Safari [fluid:1]Latest Stable Release

[fluid:1] keyboard a11y can be slightly improved if you select the "all controls" option from "Keyboard Shortcuts" under the "Keyboard & Mouse" settings. May also need to use "option + tab" for tab navigation.

General QA Guidelines

General Use

  • Does the tool behave the way that you would expect
  • Are you surprised by anything
  • Does something take longer than you would expect
  • When the tool does something unexpected or takes too long to do something, does the tool provide appropriate feedback

QA Tests


ProtocolPerform the following tests using each browser/system environment

Report issues at: http://issues.fluidproject.org/secure/Dashboard.jspa

Please search for issues before reporting them, so as to limit the number of duplicate entries.

FSS Jira Filter
UI Options Jira Filter


Unit Test

Protocol

 Launch the following websites to execute unit tests.

Site
http://build.fluidproject.org/infusion/tests/framework-tests/preferences/all-tests.html

State Tests

Description

 Ensures that the component properly traverses through the various states. 

Protocol

 Perform these tasks on the following site, perform the tests 1-10 and 11-20 in order.

Site

Mouse

Test 1: Mouse, Change Text Style

  • Procedure
    1. Using the mouse, under the "Text and Display" header, select a different text style
    2. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The text on the page should change to the selected style

Test 2: Mouse, Change Text Size (slider)

  • Procedure
    1. Complete Test 1
    2. Using the mouse, under the "Text and Display" header, move the slider to select a different text size
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The value in the textfield should change to show the value represented by the slider
    • The text on the page should change to the selected size

Test 3: Mouse, Change Text Size (textfield)

  • Procedure
    1. Complete Test 2
    2. Using the mouse, under the "Text and Display" header, type a new value in the textfield, to select a different text size
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The position of the thumb, on the slider, should change to represent the value entered in the textfield
    • The text on the page should change to the selected size

Test 4: Mouse, Change Line Spacing (slider)

  • Procedure
    1. Complete Test 3
    2. Using the mouse, under the "Text and Display" header, move the slider to change the line spacing
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The value in the textfield should change to show the value represented by the slider
    • The page should change to have the selected line spacing

Test 5: Mouse, Change Line Spacing (textfield)

  • Procedure
    1. Complete Test 6
    2. Using the mouse, under the "Text and Display" header, type a new value in the textfield to change the line spacing
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The position of the thumb, on the slider, should change to represent the value entered in the textfield
    • The page should change to have the selected line spacing

Test 6: Mouse, Change Contrast

  • Procedure
    1. Complete Test 5
    2. Using the mouse, under the "Text and Display" header, select a different contrast theme
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The contrast of the content on the page should change to the selected theme

Test 7: Mouse, Toggle Table of Contents

  • Procedure
    1. Complete Test 5
    2. Using the mouse, under the "Layout and Navigation" header, toggle the "Show Table of Contents" setting
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • No table of contents should be displayed

Test 8: Mouse, Emphasize Links

  • Procedure
    1. Complete Test 7
    2. Using the mouse, under the "Links and Buttons" tab, toggle the "underline and bold" setting
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The display of links, on the page, should match the selected setting

Test 9: Mouse, Make Inputs Larger

  • Procedure
    1. Complete Test 8
    2. Using the mouse, under the "Links and Button" header, toggle the "enlarge buttons, menus, text-fields, and other inputs" setting
    3. Using the mouse, click the "Save and Apply" button
  • Expected Results
    • The display of inputs, on the page, should match the selected setting

Test 10: Mouse, Reset

  • Procedure
    1. Complete Test 9
    2. Using the mouse, click the "Reset and Apply" button
  • Expected Results
    • The options should all be set to the initial default values
    • The page should be reset to the initial default styles
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Keyboard

Test 11: Keyboard, Change Text Style

  • Procedure
    1. Using the keyboard, under the "Text and Display" header, select a different text style
    2. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    3. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The text on the page should change to the selected style

Test 12: Keyboard, Change Text Size (slider)

  • Procedure
    1. Complete Test 11
    2. Using the keyboard, under the "Text and Display" header, tap the "tab" key until the thumb of the slider has focus
    3. Using the keyboard, tap the left and right arrow keys to select a different text size
    4. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    5. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The value in the textfield should change to show the value represented by the slider
    • The text on the page should change to the selected size

Test 13: Keyboard, Change Text Size (textfield)

  • Procedure
    1. Complete Test 12
    2. Using the keyboard, under the "Text and Display" header, tap the "tab" key until the textfield has focus
    3. Using the keyboard, type a new value in the textfield, to select a different text size
    4. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    5. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The position of the thumb, on the slider, should change to represent the value entered in the textfield
    • The text on the page should change to the selected size

Test 14: Keyboard, Change Line Spacing (slider)

  • Procedure
    1. Complete Test 13
    2. Using the keyboard, under the "Text and Display" header, tap the "tab" key until the thumb of the slider has focus
    3. Using the keyboard, tap the left and right arrow keys to select a different line spacing
    4. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    5. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The value in the textfield should change to show the value represented by the slider
    • The page should change to have the selected line spacing

Test 15: Keyboard, Change Line Spacing (textfield)

  • Procedure
    1. Complete Test 14
    2. Using the keyboard, under the "Text and Display" header, tap the "tab" key until the textfield has focus
    3. Using the keyboard, type a new value in the textfield, to select a different line spacing
    4. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    5. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The position of the thumb, on the slider, should change to represent the value entered in the textfield
    • The page should change to have the selected line spacing

Test 16: Keyboard, Change Contrast

  • Procedure
    1. Complete Test 15
    2. Using the keyboard, under the "Text and Display" header, select a different contrast theme
    3. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    4. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The contrast of the content on the page should change to the selected theme

Test 17: Keyboard, Toggle Table of Contents

  • Procedure
    1. Complete Test 16
    2. Using the keyboard, under the "Layout and Navigation" header, toggle the "Show Table of Contents" setting
    3. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    4. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • No table of contents should be displayed

Test 18: Keyboard, Emphasize Links

  • Procedure
    1. Complete Test 17
    2. Using the keyboard, under the "Links and Buttons" header, toggle the "underline and bold" setting
    3. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    4. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The display of links, on the page, should match the selected setting

Test 19: Keyboard, Make Inputs Larger

  • Procedure
    1. Complete Test 18
    2. Using the keyboard, under the "Links and Navigation" header, toggle the "enlarge buttons, menus, text-fields, and other inputs" setting
    3. Using the keyboard, tap the 'tab' key until the "Save and Apply" button has focus
    4. Using the keyboard, tap the "enter" key to save the changes
  • Expected Results
    • The display of inputs, on the page, should match the selected setting

Test 20: Keyboard, Reset

  • Procedure
    1. Complete Test 19
    2. Using the keyboard, tap the 'tab' key until the "Reset and Apply" button has focus
    3. Using the keyboard, tap the 'enter' key reset the settings
  • Expected Results
    • The options should all be set to the initial default values
    • The page should be reset to the initial default styles
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Assistive Technology (AT)

Description
Tests to ensure compatibility with Assistive Technologies.

Protocol
Perform these tasks on the following site.

Site

Test 1: AT Tests

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. Using the state tests as a guide, attempt to navigate through each state of the system
      1. Example ATs
        1. Screen Readers: JAWSNVDAVoiceOverOrca
        2. Built in AT features: WindowsMacLinuxiOSAndroid
        3. Others ATs: Speech Recognition, Screen Magnifiers, switch access, etc.
  • Expected Results
    • All states of the system should be reachable and usable while using the AT

Task Oriented Functional Tests

Description

 Ensures that the component is able to handle expected input.

Protocol

 Perform these tasks on the following site.

Site

Test 1: Change the Text Style

  • Procedure
    1. Under the "Text and Display" header, change the text style
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The text on the page should change to the selected style
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 2: Change the Text Size

  • Procedure
    1. Under the "Text and Display" header, change the text size
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The text on the page should change to the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 3: Change the Line Spacing

  • Procedure
    1. Under the "Text and Display" header, change the line spacing
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The page should change to have the selected line spacing
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 4: Change the Contrast Theme

  • Procedure
    1. Under the "Text and Display" header, change the contrast theme
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The contrast of the content on thepage should change to the selected theme
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 5: Emphasize Links

  • Procedure
    1. Under the "Links and Buttons" header, toggle the "underline and bold" setting
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The display of links should match the selected setting
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 6: Make Inputs Larger

  • Procedure
    1. Under the "Links and Navigation" header, toggle the "enlarge buttons, menus, text-fields, and other inputs" setting
    2. Trigger the "Save and Apply" button
  • Expected Results
    • The display of inputs should match the selected setting
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 7: Toggle Table of Contents

  • Procedure
    1. Under the "Layout and Navigation" header, toggle the "Show Table of Contents" setting
    2. Trigger the "Save and Apply" button
  • Expected Results
    • No table of contents should be displayed
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 8: Reset Changes

  • Procedure
    1. Change some of the settings
    2. Trigger the "Reset and Apply" button
  • Expected Results
    • The options should all be set to the initial default values
    • The page should be reset to the initial default styles
  • Stop Test
    • Reset the browser

Test 9: Persistence via Cookie

  • Procedure
    1. Change some of the settings
    2. Trigger the "Save and Apply" button
    3. Trigger the "Return To Site" button
  • Expected Results
    • The demo page should show the options in effect
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Boundary Tests

Description

 Ensures proper functionality at the input limits 

Protocol

 Perform these tasks on the following site.

Site


Test 1: Minimum Text Size

  • Procedure
    1. Under the "Text and Display" header, change the text size to the smallest value by moving the slider to the far left
  • Expected Results
    • After saving, the text should appear with the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 2: Maximum Text Size

  • Procedure
    1. Under the "Text and Display" header, change the text to the largest size by moving the slider to the far right
  • Expected Results
    • After saving, the text should appear with the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 3: Minimum Line Spacing

  • Procedure
    1. Under the "Text and Display" header, change the line spacing to the smallest value by moving the slider to the far left
  • Expected Results
    • After saving, the line spacing on the page should appear with the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 4: Maximum Line Spacing

  • Procedure
    1. Under the "Text and Display" header, change the line spacing to the largest size by moving the slider to the far right
  • Expected Results
    • After saving, the line spacing on the page should appear with the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 5: Table of Contents On

  • Procedure
    1. Under the "Layout and Navigation" header, select "Show Table of Contents"
  • Expected Results
    • No table of contents should be displayed
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 6: Table of Contents Off

  • Procedure
    1. Under the "Layout and Navigation" header, deselect "Show Table of Contents"
  • Expected Results
    • No table of contents should be displayed
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 7: Emphasize Links On

  • Procedure
    1. Under the "Links and Buttons" header, select "underline and bold"
  • Expected Results
    • The links should be emphasized
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 8: Emphasize Links Off

  • Procedure
    1. Under the "Links and Buttons" header, deselect "underline and bold"
  • Expected Results
    • The links should not be emphasized
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 9: Make Input Larger On

  • Procedure
    1. Under the "Links and Buttons" header, select "enlarge buttons, menus, text-fields, and other inputs"
  • Expected Results
    • The inputs should appear larger
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 10: Make Input Larger Off

  • Procedure
    1. Under the "Links and Buttons" header, deselect "enlarge buttons, menus, text-fields, and other inputs"
  • Expected Results
    • The inputs should not appear larger
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 16: No Preferences Changed

  • Procedure
    1. Trigger the "Save and Apply" button without having made any changes
  • Expected Results
    • The page should appear as they initially did on page load.
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 17: All Preferences Changed

  • Procedure
    1. Change all of the options
  • Expected Results
    • The page should appear with all of the style changes
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 18: Cookies Turned Off

  • Procedure
    1. Turn off cookies in the browser
    2. Change some of the settings
    3. Trigger the "Save and Apply" button
    4. Refresh the browser
  • Expected Results
    • The page should show the original default styles
  • Stop Test
    • Refresh the page and turn cookies back on


Test-to-Fail

Description

 Tests which should cause errors or not be accepted as input 

Protocol

 Perform these tasks on the following site.

Site

Test 1: Enter a text size larger than the maximum

  • Procedure
    1. Under the "Text and Display" tab, type in a text size that is larger than the maximum text size
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the maximum allowable text size
  • Stop Test
    • Refresh the browser or reset the changes

Test 2: Enter a text size smaller than the minimum

  • Procedure
    1. Under the "Text and Display" tab, type in a text size that is smaller than the minimum text size
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the minimum allowable text size
  • Stop Test
    • Refresh the browser or reset the changes

Test 3: Enter a line spacing larger than the maximum

  • Procedure
    1. Under the "Text and Display" tab, type in a line spacing that is larger than the maximum line spacing
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the maximum allowable line spacing
  • Stop Test
    • Refresh the browser or reset the changes

Test 4: Enter a line spacing smaller than the minimum

  • Procedure
    1. Under the "Text and Display" tab, type in a line spacing that is smaller than the minimum line spacing
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the minimum allowable line spacing
  • Stop Test
    • Refresh the browser or reset the changes

Test 5: Enter non-numeric characters for the text size

  • Procedure
    1. Under the "Text and Display" tab, in the text size textfield enter non-numeric characters
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change back to the last valid value
  • Stop Test
    • Refresh the browser or reset the changes

Test 6: Enter non-numeric characters for the line spacing

  • Procedure
    1. Under the "Text and Display" tab, in the line spacing textfield enter non-numeric characters
    2. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change back to the last valid value
  • Stop Test
    • Refresh the browser or reset the changes

Validation

Description
Tests to ensure that specifications are being met

Protocol
Perform these tasks on the following site.

Site

Test 1: HTML Validation

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. For each state of the system, validate the HTML markup
      1. Tools for validation
        1. Nu HTML Checker Bookmarklet
        2. Validity Chrome Extension
        3. Lighthouse Dev Tool
  • Expected Results
    • All of the HTML markup should properly validate in all states.
  • Stop Test
    • Refresh the browser to return the page to its initial state
    • Quit the tool you are using to validate

Test 2: WCAG Validation

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. For each state of the system (see: State Tests), ensure that the WCAG guidelines are being met to at least AA level
      1. Tools for validation (Note: Automated tools are not yet capable of catching all issues. It is important to go through the checklist, making use of automated tools where possible, and manual processes otherwise).
        1. WCAG AODA accessibility audit checklist
        2. tota11y Bookmarklet
        3. Lighthouse Dev Tool
  • Expected Results
    • Passes WCAG 2.0 AA requirements
  • Stop Test
    • Refresh the browser to return the page to its initial state
    • Quit the tool you are using to validate

Ad-hoc

Description
Improvised tests for quickly discovering critical issues, and uncovering ones that may be outside of formalized testing.

Protocol
Attempt to use the tool in various situations, using your imagination and freedom to explore the interface and interactions.

Site