Orator QA Test Plan
Orator QA Test Plan
Environments
Browser | Version |
---|---|
Chrome | Latest Stable Release |
Firefox | Latest Stable Release |
MS Edge | Latest Stable Release |
Safari [fluid:1] | Latest Stable Release |
[fluid:1] keyboard a11y can be slightly improved if you select the "all controls" option from "Keyboard Shortcuts" under the "Keyboard & Mouse" settings. May also need to use "option + tab" for tab navigation.
General QA Guidelines
General Use
- Does the tool behave the way that you would expect
- Are you surprised by anything
- Does something take longer than you would expect
- When the tool does something unexpected or takes too long to do something, does the tool provide appropriate feedback
QA Tests
Protocol
Perform the following tests using each browser/system environment
Report issues at: http://issues.fluidproject.org/secure/Dashboard.jspa
Please search for issues before reporting them, so as to limit the number of duplicate entries.
Unit Test
Protocol
Launch the following website to execute unit tests.
Site
http://build.fluidproject.org/infusion/tests/component-tests/orator/html/Orator-test.html
State Tests
Description
Ensures that the component properly traverses through the various states.
Protocol
Perform these tasks on the following site, perform the tests 1-6 and 7-12 in order.
Site
Demo
Mouse
Test 1: Mouseover
- Procedure
- Open the browser and navigate to the specified URL
- Using the mouse, place the cursor over the play button
- Expected Results
- The hover styling should be applied to the play button
Test 2: Mouseover (off and on)
- Procedure
- Complete Test 1
- Move the cursor away from the play button
- Expected Results
- The hover styling should be removed from the play button
Test 3: Mouse, Activate Play button
- Procedure
- Complete Test 2
- Using the mouse, click on the play button
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the top of the page through to the end
- As words are self voiced, they are highlighted
Test 4: Mouse, Activate Pause button
- Procedure
- Complete Test 3
- Using the mouse, click on the pause button
- Expected Results
- The pause button should change to a play button
- The self voicing should stop
- The last highlighted word should remain highlighted
Test 5: Mouse, Activate Play after Pause
- Procedure
- Complete Test 4
- Using the mouse, click on the play button
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the paused location
- As words are self voiced, they are highlighted
Test 6: Mouse, Select text
- Procedure
- Complete Test 5
- Using the mouse, select a range of text
- Expected Results
- The text should be selected
- A play bubble should appear over the start of the selection
Test 7: Mouse, Play text selection
- Procedure
- Complete Test 6
- Using the mouse, click the play bubble
- Expected Results
- The play bubble should turn into a stop bubble
- The selected texts should be self voiced
- Any previous self voicing highlighting should be removed
- Previous self voicing should have stopped and been replaced with the selected text.
- If the pause button was showing, it should change to a play button
Test 8: Mouse, Stop text selection
- Procedure
- Complete Test 7
- Using the mouse, click the stop bubble
- Expected Results
- The stop bubble should turn into a play bubble
- The self voicing should stop
Test 9: Mouse, Play text selection
- Procedure
- Complete Test 8
- Using the mouse, click the play bubble
- Expected Results
- The play bubble should turn into a stop bubble
- The selected texts should be self voiced from the beginning of the selection
Test 10: Mouse, Activate Play button after selection
- Procedure
- Complete Test 9
- Using the mouse, click the play button
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the top of the page through to the end
- As words are self voiced, they are highlighted
- Any previous self voicing should have stopped
- If the stop bubble was showing, it should change to a play bubble
- Stop Test
- Refresh the browser to return the page to its initial state
- Any in progress self voicing should stop
Keyboard
Test 11: Keyboard focus
- Procedure
- Open the browser and navigate to the specified URL
- Using the keyboard, tap the 'tab' key until focus is on the play button.
- Expected Results
- The focus styling should be applied to the play button
Test 12: Keyboard focus (off and on)
- Procedure
- Complete Test 11
- Tap the 'tab' key to move focus off of the play button
- Tap the 'tab' key or use 'shift-tab' until focus is back on the play button
- Expected Results
- The focus styling should be removed when focus is lost from the play button
- The focus styling should be re-applied when focus is regained to the play button
Test 13: Keyboard, Activate Play button
- Procedure
- Complete Test 12
- Using the keyboard, tap the 'enter' or 'space' key
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the top of the page through to the end
- As words are self voiced, they are highlighted
Test 14: Keyboard, Activate Pause button
- Procedure
- Complete Test 13
- Using the keyboard, tap the 'enter' or 'space' key
- Expected Results
- The pause button should change to a play button
- The self voicing should stop
- The last highlighted word should remain highlighted
Test 15: Mouse, Activate Play after Pause
- Procedure
- Complete Test 14
- Using the keyboard, tap the 'enter' or 'space' key
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the paused location
- As words are self voiced, they are highlighted
- Stop Test
- Refresh the browser to return the page to its initial state
- Any in progress self voicing should stop
Assistive Technology (AT)
Description
Tests to ensure compatibility with Assistive Technologies.
Protocol
Perform these tasks on the following site.
Site
Demo
Test 1: AT Tests
- Procedure
- Open the browser and navigate to the specified URL
- Using the state tests as a guide, attempt to navigate through each state of the system
- Expected Results
- All states of the system should be reachable and usable while using the AT
Task Oriented Functional Tests
Description
Ensures that the component is able to handle expected input.
Protocol
Perform these tasks on the following site.
Site
Demo
tasks: play, pause, play selection, stop selection
Test 1: Play
- Procedure
- Open the browser and navigate to the specified URL
- Activate the play button
- Expected Results
- The play button should change to a pause button
- The page should be self voiced from the top of the page through to the end
- As words are self voiced, they are highlighted
- Stop Test
- Refresh the browser to return the page to its initial state
Test 2: Pause
- Procedure
- Open the browser and navigate to the specified URL
- Activate the play button
- As the self voicing is reading the content, activate the pause button
- Expected Results
- The play button should change to a pause button when self voicing starts
- The page should be self voiced from the top of the page through to the end
- As words are self voiced, they are highlighted
- After the pause button is activated, it should change to a play button
- The self voicing should stop
- The last highlighted word should remain highlighted
- Stop Test
- Refresh the browser to return the page to its initial state
Test 3: Play selection
- Procedure
- Open the browser and navigate to the specified URL
- Select a range of text
- Activate the play bubble
- Expected Results
- The text should be selected
- A play bubble should appear over the start of the selection
- After activating the play bubble, it should turn into a stop bubble
- The selected texts should be self voiced
- Stop Test
- Refresh the browser to return the page to its initial state
Test 4: Pause selection
- Procedure
- Open the browser and navigate to the specified URL
- Select a range of text
- Activate the play bubble
- As the self voicing is reading the selection, activate the stop bubble
- Expected Results
- The text should be selected
- A play bubble should appear over the start of the selection
- After activating the play bubble, it should turn into a stop bubble
- The selected texts should be self voiced
- After activating the stop bubble, it should turn into a play bubble
- The self voicing should stop
- Stop Test
- Refresh the browser to return the page to its initial state
Boundary Tests
Description
Ensures proper functionality at the input limits
Protocol
Perform these tasks on the following site.
Site
Demo
Test 1: Pause immediately
- Procedure
- Open the browser and navigate to the specified URL
- Activate the play button
- Before any self voicing starts, activate the pause button
- Activate the play button again
- Expected Results
- The state of the play button should switch between play and pause as expected
- The self voicing should start from the top of the page and read all of the expected content
- All words should be highlighted as they are self voiced
- Stop Test
- Refresh the browser to return the page to its initial state
Test 2: Self voice entire page
- Procedure
- Open the browser and navigate to the specified URL
- Activate the play button
- Let the self voicing read through all expected content
- Activate the play button again
- Expected Results
- The play button should change to a pause button when self voicing starts
- The self voicing should start from the top of the page and read all of the expected content
- All words should be highlighted as they are self voice
- After reading through all of the expected content, the pause button should turn into a play button
- After re-activating the play button, self voicing should start from the top again.
- Stop Test
- Refresh the browser to return the page to its initial state
Test-to-Fail
Description
Tests which should cause errors or not be accepted as input
Protocol
Perform these tasks on the following site.
Site
Demo
*no tests yet*
Validation
________________________________________________________
Description
Tests to ensure that specifications are being met
Protocol
Perform these tasks on the following site.
Site
Demo
Test 1: HTML Validation
- Procedure
- Open the browser and navigate to the specified URL
- For each state of the system, validate the HTML markup
- Tools for validation
- Expected Results
- All of the HTML markup should properly validate in all states.
- Stop Test
- Refresh the browser to return the page to its initial state
- Quit the tool you are using to validate
Test 2: WCAG Validation
- Procedure
- Open the browser and navigate to the specified URL
- For each state of the system (see: State Tests), ensure that the WCAG guidelines are being met to at least AA level
- Tools for validation (Note: Automated tools are not yet capable of catching all issues. It is important to go through the checklist, making use of automated tools where possible, and manual processes otherwise).
- Expected Results
- Passes WCAG 2.0 AA requirements
- Stop Test
- Refresh the browser to return the page to its initial state
- Quit the tool you are using to validate
Ad-hoc
Description
Improvised tests for quickly discovering critical issues, and uncovering ones that may be outside of formalized testing.
Protocol
Attempt to use the tool in various situations, using your imagination and freedom to explore the interface and interactions.
Site
Demo