0.9 QA Plan
https://develop-0-9-1--c2lc-build.netlify.app/
Test 1: Initial Load
Load the Coding Environment
Expect:
The page loads without any errors
Test 2: Change the character’s position
Setup: Program area and scene are empty, character is at position A1
Input the alphanumeric co-ordinates (“E”, “4”) into the alphanumeric grid input
Expect:
The character appears at position E4 without drawing any lines
Input the invalid alphanumeric co-ordinates (“10”, “8”) into the alphanumeric grid input
Expect:
The character appears at position E8 without drawing any lines
Input the invalid alphanumeric co-ordinates (“$”, “10”) into the alphanumeric grid input
Expect:
The character stays at position E8
Input the invalid alphanumeric co-ordinates (“H”, “-10”) into the alphanumeric grid input
Expect:
The character appears at position H8 without drawing any lines
Input the invalid alphanumeric co-ordinates (“J”, “C”) into the alphanumeric grid input
Expect:
The character appears at position J8 without drawing any lines
Input the invalid alphanumeric co-ordinates (“L”, “#”) into the alphanumeric grid input
Expect:
The character appears at position L8 without drawing any lines
Input the invalid alphanumeric co-ordinates (“E”, “100”) into the alphanumeric grid input
Expect:
The character appears at position E8 without drawing any lines
Input the co-ordinates (“A”, “1”) into the character positioning fields
Expect:
The character appears at position A1 without drawing any lines
Test 3: Change the character’s position using directional movement buttons
Setup: Program area and scene are empty, character is at position A1
Activate the “Move Right” button three times
Expect:
The character moves right three spaces to D1 on the grid without drawing any lines
- The values in the character positioning fields are updated appropriately each time the 'Move Right' is activated
Activate the “Move Down” button two times
Expect:
The character moves down two spaces to D3 on the grid without drawing any lines
- The values in the character positioning fields are updated appropriately each time the 'Move Down' is activated
Activate the “Move Left” button two times
Expect:
The character moves left two spaces to B3 on the grid without drawing any lines
- The values in the character positioning fields are updated appropriately each time the 'Move Left' is activated
Activate the “Move Up” button one time
Expect:
The character moves up one space to B2 on the grid without drawing any lines
- The values in the character positioning fields are updated appropriately each time the 'Move Up' is activated
Activate the “Move left” button 3 times.
Expect:
The character travels to the left edge of the canvas (A2) and stops there.
- The values in the character positioning fields are updated appropriately each time the 'Move Left' is activated
Activate the “Move up” button 3 times.
Expect:
The character travels to the top corner (A1) of the canvas and stops there.
- The values in the character positioning fields are updated appropriately each time the 'Move Up' is activated
Activate the “Move Right” button 12 times.
Expect:
The character travels to the right edge of the canvas (L1) and stops there.
- The values in the character positioning fields are updated appropriately each time the 'Move Right' is activated
Activate the “Move Down” button 9 times.
Expect:
The character travels to the bottom corner of the canvas (L8) and stops there.
- The values in the character positioning fields are updated appropriately each time the 'Move Down' is activated
Use the character positioning buttons to move the character to A1
Test 4: Change the character’s rotation using rotation buttons
Setup: Program area and scene are empty, character is at position A1
Activate the “Turn left” button one time
Expect:
The character turns left 45 degrees
Activate the “Turn right” button one time
Expect:
The character turns right 45 degrees back to its original direction
Activate the "Turn Left" button 8 times
Expect:
The character completes a full 360-degree rotation left returning to its original direction
Activate the "Turn Right" button 8 times
Expect:
The character completes a full 360-degree rotation right returning to its original direction
Test 5: Simple program using '+' to add
Setup: Program area and scene are empty, character is at position A1
Select the “Move forward one square” command block
Expect:
The block appears selected
"Movement forward one square selected" is read out
Activate the '+' in the program area
Expect:
The block and a '+' are added to the program
- The block added to the program is selected
"Added movement forward one square" is read out
- Repeat step 2
Select the “Turn right 45 degrees” command block
Expect:
The block appears selected
"Movement turn right 45 degrees selected" is read out
Activate the '+' between the two blocks in the program
Expect:
The block and a '+' are added to the middle of the program
- The block added to the program is selected
"Added movement turn right 45 degrees" is read out
Activate the '+' at the beginning of the program
Expect:
The block and a '+' are added to the beginning of the program
- The block added to the program is selected
"Added movement turn right 45 degrees" is read out
Deselect the 'turn right 45 degrees' action
Expect:
The block does not appear selected
- "No movement selected" is read out
Activate the '+' at the beginning of the program
Expect:
The '+' appears selected but no block is added to the program
- There are a total of 4 action tiles in the program
Test 6: Run program
Setup:
- Test 5 above has been completed
- There is a short program in the program area
- The scene is empty
- Character is at position A1
Activate the play button
Expect:
The commands in the program area are performed by the character, drawing a line as it moves
The play button switches to a pause button while the program is running
The character produces sounds while the program is running
The stop button is available while the program is running
The command being performed by the character is highlighted while the program is running
Test 7: Refresh scene
Setup:
- Test 6 above has been completed
- There is a drawing on the scene
- Character is at position B3 facing down
Activate the 'Refresh Scene' button
Expect:
The character is moved to the A1 square
Any lines that were present on the canvas have been removed
Test 8: Delete program
Setup:
- Test 7 above has been completed
- There is a short program in the program area
Activate the “Delete” button in the program panel
Expect:
The “Delete all” dialog appears on the screen
"Delete all movements" is read out
Activate the “Cancel” button in the dialog
Expect:
- The dialog closes and the program remains the same as it was at the beginning of this test
- Activate the “Delete” button and activate the "Delete All" in the dialog
Expect:- The dialog closes and all commands in the program are removed
Test 9: Program using drag & drop
Setup: Program area and scene are empty, character is at position A1
Select the “Move forward three squares” command block and drop it in the program area
Expect:
The block is added to the program
The block appears selected both in the program area and the movements panel
- "Movement forward three squares selected" and "Added movement forward three squares" is read out
Repeat step 1 four times
Expect:
There are 5 "Move forward three squares" blocks in the program
"Added movement forward three squares" is read out after each addition
Select the "Turn right 90 degrees" block and drop it to between the third and fourth program blocks in the program area
Expect:
The block is inserted into the program
- "Movement turn right 90 degrees selected" and "Added movement turn right 90 degrees" is read out
Select the "Move forward two squares" block and drop it after the "Turn right"
Expect:
The block is inserted into the program
- "Movement forward two squares selected" and "Added movement forward two squares" is read out
Select the "Move forward two squares" block and drop it before the "Turn right"
Expect:
The block is inserted into the program
- "Added movement forward two squares" is read out
Scroll to the end of the program and add a 'Turn right 90 degrees' at the end
Expect:
The block is inserted into the program
- "Added movement turn right 90 degrees" is read out
Run the program
Expect:
The character moves along the top of the scene to the top right corner, turns right and moves to the bottom right corner and turns right again
- The play button switches to a pause button while the program is running
- The character produces sounds while the program is running
- The stop button is available while the program is running
- The command being performed by the character is highlighted while the program is running
Test 10: Remove part of the program
Setup:
- Test 9 (Program using drag & drop) above has been completed
- There is a program in the program area
- Select the first block in the program
Expect:- The control panel for the block is shown
- Activate the “delete” option in the control panel.
Expect:- The block is removed from the program
- "Deleted movement forward 3 squares" is read out
- Run the program
Expect:- The updated program runs as expected
- The character is at D1 facing its original direction
Test 11: Change playback speed
Setup:
- There is a program in the program area
- the current playback speed is set to the default playback speed (middle of slider)
- Decrease the current playback speed one setting using the slider
- Run the program
Expect:- the program runs at a slower speed than default.
- Decrease the current playback speed to the minimum using the slider.
- Run the program
Expect:- the program runs at a slower speed than in step 2.
- Attempt to decrease the current playback speed further than the minimum
Expect:- the slider remains at the minimum speed
- Run the program
Expect:- the program runs at the same speed as in step 4.
- Increase the playback speed to one setting above default using the slider.
- Run the program
Expect:- the program runs at a faster speed than default.
- Increase the current playback speed to the maximum using the slider.
- Run the program
Expect:- the program runs at a faster speed than in step 8
- Attempt to increase the current playback speed further than the maximum using the slider.
- Run the program
Expect:- the program runs at the same speed as in step 10
- Run the program and change the playback speed while it is running
Expect:- the program speed changes during playback
Test 12: Character tries to move beyond scene edge
- Load this program https://develop-0-9--c2lc-build.netlify.app/?v=0.9&t=mixed&w=Sketchpad&p=6B3A26666d3A2b2&c=aab&a=123456ABDabd
Expect:- the character is in position A1
- there is a program in the program area
- Run the program
Expect:- the character tries to move beyond the edge of the scene several times but is unable to
- the drawing is a line from A1 to H8 to L8
- the character is facing down
- Refresh the scene and delete the program
Test 13: All commands
Setup: Program area and scene are empty, character is at position A1
Create a program that uses all of the possible actions available
Run the program
Expect:
The character moves or turns as expected based on the action being run
- Each distinct movement or turn has its own unique sound that plays while it is happening
- Refresh the scene
Test 14: Stop program
Setup:
- There is a program in the program area
- The scene is empty
- The character is in position A1
- Attempt to stop the program using the stop button while the program is not running
Expect:- Nothing happens
- Activate the play button and during program playback, activate the stop button
Expect:- Program playback is stopped
- No command in the program is highlighted after the program stops
- The character remains where it was when the program was stopped and all lines drawn by the character remain on the canvas
- Run the program
Expect:- The program playback starts from the beginning.
- Refresh the scene
Test 15: Pause program
Setup:
- There is a program in the program area
- The scene is empty
- The character is in position A1
- Activate the play button
- During program playback, activate the pause button (The play button should switch to a pause button for the duration of program playback)
Expect:- A command remains highlighted to indicate the current position in program playback
- The pause button switches to a play button
- Run the program
Expect:- The program continues from where it left off and finishes
- Refresh the scene
Test 16: Toggle line drawing
Setup:
- There is a program in the program area
- The scene is empty
- The character is in position A1
- Toggle the “Draw Line” switch off
- Run the program
Expect:- The character moves and turns as expected
- No lines are drawn
- Toggle the “Draw Line” switch back on
- Run the program
Expect:- The character moves and turns as expected drawing lines as it moves
Test 17: Change theme
Complete these steps for each theme:
- Verify that all elements are visible and easily identifiable in the selected theme
- Test hover styles for all elements by hovering over each element with the mouse. Verify that all elements with hover styles implemented change to the hover style when hovered over with the mouse. Verify that the element stays visible and easily identifiable in the hover style.
- Using the tab key, test focus styles for all elements. Verify that the element is visible and clearly identifiable in its focus style, and that the focused element is clearly distinguished from other elements.
- Test selected styles for command blocks in the menu by clicking on each block to select it. Verify that each block is clearly indicated as selected when clicked on, and that it is visible and clearly identifiable when selected.
- Run a program and during running verify the hover styles and focus styles for the pause and stop buttons.
- Toggle switches and verify that they are visible and clearly identifiable in both the “on” and “off” states. Switches to test:
- Announcements switch
- Line drawing switch
- “+” node switch
- Change the character and verify that the current character is distinguished from other characters, and that all character selection buttons are visible and clearly identifiable regardless of what character is selected.
- Open the “Actions” menu and verify that the menu is readable and that all elements in the menu are visible and clearly identifiable
- Click the “Delete all” button and complete steps 1-3 for the elements in the modal.
- Click on a command block in the program to open its control panel and complete steps 1-3 for the elements in the control panel.
Test 18: Change World
- Select the world selection button
Expect:- A modal dialog with the worlds is shown
- Select the Space world radio button
Expect:- Space is selected and the background changes accordingly
- Select 'Done'
Expect:- The modal dialog closes
- The character is a spaceship
- Repeat Steps 1-3 for Jungle, Deep Ocean and Sketchpad
Test 19: Share button
Activate the “Share” button
Expect:
A message appears saying “The URL for your program has been copied to the clipboard.”
The program URL is copied to the clipboard
Open a new private browser window (incognito on Chrome)
Paste the URL of the program into the private window’s address bar and navigate to the URL.
Expect:
The program, character position, and character in the new browser window are the same as in the initial window
Test 20: URL Storage of program
- Open a new private browser window (incognito on Chrome)
- Copy the URL of the program from the address bar
- Paste the URL of the program into the private window’s address bar and navigate to the URL.
Expect:- The program, character position, and character in the new browser window are the same as in the initial window
Test 21: Announcement toggle switch
- Toggle the announcements switch off
- Activate an item in the program that has an associated announcement
Expect:- The item does not trigger an announcement
- Toggle the announcements switch on
- Repeat step 2, expecting that the item triggers an announcement.
Test 22: Announcement feature
- Verify that the announcements switch is set to on.
- Select each of the code blocks in the actions menu
Expect:- The selected code block is announced
- Add one of each code block to the end of the program
Expect:- The addition of each block is announced
- Select the first code block in the program and replace it with the “Move forward 2 squares” command block
Expect:- The replacement of the block is announced
- Select the first “Move forward three squares” code block in the program and delete it
Expect:- The deletion of the block is announced
- Select the “Turn left 180 degrees” command block and move it once to the right
Expect:- The movement of the block is announced
- Select the “Turn right 180 degrees” command block and move it once to the left
Expect:- The movement of the block is announced
Test 23: Keyboard Shortcuts
- Verify that keyboard shortcuts are enabled (open keyboard shortcuts menu and verify that the switch is set to on)
- Set the input scheme to the default input scheme using the Keyboard Shortcuts menu
- Close the keyboard shortcuts menu.
- Open the keyboard shortcuts menu using the show keyboard shortcuts menu keyboard shortcut
- Close the keyboard shortcuts menu
- Complete test 21 using the toggle announcements keyboard shortcut
- Select the “Move forward two squares” command block
- Add the command block to the end of the program using the add to end of program keyboard shortcut
Expect:- The command is added to the end of the program
- Add the command block to the start of the program using the add to start of program keyboard shortcut
Expect:- The command is added to the start of the program
- Use the announce character position and orientation keyboard shortcut
Expect:- The character’s position and orientation are announced
- Refresh scene using the keyboard shortcut
- Play a program using the play/pause program keyboard shortcut
- Pause a program using the play/pause program keyboard shortcut
- Stop a program using the stop program keyboard shortcut
- Complete test 11 using the increase and decrease playback speed keyboard shortcuts.
- Start NVDA
- Open the keyboard shortcuts menu and set the input scheme to the NVDA key bindings
- Complete steps 6-15 using the NVDA keyboard shortcuts
- Disable keyboard shortcuts using the keyboard shortcuts menu
- Close the keyboard shortcuts menu
- Attempt steps 4-15, verifying that the shortcuts do not work.
Test 24: Screen reader compatibility
Complete tests 1-23 using a screen reader. Verify the following:
The character’s position should be announced whenever program playback is stopped, paused, or ends.
When the character’s position is changed using the directional arrows or alphanumeric grid input, the character’s new position should be announced.
The state of various controls should be announced by the screen reader when they are selected
When selecting the scene in browse mode, the screen reader should convey scene information
You should be able to navigate by headers using a screen reader
Test 25: Tab order
- Open a new window with the coding environment.
- Use the tab key to select elements in the program. Verify that the order that the elements are selected matches the “tab order” list below. To test activation of an element for the tab order list, use the spacebar key.
Tab order:
Weavly
App Helper / Keyboard
Announcements toggle
Theme selector
Draw lines switch
Refresh scene button
Robot character select
Bunny character select
Rocket character select
Available actions panel
Forward 1 square
Forward 2 squares
Forward 3 squares
Backwards 1 square
Backwards 2 squares
Backwards 3 squares
Left 45 degrees
Left 90 degrees
Left 180 degrees
Right 45 degrees
Right 90 degrees
Right 180 degrees
“+” node switch
Delete all button
Rotate left 45 button
Rotate right 45 button
Up
Left
Right
Down
Co-ordinate letter
Co-ordinate number
Program panel
Goes through program panel (including “+” nodes) from left to right
On activating a block, block control panel opens
Delete block
Replace block
Move block left
Move block right
Block itself
Play/pause button
Stop button
Playback slider
Share button