1.0 QA Plan

Owned by Michelle D'Souza
Last updated: Dec 14, 2021 by Tony Atkins [RtF]
12 min read


Test 1: Initial Load

  1. Load the Coding Environment

    Expect:

    1. The page loads without any errors

Test 2: Change the character’s position

Setup: Program area and scene are empty, character is at position A2 (A1 in sketchpad world).

  1. Input the alphanumeric co-ordinates (“E”, “4”) into the alphanumeric grid input

    Expect:

    1. The character appears at position E4 without drawing any lines

  2. Input the invalid alphanumeric co-ordinates (“10”, “8”) into the alphanumeric grid input

    Expect:

    1. The character appears at position E8 without drawing any lines

  3. Input the invalid alphanumeric co-ordinates (“$”, “10”) into the alphanumeric grid input

    Expect:

    1. The character stays at position E8

  4. Input the invalid alphanumeric co-ordinates (“H”, “-10”) into the alphanumeric grid input

    Expect:

    1. The character appears at position H8 without drawing any lines

  5. Input the invalid alphanumeric co-ordinates (“J”, “C”) into the alphanumeric grid input

    Expect:

    1. The character appears at position J8 without drawing any lines

  6. Input the invalid alphanumeric co-ordinates (“L”, “#”) into the alphanumeric grid input

    Expect:

    1. The character appears at position L8 without drawing any lines

  7. Input the invalid alphanumeric co-ordinates (“E”, “100”) into the alphanumeric grid input

    Expect:

    1. The character appears at position E8 without drawing any lines

  8. Input the co-ordinates (“A”, “1”) into the character positioning fields

    Expect:

    1. The character appears at position A1 without drawing any lines

Test 3: Change the character’s position using directional movement buttons

Setup: Program area and scene are empty, character is at position A1.

  1. Activate the “Move Right” button three times

    Expect:

    1. The character moves right three spaces to D1 on the grid without drawing any lines

    2. The values in the character positioning fields are updated appropriately each time the 'Move Right' is activated

  2. Activate the “Move Down” button two times

    Expect:

    1. The character moves down two spaces to D3 on the grid without drawing any lines

    2. The values in the character positioning fields are updated appropriately each time the 'Move Down' is activated

  3. Activate the “Move Left” button two times

    Expect:

    1. The character moves left two spaces to B3 on the grid without drawing any lines

    2. The values in the character positioning fields are updated appropriately each time the 'Move Left' is activated

  4. Activate the “Move Up” button one time

    Expect:

    1. The character moves up one space to B2 on the grid without drawing any lines

    2. The values in the character positioning fields are updated appropriately each time the 'Move Up' is activated

  5. Activate the “Move left” button 3 times.

    Expect:

    1. The character travels to the left edge of the canvas (A2) and stops there.

    2. The values in the character positioning fields are updated appropriately each time the 'Move Left' is activated

  6. Activate the “Move up” button 3 times.

    Expect:

    1. The character travels to the top corner (A1) of the canvas and stops there.

    2. The values in the character positioning fields are updated appropriately each time the 'Move Up' is activated

  7. Activate the “Move Right” button 12 times.

    Expect:

    1. The character travels to the right edge of the canvas (L1) and stops there.

    2. The values in the character positioning fields are updated appropriately each time the 'Move Right' is activated

  8. Activate the “Move Down” button 9 times.

    Expect:

    1. The character travels to the bottom corner of the canvas (L8) and stops there.

    2. The values in the character positioning fields are updated appropriately each time the 'Move Down' is activated

  9. Use the character positioning buttons to move the character to A1

Test 4: Change the character’s rotation using rotation buttons

Setup: Program area and scene are empty, character is at position A1

  1. Activate the “Turn left” button one time

    Expect:

    1. The character turns left 45 degrees

  2. Activate the “Turn right” button one time

    Expect:

    1. The character turns right 45 degrees back to its original direction

  3. Activate the "Turn Left" button 8 times

    Expect:

    1. The character completes a full 360-degree rotation left returning to its original direction

  4. Activate the "Turn Right" button 8 times

    Expect:

    1. The character completes a full 360-degree rotation right returning to its original direction

Test 5: Simple program using '+' to add

Setup: Program area and scene are empty, character is at position A1

  1. Select the “Move forward one square” command block

    Expect:

    1. The block appears selected

    2. "Movement forward one square selected" is read out

  2. Activate the '+' in the program area

    Expect:

    1. The block and a '+' are added to the program

    2. The block added to the program is selected

    3. "Added movement forward one square" is read out

  3. Repeat step 2

  4. Select the “Turn right 45 degrees” command block

    Expect:

    1. The block appears selected

    2. "Movement turn right 45 degrees selected" is read out

  5. Activate the '+' between the two blocks in the program

    Expect:

    1. The block and a '+' are added to the middle of the program

    2. The block added to the program is selected

    3. "Added movement turn right 45 degrees" is read out

  6. Activate the '+' at the beginning of the program

    Expect:

    1. The block and a '+' are added to the beginning of the program

    2. The block added to the program is selected

    3. "Added movement turn right 45 degrees" is read out

  7. Deselect the 'turn right 45 degrees' action

    Expect:

    1. The block does not appear selected

    2. "No movement selected" is read out

  8. Activate the '+' at the beginning of the program

    Expect:

    1. The '+' appears selected but no block is added to the program

    2. There are a total of 4 action tiles in the program

Test 6: Run program

Setup:

  • Test 5 above has been completed
  • There is a short program in the program area
  • The scene is empty
  • Character is at position A1

  1. Activate the play button

    Expect:

    1. The commands in the program area are performed by the character, drawing a line as it moves

    2. The play button switches to a pause button while the program is running

    3. The character produces sounds while the program is running

    4. The stop button is available while the program is running

    5. The command being performed by the character is highlighted while the program is running

Test 7: Refresh scene

Setup:

  • Test 6 above has been completed
  • There is a drawing on the scene
  • Character is at position B3 facing down

  1. Activate the 'Refresh Scene' button

    Expect:

    1. The character is moved to the A2 square (A1 in sketchpad world).

    2. Any lines that were present on the canvas have been removed

Test 8: Delete program

Setup:

  • Test 7 above has been completed
  • There is a short program in the program area

  1. Activate the “Delete” button in the program panel

    Expect:

    1. The “Delete all” dialog appears on the screen

    2. "Delete all movements" is read out

  2. Activate the “Cancel” button in the dialog
    Expect:

    1. The dialog closes and the program remains the same as it was at the beginning of this test
  2. Activate the “Delete” button and activate the "Delete All" in the dialog
    Expect:
    1. The dialog closes and all commands in the program are removed

Test 9: Program using drag & drop

Setup: Program area and scene are empty, character is at position A2 (A1 in sketchpad world).

  1. Select the “Move forward three squares” command block and drop it in the program area

    Expect:

    1. The block is added to the program

    2. The block appears selected both in the program area and the movements panel

    3. "Movement forward three squares selected" and "Added movement forward three squares" is read out

  2. Repeat step 1 four times

    Expect:

    1. There are 5 "Move forward three squares" blocks in the program

    2. "Added movement forward three squares" is read out after each addition

  3. Select the "Turn right 90 degrees" block and drop it to between the third and fourth program blocks in the program area

    Expect:

    1. The block is inserted into the program

    2. "Movement turn right 90 degrees selected" and "Added movement turn right 90 degrees" is read out

  4. Select the "Move forward two squares" block and drop it after the "Turn right"

    Expect:

    1. The block is inserted into the program

    2. "Movement forward two squares selected" and "Added movement forward two squares" is read out

  5. Select the "Move forward two squares" block and drop it before the "Turn right"

    Expect:

    1. The block is inserted into the program

    2. "Added movement forward two squares" is read out

  6. Scroll to the end of the program and add a 'Turn right 90 degrees' at the end

    Expect:

    1. The block is inserted into the program

    2. "Added movement turn right 90 degrees" is read out
  7. Select the "Turn left 180 degrees" block and drag it over the program area without releasing the mouse.  Move the pointer outside the program area and release the mouse.
    Expect:
    1. The block is not inserted into the program.
    2. No announcement is made.

  8. Run the program

    Expect:

    1. The character moves along the top of the scene to the top right corner, turns right and moves to the bottom right corner and turns right again

    2. The play button switches to a pause button while the program is running
    3. The character produces sounds while the program is running
    4. The stop button is available while the program is running
    5. The command being performed by the character is highlighted while the program is running

Test 10: Remove part of the program

Setup:

  • Test 9 (Program using drag & drop) above has been completed
  • There is a program in the program area
  1. Select the first block in the program 
    Expect:
    1. The control panel for the block is shown
  2. Activate the “delete” option in the control panel.
    Expect:
    1. The block is removed from the program
    2. "Deleted movement forward 3 squares" is read out
  3. Run the program
    Expect:
    1. The updated program runs as expected
    2. The character is at D1 facing its original direction

Test 11: Change playback speed

Setup:

  • There is a program in the program area
  • the current playback speed is set to the default playback speed (middle of slider)
  1. Decrease the current playback speed one setting using the slider
  2. Run the program 
    Expect:
    1. the program runs at a slower speed than default.
  3. Decrease the current playback speed to the minimum using the slider.
  4. Run the program 
    Expect:
    1. the program runs at a slower speed than in step 2.
  5. Attempt to decrease the current playback speed further than the minimum  
    Expect:
    1. the slider remains at the minimum speed
  6. Run the program 
    Expect:
    1. the program runs at the same speed as in step 4.
  7. Increase the playback speed to one setting above default using the slider.
  8. Run the program 
    Expect:
    1. the program runs at a faster speed than default.
  9. Increase the current playback speed to the maximum using the slider.
  10. Run the program 
    Expect:
    1. the program runs at a faster speed than in step 8
  11. Attempt to increase the current playback speed further than the maximum using the slider.
  12. Run the program 
    Expect:
    1. the program runs at the same speed as in step 10
  13. Run the program and change the playback speed while it is running
    Expect:
    1. the program speed changes during playback

Test 12: Character tries to move beyond scene edge

  1. Load this program https://develop-1-0--c2lc-build.netlify.app/?v=0.9&t=mixed&w=Sketchpad&p=6B3A26666d3A2b2&c=aab&a=123456ABDabd
    Expect:
    1. the character is in position A1
    2. there is a program in the program area
  2. Run the program 
    Expect:
    1. the character tries to move beyond the edge of the scene several times but is unable to
    2. the drawing is a line from A1 to H8 to L8
    3. the character is facing down
  3. Refresh the scene and delete the program

Test 13: All commands

  1. Load this program https://develop-1-0--c2lc-build.netlify.app/?v=1.0&t=mixed&w=Sketchpad&p=123ABDdba654&c=aac&a=123456ABDabd
    Expect:
    1. Scene is empty, character is at position A1 facing southeast.
    2. Every type of command is in the program.

  1. Run the program

    Expect:

    1. The character moves or turns as expected based on the action being run

    2. Each distinct movement or turn has its own unique sound that plays while it is happening
  2. Refresh the scene

Test 14: Stop program

Setup:

  • There is a program in the program area
  • The scene is empty
  • The character is in position A1
  1. Attempt to stop the program using the stop button while the program is not running
    Expect:
    1. Nothing happens
  2. Activate the play button and during program playback, activate the stop button
    Expect:
    1. Program playback is stopped
    2. No command in the program is highlighted after the program stops
    3. The character remains where it was when the program was stopped and all lines drawn by the character remain on the canvas
  3. Run the program
    Expect:
    1. The program playback starts from the beginning.
  4. Refresh the scene

Test 15: Pause program

Setup:

  • There is a program in the program area
  • The scene is empty
  • The character is in position A1
  1. Activate the play button
  2. During program playback, activate the pause button (The play button should switch to a pause button for the duration of program playback)
    Expect:
    1. A command remains highlighted to indicate the current position in program playback
    2. The pause button switches to a play button
  3. Run the program
    Expect:
    1. The program continues from where it left off and finishes
  4. Refresh the scene

Test 16: Toggle line drawing

Setup:

  • There is a program in the program area
  • The scene is empty
  • The character is in position A1
  1. Toggle the “Draw Line” switch off
  2. Run the program
    Expect:
    1. The character moves and turns as expected
    2. No lines are drawn
  3. Toggle the “Draw Line” switch back on
  4. Run the program
    Expect:
    1. The character moves and turns as expected drawing lines as it moves

Test 17: Change theme

Complete these steps for each theme:

  1. Verify that all elements are visible and easily identifiable in the selected theme
  2. Test hover styles for all elements by hovering over each element with the mouse. Verify that all elements with hover styles implemented change to the hover style when hovered over with the mouse. Verify that the element stays visible and easily identifiable in the hover style.
  3. Using the tab key, test focus styles for all elements. Verify that the element is visible and clearly identifiable in its focus style, and that the focused element is clearly distinguished from other elements.
  4. Test selected styles for command blocks in the menu by clicking on each block to select it. Verify that each block is clearly indicated as selected when clicked on, and that it is visible and clearly identifiable when selected.
  5. Run a program and during running verify the hover styles and focus styles for the pause and stop buttons.
  6. Toggle switches and verify that they are visible and clearly identifiable in both the “on” and “off” states. Switches to test:
    1. Announcements switch
    2. Line drawing switch
    3. “+” node switch
  7. Change the character and verify that the current character is distinguished from other characters, and that all character selection buttons are visible and clearly identifiable regardless of what character is selected.
  8. Open the “Actions” dialog and verify that the content is readable and that all elements are visible and clearly identifiable
  9. Click the “Delete all” button and complete steps 1-3 for the elements in the modal.
  10. Click on a command block in the program to open its control panel and complete steps 1-3 for the elements in the control panel.

Test 18: Change World

  1. Select the world selection button
    Expect:
    1. A modal dialog with the worlds is shown
  2. Select the Space world radio button
    Expect:
    1. Space is selected and the background changes accordingly
  3. Select 'Done'
    Expect:
    1. The modal dialog closes
    2. The character is a spaceship and the background remains changed.
  4. Repeat steps 1-2, selecting a different world.
  5. Select 'Cancel'
    Expect:
    1. The original world is displayed.
  6. Repeat Steps 1-3 for Jungle, Deep Ocean and Sketchpad

Test 19: Share button

  1. Activate the “Share” button

    Expect:

    1. A message appears saying “The URL for your program has been copied to the clipboard.”

    2. The program URL is copied to the clipboard

  2. Open a new private browser window (incognito on Chrome)

  3. Paste the URL of the program into the private window’s address bar and navigate to the URL.

    Expect:

    1. The program, character position, and character in the new browser window are the same as in the initial window

Test 20: URL Storage of program

  1. Open a new private browser window (incognito on Chrome)
  2. Copy the URL of the program from the address bar
  3. Paste the URL of the program into the private window’s address bar and navigate to the URL.
    Expect:
    1. The program, character position, and character in the new browser window are the same as in the initial window

Test 21: Announcement toggle switch

  1. Toggle the announcements switch off
  2. Activate an item in the program that has an associated announcement
    Expect:
    1. The item does not trigger an announcement
  3. Toggle the announcements switch on
  4. Repeat step 2, expecting that the item triggers an announcement.

Test 22: Announcement feature

  1. Verify that the announcements switch is set to on.
  2. Select each of the code blocks in the actions menu
    Expect:
    1. The selected code block is announced
  3. Add one of each code block to the end of the program
    Expect:
    1. The addition of each block is announced
  4. Select the first code block in the program and replace it with the “Move forward 2 squares” command block
    Expect:
    1. The replacement of the block is announced
  5. Select the first “Move forward three squares” code block in the program and delete it
    Expect:
    1. The deletion of the block is announced
  6. Select the “Turn left 180 degrees” command block and move it once to the right
    Expect:
    1. The movement of the block is announced
  7. Select the “Turn right 180 degrees” command block and move it once to the left
    Expect:
    1. The movement of the block is announced

Test 23: Keyboard Shortcuts

  1. Verify that keyboard shortcuts are enabled (open keyboard shortcuts menu and verify that the switch is set to on)
  2. Set the input scheme to the default "Ctrl+Alt/Option" input scheme using the Keyboard Shortcuts menu
  3. Close the keyboard shortcuts menu.
  4. Open the keyboard shortcuts menu using the show keyboard shortcuts menu keyboard shortcut (Question Mark)
  5. Close the keyboard shortcuts menu
  6. Complete test 21 using the toggle announcements keyboard shortcut (Control + Alt/Option + x, x)
  7. Select the “Move forward two squares” command block using the keyboard shortcut (Control + Alt/Option + x, a, f, 2)
  8. Add the command block to the end of the program using the add to end of program keyboard shortcut (Control + Alt/Option + e)
    Expect:
    1. The command is added to the end of the program
  9. Add the command block to the start of the program using the add to start of program keyboard shortcut (Control + Alt/Option + b)
    Expect:
    1. The command is added to the start of the program
  10. Use the announce character position and orientation keyboard shortcut (Control + Alt/Option + i)
    Expect:
    1. The character’s position and orientation are announced.
  11. Refresh scene using the keyboard shortcut (Control + Alt/Option + r) 
  12. Play a program using the  play/pause program keyboard shortcut (Control + Alt/Option + p) 
  13. Pause a program using the play/pause program keyboard shortcut (Control + Alt/Option + p) 
  14. Stop a program using the stop program keyboard shortcut (Control + Alt/Option + s) 
  15. Complete test 11 using the increase (>) and decrease (<) playback speed keyboard shortcuts.
  16. Start NVDA
  17. Open the keyboard shortcuts menu and set the input scheme to the NVDA key bindings
  18. Complete steps 6-15 using the NVDA keyboard shortcuts, using only Alt/Option instead of Ctrl+Alt/Option
  19. Disable keyboard shortcuts using the keyboard shortcuts menu
  20. Close the keyboard shortcuts menu
  21. Attempt steps 4-15, verifying that the shortcuts do not work.

Test 24: Screen reader compatibility

  1. Complete tests 1-23 using a screen reader. Verify the following:

    1. The character’s position should be announced whenever program playback is stopped, paused, or ends.

    2. When the character’s position is changed using the directional arrows or alphanumeric grid input, the character’s new position should be announced.

    3. The state of various controls should be announced by the screen reader when they are selected

    4. When selecting the scene in browse mode, the screen reader should convey scene information

    5. You should be able to navigate by headers using a screen reader

Test 25: Tab order

  1. Open a new window with the coding environment.
  2. Use the tab key to select elements in the program. Verify that the order that the elements are selected matches the “tab order” list below. To test activation of an element for the tab order list, use the spacebar key.

Tab order: 

  1. Weavly

  2. App Helper / Keyboard

  3. Announcements toggle

  4. Theme selector
  5. World selector
  6. Pen Up/Down Toggle
  7. Character Controls

    1. Rotate left 45 button

    2. Rotate right 45 button

    3. Up

    4. Left

    5. Right

    6. Down

    7. Co-ordinate letter

    8. Co-ordinate number

  8. Actions Simplification Menu
  9. Actions
    1. Forward 1 square

    2. Forward 2 squares

    3. Forward 3 squares

    4. Backwards 1 square

    5. Backwards 2 squares

    6. Backwards 3 squares

    7. Left 45 degrees

    8. Left 90 degrees

    9. Left 180 degrees

    10. Right 45 degrees

    11. Right 90 degrees

    12. Right 180 degrees

  10. Add/Expand “+” node switch

  11. Delete all button

  12. Program panel

    1. Goes through program panel (including “+” nodes) from left to right

    2. On activating a block, block control panel opens

      1. Delete block

      2. Replace block

      3. Move block left

      4. Move block right

      5. Block itself

  13. Refresh scene button

  14. Play/pause button

  15. Stop button

  16. Playback slider

  17. Share button

Test 26: Backwards compatibility of URLS

  1. Copy and paste the following link into your browser's location bar and hit enter:
    https://develop-1-0--c2lc-build.netlify.app/?v=0.7&t=mixed&w=forest&p=b32A1A3A1A3a1a33a1a1A2a1a1a1A41a4A4A4A5a2a1a33a1a32a5a4a5&c=aab&a=123456ABDabd
    Expect:
    1. The jeep and jungle background are displayed.
    2. The jeep is on A1
    3. There are steps in the program.
  2. Copy and paste the following link into your browser's location bar and hit enter: 

    https://develop-1-0--c2lc-build.netlify.app/?v=0.7&t=mixed&w=forest&p=b32A1A3A1A3a1a33a1a1A2a1a1a1A41a4A4A4A5a2a1a33a1a32a5a4a5&c=aacaaadadafafbgbgegegfffffcfcgbgbjbjbmbmbncncndndpfpfpgpgohohnhnhogognhnhohohpgpgpfpfndndnfnfmgmgjgjgggggfffffcfcfafadcdcccccaa&a=123456ABDabd
    Expect:

    1. The jeep and jungle background are displayed.
    2. part of a cat is drawn onscreen
    3. There are steps in the program.
  3. Copy and paste the following link into your browser's location bar and hit enter: 

    https://develop-1-0--c2lc-build.netlify.app/?v=0.7&t=mixed&w=default&p=3233323323b3322233b23223222332b232232b3133133313b33331b33333332b3333b33333331b3332b3333333b3331b3333332b333b3333331b332b333333b331b333332b33b333322b221b332322b1111b322322b3b322132b2b32313b1b31313&c=aab&a=123456ABDabd
    Expect:

    1. The robot and sketchpad background are displayed.
    2. The robot is on A1
    3. There are steps in the program.
  4. Copy and paste the following link into your browser's location bar and hit enter:
    https://develop-1-0--c2lc-build.netlify.app/?v=0.7&t=mixed&w=space&p=a33333B3331B32b66666B32b3133b33333&c=aab&a=123456ABDabd
    Expect:
    1. The rocket ship and background are displayed
    2. The rocket is on A1
    3. There are steps in the program.