0.8 QA Plan

Owned by Michelle D'Souza
Last updated: Jul 08, 2021 by Justin Obara
11 min read

https://create.weavly.org/

Test 1: Initial Load

  1. Load the Coding Environment
    Expect:
    1. The page loads without any errors

Test 2: Change the character’s position and rotation

Test 2.1: Change the character’s position using alphanumeric grid input

  1. Input the alphanumeric co-ordinates (“E”, “4”) into the alphanumeric grid input
    Expect:
    1. The character appears at position E4 without drawing any lines
  2. Input the invalid alphanumeric co-ordinates (“10”, “8”) into the alphanumeric grid input
    Expect:
    1. The character appears at position E8 without drawing any lines
  3. Input the invalid alphanumeric co-ordinates (“$”, “10”) into the alphanumeric grid input
    Expect:
    1. The character appears at position E10 without drawing any lines
  4. Input the invalid alphanumeric co-ordinates (“H”, “-10”) into the alphanumeric grid input
    Expect:
    1. The character appears at position H10 without drawing any lines
  5. Input the invalid alphanumeric co-ordinates (“R”, “C”) into the alphanumeric grid input
    Expect:
    1. The character appears at position R10 without drawing any lines
  6. Input the invalid alphanumeric co-ordinates (“S”, “#”) into the alphanumeric grid input
    Expect:
    1. The character appears at position S10 without drawing any lines
  7. Input the invalid alphanumeric co-ordinates (“E”, “100”) into the alphanumeric grid input
    Expect:
    1. The character appears at position E10 without drawing any lines

Test 2.2: Change the character’s position using directional movement buttons

  1. Complete test 12 to refresh the scene
  2. Activate the “Move Right” button three times
    Expect:
    1. The character moves right three spaces on the grid without drawing any lines
  3. Activate the “Move Down” button two times
    Expect:
    1. The character moves down two spaces on the grid without drawing any lines
  4. Activate the “Move Left” button two times
    Expect:
    1. The character moves left two spaces on the grid without drawing any lines
  5. Activate the “Move Up” button one time
    Expect:
    1. The character moves up one space on the grid without drawing any lines
  6. Activate the “Move left” button ten times.
    Expect:
    1. The character travels to the left edge of the canvas and stops there.
  7. Activate the “Move up” button ten times.
    Expect:
    1. The character travels to the top of the canvas and stops there.
  8. Repeat test 2.1, step 1, using the co-ordinates “W” “12”. Activate the “Move down” button ten times.
    Expect:
    1. The character travels to the bottom of the canvas and stops there.
  9. Activate the “Move right” button ten times.
    Expect:
    1. The character travels to the right edge of the canvas and stops there.
  10. Repeat test 2.1, step 1, using the co-ordinates “A” “1”.

Test 2.3: Change the character’s rotation using rotation buttons

  1. Activate the “Turn left” button one time
    Expect:
    1. The character turns left 45 degrees
  2. Activate the “Turn right” button one time
    Expect:
    1. The character turns right 45 degrees
  3. Repeat steps 1 and 2, activating each button eight times (one 360-degree rotation) instead of once.
    Expect:
    1. The character completes a full 360-degree rotation in each direction
  4. Use the rotation buttons to set the character’s orientation back to default
  5. Complete test 12 to refresh the scene


Test 3: Simple program

  1. Whenever you add a command block to the program in this test, verify that:
    1. You do not use the “+” button to add it to the program
    2. The command is added to the program
    3. The command stays selected in the commands menu
  2. Select the “Move forward one square” command block and add it to the program
  3. Select the “Turn right 45 degrees” command block and add it to the
  4. Select the “Move forward three squares” command block and add it to the
  5. Select the “Turn left 90 degrees” command block and add it to the program
    Expect:
    1. There are 4 commands in the program

Test 4: Run program

  1. Activate the play button
    Expect:
    1. The commands in the program area are performed by the character
    2. If the “Draw lines” switch is on, the character draws lines as it moves. If the “Draw lines” switch is off, the character does not draw any lines.
    3. If the character reaches the edge of the canvas during a movement, it stops at the edge of the canvas (it doesn’t run off the canvas)
    4. The play button switches to a pause button while the program is running
    5. The character produces sounds while the program is running
    6. The stop button is available while the program is running
    7. The command being performed by the character is highlighted while the program is running

Test 5: Extended program

  1. Whenever you add a command block to the program in this test, verify that:
    1. You do not use the “+” button to add it to the program
    2. The command is added to the program
    3. The command stays selected in the commands menu
  2. If a program has already been created, complete test 14.2 to delete the program
  3. If the character is not in the default position, complete test 12 to refresh the scene
  4. Select the “Turn right 90 degrees” command block and add it to the program
  5. Select the “Move forward 3 squares” command block and add it to the program 6 times. This will result in the character attempting to run off the canvas. When running test 4 at the end of this test, make sure to verify the character does not run off the canvas.
  6. Select the “Turn left 45 degrees” command block and add it to the program.
  7. Select the “Turn left 90 degrees” command block and add it to the program.
  8. Select the “Move forward 2 squares” command block and add it to the program.
  9. Select the “Turn right 45 degrees” command block and add it to the program.
  10. Select the “Move forward 1 square” command block and add it to the program.
  11. Select the “Turn right 180 degrees” command block and add it to the program.
  12. Select the “Move backwards 1 square” command block and add it to the program.
  13. Select the “Move backwards 2 squares” command block and add it to the program.
  14. Select the “Turn left 180 degrees” command block and add it to the program.
  15. Select the “Move backwards 3 squares” command block and add it to the program.
  16. Run test 4. The character should end the program on the co-ordinates “D” “14”, facing right.
  17. Complete test 12 to refresh the scene

Test 6: Remove part of program

  1. To complete this test, you should have a program created using test 5. If this is not the case, complete test 14.2 followed by test 12, then complete test 5.
  2. Select a “Move forward 3 squares” command block in the program and delete it using the “delete” button. Repeat this for three of the “Move forward 3 squares” blocks.
    Expect:
    1. The command is removed from the program
  3. Complete test 4, verifying that the deleted commands are not performed by the character.
  4. Complete test 12 to refresh the scene

Test 7: Use “+” button to add to program

  1. If an extended program as created in test 5 and modified in step 6 is not currently entered, complete test 14.2, followed by test 12, followed by tests 5 and 6 in order.

Test 7.1: No command selected

  1. Verify that no command block is selected
  2. Attempt to add a command block to the end of the program using the “+” button
    Expect:
    1. No commands are added to the program

Test 7.2: Action selected

  1. Select the “Move forward 2 squares” command block
    Expect:
    1. The command block appears as selected
  2. Add the command block to the end of the program using the “+” button
    Expect:
    1. The selected command is added to the end of the program
  3. Select the “Move backward 1 square” command block
    Expect:
    1. The command block appears as selected
  4. Add the command block in between the first and second “move forward three squares” command blocks using the “+” button
    Expect:
    1. The selected command is added to the selected position in the program
  5. Select the “Move forward 1 square” command block
    Expect:
    1. The command block appears as selected
  6. Add the command block to the start of the program using the “+” button
    Expect:
    1. The selected command is added to the start of the program
  7. Complete test 4
  8. Complete test 12 to refresh the scene

Test 8: Change playback speed

  1. To complete this test, you should have a program created using test 5, modified using test 6 and added to using test 7. If this is not the case, complete test 14.2 followed by test 12, then complete tests 5, 6 and 7 in order.
  2. Any time you complete test 4 during this test, complete test 12 afterwards to refresh the scene.
  3. Verify that the current playback speed is set to the default playback speed (middle of slider). If it is not, use the slider to set the current playback speed to the default playback speed.
  4. Decrease the current playback speed one setting using the slider
  5. Complete test 4, verifying that the program runs at a slower speed than default.
  6. Decrease the current playback speed to the minimum using the slider.
  7. Complete test 4, verifying that the program runs at a slower speed than in step 4.
  8. Attempt to decrease the current playback speed further than the minimum using the slider.
  9. Complete test 4, verifying that the program runs at the same speed as in step 6.
  10. Increase the playback speed to one setting above default using the slider.
  11. Complete test 4, verifying that the program runs at a faster speed than default.
  12. Increase the current playback speed to the maximum using the slider.
  13. Complete test 4, verifying that the program runs at a faster speed than in step 10.
  14. Attempt to increase the current playback speed further than the maximum using the slider.
  15. Complete test 4, verifying that the program runs at the same speed as in step 12.

Test 9: Stop program

  1. Attempt to stop the program using the stop button while the program is not running
    Expect:
    1. Nothing happens
  2. Activate the play button
  3. During program playback, activate the stop button
    Expect:
    1. Program playback is stopped
    2. No command in the program is highlighted after the program stops
    3. The character remains where it was when the program was stopped and all lines drawn by the character remain on the canvas
  4. Complete test 4, verifying that program playback starts from the beginning.

Test 10: Pause program

  1. Activate the play button
  2. During program playback, activate the pause button (The play button should switch to a pause button for the duration of program playback)
    Expect:
    1. A command remains highlighted to indicate the current position in program playback
    2. The pause button switches to a play button
  3. Complete test 4

Test 11: Toggle line drawing

  1. To complete this test, you should have a program created using test 5, modified using test 6 and added to using test 7. If this is not the case, complete test 14.2 followed by test 12, then complete tests 5, 6 and 7 in order.
  2. Toggle the “Draw Line” switch off
  3. Complete test 4, verifying that the character does not draw any lines.
  4. Toggle the “Draw Line” switch back on
  5. Complete test 4, verifying that the character draws lines.

Test 12: Refresh scene

  1. Activate the “Refresh scene” button
    Expect:
    1. The character is moved to the A1 square
    2. Any lines that were present on the canvas have been removed

Test 13: Program sounds

  1. Create a program that contains one of each command
  2. Run test 4, verifying that each command plays a unique sound associated with it.

Test 14: Delete all dialog

  1. Activate the “Delete all” button
    Expect:
    1. The “Delete all” dialog appears on the screen

Test 14.1: Cancel delete all

  1. Activate the “Cancel” button in the dialog
    Expect:
    1. The dialog closes and the program remains the same as it was at the beginning of this test

Test 14.2: Confirm delete all

  1. Activate the “Delete All” button in the dialog
    Expect:
    1. The dialog closes and all commands in the program are removed

Test 15: Change theme

Complete these steps for each theme:

  1. Verify that all elements are visible and easily identifiable in the selected theme
  2. Test hover styles for all elements by hovering over each element with the mouse. Verify that all elements with hover styles implemented change to the hover style when hovered over with the mouse. Verify that the element stays visible and easily identifiable in the hover style.
  3. Using the tab key, test focus styles for all elements. Verify that the element is visible and clearly identifiable in its focus style, and that the focused element is clearly distinguished from other elements.
  4. Test selected styles for command blocks in the menu by clicking on each block to select it. Verify that each block is clearly indicated as selected when clicked on, and that it is visible and clearly identifiable when selected.
  5. Complete test 4 and verify the hover styles and focus styles for the pause and stop buttons.
  6. Toggle switches and verify that they are visible and clearly identifiable in both the “on” and “off” states. Switches to test:
    1. Announcements switch
    2. Line drawing switch
    3. “+” node switch
  7. Change the character and verify that the current character is distinguished from other characters, and that all character selection buttons are visible and clearly identifiable regardless of what character is selected.
  8. Open the “Actions” menu and verify that the menu is readable and that all elements in the menu are visible and clearly identifiable
  9. Click the “Delete all” button and complete steps 1-3 for the elements in the modal.
  10. Click on a command block in the program to open its control panel and complete steps 1-3 for the elements in the control panel.

Test 16: Change Character

  1. Change the character using the character selection buttons
    Expect:
    1. The selected character appears on the canvas
  2. Repeat step 1 for each character

Test 17: Share button

  1. Activate the “Share” button
    Expect:
    1. A message appears saying “The URL for your program has been copied to the clipboard.”
    2. The program URL is copied to the clipboard
  2. Open a new private browser window (incognito on Chrome)
  3. Paste the URL of the program into the private window’s address bar and navigate to the URL.
    Expect:
  4. The program, character position, and character in the new browser window are the same as in the initial window

Test 18: URL Storage of program

  1. Open a new private browser window (incognito on Chrome)
  2. Copy the URL of the program from the address bar
  3. Paste the URL of the program into the private window’s address bar and navigate to the URL.
    Expect:
    1. The program, character position, and character in the new browser window are the same as in the initial window

Test 19: Announcement toggle switch

  1. Toggle the announcements switch off
  2. Activate an item in the program that has an associated announcement
    Expect:
    1. The item does not trigger an announcement
  3. Toggle the announcements switch on
  4. Repeat step 2, expecting that the item triggers an announcement.

Test 20: Announcement feature

  1. To complete this test, you should have a program created using test 5, modified using test 6 and added to using test 7. If this is not the case, complete test 14.2 followed by test 12, then complete tests 5, 6 and 7 in order.
  2. Verify that the announcements switch is set to on.
  3. Select each of the code blocks in the actions menu
    Expect:
    1. The selected code block is announced
  4. Add one of each code block to the end of the program
    Expect:
    1. The addition of each block is announced
  5. Select the first code block in the program and replace it with the “Move forward 2 squares” command block
    Expect:
    1. The replacement of the block is announced
  6. Select the first “Move forward three squares” code block in the program and delete it
    Expect:
    1. The deletion of the block is announced
  7. Select the “Turn left 180 degrees” command block and move it once to the right
    Expect:
    1. The movement of the block is announced
  8. Select the “Turn right 180 degrees” command block and move it once to the left
    Expect:
    1. The movement of the block is announced

Test 21: Keyboard Shortcuts

  1. To complete this test, you should have a program created using test 5, modified using test 6 and added to using test 7. If this is not the case, complete test 14.2 followed by test 12, then complete tests 5, 6 and 7 in order.
  2. Verify that keyboard shortcuts are enabled (open keyboard shortcuts menu and verify that the switch is set to on)
  3. Set the input scheme to the default input scheme using the Keyboard Shortcuts menu
  4. Close the keyboard shortcuts menu.
  5. Open the keyboard shortcuts menu using the show keyboard shortcuts menu keyboard shortcut
  6. Close the keyboard shortcuts menu
  7. Complete test 15 using the toggle announcements keyboard shortcut
  8. Select the “Move forward two squares” command block
  9. Add the command block to the end of the program using the add to end of program keyboard shortcut
    Expect:
    1. The command is added to the end of the program
  10. Add the command block to the start of the program using the add to start of program keyboard shortcut
    Expect:
    1. The command is added to the start of the program
  11. Use the announce character position and orientation keyboard shortcut
    Expect:
    1. The character’s position and orientation are announced
  12. Complete test 12 using the refresh scene keyboard shortcut
  13. Complete test 4 using the play/pause program keyboard shortcut
  14. Complete test 10 using the play/pause program keyboard shortcut
  15. Complete test 9 using the stop program keyboard shortcut
  16. Complete test 8 using the increase and decrease playback speed keyboard shortcuts.
  17. Start NVDA
  18. Open the keyboard shortcuts menu and set the input scheme to the NVDA key bindings
  19. Complete steps 6-15 using the NVDA keyboard shortcuts
  20. Disable keyboard shortcuts using the keyboard shortcuts menu
  21. Close the keyboard shortcuts menu
  22. Attempt steps 4-15, verifying that the shortcuts do not work.

Test 22: Screen reader compatibility

  1. Complete tests 1-21 using a screen reader. Verify the following:
    1. The character’s position should be announced whenever program playback is stopped, paused, or ends.
    2. When the character’s position is changed using the directional arrows or alphanumeric grid input, the character’s new position should be announced.
    3. The state of various controls should be announced by the screen reader when they are selected
    4. When selecting the scene in browse mode, the screen reader should convey scene information
    5. You should be able to navigate by headers using a screen reader

Test 23: Tab order

  1. Open a new window with the coding environment.
  2. Use the tab key to select elements in the program. Verify that the order that the elements are selected matches the “tab order” list below. To test activation of an element for the tab order list, use the spacebar key.

Tab order:

  1. Weavly
  2. App Helper / Keyboard
  3. Announcements toggle
  4. Theme selector
  5. Draw lines switch
  6. Refresh scene button
  7. Robot character select
  8. Bunny character select
  9. Rocket character select
  10. Available actions panel
  11. Forward 1 square
  12. Forward 2 squares
  13. Forward 3 squares
  14. Backwards 1 square
  15. Backwards 2 squares
  16. Backwards 3 squares
  17. Left 45 degrees
  18. Left 90 degrees
  19. Left 180 degrees
  20. Right 45 degrees
  21. Right 90 degrees
  22. Right 180 degrees
  23. “+” node switch
  24. Delete all button
  25. Rotate left 45 button
  26. Rotate right 45 button
  27. Up
  28. Left
  29. Right
  30. Down
  31. Co-ordinate letter
  32. Co-ordinate number
  33. Program panel
    1. Goes through program panel (including “+” nodes) from left to right
    2. On activating a block, block control panel opens
      1. Delete block
      2. Replace block
      3. Move block left
      4. Move block right
      5. Block itself
  34. Play/pause button
  35. Stop button
  36. Playback slider
  37. Share button