Storytelling Tool Manual Testing Grid
This testing grid is intended to act as a set of actions which are to be attempted in a number of different environments (read: hardware and software combinations) in order to ensure broad compatibility and continued functionality of the Storytelling Tool. If the tool changes such that the devices listed are no longer supported, new functionality is added, or the list in some other way no longer accurately reflects real-world usage of the Tool, please feel free to make contributions to the grid.
When a particular device is able to pass a sequence, the applicable cell in the grid is to be marked "Pass" and the background colour changed to green (Confluence offers some quick cell background choices in its editing tools). If the device does not pass a particular sequence, the cell is to be marked "Fail" and the background set to red. If the test is completed but has issues, the cell is to be marked "Issues" and the background set to yellow. Where the sequence is not applicable, e.g. on a desktop where camera capture isn't an option, the cell is to be marked "N/A" and the background set to blue. If a particular device or sequence has not been tested, simply leave it blank.
The devices and software versions listed are based on what is currently available for use by staff at the IDRC, and though it should not necessarily be seen as prescriptive, reasonable attempts should be made to access these when testing.
For a more condensed test suite for quicker releases, try the Storytelling Tool Minimal Testing Grid
Grid last updated: March 18, 2020
Tests last completed: March 18, 2020
Action sequences:
- Story Builder: User is able to publish a story with no blocks and no other fields filled in
- Story Builder: User is able to publish a story with one empty block of each type and no other fields filled in (empty, but with empty blocks too)
- Story Builder: User is able to publish a story with two text blocks
- Story Builder: User is able to publish a story with two image blocks, both images uploaded from the file system*
- Story Builder: User is able to publish a story with two image blocks, both images captured from the device*
- Story Builder: User is able to publish a story with two image blocks, one image captured, one image uploaded*
- Story Builder: User is able to publish a story with two audio blocks, both files uploaded from the file system*
- Story Builder: User is able to publish a story with two audio blocks, both files captured from the device*
- Story Builder: User is able to publish a story with two audio blocks, one file captured, one file uploaded*
- Story Builder: User is able to publish a story with two video blocks, both files uploaded from the file system*
- Story Builder: User is able to publish a story with two video blocks, both files captured from the device*
- Story Builder: User is able to publish a story with two video blocks, one file captured, one file uploaded*
- Story Builder: User can publish a story with one block of each type and all the fields filled in
- Story Builder/Viewer: Listening to the story with the Orator plays the story completely both in the Story Preview and on viewing a published story, and the Orator highlights text as it reads
- All pages: Imagery in the editor changes along with UIO contrast theme changes
- Back and forward within the editor steps works properly, state is not remembered on page reload
* for the sequences that require two images, videos, audio files, etc. the intent is that these files be different from one another. I.e. for two uploaded images, make sure they're different images.
| Device Name | OS & Version | Browser & Version | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Apple iPad Mini 2 | iOS 12.4.4 | Safari | Pass | 3 | Pass | 3 | 3, 4 | 3 | Pass | Pass | Pass | Pass | Pass | Pass | 3 | Pass | Pass | Pass |
| Apple iPad Air 2 | iOS 12.4 | Safari | Pass | 3 | Pass | 3 | 3, 4 | 3 | Pass | Pass | Pass | Pass | Pass | Pass | 3 | Pass | Pass | Pass |
| Apple iPad Air 3 | iOS 13.3 | Safari | Pass | 3 | Pass | 3 | 3, 4 | 3 | Pass | 7 | 7 | Pass | Pass | Pass | 3 | Pass | Pass | Pass |
| Apple iPod touch (6th gen) | iOS 12.4.2 | Safari | Pass | 3 | Pass | 3 | 3, 4 | 3 | Pass | Pass | Pass | Pass | Pass | Pass | 3 | Pass | Pass | Pass |
| Lenovo P52 | Windows 10 Pro build 1909 | Chrome 80 | Pass | Pass | Pass | Pass | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | Pass | 5 | Pass | Pass |
| Lenovo P52 | Windows 10 Pro build 1909 | Firefox 74 | Pass | 3 | Pass | 3 | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | 3 | Pass | Pass | Pass |
| Lenovo P52 | Windows 10 Pro build 1909 | Edge 80 | Pass | Pass | Pass | Pass | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | Pass | 5 | Pass | Pass |
| Lenovo P52 | Windows 10 Pro build 1909 | Internet Explorer 11 | Fail | Fail | Fail | Fail | N/A | N/A | Fail | N/A | N/A | Fail | N/A | N/A | Fail | Fail | Fail | Fail |
| Gaming PC | Windows 10 Pro build 1903 | Chrome 80 | Pass | Pass | Pass | Pass | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | Pass | Pass | Pass | Pass |
| Gaming PC | Windows 10 Pro build 1903 | Firefox 74 | Pass | 3 | Pass | 3 | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | 3 | Pass | Pass | Pass |
| Gaming PC | Windows 10 Pro build 1903 | Edge 44 | Pass | 2 | 2 | 2 | N/A | N/A | 2 | N/A | N/A | 2 | N/A | N/A | 2 | Fail | Pass | Pass |
| Gaming PC | Windows 10 Pro build 1903 | Internet Explorer 11 | Fail | Fail | Fail | Fail | N/A | N/A | Fail | N/A | N/A | Fail | N/A | N/A | Fail | Fail | Fail | Fail |
| Google Pixel 3 | Android 10 | Chrome 80 | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | 1 | Pass | Pass |
| Google Pixel 3 | Android 10 | Firefox 68.6.0 | Pass | 3 | Pass | Fail | 3 | Fail | Fail | Fail | Fail | Fail | Pass | Fail | Fail | Pass | Pass | Pass |
| Samsung Galaxy S6 | Android 7.0 | Chrome 80 | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | Pass | 8 | Pass | Pass |
| MacBook Pro (2015) | macOS High Sierra 10.13.6 | Chrome 80 | Pass | Pass | Pass | Pass | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | Pass | Pass | Pass | Pass |
| MacBook Pro (2015) | macOS High Sierra 10.13.6 | Safari 13.0.2 | 6 | 3, 6 | 6 | 3, 6 | N/A | N/A | 6 | N/A | N/A | 6 | N/A | N/A | 3, 6 | 6 | Pass | Pass |
| MacBook Pro (2015) | macOS High Sierra 10.13.6 | Firefox 74 | Pass | 3 | Pass | 3 | N/A | N/A | Pass | N/A | N/A | Pass | N/A | N/A | 3 | Pass | Pass | Pass |
Issues noted in latest pass:
| Issue # | Jira | Description |
|---|---|---|
| 1 | FLUID-6412 - Getting issue details... STATUS | The Orator highlighting does not appear at all. |
| 2 | SJRK-304 - Getting issue details... STATUS | With all editing tests, the instructions on the editor do not get hidden when the first block is added, they simply get pushed below the editing space. |
| 3 | SJRK-226 - Getting issue details... STATUS | Empty Image blocks (with no image selected/uploaded) have a thin border around the placeholder image. This is a previously known issue. |
| 4 | SJRK-184 - Getting issue details... STATUS | Captured images in iOS continue to be duplicated. This is a previously known issue. |
| 5 | SJRK-336 - Getting issue details... STATUS | The Orator highlights the first word, but then highlighting doesn't continue to follow the reading. |
| 6 | SJRK-327 - Getting issue details... STATUS | Safari 13.0.2 on MacBook Pro can't reach buttons with tab, i.e. they don't seem to be in the tab order. |
| 7 | SJRK-332 - Getting issue details... STATUS | Audio capture option is not presented, can only upload. Likely not much to be done about this, as it may depend on Firefox not sending the same Android Intents as Chrome. |
| 8 | FLUID-6481 - Getting issue details... STATUS | The Orator works, but does not play more than one time. |
| 9 | SJRK-331 - Getting issue details... STATUS | The Orator isn't working in Edge 44, though it does seem to work on older versions of the tool and the build page. Maybe related to Infusion Potentia II updates. |
| 10 | SJRK-306 - Getting issue details... STATUS | IE 11 doesn't work at all, the page won't load properly, Array.prototype.find() method doesn't exist in Trident. |
| 11 | File uploading is mostly not working in Firefox 68 on Pixel 3, only tests 5 and 11 passed for some reason. |