This page contains the protocol for Universal Subtitles user testing.
...
| Panel | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
Hi [user's name]. I'm [your title] with the Floe project. Today we are looking for ways to improve the user experience of Universal Subtitles, a tool on the web used to add subtitles to any video. This is a test of Universal Subtitles design; we are not testing you. If you find something difficult or unintuitive to use, chances are that others will as well. This test of the Universal Subtitles is simply a means of evaluating the design and to discover any issues we need to address. The study is composed of four parts: we'll start with the main part of the study where you get a chance to play with Universal Subtitles. Then you will be asked to perform a few specific tasks. Afterward, we'll have a short interview about your experience, and wrap that up with a quick demographics questionnaire. The study will take about 45 minutes, and you can stop at any time during the study for any reason. Do you have any questions before we start? |
1. User Testing
1A. Undirected testing
Instructions for user (script)
There are a few browser windows open here, each on a video clip. Go ahead and watch the clips, and select one that you'd like to subtitle.
...
By looking at the screen in front of you, what can you surmise about how you should add subtitles to your video? What is your expectation of how this works?
Instructions for moderator
Give the user a selection of two videos to view (make sure that these two videos have not been subtitled before). After the user has selected a video to subtitle, send the user to the Universal Subtitles home page. Ask the user to go ahead and work toward subtitling the video they selected.
...
Ask the user to go ahead and subtitle the video. While the user is paused/typing, probe the user's thoughts. Avoid probing the user while the video is playing.
1B. Directed testing
Instructions for user (script)
...
- How would you describe your experience subtitling using Universal Subtitles overall?
- What did you find confusing or difficult to understand about subtitling?
- What did you find difficult or frustrating?
- What might we do to improve the application?
3. Post-Test Questionnaire (Demographics)
General demographics
1. What is your gender?
...