VULab Project Scope

Project Name: The Open Virtual Usability Laboratory (VULab)

Prepared by: Ron Owston

Date: October 20, 2008

Project Justification/Background

VULab is an open source software tool designed for remote accessibility and usability testing. VULab records, on its server, a video of a remote user's on-screen interactions with a web site together with any user talk. The application also has a built in survey tool that allows researchers to ask users questions at the beginning and end of their usability testing session. The video and survey data collected on the VULab server can be viewed within the tool and exported for detailed analysis to identify accessibility and usability issues with the website. Two unique features of the application that set it apart from other usability testing tools (e.g., Techsmith's Morae) are: (1) that it does not require remote users to install a software application on their computer which facilitates usability testing in public locations such as libraries or school/university labs; and (2) since all data are stored on the server they can secured and backed up on the host server rather than stored on the tester's hard drive.

VULab originated from Dr. Ron Owston's research in the Social Sciences and Humanities Research Council funded Simulation and Advanced Games (SAGE) for Learning Research Network as a tool for assessing the usability of educational games being developed by researchers in the network. A prototype of VULab was developed using a proprietary recording procedure, and the tool proved valuable in providing developers with helpful feedback on their games. Subsequent funding from the Andrew W. Mellon Foundation through the auspices of the Fluid project is allowing the tool to be developed as a fully functioning open source application for use within Fluid and by others who wish to do usability testing.

Project Goal

To develop and field test version 1.0 of VULab.

Project Description

Usability studies in VULab are organized by project. A project consists of a URL to be tested and optional pre- and post-session survey questions. The application consists of two modules: a researcher module and a tester module. In the researcher module all of the parameters of a project are set (e.g., the url to be tested, the pre- and post-survey questions, user directions, the activation and end date of the project); users are managed; and project results viewed. In the tester module users are presented with the pre-session survey, provided with task instructions, and re-directed to the website being tested. As the tester navigates the site being tested, all screen interactions are recorded; and if the user has a microphone and a 'talk-aloud' protocol to follow [PeterR: where would they get this protocol to follow?], all audio is captured. Once the website testing is completed, the tester is re-directed back to VULab where post-session questions are completed. The research is able to video the recordings and survey responses for each tester and export the data for detailed analysis.

Project Deliverables

VULab 1.0

VULab 1.0 will consist of the following components:

Researcher Module:

• user interface that meets Fluid standards for accessibility and usability
• a log in screen that provides self registration and automatic notification to the administrator of a new registration
• a project creation function to set the URL to be tested, the pre- and post-session questions, and assign users
• a survey tool that allows the researcher to set pre- and post-session questions for the tester
• a tester ID management function to create, modify, and delete tester IDs; assign testers to projects; and to automatically notify testers of their ID
• a reporting function that allows the research to view individual tester's survey responses and videos; view group aggregate survey responses; retrieve subsets of test who meet certain search criteria; and export survey responses and videos

Tester Module:

• user interface that is meets Fluid standards for accessibility and usability
• a project status screen that presents all active projects testers are enrolled in and their status i.e., completed pre survey, visited URL, completed post survey, fully completed. Users should have the option to redo a project which will delete all previous answers and video.
• A presentation module that contains the link to the (optional) pre-session questions, instructions giving the user the specific steps required for a given project, automatic re-direction to the URL to be tested, and the post-session questions
• A recording module (called RASCAL) that records audio from the tester's microphone and video of screen interactions. When testers finish the site navigation they are redirected to post survey.
• re-direction to the URL to be tested with a warning that the session is being recorded.
• A completion screen allowing the user to log out or return to do another project.

Release Schedule

1. Release of web interface for testing by the community - Oct 15, 2008
2. Release of mock up of RASCAL recording - Oct 17, 2008
3. Completion of APIs for phpESP survey tool - Oct 31, 2008
4. Integration of RASCAL into web component - Oct 31, 2008
5. Full alpha version of VULab - Nov 20, 2008
6. Testing and bug fixes completed - Dec 31, 2008
7. User management component completed - Jan 15, 2009
8. VULab 0.9 released - Jan 31, 2009
9. Additional testing and bug fixes completed - Feb 28, 2009
10. Version 1.0 released March 31, 2009

Project constraints

The deadline for completion of version 1.0 is March 31, 2009. However, since no tool with the features of VULab has been developed before, considerable research and trial-and-error testing will have to be done and and this makes it hard to accurately predict the time required to accomplish this.

Project assumptions:

The project assumes that remote recording and uploading of the recorded screens can be accomplished with reasonable efficiency so as not to impair the user's interaction with the website being tested.