Justin's Walkthrough
Sakai User Experience Walkthrough Report
(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)
Evaluation Completed by: [Insert evaluators name]
URL: http://bspace.berkeley.edu/portal/site/24cc0997-e57a-4f55-00ca-02eb0b35a6e7
Date: 22 Oct 07, 29 Oct 07
Heuristic reviews and cognitive walkthroughs were performed on a [Dell] at [screen resolution] resolution (most common resolution) and [color setting] colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using [Firefox and Win XP Pro SP2] at a [broadband] connection rate.
See UX Walkthrough Heuristics for heuristics used in this evaluation.
Scope of Walkthrough
• See UX Walkthrough Work Breakdown
User profile(s) and context of use:
Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.
• See Sakai Personas
Scenarios
Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application. See Content Management Scenariosfor a list of all possible scenarios. Evaluators will likely only cover some of the scenarios, in one of the roles, in their individual evaluations.
• General Overview (heuristic evaluation)
• Scenario 2
• Scenario 3
• Scenario 4
• Scenario 5
Assumptions for this evaluation:
• [Performed based on an instructor]
Positive Findings
All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section.
Usability Positives | Tool | Evaluator |
|---|---|---|
Clicking the help icon takes you directly to the help section for the tool it was called from. | all | JO |
Sortable tables show an arrow icon beside a heading. This shows both the method of sort (ascending/descending) and the heading which for which the sort is based on. | all sortable tables (i.e. announcements and resources) | JO |
Users are able to include only the tools they want for their site |
| JO |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Accessibility Positives | Tool | Evaluator |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Summary of Usability & Accessibility Issues Found
All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section.
Priority Legend:
High = Task cannot be completed
Medium = Task completed with significant effort and failed attempts
Low = Task completed with minor complications and/or annoyance
Usability Issues | Principle | Link to screen shots | Priority | Suggestions for solution | Tool | Component Identified? |
|---|---|---|---|---|---|---|
General |
|
|
|
|
|
|
Use of the term "Reset" for the mechanism to take the user back to the first page of the tool. Reset is often used to refer to clearing the information entered and not just returning to a previous location. | Consistency and standards |
|
| A term such as "Return" may be more appropriate | All |
|
There are three methods to organize content. Method 1 uses up/down arrows seen in the Syllabus, Method 2 uses drag and drop seen in the Site Info, and Method 3 uses numbers in a combo box seen in the Forums | Consistency and standards |
|
| Choose one or at most two methods of organization. | Syllabus, Site Info, Forums | Syllabus Items, Tool Order, Organize Forums and Topics |
Should the instructor be able to set permissions. Is this more of an admin function? | Aesthetic and minimalist design |
|
| Remove from the instructor's view | Announcements, | Permissions |
Both the Question Pools and Resources pages are used to manage a hierarchical system of information, but are designed with a different look and feel. | Consistency and standards |
|
| Choose one method to be used in all necessary areas | Resources, Quiz | Site Resources, Question Pools |
The horizontal navigation bar used for navigation within a tool is different across the tools, some user lighter text to show selection, others use darker, some are have vertical bars to separate items, others do not, some have no underline for the selected item, and etc. | Consistency and standards |
|
| Choose one method to be used in all necessary areas | All |
|
Syllabus |
|
|
|
|
|
|
Complicated work flow if the instructor just wants to add a file as the Syllabus. (Syllabus > create/edit > add > add attachment > upload file | Flexibility and efficiency of use, Match between system and the real world |
|
| Add an option along side create/edit to add a file. (System should probably also be able to display the contents of the file and not just a link) | Syllabus | |
The Add Syllabus form and editor do not display the same as the actual output. The style of bullets changes from the editor to the output. Also the preview shows the Title in a different font size and colour than is actually outputed. | Consistency and standards |
|
| The preview should be identitcal in appearance to the output on the site. The user may also like to be able to control the style of the Title. | Syllabus | Add Syllabus |
In the create/edit screen of the Syllabus tool, the "Preview" and "Reset" link perform the same actions. | Consistency and standards, Aesthetic and minimalistic design |
|
| If there are no other functions for the "Preview" link to that what is performed by the "Reset" link, "Preview" should be removed. | Syllabus | create/edit |
Redirecting a syllabus prevents other posted syllabus from being displayed. This is the case even after adding a new syllabus and verifying it's output with preview. | Error prevention, visibility of system status |
|
| The system should inform the user that a redirected website will block all other syllabus, or display all of them sequentially. | Syllabus | Redirect |
To remove a redirected syllabus (web site) the user has to erase the url and push the save button. For all other syllabus the user can remove after it by going to create/edit, which isn't even viewable if a redirect has been used. | User control and freedom, consistency and standards, error prevention |
|
| Provide an item in the syllabus list (create/edit) so that all of the syllabus can be managed in a like manor. | Syllabus | Redirect, create/edit |
When re-ordering syllabus in the Syllabus Item list, the user does not need to click the update button for the changes to take effect. Update is only needed for removal. | Error prevention, User control and freedom, Visibility of system status |
|
| Require that the user perform an update before the reordering takes effect, or rename the button to "Remove". Also disablling the button until an update/removal is needed will signal to the user when it is needed. | Syllabus | Syllabus |
Home | |
| | | | |
On the "Home" page, clicking "Options" under Worksite Information allows you to add text. However, clicking on "Options" under Recent Announcements provides no ability to add text. | Consistency and standards |
|
| Provide a link to the announcements editing page. | Home | Recent Announcements |
The "Title" and "URL" text fields are both smaller in width than the "Description" text area | Consistency and standards |
|
| The text fields do not limit the input to their size, so they should be equal in width to the Description text area, allowing for maximum visibility of information. | Home | Worksite Information > Options |
Site Info |
|
|
|
|
|
|
Should an instructor be able to add/remove users and change their role? Is this more of an admin function? | Aesthetic and minimalist design |
|
| Remove these features from the instructor, or put them under a subheading | Site Info |
|
The Edit Tool and Tool Order links have the same function but different implementations of it (with the exception of the ordering option for tool order) | Consistency and standards, Aesthetic and minimalist design |
|
| They should probably be combined or have the titles changed so that one indicates for keyboard use and the other for mouse. | Site Info | Edit Tools, Tool Order |
Slow to add/remove tools | Visibility of system status |
|
| There is a message displayed after the tool has been added/removed. There should also be a message indicating that the process is in progress. | Site Info | Tool Order |
The user can drag to add tools but can't drag to remove tools | Consistency and standards |
|
| The user should be able to drag the tool off of the column and drop it either onto the available tools column or drop anywhere off the current column to remove. | Site Info | Tool Order |
Removed tools do not show up in available tools until after a save is performed. | Visibility of system status |
|
| Tools should show up in the available tools list as soon as they are removed from the list of current tools. | Site Info | Tool Order |
The "Reset" button is used to put the order of the tools to the default ordering. The term "Reset" is also used at the top as a means to return the tool to it's start page. | Consistency and standards |
|
| One or both of these items should be renamed. The "Reset" button could be renamed to "Default Order". | Site Info | Tool Order |
Assignments |
|
|
|
|
|
|