Sakai User Experience Walkthrough Report

(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)

Evaluation Completed by: [Insert evaluators name]

URL: http://bspace.berkeley.edu/portal/site/24cc0997-e57a-4f55-00ca-02eb0b35a6e7

Date: 22 Oct 07

Heuristic reviews and cognitive walkthroughs were performed on a [Dell] at [screen resolution] resolution (most common resolution) and [color setting] colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using [Firefox and Win XP Pro SP2] at a [broadband] connection rate.

See UX Walkthrough Heuristics for heuristics used in this evaluation.

Scope of Walkthrough

• See UX Walkthrough Work Breakdown

User profile(s) and context of use:

Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.

• See Sakai Personas

Scenarios

Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application. See Content Management Scenariosfor a list of all possible scenarios. Evaluators will likely only cover some of the scenarios, in one of the roles, in their individual evaluations.

•    General Overview (heuristic evaluation)
•    Scenario 2
•    Scenario 3
•    Scenario 4
•    Scenario 5

Assumptions for this evaluation:

• [Performed based on an instructor]

Positive Findings

All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section.

Usability Positives	Tool	Evaluator
Clicking the help icon takes you directly to the help section for the tool it was called from.	all	Justin








Accessibility Positives	Tool	Evaluator

Summary of Usability & Accessibility Issues Found

All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section.

Priority Legend:

High = Task cannot be completed

Medium = Task completed with significant effort and failed attempts

Low = Task completed with minor complications and/or annoyance

Usability Issues	Principle	Suggestions for solution	Tool	Component Identified?
General
Use of the term "Reset" for the mechanism to take the user back to the first page of the tool. Reset is often used to refer to clearing the information entered and not just returning to a previous location.	Consistency and standards	A term such as "Return" may be more appropriate	All
Syllabus
Complicated work flow if the instructor just wants to add a file as the Syllabus. (Syllabus > create/edit > add > add attachment > upload file	Flexibility and efficiency of use, Match between system and the real world	Add an option along side create/edit to add a file. (System should probably also be able to display the contents of the file and not just a link)	Syllabus
The Add Syllabus form and editor do not display the same as the actual output. The style of bullets changes from the editor to the output. Also the preview shows the Title in a different font size and colour than is actually outputed.	Consistency and standards	The preview should be identitcal in appearance to the output on the site. The user may also like to be able to control the style of the Title.	Syllabus	Add Syllabus
In the create/edit screen of the Syllabus tool, the "Preview" and "Reset" link perform the same actions.	Consistency and standards, Aesthetic and minimalistic design	If there are no other functions for the "Preview" link to that what is performed by the "Reset" link, "Preview" should be removed.	Syllabus	create/edit
Redirecting a syllabus prevents other posted syllabus from being displayed. This is the case even after adding a new syllabus and verifying it's output with preview.	Error prevention, visibility of system status	The system should inform the user that a redirected website will block all other syllabus, or display all of them sequentially.	Syllabus	Redirect
To remove a redirected syllabus (web site) the user has to erase the url and push the save button. For all other syllabus the user can remove after it by going to create/edit, which isn't even viewable if a redirect has been used.	User control and freedom, consistency and standards, error prevention	Provide an item in the syllabus list (create/edit) so that all of the syllabus can be managed in a like manor.	Syllabus	Redirect, create/edit
When re-ordering syllabus in the Syllabus Item list, the user does not need to click the update button for the changes to take effect. Update is only needed for removal.	Error prevention, User control and freedom, Visibility of system status	Require that the user perform an update before the reordering takes effect, or rename the button to "Remove". Also disablling the button until an update/removal is needed will signal to the user when it is needed.	Syllabus	Syllabus
Home
On the "Home" page, clicking "Options" under Worksite Information allows you to add text. However, clicking on "Options" under Recent Announcements provides no ability to add text.	Consistency and standards	Provide a link to the announcements editing page.	Home	Recent Announcements
The "Title" and "URL" text fields are both smaller in width than the "Description" text area	Consistency and standards	The text fields do not limit the input to their size, so they should be equal in width to the Description text area, allowing for maximum visibility of information.	Home	Worksite Information > Options
Site Info
Should an instructor be able to add/remove users and change their role? Is this more of an admin function?	Aesthetic and minimalist design	Remove these features from the instructor, or put them under a subheading	Site Info
The Edit Tool and Tool Order links have the same function but different implementations of it (with the exception of the ordering option for tool order)	Consistency and standards, Aesthetic and minimalist design	They should probably be combined or have the titles changed so that one indicates for keyboard use and the other for mouse.	Site Info	Edit Tools, Tool Order
Slow to add/remove tools	Visibility of system status	There is a message displayed after the tool has been added/removed. There should also be a message indicating that the process is in progress.	Site Info	Tool Order
The user can drag to add tools but can't drag to remove tools	Consistency and standards	The user should be able to drag the tool off of the column and drop it either onto the available tools column or drop anywhere off the current column to remove.	Site Info	Tool Order
Removed tools do not show up in available tools until after a save is performed.	Visibility of system status	Tools should show up in the available tools list as soon as they are removed from the list of current tools.	Site Info	Tool Order
The "Reset" button is used to put the order of the tools to the default ordering. The term "Reset" is also used at the top as a means to return the tool to it's start page.	Consistency and standards	One or both of these items should be renamed. The "Reset" button could be renamed to "Default Order".	Site Info	Tool Order
Assignments
The descriptive text is unclear as to whether is is describing the combo boxes above or below it.	Error prevention	Information in the Assignment form needs to be borken up by white space to better seperate the sections.	Assignments	Add
Polls
When adding a poll, the user has to add the question and all possible answers on different pages. The only item that persists across pages is the question. If the user forgets what answers were already entered, they have to save, leaven and exit.	Recognition rather than recall	The user should be able to add the question and all possible answers on the same page.	Polls	Add
Schedule
When setting the time for an event, the time zone is displayed after the time combo boxes. However, the user is unable to change the time zone.	Aesthetic and minimalist design	The time zone seems to be set by the server, if the user has no control over it, it should not be displayed here.	Schedule	New Event
The legend below the Calendar is quite large and contains many items that may never appear on any one calendar all at a time.	Aesthetic and minimalist design	It would be better to only include those items which are currently visible in the calendar. Another solution would be to have a tooltip for the icon rather than include the icon in the event's title.	Schedule	Calendar
To scroll through the hours on the calendar, the user has to click on the links for earlier and later.	Consistency and standards	Users are often more familiar with using a scrollbar.	Schedule	Calendar

Accessibility Issue	Suggestions for solution	Tool
General
It isn't always clear which item is selected when tabbing through a page. The selection bounding box is not always visible.
After selection of a tool from the left navigation bar, the tab traversal resets and cycles back from the start.	It would be more helpful to the user if the tab traversal starts inside of the tool's frame	all

Justin's Walkthrough