Acessible Assessment

Work in Progress...

This page aims to develop design guidelines for building accessible assessment tools for OER (Open Education Resources) authors. Different assessment types will be investigated and accessible alternatives will be suggested.

* Please note that this is a work in progress and the following structure is not final until the research is completed.

Overview:

A key feature of computerized assessments will be the use of laptop and desktop computers as well as touch screen tablets or surfaces for delivery of tests and collection of student responses. Traditionally, assessments have been altered extensively for students who have disabilities. Accommodations have included alternate forms, such as Braille or large print paper-and-pencil tests and alternate means of presentation and response, such as the use of readers, scribes and assistive technology. One of the major goals of the accessible assessments is to generate and disseminate guidelines and recommendations for accessibility and equitable test delivery. Ideally, computer-delivered accessible tests will provide accommodations and supports to all students who need them without relying on teachers to provide them. ^*3

General Guidelines For Accessible Assessments:

Focus on what the student is being tested for, not the mechanics of responding to a test item. You can ask the following questions to clarify the test item intentions: ^*5
1. What do I want to know?
2. Why do I want to know it?
3. How can I discover this information?
4. What do I do with what I discover?
Provide text alternative (e.g. large print, braille, speech, symbols or simpler language) for any non-text content item, except in the following situation: ^*6
1. If non-text content is a test or exercise that would be invalid if presented in text. In this case, text alternative should provide a descriptive identification of the non-text content
Make assessment items adaptable, so they can be presented in different ways without losing information or structure. ^*6
1. Make sure information, structure and relationships conveyed through presentation can be programmatically determined or available in text.
2. If the sequence in which content is presented affects its meaning, a correct reading sequence can be programmatically determined
3. Content should not rely solely on sensory characteristics of the component such as shape, size, visual location, orientation, or sound
Provide users enough time to read and use content ^*6
1. For items that have a time limit set by the content, enable users to turn it off, adjust it or extend it.
2. One of the exception is when the time limit is essential and extending it would invalidate the activity
Provide ways to help users navigate, find content, and determine where they are ^*6
1. Bypass blocks of text that are repeated on multiple pages
2. Include titles to describe the purpose of the page
3. If the sequence affects the meaning or operation, focusable components should receive focus in an order that preserves meaning and operability
4. Include more than one way to locate a page within a set of pages except where the page is a result of or a step in a process
If an an error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the security or purpose of the content. ^*6
To prevent errors, for pages that submit user test responses at least one of the following is true: ^*6
1. Submissions are reversible
2. Data entered by user is checked for input errors and the user is provided an opportunity to correct them
3. A mechanism is available for reviewing, confirming, and correcting information before finalizing the submission
Content should be robust enough that can be interpreted reliably by a wide variety of user agents, including assistive technologies ^*6

Functional Considerations For Designing Accessible Assessments:

Color is not used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. ^*6
If any audio on a Web page plays automatically for more than 3 seconds, either a mechanism is available to pause or stop the audio, or a mechanism is available to control audio volume independently from the overall system volume level. ^*6
Except for captions and rendered text, text can be resized without assistive technology up to 200 percent without loss of content or functionality and does not require the user to scroll horizontally to read a line of text on a full screen window. ^*6
Try to avoid background audio. If you need background audio, give users control to turn it off or at least make it four times quieter than the foreground speech content (20 decibels lower) ^*6
Foreground and background colors can be selected by the user. ^*6
For blocks of text, make sure the width is no more than 80 characters or glyphs (40 if CJK), text is not justified (aligned to both the left and the right margins), and line spacing (leading) is at least space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing. ^*6
Make all functionality available from a keyboard ^*6
When an authenticated session expires, the user can continue the activity without loss of data after re-authenticating. ^*6
For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause , stop, or hide it unless the movement, blinking, or scrolling is part of an activity where it is essential ^*6
For any auto-updating information that (1) starts automatically and (2) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it or to control the frequency of the update unless the auto-updating is part of an activity where it is essential. ^*6
When any component receives focus, it does not initiate a change of context ^*6
Changing the setting of any user interface component does not automatically cause a change of context unless the user has been advised of the behavior before using the component. ^*6
Navigational mechanisms that are repeated on multiple pages within a set of pages occur in the same relative order each time they are repeated, unless a change is initiated by the user. ^*6
Including accommodations such as screen magnification, text to speech audio, reverse contrast, and switch systems ^*3

Assessment Types:

Type	Definition	Context	Accessibility Issues to Consider
Low Stake Assessment	Low-stakes assessment is a form of assessment encompassed by the immediate process of learning, often in a very short feedback loop, such as exercises or quizzes. Sometimes this is called "formative" assessment or even just "feedback". The essential characteristics are immediacy and the lack of serious consequences contingent on performance. ^*1	In classrooms In groups Tutoring sessions Informal learning sessions	Providing equivalent content modalities and access mechanisms wherever possible to make the system and content flexible and adaptable for different needs ^*1
Formative Assessment	The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to recognize where students are struggling and address problems immediately and by students to identify their strengths and weaknesses and target areas that need work. Formative assessments are generally low stakes, which means that they have low or no point value. Examples of Formative Assessments include: draw a concept map in class to represent their understanding of a topic submit one or two sentences identifying the main point of a lecture turn in a research proposal for early feedback ^*2	Similar to Low Stake Assessments	Similar to Low Stake Assessments
High Stake Assessment	High-stakes assessment has consequences that may make a serious impact on the life-course of the participant. An example might be a university entrance examination. It is important that a high-stakes assessment be fair to all candidates and not offer advantages to one group over another. ^*1 e.g. University Entrance or Exit Examinations, Test of English as a Foreign Language (TOEFL)	In auditoriums In classrooms Controlled individual kiosks/spaces Group auditions Individual's desired space (For projects that are submitted to the teacher online or in person)	Time requirements to complete the assessment (Addressing the automation issue) Cognitive load Required efforts Fatigue
Summative Assessment	The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. Summative assessments are often high stakes, which means that they have a high point value. Examples of summative assessments include: a midterm exam a final project a paper a senior recital Information from summative assessments can be used formatively when students or faculty use it to guide their efforts and activities in subsequent courses. ^*2	Similar to High Stake Assessments	Similar to High Stake Assessments
Self Assessment	Assessment or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard.	Individual's desired space At work	Accessible assessment tools and systems that are flexible and adapted to user needs Time requirements Cognitive load

Assessment Tools:

Tool	Description	Required capabilities to operate task	Accessibility Issues	Alternative Methods
Multiple Choice	A traditional multiple choice question (or item) is one in which a student chooses one answer from a number of choices supplied. ^*4	Cognitive	May not be easily accessible onscreen depending on the nature of the task (In shown example, the use of visual elements makes it inaccessible via alternative methods)	Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
ssertion Reason	The assertion-reason item combines elements of multiple choice and true/false question types, and allows you to test more complicated issues and requires a higher level of learning. The question consists of two statements, an assertion and a reason. The student must first determine whether each statement is true. If both are true, the student must next determine whether the reason correctly explains the assertion. There is one option for each possible outcome. ^*4	Cognitive
True/False	A true-false questions is a specialized form of the multiple-choice format in which there are only two possible alternatives. These questions can be used when the test-designer wishes to measure a student's ability to identify whether statements of fact are accurate or not. ^*4	Cognitive		Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Multiple Select	This item allows user to choose more than one response option. ^*3	Cognitive	May not be easily accessible onscreen depending on the nature of the task (In shown example, the use of visual elements makes it inaccessible via alternative methods)	Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Labeling (Graphic)	The graphic labeling task requires user to drag-and-drop labels to the correct location on an image or diagram (e.g. Venn Diagram). This type of question can be manipulated on a computer with a mouse or trackball or on a touch screen tablet. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with radio buttons, matrix, or click to select interactions
Labeling (One to One)	For this drag and drop task, the user selects a label from a list at the top of the screen and drags it to the correct location on the screen. Once the label has been used, it cannot be reused unless it is first moved back to the top of the screen. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Labeling (Unlimited)	This item is similar to one-to-one labeling except that labels can be reused. The user clicks or touches the label and drags it to the correct location. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Matching	With matching, the student matches each label with its correct response by clicking or touching one element in each column in turn. When the option in the second column is chosen, a line appears marking the connection between the two. Each option can be paired only once. ^*3	Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Matrix	The matrix item type allows the user to assign elements to categories or groups. This item duplicates the tasks required in categorization. ^*3	Cognitive		Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Categorization	The categorization item requires the user to organize elements into categories. There are two interfaces for this item type. In the drag-and-drop interface, the user clicks or touches an element and drags it into the correct category using a mouse or touch screen. With the click-to-select interface, the user clicks or selects an element to select it. Then, the user clicks or touches the category label, and the element appears in that category. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Ordering/ Ranking	Ordering/Ranking/Sequencing is a drag-and-drop task in which the student must move the elements on the screen to put them in the correct order. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Constructed Response (Text/ Numerical)	The text match question requires a student to supply an answer to a question or complete a blank within a brief piece of text, using words, symbols or numbers. ^*4	Motor skills Cognitive
Select Text/ Sore finger	A word/ Code/ Phrase is out of keeping with the rest of package. ^4 This item requires user to select one or more words, phrases, or sentences from a passage. When user selects an option, its appearance changes to verify that it has been selected. ^3	Cognitive
Interactive	Includes videos, animations, coloring, graphing, games, puzzles or any other type of question that requires high levels of interaction between user and the interface.	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Field Simulations	Offers simulations of real problems or exercises ^*4	Vision Cognitive	Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches