Acessible Assessment

Work in Progress...

Overview:

This page aims to develop design guidelines for building accessible assessment tools for OER (Open Education Resources) authors.

A key feature of accessible assessments will be the use of laptops, desktop computers and touch screen tablets/surfaces for delivery of tests and collection of student responses. Traditionally, assessments have been altered extensively for students who have disabilities, however, computer-delivered accessible assessments aim to minimize these exceptions or special arrangements where possible and provide accommodations and supports to all students who need them without relying on teachers to provide them. ^*3 Computer-aided assessments cover a broad range of tools from automated multiple-choice type questions to the use of media, simulations and virtual laboratory spaces for testing students. Thus, where sound, vision, dexterity, the ability to spell words ‘correctly’ and speedy response times can affect the outcome of assessments; accessibility issues must also be considered and explored. ^*8

General Guidelines For Accessible Assessments:

Focus on what the student is being tested for, not the mechanics of responding to a test item. You can ask the following questions to clarify the test item's intentions: ^*5
- What an item is attempting to measure?
- What elements of the item are not essential in the measurement of skill and knowledge?
- What information can we present in alternate ways so that we can make it easier for students to access content, be engaged with it and reveal what they understand?
Provide text alternative (e.g. large print, braille, speech, symbols or simpler language) for any non-text content item, except in the following situation: ^*6
- If non-text content is a test or exercise that would be invalid if presented in text. In this case, text alternative should provide a descriptive identification of the non-text content
Make assessment items adaptable, so they can be presented in different ways without losing information or structure. ^*6
- Make sure information, structure and relationships conveyed through presentation can be programmatically determined or available in text.
- If the sequence in which content is presented affects its meaning, determine a correct reading sequence
- Content should not rely solely on sensory characteristics of the component such as shape, size, visual location, orientation, or sound
Provide users enough time to read and use content ^*6
- For items that have a time limit set by the content, enable users to turn it off, adjust it or extend it.
- One of the exception is when the time limit is essential and extending it would invalidate the activity
Provide ways to help users navigate, find content, and determine where they are ^*6
- Bypass blocks of text that are repeated on multiple pages
- Include titles to describe the purpose of each page
- If the sequence affects the meaning or operation, focusable components should receive focus in an order that preserves meaning and operability
- Include more than one way to locate a page within a set of pages except where the page is a result of or a step in a process
If an an error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the validity of the test ^*6
To prevent errors, for pages that submit user test responses at least one of the following is true: ^*6
- Submissions are reversible
- Data entered by user is checked for input errors and the user is provided an opportunity to correct them
- A mechanism is available for reviewing, confirming, and correcting information before finalizing the submission
Content should be robust enough that can be interpreted reliably by a wide variety of user agents, including assistive technologies ^*6
Be aware of that computer-based testing places more demands on certain skills such as typing, mouse navigation, and the use of key combinations which may not be accessible for many users
Be aware that using multiple screens to recall a passage requires greater mental efforts by test takers
Be aware that some people become more fatigued when reading text on a computer screen than on paper specially reading long passages on the screen
The inability to see an entire problem on screen at one time is challenging for many users.
The speech recognition technology may not be an accessible alternative for collecting user responses, especially for those whose native language is not Engilsh or those with speech impairments, can be frustrated by the software’s lack of ability to differentiate many of the sounds that they make. ^*7
Calculator use is often allowed on paper/pencil tests when arithmetic is not the construct being measured. However, standardization of the type of calculator used has been very difficult and would be much easier if all students had the same online calculator to use. Use of an online calculator is challenging for some students, especially if they have not had practice with this tool in their daily work. ^*7
Capacity to have instructions repeated as often as student chooses ^*7
Maintaining place and saving completed responses during breaks ^*7
Capacity to turn off monitor/ blank screen temporarily ^*7
Test security ^*7
Capacity for self-selection of subtest order ^*7
The student must be familiar with using the technology itself. Introducing assistive technology to students with an exam deadline looming would place an added learning cost on the student ^*8
Allow students to set their own transition times between questions (but bear in mind that extra time may make the total exam burden more onerous) ^*8
if you use assistive technology to make the assessment more accessible, ask yourself if the student is being required to learn to use new technology at short notice, and whether this constitutes fair treatment. ^*8
Alternative text or “alt tags” for images

Functional Considerations For Designing Accessible Assessments:

Color is not used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. ^*6
If any audio on a Web page plays automatically for more than 3 seconds, either a mechanism is available to pause or stop the audio, or a mechanism is available to control audio volume independently from the overall system volume level. ^*6
Except for captions and rendered text, text can be resized without assistive technology up to 200 percent without loss of content or functionality and does not require the user to scroll horizontally to read a line of text on a full screen window. ^*6
- When type is enlarged on a screen, students may need to scroll back and forth, or up and down to read an entire test item. Graphics, when enlarged, may become very pixilated and difficult to view. Students who use hand held magnifiers or monocular devices when working on paper may not be able to use these devices on a screen because of the distortion of computer images. If a graphics user interface is used (versus text based), students will not have the option of altering print size on the screen. ^*7
Try to avoid background audio. If you need background audio, give users control to turn it off or at least make it four times quieter than the foreground speech content (20 decibels lower) ^*6
Foreground and background colors can be selected by the user. ^*6
For blocks of text, make sure the width is no more than 80 characters or glyphs (40 if CJK), text is not justified (aligned to both the left and the right margins), and line spacing (leading) is at least space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing. ^*6
Make all functionality available from a keyboard ^*6
When an authenticated session expires, the user can continue the activity without loss of data after re-authenticating. ^*6

- Technology is required for multiple test sessions that would allow individual students to submit their completed responses and be able to log out and back on again at another time, starting at the place where they previously left off. Careful scheduling is needed for multiple test sessions to make sure that computers are available. Test security becomes an issue if students who have responded to the same test items have opportunities to interact with each other between test sessions. ^*7

For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause , stop, or hide it unless the movement, blinking, or scrolling is part of an activity where it is essential ^*6
For any auto-updating information that (1) starts automatically and (2) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it or to control the frequency of the update unless the auto-updating is part of an activity where it is essential. ^*6
When any component receives focus, it does not initiate a change of context ^*6
Changing the setting of any user interface component does not automatically cause a change of context unless the user has been advised of the behavior before using the component. ^*6
Navigational mechanisms that are repeated on multiple pages within a set of pages occur in the same relative order each time they are repeated, unless a change is initiated by the user. ^*6
Including accommodations such as screen magnification, text to speech audio, reverse contrast, and switch systems^*3
Provide a script for sign language interpreters, so they don’t “give away” some answers
Avoidance of complex backgrounds that interfere with readability of overlying text ^*7
Capacity for any student to select calculator or dictionary option ^*7

Assessment Types:

Type	Definition	Context	Accessibility Issues to Consider
Low Stake Assessment	Low-stakes assessment is a form of assessment encompassed by the immediate process of learning, often in a very short feedback loop, such as exercises or quizzes. Sometimes this is called "formative" assessment or even just "feedback". The essential characteristics are immediacy and the lack of serious consequences contingent on performance. ^*1	In classrooms In groups Tutoring sessions Informal learning sessions	Providing equivalent content modalities and access mechanisms wherever possible to make the system and content flexible and adaptable for different needs ^*1
Formative Assessment	The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to recognize where students are struggling and address problems immediately and by students to identify their strengths and weaknesses and target areas that need work. Formative assessments are generally low stakes, which means that they have low or no point value. Examples of Formative Assessments include: draw a concept map in class to represent their understanding of a topic submit one or two sentences identifying the main point of a lecture turn in a research proposal for early feedback ^*2	Similar to Low Stake Assessments	Similar to Low Stake Assessments
High Stake Assessment	High-stakes assessment has consequences that may make a serious impact on the life-course of the participant. An example might be a university entrance examination. It is important that a high-stakes assessment be fair to all candidates and not offer advantages to one group over another. ^*1 e.g. University Entrance or Exit Examinations, Test of English as a Foreign Language (TOEFL)	In auditoriums In classrooms Controlled individual kiosks/spaces Group auditions Individual's desired space (For projects that are submitted to the teacher online or in person)	Time requirements to complete the assessment (Addressing the automation issue) Cognitive load Required efforts Fatigue Grouping arrangements Use of earphones or headphones Use of individual setting if response method distracts other students Availability/comparability/location of computers and peripherals Glare from windows or overhead lights Adaptive furniture Test security
Summative Assessment	The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. Summative assessments are often high stakes, which means that they have a high point value. Examples of summative assessments include: a midterm exam a final project a paper a senior recital Information from summative assessments can be used formatively when students or faculty use it to guide their efforts and activities in subsequent courses. ^*2	Similar to High Stake Assessments	Similar to High Stake Assessments
Self Assessment	Assessment or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard.	Individual's desired space At work	Accessible assessment tools and systems that are flexible and adapted to user needs Time requirements Cognitive load

Assessment Tools:

Tool	Description	Required capabilities to operate task	Accessibility Issues	Alternative Methods
Multiple Choice	A traditional multiple choice question (or item) is one in which a student chooses one answer from a number of choices supplied. ^*4	Cognitive	May not be easily accessible onscreen depending on the nature of the task (In shown example, the use of visual elements makes it inaccessible via alternative methods) For visually-impaired students who rely on text-to-speech software, remembering a variety of spoken information can require considerable mental effort ^8 students who rely on text-to-speech software require greater time to acquire and digest information than students who can visually read data ^8 MCQs often offer very subtle distinctions between choices. Visually-impaired students are likely to need longer to determine these directions. A dyslexic student may be unable to identify such subtle distinctions ^8 MCQs that address higher-order understanding rather than surface learning are very complex and can require numerous re-readings. This has the potential to disadvantage the visually impaired student and those using text-to-speech software ^8 Progressing between options using the tab key (for students who cannot use a mouse) can take up a great deal of additional time. ^*8	Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content Capacity for multiple options for selecting response – mouse click, keyboard, touch screen, speech recognition, assistive devices to access the keyboard (e.g., mouth stick or head wand) Option for paper/pencil in addition to computer (e.g., scratch paper for solving problems, drafting ideas) Try to avoid overly-complex answers, especially long lists with distinctions that can only be made by careful re-readings. ^8 if using graphics, ensure that high-quality recorded descriptions are available for each graphic used or that a narrative is included with video clips to describe any action taking place. ^8
Assertion Reason	The assertion-reason item combines elements of multiple choice and true/false question types, and allows you to test more complicated issues and requires a higher level of learning. The question consists of two statements, an assertion and a reason. The student must first determine whether each statement is true. If both are true, the student must next determine whether the reason correctly explains the assertion. There is one option for each possible outcome. ^*4	Cognitive	Layout issues High Cognitive load
True/False	A true-false questions is a specialized form of the multiple-choice format in which there are only two possible alternatives. These questions can be used when the test-designer wishes to measure a student's ability to identify whether statements of fact are accurate or not. ^*4	Cognitive		Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Multiple Select	This item allows user to choose more than one response option. ^*3	Cognitive	May not be easily accessible onscreen depending on the nature of the task (In shown example, the use of visual elements makes it inaccessible via alternative methods)	Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Labeling (Graphic)	The graphic labeling task requires user to drag-and-drop labels to the correct location on an image or diagram (e.g. Venn Diagram). This type of question can be manipulated on a computer with a mouse or trackball or on a touch screen tablet. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with radio buttons, matrix, or click to select interactions
Labeling (One to One)	For this drag and drop task, the user selects a label from a list at the top of the screen and drags it to the correct location on the screen. Once the label has been used, it cannot be reused unless it is first moved back to the top of the screen. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Labeling (Unlimited)	This item is similar to one-to-one labeling except that labels can be reused. The user clicks or touches the label and drags it to the correct location. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Matching	With matching, the student matches each label with its correct response by clicking or touching one element in each column in turn. When the option in the second column is chosen, a line appears marking the connection between the two. Each option can be paired only once. ^*3	Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Matrix	The matrix item type allows the user to assign elements to categories or groups. This item duplicates the tasks required in categorization. ^*3	Cognitive	Scrolling issues Variations in screen size Effects of magnification on graphics and table size	Can be printed and transcribed into braille May be accessible using text-to-speech screen readers depending on the content May be accessible to one-and two-switch systems depending on the content
Categorization	The categorization item requires the user to organize elements into categories. There are two interfaces for this item type. In the drag-and-drop interface, the user clicks or touches an element and drags it into the correct category using a mouse or touch screen. With the click-to-select interface, the user clicks or selects an element to select it. Then, the user clicks or touches the category label, and the element appears in that category. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Ordering/ Ranking	Ordering/Ranking/Sequencing is a drag-and-drop task in which the student must move the elements on the screen to put them in the correct order. ^*3	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Constructed Response (Text/ Numerical)	The text match question requires a student to supply an answer to a question or complete a blank within a brief piece of text, using words, symbols or numbers. ^*4	Motor skills Cognitive	Use of a spellcheck when achievement in spelling is being measured as part of the writing process What if the correct words appear but are misspelled? Are we in danger of measuring spelling rather than understanding? This issue can be applied equally to all students of course, but it holds a particular relevance for dyslexic students.
Select Text/ Sore finger	A word/ Code/ Phrase is out of keeping with the rest of package. ^4 This item requires user to select one or more words, phrases, or sentences from a passage. When user selects an option, its appearance changes to verify that it has been selected. ^3	Cognitive	Implications of the use of a screen reader when the construct being measured is reading
Interactive	Includes videos, animations, coloring, graphing, games, puzzles or any other type of question that requires high levels of interaction between user and the interface.	Vision Motor skills Cognitive	Actions like dragging and dropping elements onscreen are inaccessible Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches	Replace those actions with accessible alternatives like radio buttons or click to select interactions
Field Simulations	Offers simulations of real problems or exercises ^*4	Vision Cognitive	Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen Difficult to transcribe into braille or print Difficult for text-to-speech screen readers Inaccessible to switches

Available Accessible Tools:

Please test the following tools and add your comments to this page.

Tool's Name	Description	Link
Nimbletools	NimbleTools is designed for use by general education students and students with special needs. This means that a single version of a test can be created and delivered to students across a school, district, or state in an efficient and cost-effective manner. The accessibility and accommodation tools embedded within NimbleTools also assure that students with disabilities and special needs are provided with appropriate accommodations in a standardized and controlled manner.	http://nimbletools.com/demo/index.htm