Work in Progress...
Overview:
This page aims to develop design guidelines for building accessible assessment tools for OER (Open Education Resources) authors. Different assessment types will be investigated and accessible alternatives will be suggested.* Please note that this is a work in progress and the following structure is not final until the research is completed.
Overview:A key feature of computerized accessible assessments will be the use of laptop and laptops, desktop computers as well as and touch screen tablets or /surfaces for delivery of tests and collection of student responses. Traditionally, assessments have been altered extensively for students who have disabilities. Accommodations have included alternate forms, such as Braille or large print paper-and-pencil tests and alternate means of presentation and response, such as the use of readers, scribes and assistive technology. One of the major goals of the accessible assessments is to generate and disseminate guidelines and recommendations for accessibility and equitable test delivery. Ideally, computer-delivered accessible tests will , however, computer-delivered accessible assessments aim to minimize these exceptions or special arrangements where possible and provide accommodations and supports to all students who need them without relying on teachers to provide them. *3 Computer-aided
assessment itself coversassessments cover a broad range of tools from automated multiple-choice type questions to the use of media, simulations and virtual laboratory spaces for testing students. Thus, where sound, vision, dexterity, the ability to spell words ‘correctly’ and speedy response times can affect the outcome of assessments; accessibility issues must also be considered and explored. *8
General Guidelines For Accessible Assessments:
- Focus on what the student is being tested for, not the mechanics of responding to a test item. You can ask the following questions to clarify the test item's intentions: *5
- What
- Why do I want to know it?
- How can I discover this information? What do I do with what I discover
- an item is attempting to measure?
- What elements of the item are not essential in the measurement of skill and knowledge?
- What information can we present in alternate ways so that we can make it easier for students to access content, be engaged with it and reveal what they understand?
- Provide text alternative (e.g. large print, braille, speech, symbols or simpler language) for any non-text content item, except in the following situation: *6
- If non-text content is a test or exercise that would be invalid if presented in text. In this case, text alternative should provide a descriptive identification of the non-
- text content
- Make assessment items adaptable, so they can be presented in different ways without losing information or structure. *6
- Make sure information, structure and relationships conveyed through presentation can be programmatically determined or available in text.
- If the sequence in which content is presented affects its meaning, determine a correct reading sequence
- Content should not rely solely on sensory characteristics of the component such as shape, size, visual location, orientation, or sound
- Provide users enough time to read and use content *6
- For items that have a time limit set by the content, enable users to turn it off, adjust it or extend it.
- One of the exception is when the time limit is essential and extending it would invalidate the activity
- Provide ways to help users navigate, find content, and determine where they are *6
- Bypass blocks of text that are repeated on multiple pages
- Include titles to describe the purpose of
- each page
- If the sequence affects the meaning or operation, focusable components should receive focus in an order that preserves meaning and operability
- Include more than one way to locate a page within a set of pages except where the page is a result of or a step in a process
- If an an error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the security or purpose validity of the content. test *6
- To prevent errors, for pages that submit user test responses at least one of the following is true: *6
- Submissions are reversible
- Data entered by user is checked for input errors and the user is provided an opportunity to correct them
- A mechanism is available for reviewing, confirming, and correcting information before finalizing the submission
- Content should be robust enough that can be interpreted reliably by a wide variety of user agents, including assistive technologies *6
- Be aware of the following issues when designing an assessment:Computerthat computer-based testing places more demands on certain skills such as typing, mouse navigation, and the use of key combinations which may not be accessible for many users
- It may be Be aware that using multiple screens to recall a passage Some requires greater mental efforts by test takers
- Be aware that some people become more fatigued when reading text on a computer screen than on paperLong passages may be more difficult to read text on a computer screen than on paper specially reading long passages on the screen
- The inability to see an entire problem on screen at one time is challenging for many users.
- Speech recognition technology enables computers to translate human speech into a written format. Currently, speech recognition only works for some people, while others, especially those who are not native English speakers The speech recognition technology may not be an accessible alternative for collecting user responses, especially for those whose native language is not Engilsh or those with speech impairments, can be frustrated by the software’s lack of ability to differentiate many of the sounds that they make. *7
- Calculator use is often allowed on paper/pencil tests when arithmetic is not the construct being measured. However, standardization of the type of calculator used has been very difficult and would be much easier if all students had the same online calculator to use. Use of an online calculator is challenging for some students, especially if they have not had practice with this tool in their daily work. *7
- Capacity to have instructions repeated as often as student chooses *7
Maintaining place and saving completed responses during breaks *7
Capacity to turn off monitor/ blank screen temporarily *7
Test security *7
- Capacity for self-selection of subtest order *7
- The student must be familiar with using the technology itself. Introducing assistive technology to students with an exam deadline looming would place an added learning cost on the student *8
- Allow students to set their own transition times between questions (but bear in mind that extra time may make the total exam burden more onerous) *8
- if you use assistive technology to make the assessment more accessible, ask yourself if the student is being required to learn to use new technology at short notice, and whether this constitutes fair treatment. *8
- Alternative text or “alt tags” for images
Functional Considerations For Designing Accessible Assessments:
- Color is not used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. *6
- If any audio on a Web page plays automatically for more than 3 seconds, either a mechanism is available to pause or stop the audio, or a mechanism is available to control audio volume independently from the overall system volume level. *6
- Except for captions and rendered text, text can be resized without assistive technology up to 200 percent without loss of content or functionality and does not require the user to scroll horizontally to read a line of text on a full screen window. *6
- When type is enlarged on a screen, students may need to scroll back and forth, or up and down to read an entire test item. Graphics, when enlarged, may become very pixilated and difficult to view. Students who use hand held magnifiers or monocular devices when working on paper may not be able to use these devices on a screen because of the distortion of computer images. If a graphics user interface is used (versus text based), students will not have the option of altering print size on the screen. *7
- Try to avoid background audio. If you need background audio, give users control to turn it off or at least make it four times quieter than the foreground speech content (20 decibels lower) *6
Foreground and background colors can be selected by the user. *6
For blocks of text, make sure the width is no more than 80 characters or glyphs (40 if CJK), text is not justified (aligned to both the left and the right margins), and line spacing (leading) is at least space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing. *6
- Make all functionality available from a keyboard *6
- When an authenticated session expires, the user can continue the activity without loss of data after re-authenticating. *6
- Technology is required for multiple test sessions that would allow individual students to submit their completed responses and be able to log out and back on again at another time, starting at the place where they previously left off. Careful scheduling is needed for multiple test sessions to make sure that computers are available. Test security becomes an issue if students who have responded to the same test items have opportunities to interact with each other between test sessions. *7
- For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause , stop, or hide it unless the movement, blinking, or scrolling is part of an activity where it is essential *6
- For any auto-updating information that (1) starts automatically and (2) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it or to control the frequency of the update unless the auto-updating is part of an activity where it is essential. *6
- When any component receives focus, it does not initiate a change of context *6
- Changing the setting of any user interface component does not automatically cause a change of context unless the user has been advised of the behavior before using the component. *6
- Navigational mechanisms that are repeated on multiple pages within a set of pages occur in the same relative order each time they are repeated, unless a change is initiated by the user. *6
- Including accommodations such as screen magnification, text to speech audio, reverse contrast, and switch systems*3
- Provide a script for sign language interpreters, so they don’t “give away” some answers
Avoidance of complex backgrounds that interfere with readability of overlying text *7
Capacity for any student to select calculator or dictionary option *7
Assessment Types:
Type | Definition | Context | Accessibility Issues to Consider |
---|---|---|---|
Low Stake Assessment | Low-stakes assessment is a form of assessment encompassed by the immediate process of learning, often in a very short feedback loop, such as exercises or quizzes. Sometimes this is called "formative" assessment or even just "feedback". The essential characteristics are immediacy and the lack of serious consequences contingent on performance. *1 |
|
|
Formative Assessment | The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to recognize where students are struggling and address problems immediately and by students to identify their strengths and weaknesses and target areas that need work. Formative assessments are generally low stakes, which means that they have low or no point value. Examples of Formative Assessments include:
| Similar to Low Stake Assessments | Similar to Low Stake Assessments |
High Stake Assessment | High-stakes assessment has consequences that may make a serious impact on the life-course of the participant. An example might be a university entrance examination. It is important that a high-stakes assessment be fair to all candidates and not offer advantages to one group over another. *1
e.g. University Entrance or Exit Examinations, Test of English as a Foreign Language (TOEFL) |
|
|
Summative Assessment | The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. Summative assessments are often high stakes, which means that they have a high point value. Examples of summative assessments include:
Information from summative assessments can be used formatively when students or faculty use it to guide their efforts and activities in subsequent courses. *2 | Similar to High Stake Assessments | Similar to High Stake Assessments
|
Self Assessment | Assessment or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard. |
|
|
Assessment Tools:
Tool | Description | Required capabilities to operate task | Current Examples | Accessibility Issues | Alternative Methods |
---|---|---|---|---|---|
Multiple Choice | A traditional multiple choice question (or item) is one in which a student chooses one answer from a number of choices supplied. *4 |
|
|
| |
Assertion Reason | The assertion-reason item combines elements of multiple choice and true/false question types, and allows you to test more complicated issues and requires a higher level of learning. The question consists of two statements, an assertion and a reason. The student must first determine whether each statement is true. If both are true, the student must next determine whether the reason correctly explains the assertion. There is one option for each possible outcome. *4 |
|
| ||
True/False | A true-false questions is a specialized form of the multiple-choice format in which there are only two possible alternatives. These questions can be used when the test-designer wishes to measure a student's ability to identify whether statements of fact are accurate or not. *4 |
|
| ||
Multiple Select | This item allows user to choose more than one response option. *3 |
|
|
| |
Labeling (Graphic) | The graphic labeling task requires user to drag-and-drop labels to the correct location on an image or diagram (e.g. Venn Diagram). This type of question can be manipulated on a computer with a mouse or trackball or on a touch screen tablet. *3 |
|
|
| |
Labeling (One to One) | For this drag and drop task, the user selects a label from a list at the top of the screen and drags it to the correct location on the screen. Once the label has been used, it cannot be reused unless it is first moved back to the top of the screen. *3 |
|
|
| |
Labeling (Unlimited) | This item is similar to one-to-one labeling except that labels can be reused. The user clicks or touches the label and drags it to the correct location. *3 |
|
|
| |
Matching | With matching, the student matches each label with its correct response by clicking or touching one element in each column in turn. When the option in the second column is chosen, a line appears marking the connection between the two. Each option can be paired only once. *3 |
|
|
| |
Matrix | The matrix item type allows the user to assign elements to categories or groups. This item duplicates the tasks required in categorization. *3 |
|
|
| |
Categorization | The categorization item requires the user to organize elements into categories. There are two interfaces for this item type. In the drag-and-drop interface, the user clicks or touches an element and drags it into the correct category using a mouse or touch screen. With the click-to-select interface, the user clicks or selects an element to select it. Then, the user clicks or touches the category label, and the element appears in that category. *3 |
|
|
| |
Ordering/ Ranking | Ordering/Ranking/Sequencing is a drag-and-drop task in which the student must move the elements on the screen to put them in the correct order. *3 |
|
|
| |
Constructed Response (Text/ Numerical) | The text match question requires a student to supply an answer to a question or complete a blank within a brief piece of text, using words, symbols or numbers. *4 |
|
| ||
Select Text/ Sore finger | A word/ Code/ Phrase is out of keeping with the rest of package. *4 This item requires user to select one or more words, phrases, or sentences from a passage. When user selects an option, its appearance changes to verify that it has been selected. *3 |
|
| ||
Interactive | Includes videos, animations, coloring, graphing, games, puzzles or any other type of question that requires high levels of interaction between user and the interface. |
|
|
| |
Field Simulations | Offers simulations of real problems or exercises *4 |
|
|
Available Accessible Tools:
Please test the following tools and add your comments to this page.
Tool's Name | Description | Link |
---|---|---|
Nimbletools | NimbleTools is designed for use by general education students and students with special needs. This means that a single version of a test can be created and delivered to students across a school, district, or state in an efficient and cost-effective manner. The accessibility and accommodation tools embedded within NimbleTools also assure that students with disabilities and special needs are provided with appropriate accommodations in a standardized and controlled manner. | http://nimbletools.com/demo/index.htm |
Resources:
- http://www.imsglobal.org/accessibility/accessiblevers/sec9.html
- http://www.cmu.edu/teaching/assessment/basics/formative-summative.html
- http://www.ateassessments.org/
- http://www.caacentre.ac.uk/
- http://www.heinemann.com/shared/onlineresources/E03052/Accessible_Assessment_sample.pdf
- http://www.tbs-sct.gc.ca/ws-nw/wa-aw/wa-aw-assess-methd-eng.asp
- http://www.cehd.umn.edu/NCEO/onlinepubs/Policy15.htm
- http://assist-it.org.uk/assets/pdf/AAA.pdf#page=70