End-to-End Open Source Accessibility

Owned by Willie Walker
Last updated: Oct 27, 2009Version comment
9 min read

End-To-End Open Source Accessibility in One Platform

This page discusses the space of end-to-end open source accessibility on a single platform. The main premise is that the accessibility space is fragmented and we need to gain critical mass on a single platform.

The GNOME platform on top of Linux and OpenSolaris provides a good basis for this single platform. GNOME provides an open source graphical desktop environment, a graphical development toolkit, and a suite of applications. GNOME is developed by a large number of organizations spread across the planet, allowing it to draw from worldwide talent as well as quickly heal from the failure or disinvestment of any one organization. As the default graphical desktop for a large number of popular operating system distributions, GNOME also has a large deployment and support model already in place, with localizations being done for approximately 160 languages.

What is Needed for an End-to-End Solution?

To be written as part of the workshop. A primary goal is compelling access to the modern graphical user interface by people with disabilities.

  • Supporting Technology
    1. Audio
    2. Speech synthesis
    3. Braille (devices and software)
    4. Magnification
    5. Accessibility infrastructure
    6. Accessible Toolkit(s)
    7. Assistive technologies
    8. Auxiliary input devices
    9. Testing (functional, regression, automated, etc.)
    10. ...
  • Accessible Applications - E-mail, Browser, Communication Tools, Help, Educational Tools, ...
  • Deployment
  • Localization
  • Support, Maintenance
  • Education

Survey of the Existing Solution

Note that GNOME on its own is not a complete end-to-end solution for accessibility. GNOME relies upon an operating system (e.g., Linux and OpenSolaris) for fundamental support as well as other open source software from third parties.

Base Operating System

GNOME is deployed as a default component of popular operating system distributions, such as Ubuntu and OpenSolaris, and depends upon the operating system to provide needed features such as the following:

  • Graphics card drivers and basic input/output devices such as keyboards and mice. These are all part of the base operating system.
  • Support for audio devices and audio mixing. The Linux platform is migrating towards PulseAudio as a solution for audio mixing. There are admittedly current issues with PulseAudio that are causing probems with audio integration (e.g., speech synthesis engines behaving improperly), but these are being worked on in the general Linux community and should be resolved in the near future. The OpenSolaris platform is using OSS which functions well and has great audio device coverage.
  • Support for external hardware input devices. With Linux and OpenSolaris, support for connecting external input devices via serial and USB is a given. Support for video and audio input is also integrated well on both platforms. An area for improvement is with connecting/configuring switch-based devices (e.g., a Tash Single Switch), but work with the XInput2 X Server extension is expected to help resolve this issue.
  • Support for external hardware output devices. As with input devices, support for serial and USB devices for output is also a given.

Third Party Open Source Solutions

GNOME also uses components from third parties to complete its solution, including the following:

  • Underlying graphical desktop support. GNOME uses the X Windows System as its basis for the underlying graphical desktop support on Linux and OpenSolaris.
  • Speech synthesis engines. GNOME does not provide a speech synthesis engine. Instead, GNOME relies upon third parties to provide speech synthesis solutions. Existing open source speech synthesis engines that work with GNOME include eSpeak, Festival, Flite (Festival lite), and FreeTTS. GNOME also includes support for commercial speech synthesis engines, such as IBMTTS, DECtalk, Cepstral, and Loquendo.
  • Braille device drivers and contracted braille. GNOME relies upon the open source BrlTTY project for braille device support and the open source liblouis project for contracted braille support.

GNOME Accessibility

GNOME itself provides a number of built in open source accessibility features, including the following:

  • Solutions for People with Mobility Impairments Affecting Keyboard and Mouse Use
    1. Keyboard Access. Keyboard access is a core value of the GNOME project. Users should be able to access all functionality using the keyboard alone.
    2. Keyboard enhancements via StickyKeys, SlowKeys, BounceKeys, MouseKeys, and RepeatKeys. The GNOME keyboard configuration utility provides access to the AccessX/XKB features of the X Windows System and also permits the enabling/disabling of the features using the well known keyboard hot keys.
    3. Mouse enhancements via MouseTweaks. The GNOME MouseTweaks project provides support for users who are able to move the mouse pointer, but not click the mouse buttons. Features include dwell clicking, clicking by gesture, etc.
    4. Head tracking via MouseTrap. The GNOME MouseTrap project is an emerging technology that uses low cost (e.g., $25 USD) web cameras to track physical features on a user in order to manipulate the mouse. The technology is currently not a viable solution but is continually being improved. This may eventually merge with similar work being done in the AEGIS project.
    5. On screen keyboard via GOK. The GNOME On Screen Keyboard project (GOK) provides switch-based and dwell based access to the desktop.
    6. Dasher. The GNOME Dasher project integrates the Dasher predictive text entry solution into the GNOME environment.
  • Solutions for People with Visual Impairments
    1. Theming. GNOME provides support for grouping desktop/application fonts, colors, and icons in a group known as a "theme" that can be set system wide, and GNOME applications honor these themes. GNOME provides high contrast, inverse, large print, themes.
    2. Magnification. GNOME currently provides magnification via the gnome-mag project, which serves both as a standalone magnifier and as a system magnification service (e.g., for use by the Orca screen reader). The GNOME project is currently looking to move towards tighter integration with the emerging GNOME shell technology. Magnification is also supported via the eZoom plugin of an alternative window management technology known as Compiz.
    3. Screen reading via Orca. The GNOME Orca screen reader project provides access via user customizable combinations of speech, braille, and magnification.
  • Solutions for People with Hearing Impairments
    1. VizAudio. The GNOME VizAudio project is an emerging project that integrates with the libcanberra project, which is and implementation of the XDG Sound Theme and Name Specifications. VizAudio's goal is to provide on screen textual equivalents of sounds being generated by the system.
  • System Services and Infrastructure
    1. Accessibility infrastructure. GNOME provides the AT-SPI accessibility infrastructure for the creation of assistive technologies. The AT-SPI is an API-based approach to accessibility and it supported by the native GNOME toolkit, GTK+, as well as Firefox, OpenOffice, and the Java platform. AT-SPI support is also in the works for emerging GNOME technology (e.g., clutter and GNOME shell) as well as KDE's Qt toolkit.
    2. Speech synthesis. GNOME currently provides the gnome-speech module that provides a speech service for GNOME applications to use. The GNOME project is currently looking to move towards the external Speech Dispatcher project.
    3. Localization. While not specifically an accessibility solution, the localization model of GNOME enables GNOME to be available in a large number of languages (~160).

Deployment and Support

GNOME relies upon base operating system for deployments around the world. Operating system distributions, including Ubuntu, OpenSolaris, RedHat/Fedora, openSuse, Debian, etc., already provide an existing worldwide deployment and support model that includes the GNOME desktop. These solutions are available in many different languages and are also available at zero cost ($0.00).

Individuals, organizations, and countries around the world have adopted and are continuing to adopt GNOME as a viable accessibility solution. The GNOME project is seeing GNOME accessibility make the difference in people's lives around the world, including India, Spain, Portugual, Brazil, the United States, Great Britain, Germany, etc. The ONCE organization in Spain, for example, is moving large populations of people to Linux and GNOME.

Application Coverage

The following list comprises the common desktop applications that are accessible via the GNOME desktop:

  • Web browsing. Mozilla Firefox has invested considerably in accessibility and integrates well into the GNOME desktop, providing support for traditional web content as well as ARIA. Note that the Firefox browser also serves as a means for deploying accessible web applications that can be accessed using assistive technologies on the native desktop.
  • Document generation and viewing. OpenOffice has invested considerably in accessibility and integrates well into the GNOME desktop.
  • eMail. Both the GNOME Evolution eMail/calendaring/contacts application and the Thunderbird applications integrate well into the GNOME desktop.
  • Instant messaging. The Pidgin instant messenger integrates well in the GNOME desktop and provides access to AIM, IRC, Jabber, MSN, Yahoo, etc.
  • Audio/video. The GNOME Totem and Rhythmbox applications provide access to audio and video content.
  • Development tools. The open source Eclipse and Netbeans IDE's provide an accessible application development environment.
  • System administration. GNOME provides a number of system administration tools which are also accessible. These include user management, network management, printer management, etc. In addition, the Ubuntu and OpenSolaris releases provide an accessible live CD that allows people with disabilities to install and configure the operating system without assistance.

Gaps

As with everything available today, the GNOME solution is not complete. Gaps exist in at least the following areas:

  • Accessible PDF document viewing. Through various tools (e.g., translating a PDF document to a text file), PDF content can be made available for viewing, but the process is cumbersome. Adobe's Acrobat Reader claims to be accessible, but Adobe is not addressing the known bugs, which must be addressed by Adobe because it is a closed system. The GNOME Evince document viewer provides an open source alternative, but it needs work to make it accessible. Various organizations have expressed an interest in working on Evince, but no traction has been made to date.
  • Speech recognition. Command and control and dictation on the GNOME desktop is not supported well. Open source projects that run natively on Linux and OpenSolaris are available (e.g., Sphinx), but require a fair amount of integration work. Other potential solutions include attempting to integrate speech recognition running under the 'wine' application (a Windows Emulator). For example, one might run Dragon Naturally Speaking under wine.
  • Hardware input drivers. It is difficult to integrate external hardware input devices such as switches into the X desktop. The XInput2 extension may provide supprt for this, but it most likely needs attention to make sure it addresses accessibility needs.
  • DAISY document viewing. The Java-based Emerson Reader provides a solution for audio-only DAISY books. The Mozilla Foundation has also funded Benetech to create a DAISY plugin for Firefox, providing access to both audio and text-based DAISY books, thus providing some level of accessible DAISY document viewing.
  • Closed Captioning. Not sure completely, WDW recalls there being relatively spotty CC support in the video viewers on GNOME. The work Mozilla/Firefox are doing with the open video tag, however, may provide a compelling open source solution for supporting, viewing, and adding closed captions to web-based video content. The Internet Captioning Forum may also provide some good direction and solutions.
  • Hearing impairments. GNOME only provides minimal support for people with hearing impairments. VizAudio might be able to improve the situation some, but support for richer forms of audio, streaming media, and real time text are not supported. Older technologies, such as TTY, also have no GNOME-based solutions.
  • Skype. While other internet telephony applications exist for VoIP, Skype is popular and is inaccessible. The main work needed is to put pressure on the closed source Skype developers to create an accessible solution (e.g., Skype on GTK+).
  • OCR. OCR projects are emerging, but they need to be addressed with the blind user in mind (e.g., don't require the user to select regions of text with the mouse).
  • AAC. Although GNOME provides many of the building blocks, no specific application exists that is geared specifically towards alternative/augmentative communication. Dasher might come the closest, but likely needs some work. The AEGIS project may potentially yield some interesting work in this space.
  • Learning disabilities. GNOME provides little in the way of solutions for people with learning and/or cognitive disabilities. Some work might be possible in the Orca screen reader address some more mild learning disabilities, but this space needs more research. The AEGIS project is also funding work to add the concept coding framework to OpenOffice.
  • High quality text-to-speech. The existing open source speech synthesis solutions that run on Linux and OpenSolaris are marginally acceptable, but can always use improvement. Language coverage needs work as well. AEGIS potentially will fund work in this space in addition to increasing the locale coverage for the eSpeak synthesis engine.
  • Braille translation software. A Duxbury equivalent doesn't exist. A potential solution might be a plug in for OpenOffice or a standalone solution (e.g., DOTS as proposed by Eitan Isaacson). AEGIS is currently funding work to provide a braille plugin for OpenOffice.
  • Braille embosser drivers. It is unknown whether existing braille embossers will operate when connected to Linux or OpenSolaris machines. ONCE has CUPS drivers for 4 braille printers that they would like to bring to the Linux/UNIX environment.
  • Education. While a lot of material is on line, clearer and more approachable documentation is always a laudable goal. In addition, classroom-based programs for training are also highly desirable. ONCE has a plan in place to "train the trainers". In addition, Martin Owens from the Ubuntu Community has started a community project for developing curricula around Ubuntu and GNOME.
  • Out of the box experience. GNOME accessibility can be sensitive to how it is integrated in the operating system and it also requires a special setting to be enabled in the user environment. This can be improved.
  • Resources. GNOME accessibility starved for resources. If we could all center on a platform (this page proposes GNOME), we could make much more effective use of the very few resources we have.

Priorities

To be written as part of the workshop.

Roadmap

To be written as part of the workshop.