Mobile Accessibility

This document is a work in progress.

Current Work / Available Resources

Mobile Applications





Project Possibibility Applications

Audio Guardian
Barcode Reader
Color Reader
Mobile Currency Reader




Eyes Free




Loadstone GPS (Symbian Nokia)






WiFi-based point-of-interest (POI) service that allows users to tag outdoor and indoor locations
- Android port will coming soon

- depends on Firefox

Via Dux


OSM based, accessible mapping and wayfinding project for Java based mobile phones


Serial Keys


A collaborative project between the Wireless RERC and AAC RERC that provides an interface for AAC devices to communicate with Java based mobile phones


Nokia Braille Reader


Nokia Braille Reader gives SMS for the blind and visually impaired. It captures received SMS messages and brings them to the foreground for reading using Braille and tactile feedback.




mEADL is the mobile component of OpenEADL, a collection of open source and open hardware applications that may be used to extend the functionality of powered-wheelchairs and Electronic Aids for Daily Living (EADL).


Cross-platform Mobile SDKs





Python for Mobile Devices

- support for iPhone, Android, Windows Mobile/CE, Maemo, Symbian, Palm, etc...

software for mobile devices, PDA's, cell phones, handheld computers and for developers.


Phone Gap

- support for iPhone, Android, Blackberry

PhoneGap is an open source development tool for building fast, easy mobile apps with JavaScript.


Mobile FSS



- support for iPhone look
- Android skin coming soon

Mobile FSS is the mobile-friendly version of FSS. It takes advantage of core FSS philosophies like modular class names and files, it recycles many FSS class names to help take your existing content and make it mobile-friendly, and it's easy to use - just like regular FSS.


  • Java on Mobile - Runtime on which some things run but not all of phone

Open Mobile Systems






TTS & Eyes Free


- there may be a screen reader in works
- trend is for major manufactureres to build their own UI on Android


Next Generation Symbian/Nokia Phones












Eee PC (notebook)





Critical Gaps

Suggested By

Critical Gap

Comment on Importance

Guido Corona

Text to Speech Engines for Mobile Devices

Critical for audio interfaces on mobile devices

Greg Fields

TTS engine for mobile phones

This is required to provide solutions for print disabled users, at lower costs, that is open to community improvement, to minimize the gaps between persons with disabilities and the information society.

Rich Schwerdtfeger

A browser extension to enable Access For All preferences to be delivered through the browser

With HTML 5 we will have the ability to provide accessibility preferences through local storage. This would be a big win for inclusive, adaptible technologies like those developed by the Fluid Project. A plugin-in that allowed the user to configure their accessiblity preferences and convey them to web applications through local storage would, when combined with device capabilities, would truly make your mobile as well as your desktop advice a personalized solution that could adapt to the environment.

Greg Fields

PECS-based AAC application for mobile phones

Basic, general, open means of enabling communication impaired persons to communicate with the rest of the world on the most ubiquitous platform - mobile phones.

Greg Fields

Application for mobile phones that enables persons with speech impairments to used their mobile as their 'speech assistant'

Simple communication AT that connects to the TTS to enable persons with speech impairments to communicate face-to-face with the people with whom they come into contact, thereby increase independence and freedom of mobility.

Greg Fields

Screen reader for mobile phones

Basic access tech for print disabled users

Rich Schwerdtfeger


Industry is focusing on HTML 5 as a way to deliver rich content to multiple platforms and in particular mobile devices.
HTML 5 has a number of features, such as standard controls, that reduce the amount of script that need to be downloaded to the client. Also, elements like <canvas> allow web application providers to deliver drawings to a broad range of browsers. From an accessibility perspective we have a huge opportunity to integrate rich accessibility, like WAI-ARIA, into the host language. Another opportunity is for us to address personalization. Elements like the <video> and <canvas> will require alternative content for different users. We have the opportunity to include Access For All self-describing resource meta data to those elements to do things like allow the browser: swap a table out for a 2D chart rendered with <canvas> or video closed captioned to indicate an alternative video resource that is closed captioned in the language desired by the hearing impaired user. HTML 5 also provides local data storage. We could develop open source plug-ins that could feed local storage with Access For All user preferences and delivery context meta data that allows the browser to work with content to adapt it to the environment and user needs as they work throughout the day. HTML 5 is a tremendous opportunity for the accessibility community take hold of their future and deliver an inclusive framework to meet the needs of all users.

Resources Needed


Working Group Brainstorming Notes

See also Greg Fields' slides, which will be attached to this page.

  • Braille TTY exists
  • Space to take mobile beyond desktop - don't duplicate, expand
    • There is effort to extend browser capabilities to all APIs on a device
    • Zoomba
  • Gregg's "Portable Accessibility Device"
  • Using mobile phones as a "universal remote control" URC
    • Gregg
    • Georgia Tech
  • Portable OCR/barcode reader e.g. KNIB Reader
  • Gestures - standardization?
    • Gregg's kiosk work
  • auto-location of potential assistance based on preference profile
    • e.g. deaf person looking for a translator
    • social networking
    • social proximity awareness
  • iPhone shortcut button: blurs line between native and web-based. Distinction no longer matters
  • Android apps: being made in the image of iPhone interactions.
    • iPhone interactions moving to ubiquity
  • Ontario: ICT standards, expecting info to be produced according to WCAG 2.0; Employment a11y standards referencing ICT standards; having mobile device as a tool for access could decrease initial effort for organizations to meet the standards.
    • e.g. mobile RCC, mobile CART: real-time, crowd sourced closed captioning
  • User-centred design
    • Need to do user studies: observe real users in real scenarios. What really matters to the users?
  • Disability awareness training
  • Usability != checklist
    • Instead of doing "accessibility testing", make friends who can help with usability design
  • Participative Joint Design Methodologies void in all/most instances including Mobile

Demo: Jorge's Bluetooth/Serial port adapter

Problem Statement

Carve out a new space:

  • speech assistant (face-to-face)
    • real-time translation
  • universal remote (powered wheelchairs)
  • real gesture mapping -> need to involve real people
    • API (access)
    • gesture profile (cross-platform)
    • compatibility with HID/bluetooth
    • allow users to define their own gestures for particular actions?

Priority: Flexibility

Next Steps:

  • consider how to have UI that is flexible: what do we need for that?
    • access to hardware APIs

Problem Statement:

From Greg's presentation, we have a good definition of the problem space. We've identified the priority of flexibility in UI and UX. The focus should be finding a cohesive way to bring it all together and make sure that the mobile space becomes flexible.

Proposal theme possibilities:

  • making sure standards like ISO 24751, Access for All, are supported
  • don't include "new gestures" because everyone is already trying to do this

Who involved in proposal?

  • RIM


Jorge: Possible solution space: Web space

Jess: Fluid Engage demo (Safari only)
UI Options demo


  • Web as development platform for flexibility. Work there can be extended to the device.
  • Let the UI and the open web structure how we might address the points raised in brainstorming.

Relation to location-based awareness:

  • Tagging. Most system have closed APIs. Firefox opened their APIs for recording wifi info. But doesn't make sense on a laptop.
  • Difficulty: access to accelerometer not normally available from browser.
  • Prediction: Web will eventually have equal footing with native in terms of richness
  • Issue: Using the web = using data, which costs money on a mobile device.

Multi-phase proposal process:

  • First grant: user study to assess high-priority user needs; where are the common links, avenues of research or development or design that will improve accessibility of mobile devices for many
  • Second grant: follow up on findings of user study

AEGIS has identified areas of need Greg, please fill in?
They didn't cover some disability groups (deaf/blind, mobility impaired, speech impaired). Should we try to cover those groups, using their techniques?

  • Opportunities identified in Greg's slides cover these three groups.

Potential candidate: Speech Assistant:

  • real need
  • work on it would move several areas forward
  • can add translation


  • can't communicate face-to-face ad hoc

Use Cases:

  1. Face to face communication with a hearing person
    • inclusiveness
      • order coffee, talk dirty


  • speech impairement
    • temporary
    • long-term

Functional Requirements:

  • translation
  • real-time for face-to-face
  • text-to-speech
  • don't need an accessibility API
  • spell check
  • Auto Text
  • TTS direct access
  • predictive text
  • saved, forward
  • local
  • "Notepad"
  • cross-platform


  • first pass won't address cognitive impairments, such as aphasia


  • App store
  • Speech therapists
  • disability advocacy community
  • OATsoft