Mobile Accessibility

This document is a work in progress.

Current Work / Available Resources

Mobile Applications	Link	Status	Comments	Gaps
Project Possibibility Applications	Audio Guardian Barcode Reader Color Reader Mobile Currency Reader
Eyes Free	http://code.google.com/p/eyes-free/
Loadstone GPS (Symbian Nokia)	http://www.loadstone-gps.com/	production
Tagin!	http://scyp.atrc.utoronto.ca/projects/tagin	beta/experimental	WiFi-based point-of-interest (POI) service that allows users to tag outdoor and indoor locations - Android port will coming soon	- depends on Firefox
Via Dux	http://wiki.openstreetmap.org/wiki/Via-Dux	development	OSM based, accessible mapping and wayfinding project for Java based mobile phones
Serial Keys	http://www.imakenews.com/aac-rerc/e_article000772593.cfm	development	A collaborative project between the Wireless RERC and AAC RERC that provides an interface for AAC devices to communicate with Java based mobile phones
Nokia Braille Reader	http://betalabs.nokia.com/apps/nokia-braille-reader	beta	Nokia Braille Reader gives SMS for the blind and visually impaired. It captures received SMS messages and brings them to the foreground for reading using Braille and tactile feedback.
mEADL	http://scyp.atrc.utoronto.ca/projects/meadl	development	mEADL is the mobile component of OpenEADL, a collection of open source and open hardware applications that may be used to extend the functionality of powered-wheelchairs and Electronic Aids for Daily Living (EADL).

Cross-platform Mobile SDKs

Link

Status

Comments

Gaps

Python for Mobile Devices

http://www.awaretek.com/pymo.html

- support for iPhone, Android, Windows Mobile/CE, Maemo, Symbian, Palm, etc...

software for mobile devices, PDA's, cell phones, handheld computers and for developers.

Phone Gap

http://www.phonegap.com/

- support for iPhone, Android, Blackberry

PhoneGap is an open source development tool for building fast, easy mobile apps with JavaScript.

Mobile FSS

-

http://wiki.fluidproject.org/display/fluid/Mobile+FSS

- API

- support for iPhone look
- Android skin coming soon

Mobile FSS is the mobile-friendly version of FSS. It takes advantage of core FSS philosophies like modular class names and files, it recycles many FSS class names to help take your existing content and make it mobile-friendly, and it's easy to use - just like regular FSS.

Java on Mobile - Runtime on which some things run but not all of phone

Open Mobile Systems	Link	Comments
Android	http://www.android.com/ TTS & Eyes Free	- there may be a screen reader in works - trend is for major manufactureres to build their own UI on Android
Next Generation Symbian/Nokia Phones	http://developer.symbian.org/
Maemo	http://maemo.org/
OpenMoko	http://www.openmoko.com/
Eee PC (notebook)

Critical Gaps

Suggested By	Critical Gap	Comment on Importance
Guido Corona	Text to Speech Engines for Mobile Devices	Critical for audio interfaces on mobile devices
Greg Fields	TTS engine for mobile phones	This is required to provide solutions for print disabled users, at lower costs, that is open to community improvement, to minimize the gaps between persons with disabilities and the information society.
Rich Schwerdtfeger	A browser extension to enable Access For All preferences to be delivered through the browser	With HTML 5 we will have the ability to provide accessibility preferences through local storage. This would be a big win for inclusive, adaptible technologies like those developed by the Fluid Project. A plugin-in that allowed the user to configure their accessiblity preferences and convey them to web applications through local storage would, when combined with device capabilities, would truly make your mobile as well as your desktop advice a personalized solution that could adapt to the environment.
Greg Fields	PECS-based AAC application for mobile phones	Basic, general, open means of enabling communication impaired persons to communicate with the rest of the world on the most ubiquitous platform - mobile phones.
Greg Fields	Application for mobile phones that enables persons with speech impairments to used their mobile as their 'speech assistant'	Simple communication AT that connects to the TTS to enable persons with speech impairments to communicate face-to-face with the people with whom they come into contact, thereby increase independence and freedom of mobility.
Greg Fields	Screen reader for mobile phones	Basic access tech for print disabled users
Rich Schwerdtfeger	HTML 5	Industry is focusing on HTML 5 as a way to deliver rich content to multiple platforms and in particular mobile devices. HTML 5 has a number of features, such as standard controls, that reduce the amount of script that need to be downloaded to the client. Also, elements like <canvas> allow web application providers to deliver drawings to a broad range of browsers. From an accessibility perspective we have a huge opportunity to integrate rich accessibility, like WAI-ARIA, into the host language. Another opportunity is for us to address personalization. Elements like the <video> and <canvas> will require alternative content for different users. We have the opportunity to include Access For All self-describing resource meta data to those elements to do things like allow the browser: swap a table out for a 2D chart rendered with <canvas> or video closed captioned to indicate an alternative video resource that is closed captioned in the language desired by the hearing impaired user. HTML 5 also provides local data storage. We could develop open source plug-ins that could feed local storage with Access For All user preferences and delivery context meta data that allows the browser to work with content to adapt it to the environment and user needs as they work throughout the day. HTML 5 is a tremendous opportunity for the accessibility community take hold of their future and deliver an inclusive framework to meet the needs of all users.

Resources Needed

Roadmap

Links

Accessibility Solutions in Mobile Devices (not only FOSS)

Working Group Brainstorming Notes

See also Greg Fields' slides, which will be attached to this page.

Braille TTY exists
Space to take mobile beyond desktop - don't duplicate, expand
- There is effort to extend browser capabilities to all APIs on a device
- Zoomba
Gregg's "Portable Accessibility Device"
Using mobile phones as a "universal remote control" URC
- Gregg
- Georgia Tech
Portable OCR/barcode reader e.g. KNIB Reader
Gestures - standardization?
- Gregg's kiosk work
auto-location of potential assistance based on preference profile
- e.g. deaf person looking for a translator
- social networking
- social proximity awareness
iPhone shortcut button: blurs line between native and web-based. Distinction no longer matters
Android apps: being made in the image of iPhone interactions.
- iPhone interactions moving to ubiquity
Ontario: ICT standards, expecting info to be produced according to WCAG 2.0; Employment a11y standards referencing ICT standards; having mobile device as a tool for access could decrease initial effort for organizations to meet the standards.
- e.g. mobile RCC, mobile CART: real-time, crowd sourced closed captioning
User-centred design
- Need to do user studies: observe real users in real scenarios. What really matters to the users?
Disability awareness training
Usability != checklist
- Instead of doing "accessibility testing", make friends who can help with usability design
Participative Joint Design Methodologies void in all/most instances including Mobile

Demo: Jorge's Bluetooth/Serial port adapter

Problem Statement

Carve out a new space:

speech assistant (face-to-face)
- real-time translation
universal remote (powered wheelchairs)
real gesture mapping -> need to involve real people
- API (access)
- gesture profile (cross-platform)
- compatibility with HID/bluetooth
- allow users to define their own gestures for particular actions?

Priority: Flexibility

Next Steps:

consider how to have UI that is flexible: what do we need for that?
- access to hardware APIs

Problem Statement:

From Greg's presentation, we have a good definition of the problem space. We've identified the priority of flexibility in UI and UX. The focus should be finding a cohesive way to bring it all together and make sure that the mobile space becomes flexible.

Proposal theme possibilities:

making sure standards like ISO 24751, Access for All, are supported
don't include "new gestures" because everyone is already trying to do this

Who involved in proposal?

AEGIS
RIM

Deliverables:

Jorge: Possible solution space: Web space

Jess: Fluid Engage demo (Safari only)
UI Options demo

Proposal?:

Web as development platform for flexibility. Work there can be extended to the device.
Let the UI and the open web structure how we might address the points raised in brainstorming.

Relation to location-based awareness:

Tagging. Most system have closed APIs. Firefox opened their APIs for recording wifi info. But doesn't make sense on a laptop.
Difficulty: access to accelerometer not normally available from browser.
Prediction: Web will eventually have equal footing with native in terms of richness
Issue: Using the web = using data, which costs money on a mobile device.

Multi-phase proposal process:

First grant: user study to assess high-priority user needs; where are the common links, avenues of research or development or design that will improve accessibility of mobile devices for many
Second grant: follow up on findings of user study

AEGIS has identified areas of need Greg, please fill in?
They didn't cover some disability groups (deaf/blind, mobility impaired, speech impaired). Should we try to cover those groups, using their techniques?

Opportunities identified in Greg's slides cover these three groups.

Potential candidate: Speech Assistant:

real need
work on it would move several areas forward
can add translation

Problem:

can't communicate face-to-face ad hoc

Use Cases:

Face to face communication with a hearing person
- inclusiveness
  - order coffee, talk dirty

User:

speech impairement
- temporary
- long-term

Functional Requirements:

translation
real-time for face-to-face
text-to-speech
don't need an accessibility API
spell check
Auto Text
TTS direct access
predictive text
saved, forward
local
"Notepad"
cross-platform

Assumptions:

first pass won't address cognitive impairments, such as aphasia

Exploitation:

App store
Speech therapists
disability advocacy community
OATsoft