Sketch: Conversational style interaction

This is ugly looking but illustrates some design ideas I wanted to throw in to the mix before it is too late.

Description of the image:  I’m imagining this as a tablet screen.  The screen is divided into two areas.  At the top is some content, and at the bottom is a conversation area with a signing avatar and messages in the style of instant messaging.  The content at the top includes a title (“Set Up Your iPad”) in large text, a subtitle in the default size and color for the platform. The subtitle says: “This video explains some of the ways you can adjust your iPad to make it easier to use”, buttons to increase and decrease the volume, a video with a closed caption button on it, and a large ‘Continue’ button.  In the lower messaging area, the first message is a question from the avatar “Would you like me to read everything out loud?”  Yes/No buttons are attached to the right this message.  If the user does nothing, a new message from the avatar is shown and spoken: “You can answer by tapping the Yes/No buttons or by speaking”.  Next, the avatar says “Would you like to have audio description for videos?”  Again, there are Yes/No buttons, but the user could also speak.  A fourth message shown here is “You can ask a question by speaking or typing at any time”  That message has a button saying ‘Touch here for a keyboard’.  The messages would be simultaneously signed and spoken aloud as they appear on the screen.  They would appear one at a time, like an IM conversation.

Main ideas:

1.) Using an area with typical controls gives familiar-looking interaction choices to users with some experience.  If they choose to interact with these controls, we very quickly start to learn something about them and their preferences.

2.) Using a conversational interaction style would allow for flexible paths through the application that depend on previous responses.

3.) The avatar interaction would have the main focus, and is presented  in sign language and speech as well as text.  Users have several ways to respond to the avatar.  If they do not respond, or their responses are difficult to interpret, more options can be presented until something that works is found.

3.) This is still compatible with more direct manipulation selection screens, such as font size and contrast choices.  The options would appear in the top part of the screen, where users could interact with them directly, or continue in their conversation with the avatar.

4.) Just like an IM conversation, the user can take the initiative at any time, to ask a question or make a request.  For users who know what they want, this provides a shortcut.

5.) The interaction choices made by the user are also noted, and the conversation continues to use interaction methods that the user has demonstrated they can use.

6.) The conversational style provides access to prior steps in the process, and an easy way to get back to previous actions and change them.

Shari.