home people portfolio plugins contact
Plugins
ZebraTools
ZebraSpeak

ZebraSpeak
Let your work speak for itself.

The ZebraSpeak module adds text-to-speech and advanced keyboard interaction to iShell. Together, these two features enable you to create products in iShell that are accessible to blind and physically disabled users.

The Text-to-Speech works with Apple's built in speech synthesizers on Mac OS 9 and OS X. On Windows, it uses the Chant® SpeechKit® engine to provide compatibility with the main speech synthesizers for Windows, including the standard Microsoftš speech engines.

The Speak commands are very flexible, allowing you to speak the names and contents of elements, speak the contents of a text, rtf, or html file, or speak a string generated on the fly.

The keyboard interaction tools allow you to build a dynamic list of elements to receive keyboard events, so you can easily create and maintain complex keyboard navigation schemes. A special Tab Handler object is used to automate interaction with "tabbable" elements on a screen, elements that need to respond to the tab and return keys.

ZebraSpeak works with iShell 3 and iShell 4, and requires Mac OS 9, Mac OS X 10.2 or later, and Windows 95 or later.


Contents:

Speaking:
   Speak Command
   Speech Prefs Command
   Speech Mode Command
   Speech Finished Event

Keyboard control
   Tab Handler Element
   TabHandler Control Command
   Tab Event
   Key Handler Element
   Key Handler Control Command
   Get/Set Attribute Command Options
   Set Command Options
   Get Command Options

Other Notes

Speaking in iShell Overview

To use the ZebraSpeak plugin, you first need to copy ithe module to your iShell Modules folder. Once ZebraSpeak is installed, to use, all you need to do is use the Speech Prefs command to turn Text-To-Speech on, and then use the speak command to speak whatever you want.

On both Mac OS 9 and Mac OS X, the necessary software is installed and turned on by default. On Windows 2000 and XP, one or two voices are installed by default. You should provide installers for other systems. The easiest (and free) option is to distribute Microsoft SAPI installers. These can be found at Microsoft.com/Speech, although the offerings here keep changing, and finding appropriate installers can be hard, so we've mirrored the installer we use here.

 

Speak Command

The speak Command is the basic command for making iShell speak. It has a number of different variants for different situations.
Pause Speaking pauses/unpauses the current speaking text
Speak File speaks the contents of a .txt file
Speak Name speaks the name of the object
Speak Object speaks the content of an object (such as the text within a text or field object)
Speak Phrase speaks the given phrase
Stop Speaking stops the current speaking text
Speak Command options:
  • Always
speaks the text even if Text to Speech is turned off
  • Queue
 waits for all speaking text to finish, before speaking current text
  • Generate Done
 sends a speech finished event when current text is finished speaking

 

Speech Prefs Command

The Speech Prefs Command allows you to turn text to speech on and off.
TTS On Turns Text to Speech on and off (when Text to Speech is off, Speak commands will still be spoken if they are set to Speak Always).
Reselect Voice (on Win) Allows the Windows user to reselect a voice, in the same way a voice is chosen when the project is first run on a Windows machine.
When Speaking Can be set to pause other sounds or reduce their volume by a %, when Text to Speech is speaking.

 

Speech Mode Command

The Speech Mode Command allows you to switch between normal text-to-speech mode and Sentence-by-sentence, word-by-word and letter-by-letter reading modes.
Speech Mode Choose between Normal reading mode, Sentence-by-sentence, Word-by-word, and Letter-by-letter.
Use Space/Shift-Space Lets the user step forward using the Space bar, and backwards using Shift-Space.
Use Right/Left Keys Lets the user step forward using the Right arrow key, and backwards using the Left arrow key.
Use Down/Up Keys Lets the user step forward using the Down arrow key, and backwards using the Up arrow key.

 

Speech Finished Event

This event is called whenever a Speak command has completed, under the following conditions
  • the Speak command is set to Generate Done
  • the Done ID of the Speak command matches the Done ID of the Speech Finished Event

 

Keyboard Interaction Overview

This section will cover the basics of using ZebraSpeak to handle keyboard interaction in your projects. There are two ZebraSpeak elements that can be used to simplify keyboard interaction in iShell. The Key Handler Element is used to maintain a list of elements that should recieve keyboard events, and to distribute a keystroke up the list, until if finds an element that will use it.. The Tab Handler element maintains a list of "tabbable" items on a screen, and highlights the current selection.

 

Tab Handler Element

The Tab Handler provides for keyboard navigation through a screen (Tab to objects, Enter to act on them).  It allows you to specify a list of objects to be tabbed through, and choose the speech actions and mouse actions to be sent to those objects on Tab/Enter.Attributes

State
the Tab Handler must be Disabled (since it covers other important Elements) and Shown (if you need it to display Highlight boxes) to work correctly.
Size & Position
If you are using the Highlight functionality, the Tab Handler must cover all tabbable Elements on the screen for the highlights to show up correctly. It also must be on top of all Shown Elements; we find it simplest to place it as the last Element on the screen.
Highlight
this specifies whether a highlight box will be shown around the current Tabbed-to Element, and whether the highlight box is an invert box or is drawn with the specified color.
Mouse Actions you can choose which action (None, Mouse Enter, Mouse Down) is sent to an Element when it is Tabbed to , and which action (None, Mouse Down, Mouse Up, Mouse Click) is sent when Return is pressed on it (Tab Events are sent regardless of the settings in the tab handler).
Speech Actions
you can choose which action (None, Speak Name, Speak Object) is sent to an Element when it is Tabbed to, and which is sent when Return is pressed on it.
Register
 this specifies whether the Tab Handler will automatically be registered with a Key Handler, if present.  The Tab Handler will be placed on the Key Handler's Target List before Loaded (offscreen) is called, so any Elements added to the Target List in a Loaded (offscreen) method would come after the Key Handler in the list.

Activating the Tab Handler

  • If you are using a Tab Handler in conjunction with a Keyboard Handler, the Tab Handler must be the last Tab Handler in the Target List of the Keyboard Handler, for it to work.
  • If you are using a Tab Handler without a Keyboard Handler, you must Focus on the Tab Handler for it to work.
When the Tab Handler object speaks the current selection's name, it generates a Speech Finished event with the ID "Object Name Tab" for tabbing to the object, and "Object Name Return" for pressing return on the object.

 

Tab Handler Control Command

The Tabhandler Control Command allows you to manage the items that appear in a Tab Handlers list of items:
Add To Tab List adds the specified Element to the end of the specified Tab Handler's tab list
(specifying a List will add all Elements in the List to the tab list)
Remove From Tab List removes the specified Element from the specified Tab Handler's tab list
Clear Tab List removes all Elements from the specified Tab Handler's tab list
Get Tab List returns a List of all Elements in the specified Tab Handler's tab list
Get Current Focus returns the Element currently Tabbed to by the specified Tab Handler

 

Tab Event

This Event can be added to any Element, and will be triggered when a Tab Handler executes the chosen action (TabEnter/TabLeave/Return) on that Element.

 

Keyboard Handler Element

The Keyboard Handler provides more advanced keyboard control than the Tab Handler.  It can be used in situations where you would like a Tab Handler to use any tab and enter Keyboard Events, but would like other Keyboard Events (such as pressing M to access a Map) to be passed on to other Elements on the screen.  It can also be used to coordinate multiple Tab Handlers. You would typically onty have one Key Handler in a project, either in a central document that all other documents get loaded into, or in a "global" document that is included on all screens.

In a nutshell:

  • The Keyboard Handler automatically sets itself as the focus on Loaded (offscreen) when it is present, to ensure it gets all Keyboard Events.
  • It has a target list of Elements to send those keyboard events to (you can modify the target list using the Keyboard Handler Control Command).  The targets can be any type of Element.
  • It sends all Keyboard Events to the last Element on the target list; any unused Keyboard Events get sent to the next Element on the list, and so on; the Keyboard Handler itself can act on the Keyboard Events not used by any of the targets on the list.

Example: Using Keyboard Handler to pass on keyboard Events

 (see this example in  KeyboardEvents.xd in the ZebraSpeak Demo Project)

 In this example the Keyboard Handler and its target list are as follows:
       Keyboard Handler Element
          - has Keyboard Event "G"
                - Go Back
          - has Keyboard Event "B"
                - Go Back
      Keyboard Handler's Target List:
          -  Movie Element
                - has Keyboard Event "R"
                      - Run Itself True
                - contains Keyboard Event "P"
                      - Run Itself False
          - Tab Handler Element
                - by default has Tab and Enter Keyboard Events
                - contains Keyboard Event 'c'
                    - sets the screen to high contrast.

Pressing Tab or Enter, the Keyboard Handler passes these Events to the last Element on it's list, the Tab Handler, which uses them to move through it's tab list. 

Pressing C, the Keyboard Handler passes these Events to the last Element on it's list, the Tab Handler, which uses them to show a QuickTime filter box that sets the screen to high contrast.
 
Pressing R or P, the Keyboard Handler passes these Events to the last Element on the list, the Tab Handler, where they are unused. So the Keyboard Handler passes them to the next Element up the list, the Movie, where they are used to run and pause the movie. 

Pressing G or B, the Keyboard Handler passes these Events to each Element up the list, but none of the Elements use them, and so the Keyboard Handler uses them itself to go Back a screen.

Pressing any other key, the Event is passed to each Element up the list, but not used by any of them, and is also not used by the Key Handler, so nothing will happen.

Using Keyboard Handler with Multiple Tab Handlers in the same Document

At times you may want two or more Tab Handlers in the same Document, for example one Tab Handler to handle a series of buttons, and one to handle the elements of a game or exercise.  You can control them easily using a Keyboard Handler.  Remember that only the last Tab Handler in the Keyboard Handler's target list will execute Tab and Return Keyboard Events, since it is the first Element to receive them. 

(see an example in 2TabHandlers.xd in the ZebraSpeak Demo Project)

In this example, upon entering the screen, the Button Tab Handler is added to the Keyboard Handler's target list, so it is active.  It allows the user to tab between "Load Game" (button), "Save Game" (button), "Reset Game" (button), "Try Game" (the Game Tab Handler).  Pressing Return on any of the buttons "clicks" them.  Pressing Return on the Game Tab Handler makes it the active Tab Handler by adding it to the bottom of the Keyboard Handler's target list.  Now, Tab and Return will let the user play the game, and also tab to "Game Options" (the Button Tab Handler).  Pressing Return on the Button Tab Handler removes the Game Tab Handler from the Keyboard Handler's target list, which makes the Button Tab Handler the currently active Tab Handler again, since it is once again at the bottom of the list. (note, you can also stop a Tab Handler from receiving Events by hiding it, causing the other Tab Handler to receive the Events.)

Using Keyboard Handler with Tab Handlers in a series of switching sub-documents

Many iShell projects have one main container Document, and all screens are Subdocuments switched into this container.  This is also very easy to handle with a Keyboard Handler. 

(see an example in MultiSubDocs.xd in the ZebraSpeak Demo Project)

The container Document holds the Keyboard Handler; each Subdocument has a Tab Handler (where needed).  When the first Subdocument loads, it adds its Tab Handler to the Keyboard Handler's target list, making it the currently active Tab Handler.  When a Subdocument is unloaded, it's Tab Handler is automatically cleared from the list.  Then the next Subdocument, when loaded, adds it's Tab Handler, making it the active one.  And so on.

Key Handler order of operations:

Check for Space bar, arrow keys for speech mode.
If not used, goto the last item in the key handler list and check:
      if the item is a tabhandler
            if the item is shown and has tab items
                  if it's the tab key...
                        Post a tab leave event to the currently selected object.
                        Post a mouseleave event to the current selected object, if appropriate.
                        Move to the next tab item
                        Post a tab enter event to the new item
                        if appropriate, post a mouse enter event for the new item.
                        if speak object name is selected in the tab handler
                              if the object name does not begin with space, then speak the object's name
                        else
                              speak the object using the same rules as the Speak Object command.
                        trigger a tab keyboard event for the tabhandler.
                  if the key pressed is not a tab, and not escape, and the tabhandler currently has an item selected.
                        does the currentlly selected item have a corresponding keyboard event
                        if not, and the key is return
                              Post a tab return Event to the current selection.
                              Post an appropriate mouse event if needed
                              trigger a return keyboard event for the tab handler.
                              Speak the currently selected object's name, if appropriate.
            Does the tabhandler have any corresponding keyboard events
      if it's not a tabhandler
      does it have any corresponding keyboard events.
if the key is not used yet, try the above for the next last item on the keyhandler list
If the key is stilll not used, if it is enter, treat it as return and try again.
Otherwise, check to see if the KeyHandler itself has any keyboard events.

 

Keyboard Handler Control Command

Add To Target List adds the specified Element to the end of the Keyboard Handler's target list
Remove From Target List removes the specified Element from the Keyboard Handler's target list
Clear Target List removes all Elements from the Keyboard Handler's target list
Get Target List returns a list of all Elements in the Keyboard Handler's target list

 

Get/Set Attribute Command Options

Tab Handler Tab Handler Focus gets/sets the current Element tabbed to

 

Set Command Options

Runtime TTS Global Last Tab allows you to specify an Element to be at the end of every tab list in your project
(useful if you have the same navigation Element on each screen)

 

Get Command Options

Runtime Global Last Tab allows you to specify an Element to be at the end of every tab list in your project
(useful if you have the same navigation Element on each screen)
Runtime TTS Available gets whether speech engine is available
Runtime
 TTS CurrentDoneID gets the DoneID of the current Speak command
Runtime
 TTS On gets whether Text To Speech is turned on
Runtime TTS Speech Offset gets the offset of the text that is currently being spoken
Runtime
 TTS Paused gets whether Text To Speech is paused
Runtime
 TTS VolumeAdjust gets whether volume of other sounds is adjusted while Speaking

 

Other Notes

Included in the ZebraSpeak Download is a script that will disable JAWS Screen Reader when your iShell application is launched. You will need to update this file to reflect the name of your application. You will also need to document for the user how to install the script. A readme is included with the script.

 
Zebra Spot     Tel: 519.822.6788