Writing AxsJAX Scripts

Table of Contents

1 Setting Up Your Development Environment

While there are many ways to set up your development environment for working with AxsJAX, this tutorial will cover the easiest way to setup and get going. You may substitute any of the tools here for ones that you are more comfortable with that have the same functionality, but this guide will assume that you are using the tools listed here.

  1. Get Portable Firefox. This will give you a sandboxed version of Firefox with a clean profile so that you don't need to worry about conflicts with any existing extensions.
  2. Install the following extensions on your Portable Firefox:
    • Greasemonkey for injecting scripts.
    • XPather for interactively discovering XPath locators.
    • Event Spy for inspecting events as they happen.
    • Fire Vox for adding spoken output to Firefox.
  3. Download the "MICRO-PACKAGE" of Server2Go and unzip it to your hard drive.
  4. Save the axsSkel.js file under the "htdocs" directory of where you unzipped Server2Go in the previous step.
  5. Start Server2Go. You will get a browser window with a URL that looks like "http://127.0.0.1:4001/" and a page that says you are running Server2Go successfully. Make a note of what your URL is.
  6. Install axsSkelLoader.user.js as a Greasemonkey script. By default this will run on all pages; you may want to change this to only run on the page that you will be working on by going to Tools > Greasemonkey > Manage User Scripts and editing the "Included Pages" setting to just be your target page.
  7. Go to Tools > Greasemonkey > Manage User Scripts and select "axsSkelLoader". Click "Edit". Look at the line that reads:
     /* Modify this line to match where you have the script that you are working on */
     myScript.src = 'http://127.0.0.1:4001/axsSkel.js';
    
    and verify that this URL is indeed what your URL was in step 5. If not, change it to match and save the file.
  8. Go to the site you wish to apply AxsJAX to. You should get an alert box saying: "AxsSkel loaded and initialized!"

2 Applying AxsJAX to Content Rich Sites

  1. Write a Content Navigation Rule (CNR) for the site. Here is a skeleton CNR file to help you get started. An easy way to generate the XPath expressions is to use the DOM Inspector in Firefox with XPather. DOM Inspector allows you to interactively select nodes in the DOM; XPather shows you the XPath expression that would locate that selection. This will give you an initial xpath expression that you can then simplify and generalize.
  2. Replace this line:
    var cnrString = "PUT THE CNR XML HERE";
    
    in your axsSkel.js file with the CNR that you wrote in the previous step. You can get a copy-paste friendly string by using this tool.
  3. Try it out! At this point, the navigation should be working and you should be able to easily get to all the content on the page. If there is important content that you cannot reach, make sure that you are really including it in your xpath expressions. As in step 1, XPather and DOM Inspector are indispensable.
  4. If there is more functionality that needs to be enabled, you can augment the axsSkel.js file with your own functions that are tailored for the site.

3 Example: Enhancing Google Product Search Via AxsJAX

3.1 Writing the CNR file

The main trail here is the list of items returned by Product Search. Let's try to find the xpath for those items. Use DOM Inspector and click on the first result. That will select a node in the DOM tree. If that selection is too narrow, move up the tree until you select more. When you are satisfied with what the flashing rectangle has selected, note the XPath expression that you get.

Now use DOM Inspector and click on the second result. Notice that the second result's xpath is quite similar to the first result's.

The first xpath is:

 /html/body/table[3]/tbody/tr[2]
and the second is:
 /html/body/table[3]/tbody/tr[3]

Based on that, a good guess for the xpath for all the results would be:

 /html/body/table[3]/tbody/tr[*]
Let's put that into XPather and see what we get.

This is pretty close to what we want, but we are getting an extra node at the front. Notice that this extra node has a class name. To exclude it, let's change the xpath to only include nodes without class"

 /html/body/table[3]/tbody/tr[not(@class)]

Success! Now we only have the search results.

At this point, all we have to do is decide on the navigation keys we want to use and then write the CNR.

Here's the CNR file that we get.

3.2 Putting the CNR file into the JS file.

This step is pretty easy. Open up your axsSkel.js file and put in the CNR string.

This is the JS file that we get.

3.3 Load up Google Product Search and do a search.

Ok, not bad for a first attempt. It mostly works, but there are definitely some areas for improvement. For starters, the user isn't taken to the next page when they reach the end of a page of results and try to go forward. Similarly, they should be taken to the previous page if they reach the first result and try to go backward. Let's add targets with "listTail"/"listHead" triggers to address this. Just like in the first step, we use DOM Inspector and XPather to discover what the xpaths for the next/previous page links are. Also, the user should be able to just hit ENTER to go to the result, so let's add a target with ENTER as a hotkey for that as well.

Here's the CNR file that we get.

3.4 Add custom functions to improve the behavior of the page.

Now that we can easily go through the results, is there anything else that we can do to improve this page? As a matter of fact, there is! Listening closely to what the results sound like, we can hear that the spoken feedback is not exactly what we would like. For instance, the first image of the result has no alt text and some AT will read the entire URL. That image should just be omitted as there is no reason for the user to hear that. Also, the star rating is repeated since each star has an alt tag that says how many stars there are, and the text of the different parts of the result tend to run together. Let's write a short JavaScript function that will process each result by adding some smart formatting to it before speaking it. We can then use the "action" attribute of the item element in our CNR to call our function instead of using the default action of going to the node and speaking it.

While we're at it, we should also add a function that causes the first result on a page to be spoken when the page is loaded.

Here's the code for reformatting the results to create a much better auditory user interface:

 axsProductSearch.speakResult = function(item){
   var resultRow = item.elem;
   var title = axsProductSearch.getTitle(resultRow);
   var desc = axsProductSearch.getDesc(resultRow);
   var price = axsProductSearch.getPrice(resultRow);
   var seller = axsProductSearch.getSeller(resultRow);
   var ratings = axsProductSearch.getRatings(resultRow);
   var checkout = axsProductSearch.getCheckout(resultRow);
   
   var message = title + '. ' + 
                 price + '. ' +  
                         desc  +                               
                                 seller + '. ' +
                                 ratings +
                                 checkout;
   
   axsProductSearch.axsLensObj.view(resultRow);
   resultRow.scrollIntoView(true);
   axsProductSearch.axsJAXObj.speakTextViaNode(message);
 };

Now, what the user hears for the above screen shot is:

Google e. encyclopedia. $29.99. In partnership with Google, DK presents the e.encyclopedia, a revolutionary approach to children's reference publishing. An illustrated general encyclopedia … Buy this at Adoremus Books. 64 seller ratings averages out to 5 stars. Accepts Google Checkout.

Here is the JS file that we get at the end.

3.5 Wrapping Things Up

This AxsJAX script for Product Search is basically done. Of course, there are a few things to polish up such as adding controls for search refinements. To see how we're evolving this script, please see the "productsearch" directory in the AxsJAX project SVN repository.

4 Applying AxsJAX to Highly Interactive Web Applications

  1. Make sure the app is keyboard usable. The goal is to make it possible to do common actions from the keyboard very quickly; users should never need to tab through multiple things on the page just to get at something.
  2. If there are static portions on this app that could benefit from CNR, follow the steps in the section above on Applying AxsJAX to Content Rich Sites. Otherwise, replace this line:
    var cnrString = "PUT THE CNR XML HERE";
    
    with
    var cnrString = "";
    
  3. Decide on an action whose result you wish to speak,
    1. Use DOM Inspector to find and select the node of the app that contains content which you know will change when you perform an action from the keyboard.
    2. Right-click on the node in the DOM tree and choose "Watch events…" to bring up Event Spy. Make sure that "Mutation" is checked and nothing else.
    3. Perform the action from the keyboard and examine the event log to determine what events are fired when the action is performed.
    4. Using this, write code for the axsSkel.nodeInsertedHandler or axsSkel.attrModifiedHandler that will speak the result of the event.
  4. Repeat the last step until all actions have their results spoken.
  5. Try it out! Think of the different tasks that users will want to do with the the app. Is there a smooth workflow that can be done efficiently from the keyboard alone? Is there enough auditory feedback so that the application can be used without a monitor?
  6. If there is more functionality that needs to be enabled, you can augment the axsSkel.js file with your own functions that are tailored for the site.

5 Example: Enhancing Google Reader Via AxsJAX

5.1 Making the app keyboard usable

There's nothing to be done in this step since Google Reader is a power-user app that has keyboard shortcuts for pretty much everything.

5.2 Writing a CNR if necessary

This step can also be skipped for Google Reader as the navigation through the feeds and articles is done entirely by Reader.

5.3 Speaking the result of navigating the feeds

Let's start by enabling spoken feedback for navigating the list of feeds. Google Reader provides Shift + n / Shift + p to navigate forward / backward in this list. When users do this, there is a highlight that indicates where they are as shown in the screen shot below.

Now that we know what area is going to change, we can watch for changes in that area of the DOM using Event Spy.

First, start up DOM Inspector, find the area that will change, and right click on its node in the DOM tree. Then choose "Watch events…"

Check the "Mutation" category, go back to the Reader, then hit Shift + n to navigate to the next feed. You will see that the Event Log now has captured a few events.

Uncheck the "Mutation" category so that you won't accidentally get more events. Let's explore this log to see what caused the highlighting effect to appear. It would make sense that something like this is done via CSS with classname changes, so we should focus our search on the DOM events which involve class. After looking through the changes, we see something that looks quite promising. It is a DOMAttrModified event that has:

 attrName = 'class'
 prevValue = 'link'
 newValue = 'link cursor'
This is the type of change that we expected; it is a change to the classname, and the class name changes seem quite reasonable. The feed name was styled to look like a regular link before we navigated to it, and then afterwards, it still looked like a link but also acted as a cursor since it indicated our position. And finally, the target of this event points at "Cool Tools", which is the feed that we navigated to when we pressed Shift + n earlier.

With this information, we are ready to write a few lines of JavaScript to cause the user's assistive technology to speak when they navigate through the feeds. Since the event we are interested in is a DOMAttrModified event, we should modify axsSkel.attrModifiedHandler by adding a "if" condition that matches the events that are occurring with the event that we are interested in. This is what the modified axsSkel.attrModifiedHandler looks like:

axsSkel.attrModifiedHandler = function(evt){
  var attrib = evt.attrName;
  var newVal = evt.newValue;
  var oldVal = evt.prevValue;
  var target = evt.target;
  // If the target node is something that should
  // be spoken, speak it here.
  if ( (attrib == 'class') && 
       (oldVal.indexOf('cursor') == -1) &&
       (newVal.indexOf('cursor') != -1) ){
    axsSkel.axsJAXObj.speakNode(target);
  }
};

5.4 Speaking the result of user actions

We can repeat what we just did for speaking the result of navigating the feed list and do the same for other user actions. For example, reading the articles when the user navigates to them.

In this case, after looking at the Event Log, we find that the change is done by changing the ID of the article to "current-entry". Here is the code that we need to add to axsSkel.attrModifiedHandler:

  if ( (attrib == 'id') && 
       (oldVal.indexOf('current-entry') == -1) &&
       (newVal.indexOf('current-entry') != -1) ){
    axsSkel.axsJAXObj.speakNode(target);
  }  

Here is the JS file that we get after providing speech feedback for navigating the feeds and their articles.

5.5 Load up Google Reader and try it out

Just by adding two "if" statements to the axsSkel.attrModifiedHandler, we have dramatically improved the usability of Google Reader. Now, the user can use the built-in Reader shortcut keys to navigate the feeds and articles and hear spoken feedback.

There are still a few improvements that we can make. For example, users should get feedback when the articles are first loaded. To sighted users looking at the screen, it is obvious when the articles have loaded, but to enable a good eyes-free experience, it is important to let users know when the loading has finished so that they know they can proceed.

Another needed improvement is to let users know when they have starred or unstarred an entry.

5.6 Putting on the finishing touches

We can make these two improvements quite easily by using the same techniques we used earlier. Here's the code that we need to add to axsSkel.attrModifiedHandler:

 if( (target.id == 'viewer-box') &&
       (attrib == 'class') &&
       (oldVal.indexOf('hidden') != -1) &&
       (newVal.indexOf('hidden') == -1)  ){
     axsSkel.axsJAXObj.speakTextViaNode(axsSkel.ARTICLES_LOADED_STRING);
   }
   if( (attrib == 'class') &&
       (oldVal.indexOf('item-star-active') == -1) &&
         (newVal.indexOf('item-star-active') != -1) ){
     axsSkel.axsJAXObj.speakTextViaNode(axsSkel.ITEM_STARRED_STRING);
   }
   if( (attrib == 'class') &&
       (oldVal.indexOf('item-star-active') != -1) &&
         (newVal.indexOf('item-star-active') == -1) ){
     axsSkel.axsJAXObj.speakTextViaNode(axsSkel.ITEM_UNSTARRED_STRING);
   }

Here is the JS file that we get at the end.

5.7 Wrapping Things Up

This AxsJAX script for Google Reader now has most of the functionality that we want. Of course, there is still more to do. To see how we're evolving this script, please see the "reader" directory in the AxsJAX project SVN repository. Also, Reader has already begun shipping its AxsJAX script as part of the product itself.

6 References

Here are useful references for additional reading:

Author: T.V. Raman, Charles L. Chen <raman@google.com> <clchen@google.com>

Date: 2008/07/03 15:15:54

HTML generated by org-mode 6.05a in emacs 23