[Nyabs] The Death Of Screen Reader Innovation

[NYABS Board]

The Death Of Screen Reader Innovation
http://chrishofstader.com/the-death-of-screen-reader-innovation/

On the Monday of Thanksgiving week 2004, I walked into my St. Petersburg office for the last time as a Freedom Scientific employee. I had, at that point, been at the helm of the FS software engineering department for six years but, as Lee Hamilton, then CEO of Freedom Scientific told me that day, I was, “no longer capable of managing the team.” He was right, I had burned out very badly and was in terrible physical and mental shape. My repetitive strain injuries (RSI) caused me constant pain and the Vicodin prescribed by my physician had too many cognitive side effects to permit me from having a clear enough mind to perform my tasks while the steroidal injections I received for the same injuries had intense emotional side effects that felt like I was on cocaine or methamphetimine.

While my final few months at Freedom Scientific were a personal disaster for me and not too good for FS or its customers, I am very proud of the many things we accomplished during my six year tenure. During that time, Eric Damery, Glen Gordon and I invented a ton of new screen reader features now seen in all such utilities on Windows.

What Did We Accomplish?

In my time at FS, six years ending in November 2004, our little team released versions of JAWS beginning with 2.51 (a minimal update to fix an authorization problem in the 40 minute demo) to JAWS 3.31, the first screen reader to add what is now the essential virtual buffer on the Internet in all screen readers to JAWS 6 the last JAWS in which I participated. In those release we added new and interesting features with each revision and pushed the user interface of screen readers forward every six months or so.

In that period at Freedom Scientific we:

Invented the virtual buffer concept for delivering web information to JAWS users.
Invented the idea of querying applications through a private interface to gather and present information to our users – a concept used today in JAWS, Window-Eyes, NVDA, System Access and Orca screen readers.
Provided the first ever ways of reading charts and graphs in a major screen reader.
Advanced usability of office suites in a way never previously seen in a screen reader.
Invented the now ubiquitous “QUick Keys” style of navigating web pages more efficiently.
Added the JAWS “Speech and Sounds Manager” for adding tonal augmentations to information, thus, expanding the number of simultaneous semantic dimensions enjoyed by users.
Added features to “intelligently” skip beyond repeated information on web pages.
Added features to recognize similar documents and spreadsheets and automatically apply a set of adjustments for reading the data.
And many more innovations in screen reading user interfaces that you can find by looking up the “What’s New” sections of the release notes for these JAWS releases..
Returning To Windows

I hadn’t used a Windows computer or a Windows screen reader in more than five years but, recently have found that VoiceOver on Macintosh can’t provide the level of support I need so, I downloaded and installed VMWare Fusion, Windows 7 and NVDA. Once everything was up and running, I started exploring this terrific free screen reader.

The first thing I noticed was that NVDA has adopted many of the ideas that we invented in JAWS. I breathed a sigh of relief when I realized that, thankfully, most of our innovations happened before FS became crazy about patent applications and aggressive lawsuits so our most important ideas seem to have made it to other screen readers and it’s nice to see my work so widely accepted in virtually all Windows screen readers and in Orca on the GNU/Linux platform.

I continued to read the NVDA documentation to see what new and noteworthy concepts have been invented in the nine years since I walked out of Freedom Scientific on that November morning. The first thing I found was a gesture based navigation system, a notion first introduced by Apple in VoiceOver for iOS and adopted by NVDA first and, based on statements made at CSUN this year, will be added to JAWS this autumn. I looked further and found nothing new. I read up on JAWS, once the hands down leader in innovation and found nothing new there in many years either.

So, in a decade, the only new ideas in screen readers have come not from a small, highly focused screen reader company but, rather, from a mainstream super power. Ted Henter’s prediction that big mainstream companies making screen readers would result in a failure to innovate seems to be partially true, Apple innovated at first but has allowed the OSX version of VoiceOver to deteriorate and they haven’t expanded on really good ideas that are present but incomplete but, contrary to his prediction, the smaller, blindness specific companies have done nothing new or interesting in years either.

Was I That Important?

At Henter-Joyce and, after our merger, Freedom Scientific, the best ideas came out of a collaboration mostly between me, Eric Damery, Glen Gordon and Joseph Stephen. Some of these ideas started as hallway conversations with other employees, phone conversations with beta testers, questions from our technical support staff and lots of other sources. I was usually the person who wrote up the ideas into a formal specification (FS product managers seem to have some kind of innate aversion to writing anything down) and, sometimes, I originated the notion but, by the time it became a task to add it to JAWS, the concept would have been thoroughly reviewed multiple times by Eric, Glen, Joe and, often, Ted Henter.

Eric, Glen, Joe and lots of other smart and interesting JAWS users (yes, although sighted, Eric can use JAWS with any power users out there) remain at FS. I don’t think they lost all of their creativity and ability to come up with new and valuable features for JAWS so what has gone wrong in the screen reader business?

Leaders Who Are Blind

As far as I can tell, other than Mike Calvo, CEO of Serotek, makers of the System Access screen reader, I was the last blind person with direct authority over a commercial screen reader and, as System Access has never had a large user base, the last blind person with direct authority over a widely used commercial screen reader. System Access, in its hey day was a pretty innovative screen reader and Mike’s influence as a user advanced features like their “See Saw” global “dictionary” for web sites, the end of “Forms mode” (something Glen and I invented that wasn’t the greatest idea), the first ever “virtual” screen reader and a variety of other excellent concepts. I’ll contend that these happened in SA because it is a project led by a blind user of the software.

Reexamining how we invented things at Freedom Scientific, I recall that, often, the best ideas came out of frustration. Joe Stephen wanted to read notes in braille different from the contents of PowerPoint slides so we added the ability to have one stream of information go to the speech synthesizer while another went to the braille display. I got sick of making all of the JAWS settings just to read the FS financials on a weekly basis so we invented a way that JAWS could recognize different but similar spreadsheets and automatically apply the settings. We invented Quick Keys while Glen and I talked on the phone about single letter navigation in some emacs scripts. Most of our best ideas grew out of desires by actual users in our employ.

Why Doesn’t NVDA Take The Lead?

NVDA is an excellent free, no cost screen reader for Windows. As far as I know, all of its code is written by blind people who use the software as their primary means of interacting with Windows computers. Why, then, don’t they innovate?

NVDA, as far as I can tell, is written almost entirely by two guys. These guys are really smart and creative fellows and were the first to bring gesture navigation to Windows for screen reader users, a concept, while not novel, that is both powerful and useful. Nonetheless, it is a large and complicated bit of software that requires all of the maintenance of JAWS without anything approaching the FS ability to invest in a project.

Screen Reader Funding Models

There are three major ways to fund a screen reader’s development. A large mainstream company like Apple or Microsoft can make their own screen reader motivated by federal and state regulations requiring accessibility as a condition of sale. A free software screen reader like NVDA or Orca can be funded by corporate dollars contributed to the effort, by contributions from users and the general public and by selling services to support development. Lastly, a commercial technology company like Freedom Scientific or GW Micro can fund their research and development costs from sales of their products.

When I left FS in 2004, we were selling about $1.2 million worth of JAWS per month. We were spending less than one million dollars annually to make JAWS (including software engineering, testing, scripting, product management and sales roll outs) and, when I ran the project, I insisted we spend some of this money on innovation in JAWS and our other software products.

One might ask, if you made about $15 million on JAWS sales, how did Freedom Scientific spend the money? In that final year at FS, we reported a total profit to our investors of $6 million so where did $9 million in JAWS sales alone go?

The answer is simple, Lee Hamilton (then FS CEO) and the FS board of directors saw blindness and low vision hardware as their future and invested heavily in the hardware division while allowing JAWS to stagnate. The authors and publishers of the then world’s leading screen reader milked its sales dollars to try to expand their business with products like PAC Mate, Focus and a whole pile of digital magnifiers. So, for all intents and purposes, new JAWS sales and SMA dollars were spent not on the screen reader but, rather, incredibly high margin hardware devices – an outcome not terribly good for JAWS users.

At the time I left FS, it cost the company about $400 to manufacture, package and ship a PAC Mate BX 440 (then a fast processor and a 40 cell display) and we would sell them for about $5000. JAWS, on the other hand, sold for about $900 so, the most we could make on the software was limited by its price while we could enjoy $4600 of pure profit on each PM sold. Today, I’m told that a digital magnifier costs a tiny amount in parts and manufacturing but they also sell with windfall level profits.

For its business, taking the lucrative JAWS as a funding source for hardware made a lot of sense. It did nothing for the companies customers though.

The Fight

I have always been one who fights for what he thinks is right. While at Freedom Scientific, I had huge arguments with our CEO over whether or not we should invest in a wide variety of different ideas. I lost most of these battles but had my share of wins and I’m proud of the work we did with JAWS, OpenBook, MAGic and our other software projects back then. At the same time, I can only think that what stopped innovation at FS may be the lack of “fight” in those I left behind.

More so than anyone else at FS other than me, Eric Damery has always been the guy to push hard for new features that can improve the experience for JAWS users. Meanwhile, Eric is bound by the toughest non-compete agreement in the business and a salary that would be impossible for him to get in any other field. Eric is bound by contract and golden handcuffs from doing anything too bold. I was stupid, I thought I could leave Fs and continue in accessibility but, over my first two years out of FS, they threatened me with legal action nine separate times and I serve as an example for any other FS executive who may consider moving on.

Glen Gordon, one of the smartest and most creative software developers with whom I’ve ever had the pleasure of working, is the FS chief technical officer who had it really easy when Ted Henter ran the company. When Ted was around, we merely had to pitch a good idea and Ted, a user himself, would embrace it and we could go off and do the implementation. After the merger, Glen retreated and would ask me to fight the good fight, a role that ultimately destroyed my career at the company.

Joseph Stephen, Rob Gallo and some others with a high level of creativity are in positions too junior to have access to the executive committee and others in decision making positions. Hence, without Eric or one of the blinks in the executive suite fighting for an idea, something my sighted replacement hasn’t (according to reports I get from friends who remain inside FS) even once.

So, while the ideas weren’t mine exclusively, my role at FS led both the advancement of JAWS and that of screen readers from our competitors. Without JAWS leading, I doubt any Windows screen reader ever will again unless Microsoft decides to either license NVDA or make a truly usable Narrator.

What Did Users Miss?

I cannot speak to any ideas discussed within Freedom Scientific after I left the company in November 2004. I suspect Glen and Eric presented a bunch of good ideas that have been ignored. I can, however, speak to the projects I had going at FS that were canceled upon my departure. I’m not sure all of these were good ideas but, compared to what FS has done since, they were certainly bold, creative and, perhaps useful. As these notions never saw a commercial release, we cannot know the actual impact they may have had if they fell into the hands of users in the wild.

The projects they killed were:

JAWS for Macintosh was requested of us by the third party development people at Apple. I had formed a relationship with the people at Apple and, when they asked us to propose doing a Macintosh version of JAWS, I pitched the idea at an executive staff meeting and was nearly laughed out the door. In the latest marketshare figures I could find, VoiceOver and NVDA are the only two screen readers showing growth. Could we have made something better than VoiceOver? I think so.

JAWS on mainstream Windows Mobile devices was not just possible but something we already had running on both iPaq and the Dell Axim. Lee Hamilton said that such would compete with PAC Mate and, instead of allowing JAWS to grow with mainstream hardware, FS insisted on forcing its blind users into the ghetto with a massively more expensive hardware device with all of the social grace of a brick. Our friends at Code Factory did make this into a reality but with their minimal ability to market their products, uptake was slow and, as the accessibility of the Windows Mobile platform ecosystem decayed, their screen reader died on the vine.

We had started a project into making JAWS for Symbian cell phones. While today, years after the death of Nokia’s OS, this seemed like a bad idea, then, with Nokia on top of the world, we could have produced and sold a killer screen reader on more mainstream hardware. Again, the Code Factory team and my late friend Torsten Brandt, with his Talx screen reader (sadly, I could not find online references to either my late friend or his software to add as a link) got their first and provided excellent access for a whole lot of years.

We had a blind mathematician and screen reader power user designing a 3D interface for a tool for reading and writing math. Ted Henter had already started HenterMath and was doing some really interesting things with user interface for blind people who wanted to manipulate equations but he was also based in JAWS as it was, a unidimensional approach to the information. I thought we could do something really great with math for our users but, within a week or two of my departure, the project was killed. Blind people, meanwhile, have no good math tool to this day.

We had my friend Will Pearson on the payroll as a contractor working on a “machine vision” approach to screen reading. At the same time, we had a terrific gal, an awesome hacker with a solid computer science background working toward a rectangular approach to a screen reader interface. Combining these two concepts, we could have provided a layout similar to one with vision would see. We could have used positional information to better increase reading efficiency while also presenting the semantic information that comes from a visual layout – ideas that, to my knowledge, have only otherwise been explored by researchers in labs.

Where Did FS Invest In JAWS?

On the day I left FS, my friend and terrific young hacker, Waishan Lau was leading the cell phone screen reader project. Within a week of my departure, her project had been canceled and she was put back onto her previous task, namely, JAWS authorization. Waishan, now working in California at a health informatics business in Palo Alto, is one of the hottest hackers with whom I’ve ever worked. She is a brilliant and beautiful young woman who can code rings around most others in or out of FS. Her terrific talents were used almost exclusively on copy protection – a feature that no user actually enjoys.

Freedom Scientific does continue to invest in JAWS. Their advancements, though, are minimal. They support the latest and greatest MS operating systems, they try to support the most recent versions of Microsoft Office and, now and then, they add a new feature that will be mostly ignored by the user base. FS is happy making its annual release and taking the SMA dollars from its users whether or not they add anything of any actual value to those users. Supporting the latest OS may not be easy but it’s something that no other software companies charge for.

So, Maybe I Was Important

For years, I thought my years at Freedom Scientific had ended in failure. I couldn’t, due to health problems, continue to lead that team any longer. I still had the odd good idea but constant chronic pain while unable to sleep from the steroids drove me into a very bad place. Fighting every step of the way for innovation in JAWS and our other products nearly killed me.

Looking back, though, on the accomplishments we made, the major strides in screen reader user interface and the tremendous improvements we made in the lives of JAWS users in the workplace, I’m reminded that we did a lot of good stuff.

Perhaps, our success relied on the odd combination of actors in our story. Eric Damery’s undying commitment to our users, Glen Gordon’s software engineering genius telling us what was possible, Joseph Stephen adding ideas to JAWS application support while we were asleep and my creativity mixed with my insanity, a crazy that says that we can do incredible things in spite of having to fight within the company to do so that led to those great years in screen reader development. I don’t know why it worked so well for us in those days but I will state that I believe it had more to do with having a blind person in charge of the product than any of our individual contributions.

I made a huge difference at FS because, in spite of having to fight so much to get anything interesting done in the post merger era, I was willing, no matter how much pain I experienced, no matter how personally painful it was to see great ideas and support for mainstream hardware killed, I kept fighting until my last day. Without such an advocate within the company, I can’t see innovation happening again until a major screen reader is once again led by a user.

(via Instapaper)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nfbnet.org/pipermail/nyabs_nfbnet.org/attachments/20130728/22a41a08/attachment.html>