[nfbcs] Should JAWS be used for web accessibility testing (was Re: Opinions?)

David Tseng davidct1209 at gmail.com
Mon Feb 20 05:52:45 UTC 2012 

Hi guys,

Lots of thoughtful replies already to this thread.

I whole-heartedly agree that companies should test with as many AT's
(including speech recognition, magnifiers in addition to screen
readers). However, when it comes to the practical day-to-day
implementation of such a policy, it's fairly clear, at least on a
technical level, that this type of user-centric testing goes almost no
where without an accessibility expert working on the inside.

It just happens that most web developers don't speak accessibility.
The actual process of fixing many of the issues confronted by screen
reader users winds up traveling quite a convoluted path. On one end of
the spectrum, we've got web developers who live in the land of HTML,
CSS, javascript, js toolkits, web standards, etc. On the other end,
there are legacy (and not so legacy) screen readers that use in many
cases closed-source, proprietary, platform specific technologies and
techniques to read the screen. In the middle, you have again
proprietary (at least on Windows) accessibility infrastructures at the
operating system level. On top of this, you have potentially if we're
singling out web, various browsers that vary in their support of the
latest and greatest standards whether it be HTML5, or platform
specific API's like IA2, UIA, etc.

Tell a developer that "Jaws isn't reading a particular table's
contents correctly or Jaws isn't reading a form field" means that
someone has to translate down to apply x attribute to y control as the
"proper" way to inform the control of it's label.

That's just a *very* simple example. In contrast, there are some
extremely fundamental drawbacks to the way Jaws (and others) control
focus that makes it a challenge at best and impossible at worst to
deliver modern web apps to screen readers of a specific class.

Also, to point out somewhat of an assumption made on this thread, any
of the parties listed above can prove to be the weakest link. Some
screen reader issues do stem from bugs in the screen reader. Likewise,
there have been more than a few bugs in MSAA itself, the browsers, and
of course, the markup/script from the content author.

At the end of the day, automated tools provide a great front-line and
a clear, explicit path to resolving a bug (whether it be access
related or not). Testing with screen readers leads to mixed results,
but can be useful if the organization in question has the resources to
investigate and has a good *technical* contact to ask when ambiguity
arises.

Accessibility testing falls short in many cases because providing the
"what" is only half the story, if that; presenting the "why" and "how"
(to fix) a particular issue on a developer level proves invaluable.


On 2/19/12, Mike Jolls <majolls at cox.net> wrote:
> I hear you  when you speak of how the coded material is interpreted.  I
> probably don't need to go much further than to use HTML as an example ...
> you can code it so it works, but the coding ... even though it works ...
> doesn't follow the coding rules.  As a result, a screen reader would get
> confused if it was expecting a strict following of the rules (at least with
> respect to HTML) 100% of the time.  And another comment about the rapidity
> of changing technology makes it difficult to for screen reader developers to
> keep up.  Yes, I recognize all of that.  Being a developer myself, I can
> appreciate those problems.
>
> I'm going to go out on a limb here .. and maybe I need to get my skills
> updated a bit ... but it's too bad that a technology like XML couldn't be
> followed when sending information to the browser ... rather than HTML.  XML
> ... if paired with a well defined DTD would require that the XML data is
> correct and it could reduce problems for screen readers trying to interpret
> improperly formatted content.  Perhaps if the browser could receive the data
> in XML instead of HTML you'd stand a better chance of preventing the example
> I'm citing above.  Of course as I implied in the previous paragraph, HTML is
> not the only technology where problems could occur.  Many new technologies
> are developed and the screen reader companies have to keep up with those
> technologies.  Since there is such a thing as lag, I suppose the screen
> reader companies will always be behind the proverbial eight-ball.  Of course
> in the software business where I work, we call that job security.  But that
> doesn't make it easy on blind individuals that depend on using screen
> readers for their delivery system.
>
> Just a few thoughts.
>
>
> -----Original Message-----
> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
> Of Mike Freeman
> Sent: Sunday, February 19, 2012 5:45 PM
> To: 'NFB in Computer Science Mailing List'
> Subject: Re: [nfbcs] Should JAWS be used for web accessibility testing (was
> Re: Opinions?)
>
> I agree with you, Doug. Moreover, what we're dealing with here is not how
> something looks on the screen (which can be coded in a bazillion different
> ways) but how that coded material is interpreted by the screen-reader; these
> are two different things.
>
> As I say, I think the only cogent solutions are to mandate a rigorous set of
> tests with the several screen-readers or to restrict how content can be
> coded for interpretation by said screen-readers so they have a prayer of
> knowing how material will be presented to them for interpretation by the
> blind user.
>
> Mike
>
>
> -----Original Message-----
> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
> Of Doug Lee
> Sent: Sunday, February 19, 2012 3:09 PM
> To: NFB in Computer Science Mailing List
> Subject: Re: [nfbcs] Should JAWS be used for web accessibility testing (was
> Re: Opinions?)
>
> Problem is, I don't think this is so much about what features a screen
> reader should have. It's more about how screen readers implement those
> features, and how the differences among these implementations cause
> different behaviors depending on how developers developed their software and
> web sites. The API idea is good, and it's been tried, in the form of things
> like MSAA, Java Access Bridge, UIA, etc. But so far at least, the attempts
> on standardization of methods for screen readers to use have not resulted in
> complete uniformity among screen readers as to how well their features work
> in all situations.
>
> On Sun, Feb 19, 2012 at 04:54:40PM -0600, Mike Jolls wrote:
> OK, let me put my two cents in here.  And I'll hopefully make this brief.
>
> If I'm hearing this right, I'm hearing that one screen reader (say it's
> JAWS) is made the standard, and all other players should play by the
> features that are incorporated in the chosen standard software package.
>
> I say that's a bunch of hooey.
>
> What needs to happen is a standard set of functions ... arrived at by a
> board of computer savvy blind people ... should be agreed upon.  Such as...
> 1. Navigate from frame to frame
> 2. Navigate from word to word
> 3. Navigate from paragraph to paragraph
> 4. There are many more .. these are just examples.
>
> And this group of blind people should be computer educated (i.e. trained
> people who understand about computer technology, what type of data is going
> to appear on the web) who understand what needs a blind person has when they
> want to access content on the web ... that is if we're only talking about
> accessing the web.  If we're talking about accessing other things such as
> FTP servers and other technology, these people would need to be versed in
> those technologies as well.
>
> Now, with a group of people who understand the media to be accessed and what
> is possible, they agree to a standard set of functions that EVERY company
> that writes screen reader software must support.  In fact, perhaps this
> savvy group if knowledgeable in computer technology, could come up with an
> API (application program interface) that they could design that every
> producer of screen readers would have to adhere to.  If the set of features
> outlined in the API was complete, then any company producing screen reader
> technology would have to implement at least those functions, and every
> package would have at least the same basic set of functions usable by blind
> people that the standards board decided was important and necessary.  Each
> product by different vendors wouldn't have to implement them the same way
> (i.e. CTRL-R might "Read the entire line" for Company A's product, while
> CAPS-LOCK-I would do it for Freedom Scientific JAWS) ... but they would have
> to implement the agreed upon standard set of functions.  This would go for
> cheap software, or the cadillac versions.  That way, a blind person would be
> given choice, based on what he could afford.  Heck, if there was a published
> API, maybe Microsoft might even get it right and embed a screen reader in
> their O/S.
>
> In this way, different companies could produce their own products, there
> could be competition, there could be cheaper and more expensive models, and
> users would still get what they needed ... plus any additional features the
> more expensive packages offer.
>
>
> So that's what I think needs to be done to solve the issue of which screen
> reader to buy.  Which one you should buy shouldn't matter if they all
> supported the basic "have to have" features.
>
> Thoughts?
>
>
>
> -----Original Message-----
> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
> Of Mike Freeman
> Sent: Sunday, February 19, 2012 3:52 PM
> To: 'NFB in Computer Science Mailing List'
> Subject: Re: [nfbcs] Should JAWS be used for web accessibility testing (was
> Re: Opinions?)
>
> Jim:
>
> Thank you for your articulate response. Here's my answer.
>
> With regard to my setting the bar too high for testing, you bet your sweet
> posterior I am! In fact, I think that we as a society in general and
> companies in particular have become *way* too sloppy both in coding
> standards and in the level of perfection we expect from the software and
> hardware we use. There are computers using older operating systems where I
> work that have been up -- continuously -- for nearly two years. Put *that*
> in Windows' pipe and smoke it! And way too many bugs slip through in vendor
> software (both Microsoft and otherwise) these days because the testing
> wasn't sufficiently vigorous to catch them. And yes, I realize that were the
> sort of testing I advocate done, it might slow down web site and software
> changes to perhaps two a year! Bravo! That's *exactly* what I think should
> happen. If corporations and individuals knew they'd only have two chances a
> year to make changes, they might concentrate a bit more on getting things
> right the *first* time rather than using the users as beta-testers!
>
> I know you'll object that content changes more often than this on web sites.
> You're right. But I see little reason for this -- especially as a good bit
> of the change is pure glitz designed to try to attract new viewers etc. Why
> not assume people to be literate and intelligent and try to attract them
> with well-written *text* content?
>
> But I realize I'm jousting at windmills on this one. Given this, then, what
> should we do? I think your suggestion of picking a standard screen-reader
> that doesn't depend upon direct sales is impractical at best. I maintain
> that eventually *all* screen-readers will have to be maintained based upon
> sales; even the purveyors of NVDA are having to admit that they might be
> hungry and would like some contributions. If we admit that there's no such
> thing as a free lunch and if we admit that changing screen-readers involves
> more than a sighted person goes through in changing browsers because his web
> experience isn't as good with one as with another, the only alternative I
> then see short of my utopian test suite and regime would be to specify
> allowed and disallowed web constructs and software constructs and, say,
> limiting them to, for example, what one could do with standard windows
> controls as implemented in Windows-98. In other words, as I see it, we
> either allow innovation and insist upon a rigorous testing regimen or we
> specify that no innovation beyond a certain level is allowed with the
> penalty for violation being the immediate job termination of a company's CEO
> and executive board.
>
> And we know that isn't going to happen either and probably it shouldn't
> (although I confess that a good bit of nonsense in today's society would be
> solved if people knew their jobs were on the line for foisting it on the
> rest of us!).
>
> Mike
>
> -----Original Message-----
> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
> Of Jim Barbour
> Sent: Sunday, February 19, 2012 12:16 PM
> To: NFB in Computer Science Mailing List
> Subject: Re: [nfbcs] Should JAWS be used for web accessibility testing (was
> Re: Opinions?)
>
> Oh, this is very interesting.  Thank you Mike for articulating the other
> side of this.
>
> I'd like to quote and then respond two items from your message.
>
> First Quote:
> "I think the only way to do this right would be to specify that *every* site
> should be put through a suite of tests by *human* *beings,* not automated
> tools, using the following screen-readers at a minimum: JAWS, Window-eyes,
> Hal, SuperNova, System Access, NVDA, Coco (sp) and VoiceOver (both on
> i-devices and on the Mac). It's a matter for debate whether or not one
> should specify note-takers such as the BrailleSense and BrailleNote family
> also to be tested."
>
> Ignoring for the moment the word "every" in your statement above, let's
> think about large companies that want to test accessibility, such as Google,
> Yahoo, and Facebook.
>
> You're proposing that these companies stage human run usability tests on
> *each* new or changed product using 8 to 10 different AT solutions.
>
> As a comparison, none of the companies I mention above would consider doing
> this amount of mainstream usability testing against different web browsers.
> I believe that this is way to high a bar for us to expect any web site
> developer to meet
>
> Instead, we should pick a model screen reader, and begin to insist that it
> be used in *all* AT usability testing.  That screen reader could then be
> given the responsibility of reporting out problems they encounter for which
> there is no documented solution.  This puts that company in the drivers seat
> for finding a solution.
>
> For this to work, we need a screen reader whose development is not financed
> by direct sales.
>
> As to the word "every", I don't think you really mean every web site
> developer.  If I develop a photo sharing site for my family to use, say
> www.barboursphotos.com, this probably should not have to pass the
> accessibility test.
>
> The question about who should be obliged to pass any accessibility tests is
> a sticky one.
>
> And, your second quote
>
> "In fact, I think the article's author is desperately trying to find a way
> to lessen work for himself or, put another way, he is hoping he can be lazy
> and not do the sort of in-depth testing that is truly required for good
> accessibility testing."
>
> So, of course he is.  He's trying to minimize the amount of effort needed to
> make his web sites accessible.  This is reasonable and makes perfect sense
> to me.  Why wouldn't you expect any developer of any product to increase
> their efficiency?
>
> Jim
>
> On Sat, Feb 18, 2012 at 08:49:22PM -0800, Mike Freeman wrote:
>> Jim:
>>
>> I respectfully, but strongly, disagree. Although I argue in another
> message
>> that there's no good way to include or exclude a particular
>> screen-reader from accessibility or usability tests, I also think that
>> excluding a particular screen-reader amounts to a value judgment even
>> if it is not intended as such. Consider how irked Window-eyes users
>> get when everyone tests their sites against JAWS. Why should JAWS
>> users put up with the same sort of nonsense?
>>
>> In fact, I think the article's author is desperately trying to find a
>> way
> to
>> lessen work for himself or, put another way, he is hoping he can be
>> lazy
> and
>> not do the sort of in-depth testing that is truly required for good
>> accessibility testing.
>>
>> I think the only way to do this right would be to specify that *every*
> site
>> should be put through a suite of tests by *human* *beings,* not
>> automated tools, using the following screen-readers at a minimum:
>> JAWS, Window-eyes, Hal, SuperNova, System Access, NVDA, Coco (sp) and
>> VoiceOver (both on i-devices and on the Mac). It's a matter for debate
>> whether or not one should specify note-takers such as the BrailleSense
>> and BrailleNote family also to be tested.
>>
>> The only alternative I can see would be to try to get all
>> screen-readers
> to
>> behave the same way and, my friends, that ain't a-gonna happen! (grin)
>>
>> Mike Freeman
>>
>>
>> -----Original Message-----
>> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On
>> Behalf Of Jim Barbour
>> Sent: Saturday, February 18, 2012 8:21 PM
>> To: NFB in Computer Science Mailing List
>> Cc: NABS-L
>> Subject: [nfbcs] Should JAWS be used for web accessibility testing
>> (was
> Re:
>> Opinions?)
>>
>> I am in 100% agreement with the statement that JAWS should not be used
>> for web site testing.  However, my reasons differ from the ones
>> written in the article.
>>
>> It is not possible today to design and build accessible websites
>> without performing usability tests.  Further, there are too many
>> access technologies to test with them all.  So, the question is which
>> AT should be used to test, and therefore drive improvements to, web
>> site accessibility?  Whichever one gets chosen will have the
>> opportunity to informally set standards around how certain types of
>> content will be handled.
>>
>> Given this, I think JAWS is not the right answer.   Perhaps NVDA or
>> SA to go or some other screen reader I'm not aware of could step in?
>>
>> Jim
>>
>> On Sat, Feb 18, 2012 at 07:21:31PM -0800, Nicole B. Torcolini at Home
> wrote:
>> > When doing some research for a project, I found the following article.
>> What do people think?
>> >
>>
> http://clearhelper.wordpress.com/2010/03/16/stop-using-jaws-for-web-accessib
>> ility-testing/
>> > _______________________________________________
>> > nfbcs mailing list
>> > nfbcs at nfbnet.org
>> > http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
>> > To unsubscribe, change your list options or get your account info
>> > for
>> nfbcs:
>> > http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/jbar%40barcore.co
>> > m
>> >
>>
>> _______________________________________________
>> nfbcs mailing list
>> nfbcs at nfbnet.org
>> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
>> To unsubscribe, change your list options or get your account info for
> nfbcs:
>> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/k7uij%40panix.com
>>
>>
>> _______________________________________________
>> nfbcs mailing list
>> nfbcs at nfbnet.org
>> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
>> To unsubscribe, change your list options or get your account info for
> nfbcs:
>> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/jbar%40barcore.com
>>
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/k7uij%40panix.com
>
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/majolls%40cox.net
>
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/dgl%40dlee.org
>
> --
> Doug Lee                 dgl at dlee.org                http://www.dlee.org
> SSB BART Group           doug.lee at ssbbartgroup.com
> http://www.ssbbartgroup.com
> "The most exhausting thing in life is being insincere."
>         - Anne Morrow Lindbergh {American Author}
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/k7uij%40panix.com
>
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/majolls%40cox.net
>
>
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/davidct1209%40gmail.com
>

More information about the NFBCS mailing list