[nfbcs] Should JAWS be used for web accessibility testing (was Re: Opinions?)

Sun Feb 19 21:51:45 UTC 2012

Jim:

Thank you for your articulate response. Here's my answer.

With regard to my setting the bar too high for testing, you bet your sweet
posterior I am! In fact, I think that we as a society in general and
companies in particular have become *way* too sloppy both in coding
standards and in the level of perfection we expect from the software and
hardware we use. There are computers using older operating systems where I
work that have been up -- continuously -- for nearly two years. Put *that*
in Windows' pipe and smoke it! And way too many bugs slip through in vendor
software (both Microsoft and otherwise) these days because the testing
wasn't sufficiently vigorous to catch them. And yes, I realize that were the
sort of testing I advocate done, it might slow down web site and software
changes to perhaps two a year! Bravo! That's *exactly* what I think should
happen. If corporations and individuals knew they'd only have two chances a
year to make changes, they might concentrate a bit more on getting things
right the *first* time rather than using the users as beta-testers!

I know you'll object that content changes more often than this on web sites.
You're right. But I see little reason for this -- especially as a good bit
of the change is pure glitz designed to try to attract new viewers etc. Why
not assume people to be literate and intelligent and try to attract them
with well-written *text* content?

But I realize I'm jousting at windmills on this one. Given this, then, what
should we do? I think your suggestion of picking a standard screen-reader
that doesn't depend upon direct sales is impractical at best. I maintain
that eventually *all* screen-readers will have to be maintained based upon
sales; even the purveyors of NVDA are having to admit that they might be
hungry and would like some contributions. If we admit that there's no such
thing as a free lunch and if we admit that changing screen-readers involves
more than a sighted person goes through in changing browsers because his web
experience isn't as good with one as with another, the only alternative I
then see short of my utopian test suite and regime would be to specify
allowed and disallowed web constructs and software constructs and, say,
limiting them to, for example, what one could do with standard windows
controls as implemented in Windows-98. In other words, as I see it, we
either allow innovation and insist upon a rigorous testing regimen or we
specify that no innovation beyond a certain level is allowed with the
penalty for violation being the immediate job termination of a company's CEO
and executive board.

And we know that isn't going to happen either and probably it shouldn't
(although I confess that a good bit of nonsense in today's society would be
solved if people knew their jobs were on the line for foisting it on the
rest of us!).

Mike

-----Original Message-----
From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
Of Jim Barbour
Sent: Sunday, February 19, 2012 12:16 PM
To: NFB in Computer Science Mailing List
Subject: Re: [nfbcs] Should JAWS be used for web accessibility testing (was
Re: Opinions?)

Oh, this is very interesting.  Thank you Mike for articulating the
other side of this.

I'd like to quote and then respond two items from your message.

First Quote:
"I think the only way to do this right would be to specify that *every* site
should be put through a suite of tests by *human* *beings,* not automated
tools, using the following screen-readers at a minimum: JAWS, Window-eyes,
Hal, SuperNova, System Access, NVDA, Coco (sp) and VoiceOver (both on
i-devices and on the Mac). It's a matter for debate whether or not one
should specify note-takers such as the BrailleSense and BrailleNote family
also to be tested."

Ignoring for the moment the word "every" in your statement above,
let's think about large companies that want to test accessibility,
such as Google, Yahoo, and Facebook.

You're proposing that these companies stage human run usability tests
on *each* new or changed product using 8 to 10 different AT solutions.

As a comparison, none of the companies I mention above would
consider doing this amount of mainstream usability testing against 
different web browsers.  I believe that this is way to high a bar for
us to expect any web site developer to meet

Instead, we should pick a model screen reader, and begin to insist
that it be used in *all* AT usability testing.  That screen reader
could then be given the responsibility of reporting out problems they
encounter for which there is no documented solution.  This puts that
company in the drivers seat for finding a solution.

For this to work, we need a screen reader whose development is not
financed by direct sales.

As to the word "every", I don't think you really mean every web site
developer.  If I develop a photo sharing site for my family to use,
say www.barboursphotos.com, this probably should not have to pass the
accessibility test.

The question about who should be obliged to pass any accessibility
tests is a sticky one.

And, your second quote

"In fact, I think the article's author is desperately trying to find a way
to
lessen work for himself or, put another way, he is hoping he can be lazy and
not do the sort of in-depth testing that is truly required for good
accessibility testing."

So, of course he is.  He's trying to minimize the amount of effort
needed to make his web sites accessible.  This is reasonable and makes
perfect sense to me.  Why wouldn't you expect any developer of any
product to increase their efficiency?

Jim

On Sat, Feb 18, 2012 at 08:49:22PM -0800, Mike Freeman wrote:
> Jim:
> 
> I respectfully, but strongly, disagree. Although I argue in another
message
> that there's no good way to include or exclude a particular screen-reader
> from accessibility or usability tests, I also think that excluding a
> particular screen-reader amounts to a value judgment even if it is not
> intended as such. Consider how irked Window-eyes users get when everyone
> tests their sites against JAWS. Why should JAWS users put up with the same
> sort of nonsense?
> 
> In fact, I think the article's author is desperately trying to find a way
to
> lessen work for himself or, put another way, he is hoping he can be lazy
and
> not do the sort of in-depth testing that is truly required for good
> accessibility testing.
> 
> I think the only way to do this right would be to specify that *every*
site
> should be put through a suite of tests by *human* *beings,* not automated
> tools, using the following screen-readers at a minimum: JAWS, Window-eyes,
> Hal, SuperNova, System Access, NVDA, Coco (sp) and VoiceOver (both on
> i-devices and on the Mac). It's a matter for debate whether or not one
> should specify note-takers such as the BrailleSense and BrailleNote family
> also to be tested.
> 
> The only alternative I can see would be to try to get all screen-readers
to
> behave the same way and, my friends, that ain't a-gonna happen! (grin)
> 
> Mike Freeman
> 
> 
> -----Original Message-----
> From: nfbcs-bounces at nfbnet.org [mailto:nfbcs-bounces at nfbnet.org] On Behalf
> Of Jim Barbour
> Sent: Saturday, February 18, 2012 8:21 PM
> To: NFB in Computer Science Mailing List
> Cc: NABS-L
> Subject: [nfbcs] Should JAWS be used for web accessibility testing (was
Re:
> Opinions?)
> 
> I am in 100% agreement with the statement that JAWS should not be used
> for web site testing.  However, my reasons differ from the ones
> written in the article.
> 
> It is not possible today to design and build accessible websites
> without performing usability tests.  Further, there are too many
> access technologies to test with them all.  So, the question is which
> AT should be used to test, and therefore drive improvements to, web
> site accessibility?  Whichever one gets chosen will have the
> opportunity to informally set standards around how certain types of
> content will be handled.
> 
> Given this, I think JAWS is not the right answer.   Perhaps NVDA or
> SA to go or some other screen reader I'm not aware of could step in?
> 
> Jim
> 
> On Sat, Feb 18, 2012 at 07:21:31PM -0800, Nicole B. Torcolini at Home
wrote:
> > When doing some research for a project, I found the following article.
> What do people think?
> >
>
http://clearhelper.wordpress.com/2010/03/16/stop-using-jaws-for-web-accessib
> ility-testing/
> > _______________________________________________
> > nfbcs mailing list
> > nfbcs at nfbnet.org
> > http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> > To unsubscribe, change your list options or get your account info for
> nfbcs:
> > http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/jbar%40barcore.com
> > 
> 
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for
nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/k7uij%40panix.com
> 
> 
> _______________________________________________
> nfbcs mailing list
> nfbcs at nfbnet.org
> http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
> To unsubscribe, change your list options or get your account info for
nfbcs:
> http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/jbar%40barcore.com
> 

_______________________________________________
nfbcs mailing list
nfbcs at nfbnet.org
http://nfbnet.org/mailman/listinfo/nfbcs_nfbnet.org
To unsubscribe, change your list options or get your account info for nfbcs:
http://nfbnet.org/mailman/options/nfbcs_nfbnet.org/k7uij%40panix.com