[Blindmath] How useful is a GUI to blind users?

Tue Jan 10 15:38:30 UTC 2012

OK, I'm convinced. I will suffer through the pains of programming with the
SWT for any software that I publish (unless I can find a better alternative
that is also accessible), even if it does preclude the use of existing
libraries and therefore extend the development time.

Thank you everyone for your response.

Dick Baldwin

On Tue, Jan 10, 2012 at 8:37 AM, David Tseng <davidct1209 at gmail.com> wrote:

> In short, yes GUI's are useful for the blind. It is not so much that
> things are layed out visually as it is that GUI's have effectively
> abstracted away much of the complexities of software. To draw a
> mathematical analogy, it is like working with the Cartesian plane
> which serves as an abstraction for specific classes of algebraic
> expressions. With such a view, one can more easily "see" the overall
> picture (for example, relationships between regions in space, notions
> of distance, etc). Instead of working with raw information, one can
> focus on specific tasks.
>
> Back to software -- the way things are "layed" out serve as a spacial
> map of an application. Blind people also experience space so can put
> to use concepts like direction, grouping, organization, etc. Put in
> terms of an application, a desktop metaphor, for example, is just
> another place to put the stuff you're working on like on your real
> desk where you might place papers, braillers, books, CD's, etc. The
> only difference is to a blind person, the way to move around this
> virtual desktop is through a keyboard (admitedly, it's slightly less
> intuitive, but it's more so than a command line loop/prompt).
>
> As a matter of practicality as others have pointed out, for purposes
> of work or school, a GUI is either recommended or required. Working
> with others using the same interface, speaking the same language,
> exchanging the same files, pairing on the same machine is vital for
> group rpojects as a student or a shared project in a profressional
> setting.
>
> As for technical details, almost all platforms have iteratively honed
> in on the server-client pattern to solve accessibility. Specifically,
> the server (your application) interfaces with the client (the
> assistive tech) mediated by the OS through well-defined API's. In this
> way, the application can expose the raw data structures that
> encapsulate all of the information about the on screen UI. The client
> takes that raw representation and converts it into something more
> useful for the target disability or use case. That can be, for
> example, speech recognition where a spoken phrase gets translated into
> a search string which gets matched against the accessible text of a
> node in the accessibility tree to find a target node. A screen reader
> would potentially allow a user to explore the relationships between
> the structures here or just flatten the traversal to the leaf nodes.
>
>
> Graphical toolkits obscure the OS level API's, and give developers
> less control, so using a highly abstracted framework usually doesn't
> work as well if you're doing something custom.
>
> Personally, I have found that the advantages have far outweighed the
> drawbacks. Having a well-maintained, stable, multi-language, modern OS
> with well-designed, tested applications and well scripted assistive
> technology produces a productive experience. The onus of accessibility
> falls ultimately to the assistive technology which makes sense since
> the whole point of applications is to serve the diverse needs of
> people. The application itself has a restricted but also well defined
> set of responsibilities which goes a long way in people actually
> fixing their deficiencies with regard to access.
>
> On 1/8/12, Richard Baldwin <baldwin at dickbaldwin.com> wrote:
> > You wrote: " Is it possible to use the SUN image API for the graphic
> > presentation portions of the program and use SWT for the user controls? "
> >
> > Just about anything is possible when enough programming effort is
> applied.
> > What you suggest is possible but difficult to achieve in a reliable way,
> > and quite a lot of extra programming effort is required to achieve it.
> > However, that is not really the issue. The issue is that the use of the
> SWT
> > precludes the use of many existing Java class libraries.
> >
> > The great promise and benefit of object-oriented programming is "reuse,
> > don't reinvent." The great power of the Java programming environment is
> the
> > existence of massive class libraries that can be re-used to program with
> > relative ease those things that would otherwise be very difficult.
> >
> > The bottom line is that many existing Java class libraries cannot be
> > "re-used" with the SWT without a major programming effort to modify those
> > libraries. Even with a major programming effort, library modification is
> > possible only when the author of the library is willing to divulge the
> > source code for the library. Sometimes source code is available and
> > sometimes it is not available.
> >
> > IBM's approach with the SWT seems to have been "reinvent, don't reuse."
> > They should not claim that their programming environment is Java simply
> > because the compiled output will execute in a Java Virtual Machine. (The
> > compiled output of JYthon and several other programming languages will
> also
> > execute in a Java Virtual Machine but they are not Java.) IBM should have
> > renamed their programming environment JBM or something like that to make
> it
> > clear that SWT was never intended to be a member of the "write once, run
> > anywhere" family of Java products and was never intended to be compatible
> > with standard Java class libraries.
> >
> > Microsoft did something similar a few years back, but after several years
> > of floundering around with "Microsoft Java," they finally renamed their
> > programming environment C# to distinguish it from Java. And, by going
> down
> > an independent path from Java, developing their own class libraries, and
> > not claiming any degree of compatibility with Java, C# has become a very
> > respectable programming environment in its own right. IBM should do the
> > same.
> >
> > Welcome to the world of JBM.
> >
> > Thanks for the response,
> > Dick Baldwin
> >
> > On Sun, Jan 8, 2012 at 5:55 AM, Larry Skutchan <lskutchan at aph.org>
> wrote:
> >
> >> Is it possible to use the SUN image API for the graphic presentation
> >> portions of the program and use SWT for the user controls? It doesn't
> seem
> >> like either way provides any accessibility advantages to the graphical
> >> drawing, manipulation, or interpretation portions of the interface, yet
> >> SWT
> >> provides accessibility to those controls that do offer clear advantages
> >> for
> >> both blind and sighted users.
> >>
> >>
> >> On Jan 7, 2012, at 10:16 PM, Richard Baldwin wrote:
> >>
> >> > You wrote: " I thought there was generally a well-established
> principle
> >> in
> >> > software development that it is best to implement a set of
> functionality
> >> in
> >> > a way that can be connected to various presentations of user
> interface"
> >> >
> >> > You are probably speaking of something called the model-view-control
> >> > pattern, which is great in theory, but rarely exists to the full
> extent
> >> > possible in practice.
> >> >
> >> > Getting down in the weeds, here is an example of marrying Sun Java
> >> graphics
> >> > with SWT Java graphics. Standard Sun Java provides the capability to
> use
> >> > the Sun Graphics2D API to create graphics on an off-screen Image
> object.
> >> > Sun Graphics2D is a fairly extensive and powerful API.
> >> >
> >> > The SWT makes it possible to use SWT graphics (which are entirely
> >> different
> >> > from the Sun Graphics2D API) to create off-screen graphics on an Image
> >> > object. In my opinion, SWT graphics are less extensive and less
> powerful
> >> > than the Sun Graphics2D API, but that is just an opinion that others
> may
> >> > disagree with. There is no disagreement, however, that the two are
> >> > different and are incompatible.
> >> >
> >> > There is also no disagreement that an SWT Image object is different
> from
> >> a
> >> > Sun Image object, and the two are incompatible. Rather than building
> on
> >> the
> >> > Sun Image class, IBM elected to invent their own Image class in the
> SWT.
> >> > This was a decision that apparently had nothing to do with the use of
> >> > the
> >> > native GUI components because it is not a GUI issue. Thus,
> >> > incompatibilities between Sun Java and SWT Java go far deeper than
> >> > simply
> >> > the use of native GUI components or lightweight GUI components. They
> go
> >> > very deep into the guts of the programming environment.
> >> >
> >> > When graphics are developed using Suns Graphics2D API, which is the
> case
> >> > with the majority of existing Java graphics libraries, the display of
> >> that
> >> > graphic information on an SWT Composite object, requires that first,
> it
> >> is
> >> > necessary to gain access to the Sun Image object (if possible), then
> it
> >> is
> >> > necessary to reformat and transport the graphics information from the
> >> > Sun
> >> > Image object to an SWT Image object, and then it is necessary to
> display
> >> > the graphics information on an SWT Composite object.
> >> >
> >> > To begin with, the graphics data on aSun Image object may, or may not
> be
> >> > accessible to the programmer, depending on the design of the graphics
> >> > library. Even if the graphics data on the Sun Image object is
> accessible
> >> > (with or without modifying the library), the task of reformatting and
> >> > transporting that data to an SWT Image object, solely for purposes of
> >> > display, is tedious, time consuming, and can result in a lack of
> >> > responsiveness for graphics-intensive programs.
> >> >
> >> > Of course, it is always possible to modify a library if source code
> for
> >> the
> >> > library is available, but to a large extent, that defeats the purpose
> of
> >> > OOP and class libraries. It can also be very difficult in some cases.
> >> >
> >> > Dick Baldwin
> >> >
> >> > On Sat, Jan 7, 2012 at 7:57 PM, Jamal Mazrui <empower at smart.net>
> wrote:
> >> >
> >> >> Some miscellaneous points are as follows.
> >> >>
> >> >> There were text mode, screen oriented user interfaces before
> graphical
> >> >> user interfaces, e.g., WordPerfect 5.1 was a highly popular DOS-based
> >> word
> >> >> processor among blind people.  In fact, I think we were, overall,
> more
> >> >> productive with it than any GUI word processor, like MS Word,  or
> >> >> command-oriented word processor, like VIM.
> >> >>
> >> >> Those text mode, screen oriented interfaces could be highly
> accessible
> >> >> with a screen reader because it knew precisely what character (and
> >> color)
> >> >> was located in every position of a grid with 25 rows and 80 columns.
> >>  The
> >> >> point of focus was also precisely known -- what later became the
> caret
> >> in a
> >> >> GUI.  Accessibility of GUIs has been a problem in large part because
> a
> >> >> screen reader has had to build an "off screen model" of what text is
> on
> >> the
> >> >> screen, since by the time text is displayed on the screen, it is
> pixels
> >> not
> >> >> characters.  A GUI screen reader has traditionally had to intercept
> >> every
> >> >> operating system call that writes to the screen, remember where the
> >> writing
> >> >> occurred, and adjust the model every time some other OS call implies
> a
> >> >> change in the visual presentation.
> >> >>
> >> >> This process has been error prone, thus necessitating the need for an
> >> >> accessibility API by which the screen reader can interrogate the
> screen
> >> at
> >> >> any time.  Unfortunately, comprehensively implementing that
> >> accessibility
> >> >> API, on both the application and screen reader side, has also been
> >> >> error
> >> >> prone.  There are other reasons as well why GUIs have been difficult,
> >> >> including the emphasis on a mouse for input rather than the keyboard
> --
> >> the
> >> >> former usually requiring fine eye-hand coordination to be practical.
> >> >>
> >> >> Blind people, like sighted people, have a range of cognitive
> abilities.
> >> >> Some find the memorization of commands and syntax, which is generally
> >> >> required by a command-line interface, to be challenging and
> >> frustrating, as
> >> >> compared to a GUI and screen reader combination implemented
> accessibly,
> >> >> which support much more choice selection and safe exploration of
> >> >> options
> >> >> before executing a task.  Often, the most productive application will
> >> >> be
> >> >> one that offers both an accessible GUI for typical use and a
> >> well-designed
> >> >> command line interface for advanced use.
> >> >>
> >> >> I am puzzled why Swing seems to be so integrated into popular Java
> >> >> libraries.  Do they not separate visual display from underlying
> >> >> functionality?  I thought there was generally a well-established
> >> principle
> >> >> in software development that it is best to implement a set of
> >> functionality
> >> >> in a way that can be connected to various presentations of user
> >> interface
> >> >> -- unless, of course, the library is specifically targeted to
> building
> >> user
> >> >> interfaces.
> >> >>
> >> >> Regarding SWT, it is cross platform, not Windows only, though
> different
> >> >> binary files have to be distributed with other platforms.  I am a fan
> >> >> of
> >> >> wxWidgets, but agree that SWT is clearly more established in the Java
> >> >> community, and do not see any advantage that wx-based GUIs would
> offer
> >> over
> >> >> SWT.  Both GUI libraries are essentially wrappers over native
> controls
> >> of
> >> >> the platform.  Such controls tend to be the most accessible because
> >> >> they
> >> >> implement the primary accessibility API of the platform, being build
> by
> >> the
> >> >> developer of both the platform and accessibility API.
> >> >>
> >> >> Jamal
> >> >>
> >> >>
> >> >>
> >> >> ______________________________**_________________
> >> >> Blindmath mailing list
> >> >> Blindmath at nfbnet.org
> >> >> http://nfbnet.org/mailman/**listinfo/blindmath_nfbnet.org<
> >> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org>
> >> >> To unsubscribe, change your list options or get your account info for
> >> >> Blindmath:
> >> >> http://nfbnet.org/mailman/**options/blindmath_nfbnet.org/**
> >> >> baldwin%40dickbaldwin.com<
> >>
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/baldwin%40dickbaldwin.com
> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Richard G. Baldwin (Dick Baldwin)
> >> > Home of Baldwin's on-line Java Tutorials
> >> > http://www.DickBaldwin.com
> >> >
> >> > Professor of Computer Information Technology
> >> > Austin Community College
> >> > (512) 223-4758
> >> > mailto:Baldwin at DickBaldwin.com
> >> > http://www.austincc.edu/baldwin/
> >> > _______________________________________________
> >> > Blindmath mailing list
> >> > Blindmath at nfbnet.org
> >> > http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> >> > To unsubscribe, change your list options or get your account info for
> >> Blindmath:
> >> >
> >>
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/lskutchan%40aph.org
> >>
> >>
> >> _______________________________________________
> >> Blindmath mailing list
> >> Blindmath at nfbnet.org
> >> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> >> To unsubscribe, change your list options or get your account info for
> >> Blindmath:
> >>
> >>
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/baldwin%40dickbaldwin.com
> >>
> >
> >
> >
> > --
> > Richard G. Baldwin (Dick Baldwin)
> > Home of Baldwin's on-line Java Tutorials
> > http://www.DickBaldwin.com
> >
> > Professor of Computer Information Technology
> > Austin Community College
> > (512) 223-4758
> > mailto:Baldwin at DickBaldwin.com
> > http://www.austincc.edu/baldwin/
> > _______________________________________________
> > Blindmath mailing list
> > Blindmath at nfbnet.org
> > http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> > To unsubscribe, change your list options or get your account info for
> > Blindmath:
> >
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/davidct1209%40gmail.com
> >
>
> _______________________________________________
> Blindmath mailing list
> Blindmath at nfbnet.org
> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> To unsubscribe, change your list options or get your account info for
> Blindmath:
>
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/baldwin%40dickbaldwin.com
>

-- 
Richard G. Baldwin (Dick Baldwin)
Home of Baldwin's on-line Java Tutorials
http://www.DickBaldwin.com

Professor of Computer Information Technology
Austin Community College
(512) 223-4758
mailto:Baldwin at DickBaldwin.com
http://www.austincc.edu/baldwin/