[Blindmath] Issues with electronic math files and screen readers/braille displays

Sat May 12 17:24:48 UTC 2012

Sina,

All the numbers I mentioned are one dimension (button width and pixel 
width), mostly because it's easier to discuss. The buttons on the iPhone 
are rectangular, but fairly close to square.

I have not conducted a Fitts's Law study in two dimensions for auditory 
display, but I have seen people plot points in two dimensions with 
similar numbers. For graphs, if a person is moving along one graph axis 
then the other, I'd expect the accuracy to be the square of the accuracy 
of a 1D target, since there are two 1D targets. In classroom studies 
this spring, students could reliably read and write 9px x 9px targets 
with a mouse or keyboard with sonification (a later iteration of what 
you saw in August 2011). I work with regular laptops and desktops, so I 
can't speak to touchscreen needs. Again, though, I'd imagine that the 
mouse or keyboard is more precise than a finger, and so the targeting 
region can be smaller (1D or 2D).

I think maps are very different than coordinate graphs in the way we use 
two dimensions. When you want to plot a point on a graph, for example 
(2,-3), the steps are to first find the origin, then move along x to -3, 
then move up on y to 2, then mark a dot. Therefore, the number line 
model works well: it's a combination of x then y movement. I haven't 
found someone who goes diagonally from the origin! However, it's more 
natural to navigate at angles for a map, with unique country shapes and 
relationships.

Also, anecdotally, scanning a tactile graphic (or auditory graph) is 
different than scanning a visual graph for outliers (extreme values). 
They "pop out" visually. For a tactile graphic, if you don't touch the 
right spot, on a touch screen or tactile graphic, you won't know the 
outlier is there. "Flattening" the map or graph to one dimension is a 
useful way to provide a way to quickly scan horizontally (or 
vertically). Imagine a horizontal bar at the bottom of a map, telling 
you where each country is ("England", "Spain", "France", "Germany"), 
then all you have to do is move up, find the target ("Italy", 
"Switzerland", "Germany"), and move slightly to explore nearby 
understand the 2D spatial relationship. With low familiarity with the 
map, I'd guess that the horizontal-vertical approach is faster than 2D 
freeform navigation. You might, for example, miss "England" or 
"Switzerland" if you followed the trace around Europe.

There is published research in (sighted) targeting in two dimensions, 
such as MacKenzie and Buxton 1992 and more recently Wobbrock et al. 
2011, both at CHI.

I have a video demonstration about 2D graph navigation using 
sonification at bit.ly/mathgnie , under the "GNIE Software" link. It 
shows this x-then-y approach.

- Ben

On 5/12/2012 10:36 AM, Sina Bahram wrote:
> are those numbers for free form 2-d navigation or only for 1-d target selection?
>
>
> Website: www.SinaBahram.com
> Twitter: @SinaBahram
>
> -----Original Message-----
> From: blindmath-bounces at nfbnet.org [mailto:blindmath-bounces at nfbnet.org] On Behalf Of Benjamin Davison
> Sent: Saturday, May 12, 2012 8:40 AM
> To: Blind Math list for those interested in mathematics
> Subject: Re: [Blindmath] Issues with electronic math files and screen readers/braille displays
>
>   From what I understand, in iOS, the keyboard software predicts the next
> keys you will use and makes the selection area larger (even though the
> button appears to be the same size). This will definitely improve the
> accuracy. Being able to back up like you mentioned is an important
> feature, too.
>
> Maybe you're right about the 1cm estimate being too large. In my
> research, blind and sighted users can be quite accurate with a mouse and
> auditory feedback for even a 5 pixel target. On the iTouch, this would
> be about 0.32mm. I imagine a "fat finger" needs more space than that,
> but I'm not sure how much. Based on iPod touch specifications, the
> narrow screen width is about 4.99cm and the most number of keys on a
> line is 10, so the width of a button is about 5mm, or half of what I
> predicted (each button looks to be smaller, with its edges, but I don't
> think there is an actual non-selection space between buttons).
>
> Maps and graphs are different than a regular key layout, in that content
> can be overlapping and connect in many ways. I'm guessing this would
> mean a larger square-space for navigable content.
>
> - Ben
>
>
> On 5/11/2012 6:20 PM, Michael Whapples wrote:
>> Thanks for the answer. As a point of interest on the screen size thing
>> and accuracy, on a good day I can type on an iPodTouch screen in
>> portrait without hitting a wrong key straight off, but on an average
>> day I may get the key next to it on the initial touch and this
>> normally is an offset issue (IE. I tend to get the key only on one
>> side, I think normally the one to the left of where I aimed). I am not
>> sure on the exact size of the keys I have been mentioning, but I think
>> less than that one cm estimation.
>>
>> Also on the accuracy thing, with IOS and voiceover the ability to just
>> slide my finger to correct a miss is useful.
>>
>> Michael Whapples
>> On 11/05/2012 12:41, Benjamin Davison wrote:
>>> Michael,
>>>
>>> Screen sizes are tricky. It turns out that you are faster to find a
>>> target with finger movements, over hand movements, over arm
>>> movements, so a small screen makes sense for efficient activity.
>>> However, research in tactile graphics has shown that there needs to
>>> be certain spacings between elements (lines, text, etc) for the graph
>>> to be usable. Whether this applies to software on touchscreens with
>>> box-regions of "selection squares" is an open research question, but
>>> I would expect there would be some minimum threshold (probably above
>>> 1cm). Below this threshold, there will be too many perceptual and
>>> motor errors.
>>>
>>> Having two hands is a common exploration technique for tactile
>>> graphics, and useful for finding outliers and understanding the
>>> graph. Technologies for blind users should support this. Multitouch
>>> devices will detect multiple fingers. However, determining the "lead
>>> finger", where the user is paying attention, is an important part of
>>> the user experience. From what I understand, for example, when
>>> touch-reading braille with the right hand, the ring and middle
>>> fingers act as an alignment guide for the text, and many people pay
>>> attention and perceive the braille through the index (pointer)
>>> finger. An application only displaying (through braille or speech)
>>> what is at the ring finger would be misaligned with how braille is
>>> read on paper. Two hands makes this detection more complicated for
>>> the programmer, and so developers must take the time to consult
>>> empirical studies and actual users.
>>>
>>> - Ben
>>>
>>>
>>> On 5/11/2012 5:42 AM, Michael Whapples wrote:
>>>> Yes I mainly was objecting to the platform, however until one has it
>>>> on a platform they will use then the tool remains as good as
>>>> non-existent. Many of the mobile screen readers have support for
>>>> bluetooth Braille displays, but as I have a perfectly functioning
>>>> USB Braille display I cannot justify (cost, far too much) changing
>>>> it, so the Braille support in mobile screen readers adds no value in
>>>> my mind and the developers may as well have spent time on something
>>>> more useful. Of course, there may be some who it will be useful to
>>>> and it comes down to numbers as to whether the developer feels its
>>>> going to add value.
>>>>
>>>> Moving from that to comment on the actual ideas of the tool, based
>>>> purely on reading.
>>>>
>>>> Quite a lot of it seems to make sense. I personally think having the
>>>> touch part is of value, I really find I need to maintain the
>>>> physical/spatial component when using diagrams to understand how
>>>> things relate. A couple of comments/questions. One thing which
>>>> caught me out when first trying IVEO from viewplus was that I tend
>>>> to explore tactile diagrams with two hands. While this is personal
>>>> feeling rather than experimental findings, I think the reason is
>>>> that I may use one hand as a marker and the other to find something
>>>> else and then I know where the two points are relative to each
>>>> other. The problem was that two hands confused IVEO and it meant the
>>>> spoken output was not related to what I thought I was tapping
>>>> (sometimes it thought the other hand was pressing/tapping). Any way
>>>> to allow this sort of use?
>>>>
>>>> Another touchscreen question is, would you advise a certain size
>>>> screen? What I mean by this is that I actually think sometimes a
>>>> small screen is possible to be more accurate with as I can have a
>>>> finger or thumb on the edge of the device and reach over with
>>>> another finger, this then means I can feel how far/where I am
>>>> reaching because of relative finger position/muscle tension in the
>>>> hand. On larger screens one may not be able to do this and may need
>>>> to move the hand around and so loose the relative positioning
>>>> feeling. Obviously a larger screen means things can be larger to
>>>> find, I wonder if there is an optimal size.
>>>>
>>>> As an example to the above, when typing on an iPodTouch, I find I
>>>> can be quicker and more accurate using it in portrait mode rather
>>>> than landscape, I just feel landscape is too spread out.
>>>>
>>>> A different comment, I would probably have said speech recognition
>>>> is not adding value, but as you have also included natural language
>>>> recognition it possibly does. If you had not included natural
>>>> language recognition I just feel the speech commands would have been
>>>> yet another command set to remember and so does it solve something
>>>> which is a problem with the other interaction modes. Not needing to
>>>> use a specific command set means one hasn't got another command set
>>>> to remember and so can take a good guess if you cannot remember the
>>>> command in other modes.
>>>>
>>>> Hope some of this is useful.
>>>>
>>>> Michael Whapples
>>>> On 11/05/2012 02:03, Sina Bahram wrote:
>>>>> While I hear you: all you've stated an objection to is the
>>>>> platform. As you know, this is nothing more than an engineering
>>>>> effort to move to other platforms, and I agree with you in terms of
>>>>> availability and so forth. On the other hand, the research, the
>>>>> interaction techniques, the feature sets: these are platform
>>>>> agnostic. Furthermore, the development experience on Android is far
>>>>> superior to that of IOS, whereas ironically the user experience is
>>>>> far superior on IOS than it is on Android; thus, this paradoxical
>>>>> situation leads to the current implementation. Please understand
>>>>> that this is the most minor of concerns, as this not a product yet,
>>>>> but a way of showing folks that such things are possible. If we
>>>>> turn this into a product or release it as such, then of course such
>>>>> considerations are on the top of the list before anything else,
>>>>> frankly.
>>>>>
>>>>> A friend once told me that if all people criticize are the easiest
>>>>> to change of engineering efforts/details, then you're doing OK,
>>>>> *smile*, unfortunately, I'm not ready to presume that yet, but I
>>>>> might presume that you don't have any comments on the
>>>>> substance/meat of the project? if you do, please know I'm anxious
>>>>> to hear them, as I want to incorporate feedback into this.
>>>>>
>>>>> Thanks so much for your feedback and for taking the time to write
>>>>> something: so many people don't.
>>>>>
>>>>> Take care,
>>>>> Sina
>>>>>
>>>>> Website: www.SinaBahram.com
>>>>> Twitter: @SinaBahram
>>>>>
>>>>> -----Original Message-----
>>>>> From: blindmath-bounces at nfbnet.org
>>>>> [mailto:blindmath-bounces at nfbnet.org] On Behalf Of Michael Whapples
>>>>> Sent: Thursday, May 10, 2012 8:21 PM
>>>>> To: Blind Math list for those interested in mathematics
>>>>> Subject: Re: [Blindmath] Issues with electronic math files and
>>>>> screen readers/braille displays
>>>>>
>>>>> That's all fine if one has/wants an Android tablet.
>>>>>
>>>>> That comment is not to do down your work, more a call to say
>>>>> "please put
>>>>> it in a more widely useful form". Personally I have absolutely no
>>>>> interest or intent in getting an Android tablet, probably as much
>>>>> political (I dislike the Google involvement) as much as practical
>>>>> (partially cost to use ratio, and also I have already some Apple
>>>>> products so other Apple products would probably fit better for sharing
>>>>> data/information). My view is that computers are more widely useful,
>>>>> many are likely to already have one/have access to one, there is a
>>>>> dominant OS (majority are windows) and there are tools which make it
>>>>> easy to make crossplatform applications (eg. Java), etc.
>>>>>
>>>>> Michael Whapples
>>>>> On 11/05/2012 00:52, Sina Bahram wrote:
>>>>>> We're working on some solutions for Maps and also diagrams such as
>>>>>> FlowCharts in my lab. An effort, for which I was quite humbled and
>>>>>> honored, to recently receive a Whitehouse Champions of Change award.
>>>>>>
>>>>>> Currently, our system allows a blind student to interact with
>>>>>> Google maps via touch, voice, and keyboard. It runs on any Android
>>>>>> tablet, or really any decently recent Android powered device, and
>>>>>> facilitates this access to the map by utilizing the TIKISI
>>>>>> framework which I've developed as part of my doctoral studies.
>>>>>>
>>>>>> If you promise not to judge draft level copy that is very much not
>>>>>> complete, you can read more about these efforts at:
>>>>>>
>>>>>> www.AccessibleInfographics.com
>>>>>>
>>>>>> we most recently made significant progress on FlowCharts, so I'll
>>>>>> be updating the FlowCharts section soon with a write-up of our
>>>>>> progress. Videos are forthcoming.
>>>>>>
>>>>>> Hope this helps stimulate some discussion. Feel free to contact me
>>>>>> off list with questions.
>>>>>>
>>>>>>
>>>>>> Take care,
>>>>>> Sina
>>>>>>
>>>>>> Website: www.SinaBahram.com
>>>>>> Twitter: @SinaBahram
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: blindmath-bounces at nfbnet.org
>>>>>> [mailto:blindmath-bounces at nfbnet.org] On Behalf Of Michael Whapples
>>>>>> Sent: Thursday, May 10, 2012 5:59 PM
>>>>>> To: Blind Math list for those interested in mathematics
>>>>>> Subject: Re: [Blindmath] Issues with electronic math files and
>>>>>> screen readers/braille displays
>>>>>>
>>>>>> You mentioned diagrams and maps, I am not sure if there is a purely
>>>>>> computer based solution which is satisfactory for those. I know
>>>>>> some use
>>>>>> the software called the vOICe (www.seeingwithsound.com), but I think
>>>>>> even users of that admit it has its limitations for what one can
>>>>>> understand in a diagram when using it. Also while I have looked at
>>>>>> it, I
>>>>>> have never really got to grips with it, I am not putting it down by
>>>>>> saying that, many find it useful and I think it possibly has value, I
>>>>>> rather mean I personally have never managed to get on with it.
>>>>>>
>>>>>> I think certainly for the diagrams there may need to be a physical
>>>>>> diagram for blind students.
>>>>>>
>>>>>> As for the maths, well there are bits and pieces out there which
>>>>>> can do
>>>>>> parts of it, but nothing really linked up. As Steve mentioned, may
>>>>>> be if
>>>>>> a state government pushes screen reader manufacturers then may be
>>>>>> they
>>>>>> will feel there is a need and so will start working on the problem.
>>>>>> Until the screen reader providers start working on it those who do
>>>>>> try
>>>>>> and make maths accessible will be working in quite a constrained
>>>>>> environment.
>>>>>>
>>>>>> Michael Whapples
>>>>>> On 10/05/2012 14:40, Patricia Balassone wrote:
>>>>>>> Thank you all, your feedback contains the kind of information I
>>>>>>> need to pass along. Although I only know a little about Nemeth
>>>>>>> and even less about programming, I know enough that I am also
>>>>>>> skeptical of the possibility for adequate adaptations.
>>>>>>> Unfortunately, providing a brailled hard copy with all possible
>>>>>>> answer scenarios for an exam that is answer driven is
>>>>>>> unrealistic. One of my suggestions, even though it will still be
>>>>>>> cumbersome, will be to supply tactile copies of all maps,
>>>>>>> diagrams, and equations for braille readers. Again, thank you for
>>>>>>> allowing me to tap into your knowledge and experience.
>>>>>>> Patricia Balassone
>>>>>>> CONFIDENTIALITY NOTICE: This email and any files transmitted with
>>>>>>> it are confidential and are intended solely for the use of the
>>>>>>> individual or entity to which they are addressed. This
>>>>>>> communication may represent the originator's personal views,
>>>>>>> opinions, conclusions and other information which do not
>>>>>>> necessarily reflect those of Lansing School District and are not
>>>>>>> given nor endorsed by Lansing School District unless otherwise
>>>>>>> specified. If you are not the original recipient or the person
>>>>>>> responsible for delivering the email to the intended recipient,
>>>>>>> be advised that you have receive this email in error, and that
>>>>>>> any use, dissemination, forwarding, printing, or copying of this
>>>>>>> email is strictly prohibited. If you received this email in
>>>>>>> error, please notify the sender immediately.
>>>>>>>
>>>>>>> VIRUS WARNING: All reasonable precautions have been taken to
>>>>>>> ensure no viruses are present in this email or any attachments.
>>>>>>> However, Lansing School District cannot accept responsibility for
>>>>>>> loss or damage arising from the use of this e-mail or attachments
>>>>>>> and recommends that you subject this e-mail and any attachments
>>>>>>> to your own virus scanning procedures prior to viewing or opening.
>>>>>>>
>>>>>>> UCE (SPAM) NOTE: Lansing School District takes an aggressive
>>>>>>> approach to unsolicited commercial e-mail (UCE). All UCE sent to
>>>>>>> this e-mail address will be forwarded with complete Internet
>>>>>>> headers to various anti-spam blacklisting agencies; and all UCE
>>>>>>> with non-working or fraudulent opt-out mechanisms will be
>>>>>>> forwarded to the Federal Trade Commission for investigation and
>>>>>>> possible prosecution under the CAN-SPAM Act of 2003.
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Blindmath mailing list
>>>>>>> Blindmath at nfbnet.org
>>>>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>>>>> To unsubscribe, change your list options or get your account info
>>>>>>> for Blindmath:
>>>>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/mwhapples%40aim.com
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Blindmath mailing list
>>>>>> Blindmath at nfbnet.org
>>>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>>>> To unsubscribe, change your list options or get your account info
>>>>>> for Blindmath:
>>>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/sbahram%40nc.rr.com
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Blindmath mailing list
>>>>>> Blindmath at nfbnet.org
>>>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>>>> To unsubscribe, change your list options or get your account info
>>>>>> for Blindmath:
>>>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/mwhapples%40aim.com
>>>>>>
>>>>> _______________________________________________
>>>>> Blindmath mailing list
>>>>> Blindmath at nfbnet.org
>>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>>> To unsubscribe, change your list options or get your account info
>>>>> for Blindmath:
>>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/sbahram%40nc.rr.com
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Blindmath mailing list
>>>>> Blindmath at nfbnet.org
>>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>>> To unsubscribe, change your list options or get your account info
>>>>> for Blindmath:
>>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/mwhapples%40aim.com
>>>>>
>>>>
>>>> _______________________________________________
>>>> Blindmath mailing list
>>>> Blindmath at nfbnet.org
>>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>>> To unsubscribe, change your list options or get your account info
>>>> for Blindmath:
>>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/davison%40gatech.edu
>>>>
>>> _______________________________________________
>>> Blindmath mailing list
>>> Blindmath at nfbnet.org
>>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>>> To unsubscribe, change your list options or get your account info for
>>> Blindmath:
>>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/mwhapples%40aim.com
>>>
>>
>> _______________________________________________
>> Blindmath mailing list
>> Blindmath at nfbnet.org
>> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
>> To unsubscribe, change your list options or get your account info for
>> Blindmath:
>> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/davison%40gatech.edu
>>
> _______________________________________________
> Blindmath mailing list
> Blindmath at nfbnet.org
> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> To unsubscribe, change your list options or get your account info for Blindmath:
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/sbahram%40nc.rr.com
>
>
> _______________________________________________
> Blindmath mailing list
> Blindmath at nfbnet.org
> http://nfbnet.org/mailman/listinfo/blindmath_nfbnet.org
> To unsubscribe, change your list options or get your account info for Blindmath:
> http://nfbnet.org/mailman/options/blindmath_nfbnet.org/davison%40gatech.edu