[Tactile-Talk] My Introduction And Remarks From 2024 Science Division Talk

Wed Aug 28 01:05:09 UTC 2024

I like your descriptions of the problems and the features.  Some of your
line vanishing or interfering problems can be fixed by changing the filters
and inverting pins up to pins down.  My favorite thing lately has been I got
the Black hole pictures when they first came out in 2019 the day they came
out and the new ones in 2024 the day they came out and they come up great on
the Graphiti if you set the filters and the inversion correctly.  

One thing that needs to be worked on for all the tactile displays is color
filters.  The filters currently on the Graphiti are brightness filters.  I
think if we add color filters it will make controlling the pictures amazing.
In fact, I had coded a couple python scripts to show this through the USB
protocol.  

John, I think I also told you about the Anscombs Quartets I got from MIT
when they were doing some research and was able to create code that I could
see all four on the Graphiti by reducing the images.  It really shows what
Anscombs is all about.

I like writing games for the Graphiti like the Tetris I play.  With that
said I am now more interested in larger data sets since I just taught a Data
Science class.  WE didn't use the Graphiti in it, but we did use swell
paper, and I used an embosser to see more detail in a larger area without
having to zoom.  I think multiple tools are always better than just one.  I
am still waiting on a few libraries to come out of Alpha that let you view
graphs on regular braille displays with a combo of seeing them partially,
hearing their sonification, hearing descriptions, and being able to ask
questions of AI.  I think in the future these libraries will also support
these tactile displays.   I am hoping the Holy braille folks, and the
Kadence folks join the list and talk about the things they are doing.  Once
these libraries are released I will post some blogs about them and will make
sure to let people know about them here.   One cool feature about one of the
libraries is after you save the image you can pass it around and the person
who gets it does not have to have the development chops to view it.  It just
works in a web browser with what ever tools you have.  If you have AI keys
you can use it with them or if you have a braille display you can use it or
just sonification or speech if that is what you like.  I am pretty excited
about this one.

WE need to make sure the companies making these displays try to use the same
protocol.  So far Orbit is the only one that has a protocol you can request
for USV control but maybe they will all get together and support a Open
protocol so that you can use software from each device.  Even though they
all have extreme differences of features and usage.

I am thinking of writing something like The Canute has to show live sports
events for the Graphiti.  I thought something like the first courts method
of watching American football would be cool on the Graphiti.  So far I just
have a basic o's and x's display of a single offence and defense.  I want to
now hook it up to a live data feed but some of those data feeds are
expensive.

Anyway that is my ramblings for now.

From: Tactile-Talk <tactile-talk-bounces at nfbnet.org> On Behalf Of John
Miller via Tactile-Talk
Sent: Tuesday, August 27, 2024 6:39 PM
To: tactile-talk at nfbnet.org
Cc: John Miller <johnmillerphd at hotmail.com>
Subject: [Tactile-Talk] My Introduction And Remarks From 2024 Science
Division Talk

Hello,

My name is John Miller and I am the moderator of the tactile-talk list.

I look forward to hearing from others about how the blind are using tactile
graphic displays either professionally or at play.

I want to thank those who have stepped forward and introduced themselves.

I am a totally blind electrical engineer. At work I have the opportunity to
do data visualization. I use a Graphiti tactile graphic display from Orbit
Research to review certain datasets.  These days a standard computer monitor
has 1920 by 1080 pixels which is many more pixels than any tactile display.
Because of the resolution of the tactile display and the small number of
pins on a tactile display, we need to be thoughtful in how we examine an
image with the display. I would like to hear experiences from sighted
colleagues, teachers, and other blind individuals interacting with tactile
graphic displays.  Did you have an opportunity to interact with a tactile
graphic display just for a few minutes perhaps at a blindness convention? Do
you have access to a tactile graphic display while taking a high school or
college course? What kind of things have you made sense of while using this
technology? Quite a sizeable community subscribed to this list in the last
couple of days.  It is my guess that most of us do not currently have access
to a tactile graphic display because of the cost.  I greatly benefited from
creating my own tactile graphics on paper or what methods came to hand.  I
am very thankful for the many volunteers who provided me paper tactile
graphics in the past.  If list members wish to discuss how to make a tactile
graphic with Swell Paper, on braille paper, or with 3-D modeling on this
list, I think that is a fine idea.

Here are some remarks about the Graphiti I shared at the 2024 NFB Science
and Engineering Division meeting in Orlando, Florida.

With the help of David Andrews this list Tactile-Talk has been created.

To subscribe to the list send a message to tactile-talk-subscribe at nfbnet.org
<mailto:tactile-talk-subscribe at nfbnet.org>  and leave the subject and body
blank.

To post to the list send an e-mail to tactile-talk at nfbnet.org
<mailto:tactile-talk at nfbnet.org> . Please invite others who might be
interested to join.

One of the goals of this list is to capture findings and best practices for
using  a specific tactile graphic display.

If you are a blind person and have used the Graphiti to help accomplish a
particular task please contact me at johnmillerphd at hotmail.com
<mailto:johnmillerphd at hotmail.com>  and mention what kind of task Graphiti
assisted you in.  I am equally interested in hearing from an educator of the
blind, a parent of a blind K-12 student, or a sighted friend who would like
to report the kind of activity you have seen a blind person do with the
Graphiti that seems easier or different than using speech output,
refreshable braille, or hard copy braille graphics only. My request would
equally go out to those who have worked with a Monarch tactile Graphic
Display from HumanWare. If you feel comfortable, just post some of your
experiences to the list and we can all learn from each other.

At the 2024 annual Science Division meeting I shared about how to sign up to
the Google group blindgraph.  One of the pieces of data from this resource
is a folder of 41 PDF images and corresponding comma delimited CSV files.

In an upcoming update I will also make available the corresponding PNG files
suitable for interacting with AI applications. 

The files contain plots of sine waves, parabolas, line plots of other
algebraic expressions, scatter plots, histograms, block diagrams,
blueprints, and other technical figures.

I invite you to review these images with a Graphiti or other tactile graphic
display. 

Also I invite you to review the PNG files using the Picture Smart AI feature
in Jaws for Windows.

With Jaws for Windows make a notation of which version you are using.  I
recommend the latest which is currently JAWS 2024.06.121.

How well does each image render for you as a tactile graphic? What
additional metadata would help you in reviewing images of this type?

I obtained a Graphiti in September 2023 and have summarized some findings
from my experiences with it between that time and June 2024.  The Graphiti
has both a HDMI mode and a USB communications mode.  This report describes
accessing information with the HDMI mode only.  Using the USB mode is a
promising capability that requires additional exploration.

The Graphiti is quiet. It is extremely quiet compared to braille embossers
on the market. 

It renders a new image in five seconds.  In HDMI mode, the Graphiti
constantly updates when something changes on the monitor of my laptop. 

When I am typing up code to form a new plot, I toggle the space bar on the
Graphiti to stop what I call "image chatter".

This action stops the pins from moving up and down as Graphiti updates an
image. Just toggle the space bar on Graphiti again when ready to view a new
image that is displaying to your computer monitor.

I participated in a Microsoft Teams meeting with Venkatesh Chari from Orbit
Research and a group of researchers from MathWorks who create the product
Matlab. The objective of the meeting was to explore the increase of
accessibility of STEM for the blind. Venkatesh traveled to the MathWorks
office in Massachusetts while I participated remotely from San Diego.  A few
days earlier I sent a Matlab script to MathWorks that generated a figure.
During the meeting a Mathworks engineer modified the script forming a
modified image.  Because the engineer was using screen sharing in Microsoft
Teams and because I was using the Graphiti, I was able to examine the
modified graphic in real-time on the Graphiti.  This is just one use case
that is very exciting about how the blind can remotely consume graphical
data in a new way using tactile graphic displays.

I have had a positive experience reviewing a folder of PNG files using the
Graphiti.  The Windows image viewer has an easy to use rotate by 90 degrees
feature.  It can be helpful to rotate a crowded axis to the horizontal axis
for the Graphiti since it has a larger number of pins in the horizontal
direction.  The Graphiti has 60 pins in the horizontal direction and 40 pins
in the vertical direction.

The Graphiti contains several different filters for how it processes HDMI
data. Selecting one of these filters is achieved by pressing the center of
the arrow controls. I believe I selected the 4th filter out of 8 to achieve
tactile contrast between red and green lines. The Graphiti permits 5
different tactile heights. Using the particular filter described above, the
red line appears as a high tactile line while the green line appears as a
medium height tactile line. Using this mode I examined a figure containing a
red wide parabola and a green narrow parabola displayed on the same image.

One thing I like about the Graphiti is that your hands can be on the display
while it is updating and this does not change the updated image. This can be
very helpful for comparing one image to another or for when getting context
by zooming in or zooming out of an image. Just as a reminder, the Graphiti
has a braille keyboard for issuing graphics commands. I believe zooming out
is dot 2 and zooming in is dot 5. I have had good success reviewing a plot
of data saved as a PNG file.  I have not had so much luck reviewing this
plot once it has been pasted into PowerPoint. 

I felt the Graphiti display as a sighted colleague created a smiley face in
Paint. 

He added funny ears, a mustache, a beard, and a box-shaped hat. 

As each feature was drawn, I could see the updated image with Graphiti.  My
colleague brought up at random stock images of a car, a stick figure of a
person, and a cartoon drawing of a house. Each time I called out the correct
name of the rendered drawing. Looking at stock images was a lot of fun but
it also felt like I was building a skill in terms of how to make sense of a
drawing without any context. Such a skill might be just the thing we tactile
graphics readers could use as we interact with more and more tactile
graphics.

On the other hand, I was not able to identify a detailed image of a modern
spacecraft. Detailed and busy images displayed on the Graphiti often do not
make sense to me. Some images may not render well because of their contrast.
In this case, all the pins might be raised.

I have used Graphiti regularly to review line plots generated by Matlab from
MathWorks.  I also find Graphiti works well in displaying the graphical
output from the website www.desmos.com <http://www.desmos.com>  when
displaying the graph of an equation.

The HDMI mode has some remaining challenges for tactile viewing.

When a line or object is too close to a desired object, a portion of the
desired object can appear erased.

When viewing plots of a function I find it helpful to disable the horizontal
axis.  In this case the function is more likely to display in its entirety.
I toggle the horizontal axis on and off to collect the information that I
need from the plot.

I observed that a line connecting two boxes in a block diagram can appear
erased when the boxes are too close together.

I generated an image in PowerPoint of two boxes connected by a thick line.
The Graphiti displayed the image correctly including displaying the
connecting line.  As I moved the boxes closer to each other at some point
the boxes displayed nicely but the line between them does not display. In
addition it takes a little bit of guess and check to learn how thick a line
needs to be before it displays nicely.  Similarly small dots on a scatter
plot may need to be enlarged before they display well.

There may be some image filtering going on in the HDMI driver itself that
impacts how tactile images are displayed. 

Learning more about which images display well and when a region of an image
displays poorly on a tactile display can help with choices of how to
generate the image for best viewing quality.

I find the strength of the Graphiti is examining functions like plots and
also examining 2-D data.

I would like to hear about successes and challenges from others.

Very Best,

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nfbnet.org/pipermail/tactile-talk_nfbnet.org/attachments/20240827/8200546b/attachment-0001.htm>