[NFBCS] Sonification of graphs

[Doug Lee]

I'd break the graph sonification problem into two parts: Producing the sounds, and determining what sounds to produce.

The first of those is by far the easier, I'd say. NVDA's progress bar indications, and if I daresay, my older JGauge indications for progress bars in JAWS, are examples of ways to do that. (JGauge, old though it is, remains
at https://www.dlee.org/jgauge/ )

The problem, then, is how to know what sounds to make. Direct interpretation of graphical presentation for translation to sound tends to be a complex task, and rather tied to what sort of graph you're expecting. The type of
sonification we're discussing here might work well for bar graphs, single-line plots, and perhaps even geographical presentations; but a pie chart would be completely nonsensical if presented this way.

The normal approach to this problem, I believe, is to get at the original data and interpret that for sound rather than trying to interpret the graphical presentation itself. That, of course, moves the problem into the realm
of accessibility that is shared by every control type in modern software: What data is made available to assistive technology so that interpretation is possible?

Back to the progress bar example: Progress bars typically provide a percent-complete to assistive technology. That's all NVDA or JAWS would need in order to make a sound that represents the state of progress.

All that said...

If you have a graph that you want translated to sound, and you know what sort of graph it is, and that a translation to sound actually makes sense, there's probably a way to do it. It would usually be preferable, though, to
get at the data that generated it instead, and present sound from that.

Thoughts welcome of course.

On Wed, Apr 01, 2020 at 10:40:34AM -0230, NFBCS mailing list wrote:
Hi Tracey.

A programmer I am not and to be honest I have no current use for the
technology of which you are speaking, however; given the way that nvda uses
beeps to sonify the progress bars on screen as you install software and the
like, I wonder if that technology nvda employs or that particular method of
technical leverage can be used to sonify the graphs as you suggest below?

Just a thought!

 

 

From: NFBCS <nfbcs-bounces at nfbnet.org> On Behalf Of Tracy Carcione via NFBCS
Sent: Wednesday, April 1, 2020 10:34 AM
To: 'NFB in Computer Science Mailing List' <nfbcs at nfbnet.org>
Cc: Tracy Carcione <carcione at access.net>
Subject: [NFBCS] Sonification of graphs

 

I heard a piece on Marketplace on NPR last night that played a graph
sonification.  Where a print graph line would go up and down, the sound went
up and down a piano keyboard.  It was so great.  I understood immediately
what it was showing, even without the presenter's detailed explanation.

I sure wish there were more sonified graphs out there, or there was a bit of
software I could run that would grab a print graph and sonify it.  I expect
it wouldn't work for all graphs, but even some would be helpful.

Tracy



-- 
Doug Lee                 dgl at dlee.org                http://www.dlee.org
Level Access             doug.lee at LevelAccess.com    http://www.LevelAccess.com
"I honestly believe it is better to know nothing than to know what ain't so."
- Josh Billings, 1818-1885 (in "Solemn Thoughts")