[BlindRUG] Sonification of plots in R - an experiment

[Vincent Martin]

I am editing some work related to this right now.  After it is accepted for
publications, I can share it.  

 

From: BlindRUG [mailto:blindrug-bounces at nfbnet.org] On Behalf Of Godfrey,
Jonathan via BlindRUG
Sent: Tuesday, April 05, 2016 5:28 PM
To: Blind R Users Group <blindrug at nfbnet.org>
Cc: Godfrey, Jonathan <A.J.Godfrey at massey.ac.nz>
Subject: Re: [BlindRUG] Sonification of plots in R - an experiment

 

Hi Robin et al.,

 

There have been a few discussions about this idea before now, but your work
looks like it has gone further than other attempts I've seen to date. Good
stuff.

I'll send you the work Karen Vines and I did  off-list.

 

I think there are a few lessons from our experiences I'd like to share with
you and the group.

 

We found it difficult to get a great handle on what the sonified result
could show. We couldn't really tell the difference between a normal curve
and a variety of other symmetric unimodal distributions for example,
especially if we used the exact observations instead of  a smooth curve.

 

I'm interested in using any means, including sound, to help show a blind
person what appears in a graph window. I'm sure I have the components that
deliver the contents of a graph window but I haven't gone as far as the
sonification yet. This should be fairly easy to do once I get the text
descriptions for some content  sorted.

 

The main problem I encounter is that the standard plot() command that
generates a scatter plot in many instances is a family of commands and the
resulting graph windows can vary considerably. It would be much better if we
could convince the entire R community to use xyplot() from the lattice
package or some such, but that's not likely!

 

I would prefer to see your functions use (x, y) or (y~x) instead of the
current (y, x) as they are more standard practice and match the options for
plot) etc.

 

I'll write more on this topic after e hear from others. Off to do some
teaching now.

Jonathan

 

 

 

From: BlindRUG [mailto:blindrug-bounces at nfbnet.org] On Behalf Of Williams,
Robin via BlindRUG
Sent: Wednesday, 6 April 2016 3:48 a.m.
To: BlindRUG at nfbnet.org <mailto:BlindRUG at nfbnet.org> 
Cc: Williams, Robin
Subject: [BlindRUG] Sonification of plots in R - an experiment

 

Dear all,

 

In collaboration with a colleague here at the University of Exeter, I've
started to experiment with the idea of producing sound scapes of plots. The
idea was motivated by my current work, which requires me to look at lines of
best fit (often using polynomial smoothers) through many datapoints.
Essentially, we can produce a soundscape of a plotted function, or plotted
data points. The idea is similar to the Mathtrax software produced by NASA
some years ago, except we can now harness the power of R. The code is based
on the tuner package. Essentially, we create a signal that represents the
value of a plotted function using Fourier series, and output the wave to a
.wav file. This can be played from within R, using a software such as
Windows Media Player under Windows, or mplayer under linux. You can choose
from four forms of wave which all sound quite different to me. Datapoints
can be indicated with pulses, and tick marks on the x axis are indicated
with a short blast of the distinctive sawtooth wave. The soundscape moves
from left to write, in order to mimic moving along the x axis. So far, I've
used the sonify() function to examine distributions whose PDF I was
unfamiliar with, and to look at a nonparametric approximation to a line of
best fit estimated from data. I've had to look at many of these plots over
the years, which have often required sighted assistance, and so this
represents quite a breakthrough in terms of working efficiency.

The code can be obtained from here:

https://raw.githubusercontent.com/sieste/sonic-plot/master/Rsonify/sonify.R

For example, you could try

X=seq(-5,-5,.01)

Y=sin(x)

Sonify(y,x,ticks=seq(-4,4,1),pulse_len=0) # without pulses for the
datapoints

Sonify(y,x,ticks=seq(-4,4,1),pulse_len=0.2) # to hear pulses at the
datapoints

 

At this stage I would like to gather any feedback, and suggestions as to how
the script could be made more useful still. Please send me any bugs you find
in the code.

 

Best wishes

Robin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nfbnet.org/pipermail/blindrug_nfbnet.org/attachments/20160405/e12041c6/attachment.html>