[blindkid] Fw: Playing by Ear

[Deborah Kent Stein]

This is a very interesting article, although I find the last sentence 
disturbing.

Debbie



> Science 30 September 2011:
> Vol. 333 no. 6051 pp. 1816-1818
> DOI: 10.1126/science.333.6051.1816
> Playing by Ear
>  1.. Sara Reardon
> Audio-based computer games are helping blind volunteers learn navigation 
> skills and may unlock mysteries of the sightless mind.
>
> BOSTON-Like the dull, bare walls of the classroom here at the Carroll 
> Center for the Blind, the video game that 28-year-old Rachel Buchanan is 
> playing on a laptop isn't much to look at. Onscreen, there's just a simple 
> rendition of the floor plan of one of the center's administrative 
> buildings, laid out on a grid with each cell corresponding to one step. 
> Though blind since childhood as the result of optic nerve damage, Buchanan 
> navigates her game avatar through the maze quickly, keeping it close to 
> the walls like someone guiding themselves by touch. The secret to her 
> speed is inside the headphones Buchanan wears, which immerse her in a 
> three-dimensional labyrinth of sound.
>
> A knock in one earphone or the other indicates a door on that side. The 
> sound of footsteps ascends in tone as Buchanan walks her avatar up stairs. 
> Furniture pings when bumped, and the jewels she is seeking twinkle more 
> loudly as she approaches them. All in all, Buchanan says she feels 
> physically present in the maze's corridors. "Ah! Go away!" she yelps 
> suddenly at one point when she hears the warning of a passing 
> "monster"-her fingers fly over the keys to skitter her avatar toward 
> safety, into a section that announces itself as "Women's Bathroom."
>
> Once they drag Buchanan away from the game and into the real building next 
> door-thankfully free of monsters-neuroscientist Lotfi Merabet and study 
> coordinator Erin Connors, both of the Massachusetts Eye and Ear Infirmary, 
> instruct her to go to various rooms and exits, noting the routes she 
> takes. Like the other blind volunteers in Merabet's study, she had never 
> been in the building before playing the Audio-Based Environment Simulator 
> (AbES, pronounced "abbess") game. But after just half an hour of 
> monsterdodging and jewel-hunting in the virtual building, these gamers 
> learn its layout so well that they can quickly navigate themselves to any 
> room in the real building.
>
> A rehabilitation therapist and self-proclaimed computer-game enthusiast, 
> Buchanan, Merabet says, is one of the "rock stars" among those who have 
> participated in his studies of computer games as indoor navigational tools 
> for the blind. So far, 10 volunteers between the ages of 18 and 45, all of 
> them blind since birth, have played the AbES game and then successfully 
> navigated the actual administrative building. Those are encouraging 
> results as indoor navigation, where GPS systems don't work and guide dogs 
> aren't always welcome, is a special challenge for many blind people.
>
> AbES "frees the blind from relying on special devices to navigate when all 
> they need is already embedded in their brain," says cognitive 
> neuroscientist Ladan Shams of the University of California, Los Angeles. 
> "That must be very liberating." The goal, adds neuroscientist Franco 
> Lepore of the University of Montreal in Canada, would be for the blind 
> gamers to learn to develop maps that the sighted build subconsciously, so 
> that when deciding to go to a place, the directions will just "click in 
> your mind."
>
> Still, Merabet is hoping to do more than use AbES to help the blind. With 
> funding from the National Eye Institute, he and colleagues have begun 
> imaging the brain activity of both blind and sighted people as they play 
> the game. The comparison is meant to reveal how the navigation techniques 
> used by blind and the sighted brains differ. "Myths abound," he says, 
> about how the blind learn, as well as about what they can and cannot do, 
> such as grasp abstract visual concepts. As neuroscientists working at the 
> edge of the educational community, "we can come with the data to prove 
> that they are myths." Game navigation, he adds, "is an interesting 
> neuroscience question that happens to have an end product."
>
> The streets of Santiago
> AbES has its roots in the streets of Santiago, Chile, a place where blind 
> children are among the poorest children in the city; they wouldn't 
> normally have access to cutting-edge technology as do Buchanan and the 
> students at the well-endowed Carroll Center. Yet thanks to computer 
> scientist Jaime Sánchez, it was the kids of Santiago who nearly 20 years 
> ago got to play-test the first audio-based computer game for the blind.
>
> Sánchez, a researcher at the University of Chile in the city, had been 
> working on educational computer games in the early days of the industry in 
> the 1980s. In 1993, he and the whole gaming industry experienced the DOOM 
> revolution. DOOM was the first popular game to present a three-dimensional 
> field of action at eye level rather than from a bird's-eye view above. A 
> DOOM player had to navigate an avatar through its infamously complex map 
> of corridors while simultaneously spotting and mowing down demons with a 
> machine gun. Similar games, known as first-person shooters, now dominate 
> much of the video-game industry.
>
> Sánchez, who was working on developing audio-based games as learning tools 
> for disabled children, wondered whether they could navigate DOOM's unique 
> system. He used the map as the basis for his own game, AudioDOOM, which he 
> released in 1998. The "mother of AbES," AudioDOOM incorporated a similar 
> system of sound-emitting walls, as well as monsters that run toward the 
> player with increasingly loud footsteps. Sánchez recruited half a dozen 
> children between the ages of 8 and 11, all blind since birth and mostly 
> from poor neighborhoods, and watched the children play the game in their 
> homes or schools.
>
> As a proof of the idea that the blind could navigate a computerized map 
> using only sound, AudioDOOM was an "astonishing" success, Sánchez says. 
> Curious as to how well they had learned the game's map, the researchers 
> also gave the children building blocks and asked them to reconstruct it 
> from memory. Although they hadn't been instructed to keep track of their 
> movements while playing, the children were all able to recreate the map to 
> scale.
>
> Sanchez wondered whether this ability was unique to blind children, so the 
> researchers recruited sighted children of the same age and from the same 
> area of Santiago and had them play AudioDOOM while blindfolded. They did 
> poorly. Despite the auditory cues, the sighted children hadn't even 
> realized they were in corridors, and their reconstructions were a mess.
>
> Sánchez and Merabet met at a conference in 2008, and the two agreed to 
> collaborate on studying the differences between blind and sighted gamers. 
> Suspecting that similar audio-based games would have potential as a 
> rehabilitation tool, the pair sought to make a game located in a place, 
> complete with open spaces and multiple floors, that actually exists. They 
> found a home for the AbES project at the Carroll Center, where Merabet 
> could both recruit subjects and run tests.
>
> The arguments for using such games to teach navigation are numerous, he 
> and Sánchez contend. The games are simple for the blind students to play: 
> "We don't have to teach them anything," Merabet says. And while plenty of 
> research has demonstrated the effectiveness of virtual reality systems as 
> training and navigation aids, the researchers speculated that putting 
> these tools in the context of a game would make people play longer while 
> still learning subconsciously. And even if AbES didn't turn out to be good 
> at teaching navigation, at worst the blind students at the Carroll Center 
> would have a new game.
>
> Yet AbES proved effective right away, Merabet reported at the Envision 
> Conference in St. Louis, Missouri, last week. The researchers directed 
> half of the blind students through AbES's virtual building as though 
> taking them on a tour. The others were left to explore on their own in a 
> "game mode," complete with jewels and monsters. When the researchers then 
> assessed the blind students' ability to navigate the real building, all of 
> them could easily find any room or exit. But there was a subtle difference 
> between the two groups: Students who had played in game mode for half an 
> hour were significantly more creative at finding the quickest route to an 
> exit. Those who had been led were far less efficient
>
> That's because play is a more natural way to navigate, Merabet speculates. 
> With a challenge, a reward system, and a way to hold a player's attention, 
> video games mirror "the way the brain likes to work," he says.
>
> AbES could also teach the blind how to infer directions, says experimental 
> psychologist Nicholas Giudice of the University of Maine, Orono. With most 
> rehabilitation and navigation training, says Giudice, who is blind 
> himself, "there's an emphasis on route learning and not enough on how to 
> build a cognitive map" that would allow creative problem-solving. After 
> all, if a route is blocked, it's useless. "If something like Lotfi's game 
> can get people to think in these global contexts, that's going to affect 
> almost everything they do" with spatial tasks, he says.
>
> Just a few more minutes
> Merabet and Sánchez are currently working on an "AudioZelda" version of 
> AbES that maps the entire campus. Similar to the famous video game Legend 
> of Zelda, play Merabet and Sánchez are currently working on an 
> "AudioZelda" version of AbES that maps the entire campus. Similar to the 
> famous video game Legend of Zelda, players will have to run from building 
> to building collecting jewels as well as keys to the different buildings. 
> Sánchez has also developed a game for the blind based on the Metro de 
> Santiago-the second largest underground metro system in Latin America. 
> Still interested in educational gaming, he's further working to turn the 
> audio navigation technology into a way to teach blind children topics such 
> as anatomy (navigating a virtual human body Fantastic Voyage-style) and 
> geometry so they can "start school at the same level as sighted" children, 
> he says.
>
> The top priority for the researchers, however, is to develop software that 
> can create an audible map based on any provided floor plan. Merabet says 
> he frequently receives requests from blind students who want a game based 
> on their new university, or a mall in their hometown, so they can learn to 
> navigate it before going there.
>
> In all of the AbES iterations, Merabet says blind volunteers such as 
> Buchanan are active consultants whose feedback is crucial. Recently, these 
> volunteers began providing another type of feedback: images of their brain 
> activity. For more than a decade, neuroscientists have studied the brains 
> of sighted people as they've learned to navigate mazes or played video 
> games inside MRI machines or PET scanners. Merabet is now studying whether 
> the brain activity of blind people doing these puzzles differs.
>
> As all of his subjects have been blind since birth, the visual cortex, 
> which makes up 30% to 40% of the brain's cortical surface, has never 
> received visual stimulation. In the past decade, however, researchers have 
> found numerous ways that brains of the blind repurpose this "real estate:" 
> the region is active when they read Braille, interpret language, and 
> localize sounds, to name just a few.
>
> The team has adapted AbES so that the subjects can play it inside an fMRI 
> scanner. Given the previous data on brain-region repurposing, it wasn't a 
> surprise to Merabet that the visual cortex of his blind subjects' brains 
> was active during game playing. His team is now trying to dig up some more 
> specifics. As a volunteer plays the game, the scanner records brain 
> activity continuously. When the player encounters a monster or stops to 
> figure out where he is in the maze, AbES time-stamps the event. This 
> allows the researchers to determine exactly which parts of the brain are 
> actively making navigation decisions at that point.
>
> The researchers' early results suggest that at these junctures, sighted 
> players generally use the memory center, the hippocampus, to remember 
> where they are and decide what to do. But it is at these decision points 
> that blind players' visual cortices activate most robustly. The 
> researchers plan to test people who became blind late in life, to 
> determine whether the adult brain's wiring is still malleable enough to 
> use the visual cortex in this way.
>
> The addictive nature of computer games has provided the researchers with 
> willing test subjects. When he opens the fMRI scanner after a session, 
> Merabet says he often finds the volunteers still playing AbES. "Just a few 
> more minutes. I need to finish this level!" they plead, he says. The blind 
> students at Carroll Center are even competing to see who can collect the 
> most jewels, he has heard. "Blindness is so isolating," Buchanan explains. 
> "Being able to play games, that's the best."
>