[blindkid] Fw: Playing by Ear

[Carol Castellano]

Very interesting.  A few thoughts...Seems to 
validate the discovery method of travel!   I 
wonder if blind people who are spatially 
challenged do well with this technique.  Lastly, 
when will they stop calling that section of the brain the VISUAL cortex!

Have you sent the article to Daniel Kitsch?  i bet he'd be interested.
Carol

At 12:40 AM 10/3/2011, you wrote:

>This is a very interesting article, although I 
>find the last sentence disturbing.
>
>Debbie
>
>
>
>>Science 30 September 2011:
>>Vol. 333 no. 6051 pp. 1816-1818
>>DOI: 10.1126/science.333.6051.1816
>>Playing by Ear
>>  1.. Sara Reardon
>>Audio-based computer games are helping blind 
>>volunteers learn navigation skills and may 
>>unlock mysteries of the sightless mind.
>>
>>BOSTON-Like the dull, bare walls of the 
>>classroom here at the Carroll Center for the 
>>Blind, the video game that 28-year-old Rachel 
>>Buchanan is playing on a laptop isn't much to 
>>look at. Onscreen, there's just a simple 
>>rendition of the floor plan of one of the 
>>center's administrative buildings, laid out on 
>>a grid with each cell corresponding to one 
>>step. Though blind since childhood as the 
>>result of optic nerve damage, Buchanan 
>>navigates her game avatar through the maze 
>>quickly, keeping it close to the walls like 
>>someone guiding themselves by touch. The secret 
>>to her speed is inside the headphones Buchanan 
>>wears, which immerse her in a three-dimensional labyrinth of sound.
>>
>>A knock in one earphone or the other indicates 
>>a door on that side. The sound of footsteps 
>>ascends in tone as Buchanan walks her avatar up 
>>stairs. Furniture pings when bumped, and the 
>>jewels she is seeking twinkle more loudly as 
>>she approaches them. All in all, Buchanan says 
>>she feels physically present in the maze's 
>>corridors. "Ah! Go away!" she yelps suddenly at 
>>one point when she hears the warning of a 
>>passing "monster"-her fingers fly over the keys 
>>to skitter her avatar toward safety, into a 
>>section that announces itself as "Women's Bathroom."
>>
>>Once they drag Buchanan away from the game and 
>>into the real building next door-thankfully 
>>free of monsters-neuroscientist Lotfi Merabet 
>>and study coordinator Erin Connors, both of the 
>>Massachusetts Eye and Ear Infirmary, instruct 
>>her to go to various rooms and exits, noting 
>>the routes she takes. Like the other blind 
>>volunteers in Merabet's study, she had never 
>>been in the building before playing the 
>>Audio-Based Environment Simulator (AbES, 
>>pronounced "abbess") game. But after just half 
>>an hour of monsterdodging and jewel-hunting in 
>>the virtual building, these gamers learn its 
>>layout so well that they can quickly navigate 
>>themselves to any room in the real building.
>>
>>A rehabilitation therapist and self-proclaimed 
>>computer-game enthusiast, Buchanan, Merabet 
>>says, is one of the "rock stars" among those 
>>who have participated in his studies of 
>>computer games as indoor navigational tools for 
>>the blind. So far, 10 volunteers between the 
>>ages of 18 and 45, all of them blind since 
>>birth, have played the AbES game and then 
>>successfully navigated the actual 
>>administrative building. Those are encouraging 
>>results as indoor navigation, where GPS systems 
>>don't work and guide dogs aren't always 
>>welcome, is a special challenge for many blind people.
>>
>>AbES "frees the blind from relying on special 
>>devices to navigate when all they need is 
>>already embedded in their brain," says 
>>cognitive neuroscientist Ladan Shams of the 
>>University of California, Los Angeles. "That 
>>must be very liberating." The goal, adds 
>>neuroscientist Franco Lepore of the University 
>>of Montreal in Canada, would be for the blind 
>>gamers to learn to develop maps that the 
>>sighted build subconsciously, so that when 
>>deciding to go to a place, the directions will just "click in your mind."
>>
>>Still, Merabet is hoping to do more than use 
>>AbES to help the blind. With funding from the 
>>National Eye Institute, he and colleagues have 
>>begun imaging the brain activity of both blind 
>>and sighted people as they play the game. The 
>>comparison is meant to reveal how the 
>>navigation techniques used by blind and the 
>>sighted brains differ. "Myths abound," he says, 
>>about how the blind learn, as well as about 
>>what they can and cannot do, such as grasp 
>>abstract visual concepts. As neuroscientists 
>>working at the edge of the educational 
>>community, "we can come with the data to prove 
>>that they are myths." Game navigation, he adds, 
>>"is an interesting neuroscience question that happens to have an end product."
>>
>>The streets of Santiago
>>AbES has its roots in the streets of Santiago, 
>>Chile, a place where blind children are among 
>>the poorest children in the city; they wouldn't 
>>normally have access to cutting-edge technology 
>>as do Buchanan and the students at the 
>>well-endowed Carroll Center. Yet thanks to 
>>computer scientist Jaime Sánchez, it was the 
>>kids of Santiago who nearly 20 years ago got to 
>>play-test the first audio-based computer game for the blind.
>>
>>Sánchez, a researcher at the University of 
>>Chile in the city, had been working on 
>>educational computer games in the early days of 
>>the industry in the 1980s. In 1993, he and the 
>>whole gaming industry experienced the DOOM 
>>revolution. DOOM was the first popular game to 
>>present a three-dimensional field of action at 
>>eye level rather than from a bird's-eye view 
>>above. A DOOM player had to navigate an avatar 
>>through its infamously complex map of corridors 
>>while simultaneously spotting and mowing down 
>>demons with a machine gun. Similar games, known 
>>as first-person shooters, now dominate much of the video-game industry.
>>
>>Sánchez, who was working on developing 
>>audio-based games as learning tools for 
>>disabled children, wondered whether they could 
>>navigate DOOM's unique system. He used the map 
>>as the basis for his own game, AudioDOOM, which 
>>he released in 1998. The "mother of AbES," 
>>AudioDOOM incorporated a similar system of 
>>sound-emitting walls, as well as monsters that 
>>run toward the player with increasingly loud 
>>footsteps. Sánchez recruited half a dozen 
>>children between the ages of 8 and 11, all 
>>blind since birth and mostly from poor 
>>neighborhoods, and watched the children play 
>>the game in their homes or schools.
>>
>>As a proof of the idea that the blind could 
>>navigate a computerized map using only sound, 
>>AudioDOOM was an "astonishing" success, Sánchez 
>>says. Curious as to how well they had learned 
>>the game's map, the researchers also gave the 
>>children building blocks and asked them to 
>>reconstruct it from memory. Although they 
>>hadn't been instructed to keep track of their 
>>movements while playing, the children were all 
>>able to recreate the map to scale.
>>
>>Sanchez wondered whether this ability was 
>>unique to blind children, so the researchers 
>>recruited sighted children of the same age and 
>>from the same area of Santiago and had them 
>>play AudioDOOM while blindfolded. They did 
>>poorly. Despite the auditory cues, the sighted 
>>children hadn't even realized they were in 
>>corridors, and their reconstructions were a mess.
>>
>>Sánchez and Merabet met at a conference in 
>>2008, and the two agreed to collaborate on 
>>studying the differences between blind and 
>>sighted gamers. Suspecting that similar 
>>audio-based games would have potential as a 
>>rehabilitation tool, the pair sought to make a 
>>game located in a place, complete with open 
>>spaces and multiple floors, that actually 
>>exists. They found a home for the AbES project 
>>at the Carroll Center, where Merabet could both recruit subjects and run tests.
>>
>>The arguments for using such games to teach 
>>navigation are numerous, he and Sánchez 
>>contend. The games are simple for the blind 
>>students to play: "We don't have to teach them 
>>anything," Merabet says. And while plenty of 
>>research has demonstrated the effectiveness of 
>>virtual reality systems as training and 
>>navigation aids, the researchers speculated 
>>that putting these tools in the context of a 
>>game would make people play longer while still 
>>learning subconsciously. And even if AbES 
>>didn't turn out to be good at teaching 
>>navigation, at worst the blind students at the 
>>Carroll Center would have a new game.
>>
>>Yet AbES proved effective right away, Merabet 
>>reported at the Envision Conference in St. 
>>Louis, Missouri, last week. The researchers 
>>directed half of the blind students through 
>>AbES's virtual building as though taking them 
>>on a tour. The others were left to explore on 
>>their own in a "game mode," complete with 
>>jewels and monsters. When the researchers then 
>>assessed the blind students' ability to 
>>navigate the real building, all of them could 
>>easily find any room or exit. But there was a 
>>subtle difference between the two groups: 
>>Students who had played in game mode for half 
>>an hour were significantly more creative at 
>>finding the quickest route to an exit. Those 
>>who had been led were far less efficient
>>
>>That's because play is a more natural way to 
>>navigate, Merabet speculates. With a challenge, 
>>a reward system, and a way to hold a player's 
>>attention, video games mirror "the way the brain likes to work," he says.
>>
>>AbES could also teach the blind how to infer 
>>directions, says experimental psychologist 
>>Nicholas Giudice of the University of Maine, 
>>Orono. With most rehabilitation and navigation 
>>training, says Giudice, who is blind himself, 
>>"there's an emphasis on route learning and not 
>>enough on how to build a cognitive map" that 
>>would allow creative problem-solving. After 
>>all, if a route is blocked, it's useless. "If 
>>something like Lotfi's game can get people to 
>>think in these global contexts, that's going to 
>>affect almost everything they do" with spatial tasks, he says.
>>
>>Just a few more minutes
>>Merabet and Sánchez are currently working on an 
>>"AudioZelda" version of AbES that maps the 
>>entire campus. Similar to the famous video game 
>>Legend of Zelda, play Merabet and Sánchez are 
>>currently working on an "AudioZelda" version of 
>>AbES that maps the entire campus. Similar to 
>>the famous video game Legend of Zelda, players 
>>will have to run from building to building 
>>collecting jewels as well as keys to the 
>>different buildings. Sánchez has also developed 
>>a game for the blind based on the Metro de 
>>Santiago-the second largest underground metro 
>>system in Latin America. Still interested in 
>>educational gaming, he's further working to 
>>turn the audio navigation technology into a way 
>>to teach blind children topics such as anatomy 
>>(navigating a virtual human body Fantastic 
>>Voyage-style) and geometry so they can "start 
>>school at the same level as sighted" children, he says.
>>
>>The top priority for the researchers, however, 
>>is to develop software that can create an 
>>audible map based on any provided floor plan. 
>>Merabet says he frequently receives requests 
>>from blind students who want a game based on 
>>their new university, or a mall in their 
>>hometown, so they can learn to navigate it before going there.
>>
>>In all of the AbES iterations, Merabet says 
>>blind volunteers such as Buchanan are active 
>>consultants whose feedback is crucial. 
>>Recently, these volunteers began providing 
>>another type of feedback: images of their brain 
>>activity. For more than a decade, 
>>neuroscientists have studied the brains of 
>>sighted people as they've learned to navigate 
>>mazes or played video games inside MRI machines 
>>or PET scanners. Merabet is now studying 
>>whether the brain activity of blind people doing these puzzles differs.
>>
>>As all of his subjects have been blind since 
>>birth, the visual cortex, which makes up 30% to 
>>40% of the brain's cortical surface, has never 
>>received visual stimulation. In the past 
>>decade, however, researchers have found 
>>numerous ways that brains of the blind 
>>repurpose this "real estate:" the region is 
>>active when they read Braille, interpret 
>>language, and localize sounds, to name just a few.
>>
>>The team has adapted AbES so that the subjects 
>>can play it inside an fMRI scanner. Given the 
>>previous data on brain-region repurposing, it 
>>wasn't a surprise to Merabet that the visual 
>>cortex of his blind subjects' brains was active 
>>during game playing. His team is now trying to 
>>dig up some more specifics. As a volunteer 
>>plays the game, the scanner records brain 
>>activity continuously. When the player 
>>encounters a monster or stops to figure out 
>>where he is in the maze, AbES time-stamps the 
>>event. This allows the researchers to determine 
>>exactly which parts of the brain are actively 
>>making navigation decisions at that point.
>>
>>The researchers' early results suggest that at 
>>these junctures, sighted players generally use 
>>the memory center, the hippocampus, to remember 
>>where they are and decide what to do. But it is 
>>at these decision points that blind players' 
>>visual cortices activate most robustly. The 
>>researchers plan to test people who became 
>>blind late in life, to determine whether the 
>>adult brain's wiring is still malleable enough 
>>to use the visual cortex in this way.
>>
>>The addictive nature of computer games has 
>>provided the researchers with willing test 
>>subjects. When he opens the fMRI scanner after 
>>a session, Merabet says he often finds the 
>>volunteers still playing AbES. "Just a few more 
>>minutes. I need to finish this level!" they 
>>plead, he says. The blind students at Carroll 
>>Center are even competing to see who can 
>>collect the most jewels, he has heard. 
>>"Blindness is so isolating," Buchanan explains. 
>>"Being able to play games, that's the best."
>
>
>_______________________________________________
>blindkid mailing list
>blindkid at nfbnet.org
>http://nfbnet.org/mailman/listinfo/blindkid_nfbnet.org
>To unsubscribe, change your list options or get 
>your account info for blindkid:
>http://nfbnet.org/mailman/options/blindkid_nfbnet.org/blindchildren%40verizon.net