[Njabs-talk] Blind Driver Challenge: The Interface that Touches the Mind

[EVELYN E. VALDEZ]

 The Interface that Touches the Mind:
					Advancing Beyond Autonomous Vehicles
							   by Dennis Hong

On Thursday afternoon, July 8, 2010, Dr. Dennis Hong,
 director of the Robotics and Mechanisms Laboratory at the Virginia
 Polytechnic Institute, addressed the convention. He is leading the group 
 of Virginia Tech engineering students who are working with the NFB to 
 develop the blind-drivable car. Here is what Dr. Hong said:

	  Driving-sighted people like me, we do it every day, and we take it 
 for granted. When you need to go to the grocery store, you just get in your 
 car and start driving. You drive to school, pick up your dearly loved 
 children, and then take them to soccer practice. You hit the open road and enjoy a
 road trip with your friends and family with freedom and joy. Well, in
 modern society driving is really a necessity. It takes you from point A 
 to point B. It's a means of getting you to your destination whenever and
 wherever that may be. At the same time driving is fun and exciting. Some
 people even consider it an expression of power. However, most important,
 driving is really about freedom. Driving is really about independence.
 Unfortunately, however, not everyone has the privilege of driving, mostly
 because of physical challenges, blindness being one of the reasons
 affecting people the most.
  We want to change this. We want to give the blind the ability to
 drive. So in 2007 Virginia Tech accepted a challenge proposed by the
 National Federation of the Blind Jernigan Institute to develop a car that
 can be driven by the blind safely and independently. It was called the
 Blind Driver Challenge. As far as I know, we are the only group in the
 nation that has accepted this call. A lot of people thought we were 
 crazy.
Some people still do, and, as a matter of fact, to be honest, half the 
 time I actually think we are crazy myself.
But the real story behind the challenge goes like this. At the time
 when NFB announced the Blind Driver Challenge, we already had a very 
 active research program in autonomous vehicles at Virginia Tech. For example, we
 won third place at the DARPA Urban Challenge and won a 
 half-million-dollar
 award. This competition was about developing a fully autonomous vehicle
 that can maneuver a sixty-mile course in the urban environment. The 
 vehicle had to obey all the California traffic laws, merge into moving traffic,
 navigate traffic circles, negotiate intersections, avoid a variety of 
 free-standing obstacles, and even park itself--all with no human intervention.
 So we thought we could tackle the challenge proposed by the NFB. We had
 already successfully developed an autonomous vehicle, so we thought, "How
 hard could it be to develop a car for the blind?" Well, we couldn't have
 been more wrong. We quickly realized that what the NFB wanted was not a
 vehicle that could drive a blind person around, but rather a vehicle that
 a blind person could actually operate by making active decisions.
 Realizing this, we had to start from scratch; we had to go back to 
 the drawing board and rethink how we could pull it off. Sometimes we doubted
 whether it was even possible, but, when we realized the importance of 
 this mission for the blind community and the huge positive impact it could 
 have on society, we understood that the potential for the technologies we'll 
 be developing along the way would have more far-reaching impact than just
driving. So in 2008, with thirteen very talented and hard-working, smart
 undergraduate students and only $3,000 in funding, we started developing
 our first vehicle for the blind.
 Let me tell you a story. I still remember when the folks from the 
 NFB first visited my lab, the Robotics and Mechanisms Laboratory (RoMeLa) at
Virginia Tech, and the stupid mistakes and awkward situations I created 
 due to lack of experience and understanding of blindness. At the time I did 
not know anyone personally who was blind, and I do not have any blind members
of my family, so I had all those stereotypes in my head about blind 
people and erroneous assumptions about blindness.
 Though we are good friends now, I have to confess that I felt very
uncomfortable when I first met Mark Riccobono, the executive director of
the NFB Jernigan Institute. He was the very first blind person that I had
a true conversation and interaction with. When he first visited our lab a 
few years ago, actually I was not at all prepared. I didn't know what to do. 
We have so many visitors through our lab from high-profile sponsors to K-12
students, so we are always ready and prepared to greet visitors and give
tours. However, that day, when Mark came to our lab in the basement of
Randolph Hall, he was sharply dressed in a suit and tie as usual. First I
tried to grab his arm to guide him. I was giving a tour of our 
laboratory,
and I constantly said, "As you can see," then I quickly remembered and
thought to myself, "Ahho, I just said, `As you can see' to this blind 
 man," which then created this awkward pause. I know better now. Then in the
 afternoon I remembered the movie, Daredevil, with Ben Affleck, this 
 heroic blind guy. I wanted to be polite and actually brushed my teeth twice 
 after lunch, remembering from the movie that blind people have a heightened,
 almost super-hero-like ability to smell. You might laugh, but that was 
 how I thought of the blind at the time. You would be surprised how many 
people in our society still think the way I did.
Then, as I and my students constantly met with the folks at the NFB
 Jernigan Institute, visiting the NFB headquarters in Baltimore overnight,
 interacting with students from schools for the blind, and working 
 together with blind engineers, we started to learn more about all the
misunderstandings about blindness. One of the biggest among them is the
ability of the blind to perform at jobs. I have learned and personally
witnessed that, contrary to general belief, there are really very few 
jobs that blind people cannot do well. Throughout my work on this project I've
been talking and emailing back and forth with blind people from all over
 the world and was surprised at the jobs these people have. They range 
 from office managers, farmers, IT specialists to auto mechanics. Some ride
horses as a hobby, and I even talked to a hobbyist who is a drag racer.
 Gradually I understood that with just a little technology the blind can
 really do almost anything that a sighted person can do.
 We need for the rest of society to understand this, and what better
way to deliver this message to society than for a blind person to drive a
car? At the same time, if this vehicle becomes a reality, which it will,
and is available to the general public, the impact on the blind would be
huge, opening even more doors to new jobs by providing a safe, independent
>> means of transportation.
What is this car for the blind? How does it work? We don't have a 
lot of time, and the vehicle is very complicated, so I will give you a very
quick overview of how it works. There are three parts to the system: part
1, perception; part 2, computation; and part 3, nonvisual user 
 interfaces.
 So in the first stage, perception, this vehicle has different kinds of
 sensors all around it, from laser range-finder sensors to cameras. The
 laser range-finder sensor shoots out a laser, and, if there is an object 
 in front of it, it bounces back, and a computer measures the time of flight.
So, if you know the speed of light, which you do, then you can measure 
 the distance of objects. The laser shoots around, scans the environment, and
 makes a map around the vehicle. The camera system looks all around the
 vehicle by use of some very sophisticated computer vision algorithms, to
 identify and classify objects so that the vehicle knows, oh, a tree is 
 over there; a rock is over here. These are the lanes. A vehicle is to the 
left, and it's going at such-and-such speed and direction.
The second step is computation. This is a vast amount of data from 
the sensors, and that sensory information is fed into the computer, and the
computer tries to generate a world model. This is essentially a map 
around the vehicle that the computer can understand. Now the challenge is the
third stage--nonvisual user interfaces. How do we move or channel these
ast amounts of real-time information to a person driving the vehicle
without using vision? This is a challenge, so during the past three or 
four years we've been working on many, many different types of nonvisual
interfaces. I'm sure you've probably heard some about the vibrating vest,
the AirPix device, and the glove. By the way, some of my students were
 here. They left yesterday, but they brought some of the interfaces and 
 did a demonstration. Did anybody have a chance to play with those? [applause]
We also brought the vehicle that will eventually become the next 
 generation
 Blind Driver Challenge vehicle. It is a Ford hybrid Escape, very 
 exciting.
 One thing I want to point out is a more philosophical approach: what
 really is this Blind Driver Challenge vehicle? You know, we already have 
 a fully autonomous vehicle. Is the challenge just to put a blind person in
 it? Is that the Blind Driver Challenge vehicle? No. We want people 
 actually to drive the vehicle, so we have two types of interfaces. One is called 
 the instructional cue interface. The other is the informational cue 
interface.
Let me explain a little more, because this is a rather important concept.
The instructional user interface includes the drive grip, which is like a
glove, that has five ring motors on the knuckles and tells you how to 
 turn the steering wheel. With this device the computer makes the decision: 
turn right, turn left, stop, go, push on the brake. So this information or
instruction is transmitted to the driver through these interfaces. We 
call this the "backseat driver problem." This is not really driving. You're
driving, but you are really following orders from a computer. You can 
call that a "Blind Driver Challenge" vehicle, but it is not our vision.
We started from there. Now we are moving towards the information cue
devices. The AirPix is a good example. AirPix is a small tablet-like 
device that has holes on it and compressed air comes out and forms an image of 
the map around the vehicle. You put your hand over it and feel, "Oh these are
 the roads. That's a tree over there; there is a moving vehicle to my
 right." The computer provides information about the vehicle so that it is
 you, the driver, who make active decisions. That is the concept.
 We started with instruction cues and are moving toward information
 cues, and the future is great, and it's looking good. Let me give you a
 brief timeline. In 2008 we started a feasibility study. We started with
 this low-cost dune buggy that we bought on eBay for $2,000. We only had
 $3,000, so we used two-thirds of it. Then we got a bunch of donations of
 equipment from companies, and then we generated these first-generation
 interfaces, which included vibrating chairs and vibrating vests, a click-
 wheel interface, and other things.
  In 2009, last year, we had our first successful test run in early
 summer, and, as Mark Riccobono mentioned, we brought the vehicle to the
 Youth Slam event at the University of Maryland and had two hundred 
 students from all over the nation who came. Some of them had the chance to drive 
 it, and the experience was tremendous. When I think of it, I get tears in my
 eyes.
  This year we are developing the next-generation vehicle. The red
 buggy demonstration that we had last year was really a feasibility
 experiment; it was run in a parking lot. The lanes were defined by red
 traffic cones. It was a very controlled environment, but now the next-
 generation vehicle is going to be running on real roads. This is a real
 car, and this is going to be the real Blind Driver Challenge vehicle. I 
 am very excited about this.
	  As you have probably heard, this has been all over the news, even
 internationally. It has been on the cover of several magazines, on TV 
 news, everywhere. I am literally getting hundreds and hundreds of emails,
 letters, and phone calls from people all over the world. Most of them are
 positive: "Dr. Hong, this is great. Thanks for doing this." Some of them
 give us advice and feedback. But from time to time this is a 
 controversial
 project, and I do get letters, most of them from sighted people, saying,
 "Dr. Hong, are you out of your mind? We already have teenagers texting
 while driving, which is dangerous. What do you think you are doing 
 putting blind people on the road?" To be honest with you, it is rather a valid
 concern, so this is good news, bad news, good news, bad news, but I get a
 lot of questions from the community saying, "When can I buy this vehicle.
 When can I drive the vehicle?" Well you will be able to drive the vehicle
 soon in a test track, in a controlled situation. When will you be able to
 buy it? That's the bad news. This vehicle will not be a real product for
 the general consumer until it's proven 100 percent safe, at least as safe
 as a regular vehicle today. The good news is that I truly believe it can 
 be done. [applause]
Now again, the bad news is that, aside from these technical
 difficulties, a hurdle which we really can tackle, there are many, many
 other issues. How is a driver's license going to be issued? How is
 insurance going to cover this? The social acceptance. A lot of issues 
 need to be addressed; nonetheless, this is a very exciting project. You will
 actually have a chance to drive this vehicle. January 29 at the Daytona
 International Raceway, we will have the first sneak peek, public
 demonstration, and next year at the national convention we will have the
 full demonstration and it's very, very exciting.
  When I talk to my students who work on this project, I always ask
 them, how many chances in your lifetime do you have an opportunity to
 change the world? This is actually that moment, so we are the Virginia 
 Tech Blind Driver Challenge Team, and we expect to see spectacular things 
 coming in the next few years. .