NFBNJ-Seniors] article about the OrCam products

[nancy Lynn]

MyReader and MyEye from OrCam: Text and Item Recognition at the Touch
of a Finger

Bill Holton

In a recent AccessWorld article,
Choice Finds from the ATIA 2016 Conference Exhibit Hall, we took a
first look at the OrCam from Israeli-based OrCam Technologies. 
The OrCam is a wearable device that detects what you are looking at,
recognizes the presence of text, and speaks the text aloud. The OrCam
can also identify various products, US currency, credit cards, and even
faces.

While many individuals with visual impairments use smartphones and
other mobile devices to accomplish most of these tasks, there is a
growing population of people who are newly blind for whom such a device
might enable a significant step toward personal independence. People
who are newly blind and who do not have substantial technology
experience and training may find this device of particular benefit.

At the time of the previous AccessWorld mention, OrCam was not yet
available for purchase. It is now available with training via a network
of dealers.
Recently I had a chance to put the device through its paces in an
extended training session. Here's what I found.

A quick Tour of OrCam

The OrCam has two components. The first is the processor unit, which is
5.5 inches by 2.25 inches by .8 inches, and weighs just over 5.5
ounces. 
It fits
easily in the palm of the hand, and is designed to fit into a
standard-size hip pocket. A belt clip is also provided, but using a
handbag might be problematic.
A shoulder bag carried on the right shoulder might work, or a backpack,
since there is a 32-inch cable connecting the processor unit to the
OrCam. Hopefully, the company will soon be able to offer the option of
either Bluetooth or other wireless connectivity.

The processor unit has an audio jack, a charging port, and three
buttons: a Power button and Volume Up/Down rocker on one edge, and on
the opposite edge the Trigger button, which the user can press to
initiate a scan or hold down for two seconds with either of the Volume
buttons to open the Settings menu.
As you will soon see, however, you can access most essential OrCam
features without using the physical Trigger button.

The second component is the camera/speaker head unit, which is
connected to the other end of the cable. This component snaps on to the
right arm of nearly any standard-size pair of eyeglasses or sunglasses.
(Note: wire frame glasses cannot currently be used with the OrCam.)
When attached properly, the OrCam 8-megapixel camera is positioned at
the very front of the glasses, and the speaker is positioned toward the
rear. Weighing in at just over 1 ounce, the head unit does not feel
awkward at all to wear. Some prerelease versions of OrCam Eye used bone
conduction earphones, but for the release version these have been
replaced by a single speaker with sound output directed toward the
user's right ear. Those who have hearing deficits in this ear may have
a problem using OrCam out of the box. Such users may need to connect
their own headphones, external speaker, or hearing aid using the
processor unit's audio jack.

The non-swappable battery is rated for 4.5 hours of continuous use. It
can be recharged in four hours using a USB power adapter, computer
port, or car charger. At the beginning of my two-hour training session
the battery was at 100 percent. It ended at 50 percent, so at first
glance these numbers do seem fair.

The Two Flavors of OrCam

OrCam comes in two versions: OrCam MyEye, which is priced at $3,500,
and OrCam MyReader, which costs $2,500. Let's take these configurations
one by one, describing what they can and can't do. Keep in mind that,
as is the case with most camera-based accessibility devices and apps,
there is a certain learning curve involved. I only had one training
session, so I also sought out feedback from an OrCam veteran, which you
will find later in this article.

OrCam MyReader

OrCam MyReader focuses on text recognition. There are two ways to
instruct the device to identify and speak recognized text. First, look
directly at the page or label of text. This can be a bit tricky, at
least at first. I tended to look higher and a bit to the right of my
intended target. With practice I quickly improved my aim. Now, with the
target text directly ahead, press the Trigger button on the processor
unit. You will hear a shutter sound as the image is taken. A second or
two later the text will begin speaking.

Alternatively, place a finger near the top center of the page or block
of text you wish to have read. You will need to place your finger so
that the fingernail is facing up. OrCam recognizes the fingernail, and
uses it to set the camera's focus. After hearing a confirmation chirp,
remove your finger from the text.
This action "triggers" OrCam to auto-snap the picture. If you do not
hear a chirp, remove your finger from view and then try again.

The developers of OrCam have built their own, proprietary text
recognition engine from the ground up. I found the recognition equal to
or better than other OCR products, including the KNFB Reader. OrCam
prompts you if the text is upside down or too blurry to recognize, and
seems to do a good job cleaning up off-center images. It's also quick,
and again, there is no data connection needed. You do need adequate
lighting, however. OrCam tended to fail in poor light conditions, and
there was way to know if the failure was caused by text not being
included in the view or because the light was inadequate. 
This
lead to several instances where I kept trying to get the text in view,
when my problem was actually poor lighting. Perhaps an inadequate
lighting message should be added.

The text is read using either the Ivona, Brian, or Kendra voice. I
found the results clear and easy to understand, even with the small
built-in speaker.
You can change voices, speed, and volume via the Settings menu.

If you wish to stop text reading, simply place your hand in front of
the object you're reading. To scan the text: after speech begins, place
your finger against the text and slide it down toward where you'd like
to skip to. 
This was a bit tricky, and my attempts resulted in many recognition
stops and restarts.
With practice I was able to perform the gesture about half the time. I
suspect that with further practice I could improve this gesture even
more, but I'm not sure it is a feature I would use enough to warrant
the effort.

Using OrCam I was able to scan the left page and the right page of a
book with high speed and accuracy. OrCam does not save your recognized
text after it has been read. You cannot scan ahead in a book as you
listen to previous pages, or save your scanned text to a computer or
other device. 
And speaking
of computers, when I pointed OrCam at my computer screen it detected I
has an Excel spreadsheet open and began reading the data one row at a
time.

Taking OrCam to my pantry, I could touch my finger to a can of corned
beef hash, for instance, or a box of crackers, and OrCam would speak
most visible text. This text tended to be cooking instructions, or
nutritional information, from which I could often as not determine what
was in the package. Product names were problematic, because they are
usually printed in unusual fonts OrCam can't recognize. There is a way
to more consistently identify products, which I will describe in the
next section. For now, it was time to hit the road.

On the Go with OrCam MyEye

Heading outdoors on the day of my training session I was startled to
learn that, even though I have lived in my current house for eight
years, I never knew that the street number was printed on top of my
garage door. As the instructor and I headed out, the UPS man arrived
with two Amazon packages. I was easily able to tell that one of the
boxes was for me and one for my wife by simply touching the label with
my fingertips, then moving my finger away for the picture to snap. On
one of the packages it took me a while to find the label, since it felt
so much like the packing tape. Assuming the daylight was sufficient,
when OrCam did not chirp I knew I had to pull my finger away and try
again.

Corner street signs were also readable, but I cannot say this feature
is particularly useful. I had to be in precisely the right spot, and
know at which of the four corners the sign was located. Trees and other
branches typically interfered, and I had no way to tell if I was even
close to being on target.

Heading to a nearby fast food restaurant/gas station/convenience store,
I was delighted when OrCam read many of the outdoor signs. I do not
think it will be able to distinguish different stores in a strip mall,
as most signs use unusual fonts, and OrCam does not do well with
lighted neon.

At the order counter, I could pick up snatches of the menu board
overhead. I could not read it all, so I had to rely on a printed menu.

By happenstance, the previous occupier of the table I selected had left
a newspaper, so between bites of burger I tried reading a few articles.

OrCam did
an excellent job columizing and only reading down one column at a time.

I had to start the text recognition again for the next column. I also
needed to have some sense of where things were on the page. OrCam
doesn't identify when you're pointing at graphic or picture; it simply
doesn't recognize these elements on a target.

When using the Trigger button, as opposed to the point gesture, for
reading a newspaper, OrCam reads the text on the entire page, starting
at the top-left corner to the bottom-right corner. OrCam announces
"reading next text block" to distinguish between headlines and
different articles.

Our final stop was inside the convenience store, where I was blown away
by OrCam's capabilities. Walking up to a shelf, I touched a finger to
an item.
It read the item name and the price. It was a bag of dog food. I moved
on to another unknown item in a blister pack. OrCam informed me it was
a tire gauge, and also provided the price. Last stop--the refrigerator
case, where I pulled the door open and began touching beverage bottles.
Nearly every time it read what it was and the price tag. It was much
easier than my own pantry or refrigerator, since stores always display
items with their labels front and center.

OrCam MyEye

Along with all the features described above, OrCam MyEye can recognize
currency, credit cards, household or work items you have previously
identified and placed into OrCam's memory using voice memos/audio tags,
and the faces of friends, family, and coworkers.

Currency must be touched or viewed using the trigger button in order to
have it recognized. The various denominations have been pre-added to
the OrCam's memory. Other items you will need to add manually.

OrCam MyEye will store and recognize up to 150 credit cards, pantry
items, household cleaners, and other objects from the size of a pack of
playing cards to a box of cereal. To add an item to the device's
memory, press and hold the Trigger button until OrCam prompts you with
"Start new product learning.
Please point at the product three times at different positions." It's
best to take one photo with the item at arm's length, a second closer
up, and the third using a different background.

After snapping the photos you are prompted to add an audio name or
description of the item. Multiple image sets can be taken, if, say, you
want to add the front and back sides of a box, but this will count as
two items toward your maximum of 150. Note: You must still have the
item in order to clear it from memory.

Now, when you wish to identify that box of teabags or distinguish your
sugar canister from your flour canister, touch the item and then pull
away your finger, as though you were identifying text. OrCam will speak
your audio message.

Identifying faces works similarly to object identification. A person
only needs to be photographed and entered into OrCam one time. Press
and hold the trigger button for about two seconds. The OrCam device
will ask you to please name the person in front of you after the beep.
Then you confirm the person by pressing on the trigger button again to
complete the facial recognition entry process.

Now, whenever that person comes into view, OrCam MyEye will play your
audio tag. Remember, it's playing through a speaker, and though the
volume is fairly low, you should probably avoid names such as "My
stupid boss," or "The guy who owes me money."

My trainer had pre-entered his facial image into OrCam MyEye. At the
fast food restaurant when he wandered away I tried to locate him. It
took a while, but finally he came into view and I knew in which
direction to look and speak.

One last feature I discovered on my own was when OrCam MyEye abruptly
announced, "One person is in front of you." OrCam can recognize that
there are up to eight faces in front of you--a handy feature if you're
standing in a line, or if you're at a party wondering if you're
speaking to a person or a floor lamp.

An OrCam MyEye User Story

As mentioned above, since I only had one OrCam training session I
thought it prudent to check in with another, more experienced user. So
I spoke with Boyd Boyd, 80, who lives in Titusville, Florida. Boyd has
retinitis pigmentosa, and her vision is limited to four degrees. Her
granddaughter is an optician who followed the progress of OrCam as it
was being developed and obtained a device for Boyd as soon as one was
available.

Boyd uses a speech-enabled iPad and Kindle, but she usually prefers to
use her OrCam MyEye to review the screens. "I don't have to worry about
accidentally touching something I didn't mean to and changing things,"
she says.

Boyd also uses her OrCam to read her print Bible and daily devotional. 
Her husband teaches Sunday school, and she was amazed when she could
follow along with his PowerPoint presentations. "OrCam actually helps
me orient my scans," she says. "If my finger isn't pointing directly at
the PowerPoint display it tells me, 'There's more text to the right.'"

Boyd doesn't have many household items in her object recognition
database. She doesn't need to. "I can usually tell what I'm looking at
just by the snatches of text I hear," she says. "It's very handy when
I'm looking for something in the refrigerator--especially when you have
a husband who never puts the mustard back where it belongs."

Boyd does use the facial recognition. Besides her husband, she's also
programed her OrCam with images of a number of neighbors and church
friends, her two daughters, four grandchildren, and their spouses,
along with her four great-grandchildren. "My husband can't sneak up on
me anymore," she laughs.

Recently, the family threw a big celebration for Boyd's 80th birthday. 
"They gave me eight slips of paper with printed messages like, 'Today
you're having your nails done,' and 'You're going on a shopping trip
with your daughters.'"

Final Thoughts

OrCam MyReader and OrCam MyEye definitely work as advertised. For the
newly blind or individuals with physical or cognitive limitations that
prevent them from using a touch screen mobile device, the MyReader and
MyEye are excellent ways to answer the question: "What is this?" Office
workers and others who frequently socialize in groups may also enjoy
the OrCam MyEye.

If I have any hesitation about the OrCam it is because of the two
different models. Many years ago when IBM started selling printers
under the Lexmark name they offered two models of laser printers: a
4-page-per-minute and an 8-page-per-minute. The printers were
identical, other than the fact that the company had disabled the higher
speed on the low-end model. IBM doesn't sell printers anymore.
Similarly, OrCam MyReader and MyEye seem identical other than the
preloaded software. MyReader is upgradable to the MyEye, but I think
the company would have been better served by setting a single price for
the full-featured model. I also wonder how long it will be until
someone designs a similar headset unit and connects it wirelessly to an
Android smartphone.

Who knows? Perhaps the company is already hard at work on that very
thing.

Product Information

Products: OrCam MyReader and OrCam MyEye Available from:
OrCam Technologies
1-800-713-3741 (US)

OrCam is available in English, French, German, and Hebrew in the
following countries: United States, Canada, Australia, France, Germany,
Austria, Switzerland, UK, and Israel. OrCam will soon be released in
Spanish.

Manufacturer's Comments

OrCam's mission is to empower the lives of people who are blind or
visually impaired by harnessing artificial vision innovation.

The wearable OrCam assistive technology device provides independence by
effectively, discreetly, and instantly reading printed text from any
surface, recognizing faces and identifying products and money notes.

Powered by leading minds in the computer vision and machine learning
fields, OrCam's team includes dedicated software, computer, and
electrical engineers; hardware design experts; and a passionate
customer service team--including sighted, low vision and blind
members--to provide a visual aid through a discreet, mobile and
easy-to-use interface.