[NFBMO] From the Freedom Scientific Blog: Picture Smart AI: How We Got Here

Sat Mar 16 02:33:02 UTC 2024

From: Freedom Scientific Blog <donotreply at wordpress.com> 
Sent: Friday, March 15, 2024 1:03 PM
To: dangarcia3 at hotmail.com
Subject: Picture Smart AI: How We Got Here

In the constantly shifting sands of technology, true revolutions are rare.
Yet, as I reflect on my 25-year journey as a JAWS user and my role at the
helm of Freedom Scientific's software team over the past year and a half, I
am thrilled to share a devel. 

<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.freed
omscientific.com%2Fpicture-smart-ai-how-we-got-here%2F&data=05%7C02%7C%7C8ff
0ed1c43984914b52608dc451a2c47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C6
38461225943161695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=jcj3ual2F5r9E847gai%2BQu
l7uTuj2LX7m0J6GjOPGao%3D&reserved=0> Read on blog or Reader 

<https://i0.wp.com/blog.freedomscientific.com/wp-content/uploads/2017/08/cro
pped-Freedom-Banner-1.jpg?resize=256%2C256&ssl=1> 

<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.freedo
mscientific.com%2F&data=05%7C02%7C%7C8ff0ed1c43984914b52608dc451a2c47%7C84df
9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638461225943190519%7CUnknown%7CTWFpbG
Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0
%7C%7C%7C&sdata=nSyaXm7ehSHPEYuEWfwWYogswP4sOvyCvzs%2FV2RKUq8%3D&reserved=0>
Freedom Scientific Blog 

<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.freed
omscientific.com%2Fpicture-smart-ai-how-we-got-here%2F&data=05%7C02%7C%7C8ff
0ed1c43984914b52608dc451a2c47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C6
38461225943202285%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ZepTu4oRXfjXa0nMdRE6Bz0H
iluSwwUvIryJhWBAl%2FA%3D&reserved=0> Read on blog or Reader 

Picture Smart AI: How We Got Here 

<https://1.gravatar.com/avatar/daf399cb093af5ecb4ac7e150df53c646b8cfa24bef4b
1669c7821716dcc064b?s=96&d=https%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a
11cd5ca435acc9bb6523536%3Fs%3D96&r=G> 

Ryan Jones

March 15 

In the constantly shifting sands of technology, true revolutions are rare.
Yet, as I reflect on my 25-year journey as a JAWS user and my role at the
helm of Freedom Scientific's software team over the past year and a half, I
am thrilled to share a development that marks a genuine leap forward: the
introduction of Picture Smart AI. This isn't just another step; it's a giant
leap, transforming accessibility for those of us who are blind or have low
vision.

The Dawn of a New Era with AI

Early in 2023, the potential of AI to revolutionize accessibility became
unmistakably clear. It promised a future where some of the digital barriers
that have long stood firm could finally be dismantled. Our commitment at
Freedom Scientific has always been to pioneer advancements that break down
these barriers. We started the AI journey by introducing the Face in View
feature in October which provides feedback to help someone get centered and
aligned in the camera feed for online video calls.

Project Horizon: The Challenge That Became a Milestone

Back in the Summer we started to realize the potential of how AI technology
could break down one of the biggest barriers we have always faced. Detailed
and dynamic descriptions of images and graphics has always seemed to be this
elusive dream; it's been getting closer over the years but was always just
over the horizon. To me, it is one of the last frontiers to conquer in
traditional screen reading. When we realized we could take a sledgehammer to
this barrier, we knew we had to jump. Our excitement was amplified when the
winning entry to the Next Big Thing event in November was also the idea of
integrating AI powered image descriptions into JAWS. In early December I
posed a challenge to our software team. I asked them to integrate this new
technology into JAWS, to do it in record time, and to make sure it was
available to all our users around the world. The team accepted the challenge
and Project Horizon was born. The good news was that we already had the
framework built. The initial Picture Smart feature was introduced in 2019
and was powered by early iterations of AI technology. It served its purpose
and provided information about images that we would not have otherwise been
able to access. However, the older technology pales in comparison to what is
available now from companies like OpenAI and Google. Three months and a lot
of hard work later and here we are; Picture Smart AI is released and
available for all to try out. We chose to first place it in our Early
Adopter Program as a signal that we are nowhere near done with it. In fact,
we are well on our way in developing the next phase of Picture Smart AI but
we wanted to get this out for everyone to start using now.

Picture Smart AI in Action: A Personal Perspective

Picture Smart AI is more than a feature; it's a vision come to life. Using
advanced AI algorithms, it interprets and verbalizes the content of images,
bringing them to life for those of us who rely on JAWS to interact with the
digital environment. This technology will revolutionize the way we engage
with digital content, offering a level of independence and interaction that
was previously unattainable.

One way this recently came to life for me was when I discovered that an
incorrect company logo was being used in a PowerPoint slide I was reviewing.
I used Picture Smart AI to describe the slide and it identified the company
logos that were present which alerted me that one of them was for the wrong
company. Traditional screen reading could not tell me this because the logo
was an image. I've always been a pretty good proof reader with JAWS but this
was the first time I was able to discover something that was purely
graphical information.

Another way this new technology solved a problem for me is when I was
looking to purchase a new coffee machine. I was reviewing product pages on a
popular shopping site that rhymes with Hamazon and wanted to ensure the one
I purchased would have buttons and not touch screen controls. Normally I
would need to dig through dozens of reviews hoping to find someone mention
this kind of info. This time though I reviewed the descriptions of several
of the product photos and got quick confirmation that the machine I was
interested in did in fact have physical buttons.

Each day we are discovering new ways to benefit from Picture Smart AI and I
love hearing about how others are using it to transform their lives. I ask
everyone to please share with us and the community at large how you find it
useful. Together we will explore this new world that has opened up.

Sometimes the picture is distorted

One thing I need to be clear about is that AI powered image descriptions are
not perfect. There are many factors involved but the bottom line is that
sometimes the description has errors or may even be mostly incorrect. The
term for this is "hallucinations" and it happens when the AI makes
assumptions or misunderstands what is in the picture. I have no doubt
hallucinations will become less frequent as time goes on, in fact, we are
already seeing less of it, but it will probably be something we will always
have to deal with. One way we are addressing this is by providing
descriptions from two independent services so you can compare results and
look for discrepancies. Learn more about this by reading my post about
<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.freedo
mscientific.com%2Fpicturesmartchallenges%2F&data=05%7C02%7C%7C8ff0ed1c439849
14b52608dc451a2c47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638461225943
246281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
k1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=q8F1ocQxE3Uma1AXeefKFiGSCSbEeALg331
1bGaC7V0%3D&reserved=0> AI image description hallucinations.

Looking Ahead with Hope and Excitement

As we look to the future, I am filled with excitement for the possibilities
that lie ahead, not just for Picture Smart AI, but for the entire field of
accessibility technology.

I invite you to join us in this journey, to share your experiences,
challenges, and successes. Your insights are invaluable as we continue to
evolve and enhance Picture Smart AI. Together, we can continue to break down
barriers and open up a world of possibilities for visually impaired users
around the globe. Read more and learn how to use Picture Smart AI at the
<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.freedo
mscientific.com%2Ftraining%2Fjaws%2Fnew-and-improved-features%2F&data=05%7C0
2%7C%7C8ff0ed1c43984914b52608dc451a2c47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7
C1%7C0%7C638461225943255091%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=JDebFGCrKcbBwj
TsZXf0ga%2FwR5cA1km8zHUrNu1Yc%2FI%3D&reserved=0> Picture Smart AI feature
page.

Comment 

Freedom Scientific Blog C 2024. Manage your email settings or unsubscribe. 

<https://s0.wordpress.com/i/emails/marketing/wpcom/footer/wpcom-jp-dual-2x.p
ng> 

Get the Jetpack app 

Subscribe, bookmark, and get real-time notifications - all from one app! 

<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple
.com%2Fapp%2Fapple-store%2Fid1565481562%3Fpt%3D299112%26ct%3Dnew-post%26mt%3
D8&data=05%7C02%7C%7C8ff0ed1c43984914b52608dc451a2c47%7C84df9e7fe9f640afb435
aaaaaaaaaaaa%7C1%7C0%7C638461225943301578%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=
1Eh4xnWN4XFTxjFPoYC%2BC4gD65W1r2tb07fp8EJds%2FI%3D&reserved=0> 

Automattic, Inc. - 60 29th St. #343, San Francisco, CA 94110   

<https://pixel.wp.com/t.gif?has_featured_image=0&subscriber_id=37160555&_ui=
fc167a0ec1b46425184b5215c8b24798&_ut=anon&email_domain=hotmail.com&blog_id=1
34131076&post_id=2467&user_email=dangarcia3%40hotmail.com&date_sent=2024-03-
15&email_id=1b858cfbee09b72b75d2bf50dd3ac0a5&email_name=new-post&template=ne
w-post&_en=wpcom_email_open&browser_type=php-agent&_aua=wpcom-tracks-client-
v0.3&blog_tz=-4&blog_lang=0&_ts=1710525787475>
<http://pixel.wp.com/b.gif?blog=134131076&post=2467&subd=blog.freedomscienti
fic.com&ref=&email=1&email_o=jetpack&host=blog.freedomscientific.com> 

-- 

**Private voting is a right, it should not require sight. 
<https://nfbmo.org/PrivateVoting>**

The National Federation of the Blind 
knows that blindness is not the characteristic that defines you or your 
future. Every day we raise the expectations of blind people, because low 
expectations create obstacles between blind people and our dreams. You can 
live the life you want; blindness is not what holds you back.

<https://nfbmo.org/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nfbnet.org/pipermail/nfbmo_nfbnet.org/attachments/20240315/334d5e5b/attachment.html>