[nfbwatlk] Facebook working on artificial intelligence that can tell what’s in photos, CNBC, November 3 2015

[Nightingale, Noel]

Link:
http://www.cnbc.com/2015/11/03/facebook-artificial-intelligence-photos-web-summit.html

Text:
Facebook working on artificial intelligence that can tell what’s in photos
Arjun Kharpal | @ArjunKharpal
Tuesday, 3 Nov 2015 | 9:17 AM ETCNBC.com

Facebook is testing an artificial intelligence (AI) feature that will allow it to answer questions about a photo, a feature aimed at helping blind people "see" images uploaded to the social network.

The U.S. giant has been pushing hard to develop its AI capabilities – alongside Apple and Google – with the technology being seen as key for the future of these companies.

At Web Summit conference in Dublin, Ireland, Facebook's chief technology officer Mike Schroepfer, talked about the firm's advances in the space. Earlier this year, the social media titan unveiled a technology called "Memory Networks" which allows a machine to perform sophisticated questioning and answering. He showed an example where a computer was fed the synopsis of the "Lord of the Rings" and was able to answer questions about the story.

But after hints over the past few weeks about the next steps, Schroepfer confirmed that the company is trialing a technology that will allow a machine to identify the different aspects of a picture and then answer questions about it, a feature that could be potentially helpful for those who are visually impaired.

"So much of the world is visual. So much of sharing on social networks is photos and videos. Much of your brain is dedicated to processing visual imagery. So one of the keys to building systems that work is teaching computers to understand the visual world," Schroepfer said during a keynote at Web Summit.

"Next month, we're going to talk about some breakthrough work we've done…on image segmentation."

Helping the blind to 'see'

The Facebook technology chief described an image of a baseball game with a person batting, a pitcher and a catcher. The AI system will be able to tell where the bat that a player is holding ends, an a player's arm begins, for example. A user would be able to questions Facebook's AI service and receive answers about what is happening in the picture.

"Imagine that you are one of the hundreds of millions of people with some sort of vision disability and you have trouble participating in the visual part of social networks. And one of your friends, who just had a baby, posts a photo and captions it. There's technology already out there to read all the text on the screen to you…but you wanted to learn more about what this photo is. We built a system that allows you to ask questions about a photo that it's never seen before," Schroepfer said.

Facebook search?

But it could also fit into Facebook's broader push into the search space. Last month, Facebook said it had indexed 2 trillion posts that would allow users to search for anything via the social network and be served public posts related to the topic. If this AI image recognition technology is built in, it could surface a much more complex result that includes more images. Schroepfer did not outline how the AI tech would fit into the service but said it would be used to help surface relevant posts in a user's newsfeed.

"With all of this data we could possibly consume for your newsfeed, all of the hundreds of millions of photos being uploaded every day, we want to do the very best job we can to make sure we show you exactly what you want...By understanding just by looking at the pixels what's in this photo we can do a better job of showing you what you want, and not showing you what you don't want," the CTO said.

AI has been a big focus of Facebook's of late. Earlier this year, the firm said it was trialing a service through its Messenger app called "M", a digital personal assistant much like Apple's Siri. But instead of using just AI, it also has real people behind the service to carry out some tasks that machines cannot carry out yet.