Google introduces a new feature based on artificial intelligence in the Lookout application – Assisted Vision. Thus, visually impaired people can discuss with the artificial intelligence to ask it, through several questions, all the details of the image.
Describing an image is no longer too complicated a challenge for a good AI. However, Google has managed to take this concept one step further to make it more functional and impressive. The American giant has just revealed Picture & Answer functionality in the Accessibility App Lookout – Assisted vision And we were able to give a brief demo of it during Google I/O 2023 a few days ago.
The concept can be easily summed up: you ask any question to understand what is in the picture and the AI gives you a clear answer every time. The purpose of this feature – and monitoring in general – is to help visually impaired people know what is in a photo or graphic. Hence the concept of questions and answers, which means questions and answers. The show, she had a pretty amazing little thing.
Basically, you upload an image to the app and ask the AI to describe it. The latter will perform and explain to you what we see. This first description is effective and may suffice if you just want a quick summary. However, for those who need more details, simply ask a question, in writing or out loud.
Artificial intelligence answers your questions to describe the image
During the demo at I/O, Lookout’s Image Q&A was confronted with an image and here’s the conversation it had afterwards with the human user.
artificial intelligence: In this picture I can see a dog running on the beach. I can also see he has a bullet in his mouth. In the background I can see water, mountains and sky.
humans: What is a dog breed?
artificial intelligence: The dog is a mongrel.
humans: Does the dog look playful?
artificial intelligence: Yes, the dog is playful.
humans: Ocean description.
artificial intelligence: The ocean is calm and there is a mountain in the background.
humans: Are the waves big?
artificial intelligence: No, the waves are small.
humans: Are there clouds in the sky?
artificial intelligence: Yes, there is a cloud in the sky.
humans: Is the beach crowded?
artificial intelligence: No, the beach is empty.
The conversation is not necessarily the most natural, but it is fluid and clear enough to represent an interesting solution for visually impaired people who on the Web often have to be content with very brief descriptions.
There, they have a tool with great potential to give them an accurate idea of the image.
We photographed the demonstration in question. Let’s simply specify that this recording is not of high quality, it was mainly intended to support the writing of this article.
YouTube linkSubscribe to Frandroid
Finally, remember that Lookout has been an app that has been available for some time on the Play Store. This accessibility tool already has many interesting features for blind people: reading business and loyalty cards or labels, detecting the value of banknotes and helping to scan a document.
To follow us, we invite you to Download our Android and iOS app. You can read our articles and profiles and watch the latest videos on YouTube.
