Google DeepMind: Now able to read on lips

Can you "read" what she says? The most recent AI effort of Google DeepMind can. A new project by DeepMind and the University of Oxford has created a lip-reading system that can give the best professionals a run for their money.

System deciphering whole sentences

The newer AI system was formed with about 5000 hours of six different television programs in the UK.

This includes Newsnight, BBC Breakfast, and Question Time, all with a total of about 118,000 sentences. Apparently the researchers formed the AI ​​with shows that was aired between January 2010 and December 2015. They tested the new system on programs broadcast between March this year for the September period. However, the results are incredibly magnificent. The system has apparently accurately deciphered whole sentences.

Lip reader: able to check about 200 clips randomly selected

According to the researchers, the AI ​​also outperformed a professional lip-reader who attempted to decipher about 200 clips selected at random. The lip reader was able to check about 12.4 percent of the clips without error, and the AI ​​was able to make a superb 46.8 percent. Many of his errors are also centered on the missing "-s" at the end of some words. This means that the system outperforms other automated lip-reading systems. Ziheng Zhou of Oulu University said this is a big step as it is difficult to train IA to read on the lips without substantially large data set. In particular, clips should be prepared by researchers to learn the machine. Unfortunately, a lot of audio and video streams are sometimes out of sync, which almost made it impossible for the AI ​​to learn the associations between words and lip movement. Regardless, if scientists were able to correct this error, then it is really possible for the AI ​​to learn in the same way.

Interestingly, AI was able to "realign" audio and video streams that were out of sync. It then automatically processed the value of the video and audio hours to be ready for the "challenge". Now, the same researchers are wondering exactly how this new lip-reading achievement can be used. Zhou thinks the system can be adapted to consumer devices so that the AI ​​can better understand what we are trying to say.