Google unveils AI than can find lost specs

The firm showcased multimodal features in Gemini Nano, an AI assistant that runs “on device” on its Pixel phone, and in the Gemini App.

It also demonstrated a prototype scam alert feature it was testing for Gemini Nano that could listen to a phone call and warn that it was a scam, without any information about the call leaving the phone.

The new AI powered demos were revealed at Google I/O, the firm’s annual presentation for software developers.

A quick AI-powered transcription of proceedings, by BBC News, suggested that the word “multimodal” came up at least 22 times.

Speakers such as Sir Demis Hassabis, the head of Google Deepmind, repeatedly stressed the firm’s long-running interest in multimodal AI and emphasised that its models were “natively” able to handle images, video and sounds and draw connections between them.

He showcased project Astra which is exploring the future of AI assistants. In a demo video of its capabilities, it was able to answer spoken questions about what it was seeing through a phone camera. At the end of the demo a Google employee asked the virtual assistant where they had left their specs, to which it replied that it had just seen them on a nearby desk.

There was also a “live” demo of using video when searching Google. Google Search was able to suggest ways to fix a broken record player, having been shown it malfunctioning.

Also in the announcement:

New AI features such as summarising all the emails on a certain topic will come to Google stalwarts such as Gmail.

And looking much further into future, there was also a demo of a prototype system that would create a virtual “team-mate” who would could be told to perform certain tasks such as attending multiple online meetings at once.