Meta is working on an AI 'builder bot' for the metaverse

Mark Zuckerberg may still be explaining exactly what the metaverse is, but the Meta CEO just showed off another technology he says will be crucial to the company’s vision of creating immersive environments. Meta's AI researchers are working on a voice-powered bot that allows users to change their virtual surroundings with voice commands.

Zuckerberg demoed an early version of the concept, called Builder Bot, at a virtual event detailing the company’s latest AI research.

“It enables you to describe a world and then it will generate aspects of that world for you,” Zuckerberg said. In his demo, he and another Meta employee used voice commands to create a very basic beach scene with a picnic table, clouds, and, yes, Zuckerberg’s signature hydrofoil. They also added some sound effects of seagulls and ocean waves.

Zuckerberg used the Builder Bot to create a hydrofoil. (Facebook)

For now though, the technology seems to be in a relatively early stage. The environment Zuckerberg generated looked much flatter and lower-res than past metaverse demos, though his legless avatar was present much like other recent demos. “As we advance this technology further, you're going to be able to create nuanced worlds to explore and share experiences with others with just your voice,” he said.

The builder bot is part of a larger AI project called Project CAIRaoke that aims to create the kind of conversational AI necessary to create these virtual worlds. Facebook isn’t the first to experiment with this type of research. OpenAI showed off a neural network capable of generating images from text last year. But Zuckerberg said that project CAIRaoke would be central for the Meta’s future.

“In the metaverse, we're going to need AI that is built around helping people navigate virtual worlds as well as our physical world with augmented reality,” Zuckerberg said. “When we have glasses on our faces, that will be the first time that an AI system will be able to really see the world from our perspective: See what we see, hear what we hear and more.”

Zuckerberg’s comments come as he is trying to pivot the company from social networking to metaverse technology, a move that has so far proved to be incredibly expensive. But as the core Facebook service starts to lose users for the first time ever, Zuckerberg has said “metaverse” experiences like the company’s VR Horizon Worlds will drive future growth.