World Models Become Robot Playgrounds: Google Genie 4 combines with Gemini models to build better video game worlds, robotics training

Google says that Genie 4, its latest world model, will be used in tandem with Gemini models to create digital practice spaces for robots, giving them endless worlds to train in. So far, though, Google’s demos suggest there’s still a long way to go. 

What’s new: On October 27, Google announced Genie 4, which it has released to a limited group of testers in a research preview. Genie 4 produces interactive worlds that Google says are even more realistic than its predecessor, Genie 3. Users can change a scene midway through with a text prompt, but now Gemini models help work those changes into the scene. Genie 4 also adds sound, which allows it to create real-time audio for each world it generates, whether it’s two people chatting on a motorcycle or snow crunching under a crowned crane’s feet. Google AI Ultra subscribers as well as some developers should be getting access in the weeks ahead. 

How it works: Genie 4 doesn’t have a technical report out yet, but Google’s announcement explains how Gemini models can help direct what Genie 4 renders.

• Navigation: Users move through a scene with video game-style mouse and keyboard controls.
• Text directions: While Genie 4 renders a scene, Gemini 3.8 Flash works alongside it as a director. It breaks the user’s goal into steps, then double-checks each new frame to make sure each of those steps has been executed, adjusting its plan if the user inserts changes as things go along.
• Video: Gemini Omni, Google’s video generation model, helps Genie 4 render more realistic scenery. Google demoed examples including a maple seed gracefully spinning and landing on grass, as well as people shuffling along a 1980s-era train platform.
• Robotics: In a demo, Google’s reasoning model Gemini 4 Argon broke a task into text-based steps for Genie 4 while it rendered a simulation of Apptronik’s Apollo humanoid robot playing checkers with a person. Google didn’t describe a robot control system in the loop.

Results: Google’s announcement was full of curated demos, but the company didn’t offer any quantitative evaluations to back them up. Google says a simulation can run up to two minutes, roughly double Genie 3’s limit, but DeepMind research engineer Hang Qi noted that reaching that limit doesn’t guarantee that a scene will keep its consistency the whole time, according to Bloomberg. Google says it’s still working to improve complex physical interactions. One of its demos even includes a physics slip: A marble bust that drops onto a watermelon settles on its plinth when it should topple over the side. “There’s only so much that you can do with two minutes,” Google DeepMind product manager Diego Rivas said, per TechCrunch. “We’re still in the research journey. We’re not necessarily in the end product journey.” 

Behind the news: Genie 4 has plenty of company: The world models field is getting more crowded, although not every world model operates the same way. Like GWM Worlds 2, the model Runway released in September, Genie 4 generates video in real time. Marble, from AI pioneer Fei-Fei Li’s startup World Labs, and HY-World 2.0, an open-weights model from Chinese tech giant Tencent, build persistent 3D scenes instead. 

• Robotics isn’t the only field disrupted by the new wave of world models. When Genie 3 opened to paying subscribers in January, it sparked a sharp selloff of video game stocks. In October, electric-vehicle maker Rivian bought world-model startup Third Dimension to bring self-driving simulation in house.
• The field is also consolidating. AMD agreed in September to buy World Labs for about $8.2 billion, and, per The Wall Street Journal, Amazon has been in talks to acquire Decart, whose products include world models for robotics and driving simulation. 
• Genie 3 served as an environment for SIMA, DeepMind’s agent for 3D virtual worlds. Waymo later built a driving simulator on Genie 3 to generate unusual scenarios (things like a tornado or an elephant in the road).
• Runway has also pushed its world models toward robotics. In February it announced that robot-control policies acting inside a robotics version of its video model earned simulated scores that closely tracked their real-world results in tabletop tests. In September, Runway debuted a research preview of GWM Worlds 2, a real-time world model with audio. It also announced Praxis-1, a model for controlling real robots that it plans to release with open weights. 

Why it matters:  Google’s clearest evidence that simulation can improve real robots comes from a different approach from other companies. A few weeks before Genie 4’s debut, Google showed Gemini 4 Argon writing physics simulations in the MuJoCo engine based on photos and video. From a single photo of a tabletop, Argon generated more than 4,500 practice episodes, and fine-tuning a robot-control model on them raised its real-world success rate from 56 percent to 75 percent. Google’s proposed path forward, which it calls hybrid world models, would combine elements of both the structured reasoning of language and code models with a generative model that renders pixels. Theoretically that could give generated environments reliable cause and effect, which is something video models often lack.

We’re thinking: A good robot simulator lets plans fail, since robots learn from mistakes as well as successes. Genie 4 pairs its renderer up with a Gemini director that checks whether each step of a plan succeeded. This can accidentally drift the world toward whatever the intended outcome is. In the marble bust clip, the model itself seems to choose a neat landing on the plinth instead of the toppling over the side like physics says it should. Our question for DeepMind: Can a plan fail in Genie 4? We’d like to see that Apptronik robot hand mess up and drop a few checkers, too.