Genie 4 can simulate a world you can steer, and now that world can talk back 🌍

Most AI video I watch is a recording: you write a prompt, wait, and get a clip. A world model is a different idea. It generates the next moment of a scene in real time, while you move through it and change it. On 27 October, Google DeepMind announced Genie 4, and it's the most exciting step I've seen yet in that direction.

What Genie 4 adds

The announcement, by Raia Hadsell, Diego Rivas and Hang Qi, groups the progress into four capabilities:

  • Steering in real time: you navigate with ordinary keyboard controls, and you can change the scene with plain language while it runs, such as changing the weather, moving the camera or adding a person. Gemini adapts each prompt to fit what is already on screen and keeps the world progressing coherently between your inputs.
  • More realistic physics and people: DeepMind says Genie 4 simulates aerodynamics, collisions and more, and that the biggest leap is in how people move, react and can be steered.
  • Native audio: this is DeepMind's first real-time world model that generates sound itself, from wind and waves to interactive dialogue where characters react to you and to events in the world.
  • Complex, multi-step events: Gemini 3.8 Flash acts as a guiding agent in the loop. It watches the scene, splits a goal into steps, checks incoming frames to verify each step worked, and changes its plan when the world shifts or you intervene.

Why I find this beautiful

That last point is the part I keep coming back to. It's an agent supervising a simulation, frame by frame, and correcting course. DeepMind also shows Gemini 4 Argon breaking a task into steps that are fed to Genie 4 as text captions, in a demo of Apptronik's Apollo humanoid playing checkers. If you care about training robots and agents, that's the real promise: safe, endlessly varied practice worlds before anything touches the physical world.

DeepMind calls the longer-term direction "hybrid world models": the structured reasoning of language and code models combined with pixel-level simulation, where Gemini plans the narrative and Genie renders how the world looks, sounds and responds.

The tester stories are lovely too. Filmmakers use it as a "draft mode" to run several takes of a scene live, photojournalists explore turning a single archival photo into a grounded simulation, and game developers go from an idea to an explorable prototype before writing engine code. I can also imagine it in a classroom: walking through a historical place, or watching how a flash flood unfolds, instead of only reading about it.

The honest part

DeepMind is clear that this is early research. Coherence holds over short sessions, but details and object permanence can drift in longer runs, after many prompt edits or while navigating. Complex physical interactions, audio accuracy, legible text and long-term consistency are still open problems. Access is a research preview for trusted testers for now, coming soon to Google AI Ultra subscribers in Project Genie and to early access partners in the Gemini API and Google AI Studio. All generated video and audio carries SynthID watermarking, and there is no open model or paper to inspect yet.

Sources

We used to dream about stepping inside a picture. Now the picture is starting to listen, respond and remember where things are, at least for a while.

Reacties

Een reactie posten

Populaire posts van deze blog

Meta just shared the full recipe for a 30B agent model you can run at home 🧠

When Meta released Muse Glimmer in August, the pitch was simple: an open-weight, Apache 2.0 model built for AI agents that run on your own Mac or PC. What was missing was the "how". On 19 October, Meta Superintelligence Labs published the full technical report, and for anyone who cares about distillation and local agents, it is one of the most useful reads in a while. What the model is Muse Glimmer is a dense model of about 29.6 billion parameters, including a 1.8B perception encoder, so it reads text and images and writes text. It has a 128K+ context window and was trained on data from more than a hundred languages. The design goal was strict: everything has to fit in 32 GB of GPU memory, and a smaller 4-bit variant targets 24 GB cards. The parts I found most interesting Dense beat mixture-of-experts here. That surprised me. The team trained scaling ladders for a dense model and two MoE designs with the same memory footprint. Under a fixed memory ceiling and their compu...
Meer lezen

AI that can actually "see" a crystal 🔬

Most of the solid stuff around us, from battery materials to chips, is made of crystals: atoms stacked in neat, repeating 3D patterns. Large language models are great at reading papers about those materials, but they have never really looked at the structures themselves. A team at MIT just showed a clever way around that, and the trick is one I keep coming back to: good synthetic data. What they built The model is called CLaC, short for Contrastive Language-Crystal model, and it is described in a paper in Nature Communications by Yang Jeong Park, Mayank Kumaran, Chia-Wei Hsu, Elsa Olivetti and Ju Li. MIT News covered it on 18 October 2026. If you know CLIP, the model that taught image generators to connect pictures and words, you already get the idea. CLaC has two branches: a graph neural network that reads the 3D crystal structure, and a text encoder that reads a description. Contrastive training pulls matching structure-text pairs together in one shared space and pushes mismatche...
Meer lezen

A 35B model on a Mac mini, using just 2.9 GiB of memory 🚀

There's something special about running a capable language model on hardware you own: no API bill, no data leaving the house, just a small box on your desk doing the thinking. A paper that's currently #2 on Hugging Face's trending list takes that idea further than I expected: it runs a 35-billion-parameter model on a Mac mini at about 20 tokens per second while using under 3 GiB of memory. The trick: leave the experts on the SSD The paper is called The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction , and the system is named Edge0 . It builds on a Mixture-of-Experts (MoE) model, Qwen3.6-35B-A3B. In an MoE, each token only uses a few "experts" per layer (here 4 out of 256 in each of 40 layers), so most of the weights sit idle at any given moment. Edge0 takes that literally. The expert weights (quantized to int4) stay on the SSD and are streamed in only when needed. The clever part is a small prerouter : a tiny trained he...
Meer lezen