Dataset: Rocket Science Dataset (1,000 hours of 4-player Rocket League gameplay)
Imagine four players in a fast-paced game of car soccer. Now take away the physics engine, the renderer, and every line of code that makes the ball bounce. Replace all of it with a single neural network that has ingested 10,000 hours of recorded gameplay. You feed it a gamepad input. It predicts the next frame. Multiply that by four players, keep them all in sync, and you have MIRA, a multiplayer interactive world model that can run a game of Rocket League forever without a physics engine. Kyutai and General Intuition, in collaboration with Epic Games, built MIRA to show that interactive world models trained on pixels and actions can stay stable over long rollouts, a problem that has haunted causal video generation for years. The model has 5 billion parameters and runs four simultaneous player views at 20 frames per second on a single Nvidia B200 GPU. The team also released a 1,000-hour open dataset called Rocket Science so others can reproduce the work or build on it.
What exactly is MIRA and how was it trained?
MIRA is a diffusion transformer that learns to simulate a video game. It does not run Rocket League. It watches recordings of Rocket League being played and learns to predict what comes next, conditioned on what players are pressing on their controllers. The training data is entirely synthetic: roughly 10,000 match-hours of 2v2 Rocket League, generated by self-play between bots. No human gameplay was used. Every car in every match is driven by Nexto, the highest-skilled publicly available Rocket League bot, which emits three-valued axis commands for steer, throttle, yaw, pitch, and roll, plus binary jump, boost, and handbrake inputs at 15 times per second. The model itself never sees the underlying physics state. It only ever sees pixels and actions. The physics state, logged through BakkesMod, the community modding framework that exposes Rocket League's internal state, is reserved strictly for evaluation.
The architecture pairs a 5B-parameter diffusion transformer with a 600-million-parameter video representation codec. The codec compresses each frame into a compact latent representation. The world model then learns to predict future latents from past latents and player actions. For training, per-player video is re-encoded to a standardized 288p at 20 fps and tiled into fixed-length 4-second chunks. At inference time, the model unrolls autoregressively, feeding generated latents back into the transformer to produce more frames.
The most notable architectural claim is stability. Causal video models typically diverge over long rollouts, accumulating errors until the output degrades. Most researchers lean on dedicated tricks and multi-stage training to avoid this. MIRA gets stability from a relatively simple recipe. The team trains using diffusion forcing in the latent space, and during training the model sees noised versions of past frames so it learns to deal with uncertainty. This lets them stabilize rollouts by intentionally adding noise to past frames at inference time: the model learns not to trust the past too much. The result is a model that, as far as the team can tell, can run infinitely without diverging.
Does the simulation actually hold up?
By the team's own account, the model simulates many aspects of Rocket League. Cars move, they can kick the ball, you can score goals. The four car views stay in sync across four screens. The model tracks the boost meter and prevents boosting when the tank is empty. Demolitions work. It even displays event messages like "Shot on goal" or "Save." These are emergent behaviors from a model that has no physics engine, no rendering engine, and no explicit 3D representation. It is just videos and actions crammed into a transformer.
The chart above shows the team's qualitative breakdown of which game mechanics MIRA reproduces well versus where it breaks down. Core mechanics like car movement, ball interaction, and the boost system are reproduced with high fidelity. Replays, however, are described as "funky." The model sometimes produces out-of-distribution behaviors when action sequences drift far from the training data, though it can sometimes recover on its own. The team notes that the model's ability to keep going forever without hard resets is a strong outcome, but they caution that the narrow visual domain of Rocket League likely makes this easier than it would be on messy real-world video. Stability is not a given even in this controlled setting; it only happens with their video representation codec, while their earlier semantic distillation codecs would diverge faster and never recover.
Why does a Rocket League world model matter for builders?
If you are building agents, robots, or simulation environments, MIRA demonstrates a recipe for stable, interactive, multi-agent world modeling using purely learned, action-conditioned video prediction. That has direct implications for anyone working on physical AI, where real-world data is messier and scarcer than game footage. The team frames Rocket League as a stepping stone: before you can make world models work on real-world data, you need to show they work in a controlled setting.
For builders, several things stand out:
- Data strategy. MIRA was trained on 10,000 hours of entirely bot-generated gameplay. This is a synthetic data pipeline that produces synchronized, action-conditioned video with no need for human annotation. If you are building a world model for a simulated environment, this is a blueprint for generating your own training data through self-play.
- Codec design. The single biggest leap in modeling performance came not from scaling the transformer but from getting the video representation codec right. The codec compresses each frame into a compact latent, and the world model predicts future latents. If you are working on video prediction, the lesson is that your representation space matters more than your model size.
- Long-rollout stability. The team achieved infinite rollout stability without multi-stage training or dedicated anti-drift tricks. The recipe is diffusion forcing in latent space plus noise injection during training so the model learns to handle uncertainty in its own past predictions. This is directly relevant to anyone building agents that need to plan over long horizons or robots that need to maintain a world model over extended operation. If you are working on robot learning pipelines, LeRobot's DAgger correction loop tackles a similar problem from the opposite direction: turning robot failures into training data to close the gap between policy and reality. MIRA's approach is to make the world model itself robust to drift.
- Open resources. The 1,000-hour Rocket Science dataset at 720p includes paired action streams and physics states. The training and inference code is on GitHub. If you want to experiment with action-conditioned world models, you can start today without recording your own data.
The compute footprint is also worth noting. Running four player views at 20 fps on a single B200 is real-time but not trivial. This is not something you can deploy on a consumer GPU. For a team building interactive simulations, the B200 requirement sets a floor on infrastructure costs. Scaling to more complex environments, higher resolutions, or more players will push that floor higher.
What are the limits and what should you watch next?
The team is transparent about the limits. Rocket League is a visually narrow, highly structured environment. The stability results may not generalize to messier domains. Using a single bot (Nexto) for all four cars removes any dependence on human players but also removes behavioral diversity. The model's behavior is bounded by what Nexto does, which means it will not surprise you with novel strategies. The replay system is unreliable. And the team explicitly states that the world model technology is not being used to develop Rocket League.
For builders deciding where to place bets, the questions to watch are:
- Does the codec-first approach transfer to real-world video? The team credits their video representation codec with both the performance leap and the infinite stability. If this holds on messier data, it changes the economics of world model training. You would invest in representation learning before scaling transformers.
- Can multi-agent world models scale beyond four players? MIRA handles four synchronous views. Many real-world scenarios, from traffic simulation to multi-robot coordination, involve more agents and more complex interactions. The compute cost per player view is a real constraint.
- Does action-conditioned video prediction become a viable alternative to physics engines for training agents? If you can train agents inside a learned world model instead of a hand-coded physics simulator, you remove a major engineering bottleneck. The catch is that the world model needs to be accurate enough that policies trained inside it transfer to reality. MIRA is not there yet for real-world robotics, but it is a proof of concept that the approach can work in a controlled domain.
The broader research context is moving fast. The MIRA team's release of code, data, and a live demo is a strong contribution to a field that has been bottlenecked by instability and closed implementations. If the codec-first recipe transfers, the next wave of world model papers will look very different from the last one.
The real game is not Rocket League
MIRA simulates a video game without a physics engine. That is a neat trick. The reason it matters is that the same recipe, a stable video representation codec plus diffusion forcing with noise injection, might work on noisier data. If it does, the bottleneck for training physical AI agents shifts from hand-engineering simulators to generating synthetic data and learning the world model itself. That is a much better bottleneck to be stuck at, because it is one you can scale with compute. For now, MIRA is a proof of concept in a controlled domain. Treat it as a signal of where world model research is heading, not as a drop-in replacement for your physics engine.
