By Yishui, from QbitAI
QbitAI | Official account QbitAI
Wait, the "gaming world model" that is famously compute-hungry has somehow been squeezed into a single RTX 5090?!
And it carries two very enticing labels:
domestically developed, open source。
Just now, Skywork AI, a subsidiary of Kunlun Wanwei, releasedMatrix-Game 4.0, reportedly"the world's first gaming world model that can run in real time on a single consumer-grade graphics card"。
Quickly flipping to the Demo — honestly, the 480p quality does still have room for improvement, but watch a few more seconds:
Hey, something's off!!
In typical gaming world model Demos of the past, the player presses a direction key and the model generates the next frame of movement. You can roam the world, scenes move, but whether there's actually a real game running underneath is hard to say.
But Matrix-Game 4.0 is different.
When the player takes a hit, the shield durability drops first; once the shield breaks, health starts draining; and when health hits zero, the game truly ends.
Video address: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
Wow, so it's not just the visuals moving — the game logic underneath is actually running too.
Even more outrageous, this whole setup isn't sitting on some luxurious compute cluster.
Just one RTX 5090.
Press a key and the screen responds in under 0.15 seconds; it runs at 60 frames per second; the whole system's peak VRAM usage is about 18.5GiB。
The gameplay can even be modified.
Let a Coding Agent tweak the game code, and an original time-trial instantly becomes a pursuit mission to intercept a target vehicle.
Video address: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
One card, carrying a game that AI renders in real time and that runs according to code rules.
How was this done?
Putting a Harness on the video model
This time, Matrix-Game 4.0 did not take the usual path.
The team's core judgment is:
Rather than letting a video model implicitly memorize the world, it's better to let a game engine explicitly maintain the world.
The route represented by Google's Genie was the first to let players step into AI-generated real-time worlds — wherever you go, the model generates the visuals as you go.
But a world you can walk around in doesn't mean a game you can actually play.
An occasional extra tree or a missing window in the visuals will at most break the immersion. But if NPC positions, player health, and win-or-lose outcomes of quests are also improvised by the model on the fly, what's left to play??
So what's more troublesome than visual glitches is actually rules running wild.
In the team's view, current-stage video generation is more like a form of "intersemiotic translation," responsible for converting text, actions, and scene information into continuous footage.
When it comes to "painting," video models are indeed good at it.
Yet the common approach in the past insisted that the model both remember the entire world, work out the game rules, and keep generating frames nonstop.
Good grief, one model juggling several jobs at once can hold up for a short while, but over time, you know how it goes...
△Image generated by AI
So with Matrix-Game 4.0, the team simply split up the work:
the game engine is responsible for running the game correctly, and the video model is responsible for painting the visuals well。
Once split this way, the world state is no longer hidden inside the pixels generated by the model, but is explicitly recorded in code.
Even if the visual generation occasionally deviates, it won't directly rewrite things like the player's health, item ownership, or quest progress.
The team summarizes this approach as"putting a Harness on the video model"：
1. The player first expresses an idea in natural language, and the Coding Agent is responsible for turning it into game code;
2. The game engine acts as the "referee," maintaining world state, enforcing game rules, and producing a structured sketch that won't fall apart;
3. The video model acts as the "painter," generating the final frames in real time according to the sketch.
To put it plainly: what should be deterministic goes to code, and what calls for creativity goes to AI.
As for how well this division of labor works, just look at the demo.
At first, the engine outputs a structural scene made of blocks, skeletons, and simple terrain。
which looks rather like an "unfinished rough build."
But where the characters are, where the items are, and who is moving, is already laid out crystal clear.
Once the video model takes over, castles, grass, textures, and lighting are filled in in real time — what was just a 3D rough model instantly becomes a complete game scene.
Video URL: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
Being able to paint it is one thing; the rules underneath are genuinely running too.
Take the shield as an example: after the player picks it up, the engine formally writes it into the game state. When hit, the shield's durability is reduced first, and only after the shield fails does the character's health drop; when health reaches zero, the game ends.
The video model is responsible for painting the process, while exactly how it is settled is entirely decided by code.
Video URL: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
Even more interesting, these rules can keep being modified.
Want to increase a weapon's fire rate? Just tell the Coding Agent in natural language.
Once the underlying code changes, the weapon's actual attack logic changes accordingly.
Video URL: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
A combat scenario ran the whole method through from start to finish:
the Coding Agent handles modifying, the engine handles computing, and the video model handles painting.
AI-generated worlds are also moving from "looking like a game" to "truly running by game rules."
How do you run the game on a single RTX 5090?
With the approach sorted out, the next level is the truly tough part:
Compute power.。
After all, having a video model render a game in real time doesn't sound like a job a single consumer-grade GPU could handle.
At almost the same time, NVIDIA's research team also releasedStreamRender-H3, likewise taking the route of "code runs the world, video model generates the visuals."
But the deployment configuration shown officially is8 GB200 GPUs. Each video chunk uses two-step denoising, and the diffusion rendering time per streaming generation step is 218ms.
And that's just the diffusion rendering stage. Counting video chunk collection, decoding, and playback, if one complete feedback loop is calculated as two video chunks, the end-to-end latency is about545ms。
By a rough comparison with this set of data, Matrix-Game 4.0 takes about 112ms from keypress to display, roughly the estimated end-to-end latency of StreamRender-H3'sone fifthof StreamRender-H3's estimated end-to-end latency; and the hardware configuration went from 8 GB200s,down to a single RTX 5090。
Of course, the two differ somewhat in resolution and how latency is measured, so they can't be ranked directly.
But from an engineering standpoint, a single consumer-grade GPU running at this speed is already impressive enough.
And this GPU didn't suddenly shoulder everything.
Starting from Matrix-Game 1.0, the team has been filling in the gaps for game world models:
1.0 first made the generated world understand player actions, 2.0 solved real-time performance, 3.0 added long-term memory, and 3.5 kept stabilizing space and characters.。
△ Image generated by AI
By 4.0, the team began facing another question:
When does this real-time generated world that can also remember count as a game?
The answer has already been revealed earlier: hand the rules over to code and the game engine.
But to make rules and visuals run simultaneously on a single RTX 5090, just redrawing the division of labor isn't enough.
The whole pipeline also needs to be lightened from an engineering perspective.。
First reduction: reduce what the model has to guess on its own.
The first step is to make the generation task simpler.
"Painting from thin air"?? NONONO—how is that ever easier than "polishing from a sketch."?
To teach the model this skill, the team created a batch of paired data using UE5:one side is structural frames, the other is the corresponding refined frames.。
The former ignores fancy stuff like materials and lighting, keeping only scene structure, object positions, and motion trajectories; the latter is the final effect to be generated.
With training like this, the game engine only needs to render the structural frame first, and the video model can then fill in materials, lighting, and environmental details accordingly.
But a sketch alone isn't enough。
For the same block, the model can't tell whether it's a shield, an NPC, or a barricade.
So the system also reads the visible objects and real-time state provided by the engine, dynamically assembling a Caption to tell the model what's in the scene and what's happening.
Video link: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
The structured scene takes care of conveying "where," while the dynamic Caption explains "what it is and what it's doing."。
Wow, the blueprint and the instruction manual are handed right over.
Now the video model just has to draw accordingly.
Second reduction: cutting down what the model has to remember on its own
Getting a single picture right isn't enough—the game has to keep running.
The problem is, the longer the video, the more historical frames there are.
Having the model go through the records from scratch at every generation is not only laborious, but frames that went slightly off track earlier may also be carried all the way down.
But wouldn't it be nice to just put the key frames right in front of it~
Matrix-Game 4.0 is a V2V model built on Wan2.2 5B Fun Control. At each generation, besides referencing the current structural frame, it also calls on a spatial memory reference image, a first frame for connection, and recent historical information in the cache。
Where things should be, what they look like, and how they connect are all hinted in place, so the probability of the picture drifting off naturally goes down.
Third cut: reduce what the model needs to compute
The final cut strikes directly at the computation itself.
A diffusion model generates images a bit like repeatedly editing from a screen full of noise—one pass isn't enough, so edit a few more times until the picture takes shape.
A game can't wait that long.
The team used few-step autoregressive distillation to compress this process to only 1 or 2 editing steps, generating only four frames at a time。
Fast computation isn't enough—the GPU waiting for the CPU to issue instructions and historical data being shuttled back and forth also slow things down.
So CUDA Graph, FP8 fused operators, a KV ring cache, and a lightweight codec were all brought in:
Compute once less whenever possible, and don't shuttle data back and forth unless necessary.。
In the end, the text encoder, DiT, and VAE all reside permanently on a single RTX 5090.
With single-step denoising, the video model's steady-state generation speed reaches 83 FPS, and about 57 FPS with two-step denoising.
The actual game runs at 60 FPS, and less than 0.15 seconds after a player presses a key, the picture responds.
Honestly, seeing this, I feel the truly impressive thing about 60 frames on a single card is no longer the number "60" itself.
Game world models that previously could only run on compute clusters can now be moved directly onto a developer's desktop with just one RTX 5090.
Video link: https://mp.weixin.qq.com/s/dOIA_LEFwE_A3tUTwQwEyA
Even more remarkable, just as the hardware threshold was cut down, Kunlun Tech followed up with another move:
Open-sourcing。
Chinese AI pushes open source into game world models
The act of "open-sourcing" carries more weight in game world models than one might imagine.
Every time I see a world model release, I get a familiar feeling:
The demo is indeed stunning.
Snow mountains, cities, deserts generated at will, the character walking forward when the player presses the arrow keys, videos flooding social platforms everywhere.
And then?
And then nothing.。
The model weights are unavailable, the inference code invisible, and developers—let alone downloading it locally and trying changing a rule or swapping a scene—can do none of that.
A quick scan of the industry shows that most players open up products, APIs, or research demos,while those who truly hand the model and code to developers remain a minority.。
So this open-sourcing of Matrix-Game 4.0 is all about using it without heartburn (manual doge).
Key point: this isn't a sudden change of heart today—the company has been committed to open source ever since Matrix-Game 1.0.
Looking back, one very important reason for laying the technology out so early seems to be:
The world model track is getting more and more crowded, yet the technical routes are far from converging.
△Image generated by AI
Some are still betting on video generation, some focus on 3D space, and others are bringing game engines back into the mix.
They all go by the name of world models, but the underlying solutions have long since been diverging.
The only thing gradually becoming clear is that being able to generate video alone is no longer enough—the world has to be enterable and interactive, and ideally able to keep running over the long term.
As for exactly how to take that step, everyone is still frantically experimenting.
And precisely because there is no standard answer, open source has a role to play。
Once the model is handed over to developers, whether it can be reproduced, whether it breaks down in a different scenario, and what new tricks can be squeezed out of it—just get hands-on and roll through it.
Whether it works well, the community will naturally give an answer.
And this playbook—we couldn't be more familiar with it.
Large language models went open source, multimodal models continued going open source, and now Chinese open-source forces, with Kunlun Tech as a representative, have rolled right along into game world models.。
Well well, a new track just heated up, and the familiar recipe is already being served.
Of course, Matrix-Game 4.0 is still just a beginning for now.
The 480p image quality can hardly be called refined, and there is still a way to go toward an AI world that is more stable, runs longer, and lets more people play simultaneously.
But the path isn't set yet, so of course the answers shouldn't be written behind closed doors by just a few companies.
After Google's Genie, the solution offered this time by domestic players is:
The code keeps the world in order, and the AI draws the world。
A single RTX 5090 can run it, and both the model and the code were released together.
As for what developers will end up making of it?
All we can say is:
Matrix-Game 4.0 is a big stage—if you like it, come on over.
[Big Stage Portal]:
https://github.com/SkyworkAI/Matrix-Game
https://huggingface.co/Skywork/Matrix-Game-4.0
https://matrix-game-v4-0.github.io/page/