By Mengyao, from QbitAI
QbitAI | WeChat Official Account QbitAI
Wow,ByteDancehas this time turned AI generation into an open world?!
Look — with just one sentence of input, the AI conjures up a room from the perspective of a tiny miniature figure out of thin air~
Right after that, with some frantic keyboard and mouse action, we can hop on a paper airplane andfly freelythrough this cyber world, going wherever we want:
And that's not all — in this AI-generated world, even the things in front of you can be created on the spot!
Want a spoon? Just conjure one out of thin air, and even…
we can alsointeractwith the spoon, turning everyday tableware into a large-scale rock climbing scene —
Of course, this world can do far more than that.
Thanks to themagic doorfeature, two completely unrelated scenes can be directly connected.
One second you're indoors, and the next second you push open a door and step into outer space?!
This magical scene appeared on the just-concluded stage ofDouyin Group's '2026 Wonderful Night of Wonder'tech show, with Tim from Film Hurricane as the demonstrator.
What's even more noteworthy is that this series of operations, worthy of being called cyber creation, is powered by theworld modelthatByteDancehas been quietly developing. (Shocked.jpg)
Although the model has not yet been officially released, judging from the live demonstration, it has already shown capabilities in3D scene generation, free exploration, real-time interaction, and object editing,Even the ability tofreely connectbetween different spaces~
This time, ByteDance used a live on-stage demonstration to give its own world model an unexpected"debut", truly showing off its muscles a bit early.
It's quite touching to think about—over the past few years, from Fei-Fei Li's World Labs, to Google DeepMind's Genie, to NVIDIA's Cosmos.
A host of tech giants have stepped into the arena one after another, trying to get AI to further understand spatial structure, simulate real-world laws, and even predict how environments change in response to human actions.
Now, AI is conjuring the worlds of science fiction movies into reality.
From drawing a world to bringing an entire world to "life"
I guess the earliest group of people playing with AIGC years ago probably never imagined—
that one day AI-generated videos could become a cyber 3D world you could walk into, interact with, and flexibly edit.
I counted a little—from the beginning of this year until now, I've tried roughly six or seven world model products。。。
The level of shock was no less than watching Interstellar one day and then finding yourself able to walk into the screen the next, casually moving a black hole into your own living room.
So, how did AI make such a smooth "server switch" from image and video generation to world building?!
This story has to start with a technical shortcoming of traditional multimodal generation. (doge)
Today, world models have branched into multiple technical routes, from abstract state prediction and physical causal modeling to generative 3D worlds, each with its own focus.
What we'll mainly discuss here is the route related to multimodal generation and spatial intelligence:
having AI learn the spatial structure and laws of change of the real world, and then generate, reconstruct, and simulate a 3D world that can be continuously explored and interacted with in real time.
It needs to be emphasized that there are two key terms here:spatial structureandlaws of change。
And these two capabilities happen to address a fundamental problem that visual generation technology has not yet fully solved over the past few years.
△AI generated
In 2024, OpenAI released Sora, and its stunning generated videos left the entire internet reeling alongside Sam Altman in astonishment.
But compared with those screen-flooding videos from back then, the technical report OpenAI released at the same time is actually more worth savoring today——
"Video Generation Models as World Simulators", literally translated, meansVideo Generation Models as World Simulators。
At that time, OpenAI discovered that as video generation models scaled up, the models began to exhibit a degree of three-dimensional consistency and object permanence, and could even simulate simple environmental interactions.
But the awkward thing is that a model capable of generating realistic world imagery may still fail to handle even the most basic "physical relationships."
For example, Sora can generate a scene of a person running on a treadmill, where the figure, lighting, and camera work all look convincingly real.
But on closer inspection, the relationship between the characters' movements and the treadmill's motion may not conform to real-world physical logic — the visuals look very realistic, yet the laws of physics seem a bit random??
△Image source: OpenAI official website
Behind this actually lies a fairly fundamental problem of traditional visual generation technology.
Over the past few years, whether diffusion models or Transformer-based generative architectures, mainstream image and video models have mainly relied on learning from the massive amount of visual data inStatistical laws, to generate images that match the input conditions.
This set of methods has been quite successful, even able to implicitly learn part of the three-dimensional geometry, object relationships, and physical laws.
ButVisual spatiotemporal continuity does not necessarily correspond to a stable world state that can be intervened upon by external actions.——
For example, keeping a cup's appearance consistent throughout a video has become relatively easy for today's generative models.
But when it is picked up, put down, or even pushed off the edge of a table, the model must also continuously track spatial position, object relationships, and state changes, and predict physical outcomes such as collisions and falls.
In other words, there is still the problem of world-state modeling standing between generating a video that conforms to physical intuition and reliably predicting how an action changes the environment.
And this is precisely where world models excel.
In June this year, Fei-Fei Li's team published an article titled "A Functional Taxonomy of World Models."
From a functional perspective, the article divides world models into three types, which also gives a clearer explanation of what "real capabilities" world models actually possess—
- Renderer: in charge of the worldIt seemswhat it looks like, outputting visually rendered images for humans, with the core goal of image realism and visual expressiveness.
- Simulator: responsible for the world — what itactuallylooks like, maintaining scene states that conform to geometry and physical causality.
- Planner: responsible for how the agent shouldact next..., outputting reasonable actions based on the current state of the world and given goals.
What makes this division worth savoring is that it precisely explains why, even though AI can already generate videos that are indistinguishable from reality, we still go to such great lengths to research world models.
From generating photorealistic visual imagery, to maintaining environmental states consistent with the laws of physics, to supporting autonomous decision-making and action by agents—
the spatial representation, dynamic prediction, and action planning capabilities involved behind all of this are precisely the technical challenges that traditional multimodal generative models have not yet adequately solved.
And now, these world model concepts in the papers that once seemed quite out of reach have begun to be gradually "built" by AI.
△Image source: the article "A Functional Taxonomy of World Models"
Take, for example,the Cyber Wonders Night 2026tech show by ByteDance Group, and the cyber creation operations demonstrated there based on ByteDance's AI models.
Generating imagery from a single sentence is, by today's standards, nothing remarkable at all.
But when a person can actually walk into this AI-generated scene and move through it freely, what the model needs to handle has extended from generating the single image before one's eyes to maintaining the spatial structure of the entire scene.
No matter which direction the person moves or from which angle they observe, the terrain, objects, and their relative positions in the scene all need to maintain sufficient geometric consistency.
What this reflects is precisely the world model'sspatial representation capability— the foundation for making an AI-generated scene truly stand up to repeated exploration.
Want a spoon? The AI can generate one out of thin air, and you can even crawl along the spoon, enabling real-time action interaction.
Behind these seemingly simple operations lies the world model'sreal-time generation, environment state modeling, and dynamic prediction capabilities——
. The model must update the objects and spatial relationships in the scene in real time according to the user's actions, continuously predict the environmental changes triggered by those actions, and maintain consistency between previous and subsequent states.
Even space itself can be modified on the fly — draw a "magic door" and you can directly connect a real room to outer space.
At this point, the world model's capabilities extend another layer further —3D scene reconstruction, controllable spatial editing, and cross-scene continuity modeling.
The model can dynamically reorganize the geometric relationships between different scenes while preserving the spatial structure of the original scene, supporting continuous cross-scene exploration and real-time interaction:
From maintaining spatial structure, to responding to real-time interaction, to supporting scene editing and cross-space recombination, the capabilities of ByteDance's AI model all point to the core characteristic of world models —
what AI generates is no longer just content; it generates a digital world that can persist, continuously change, and continuously interact.
Over the past few years, large language models have continually expanded AI's ability to process language and knowledge, while multimodal models have extended generative capabilities to images, sound, and video.
Now, with the development of world models, AI has begun to further explore spatial structure, environmental states, and their laws of evolution.
The technological development of generative AI is also shifting from how realistic the visuals look to whether the generated environment can persist, respond to interaction, and even simulate the workings of the real world.
World models let AI develop "real skills" in virtual worlds
Of course, if world models only let humans explore the cyber world, that would somewhat underuse their potential.
Viewed in the broader industrial landscape, they hold another layer of value with even more imagination —
letting machines train at scale and repeatedly trial-and-error in virtual worlds, then transferring the acquired abilities to reality.
After all, the development of physical AI such as robots and autonomous driving has long been constrained by real-world limitations like the high cost of collecting real data, long trial-and-error cycles, and the difficulty of reproducing dangerous or extreme scenarios.
World models offer a new path: transforming real-world environments into generative, controllable, and repeatedly verifiable virtual training grounds.
From generating scarce training data to simulating complex environments and validating action strategies, some training processes that originally depended on real environments are expected to be moved into the virtual world.
This also allows world models to extend from content generation into the training and validation stages of physical AI, becoming an important technological exploration direction in fields like robotics and autonomous driving.
△AI generated
First, let's talk about "training data."
For physical AI systems like autonomous driving, the more critical the extreme scenario, the scarcer the training data tends to be.
For example, low-frequency, high-risk scenarios like sudden accidents or extreme weather may occur only a few times over a huge number of miles driven — and world models are exactly suited to generate and repeatedly simulate these scenarios on demand.
At Douyin Group's "2026 Night of Wonderful Wonder" technology show, they demonstrated just such an autonomous driving demo —
an ambulance suddenly approaching, a pedestrian crossing the road, a traffic officer directing on the spot.
All kinds of complex road conditions were brought into AI-generated virtual roads, where vehicles could simulate driving, handle emergencies, and even analyze driving performance through multi-camera replays.
The long-tail scenarios that once required extensive real-world road testing to accumulate can now be mass-generated through world models and used for the repeated training and validation of autonomous driving systems.
Beyond training data, the industrial value of world models is also reflected in another key aspect—improving the training and validation efficiency of physical AI.
As everyone knows, real robots have to perform actions and wait for feedback during each training run, and after failures they may require manual reset and carry the risk of hardware damage.
But in a three-dimensional world, this time-consuming trial-and-error process has the opportunity to become repeatable, parallelizable simulated experiments.
The household robot demonstration at ByteDance's 2026 Wonderful Night of Wonder technology show reflected a similar approach.
In an AI-generated virtual home environment, a robot can practice household operations such as grasping, carrying, and placing objects, and can also adjust its action strategies and try to complete tasks while facing constantly changing object positions and spatial layouts.
Even if an operation fails, it can quickly reproduce the problem and adjust strategies in the virtual environment for repeated training and verification.
Compared with operating over and over in reality and manually resetting, world models give robots the opportunity to try and err in parallel across different scenarios, mastering the ability to cope with real environments at lower cost and faster speed.～
Looking at the industry as a whole, tech giants have already begun exploring the industrial application of world models along different directions.
In March (3) of this year,NVIDIAcombined the Cosmos world model with the Isaac simulation platform, allowing robots to generate data and train repeatedly in virtual environments.
This industrial simulation solution is expected to greatly reduce the cost of robots from training to actual deployment.
△Image source: NVIDIA official website
What is even more worth mentioning is that this wave of enthusiasm has even drawn upstream chip giants in.
Just in the past September (9),AMD announced that it would spend 8.2 billion US dollars to acquire Fei-Fei Li's World Labsaiming to lay out the computing power demand for world models in advance and pave the way for next-generation chip development.
From robots practicing in virtual settings, to industrial simulation, and then to chip giants personally entering the field.
The industrial competition around world models is extending along the industrial chain, from model capabilities and practical applications, gradually deepening into underlying computing power architecture.
In the global race of world models, Chinese players' technical accumulation is starting to pay off
World models are truly hot, but behind the boom, quite an interesting industrial phenomenon is gradually emerging.
Manufacturers at home and abroad are all exploring how to build three-dimensional worlds that are spatially stable and capable of real-time interaction, but their technical accumulations, R&D focuses and application paths differ considerably——
The technical goals are gradually converging, yet each has its own way of realizing them.
Take overseas examples: Google DeepMind's Genie and Fei-Fei Li's World Labs are both exploring general-purpose world simulation capabilities, focusing on spatial structure, environmental dynamics modeling and interactive scene generation.
NVIDIA's Cosmos is closer to the practical needs of physical AI, combining world models with robot simulation and synthetic data generation, attempting to connect virtual training with real-world applications.
These several routes each have their own emphasis, but all ultimately point to one goal:making AI-generated virtual environments gradually possess the ability to simulate the real world.
Now let's turn our attention to China.
One advantage worth noting is that, compared with the overseas exploration path emphasizing general-purpose world simulation, domestic manufacturers' technical capabilities accumulated over the past few yearsin the fields of image and video generation provide another quite competitive evolution path for world models.
In July (7) of this yearSeedance 2.5extended the duration of a single video generation to 30 seconds, and also strengthened multimodal reference control, camera movement and continuous narrative capabilities.
The image generation model released in the same period,Seedream 5.0 Pro, further strengthens spatial position understanding, structural consistency, and interactive precision editing.
A user can circle a location on the image, or even casually scribble a few strokes, and the model will modify the specified area—truly draw where you want and change it there.
From consistency of spatial structure, to dynamic continuity over long sequences, to precise response to external operations, these are precisely the unavoidable technical problems in building an interactive three-dimensional world.
The technologies that domestic vendors have accumulated over the past few years in visual generation and controllable editing are now beginning to show the potential to extend toward general world models.
In 1895, the Lumière brothers brought a train pulling into a station to the silver screen.
People at that time could hardly have imagined that this new invention would, over the following century and more, completely change the way humanity records and presents the world.
More than a century later, with the successive emergence of computer graphics and virtual reality technology, humanity began attempting to move from watching the world on a screen toward immersive experience and exploration of digital worlds.
Today, the emergence of world models has brought a new turning point to this technological evolution spanning more than a hundred years—
machines are beginning to learn the spatial structures and operating laws of the real world, attempting to generate virtual environments that can be continuously explored, interacted with in real time, and even used to simulate reality.
More than a hundred years ago, humanity learned to record the world with moving images.
More than a hundred years later, we are beginning to try to make moving images become a world that one can personally enter and even change reality.
More than a hundred years later, this time we are finally beginning to try to step into the images we ourselves created.