Yishui, from Aofei Temple
QbitAI | Public Account QbitAI
Play games while upgrading yourself?
Wow, Huawei has actually made AI learn “get stronger while playing” right away?!
A recent study by Huawei Noah’s Ark Lab, in collaboration with teams from UCL and others, has developed an AI intelligence system called Memento 3.
All 25 games and 183 levels from the ARC-AGI-3 public collection have been completed.
It wasn’t just relying on blind trial and error that got through.
Humans need 17135 operations to complete the same set of levels.
And Memento 3 was used only 7518 times. That’s about 56% less than humans, and it finally achieved a score of 100 in RHAE (Relative Human Action Efficiency).
Even more absurdly, this system uses Claude Opus 5 as the base model, without updating the model weights at all, and without any fine-tuning for the game.
It just lets him play by himself, figure out the rules, and write the code. If he makes a mistake, he will reflect on it and make changes.
Then… I finished the level with a perfect score.
For comparison, Claude Opus 5 has an average RHAE of only about 40.7 points under the official standard testing framework of the ARC Prize.
With the same mold, but using a smart agent system, the score actually soared by 59.3 points directly.
Not over yet.
This method can not only be used for puzzle games, but the research team also made it play the classic Atari game Pong.
As a result, the AI figured out its own control strategy, and then without having to request reasoning from the large model, it won three consecutive games with a score of 21:0 against the opponent.
No, none of the large models are involved in the subsequent operations. How can the game be played so well?!
The key lies in the title of this study:
Model-Based Recursive Self-Improvement, Recursive Self-Enhancement Based on World Models.
Huawei AI directly passes 25 games, with 56% fewer operations than humans
Let’s first talk about what these 25 games are all about.
They come from ARC-AGI-3, launched by the ARC Prize. Those who are interested in AI evaluation should be familiar with the ARC series.
But there is a very obvious change in this generation:
It’s no longer just about giving questions and solving them; this time, we’re actually putting AI into a foreign game world.
Rules are something that AI needs to guess on its own; targets are things that AI needs to find on its own; and what the operation will produce is something AI needs to test on its own.
AI: It’s all depending on me~
These 25 public games contain a total of 183 levels, and most of the graphics are made up of simple pixels, blocks, colors, and shapes.
At first glance, it even looks like a collection of old games from decades ago.
But don’t be fooled by this simple painting style.
For AI, how to operate isn’t the most difficult part; the real trouble is not knowing at all how this world works.
What happens when you step on a special floor? A door won’t open, is it because the colors don’t match, or is the shape mismatched?
What’s more troublesome is that the subsequent levels might introduce new rules, overturning all the experience gained before.
Don’t talk about AI; human players can also get stuck.
The test data from the ARC Prize shows that in the puzzle game r11l that requires controlling amoeba movement and color changes, all 10 participants completed the game; whereas in another puzzle game tr87 that tests symbol decryption ability, 12 people tried, and only 6 succeeded.
Before Huawei AI, various research teams were already competing fiercely:
Princeton University's Continual Harness has only cleared 3 games and scored 20.5 points.
The DreamTeam developed by NVIDIA researchers is a bit better; it completed 6 levels with a score of 38.1.
At a higher level, OPINE-World developed by independent researcher David Courtis, and NOOA by NVIDIA’s team, scored 78.4 points and 85.1 points respectively.
The most impressive one is baseline1 by independent developer Sergey Rodionov. It not only connects all 25 games, but also got 99 points, just 1 point short of a full score.
But this time, the Huawei team directly made up that last 1 point too.
Memento 3 not only cleared all 25 games and 183 levels with a score of 100, but also reduced the total number of operations to 7518, which is 829 less than baseline 1’s 8347.
Not only did I pass the level, but I also did it with fewer steps.
How was that done??
No need to train large models, AI can achieve “leveling up” by writing its own strategies.
I flipped through the paper, and the core method of Memento 3 is called Code as Model.
It sounds a bit abstract, but basically it’s about having the AI prepare two things for itself during the game.
A constantly updated game guide, along with a set of code that simulates game operation.
- Rulebook: A Markdown document that records the patterns discovered by AI in games, such as which actions trigger certain effects, how to complete the game, and what rules are still unclear.
- Executable World Model: AI will directly convert the rules it summarizes into executable Python code, which is like creating a simplified game simulator.
With these two things, AI can first simulate different paths in its own built simulator, and then decide what to do next.
But here comes the problem: what if the rules of the game that we start understanding are wrong?
Don’t worry, Memento 3 not only writes guides by itself, but if it finds a mistake, it can also reverse it and start over.
An example is shown in the paper.
In a puzzle game, AI encounters a special floor. Initially, it seems that when stepping on it, the pattern carried by the character will be mirrored according to the direction of entry.
It was only after actual testing that it was discovered that the pattern is not flipped, but rotated 90 degrees clockwise, and this has nothing to do with the direction of entry.
So, the AI modified the rule manual, and the corresponding Python code was also updated accordingly.
However, after the change, it still needs to be verified.
The system will re-simulate the previous operation, comparing each cell of the 64×64 game image to confirm that the code can accurately reproduce the real results.
It doesn’t match, so keep editing.
Just like that, while exploring, AI corrects errors, and the rules become more accurate with each change.
And the experience learned earlier can also be applied in subsequent levels.
For example, in the symbol decryption game tr87, the AI initially couldn’t distinguish whether the orientation of the symbols affects their meaning. Later, it was found that symbols in different directions might represent the same letter.
As new levels appear, it also overturns the previous understanding of the cursor position.
Throughout the entire game, the same rule manual was iterated 11 times.
To facilitate backtracking, the research team even introduced Git version control.
Every change to the rules and code is recorded. If you notice a mistake, you can view previous versions and restore them directly when needed.
Wow, even while playing a game, you can take care of maintaining the code repository.
This is also what the research team refers to as Recursive Self-Improvement (RSI).
The parameters of the large model don’t need to be changed; AI can become more proficient by continuously accumulating experience, modifying rules, and updating code.
Moreover, this method is indeed supported by data.
The research team conducted control experiments in three games:
When keeping the rules manual, the AI used a total of 617 operations; without the rules manual and only keeping the code world model, it required 677 operations.
Equivalent to having this guide, with the number of operations reduced by about 9%, and the number of execution rounds for the AI also reduced by about 18%.
But the research team is not satisfied with this.
Since a world model can help AI avoid unnecessary detours, what about using two at the same time?
So, they let the AI maintain two sets of world models simultaneously, sharing game operation records, but each applying different rule assumptions.
As a result, in the game wa30, it took 899 operations to complete the single-world model, while switching to the dual-world model reduced it directly to 597 operations.
The number of operations has decreased by approximately 34%.
Of course, this improvement has been verified only on wa30, and it cannot be directly applied to other games yet.
But at least in these experiments, Memento 3 proved one thing:
Without needing to retrain the large model, AI can also improve by continuously correcting its understanding of the world, and it gets stronger with each use.
The game ends at 21:0, and the AI doesn’t even need to use the large model anymore
Is this method only used for puzzle games?
The research team decided to add some more difficulty, and directly challenged Memento 3 with the classic arcade game Atari Pong.
Unlike the previous puzzle games, this time the ball keeps moving and bouncing. The AI not only needs to understand the rules of the game, but also has to determine the ball’s movement path in real time and control the paddle to catch the ball.
This time with the results, we actually discovered something new.
At first, AI also didn’t know the rules of the ball’s bounce, so it could only try and figure it out while playing.
It initially thought that the racket could be divided into four areas, and if the small ball hit different positions, it would bounce in the corresponding directions.
The result bounced back horizontally suddenly.
Eh? This doesn’t match the rules summarized before at all.
So, the AI re-analyzed the collision position, modified the bounce formula, and later encountered new trajectory deviations, so it continued to adjust, gradually forming a movement pattern that better matches the actual situation.
When the control strategy became mature, the research team simply let this program operate independently.
No longer calling Claude Opus 5, and no longer updating rules.
As a result, three different starts resulted in a 21:0 score.
Wow, first it was struggling to catch the ball, and then he just shut down the opponent completely?????
And note that, at this point, large models no longer need to take part personally.
It is only responsible for exploring rules and summarizing playing strategies in the early stage. Once the strategy is learned, it is directly passed to the code for execution.
For comparison, if no world model is used and only the large model decides how to act based on the current image, the final score is only -19, which is not much different from the random, random actions of -20.7.
In this comparison, the gap becomes evident.
Instead of having the large model rethink every step, let it first learn how to play, and then hand over its playing style to the program.
And the efficiency is also quite impressive.
The paper data shows that Memento 3 used approximately 9500 game frames to learn this control strategy.
In contrast, some traditional reinforcement learning methods often require hundreds of thousands or even hundreds of millions of training frames.
Of course, this doesn’t mean that Memento 3 has no costs.
After all, it uses Claude Opus 5 with the Extra-High reasoning strength enabled, and the processes of exploration, code generation, and verification all consume a lot of Tokens and computing power.
Moreover, currently the experiments mainly focus on game environments with relatively clear rules. Whether this method will still work in a more complex and variable real world remains to be verified.
However, this study points out a direction that deserves attention:
Recursive self-improvement (RSI) does not necessarily require changing the underlying weights of the model. By constructing a checkable, executable, and roll-backable external world model, a frozen general large model can also self-iterate and evolve rapidly in unfamiliar domains.
Paper address:
https://arxiv.org/abs/2610.11794
Reference link:
https://x.com/ScarletKc/status/2109169173540884935