By Jay, from QbitAI
QbitAI | Official account: QbitAI
Google's new RSI achievement is now open source.
It's calledHarness², letting large models modify their own plugin code, system prompts, toolchains, and skill libraries while running.
Yes, RSI at the Harness layer.
And the results are impressive.
GLM-5 equipped with this RSI-specialized harness soared by11.5 points——
on the office benchmark WorkBuddy-Bench, instantly overtaking the native Gemini 3.5 Flash.
In short, it's an outrageously powerful plugin — put it on any model and the benchmark scores take off.
Currently, the project has been fully open-sourced and already supports OpenCode and Codex.
Why does "modifying itself" keep failing
In the past, the industry pursued many recursive self-improvement (RSI) approaches to make agents smarter.
The idea sounds beautiful: let AI write its own code, modify its own prompts, and optimize its own systems. But once placed in the real open world, nearly all attempts fail completely.
Why is that?
It mainly gets stuck on two fatal problems:
No standard answers.
In the lab, grinding problems comes with unit tests and Ground Truth to tell you whether you're right. But in real work scenarios — writing a business proposal or reviewing a legal contract — there's simply no one online to grade you.
When the AI modifies its own prompt, how does it know whether the change made it smarter or just made things up?
Overfitting to single tasks.
After finally hacking the scaffold to fix a particular task, the next task turns out to be writing a web crawler, and all the previously modified rules become dead weight.
Directly applying the modified scaffold to the next task, the system quickly falls into a noise spiral, getting dumber with every change.
The Harness² proposed by the Google team this time is aimed squarely at these two fatal problems.
They built atwo-layer recursive architecture。
Two-layer recursion: one hand runs experiments, the other takes notes
The "squared" in the name Harness² refers to two nested recursive loops:Task-level recursion(Task-level) andHarnessing-level recursion(Harnessing-level).
The first level,Task-level recursion: running controlled experiments when there is no standard answer.
Faced with this new task at hand, the system first uses the initial scaffold to run a version of the reference trajectory. Whether this version's results are correct doesn't matter for now; the core is to see what deficiencies the model exposes during execution.
Immediately afterward, it activates a mechanism calledPropose–Probe–Compose :
Propose：
A dedicated editor role proposes two modification plans targeting the trajectory just produced.
One set is "Generic Patching" (Invariant), which specifically catches cross-domain common problems, such as planning chaos, tool-calling infinite loops, and context explosion;
The other set is "Domain-Specific Patching" (Domain), targeting the specific knowledge and constraints of the current business.
Probe：
Both of these modified candidate scaffolds are put back into the same task and run separately.
Even without a judge, that's fine: the system captures their execution traces and the reference trace for behavioral contrast (Behavioral Contrast). It checks whether the changes actually solved the step that got stuck, and whether they introduced any new logical regression.
Compose：
The composer puts all execution evidence together, picks out the changes that are genuinely proven effective by behavioral evidence, and merges them into the final execution scaffold.
That's not all. It also supports Parallel Recursion, spreading out $k$ candidate solutions at once for concurrent probing, synthesizing the best from a wider exploration space.
Amazing.
The second layer,Improvement-level recursion: distilling the experience of patching scaffolds into a meta-capability.
For many systems, once the current task is done and the deliverables handed over, the lifecycle ends there.
But the real power of Harness² lies in what comes next—
The entire debugging record (reference trace, candidate modifications, contrast evidence, composition decisions) is handed to the reflection module (Experience Reflector).
After reviewing everything, the reflection module writes an EXPERIENCE.md, clearly storing why it was changed this way, under what conditions it worked, and what consequences it brought, into thecontinuous improvement experience library(Harness Improvement Library).
Then, when the next new task comes in:
What the execution side receives is still a clean, untouched base scaffold.
But the "improvement brain" responsible for patching the scaffold already holds a thick historical experience manual in its hands. During the proposal and composition phases, it can directly retrieve effective components and pitfall-avoidance guides from past similar scenarios.
What improves is not just the scaffold at hand, but the future ability to improve scaffolds.
This turns ad-hoc patches from a single task into a lifelong-learning cognitive asset.
Benchmark Results
No matter how beautiful the mechanism sounds, it ultimately has to speak with data from the real battlefield.
The team directly took on three heavyweight hardcore professional benchmarks:
WorkBuddy-Bench: covering three high-difficulty office workflows—Code, Web, and Office suites;
JobBench: long-horizon end-to-end complex professional job tasks;
LAB: Harvey's open-sourced top-tier legal agent benchmark.
The base model chosen wasGemini 3.5 FlashandGLM-5, the execution scaffolds chosen were the mainstream open-sourceOpenCodeandCodex。
. The results were extremely dramatic.
In the WorkBuddy-Bench test, with only a single-step Harness² improvement ($k=1$):
GLM-5 under OpenCode saw its composite score skyrocket from 64.2 to 75.7, a jump of 11.5 points; under Codex it also rose from 64.4 to 75.3.
Gemini 3.5 Flashimproved from 72.4 to 77.5 under OpenCode, and from 74.5 to 77.5 under Codex.
Look at this data:
the scaffold-optimized GLM-5 (75.7 points / 75.3 points),directly crushed the original unmodified-scaffold Gemini 3.5 Flash (72.4 points / 74.5 points)。
On JobBench, which covers end-to-end job tasks, the single-step improvement likewise brought across-the-board gains, with Gemini 3.5 Flash rising by4.5 points，GLM-5 rose3.6 points; on the legal benchmark LAB, GLM-5's pass rate for some parts was directly raised significantly11.8 percentage points。
If the parallel budget is raised to $k=5$ (5 concurrent candidate paths), performance rises further: on WorkBuddy-Bench it gains up to another3.9 points, JobBench rises again4.2 points, LAB pass rate rises again6.7 percentage points。
What's even more remarkable is the cross-domain transfer testing of the experience library.
The team took the experience library accumulated from running tasks on JobBench, unplugged it and plugged it straight into WorkBuddy-Bench, and with no additional probing at all, painlessly gained +0.6 on code, +0.5 on web, and +2.9 on office.
Over the past two years, expectations for AI have been almost entirely staked on "base model scaling"—people assumed that as long as they waited for the next-generation GPT or the next-generation Gemini, the application problems at hand would resolve themselves.
But reality has thrown a bucket of cold water on all practitioners: base models are getting more and more expensive, marginal gains are slowing down, and the long-tail bugs in real-world deployment scenarios remain endless.
Give the model as large a context window as you want, and it will still cut corners when reconciling across files; give it as high a benchmark score as you want, and after writing a report it still won't proactively open the file to verify the formatting.
These things can never be fed by pretraining parameters alone.
What it needs is a set of Runtime cognitive engineering that can self-sense, self-repair, and continuously evolve during execution.Runtime cognitive engineering。
In the past, when we built scaffolding, engineers sat at their computers hand-writing Prompts, configuring tools, and fixing pipelines one by one—exhausting themselves half to death and still leaking everywhere.
Now Harness² has proven a new path:
returning the power to make rules to the Agent itself.
Paper authors
Finally, as usual, let's talk about the paper's authors.
There are 9 in total, and the concentration of Chinese researchers is extremely high—a full 78%...
Since the article is limited in length, three will be introduced here in order.
Ruiyao Xu, the first author and corresponding author of the paper.
He is a PhD student in the Department of Statistics and Data Science at Northwestern University in the US, advised by Kaize Ding.
His main research directions include large language models, graph machine learning, anomaly detection, and AI Agent systems.
This paper is the work he did during his internship at Google.
Yanfei Chen, a Senior Research Engineer at Google Cloud AI Research.
He received his bachelor's degree from Peking University, majoring in Electrical Engineering and Economics, and earned his Ph.D. in Industrial Engineering from the University of Pittsburgh in the United States in 2018.
He then worked as a Research Engineer at the machine learning company Qeexo, joined Google in 2020, and has contributed to projects such as Vertex AI Search and Gemini Enterprise Deep Research, focusing mainly on large language model agents, retrieval-augmented generation, and tool learning.
Zhongying CuiZhu, also known as Zoey CuiZhu, is a Research Engineer at Google Cloud AI Research.
She received her Ph.D. in Electrical Engineering from Stanford University in 2019, and mainly researches general agents, verifiable agents, and agent quality evaluation.
Reference link:
[1] https://arxiv.org/html/2610.14224v1