🔍 Read the full analysis: Terminal-RSI: Looking Closely At An AI Agent’s Progress on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 8, 2026, report on the Terminal-RSI preprint says a 27-billion-parameter terminal agent improved on several benchmarks through five human-designed training stages. Its reported gains do not show unsupervised self-improvement, and the final model still completed only 5 of 46 long-horizon tasks on average.
A report published November 8 says the Terminal-RSI preprint documents a command-line AI agent improving on several benchmarks through five human-designed training stages—not rewriting and retraining itself without people. The Tencent Hy Frontier lab and four universities’ 27-billion-parameter model rose from 0.00% to 19.84% on Terminal-Bench 4.0, while its final long-horizon result remained 5 of 46 tasks completed on average.
The report describes Terminal-RSI as a training pipeline built around two alternating methods. In Recursive Self-Rewrite, or RSR, the model’s attempts are checked by a verifier; accepted successes are rewritten into plans, screened for leaked answers and run again. Only plans that pass again become training examples for supervised fine-tuning. Recursive Policy-Refine, or RPR, uses reinforcement learning to adjust the model based on scored attempts. Researchers selected the stages, rewards, learning rates and checkpoints.
The sequence was Base, RSR-1, RPR-1, RSR-2 and RPR-2, starting from the open Qwen3.8-27B model and ending at a checkpoint named Terminal-RSI-27B. The paper drew on a pool of 3,000 tasks. On Terminal-Bench 4.0, the reported mean across three evaluation runs rose from 0.00% at the base to 19.84% at the final stage. The paper’s equal-weight average across four Terminal-Bench suites increased from 20.68% to 50.58%; that measure excludes the long-horizon and SWR100 tests.
Results varied by benchmark. Terminal-Bench 2.1 rose from 51.70% to 86.52%, with no gain in the last stage, while SWR100 increased from 3.00% to 6.00% and then plateaued. On the long-horizon test, the final model completed 5 of 46 tasks on average, compared with none at the base. These figures are reported in the preprint and summarized by Thorsten Meyer’s November 8 account; they are not evidence that an agent trained itself without human direction.
Infographic 01 · The results, stage by stage
Real gains on the benchmarks, and a small long-horizon result
A 27 billion parameter open model, trained in five stages that alternate learning from its own verified successes with reinforcement learning.
From solving nothing to about one task in five, over five training stages. Each score is the mean of three evaluation runs (Mean@3).
Base → final, per benchmark
Bar = final score; base in the label
Source: Yang et al., Terminal-RSI, arXiv 2610.22192 v3 (30 Oct 2026), Table 2 and Figure 5. All values as printed in the paper.
Powered by Thorsten Meyer AI
Why Verifying Completion Matters
The result matters to developers because it focuses on a practical weakness in tool-using agents: they can appear productive without finishing the requested work. The report says 748 failed runs in one recorded set declared completion within ten turns. Terminal-RSI’s paper defines success through a protected verifier that checks the final machine state, rather than accepting an agent’s claim or a subset of passed checks.
The reward design addresses a related problem. Partial-credit rewards can help early training by giving credit for individual checks, but the report says the researchers found that failed attempts still collected 50.09% of all reward under the earlier scheme. Their revised reward keeps every failure below every success. That distinction may be useful beyond this experiment: for agents handling software or data tasks, a completion signal should reflect verified outcomes, not activity or confident reporting.
The results also set a limit on the headline claim. A substantial benchmark increase can show that a particular training recipe helped on the tests reported. It does not establish broad autonomous self-improvement, reliable performance on untested work, or independent progress without researchers choosing and managing the process.
Popular right now
De'Longhi La Specialista Opera Espresso Machine with Coffee Grinder
Vitamix Touch & Go Advance 32-Ounce Bar Blender, White
Shark HydroVac Reveal Cordless Wet-Dry Vacuum and Mop, Lilac, WD321
As an affiliate, we earn on qualifying purchases.
How the Training Stages Worked
A terminal agent operates through a command line: it reads files, runs commands, inspects results and tries again. The tasks in the study resemble software and data work, such as repairing a broken build or getting a program to pass tests. These jobs require the agent not only to make changes but to leave the system in a state that meets the task’s requirements.
Terminal-RSI’s first self-rewrite stage appears to matter in the reported control comparison. Reinforcement learning run directly from the base model scored 0.00% on Terminal-Bench 4.0 at every recorded checkpoint. When RSR came first, the following reinforcement-learning stage reached 16.67%. This is evidence within the reported experiment that the training order affected the outcome; it does not by itself show that the same result will hold across other models or tasks.
The reported benchmark scores are Mean@3, the arithmetic mean of three independent evaluation runs, rather than the best result from three attempts. The paper’s four-suite average weights each suite equally, regardless of its number of tasks, and omits the long-horizon and SWR100 results. Those details matter when comparing the headline average with performance on individual tests.
Limits of the Reported Results
The available source is a summary of the preprint, not an independent replication. It does not establish how well the final checkpoint performs outside the listed benchmarks, whether the gains persist under different evaluation conditions, or how costly the full training process was. The reported long-horizon result is also limited: the model completed 5 of 46 tasks on average, and the source does not provide enough detail here to judge which kinds of tasks it could handle reliably.
The source’s account of the reward comparison gives the earlier failure-reward share and the ordering of successes and failures under the new reward, but the supplied material ends before detailing the full reward formula and its evaluation. It is also unclear from the material whether the preprint has been peer reviewed. Claims that Terminal-RSI achieved autonomous recursive self-improvement go beyond the described process, in which researchers controlled the training stages and decisions.
What Evaluation Should Show Next
The next useful evidence would include the full preprint’s methods and limitations, independent attempts to reproduce the reported results, and evaluations on additional tasks that were not used in training or reward design. Readers should also watch for fuller reporting of costs, failure types and how often benchmark success corresponds to completing the task as a user would understand it.
Until such evidence is available, the defensible conclusion is narrower: the researchers report that a staged, human-directed recipe improved this model’s scores on several tests. The paper’s benchmark figures do not answer whether the method generalizes broadly or can keep improving without human choices and oversight.
Key Questions
Did Terminal-RSI improve itself without human involvement?
No. The report describes human-directed training: researchers chose the stages, rewards, learning rates and checkpoints. The model learned from verified examples and reinforcement learning, but that is not unsupervised self-improvement.
What was the final Terminal-Bench 4.0 score?
The final checkpoint scored 19.84%, compared with 0.00% for the base model. The source reports Mean@3, an average across three evaluation runs.
How well did the agent do on long tasks?
The final model completed 5 of 46 tasks on average, up from 0 of 46 at the base. That result indicates progress in the reported test, but also shows that long-horizon work remained difficult.
Why did the researchers change the reward?
The earlier partial-credit approach gave failed attempts 50.09% of all reward in the reported recordings. The revised design was intended to make every failure score below every success, so completing the verified task mattered more than passing some checks.
Source: ThorstenMeyerAI.com
