The question
Can a changed model do better on new tasks without losing what it could already do?
RESEARCH
We compare a model before and after adjustments to see whether gains carry over to unfamiliar tasks, rather than only appearing in the original test.
Back to researchCan a changed model do better on new tasks without losing what it could already do?
Early simulations outperformed a simple baseline, but still fell short of the improvement threshold when compared with a stricter control.
There is no evidence yet that the model can keep improving on its own, and these changes have not been confirmed in any public product.
RSI-RECURSION-PILOT-002 · 2026-10-08
This technical report is based on the original experiment records; it is not a published paper. It explains the research question, method, results, and scope.
This study compares two ways to choose the next subtask: one adapts its order using prior feedback, while the other follows a fixed curriculum. The question is whether feedback-based selection improves scores under the same budget and keeps that advantage in new synthetic test worlds. The measured gain came from an explicitly written software selector, not an AI model modifying itself.
We first compared feedback-based selection, a fixed curriculum, random budgeting, and simple priority selection across 24 development worlds. We then froze the same strategy and compared it again in 48 separately generated descendant test worlds. Both stages used synthetic tasks and the same resource budget, with an independent verifier checking output integrity. This checks whether the strategy reproduces its result in these generated tasks, not whether it passes an independent real-world evaluation.
Mean development scores were 0.7619 for feedback-based selection, 0.4870 for the fixed curriculum, 0.4612 for random budgeting, and 0.4991 for simple priority. Across 48 descendant test worlds, the scores were 0.7560, 0.4744, 0.5173, and 0.4317, respectively. The feedback-minus-fixed difference in the descendant stage was 0.2816, with a 95% bootstrap interval from 0.2097 to 0.3585. All 51 integrity checks passed.
In researcher-designed synthetic worlds, the feedback selector scored higher than the fixed curriculum, and the difference appeared again in newly generated descendant worlds. The score measures how the software chooses tasks from feedback; it does not measure model reasoning, learning, or self-improvement. The task worlds and candidate strategy came from the same research effort, with no real product integration or external blinded scoring.
A stronger evaluation would use independent researchers to create different task families, preregister the budget and acceptance threshold, score blindly, and include distorted feedback and unfamiliar conditions. If the strategy loses its advantage under those tests, the result should remain limited to the original synthetic setting.
Scores and confidence intervals are from the synthetic experiment record. The 48 worlds are independent descendant tests within one authored family, not 48 real-world applications.
Feedback-based task selection scored above the fixed curriculum; random budgeting and simple priority are included for comparison.
The mean difference between feedback-based selection and the fixed curriculum was 0.2816, with a 95% bootstrap interval of 0.2097–0.3585. This is a result from a researcher-designed software selector, not a model improving itself.
| Test condition | Observed value | Interpretation |
|---|---|---|
| Development worlds (24) | 0.7619 vs 0.4870 | Feedback-based selection vs fixed curriculum; random and simple-priority controls were also included. |
| Descendant test worlds (48) | 0.7560 vs 0.4744 | Feedback-based selection vs fixed curriculum; 95% bootstrap interval for the difference: 0.2097–0.3585. |
| Integrity checks | 51 / 51 passed | Checks output and evidence-chain integrity; this is not external validation. |
Source record: RSI-RECURSION-PILOT-002 outcome.json; SHA-256 ae54e163d6d3d48ae73260e92eb46e05678cd33621968290ff8c7525da751567. Research note based on the experiment record; not a peer-reviewed paper.
This external study examines bias in model-based judging, including answer position, verbosity, and self-enhancement, and compares judges with human preferences using MT-Bench and Chatbot Arena. It is a reminder to validate evaluation methods too; the experiment on this page used synthetic program scores, not an LLM judge.
There is no evidence yet that the model can keep improving on its own, and these changes have not been confirmed in any public product.
Explore other research topics