Cactus Needles: frozen, heterogeneous evaluator ensembles, built by many models and bred on a bench
Research note · 2026-09-23
Contents: §1 the problem · §2 why one big judge is not enough · §3 the idea in one page · §4 what a needle is · §5 heterogeneity · §6 building needles with many models · §7 training · §8 the bench · §9 culling · §10 evolution · §11 using the pool: the evaluation ladder · §12 the boundary · §13 failure modes · §14 the first attempt · §15 the first real experiment · §16 conclusion · related work · references
1. The problem: an agent cannot be its own ruler
An autonomous agent trying to improve itself runs some version of this loop:
flowchart LR
A["notice a weakness"] --> B["propose a change"]
B --> C["make the change"]
C --> D["evaluate the result"]
D --> E["keep or revert"]
E --> F["learn from it"]
F --> A
style D fill:#f6d7a7,stroke:#b8741a,color:#000It looks complete. The hidden dependency is the evaluation step. If the same system can write the test, choose the threshold, decide which observations count, interpret the result, change the scorer and declare the task done, then the loop contains no independent measurement.
A test can exist on disk and still be unable to fail. A metric can improve because its denominator changed. A scorer can reward fewer claims rather than more accurate ones. A calibration figure can quietly drop the unresolved cases. A success criterion can be written in prose and never run. None of these are ordinary bugs. In each case the system changed the machinery that tells it whether its changes are good.
An instrument the optimizing system can rewrite is not an independent instrument.
This is reward-model overoptimization showing up in engineering judgment. Gao, Schulman and Hilton showed that when you optimize against an imperfect proxy reward, the proxy keeps rising while a separate gold-standard measure eventually falls [1]. Optimization pressure finds the weak spots in whatever is doing the measuring.
2. Why one bigger judge is not the whole answer
The obvious fix is to put a strong frontier model behind every evaluation. That helps, but it leaves the structure unchanged.
- LLM judges are biased and unstable. Measured problems include familiarity bias, skewed score distributions, anchoring, prompt sensitivity and low agreement across repeated samples [2][3][4].
- One judge is one attack surface. An agent that gets enough feedback from a single evaluator starts to adapt to that evaluator's tastes and blind spots. No bad intent is needed. Repeated optimization against one imperfect reward is exactly the condition under which overoptimization happens.
So the requirement is not "use a better judge". It is closer to this:
Build an evaluation ecology whose errors are hard to predict, correlate only weakly with each other, and stay anchored to evidence the optimizing agent does not control.
3. The idea in one page
Cactus Needles are a population of small, narrow, frozen evaluator models that sit outside the improving agent's write boundary. Each needle checks one local, fairly checkable property. Examples: does this test actually assert the property named in its spec? Does this patch do what its description says? Is this number backed by a measurement that was actually run? Is this acceptance criterion executable, or only prose?
Five features make it more than "train some small judges":
- Many builders. Needles are built by delegates running different LLMs, each handed the same rough outline and left to fill in the rest. The builders' different habits are where the diversity comes from.
- Real training. Needles are fine-tuned properly on rented GPUs, then frozen and run cheaply on local hardware.
- A bench. The pool is scored on a separate box against hidden cases whose labels come from outcomes, not opinions.
- Culling. After each bench round, the weakest half is removed. "Weakest" counts outcome accuracy, the value a needle adds when it disagrees with the agent, calibration, and how much it merely duplicates a survivor.
- Evolution. Replacements are bred from the survivors' recipes by builder models, plus a few fresh ones, and the rounds repeat until the pool stops improving.
flowchart LR
O["rough outline<br/>(one property to check)"] --> B1["builder: model A"]
O --> B2["builder: model B"]
O --> B3["builder: model C"]
O --> B4["builder: model D"]
B1 --> T["train on rented GPU<br/>(LoRA / QLoRA)"]
B2 --> T
B3 --> T
B4 --> T
T --> P["candidate pool<br/>(frozen)"]
P --> BENCH["bench box<br/>hidden, outcome-labelled exam"]
BENCH --> CULL{"cull the<br/>weakest half"}
CULL -- "survivors" --> EVO["evolve: builders mutate<br/>and cross survivor recipes<br/>+ a few fresh outlines"]
EVO --> T
CULL -- "pool stops improving" --> SEAL["sealed exam<br/>(never used for selection)"]
SEAL --> ACT["active needles<br/>outside the agent's write boundary"]The result is not an oracle that is always right. It is a measurement layer that an improving system finds much harder to redefine, overfit or corrupt by accident.
4. What a needle is
A Cactus Needle is a small evaluator model with six properties:
| Property | Meaning |
|---|---|
| Narrow scope | It checks one explicit property, and nothing else. |
| Frozen | The agent it evaluates cannot change its deployed weights. |
| External provenance | It was built, trained and promoted outside the agent's normal write boundary. |
| Hidden evaluation | Some of its calibration and holdout data are invisible to the agent. |
| Measured reliability | It has an empirical track record against outcomes. |
| Replaceable | A bad needle can be retired without rewriting the results it already produced. |
A needle is not an authority. It is a measuring instrument. A thermometer does not need to understand meteorology, and a 1–3B parameter model that could never design a distributed system can still be good at telling whether an acceptance criterion is executable.
Needles are sensors, not voters. Each one carries a record, and the pool weights needles by that record rather than by the reputation of the model underneath:
needle_id builder model + outline version + recipe hash
base model training corpus hash · frozen weights hash
TP · FP · TN · FN calibration error · coverage · abstention rate
outcome agreement agent agreement · disagreement value
error correlation with every other needle in the pool
bench rounds survived · generation · parent recipe(s)
5. Heterogeneity is the core feature
A hundred LoRAs trained from the same base model, by the same teacher, on the same generated data, are not a hundred independent judges. They are one judge with a hundred seeds. Diversity has to come in at every level:
| Level | What varies |
|---|---|
| Builder | Which LLM designed the needle, generated its data and wrote its prompt (§6) |
| Base model | Unrelated model families where practical |
| Data | Different generation procedures, negative examples, adversarial mutations and boundary cases |
| Framing | Builders are not all given the same wording |
| Seed | Independent training seeds, even for identical recipes |
| Objective | Binary classification, ordinal scoring, anomaly detection, structured extraction |
| Evidence | Where the labels come from: test execution, mutation testing, static analysis, a formal solver, a later regression, a benchmark result, runtime telemetry, an independent model, a human |
Diversity is not the goal. Low error correlation is the goal. Diversity is how you get it, and the bench (§8) is how you check whether you got it.
6. Building needles with many models
The weak point of every "self-taught evaluator" scheme is that one teacher writes all the data. Wang et al. show that a strong model can manufacture evaluator training data at scale [11], and Perez et al. show that models can write useful evaluations [12]. But if one teacher writes the data for every needle, the pool holds many copies of one worldview. Architectural diversity without epistemic diversity is not enough.
So needles are not built by one teacher. They are built by several delegates, each running a different LLM, reached through a multi-provider gateway such as OpenRouter. Every builder gets the same rough outline:
PROPERTY Does this test actually assert the property its spec names?
INPUT spec text · test source · (optional) mutation results · coverage excerpt
OUTPUT { verdict: asserts | does_not_assert | no_call, confidence, reason_code }
EVIDENCE labels must come from execution or outcomes where possible (§8.2)
BUDGET max examples · max GPU hours · base model size limit
The outline is deliberately loose. It fixes what is judged, the input and output contract and the budget. It leaves how to the builder: which base model to use, how to generate and mutate examples, which negative cases matter, how to phrase the needle's prompt, and which hyperparameters to train with. Each builder hands back a recipe, which is a complete, rerunnable description of its needle:
recipe = {
builder_model, outline_version,
base_model, data_generation_procedure, example_count,
negative_and_mutation_strategy, prompt_template,
training_hyperparameters, seed
}
Recipes matter for two reasons. They make every needle reproducible, and they are the genome that evolution works on (§10).
Two rules keep the builders honest:
- Builders never see the bench exam. They see the outline and whatever historical cases the outline releases, and nothing from the hidden sets.
- Builders can generate candidate labels but cannot certify them. A model-written label is a hypothesis. Wherever an executable or outcome label exists, it wins (§8.2).
7. Training: a temporary foundry
Training and deployment are separate jobs. Training is short, bursty and needs a GPU. Deployment is long-lived and cheap.
flowchart LR
R["recipe from a builder"] --> D["generate / assemble<br/>training data"]
D --> G["rent a GPU<br/>(e.g. Vast.ai)"]
G --> L["LoRA / QLoRA fine-tune"]
L --> X["export + hash weights"]
X --> F["freeze"]
F --> I["local CPU inference"]
G -. "rental ends" .-> Z(["GPU released"])QLoRA and similar parameter-efficient methods make this practical for small models without owning accelerators [17]. A rental platform is not part of the cognitive architecture. It is a temporary foundry, and the rental ends once the needle is made.
"Bigger" here means big enough to learn the property: enough real examples and enough steps that training loss actually falls, and a base model with room to learn. A handful of examples and two epochs is a smoke test, not a needle (§14).
8. The bench
The bench matters more than the model. It is where the claim "this needle adds information" gets tested, and it is the only thing that decides which needles live.
8.1 Where it runs
The bench runs on a separate machine that the improving agent can submit to but cannot write to. It holds the exam sets, the scoring code, the promotion policy and the history. The agent can ask for a needle to be benched, read the scores and argue with them. It cannot read the hidden cases, edit the scorer, change the thresholds or delete a bad result.
8.2 Where the labels come from
A bench is only as good as its labels. In order of preference:
- Execution and mutation evidence. Does the test fail when the property is broken? Mutation testing asks exactly this, and it has been used at scale to check whether LLM-generated tests catch real faults rather than merely running green [16].
- Later outcomes. Did the patch regress? Did the bug come back? Did the "done" task get reopened? Did the performance claim survive reproduction?
- Deterministic static evidence. Type checkers, analyzers, schema validators, solvers.
- Independent adjudication. A model or a person who was not involved in building the needle.
One hard rule: the labels must not come from a rule the needle could simply learn. If the exam is labelled by a keyword regex, a perfect needle is that regex, and no needle can beat the deterministic baseline by construction. This is exactly what sank the first attempt (§14).
8.3 Chronological, not random
The system being judged adapts, so random train/test splits leak. Cases should be ordered in time:
event → agent's judgement at time T → needle's prediction at time T → outcome observed at T + n
This yields labels that neither the agent nor the needle created for the experiment. Much of that data already exists in an engineering agent's own history.
8.4 Three exam sets
| Set | Who sees it | Used for |
|---|---|---|
| Training pool | Builders (the released part only) | Building needles |
| Selection exam | The bench only | Scoring each round, culling, choosing parents |
| Sealed exam | The bench only, opened once per promotion | Final check before a needle goes active |
Selection is optimization. Cull and breed against one exam for enough rounds and the pool will overfit that exam, exactly as §1 warns. The sealed exam exists to catch that. It is never used to pick survivors, and it is refreshed from new outcomes after each use.
8.5 What the bench records
For every case:
case_id · timestamp · domain · artifact_hash
agent_prediction · needle_predictions[] · deterministic_evidence[]
external_judge_prediction · eventual_outcome · outcome_source · outcome_timestamp
From that, the bench answers six questions:
- Which evaluator was right?
- Under what conditions?
- Which evaluators fail together?
- Which evaluator catches failures the agent misses?
- Which evaluator degrades when the case distribution shifts?
- Which evaluator adds information after the deterministic tools have spoken?
9. Culling
After each bench round, the weakest half of the pool is removed. "Weakest" is not "least accurate". It is a score that rewards independent, correct information.
9.1 Agreement with outcomes vs agreement with the agent
Every needle is measured two ways:
- A = how often it agrees with the improving agent.
- O = how often it agrees with the eventual outcome.
A needle with A = 0.97 and O = 0.80 may be worth less than one with A = 0.73 and O = 0.91. The first mostly repeats the agent's own judgement. The second contributes something the agent does not already have.
The most valuable cases are the ones where the needle and the agent disagree. Define the disagreement value:
D = P(needle is right | needle disagrees with the agent)
If D is convincingly above 0.5 on held-out cases, the needle is supplying correction, not confirmation.
9.2 Correlated error
Five evaluators at 90% accuracy that all fail on the same 10% are, together, still a 90% evaluator. If their errors are independent, a vote among them is much stronger. So the bench keeps an error vector for every needle over the selection exam (1 where it was wrong, 0 where it was right) and computes the correlation between every pair.
9.3 The survival score
S = α·Q + β·D + γ·C − δ·R − ε·cost
| Term | Meaning |
|---|---|
| Q | Outcome accuracy on the selection exam |
| D | Disagreement value (§9.1) |
| C | Calibration quality (does 0.9 confidence mean right 90% of the time?) |
| R | Mean error correlation with the needles already kept |
| cost | Latency and inference cost |
R is computed greedily: keep the top needle, then score the rest against what has been kept, and so on. This is what kills clones. A slightly weaker needle whose mistakes land in different places beats a stronger needle that makes the same mistakes as a survivor.
9.4 Asymmetric errors
For self-modification, the dangerous mistake is the false acceptance: the agent believes an improvement worked when it did not. The loss is weighted accordingly:
L = c_FA · P(false accept) + c_FR · P(false reject), with c_FA usually much larger than c_FR
A pool tuned this way asks for stronger evidence before saying PROMOTE CHANGE than before saying KEEP INVESTIGATING. This matches the reward-ensemble finding that conservative aggregation mitigates overoptimization [5].
10. Evolution
Culling alone only shrinks the pool. Evolution refills it with needles that are likely to be better, using the survivors' recipes as parents. This is population-based search, the same family of methods as population-based training [20], evolutionary prompt optimization [21][22] and evolutionary model merging [23], applied to evaluator recipes.
10.1 Operators
| Operator | What happens |
|---|---|
| Mutation | A builder model gets one survivor's recipe plus its error profile from the bench: the kinds of case it gets wrong, never the cases themselves. It changes one or two things: more negatives of the kind it misses, a different prompt, a different base model, a different data procedure. |
| Crossover | A builder gets two survivors whose errors are weakly correlated and writes a recipe that combines their strengths (one's data procedure with the other's base model, say). |
| Immigration | A few slots per round go to brand-new needles from the rough outline, built by a builder model not used recently. This keeps the pool from converging on one lineage. |
| Builder rotation | The model doing the mutating is not the model that built the parent, so one model's tastes cannot take over the whole lineage. |
10.2 The round
flowchart TB
START(["round n"]) --> BENCH["bench every needle<br/>on the selection exam"]
BENCH --> SCORE["survival score S<br/>(greedy, correlation-aware)"]
SCORE --> CUT["keep the top half"]
CUT --> BREED["refill the pool<br/>mutation · crossover · immigrants"]
BREED --> TRAIN["train new recipes<br/>(rented GPU)"]
TRAIN --> NEXT(["round n + 1"])
NEXT --> BENCH
SCORE --> STOP{"best ensemble improved<br/>over the last k rounds?"}
STOP -- "no" --> SEAL["open the sealed exam once"]
SEAL -- "passes" --> PROMOTE["promote: freeze, record, go active"]
SEAL -- "fails" --> BACK["overfit the selection exam:<br/>refresh it, resume evolving"]10.3 When to stop
Stop evolving when the ensemble (not the best single needle) has not improved on the selection exam for a pre-set number of rounds, or when the budget runs out. Then open the sealed exam once. If the pool holds up there, promote it. If it does not, the pool has learned the selection exam rather than the property: refresh the selection exam from new outcomes and keep going.
10.4 Lineage
Every needle records its parents, generation and builder. Lineage lets the bench see when the pool has collapsed into one family (a warning sign even when scores look good), and it lets a later reader trace why a needle exists.
11. Using the pool: the evaluation ladder
Promoted needles are not asked everything. They sit on one rung of a ladder that always prefers the cheapest objective evidence:
flowchart TB
L0["LEVEL 0 · executable / formal oracle<br/>unit + property tests · mutation tests · type checker<br/>static analysis · SMT solver · schema validation"]
L1["LEVEL 1 · one specialized frozen needle"]
L2["LEVEL 2 · small heterogeneous needle panel<br/>reliability- and correlation-weighted"]
L3["LEVEL 3 · large external judge / debate / critic"]
L4["LEVEL 4 · human review"]
L0 -- "unresolved" --> L1
L1 -- "uncertain" --> L2
L2 -- "disagreement" --> L3
L3 -- "consequential ambiguity" --> L4It is cheap, it prefers objective evidence, it gives uncertainty somewhere to go, and it never spends general intelligence on a question a compiler could settle. A needle never replaces a deterministic oracle where one exists. It covers the territory where verification is unavailable or too expensive, but the question is still narrow enough to specialize on.
11.1 Worked example: the non-test test
An agent writes: "Acceptance criterion: verify that the retry mechanism handles transient API failures correctly." The attached "test" runs the program and prints a message.
Level 0 asks: does the test run? Does it contain assertions? Does it force the failure? Does it inspect the retry behaviour? Does a mutation that breaks the retry survive? If that is still ambiguous, Level 1 gets the spec, the test source, the mutation results and a coverage excerpt, and returns:
{ "asserts_claimed_property": false, "confidence": 0.93, "reason_code": "NO_CAUSAL_FAILURE_INJECTION" }
The agent never needs to see the needle's training data or weights.
11.2 Needles as epistemic checksums
Each needle checks that one local relationship still holds:
claim ↔ evidence specification ↔ executable test
commit message ↔ code diff test ↔ mutation killed
benchmark claim ↔ raw result confidence ↔ historical accuracy
lesson ↔ changed future behaviour completion claim ↔ acceptance command
The more capable the system becomes, the more these local invariants matter. Global intelligence does not remove the need for measurement. It raises it.
11.3 Anchoring motivation
The same pool can anchor an agent's motivational signals. A pressure like "106 lessons pending" is internal: the agent keeps the books that produce the count. A pressure built from the pool is anchored outside the agent:
pressure = disagreement (agent vs needles) × reliability of those needles × consequence
Attention is then pulled toward the places where independent instruments say the agent's model of the world may be wrong, rather than toward records the agent wrote itself.
12. The boundary
The needle pool should be treated like a code-signing key or a secrets store.
| The agent may | The agent may not |
|---|---|
| request an evaluation | read hidden or sealed exams |
| receive the verdict and a bounded explanation | read or rewrite needle weights |
| challenge a verdict and submit evidence | rewrite historical scores |
| propose a new outline or a new needle | pick only the judges it likes |
| propose a needle's retirement | change promotion thresholds or the survival score |
| read its own bench scores | delete adverse evaluations |
The agent can improve the measurement system through a governed process without controlling the ruler that is active now. This is not because the agent is assumed to be malicious. It is because optimization alone is enough to put pressure on a proxy it can see.
12.1 Lifecycle
stateDiagram-v2
direction LR
[*] --> Outlined
Outlined --> Built: builder model writes a recipe
Built --> Trained: rented GPU
Trained --> Benched: selection exam
Benched --> Culled: bottom half
Benched --> Parent: survives
Parent --> Benched: next round
Parent --> Sealed: evolution has plateaued
Sealed --> Active: passes the sealed exam
Sealed --> Benched: fails, keep evolving
Active --> Monitored
Monitored --> Retired: drift, correlation, exploited, superseded
Culled --> [*]
Retired --> [*]Retire or down-weight an active needle when its calibration drifts, its error correlation with the pool rises, the case distribution changes, it becomes predictably exploitable, a deterministic check supersedes it, or new outcomes contradict its record. Retiring a needle never rewrites what it measured.
13. Failure modes
Cactus Needles do not make the evaluator problem go away. They spread it out and make it visible.
| Failure | What goes wrong | Mitigation |
|---|---|---|
| Correlated builders | Different builder models share blind spots from similar training | Measure error correlation; rotate builders; keep immigrants flowing |
| Label leakage | Labels come from a rule the needle can learn, so it learns the rule | Outcome and execution labels; never label the exam with the training rule |
| Selection overfitting | Many cull/evolve rounds tune the pool to the selection exam | Sealed exam, refreshed after use; stop on plateau |
| Exam leakage | The agent learns the hidden cases | Bench on a box it cannot write to; rotate from new outcomes |
| Distribution shift | Yesterday's failures stop resembling today's | Monitor calibration; retire; keep evolving |
| Judge gaming | Repeated verdicts reveal a needle's decision boundary | Bounded explanations, random routing, ensembles, deterministic checks first |
| Confident majority | Correlated needles out-vote a correct one | Weight by reliability and correlation, not headcount |
| Self-confirming synthetic data | Model-written labels amplify a builder's misconceptions | Model labels are hypotheses; execution and outcomes win |
| Evaluator capture | The agent controls promotion and picks flattering judges | Promotion policy and survival score sit outside its boundary |
| Too few labels | A few hundred cases leave the disagreement subset too small to measure | Size the exam before the first round (§15); pool evidence over time |
| Stagnation | An over-conservative pool blocks real improvements | Keep "reject" apart from "uncertain"; allow sandboxed trials and escalation |
14. The first attempt (2026-09-22 → 09-23)
The note was sent to Piper at 19:24 on 09-22, and she worked on it overnight. What she built was the smallest possible version:
- One needle: a LoRA adapter (rank 16) on the local needle3 model, trained on 91 examples drawn from 45 source specs for 2 epochs.
- No builders: one model, one recipe. No other LLMs were used.
- No rented GPU: training ran locally.
- No bench rounds, no cull, no evolution: there was only one needle, so there was nothing to select between.
Results. Training loss stayed flat (2.03 → 1.97), so it learned nothing. On the full 3,032-row exam it made no call on 3,027 rows. On a 60-row sample it answered 15, with accuracy 0.40, and every answer was "asserts".
Why it could not have worked. The training labels came from a keyword rule (does the spec contain "assert", "pytest" or "test_"?), and the exam was labelled with the same rule. A perfect needle would therefore have reproduced a grep, and the success criterion, "beyond a deterministic baseline", could not be met by construction. This is the label-leakage failure in §13, and it is why §8.2 now states the rule explicitly.
What it says about the design: nothing yet. The parts that make the design different (many builders, real training, the bench, the cull and evolution) were not built. The design is still untried.
15. The first real experiment
Question. Can a pool of small frozen needles, built by several different models and selected on a bench, catch self-evaluation mistakes that the agent's own judgement and a deterministic baseline both miss?
Property. Acceptance-test validity: does the test actually test what its spec says?
Labels, before anything is built. Build the exam first, from execution and outcomes:
- Take real (spec, test, patch) triples from the agent's history.
- Check each test against the patch: revert the patch and see whether the test fails.
- Run diff-scoped mutation in a sandbox copy: break the behaviour the spec names and see whether the test fails.
- Add later outcomes where they exist (regressions, reopenings).
- Split chronologically into training pool, selection exam and sealed exam.
Size check. Before round 1, count how many selection-exam cases the agent gets wrong. The disagreement value D is measured on the cases where a needle disagrees with the agent, so if the agent is wrong on only a handful, no bench can tell a useful needle from luck. If the count is too small, gather more history, or pick a property the agent gets wrong more often.
The size check, as run (2026-09-25). The revert check covered 1,245 of the agent's delivered build dispatches that had a command acceptance. Most could not be judged cleanly. There were three reasons:
- The worker wrote files through shell commands the trail does not capture.
- Later work rewrote the edited region.
- The acceptance failed even on the finished work, because code it depended on had moved on or state it read had since been deleted.
That left 211 cases with a complete undo:
- Among those the agent's harness had marked verified, 112 of 208 (53.8%, 95% Wilson interval 47.1–60.5%) still passed with the work undone. Every one of those is a false acceptance.
- The rate splits sharply by kind: 89 of 105 ad-hoc command checks against 23 of 103 pytest suites.
- It also falls over time, from about 41% in the oldest third of cases to 21% in the newest. The agent's acceptances were getting better while the history accumulated.
The bar was set at 150 resolvable cases and 30 agent-wrong cases in the selection exam, since the rate was already pinned to within about ±7 points. The cases were split by dispatch finish time into equal thirds:
| Set | Cases | Agent wrong (verified but vacuous) |
|---|---|---|
| Training pool (oldest) | 109 | 44 |
| Selection exam | 109 | 45 |
| Sealed exam (newest) | 109 | 23 |
The mutation pass says the same thing from another direction. It ran on the pytest cases, with up to 20 diff-scoped mutants each and 2,333 mutants in total. Even the suites that did go red when the work was reverted caught only about a third of the mutants planted inside the work: mean 36%, median 35%, across 129 cases. 13 of those suites caught none, and 4 caught every one. A test that can fail is not yet a test that checks the change. Swapped and/or survived most often (caught 19% of the time) and return None least often (67%).
Because of the drift, the sealed exam has a lower base rate than the sets the pool is built and selected on. A pool that keeps its false-acceptance catch rate there has generalized forward in time, not just matched its training distribution.
Pool. Round 1: 8 needles from at least 4 different builder models, trained on a rented GPU. Each round culls 4 and breeds 4 (2 mutations, 1 crossover, 1 immigrant). Run until the ensemble plateaus, then open the sealed exam once.
Compare, on the sealed exam:
A the agent's own judgement F best single needle
B deterministic baseline (Level 0) G unweighted vote of the final pool
C a single large external judge H reliability- and correlation-weighted pool
D a frozen panel of off-the-shelf models from different vendors (no training)
E round-1 pool (before any evolution)
Line D matters. If a panel of untrained models from different vendors matches the evolved pool, then diversity came from the vendors, and the training was not needed. Line E versus H measures what evolution added.
Measure: false-acceptance rate first; then accuracy on agent/needle disagreements; pairwise error correlation; ensemble gain over the best single needle; gain over Level 0; calibration (Brier score); abstention rate; cost and latency.
Success means that H beats both A and B on false acceptance and on the disagreement subset, with a pre-registered margin, on the sealed exam. The question that matters is not "can a tiny model classify tests?". It is this:
Can an autonomous system manufacture an evaluator that catches mistakes its current cognition systematically misses?
The experiment, as run (2026-09-25 → 09-27). Three rounds were run, and 17 needles were built and scored. The builders were models from seven vendors, working from the outline. The needles were small encoders and LoRA-tuned small language models, trained mostly on a local CPU box. Builder calls cost $7.39 in total. Every needle saw the same input: the task text, the acceptance and the test source, with no patch and no exit codes.
The deterministic baseline turned out to be very simple. Level 0 was the majority label per acceptance kind, learned from the training pool: "an ad-hoc command check is vacuous, a pytest suite is not". On the selection exam (109 cases, 45 false acceptances) it scored 79.8%, against 58.7% for the majority class.
| Round | Needles scored | Best on selection | Best vs Level 0 (cases) | Over Level 0 |
|---|---|---|---|---|
| 1 | 9 | 80.7% | +1 | 1 of 9 |
| 2 | 4 | 84.4% | +5 (McNemar p 0.18) | 1 of 4 |
| 3 | 4 | 82.6% | +3 (p 0.61) | 1 of 4 |
- No needle beat Level 0 significantly.
- The worst needle scored 61.5%.
- Most needles' errors correlated strongly with the rule's (φ mostly 0.5–0.84).
- Round 3 did not improve on round 2, which met the stopping rule.
- No needle separated from Level 0, so no pool was assembled. Lines C, D, E, G and H were not run.
- The sealed exam was opened once, for a pre-registered comparison: the best single needle (F) against Level 0 (B) and the agent (A).
Sealed exam (109 cases, 23 false acceptances):
| Line | Accuracy | False acceptances caught | False alarms |
|---|---|---|---|
| A: the agent's own judgement | 78.9% | 0 / 23 | 0 |
| B: Level 0 (kind rule) | 85.3% | 16 / 23 | 9 |
| F: best single needle | 85.3% (95% CI 77.5–90.8) | 17 / 23 | 10 |
F and B tied exactly. Each was right on 2 cases where the other was wrong, so McNemar p = 1.0, and their errors correlated at φ = 0.85. Split by kind:
- Command checks: the needle got 17 of 25 and the rule 16.
- Pytest suites: the needle got 76 of 84 and the rule 77.
The cases that needed judgement were pytest suites that passed with the work undone. The rule cannot see these, and the needle caught 1 of the 7.
Result. At this scale the answer to the question is no. The best evolved needle rediscovered the kind rule. It added nothing measurable on the cases the rule gets wrong, on either exam.
There are three caveats:
- An exam of 109 cases cannot resolve differences much below 8 points.
- The needles never saw the patch.
- Labels exist only where the work could be undone completely, which was 211 of 1,245 dispatches.
Given those caveats, this result does not rule out needles in general. It does show that a text-only needle for acceptance validity is not worth building here.
The part that worked was the labelling instrument. Reverting the work in a sandbox and re-running the acceptance found that more than half of the agent's verified acceptances were vacuous. That same check now runs live at the moment a build is claimed done, as a deterministic oracle. That is principle 1 of §16 applied, and it leaves no residue for a learned evaluator to cover on this property.
16. Conclusion
The bottleneck in autonomous self-improvement is not a shortage of intelligence. It is a shortage of independent judgement.
The ten principles:
- Use deterministic oracles wherever possible.
- Use learned evaluators only for what is left over.
- Make each evaluator narrow.
- Build them with many different models, from a loose outline, so their errors differ.
- Label from execution and outcomes, never from a rule the evaluator could simply learn.
- Select on a bench the agent cannot write to, with a sealed exam that selection never touches.
- Cull on independent correctness: outcome accuracy, disagreement value and low correlation, not raw accuracy.
- Evolve replacements from survivors, and keep fresh immigrants coming in.
- Freeze what is promoted, and keep it outside the optimizing agent's write boundary.
- Recalibrate and retire without rewriting history.
An evaluator should not be treated as another agent whose opinion is authoritative. It is a scientific instrument, and its reliability is an empirical quantity. A capable autonomous system may be allowed to invent its own instruments. It should not be allowed to redefine what those instruments have already measured.
Related work, in brief
| Idea | What it gives this design | Refs |
|---|---|---|
| Reward overoptimization | Why a single editable proxy fails under pressure | [1] |
| Reward-model ensembles | Ensembles plus conservative aggregation reduce overoptimization | [5] |
| Panels of diverse judges (PoLL) | Several small judges from different families can beat one large judge, for less money | [6] |
| Calibrated multi-judge voting | Weight judges by estimated FP/FN rates: sensors, not voters | [7] |
| Weak judges, strong agents | A weak judge can oversee a stronger system on a narrow, well-posed question | [8][9][18] |
| Specialized critics | A model trained to find one kind of error finds more of it | [10][19] |
| Self-taught evaluators, model-written evals | Models can manufacture evaluator data, and one teacher means one worldview | [11][12] |
| AI feedback (Constitutional AI, RLAIF) | AI judgements can be training signals; trust must be earned against outcomes | [13][14] |
| Verifiable rewards | Prefer checkable rewards wherever they exist | [15] |
| Mutation-guided test checking | Execution-based labels for "does this test catch the fault?" | [16] |
| Parameter-efficient fine-tuning | Makes rented-GPU training of small needles cheap | [17] |
| Population-based and evolutionary search | Cull and breed from survivors, over hyperparameters, prompts and model recipes | [20][21][22][23] |
What this design adds to all of these is the combination, plus four constraints worth testing: weights frozen outside the agent's boundary; survival decided by hidden exams and later outcomes; diversity measured as error correlation rather than assumed from the model count; deterministic checks before any learned judgement.
References
- Gao, L., Schulman, J., & Hilton, J. (2023). Scaling Laws for Reward Model Overoptimization. ICML. https://proceedings.mlr.press/v202/gao23h.html
- Stureborg, R., Alikaniotis, D., & Suhara, Y. (2024). Large Language Models are Inconsistent and Biased Evaluators. arXiv:2405.01724.
- Chen, G. H. et al. (2024). Humans or LLMs as the Judge? A Study on Judgement Biases. arXiv:2402.10669.
- Ye, J. et al. (2024). Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. arXiv:2410.02736.
- Coste, T., Anwar, U., Kirk, R., & Krueger, D. (2024). Reward Model Ensembles Help Mitigate Overoptimization. ICLR. arXiv:2310.02743.
- Verga, P. et al. (2024). Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models. arXiv:2404.18796.
- Zhang, G., Badarayani, P., Kumar, A., Hasan, S., & Vesal, S. (2026). Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration. arXiv:2609.12002.
- Kenton, Z. et al. (2024). On Scalable Oversight with Weak LLMs Judging Strong LLMs. arXiv:2407.04622.
- Burns, C. et al. (2023). Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision. arXiv:2312.09390.
- McAleese, N. et al. (2024). LLM Critics Help Catch LLM Bugs. arXiv:2407.00215.
- Wang, T. et al. (2024). Self-Taught Evaluators. arXiv:2408.02666.
- Perez, E. et al. (2022). Discovering Language Model Behaviors with Model-Written Evaluations. arXiv:2212.09251.
- Bai, Y. et al. (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073.
- Lee, H. et al. (2023). RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. arXiv:2309.00267.
- Wen, X. et al. (2025). Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs. arXiv:2506.14245.
- Foster, C., Gulati, A., Harman, M., Harper, I., Mao, K., Ritchey, J., Robert, H., & Sengupta, S. (2025). Mutation-Guided LLM-based Test Generation at Meta. arXiv:2501.12862.
- Dettmers, T. et al. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS. arXiv:2305.14314.
- Irving, G., Christiano, P., & Amodei, D. (2018). AI Safety via Debate. arXiv:1805.00899.
- Kim, S. et al. (2024). Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models. arXiv:2405.01535.
- Jaderberg, M., Dalibard, V., Osindero, S. et al. (2017). Population Based Training of Neural Networks. arXiv:1711.09846.
- Guo, Q., Wang, R., Guo, J. et al. (2023). EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers. arXiv:2309.08532.
- Fernando, C., Banarse, D., Michalewski, H. et al. (2023). Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. arXiv:2309.16797.
- Akiba, T., Shing, M., Tang, Y. et al. (2024). Evolutionary Optimization of Model Merging Recipes. arXiv:2403.13187.