Summary / The 27 components
04 Code and context retrieval
Finding the small relevant part of a large repository for the step at hand.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
Nothing in the harness.
What is missing
Query extraction from the task and failing-test output, ripgrep plus BM25 recall over signatures and docstrings, and a 4K-token slice injected into REPAIR with an untrusted-data banner (arm h4).
Reviewer Claude, 2026-10-04; close-out verdicts 2026-10-23
Reviewer's read: Closed 10-22: built as the hx localize stage; no detectable gain
The spec forbade the model from browsing ("the model never browses"), which would have removed the one retrieval method her workers demonstrably use. The reconciliation dropped that rule. As with 03, the evidence is that workers retrieve for themselves; a pushed slice would have to beat grep-in-the-loop, and nobody has measured whether it does.
Close-out, 10-22 (pre-registered, boxed, Claude): built for real as stage S1 (localize) of the close-out bundle arm hx (A12): h0b + localize pre-pass + repro-first with failure digest + repro-triggered reflect/repair (2 rounds) + critic with one revision. hx against h0b, verified-hard 38 + pro-hard 20, 3 seeds: 103 against 94 cells (verified 65/114 against 55, pro 38/60 against 39). Gain +3.0 tasks of 58, paired bootstrap CI [−1.0, +7.0]; 4 regressions on tasks h0b always solved (limit 2); 2.9× h0b cost per cell. No detectable gain at this model. The pre-registered rule closes every bundled component together, with no halving round. Terminal: not re-run.
How it is measured no detectable gain (hx, 10-22)
Arm h4 against h0-v41, with gold-entity recall@5 of the slice reported alongside the resolve rate.
Adoption rules shared by every arm: How the tests work.
Research & attempts
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.