Summary / The 27 components
21 Multi-agent development
Separating architect, coder, tester, reviewer and debugger roles.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
Nothing as roles. The N sessions in 16 are identical workers; the driver is the only orchestrator.
What is missing
Everything; the spec itself said only the validator role is adopted unconditionally.
Reviewer Claude, 2026-10-04; close-out verdicts 2026-10-23
Reviewer's read: Closed 10-22: built as the hx critic stage; no detectable gain
The spec was self-limiting in the right way: every role beyond the judge had to earn its keep one at a time. The deterministic validator (18) already fills the judge role. No role has been proposed with a measurement attached.
Close-out, 10-22 (pre-registered, boxed, Claude): built for real as stage S3 (critic + one revision) of the close-out bundle arm hx (A12): h0b + localize pre-pass + repro-first with failure digest + repro-triggered reflect/repair (2 rounds) + critic with one revision. hx against h0b, verified-hard 38 + pro-hard 20, 3 seeds: 103 against 94 cells (verified 65/114 against 55, pro 38/60 against 39). Gain +3.0 tasks of 58, paired bootstrap CI [−1.0, +7.0]; 4 regressions on tasks h0b always solved (limit 2); 2.9× h0b cost per cell. No detectable gain at this model. The pre-registered rule closes every bundled component together, with no halving round. Terminal: not re-run.
How it is measured no detectable gain (hx, 10-22)
One role at a time, each as an arm against h1-v41.
Adoption rules shared by every arm: How the tests work.
Research & attempts
- Build Spec 21 — Multi-agent Development (spawn, not summon) Spec 2026-09-06
- Coding-Harness Component 21/27 — Multi-agent Development Paper 2026-09-06
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.