Piper · Coding harness
Log in

Summary / The 27 components

22 Independent reviewer (critic)

A second process that challenges a candidate, with no self-grading.

Latest—not scored
Baseline— 
Live use—last 7 days

Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.

What is built now

A REVIEW state in the driver's graph, switched off. The reviews table is empty.

What is missing

The blinded review packet, the critic call, falsification routed to real tests, and the single send-back to REPAIR (arm h7).

Reviewer Claude, 2026-10-04; close-out verdicts 2026-10-23

Reviewer's read: Closed 10-22: built as the hx critic stage; no detectable gain

The spec's best idea is that the critic never votes: a finding counts only once a test reproduces it. Elsewhere in her system the same idea is already live (a gate that refuses builds whose acceptance was green before the change), and it caught real vacuous work on its first night. In the harness, with the official grader downstream, a critic has to beat the vote on the cells the vote gets wrong, and on dev there are five.

Close-out, 10-22 (pre-registered, boxed, Claude): built for real as stage S3 (critic) of the close-out bundle arm hx (A12): h0b + localize pre-pass + repro-first with failure digest + repro-triggered reflect/repair (2 rounds) + critic with one revision. hx against h0b, verified-hard 38 + pro-hard 20, 3 seeds: 103 against 94 cells (verified 65/114 against 55, pro 38/60 against 39). Gain +3.0 tasks of 58, paired bootstrap CI [−1.0, +7.0]; 4 regressions on tasks h0b always solved (limit 2); 2.9× h0b cost per cell. No detectable gain at this model. The pre-registered rule closes every bundled component together, with no halving round. Terminal: not re-run.

How it is measured no detectable gain (hx, 10-22)

Arm h7 against h1-v41 on the harder set; a planted-defect fault test first.

Adoption rules shared by every arm: How the tests work.

Research & attempts

Score history

No scores yet. The first entry is the baseline for suite v1.

Experimental record

One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.

No interventions recorded yet.