Piper · Coding harness
Log in

Summary / Harness arms

A12 hx (close-out bundle)

h0b plus four stages built for the 10-21 close-out: S1 localize pre-pass (03-08), S2 repro-first with failure digest (13, 12), S4 repro-triggered reflect and repair up to 2 rounds (14, 15), S3 independent critic with one revision (22, 21).

Latest63.3v4p
Baseline—1 run in v4p
Live use—last 7 days

Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.

What is built now

Built 10-21 by Claude in piperharness/closeout.py, two hooks in the driver, default-off: every existing arm runs byte-identical. Scratch files never enter the patch; the critic pins the change with a commit, never git stash (worktrees share one stash). Smoke on 3 tasks: every stage fired, 3/3 repro scripts valid, critic approved 3/3, box held (curl, pip download and searches for the answer store all failed).

What is missing

Nothing: closed.

Reviewer Claude, 2026-10-04; close-out verdicts 2026-10-23

Reviewer's read: No detectable gain (10-22)

Run 10-21 22:14 to 10-22 11:47, boxed, same model as h0b. Verified-hard 65/114 against h0b's 55; pro-hard 38/60 against 39. Over both sets +3.0 tasks of 58, CI [−1.0, +7.0], 4 regressions on tasks h0b always solved, $0.064 per cell against $0.022. The verified gain did not carry to pro. The smoke showed why it was likely: the model's own reproduction usually passes once its patch is in, while the hidden tests check something the issue text does not show, the same limit c14 hit on 10-12.

How it is compared no detectable gain (10-22)

Pre-registered (closeout-20261021): paired bootstrap against h0b's 10-18 boxed rows over both sets; KEEP needs CI excluding 0, ≥ +4.2 tasks, ≤ 2 always-solved regressions, gold-free.

Adoption rules shared by every arm: How the tests work.

Research & attempts

No documents yet.

Score history

Suite v4 retired
0255075100Oct 2257
Suite v4p current
0255075100Oct 2263.3

Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.

DateSuiteTypeScore±Live useCostNotes
2026-10-22v4psealed 63.3— —$3.96 hx BOXED on pro-hard, k=3: 38/60.
2026-10-22v4sealed 57— —$7.14 hx BOXED on verified-hard, k=3: 65/114. Close-out: with pro-hard, +3.0 tasks vs h0b, CI [−1.0, +7.0], NO DETECTABLE GAIN.

Experimental record

One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.

No interventions recorded yet.