Piper · Coding harness
Log in

Summary / Harness arms

A2 h0 (plumbing)

The baseline worker inside the harness: driver, git workspace, container execution, VERIFY. No gates, no budgets.

Latest65v4p
Baseline—1 run in v4p
Live use—last 7 days

Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.

What is built now

v4-flash era (09-08): 115/150, against baseline Δ −1.3, CI [−8.0, +5.3], undetermined. Plumbing costs nothing. Current model (09-15): 128/150 (85.3%), the parent of every later arm. Held-out plus regression (09-29): 93/100.

What is missing

Nothing; it is the parent arm.

Reviewer Claude, 2026-10-04; close-out verdicts 2026-10-23

Reviewer's read: Neutral, as intended

Its first version was rejected at −8.7 points because of a 40-turn budget the baseline did not have. Relabelling those 150 runs as h0-v1-budgeted instead of deleting them kept the record honest.

How it is compared measured

Parent of every arm since 09-14.

Adoption rules shared by every arm: How the tests work.

Research & attempts

No documents yet.

Score history

Suite v1 retired
0255075100Sep 876.7
Suite v2 retired
0255075100Sep 1585.3
Suite v3 retired
0255075100Sep 2993
Suite v4 retired
0255075100Oct 1848.2
Suite v4p current
0255075100Oct 1865

Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.

DateSuiteTypeScore±Live useCostNotes
2026-10-18v4psealed 65— —$1.46 h0b BOXED on pro-hard, k=3: 39/60 (seeds 14, 12, 13 of 20).
2026-10-18v4sealed 48.2— —$2.35 h0b BOXED (harness 0.8.0, no network) on verified-hard, k=3: 55/114 (seeds 20, 19, 16 of 38). Suite v4 = boxed; not comparable with pre-box rows.
2026-09-29v3sealed 93— —— h0-v41 on holdout 20 + regression 30, k=2: 93/100. Suite v3 = held-out sets.
2026-09-15v2sealed 85.3— —— h0-v41 on dev, k=3: 128/150. Model deepseek-flash (V4.1). Suite v2 = same tasks on the new model; not comparable with v1.
2026-09-08v1sealed 76.7— —— h0 on dev, k=3: 115/150. vs baseline Δ −1.3, CI [−8.0, +5.3], undetermined. deepseek-v4-flash.

Experimental record

One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.

No interventions recorded yet.