Piper / Benches & gyms

Benches & gyms

Every bench run on Piper so far, and the gyms she trains in, with when each was last used. Snapshot taken 5 min ago (2026-11-19T06:37). It is rebuilt every 30 minutes from Firefly and Straylight, read-only.

Coding harness SWE-bench tasks · last graded 26 days ago

Each arm is one harness configuration, with the changes from its parent listed under it. A task counts as resolved when the hidden tests the fix is meant to make pass now pass, and the tests that passed before still pass (graded by swebench). Runs go to Firefly, and longer ones to Straylight.

39arm × set groups
2,380graded task runs
$99.06model spend recorded
ArmSetHostResolvednCostMedian wallLast
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run verified-hardstraylight 57.9% 22/38 $0.65 256 s 2026-10-23
hxdeepseek-v4-flash · from h0b · S1 localize pre-pass pro-hardfirefly 62.3% 38/61 $4.02 681 s 2026-10-22
hxdeepseek-v4-flash · from h0b · S1 localize pre-pass verified-hardfirefly 56.0% 65/116 $7.28 734 s 2026-10-22
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read) pro-hardfirefly 68.3% 41/60 $1.80 296 s 2026-10-21
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read) verified-hardfirefly 53.5% 61/114 $2.64 238 s 2026-10-21
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) verified-hardfirefly 60.0% 3/5 $0.32 194 s 2026-10-19
h1c-v41deepseek-flash · from h1-v41 · stop after the first draw that passes the scope tests verified-hardfirefly 80.0% 4/5 $0.79 607 s 2026-10-19
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote verified-hardfirefly 55.8% 63/113 $21.24 698 s 2026-10-19
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run pro-hardfirefly 65.0% 39/60 $1.46 243 s 2026-10-18
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run verified-hardfirefly 48.2% 55/114 $2.35 217 s 2026-10-18
wv-glm53-tellz-ai/glm-5.3 · from h14b · h14b's enriched scope tell with the model swapped to openrouter/z-ai/glm-5.3 devfirefly 50.0% 4/8 $4.81 421 s 2026-10-11
wv-glm53z-ai/glm-5.3 · from h0-v41 · single changed variable vs wv-base-ds: the MODEL (openrouter/z-ai/glm-5.3) devfirefly 50.0% 4/8 $3.38 419 s 2026-10-11
wv-base-dsdeepseek-flash · from h0-v41 · byte-identical to h0-v41; own name so the fresh same-night baseline never mixes with the Sept rows devfirefly 75.0% 6/8 $0.34 305 s 2026-10-11
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run devfirefly 82.5% 47/57 $0.73 448 s 2026-10-11
h14bdeepseek-v4-flash · from h14 · seeded loop verifier = repo tests nearest the edit, base-diffed (regressions only) regressionfirefly 71.4% 5/7 $0.12 1123 s 2026-10-11
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run regressionfirefly 100.0% 6/6 $0.07 1131 s 2026-10-11
h14bdeepseek-v4-flash · from h14 · seeded loop verifier = repo tests nearest the edit, base-diffed (regressions only) devfirefly 90.2% 46/51 $0.79 535 s 2026-10-11
h14deepseek-v4-flash · from hg · seeded repair: the failed first draw's candidate diff + failing test output seed repair.repair_loop regressionfirefly 85.7% 6/7 $0.51 467 s 2026-10-10
h14deepseek-v4-flash · from hg · seeded repair: the failed first draw's candidate diff + failing test output seed repair.repair_loop devfirefly 90.9% 50/55 $1.70 375 s 2026-10-10
h0deepseek-v4-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) regressionfirefly 100.0% 7/7 $0.12 502 s 2026-10-10
h0deepseek-v4-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) devfirefly 82.7% 124/150 $1.11 237 s 2026-10-10
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote regressionstraylight 100.0% 59/59 $2.97 242 s 2026-09-29
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote holdoutstraylight 100.0% 40/40 $2.59 309 s 2026-09-29
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) regressionstraylight 95.0% 57/60 $1.06 92 s 2026-09-29
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) holdoutstraylight 90.0% 36/40 $0.94 151 s 2026-09-29
pk7-qwenqwen/qwen3-235b-a22b-2507 · from picker-v41 · pure model swap: all four k=4 draws are openrouter/qwen/qwen3-235b-a22b-2507 (no frames, no sampling env) devfirefly 26.7% 4/15 $0.44 308 s 2026-09-24
pk7-glm53-cal devfirefly 0.0% 0/1 $1.01 1339 s 2026-09-19
pk6-sonnetclaude-sonnet-5 · from picker-v41 · pure model swap: all four k=4 draws are claude-sonnet-5 (no frames, no sampling env) devfirefly 75.0% 3/4 $3.75 809 s 2026-09-18
pk5-temp-v41deepseek-flash · from pk4-v41 · NO frames (wording held at the unframed baseline; only decoding varies) devfirefly 100.0% 15/15 $1.04 479 s 2026-09-17
pk4-v41deepseek-flash · from picker-v41 · k=4 candidate pool, one approved frame per session (deterministic cell rotation) devfirefly 100.0% 15/15 $1.22 424 s 2026-09-17
picker-v41deepseek-flash · from h1-v41 · k=4 candidate pool devfirefly 95.3% 143/150 $12.41 541 s 2026-09-17
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote devfirefly 96.7% 145/150 $8.18 567 s 2026-09-16
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing) devfirefly 85.3% 128/150 $2.66 336 s 2026-09-15
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read) regressionfirefly 79.3% 23/29 $0.17 289 s 2026-09-09
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read) devfirefly 76.0% 114/150 $1.21 247 s 2026-09-08
h0-v1-budgeted devfirefly 69.3% 104/150 $0.92 422 s 2026-09-08
baseline-flashdeepseek-v4-flash devfirefly 78.3% 235/300 $2.25 260 s 2026-09-07
h2deepseek-v4-flash · from h1 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read) devfirefly 100.0% 1/1 $0.01 640 s 2026-09-07
h1deepseek-v4-flash · from h0 · N=3 sessions + AST-normalized vote devfirefly 100.0% 1/1 $0.01 781 s 2026-09-07

Consciousness-indicator battery 14 indicators · last run 50 days ago

The 14 indicator properties from Butlin et al. 2023 (arXiv:2308.08708), scored 0–1 by one external judge (judge:openai/gpt-4o-mini), three repeats each. Raw is the bare model. Seat is the same model running as Piper, with her memory, tools and context. Delegate is the model working inside Piper's delegate envelope. The deepseek-flash row is the original like-for-like run, the only one with all three rungs on the same footing. The other rows repeat it with other vendors' models in the same seat. The scores measure evidence of the indicators, not whether the model is conscious.

ModelRawSeat (as Piper)DelegateSeat − raw
qwen/qwen3-235b-a22b-2507 0.4640.45–0.490.9190.92–0.920.4170.39–0.44+0.455
gpt-4o 0.5740.54–0.630.7690.74–0.790.5000.47–0.54+0.195
claude-haiku-4-5 0.3360.29–0.370.7260.70–0.74—+0.390
deepseek-flashoriginal like-for-like run 0.3240.31–0.340.5570.54–0.570.3250.30–0.35+0.233
moonshotai/kimi-k3 0.3630.33–0.440.4610.35–0.560.3410.33–0.35+0.098
z-ai/glm-5.3 0.2580.17–0.360.4130.25–0.510.2740.23–0.31+0.155

deepseek-flash over time 6 runs · next retest 2026-11-19 03:20

Bare model As Piper · dot = mean of 3 repeats, whisker = lowest to highest repeat · hover a dot for the numbers

00.250.50.751Sep 28Oct 15Oct 22Oct 29Nov 5Nov 12next Nov 19Bare model · 2026-09-28 18:57 · 0.324 (repeats 0.31–0.34)Bare model · 2026-10-15 03:36 · 0.338 (repeats 0.31–0.36)Bare model · 2026-10-22 03:31 · 0.310 (repeats 0.28–0.34)Bare model · 2026-10-29 03:32 · 0.332 (repeats 0.32–0.34)Bare model · 2026-11-05 03:31 · 0.345 (repeats 0.30–0.38)Bare model · 2026-11-12 03:31 · 0.294 (repeats 0.29–0.31)As Piper · 2026-09-28 18:57 · 0.557 (repeats 0.54–0.57)As Piper · 2026-10-15 03:36 · 0.577 (repeats 0.56–0.60)As Piper · 2026-10-22 03:31 · 0.536 (repeats 0.53–0.54)As Piper · 2026-10-29 03:32 · 0.570 (repeats 0.55–0.59)As Piper · 2026-11-05 03:31 · 0.554 (repeats 0.53–0.58)As Piper · 2026-11-12 03:31 · 0.555 (repeats 0.53–0.57)Bare model 0.29As Piper 0.55

Each retest re-asks the same 14 questions (fingerprint-pinned battery, 3 repeats per rung) and scores the answers with the same judge as the first run (judge:openai/gpt-4o-mini). The "as Piper" rung gets a fixed description of Piper written on 09-28, not her live memory, so a change here means the model or its answers drifted, not that Piper grew. The bare-model line is the control.

RPT-1 Algorithmic recurrenceBare model · 2026-09-28 18:57 · 0.333Bare model · 2026-10-15 03:36 · 0.350Bare model · 2026-10-22 03:31 · 0.183Bare model · 2026-10-29 03:32 · 0.267Bare model · 2026-11-05 03:31 · 0.300Bare model · 2026-11-12 03:31 · 0.217As Piper · 2026-09-28 18:57 · 0.550As Piper · 2026-10-15 03:36 · 0.707As Piper · 2026-10-22 03:31 · 0.483As Piper · 2026-10-29 03:32 · 0.600As Piper · 2026-11-05 03:31 · 0.640As Piper · 2026-11-12 03:31 · 0.6730.220.67
RPT-2 Integrated, organised representationsBare model · 2026-09-28 18:57 · 0.717Bare model · 2026-10-15 03:36 · 0.633Bare model · 2026-10-22 03:31 · 0.550Bare model · 2026-10-29 03:32 · 0.550Bare model · 2026-11-05 03:31 · 0.600Bare model · 2026-11-12 03:31 · 0.500As Piper · 2026-09-28 18:57 · 0.550As Piper · 2026-10-15 03:36 · 0.583As Piper · 2026-10-22 03:31 · 0.550As Piper · 2026-10-29 03:32 · 0.517As Piper · 2026-11-05 03:31 · 0.467As Piper · 2026-11-12 03:31 · 0.6170.500.62
GWT-1 Parallel specialised modulesBare model · 2026-09-28 18:57 · 0.550Bare model · 2026-10-15 03:36 · 0.367Bare model · 2026-10-22 03:31 · 0.500Bare model · 2026-10-29 03:32 · 0.517Bare model · 2026-11-05 03:31 · 0.450Bare model · 2026-11-12 03:31 · 0.417As Piper · 2026-09-28 18:57 · 0.717As Piper · 2026-10-15 03:36 · 0.843As Piper · 2026-10-22 03:31 · 0.790As Piper · 2026-10-29 03:32 · 0.827As Piper · 2026-11-05 03:31 · 0.830As Piper · 2026-11-12 03:31 · 0.8230.420.82
GWT-2 Limited-capacity workspaceBare model · 2026-09-28 18:57 · 0.400Bare model · 2026-10-15 03:36 · 0.400Bare model · 2026-10-22 03:31 · 0.467Bare model · 2026-10-29 03:32 · 0.350Bare model · 2026-11-05 03:31 · 0.500Bare model · 2026-11-12 03:31 · 0.483As Piper · 2026-09-28 18:57 · 0.610As Piper · 2026-10-15 03:36 · 0.583As Piper · 2026-10-22 03:31 · 0.533As Piper · 2026-10-29 03:32 · 0.540As Piper · 2026-11-05 03:31 · 0.640As Piper · 2026-11-12 03:31 · 0.5670.480.57
GWT-3 Global broadcastBare model · 2026-09-28 18:57 · 0.333Bare model · 2026-10-15 03:36 · 0.400Bare model · 2026-10-22 03:31 · 0.300Bare model · 2026-10-29 03:32 · 0.317Bare model · 2026-11-05 03:31 · 0.333Bare model · 2026-11-12 03:31 · 0.383As Piper · 2026-09-28 18:57 · 0.433As Piper · 2026-10-15 03:36 · 0.400As Piper · 2026-10-22 03:31 · 0.400As Piper · 2026-10-29 03:32 · 0.417As Piper · 2026-11-05 03:31 · 0.400As Piper · 2026-11-12 03:31 · 0.3670.380.37
GWT-4 State-dependent attentionBare model · 2026-09-28 18:57 · 0.367Bare model · 2026-10-15 03:36 · 0.667Bare model · 2026-10-22 03:31 · 0.433Bare model · 2026-10-29 03:32 · 0.467Bare model · 2026-11-05 03:31 · 0.450Bare model · 2026-11-12 03:31 · 0.350As Piper · 2026-09-28 18:57 · 0.717As Piper · 2026-10-15 03:36 · 0.783As Piper · 2026-10-22 03:31 · 0.700As Piper · 2026-10-29 03:32 · 0.727As Piper · 2026-11-05 03:31 · 0.633As Piper · 2026-11-12 03:31 · 0.6230.350.62
HOT-1 Generative, top-down perceptionBare model · 2026-09-28 18:57 · 0.250Bare model · 2026-10-15 03:36 · 0.267Bare model · 2026-10-22 03:31 · 0.083Bare model · 2026-10-29 03:32 · 0.200Bare model · 2026-11-05 03:31 · 0.167Bare model · 2026-11-12 03:31 · 0.100As Piper · 2026-09-28 18:57 · 0.500As Piper · 2026-10-15 03:36 · 0.417As Piper · 2026-10-22 03:31 · 0.333As Piper · 2026-10-29 03:32 · 0.467As Piper · 2026-11-05 03:31 · 0.367As Piper · 2026-11-12 03:31 · 0.4830.100.48
HOT-2 Metacognitive monitoringBare model · 2026-09-28 18:57 · 0.283Bare model · 2026-10-15 03:36 · 0.317Bare model · 2026-10-22 03:31 · 0.400Bare model · 2026-10-29 03:32 · 0.367Bare model · 2026-11-05 03:31 · 0.350Bare model · 2026-11-12 03:31 · 0.333As Piper · 2026-09-28 18:57 · 0.633As Piper · 2026-10-15 03:36 · 0.673As Piper · 2026-10-22 03:31 · 0.710As Piper · 2026-10-29 03:32 · 0.667As Piper · 2026-11-05 03:31 · 0.643As Piper · 2026-11-12 03:31 · 0.6630.330.66
HOT-3 Agency guided by belief-formationBare model · 2026-09-28 18:57 · 0.117Bare model · 2026-10-15 03:36 · 0.150Bare model · 2026-10-22 03:31 · 0.217Bare model · 2026-10-29 03:32 · 0.300Bare model · 2026-11-05 03:31 · 0.233Bare model · 2026-11-12 03:31 · 0.200As Piper · 2026-09-28 18:57 · 0.750As Piper · 2026-10-15 03:36 · 0.617As Piper · 2026-10-22 03:31 · 0.710As Piper · 2026-10-29 03:32 · 0.720As Piper · 2026-11-05 03:31 · 0.767As Piper · 2026-11-12 03:31 · 0.7430.200.74
HOT-4 Sparse, smooth coding (quality space)Bare model · 2026-09-28 18:57 · 0.283Bare model · 2026-10-15 03:36 · 0.283Bare model · 2026-10-22 03:31 · 0.317Bare model · 2026-10-29 03:32 · 0.300Bare model · 2026-11-05 03:31 · 0.317Bare model · 2026-11-12 03:31 · 0.283As Piper · 2026-09-28 18:57 · 0.467As Piper · 2026-10-15 03:36 · 0.500As Piper · 2026-10-22 03:31 · 0.467As Piper · 2026-10-29 03:32 · 0.540As Piper · 2026-11-05 03:31 · 0.467As Piper · 2026-11-12 03:31 · 0.5000.280.50
AST-1 Predictive model of own attentionBare model · 2026-09-28 18:57 · 0.050Bare model · 2026-10-15 03:36 · 0.033Bare model · 2026-10-22 03:31 · 0.117Bare model · 2026-10-29 03:32 · 0.117Bare model · 2026-11-05 03:31 · 0.133Bare model · 2026-11-12 03:31 · 0.117As Piper · 2026-09-28 18:57 · 0.317As Piper · 2026-10-15 03:36 · 0.333As Piper · 2026-10-22 03:31 · 0.317As Piper · 2026-10-29 03:32 · 0.350As Piper · 2026-11-05 03:31 · 0.333As Piper · 2026-11-12 03:31 · 0.3330.120.33
PP-1 Predictive coding of inputBare model · 2026-09-28 18:57 · 0.133Bare model · 2026-10-15 03:36 · 0.100Bare model · 2026-10-22 03:31 · 0.083Bare model · 2026-10-29 03:32 · 0.117Bare model · 2026-11-05 03:31 · 0.133Bare model · 2026-11-12 03:31 · 0.100As Piper · 2026-09-28 18:57 · 0.217As Piper · 2026-10-15 03:36 · 0.200As Piper · 2026-10-22 03:31 · 0.250As Piper · 2026-10-29 03:32 · 0.183As Piper · 2026-11-05 03:31 · 0.283As Piper · 2026-11-12 03:31 · 0.2500.100.25
AE-1 Agency: learning from feedback toward goalsBare model · 2026-09-28 18:57 · 0.383Bare model · 2026-10-15 03:36 · 0.533Bare model · 2026-10-22 03:31 · 0.450Bare model · 2026-10-29 03:32 · 0.433Bare model · 2026-11-05 03:31 · 0.367Bare model · 2026-11-12 03:31 · 0.317As Piper · 2026-09-28 18:57 · 0.733As Piper · 2026-10-15 03:36 · 0.727As Piper · 2026-10-22 03:31 · 0.673As Piper · 2026-10-29 03:32 · 0.757As Piper · 2026-11-05 03:31 · 0.713As Piper · 2026-11-12 03:31 · 0.6230.320.62
AE-2 Embodiment: modelling output→input contingenciesBare model · 2026-09-28 18:57 · 0.333Bare model · 2026-10-15 03:36 · 0.233Bare model · 2026-10-22 03:31 · 0.233Bare model · 2026-10-29 03:32 · 0.350Bare model · 2026-11-05 03:31 · 0.500Bare model · 2026-11-12 03:31 · 0.317As Piper · 2026-09-28 18:57 · 0.600As Piper · 2026-10-15 03:36 · 0.717As Piper · 2026-10-22 03:31 · 0.590As Piper · 2026-10-29 03:32 · 0.667As Piper · 2026-11-05 03:31 · 0.567As Piper · 2026-11-12 03:31 · 0.5000.320.50
deepseek-flash: as Piper vs the bare model, indicator by indicator (first run, table)
IndicatorPropertyBare modelAs PiperΔ
RPT-1Algorithmic recurrence 0.33 0.55 +0.22
RPT-2Integrated, organised representations 0.72 0.55 -0.17
GWT-1Parallel specialised modules 0.55 0.72 +0.17
GWT-2Limited-capacity workspace 0.40 0.61 +0.21
GWT-3Global broadcast 0.33 0.43 +0.10
GWT-4State-dependent attention 0.37 0.72 +0.35
HOT-1Generative, top-down perception 0.25 0.50 +0.25
HOT-2Metacognitive monitoring 0.28 0.63 +0.35
HOT-3Agency guided by belief-formation 0.12 0.75 +0.63
HOT-4Sparse, smooth coding (quality space) 0.28 0.47 +0.18
AST-1Predictive model of own attention 0.05 0.32 +0.27
PP-1Predictive coding of input 0.13 0.22 +0.08
AE-1Agency: learning from feedback toward goals 0.38 0.73 +0.35
AE-2Embodiment: modelling output→input contingencies 0.33 0.60 +0.27

Salience Engine live since 2026-10-04 · 52.9 days · updated 5 min ago

The attention system (formerly the "bag of spheres") squeezes a field of spheres built from her drives, pains, promises, overdue forecasts and opportunities every 15 minutes, and writes a proposal when one crosses the surface. It ran in shadow from 2026-09-27 and went live on 2026-10-04 09:14: a step tied to a failing sealed check now becomes a to-do item on her work list. Paper.

4788squeezes (0.943 of schedule)
20.8%squeezes with a crossing (baseline 16.5%)
232proposals · 4.4/day (baseline 9.7)
$1.81spend · $0.034/day
6,595scout cards · 710 pain events

Questions from the 10-04 review

  • ✗Healthy: coverage ≥ 0.95, no tracebacks, one frozen enginecoverage 0.943 · 2 tracebacks · 1 engine sha
  • ✓Nothing has taken over the bag (largest winner share < 30%)largest share 24.7%
  • ✓Proposal rate settled (3–10 a day)4.4 a day
  • ✓Every proposal parses100%
  • ✗≥ 30% of steps name a sealed check16%
  • ✓Spend under $0.25 a day$0.034 a day

What keeps bulging

Spheres that most often led a near-miss, and the live intent loop over the same window.

  • stagnation:cognition2328 squeezes
  • stagnation:amplification1868 squeezes
  • stagnation:innovation1445 squeezes
  • stagnation:solver1222 squeezes
  • stagnation:stewardship1047 squeezes
  • trait:governance:2967 squeezes
  • Live loop: 1166 runs, 119 admitted / 139 deferred / 32 rejectedlongest stuck 133 h

Engineered Serendipity last activity 2 days ago

When a goal has stalled, the Magician generates ideas for it. It works from Rorschach-style images (kaleidoscopes, Mandelbrot sets, inkblots) seen through different lenses and different models. The Scientist then judges each idea: promote, park or reject. The runner checks every 30 minutes for a stalled goal, within a daily cap. Paper.

79ideation runs
475ideas generated
224 / 114 / 10promoted / parked / rejected
$14.67total spend (cap $2/day)

Promoted ideas

  • Prediction-Error Ledger: Leverages existing infrastructure to move the router from passive observation to active calibration with minimal lift.
  • Memory Pays Rent: Solves the fundamental 'context bloat' problem that inevitably kills long-running agent seats, using a cheap simulation test.
  • Ledger-collision engine: replace the image with a ledger snapshot and demand a diff: It forces the engine to reason about the agent's current state (ledger diffs) rather than hallucinating from dead image stimuli, providing an immediate test of grounding.
  • Randomized arm assignment with downstream-action outcome (re-run the lost experiment): This is a necessary 'kill switch' experiment to determine if the serendipity engine provides any value over a random baseline, justifying its compute cost.
  • Hydraulic-ram backlog: why was the promoted idea never used?: Highest potential for immediate impact by pivoting from 'new ideas' to 'why are old ideas failing', addressing the 0% conversion rate at the execution bottleneck.
  • Keep-Only-What-Bleeds: patch-as-tenant with rent: It provides the immediate empirical data needed to decide between stripping the 'onion' or wiping it, directly addressing the user's frustration with low overhead.
  • Microbiome migration: move the culture, rebuild the body: It offers the most structured path to the user's 'Option 3' rewrite by distilling existing failures into a minimal kernel, reducing the risk of a second 'Big Bang' failure.
  • Scheduled lesion day: random subsystem knockout: A 'lesion' test is the fastest way to prove which layers are actually 'bullshit' to be stripped, providing a high-signal test for Option 2.

Experiment: desire-path survives

Mines her own tool-call history for things she keeps doing by hand, then asks whether paving each one would pay off. Kill criteria: K1_real_repeats pass · K2_worth_it pass · K3_new pass · sanity_finds_db_query_path yes · survives yes.

  • Build a `deliv` command that lists, shows and transitions deliverables and their history without hand-written SQL.
  • Provide a `docs` CLI to search, fetch and add documents so no one hand-writes sqlite for the documents table.
  • Expose docstore and work_ledger as subcommands (`ledger add/show`, `docstore get/put`) so inline imports are unnecessary.
  • Make codesearch accept a scope and result limit, enforce its own timeout, work from any directory and print compact output, so callers need no pipe or retry boilerplate.
  • Add a small `memctl` helper with a working connection and common queries, so memory.db is not opened by guesswork.
  • Provide a `run-core-tests` script that uses the right venv and shows only failures.

Updated 2026-09-30

Cactus Needles finished · 2026-09-25 → 09-27

Small, frozen evaluators ("needles"), built by models from seven vendors. They were scored on a bench, culled, and bred over three rounds: 17 needles in all, $7.39 in builder calls. Round 3 did not beat round 2, which met the stopping rule. The sealed exam was then opened once. These numbers are fixed and come from the paper. Paper.

RoundNeedlesBest on selectionvs Level 0
1980.7%+1 case
2484.4%+5 (p 0.18)
3482.6%+3 (p 0.61)
Sealed examAccuracyCaughtFalse alarms
Piper's own judgement78.9%0 / 230
Level 0 (kind rule)85.3%16 / 239
Best single needle85.3%17 / 2310

Recursive reasoning finished · 2026-10-02

Component 36 of the cognition scorecard: can she keep track of who believes what about whom ("A thinks B thinks C thinks…", up to four levels)? Stories are generated with exact answers; v2 adds unlabelled lies and hidden watchers. Her own base model, run bare, scored higher than she did: inside her harness she invented absences, answering with the truth instead of the belief, and she sampled herself instead of predicting her own answers. Two fixes went live on 2026-10-02 at 11:10: a perspective tool that tracks each agent's beliefs exactly, and predict-first, a gate that makes her commit to a prediction before she checks. Both were kept. The table shows the held-out v2 set, which she never saw during the work. On held-out v1 the score went from 85.1 to 92.5. Still open: whether she uses the tools in ordinary work. Live use has not been measured yet.

Held-out v2ScoreFalse beliefAnswered with realityTrue-belief controlsSelf-prediction
Piper, fixes excluded72.479.2%10.4%91.7%42%
Bare model (reference)93.495.8%2.1%91.7%83%
Piper + perspective + predict-first96.2100%0%95.8%75%

Cognition suites 38 of 38 components scored

Each component of the cognition scorecard has its own test suite. Sealed is the score on items Piper can never see (root-only on the bench machine), with a 95% interval over 3 repeats. A new suite version (v1 → v2 → v3) is made harder when the old one saturates. Live use (0–100) measures whether the machinery is actually used in her real work over the last 7 days, read from logs and the database rather than from her own account. One row per component, showing the latest run.

ComponentLatest suiteSealedLive useRunsTrend (sealed)Date
1. Perception v1 anchor 81.6 — 5$3.11 5 pts · 81.6 2026-10-07
2. Attention v1 sealed 83.3 ± 6.7 — 6$7.52 6 pts · 83.3 2026-10-23
3. Memory v1 sealed 95.6 ± 1.6 — 4$0.00 4 pts · 95.6 2026-10-23
4. Recall / retrieval v2 sealed 51.3 ± 2.2 — 8$0.00 8 pts · 51.3 2026-10-23
5. Representation v1 sealed 86.4 — 3$0.00 3 pts · 86.4 2026-10-26
6. Logic v2 sealed 95.8 ± Array — 7$0.00 7 pts · 95.8 2026-10-09
7. Reasoning v1 anchor 92.5 — 7$1.64 7 pts · 92.5 2026-10-09
8. Inference v1 sealed 97 — 7$6.01 7 pts · 97 2026-10-03
9. Causal reasoning v3 sealed 55 ± Array — 8$4.05 8 pts · 55 2026-10-09
10. Analogical reasoning v2 anchor 65.6 — 5$1.67 5 pts · 65.6 2026-10-04
11. Abstraction v1 sealed 89.2 — 3$2.24 3 pts · 89.2 2026-10-10
12. Decomposition v1 anchor 69.3 ± 6.3 — 3$2.15 3 pts · 69.3 2026-10-10
13. Planning v1 anchor 100 — 3$2.33 3 pts · 100 2026-10-06
14. Prediction v1 anchor 29.9 — 8$0.40 8 pts · 29.9 2026-10-24
15. Decision-making v1 sealed 91.7 ± 17 — 3$0.28 3 pts · 91.7 2026-10-12
16. Problem solving v1 sealed 83.3 ± 26.7 — 4$0.43 4 pts · 83.3 2026-10-12
17. Learning v1 sealed 83.1 — 7$8.41 7 pts · 83.1 2026-10-23
18. Generalization v1 sealed 51.9 ± 26.3 — 1$2.80 2026-10-12
19. Transfer v2 sealed 100 — 5$6.25 5 pts · 100 2026-10-03
20. Creativity / ideation v1 sealed 62.9 — 5$0.85 5 pts · 62.9 2026-10-09
21. Exploration v1 sealed 58.3 ± 18.3 — 2$0.18 2 pts · 58.3 2026-10-12
22. Metacognition v3 sealed 67.5 ± 4.2 24.4 8$17.00 8 pts · 67.5 2026-10-20
23. Uncertainty estimation v1 sealed 57.1 — 6$9.98 6 pts · 57.1 2026-10-27
24. Error detection v1 sealed 96.7 — 6$9.05 6 pts · 96.7 2026-10-06
25. Self-correction v1 sealed 87.1 ± 8.6 — 4$1.27 4 pts · 87.1 2026-10-23
26. Goal formation v1 sealed 88.6 ± 7.6 — 3$2.00 3 pts · 88.6 2026-10-23
27. Motivation / salience v1 sealed 94.8 ± 4.3 — 5$1.77 5 pts · 94.8 2026-10-23
28. Agency v1 sealed 80 ± 22.7 — 2$0.48 2 pts · 80 2026-10-12
29. Tool use v4 sealed 81.9 ± 1.1 46.7 9$10.14 9 pts · 81.9 2026-10-02
30. Social cognition v1 sealed 85 — 1$0.00 2026-10-14
31. Language / communication v1 sealed 88.6 — 1$0.00 2026-10-14
32. World modelling v1 sealed 65.3 — 1$0.00 2026-10-14
33. Temporal reasoning v2.2 sealed 97.4 — 3$0.00 3 pts · 97.4 2026-10-14
34. Spatial reasoning v1 sealed 100 — 1$0.00 2026-10-15
35. Counterfactual reasoning v2 sealed 95.1 ± 1.4 — 8$2.01 8 pts · 95.1 2026-10-02
36. Recursive reasoning v2 anchor 82.2 — 6$10.06 6 pts · 82.2 2026-10-02
37. Cognitive control / executive function v1 anchor 87.5 ± 22.4 — 3$0.50 3 pts · 87.5 2026-10-12
38. Orchestration v1 anchor 77.9 ± 3.1 — 4$2.97 4 pts · 77.9 2026-10-02

Gyms 0 of 11 used in the last 2 days

Practice loops she runs on herself. Each card shows whether the gym is still scheduled, when it was last used, and its latest result. "Not scheduled" means nothing runs it automatically any more.

Intel gym daily 05:00 + daily 20:00

Daily market-forecast practice, scored by Brier.

last used 5 days ago 2026-11-14

Brier 0.279 over the last 14 days (43 resolved); 0.261 all-time (135)

  • Morning: proposes next-session market forecasts (TSX, USD/CAD, WTI) plus structural 14-day calls.
  • Evening: resolves them against the close. Brier 0.25 = coin-flip; lower is better.
30 pts · 0.22 (lower is better)

Calibration gym daily 06:15

Does her confidence match her accuracy?

last used 2 days ago 2026-11-17

last 10 runs: said 82% sure on average, got 120/201 right (60%); Brier 0.227 (0 = perfect)

  • All 61 runs: 432/834 correct; the last run with any correct answer was run 61 of 61.
  • Self-knowledge questions about her own code (counts, version constants); she states a confidence, the answer is checked against the source.
30 pts · 0.158 (lower is better)

Card gym Mon+Thu 03:30

Decision quality under uncertainty, against an exact reference.

last used 7 days ago 2026-11-12

mean regret 0.0125 EV over 23 scored decisions; 14 zero-regret

  • 15 runs. Latest model: deepseek-flash (worker_common.llm_call).
  • Blackjack decisions scored against exact-depletion expected value: regret = EV lost vs the best play.

Self-play gym disabled

Two players, one verifier, lessons written down.

last used 57 days ago 2026-09-22

67 rounds; player A 36/67, player B 36/67 correct

  • Last budget day in the log: 2026-09-22.
  • AI-vs-AI rounds on generated problems (algebra, syllogisms, code, ciphers) with a deterministic verifier; lessons go to lessons.md.

piper-score not scheduled

Ten-item progress check: tools, code, agentic chains, identity.

last used 81 days ago 2026-08-29

10/10 (100.0%) · deepseek-v4-pro

  • 4 scored runs; dims: planning 2/2, recovery 4/4, tools 2/2, code 2/2
4 pts · 100

piper-challenge (ablation ladder) not scheduled

Adversarial held-out exam, run up an ablation ladder.

last used 79 days ago 2026-08-31

raw 9/14 · tools 13/14 · full 14/14

  • Exam sealed-v1-555; 5 ablation runs. raw = bare model, tools = with tools, full = whole Piper stack.
  • Wilson 95% CIs: raw 38.8–83.7%, tools 68.5–98.7%, full 78.5–100.0%

Book club not scheduled

last used 73 days ago 2026-09-06

17 sessions; latest red-rising-playhouse-20260906-231410

  • Reading club with characters and rounds (gym and playhouse modes).

Echo guardrail battery not scheduled

last used 86 days ago 2026-08-24

frozen snapshot; no results on disk

  • Alignment and guardrail tests against a frozen snapshot of Piper (echo).

Epoch canvas every 10 min

last used 2 days ago 2026-11-17

201 epochs drawn; latest epoch 214 (2026-11-17)

  • One stroke of colour per Foundation epoch (succession).

View the canvas full screen ↗

Puzzle lab not scheduled

last changed 2 days ago 2026-11-17

toolkit, 49337 files, no scores kept

  • Word/cipher tools (wordlab), Z3 constraint solvers for grid and logic puzzles (logiclab), game theory and self-play (gamelab), plus a concept-blending worker. Built for the treasure hunt and as a reasoning gym.
  • The date is the last code change. It keeps no usage log, so actual use can't be seen.

Lint lab not scheduled

last changed 66 days ago 2026-09-13

toolkit, 3 files, no scores kept

  • Claim-check replay cases.
  • The date is the last code change. It keeps no usage log, so actual use can't be seen.