Piper: Skill Compilation
Turning work she has already solved into procedures she no longer has to reason through
Research programme proposal · v0.2.0 · 3 November 2026
Shared copy. Machines other than Firefly are named by their role. Nothing else differs from the full copy.
| Version | Date | What changed |
|---|---|---|
| v0.1.0 | 3 Nov 2026 | First version: the plan, plus the external reviewer's assessment verbatim (Appendix A). Kept unchanged here. |
| v0.2.0 | 3 Nov 2026 | Stage 0 run (offline, read-only). §2 of v0.1.0 was wrong: about 95% of the pre-send log it quoted is her test suite, not real dispatches. She already has a skill compiler of her own, and her skills are almost never read. Skill #1 changes from the acceptance-bar builder to budget-fit, now built and replay-tested offline (§5). |
Status: Stage 0 done, skill #1 built offline, nothing installed. All work so far ran on read-only copies of her logs, off Firefly. Piper has not been told. Nothing is wired in before the freeze ends on 8 November and the surprise-why loop goes in on 9 November (§8).
1. The idea in one paragraph
Piper remembers what worked (episodic memory) and is starting to predict what will work (the change ledger, foresight priors, the JEPA proposal in the companion paper Predictive Self-Improvement Engine). What she does not do is stop paying for work she has already figured out. When a familiar task comes back, her worker reasons through it from scratch and often makes the same mistake as last time. Skill compilation turns a task she has solved many times into a small, tested, versioned program with explicit conditions for when it applies. The model is kept for the parts that are actually new. The question the programme asks is the reviewer's:
Can I solve a problem today in a way that permanently reduces the amount of intelligence needed to solve similar problems tomorrow?
2. What Stage 0 found (offline, read-only, 26 Oct – 3 Nov 2026)
Sources were copied off Firefly and read on another machine: the pre-send, failure, refusal and reuse logs, the work ledger (6,236 deliverables, 5,105 verified), the cost ledger, the budget-guard log and 1,138 worker session transcripts (182 MB). Scripts and outputs are listed in Appendix B.
2.1 She already has a skill compiler. What she lacks is execution and use.
core/skill_compiler.py1.11.0 (2,582 lines, built 25–28 Oct) mines her own transcripts for repeated command procedures. It matches on what a command is for, not its exact text (--normalise), and writes them out as proseSKILL.mdfiles. 14 of the 21 skills in~/.pi/agent/skills/came from it. It also tracks failing steps week by week (--failures --trend).core/skill_suggest.pyis the read side: it matches a command or an objective to an installed skill. Nothing calls it.- All 21 skills are prose. None has code, so using one still costs a full round of reasoning.
- They are almost never opened. In 8 days, 18 of 1,138 worker sessions (1.6%) touched any
SKILL.mdwith a tool call, about 37 reads in all. Her own day sessions read none.
So Stages 3 and 4 of v0.1.0 ("hand her the compiler", "she improves the compiler") are partly done already: she built the miner herself. The missing pieces are (a) compiled procedures that run as code, (b) delivery at the moment of need, not a note the model has to remember to read, and (c) a lifecycle with tests and a measured effect.
2.2 Correction: the acceptance-bar numbers in v0.1.0 were mostly her test suite
v0.1.0 quoted 4,567 pre-send refusals in 8 days. About 95% of those rows come from her own test suite writing into the production log: millisecond bursts, task_id null, fixture objectives such as "do the thing", "Refactor the parser for speed" and d-budgettest (923 rows alone). Matched against objectives that exist as real dispatches or deliverables, real refusals were 61–71, against 918 dispatches sent. She passes the pre-send check on the first try about 92% of the time. Real refusals by verdict: ALREADY_GREEN 37, CANNOT_PASS 9, PRESENCE_ONLY 6, EMPTY_TARGET 5, UNEVALUABLE 4. The acceptance bar is a much smaller problem than it looked, so it drops to a later candidate. Anything else that reads that log has been over-counting by the same factor. That is held for her for after the freeze (§2.5).
2.3 Money is not the constraint
Total spend in the 8 days was $55.26. Workers cost $24.99 over 1,075 runs ($0.023 a run, median 17 model calls). Composing a dispatch costs at most about $0.003. At these prices the reviewer's break-even test (compile cost ÷ saving per use) passes for almost anything that repeats. The useful measures are turns, wall-clock inside a worker's budget, and outcomes, not dollars (§6).
2.4 The family that does qualify: budget refusals
Worker sessions had 23,311 bash steps; 1,267 failed (5.4%), and 55% of sessions had at least one failure. Most failure kinds are a long tail (tracebacks, missing files, grep exit 1). One is a single mechanical procedure:
- Her budget guard refuses a command whose worst case is over the clock: "refused: worst case W s, you have R s … Rerun with timeout at most N seconds." There were 307 such refused steps in 191 sessions (17% of all worker sessions), and they are rising: 8–19 a day in late October, 67 on 2 November and 75 on 3 November. Her guard 1.4.0 (30 Oct) made the advice reachable, but that didn't stop the rise.
- 258 of the 303 logged refusals already declared a
timeout. 154 would have fit the clock and were refused only by the guard's 30-second margin. - The guard already lowers the bound by itself (
clamp), but only for a singletimeout N cmd. That has happened 45 times in total. Multi-part commands (cd X && timeout N …; echo; cat) are refused. - What workers did next: 130 of 311 refused steps were followed by the same command with only the numbers changed. That is the repeated, successful, mechanical behaviour the reviewer said to compile.
- Sessions with a budget refusal ended budget-exhausted 4.2% of the time, against 1.5% without one (8/191 vs 14/947; small numbers).
2.5 Two findings held for her until after the freeze
- The budget guard is blind after a heredoc.
worst_case("cat > x.sh <<'EOF' … EOF\ntimeout 400 python3 run.py")comes out at 1 s, so the command is allowed with 300 s left. 645 of 1,151 worker commands with atimeoutafter a heredoc were undercounted, 367 of them with a bound of 120 s or more. - Her test suite writes to the production pre-send log (§2.2).
Both go on her daily list after 8 November as symptom and repro, with no fix, per the daily-list rule.
3. Corrections to the reviewer's note
- "63 against a bar of 16". These were not checks that fired 63 times. They were 63 refused envelopes in the week after prediction 9 (12–19 October, mostly C21 blast-radius, 42). The data supports compiling the behaviour that passes a check, not the check itself, which is already code. That is the reviewer's own point in §2, and budget-fit follows it: it compiles the worker's recovery, not the guard.
- "Compile the dispatch contract / pre-send tests." Not on current data: real pre-send refusals are rare (§2.2).
- The cited papers were not checked. The reviewer could not find the ATHENA paper. Lifting Traces to Logic (ICML 2026) and Autonomous Skill Development (Oct 2026) are the reviewer's citations and have not been read.
- Rigour level. The reviewer's Phase A–C (30–50 tasks, a hidden split, matched comparison) is proof-grade. After 8 November the operator's standing direction is quick before/after reads around a change. This paper keeps what protects correctness (replay on real cases, mutants, preconditions checked before a skill runs) and drops what only serves proof.
4. Design
4.1 A compiled skill is a folder she already knows how to use
~/.pi/agent/skills/<name>/
SKILL.md # existing convention: when to use, near-miss boundaries, the WHY
run.py # the compiled procedure: explicit inputs -> outputs, no model calls
skill.json # contract + lifecycle (below)
tests/ # replay of real cases + mutants
HISTORY.md # why each version exists (her WikiSkill rule)
skill.json holds the preconditions (checked by run.py before it does anything; if one fails it returns abstain and things go on as today), the postconditions (checked before it returns a result), depends_on (hashes of the code it relies on; a change sends the skill to probation), the state (candidate / active / probation / retired), provenance (who built it, from which traces, at what cost), and stats.
4.2 Delivery: at the hook, not in the prompt
§2.1 shows that a skill the model has to remember to read is rarely read. A compiled skill should therefore be called by the harness at the point where the need appears: a pi tool_call / tool_result hook, a dispatch-path call, a gate's refusal branch. The model still sees what happened (the result carries a one-line note), so it isn't hidden from her. SKILL.md stays for the cases where she does choose to call it.
4.3 Lifecycle
- candidate → active: replay and mutants pass, and a quick live read shows the effect.
- active → probation: 2 failures in its last 10 runs, a
depends_onhash changes, or the shadow sample shows drift. - probation → active/retired: re-tested. Retired skills stay on disk for history and are never run automatically.
- Shadow sample: about 1 in 10 eligible cases still goes the old way, so drift shows up.
5. Stages
Stage 0 · Mine. DONE 3 Nov (offline). Results in §2.
Stage 1 · Skill #1: budget-fit. BUILT offline 3 Nov, candidate. Given a command and the seconds left, it lowers the declared timeout bounds as little as needed so the guard allows the command. It uses the guard's own budget_guard.py as the judge: a timeout word counts only if lowering it lowers the guard's worst case, and a result is returned only if the guard then allows it. So the skill and the guard can't disagree, and a heredoc body or quoted string is never edited. It abstains on unbounded or wrap-up refusals, when no counted bound exists, or when a bound would have to drop below 10 s or below half its declared value (past that, the work probably won't finish, and detaching or splitting it is the worker's call).
| Replay on the 303 logged budget refusals (26 Oct – 3 Nov) | |
|---|---|
| Fitted (guard allows the rewritten command) | 214 |
| Abstained (left for the worker, as today) | 36 |
| Already allowed by the current guard file | 53 |
| Postcondition failures | 0 |
| Bounds kept, median (min) | 83% (50%) of declared |
Mutants (heredoc body, quoted string, loop, &, no timeout, >50% cut, single bound, two segments, nested ssh, wrap-up) |
11 / 11 pass |
The hand-written version doubles as the reviewer's "simple manual procedure" baseline. Her own skill_compiler can be pointed at the same traces later. If it can't produce something equivalent, that is the gap to work on in Stage 3.
Stage 2 · Wire it in (after 9 Nov). A drafted patch to her budget-guard-tool.ts calls budget-fit from the guard's refusal branch, but only for rule: budget, and only when worker_policy.json sets budget_fit: true (default off). On fitted it runs the rewritten command and appends a one-line note to the result. On anything else the refusal stands exactly as today. One item is still open before install: confirming the note reaches the result. Quick read afterwards: budget refusals per day, turns per refused step, budget-exhausted endings.
Stage 3 · Her compiler compiles to code. Extend her skill_compiler from prose output to run.py + skill.json + replay, and wire skill_suggest (or a hook) so that skills are actually used. Then she takes the next family. Candidates seen in Stage 0: settling change_ledger rows (33 dispatches in 8 days, already a prose skill), repairing stored acceptances (23, 6 ended budget-partial), restarting a unit (20), and the ALREADY_GREEN bar (37 real refusals).
Stage 4 · The recursive step. When she changes the compiler, run the old and new versions on the same traces. If the new one produces a skill the old one couldn't, and that skill goes active and measurably helps, that is a concrete improvement chain. It costs one extra offline run.
6. What we measure (quick reads, not windows)
| Measure | Want |
|---|---|
| Refusals per day in the family (skill #1: budget refusals) | down from 67–75/day |
| Turns spent per refused step | toward 0 |
| Fitted commands that then time out at the lowered bound | low; >20% → probation |
| Wrong invocations (caught by postconditions or the shadow sample) | ~0 |
| Abstention rate | reported; very high means preconditions too tight |
| Session endings budget-exhausted / timeout | not up |
| Skill reads/calls per worker session (library-wide) | up from 1.6% |
7. Guard rails, kept short
- No gate loosened. budget-fit only acts after the guard refuses, and the guard re-judges its output.
- Preconditions run before execution, never left to the model's judgement alone.
- Probation on dependency change. budget-fit records the guard's file hash. When her guard changes (for instance a heredoc fix), the skill is re-tested.
- Premature compilation: promote only from repeated evidence (here, 130 hand retries) plus replay and mutants.
- Not a reason to hold back. After 8 November the operator talks to her, tinkers and codes with her freely. Operator or Claude involvement is labelled in provenance, never treated as contamination.
8. Where it sits in the queue
| When | What |
|---|---|
| 3 Nov | Stage 0 done; budget-fit built and replay-tested offline; extension patch drafted (not installed) |
| 8 Nov | freeze ends; the two §2.5 findings go on her daily list |
| 9 Nov 00:05 | surprise-why loop wires in (already scheduled) |
| from ~10 Nov | Stage 2: install budget-fit behind its flag, quick read after a few days |
| ≥ 12 Nov | simplification sphere. If it changes the guard, budget-fit goes to probation and is re-tested |
| then | Stage 3 (compiler → code; skills actually delivered), then Stage 4 |
Compiled skills are plain code, so they survive the model decision around 10 November. Skill compilation goes ahead of JEPA: it needs no new dataset or training.
Relation to the JEPA proposal. JEPA asks what will this change do? Skill compilation asks have I already learned how to do this? A familiar task goes straight to a skill. A new one is deliberated, and JEPA helps there once it exists. Each compiled skill's log (preconditions, verdict, outcome) is also clean labelled data for JEPA.
9. Kill condition
If after 3 weeks of Stage 2 budget refusals per day haven't fallen, or fitted commands time out at their lowered bound more than 20% of the time, retire budget-fit and write down why. If Stage 3 produces no skill of hers that runs as code and is actually used within 4 weeks, the compiler stays a reporting tool and this paper records the negative result.
Appendix B. Stage 0 files (off Firefly)
| File | What |
|---|---|
mine/stage0.py, STAGE0.md |
envelopes, refusals, cost, families, bar trace set (bar_pairs.jsonl 37, bar_verified.jsonl 5,003) |
mine/worker_waste.py, WORKER-WASTE.md |
failing steps in 1,138 worker sessions, measured with her own skill_compiler extractors |
mine/real_refusals.json |
the 61 real pre-send refusals |
skills/budget-fit/ |
run.py, skill.json, SKILL.md, HISTORY.md, tests/replay.py, tests/REPLAY.md |
drafts/budget-guard-tool.budget-fit.patch |
delivery hook, default off, not installed |
Appendix A. External reviewer's assessment (3 November 2026), verbatim
Kept unchanged, as received from the operator. Corrections are in §3.
Skill Compilation for Piper: I like this more than JEPA as a near-term project
My assessment: 9/10 as an architectural direction, but approximately 6/10 for the specific implementation you've described.
I think skill compilation could address one of Piper's most persistent problems: she accumulates experience, but doesn't necessarily become more efficient at performing tasks she's already solved.
The critical idea is:
Piper should not have to pay the full reasoning cost of solving the same problem twice.
That is different from simply remembering how she solved it.
And potentially much more valuable.
One qualification: I couldn't independently locate the exact January 2026 ATHENA paper from the title and quotations you provided. Several unrelated projects use that name. I'm therefore treating your three-tier description as the proposed architecture, rather than attributing unverified findings to that paper.
The broader concept is supported by related research. For example, Lifting Traces to Logic (ICML 2026) demonstrates an approach to converting agent interaction traces into modular programs with explicit conditional logic. That's closely related to what you're proposing for Piper. Proceedings of Machine Learning Research
1. Why this fits Piper particularly well
Consider her existing architecture.
Piper subsystem Current function Skill compilation opportunity Trinity / persistent memory Retains accumulated experience Identify recurring successful procedures Dispatch system Executes engineering work Reuse validated execution patterns Acceptance tests Evaluate completed work Compile recurring verification procedures Change ledger Records modifications and outcomes Supply evidence for skill induction Salience / concern ranking Prioritizes attention Identify opportunities for applying skills RSI machinery Attempts self-improvement Improve how future modifications are performed The distinction is between three forms of learning.
Episodic learning: Piper remembers that a particular modification worked.
Predictive learning: Piper estimates whether a similar modification will work again.
Procedural learning: Piper possesses a validated procedure for performing the modification without reasoning through every step.
The third form is particularly interesting because it changes how Piper performs future work.
Example: Piper encounters a recurring dispatch problem
Currently, the process might look like this:
- Piper identifies a task.
- She reasons about the applicable dispatch rules.
- She examines the acceptance-test requirements.
- She creates an execution plan.
- She performs the task.
- She verifies the result.
Suppose much of that process requires repeated LLM calls.
With skill compilation, Piper could eventually recognize:
"This is the same class of problem I've successfully handled eleven times."
She then invokes an executable procedure that performs the established steps.
The LLM remains responsible for genuinely novel decisions.
The objective isn't to remove reasoning. It's to reserve reasoning for situations that require it.
2. The part I'd change: Don't compile rules. Compile successful behaviours.
This is my biggest criticism of your proposed implementation.
You suggest targeting her dispatch contract and pre-send tests because they fire frequently without producing the desired effect.
That's a reasonable place to look, but it risks repeating an existing architectural problem.
Piper already has considerable governance and validation machinery.
If you compile more checks into deterministic procedures without changing what they accomplish, you could simply make her existing inefficiencies execute faster.
I would instead distinguish three possible compilation targets.
Skill type Example Value Verification skill Execute a known test sequence and validate its result Reliability Execution skill Perform a repeatable engineering procedure Efficiency Cognitive skill Apply a proven method for choosing or improving an engineering approach Potential improvement in decision quality The third category interests me most, although it's also the most difficult to compile safely.
For example, suppose Piper repeatedly succeeds when she follows this pattern:
- Identify two overlapping rules.
- Compare their actual operating conditions.
- Construct a unified replacement.
- Test the replacement against historical cases.
- Remove the redundant rules.
- Verify that no previously permitted behaviour is blocked.
This is directly relevant to your earlier idea about consolidating redundant rules and gates.
Piper could compile the repeated structure into a parameterized procedure.
The LLM would still identify which rules might overlap and judge whether consolidation is appropriate.
But the mechanical comparison, regression testing and verification could become reusable executable components.
That's considerably more useful than simply adding another mandatory check.
3. The architecture I'd build: A procedural learning pipeline
I'd keep the three-tier concept, but add a compiler and a promotion system.
PIPER | Novel problem? | +----------+----------+ | | YES NO | | Tier 3: WILL Search skill library LLM deliberation | | Skill found? | / \ | YES NO | | | | Tier 1: SKILL | | Execute | | | | +----------------+----------+ | Verify outcome | Tier 2: HABITS Store experience | SKILL COMPILER | Find regularities | Generate candidate | Test against history | Independent validation | Promote / reject | Tier 1: SKILLSThe important addition is that a successful execution trace doesn't automatically become a skill.
It becomes evidence from which a reusable skill might be constructed.
That distinction is essential.
The skill compiler should perform five operations
1. Discover: Find recurring patterns across previously successful work.
2. Generalize: Separate what is common across executions from task-specific details.
3. Compile: Convert the common procedure into an executable routine with explicit inputs, outputs and conditions.
4. Verify: Test the new routine on historical cases and independent withheld cases, including failures.
5. Promote: Add it to Piper's active skill library only after it demonstrates acceptable reliability and cost savings.
Each compiled skill should also contain:
- Preconditions specifying when it is applicable.
- Postconditions specifying what successful execution guarantees.
- Explicit failure and abstention conditions.
- A version and provenance record.
- A rollback or recovery procedure.
- A mechanism for detecting when it becomes obsolete.
I would use versioned Python functions or tightly constrained declarative workflows for the first implementation.
They are inspectable, testable and relatively inexpensive to execute.
There's related research worth reviewing here. Autonomous Skill Development (October 2026) describes agents creating verifiable symbolic skills, storing them in versioned libraries and reusing them for deterministic computation. HEVS Publications
4. The innovation I'd add: Skill promotion AND skill demotion
Your earlier interest in getting Piper to eliminate redundant or obsolete rules applies directly here.
A skill library that only grows will eventually become another accumulation problem.
Piper should be able to recognize when a compiled procedure is no longer useful.
I'd establish four states:
State Meaning Candidate Discovered pattern, not yet validated Active Validated procedure available for execution Probation Evidence of degradation, uncertainty or changed conditions Retired No longer eligible for automatic execution Skills could enter probation following repeated failures, dependency changes, conflicting results or degradation against a newer method.
A retired skill would remain available for historical analysis and reproducibility.
Crucially, the skill's applicability conditions should be checked before the procedure runs, not left entirely to the LLM.
If those conditions are not satisfied, Piper returns to deliberation.
That prevents a learned habit from becoming an inflexible rule.
5. The relationship between JEPA, JEV and skill compilation
This is where I think the broader architecture becomes interesting.
Assuming JEV is your proposed decision-selection mechanism, you could organize the three systems as follows:
System Question answered JEPA What consequences should I expect? JEV Which action should I choose? Skill compiler Have I learned how to perform this efficiently? But I would avoid making them three mandatory stages for every decision.
For a familiar, low-risk task, the system might go straight to a validated skill.
For an unfamiliar or high-risk task, it should deliberate, consider predictions, choose an action and verify the outcome.
The skill compiler works primarily afterward, studying the traces and determining whether a reusable procedure exists.
A good architecture reduces unnecessary deliberation rather than running every cognitive subsystem on every task.
This also creates a useful division between two kinds of uncertainty.
JEPA can estimate uncertainty about an action's consequences.
The skill system can determine whether a previously validated procedure applies to the current situation.
If either uncertainty is excessive, Piper should deliberate rather than execute automatically.
6. The experiment I would run first
I would not immediately turn the compiler loose on Piper's dispatch contract.
Instead, I'd identify one narrow, repetitive task family with reliable historical evidence.
The repeated acceptance-test workflow would be a reasonable candidate, provided you first address the vacuous-test problem identified in the earlier proposal.
Experiment: Can Piper compile a reusable verification procedure?
Phase A — Baseline
Select approximately 30–50 representative, independent verification tasks.
Measure Piper performing them through her existing process.
Record total tokens, LLM calls, elapsed time, verification accuracy and failures.
Phase B — Compilation
Give the compiler several successful and unsuccessful traces from a separate development set.
Have it construct a reusable verification procedure.
Validate the resulting procedure against hidden positive and negative cases.
In particular, use deliberately broken implementations to confirm the procedure detects failures.
Phase C — Comparison
Run the original and compiled approaches on matched tasks.
Evaluate:
Metric Desired result LLM token consumption Lower Execution time Lower Verification accuracy Equal or better Missed regressions No increase Incorrect skill invocation Near zero Recovery from unexpected inputs Equal or better Total cost, including compilation Lower over repeated use Also compare against a simple, manually written procedure.
Otherwise, you might demonstrate only that deterministic code is cheaper than LLM deliberation, rather than that Piper has learned to produce useful procedures herself.
A critical measurement: Break-even point
Compiling a skill has a cost.
Suppose:
- Discovering, generating and validating a skill costs $2.
- Each ordinary execution costs $0.05.
- Each compiled execution costs $0.002.
The skill saves $0.048 per invocation.
It would take approximately 42 invocations to recover the initial compilation cost.
That means a skill used twice might not be worth compiling, even if it works perfectly.
I'd make expected reuse frequency part of the promotion decision.
For Piper, whose operating budget matters, this is especially relevant.
7. The connection to RSI is stronger than it first appears
Consider three progressively more interesting outcomes.
Level 1: Piper compiles a skill
Piper discovers that she repeatedly performs the same verification procedure.
She creates a validated executable version.
Future verification becomes cheaper.
Result: Procedural learning and efficiency improvement.
Useful, but not necessarily recursive.
Level 2: Piper compiles a skill that improves her engineering
Piper discovers that a particular method of debugging repeatedly produces successful results.
She compiles its reusable components.
Future engineering tasks become more efficient, enabling more attempts within the same budget.
Result: An improvement to her engineering process.
This could produce persistent capability gains, but would still require evidence that those gains actually occur.
Level 3: Piper improves her own skill compiler
Now imagine Piper notices that her compiler struggles to recognize equivalent procedures with different variable names.
She independently modifies its generalization mechanism.
The modification passes independent evaluation.
The improved compiler subsequently discovers a useful skill that the previous compiler couldn't generate.
That new skill improves another part of her engineering system.
Now you have a plausible candidate for a causally supported recursive improvement chain.
You would still need to establish the counterfactual: would the additional skill have been discovered without the compiler modification?
But the chain is much more concrete than a general claim that Piper is becoming smarter.
8. One concern: The 63-versus-16 measurement
You mentioned that Piper's pre-send tests fired 63 times against a bar of 16, but were read rather than used.
I would be careful with that interpretation.
A high firing count establishes exposure, not effectiveness.
It doesn't establish whether Piper ignored the tests, whether they provided no new information, or whether they prevented mistakes that would otherwise have occurred.
Before compiling those checks into procedures, I'd measure:
How many decisions would have differed if the tests had not run?
Then measure whether those differences would have improved or worsened outcomes.
If the answer is essentially zero, the appropriate intervention might be eliminating redundant checks rather than compiling them.
This distinction matters because your goal is not to increase procedural activity.
It's to improve outcomes while reducing unnecessary computation.
9. My biggest concern: Premature compilation
Here's a failure mode I would specifically warn against.
Piper encounters a difficult problem, eventually solves it, and decides the solution should become a reusable skill.
But perhaps the solution worked only because of unusual circumstances.
If she compiles it too aggressively, she could turn a one-off workaround into a permanent procedure.
The resulting system might become faster but less adaptable.
This is why I'd require:
Repeated evidence + generalization tests + explicit applicability conditions before promotion.
I would also reserve some recurring executions for the original deliberative process.
That gives Piper an ongoing comparison against which to detect skill degradation.
The worst outcome would be a system that becomes increasingly efficient at repeating obsolete solutions.
10. Where I'd place this among Piper's potential improvements
Project Primary benefit Implementation difficulty Near-term measurability JEPA predictive world model Anticipating consequences High Moderate Skill compilation Reducing repeated reasoning Moderate High Automated rule consolidation Reducing architectural complexity Moderate High Counterfactual decision engine Improving action selection High Moderate I'd put skill compilation ahead of JEPA in the development queue.
Not because it has greater theoretical potential, but because it appears easier to implement, validate and evaluate with Piper's existing data.
It could also help make JEPA development cheaper later.
A successful skill compiler might automate portions of dataset validation, evaluation, experiment setup and regression testing.
However, I would initially keep it separate from changes to Piper's governing mechanisms.
That preserves the integrity of the current RSI measurements.
My recommendation
Build a self-compiling procedural memory system, not merely another collection of reusable functions.
The difference is that Piper herself should eventually be able to:
- Recognize repeated successful behaviour.
- Identify which parts are generalizable.
- Produce a candidate executable procedure.
- Verify that procedure independently.
- Promote it into her active skill library.
- Recognize when it becomes obsolete and replace or retire it.
The first few skills may be mundane.
That's fine.
The objective is to establish a reliable mechanism through which experience can be transformed into enduring, reusable capability.
The question I'd ultimately want Piper to answer
"Can I solve a problem today in a way that permanently reduces the amount of intelligence required to solve similar problems tomorrow?"
That's a meaningful form of cumulative procedural learning.
And if she eventually learns how to improve the machinery performing that compilation, you have another potential route toward recursive self-improvement.
Bottom line: I consider skill compilation a stronger near-term engineering investment than JEPA. I'd run it as an isolated experiment after November 8, beginning with one repetitive verification task family and requiring evidence of both correctness and net cost savings before integrating it into Piper's normal workflow.