Skip to content
UpSkill Design System

Self-improving loops

What it is

Several mechanisms in this system change their own behavior, or their own record of the truth, based on evidence gathered while the system runs — rather than staying fixed the way a plain rule or a plain gate does. This page is the family portrait: it names the four groups these mechanisms fall into, points at the page or decision record (ADR) that owns each one’s full mechanics, and draws the line between a loop that learns and a gate that only checks.

The four groups:

Group What changes, and from what evidence
Learning loops A finding from a real review becomes a permanent addition to a contract — component metadata, a token convention, or a curated list — so the next agent inherits it as a checked fact.
Measurement loops A committed, append-only record of outcomes accumulates so a costly step (an adversarial reviewer, a docs critic) can be judged on real numbers instead of defended by intuition.
Self-retiring ledgers A tracked list of known debt is designed to shrink to empty, and mechanically forbidden from growing back once new work no longer needs it.
Self-stabilizers The system polices its own upkeep — its instruction budget, its own diffs, its own documentation’s readability — with a script or subagent rather than a person remembering to check.

Why it’s built this way

None of these loops were designed in one sitting as “a self-improvement system.” Each one is the fix for a specific failure this project already lived through, generalized just far enough to be reusable. CLAUDE.md grew past its adherence budget before ADR-017 capped it; .claude/handoff/ mixed live and dead work before ADR-015 gave it a lifecycle; a review finding used to die in a PR comment before /extract-learnings gave it a permanent home; a plausible-sounding idea (feed the cross-component pattern file into every generation task) turned out to help one task and hurt another before ADR-013’s harness measured the split.

The common thread is the project’s standing principle, stated plainly in Context engineering: measure, don’t assert. A rule that “more context helps” or “the adversarial stage is worth it” is a belief until something records what actually happened. These four groups are the four different shapes that measuring can take: routing a finding into a durable artifact, keeping receipts in a ledger, letting a tracked list empty out, or having the system check its own diffs.

It matters that these are documented as one family rather than four unrelated facts, because the same anti-pattern threatens all of them: a loop that nobody re-reads decays back into a plain rule nobody trusts. Naming them together is itself part of keeping them honest.

How it works, concretely

Learning loops

/extract-learnings is the engine: it reads a component’s .review.json after an adversarial review and routes each finding to the narrowest durable home — ARIA and keyboard findings into the component’s spec (accessibility.ariaAttributes, accessibility.keyboardInteractions; see Machine-readable metadata), consumer misuse into metadata usage.antiPatterns, token-convention violations into /tokens-author’s Conventions section, contrast misses into the curated PAIRS array in scripts/token-contrast-check.js (never a silent waiver). Running --all widens the scope one notch further: a pattern repeating across two or more components may be proposed as a CLAUDE.md addition — never auto-applied, always confirmed by the developer. A .learnings.json file marks a run as processed; scripts/sense.js reads its absence to populate the pending-learnings backlog in .claude/STATUS_QUO.md, so unprocessed findings stay visible rather than quietly forgotten.

The docs-scribe critic (ADR-018) is the same shape applied to prose instead of code. It is a read-only subagent (.claude/agents/docs-scribe.md) spawned once per real-drift /docs-sync run, judging the rewritten sections for three audiences — product manager, product designer, software engineer — against docs/08-glossary.md as the term canon. Its finding types are a closed set (undefined-term, undefined-prerequisite, glossary-gap, buried-lead, sentence-density, missing-example, audience-mismatch) — it cannot invent a new kind of complaint. A glossary-gap finding is a proposal only: a human confirms before the glossary actually grows, the same “propose, don’t auto-apply” discipline --all uses. The prerequisite-chain rule (recurse one level into a definition’s own vocabulary, no deeper) exists because of a real failure: a glossary fix for “Glob pattern” once passed review while quietly leaning on an undefined “wildcard,” which itself leaned on an undefined “file path.” See Agentic moments for the full moment-9 procedure.

flowchart LR
    CHECK["docs:check<br/>CI, staleness detection"] --> TRIGGER(["/docs-sync"]):::command
    TRIGGER --> REWRITE["Rewrite stale sections<br/>main session"]
    REWRITE --> SCRIBE["docs-scribe review<br/>read-only subagent"]
    SCRIBE --> APPLY["Apply accepted findings<br/>+ open PR docs-sync/date"]

    APPLY -.-> PROPOSE["glossary-gap proposal"]
    PROPOSE -.->|human confirms| GLOSSARY["docs/08-glossary.md<br/>term canon"]
    GLOSSARY -.-> SCRIBE

    classDef command fill:#fff,stroke:#333,stroke-width:1px

The shape mirrors /add-component’s loop in Agentic moments: a deterministic check hands off to a bounded LLM pipeline, a read-only reviewer stage reports findings rather than fixing anything itself, and the one finding type that can grow a durable contract (glossary-gap) loops back — after human confirmation — into the exact artifact (docs/08-glossary.md) the next reviewer run reads as its term canon. The difference is what closes the loop: /extract-learnings is a separate, later-triggered command, while here the glossary update happens inside the same /docs-sync run, gated only by the developer’s confirmation before the PR opens.

Measurement loops

.claude/handoff/run-ledger.json is the receipts drawer. It is append-only and committed, promoted from the gitignored per-run <Name>.run.json files by npm run handoff:tidy (scripts/handoff-tidy.js). Its most load-bearing field is reviewerFindingsBeyondGateCount — the number of findings a /review-component adversarial reviewer caught that the deterministic gate (metadata validation, typecheck, build, a11y) could not. That single number across every logged run is the empirical answer to “does the adversarial stage earn its Claude-Pro usage-window cost,” a question ADR-007 leaves open on purpose rather than settling by argument. Since ADR-029 each entry also records path (full or standard) and risk (interactive or display), because the reviewer now spawns only for interactive components. A later readout can compare findings-beyond-gate with and without the reviewer, per risk tier. Entries from before the change were back-filled as full. ADR-018 commits the docs-scribe critic to being judged the identical way once it has 2–3 runs behind it.

The pattern-accuracy harness (scripts/pattern-accuracy-harness/) is the same instinct run as a formal experiment rather than an ongoing tally: a pre-registered 7-task, 2-arm A/B (with and without component-patterns.json in the prompt), scored by score.js against the repo’s real deterministic gates plus a trap checklist. It found the pattern file helps layout/composition tasks (violations roughly halved) and hurts scaffold tasks (violations rose) — the honest, mixed result behind ADR-013’s “layout-generation only” rule. A later amendment fixed a scorer false positive (an @attr-position exemption in the trap checker) and rescored all 14 cells from the retained outputs without rerunning the tasks — the instrument correcting itself, and the correction is documented as a dated ADR amendment rather than a silent edit. Since the ADR-027 spec rollout, the harness is frozen: its tasks and scorer assume metadata that still carried states, tokens and accessibility, so run.js refuses to run on a later commit and points to a worktree at the commit the results were measured on (ADR-029 amendment 2026-10-12). It stays in the repo as the evidence behind the decision, not as a tool for the current architecture; a new question gets a new, pre-registered eval.

Self-retiring ledgers

Two tracked-debt lists are built to shrink to nothing and stay there. scripts/a11y-backlog.json waived pre-existing interactive components that lacked a behavioral accessibility test; the backlog is now [], and — enforced by ADR-008, see Accessibility — a new interactive component is not allowed to be added to it, only required to ship its test. scripts/token-contrast-waivers.json works the same way for known contrast failures: each entry carries a reason and a GitHub issue, and any new contrast miss is routed through /extract-learnings into the curated PAIRS list, never quietly waived. Both lists are evidence the system holds itself to “waivers shrink, they don’t grow” as a mechanical rule, not a promise.

Self-stabilizers

These keep the system’s own instruction surface and documentation lean without a person remembering to police it. npm run claudemd:check fails CI when CLAUDE.md exceeds 200 lines / 20KB, when AGENTS.md exceeds 8KB or the two together exceed 24KB (ADR-029), or when a nested package AGENTS.md lacks the sibling CLAUDE.md that makes Claude Code load it — ADR-017. scripts/sense.js’s writeIfChanged helper blanks wall-clock-only fields before writing, so re-running npm run sense with no real change produces zero diff instead of manufacturing one every day.

The clock-reset convention is this same idea applied to the docs site itself, and it is used in every page’s frontmatter without ever being explained until now. Each docs/NN-*.md page declares its load-bearing sources: in frontmatter; npm run docs:check fails when any listed source has a commit newer than the page, flagging it stale. Sometimes a source changed but the page’s prose is still fully accurate — nothing in it needs rewriting. Rather than force a cosmetic rewrite just to touch the file, the convention is to append a one-line # comment inside the frontmatter block: # clock reset YYYY-MM-DD: <what changed> <why the page is unaffected>. That comment’s own commit becomes the doc’s “last touched” timestamp, resetting docs:check’s staleness clock honestly — the page was reviewed against the change and found accurate, which is different from a rewrite (where the prose itself changes) and different from ignoring the failure (where nobody looked). You can see the convention used repeatedly at the top of this page’s sibling pages, for example 09 — Context engineering’s frontmatter.

Screenshot baselines (ADR-019) close the loop with trust promoted on evidence: re-baselining a component’s canonical screenshot is human-approved, the CI perceptual-diff step runs advisory (continue-on-error) rather than blocking because cross-OS antialiasing makes local and CI ratios differ, and the documented escalation path — CI-generated baselines first, then blocking once diff ratios are quiet — only tightens the gate once the evidence says it is safe to.

Pure gates, for contrast

Not everything that runs automatically is a learning loop. npm run metadata:validate, npm run layout:validate, and the docs:check staleness detection itself are pure gates: they check a fixed rule against the current state and pass or fail, but they never route a finding anywhere, never accumulate evidence, and never change what they check for next time. They are backstops the loops above depend on, not instances of the pattern this page describes.

Summary table

Mechanism Feedback signal Durable home it updates ADR
/extract-learnings Adversarial-review findings (.review.json) Component metadata, /tokens-author conventions, contrast PAIRS 007
Docs-scribe critic First-read comprehension findings on rewritten doc sections Rewritten prose, proposed glossary entries 018
run-ledger.json Per-run gate + reviewer telemetry (reviewerFindingsBeyondGateCount) Append-only ledger, judges the adversarial stage’s ROI 015, 018
Pattern-accuracy harness Pre-registered 7-task × 2-arm A/B scored against real gates component-patterns.json’s consumption rule (/layout-generation only) 013
a11y-backlog.json Behavioral test coverage per component Shrinking waiver list, now [] 008
token-contrast-waivers.json Known contrast failures with a tracked issue Shrinking waiver list; new misses go to PAIRS instead 008
claudemd:check Line/byte count of CLAUDE.md and AGENTS.md; cross-file duplicate lines; paths: presence on rules Enforced budget on the always-loaded instruction files 017
Clock-reset convention A source changed but the page’s prose didn’t Frontmatter comment resetting docs:check’s staleness clock — (convention, no dedicated ADR)
Screenshot baselines Perceptual diff ratio, advisory in CI Committed PNG baselines; escalation path to blocking 019