Files
gcp_solution_architecture_a…/docs/adr/0004-lessons-and-overlays-arrive-as-skills.md
Jonathan Boniface c9f59be274 docs: accept ADR-0004, lessons and overlays arrive as Skills
Accepted by the owner on 2026-10-08 with its six decisions as written:
a person approves every lesson; a lesson must be shown to help by
evaluation; lessons are per Cloud provider; overlay Skills come only from
configured directories; each Execution records the Skills it loaded; and
lesson candidates are client data kept out of the repository.

- CONTEXT.md gains Lesson candidate, Lesson and Overlay Skill.
- Iteration 4 step 9 is complete; steps 10 and 12 name the decisions they
  must meet and are unblocked. Neither is built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:05:03 +01:00

3.6 KiB

status
status
accepted

Lessons and overlays arrive as Skills

An Execution starts clean (CONTEXT.md) unless it is a Change request with a named Baseline (ADR-0006), so nothing one Execution learns may flow into the next as hidden state. What an Execution gets wrong is still worth keeping. We decide that everything that changes how agents behave between Executions is a Skill change, of one of two kinds.

  • Lessons. Each Execution records lesson candidates: Revisions and their reasons, validator findings, tool refusals, the user's change requests, and work that stopped. An offline curator proposes lessons from candidates across Executions. A person approves each one. An approved lesson is added to a lessons Skill that the provider bundle assigns to roles.
  • Overlays. An organisation's standards or one engagement's constraints are written as their own Skills. The provider bundle lists the ones every Execution loads. A caller may name further overlay Skills when it starts an Execution.

Vendored Skills stay unmodified (ADR-0002); lessons and overlays sit beside them.

Decisions

Accepted by the owner on 2026-10-08, agreeing to each of these:

  1. No lesson takes effect without a person's approval. The curator proposes; it never edits a Skill that agents load. We prefer slower learning to an agent that rewrites its own instructions.
  2. A lesson must be shown to help. It is kept only if the evaluation, run with and without it, holds quality and lowers Revisions, time or tokens.
  3. Lessons are per Cloud provider. The lessons Skill is listed in that provider's bundle, since most lessons name its products.
  4. Overlay Skills come only from directories the deployment configures. A caller names an overlay by its Skill name; it cannot upload one. Each passes the same Agent Skills validation as bundled Skills, and an unknown name refuses the Execution at the start.
  5. Every Execution records the Skills it loaded and a content hash of each, so a result can be traced to the instructions that produced it.
  6. Lesson candidates are client data. They quote user feedback and name resources. They are stored with the Execution's records, never in the repository, and a lesson is written so that it carries no client detail.

Considered Options

  • Memory carried between Executions (a vector store of past Deliverables or feedback, retrieved at run time): rejected. It breaks "every Execution starts clean", can leak one client's content into another's Execution, and cannot be reviewed or rolled back as a diff.
  • Automatic Skill edits (the curator applies lessons directly): rejected for now. Without review, one bad Execution can degrade every later one, and nothing ties a regression to its cause.
  • Editing the vendored Skills in place: rejected by ADR-0002. Upstream changes must arrive as a clean diff.
  • Overlays as prompt text passed by the caller: rejected. Free text bypasses Skill validation and progressive loading, and it is an injection path.

Consequences

  • Learning is slow by design: a lesson reaches agents only after capture, proposal, review and an evaluation run.
  • The evaluation must be able to see a difference a Skill makes. Its structural checks cannot, so the judged checks and the thresholds on Revisions, time and tokens become part of the loop.
  • The glossary gains Lesson candidate, Lesson and Overlay Skill.
  • Skills can no longer be loaded once at start-up for all Executions: the runtime must resolve a role's Skills per Execution.