Files
gcp_solution_architecture_a…/.agents/plans/iter-04-learning-and-skills/steps/step-10.md
Jonathan Boniface a7bc5310f3 feat: the lessons loop, and six lessons approved and published
Capture, distil, review, publish, retire (ADR-0004). Lesson candidates of
past Executions are grouped into what recurred; a lesson-curator role,
outside any Execution, proposes a lesson per group worth one; each proposal
is an issue file a person approves or rejects; an approved lesson is
written into the provider's lessons Skill, which a Specialist is given only
once it holds lessons for that Specialist's Deliverable. LESSONS_ENABLED
off runs without them, for an evaluation with and without.

The curator proposed six lessons from 21 Executions. The owner approved all
six on 2026-10-09 and they are published: five for the target architect,
one for the IaC author. None is proved by an evaluation yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:19:32 +01:00

4.1 KiB

Step 10 — Implementation Detail

Built on 2026-10-09: the loop, and its first six lessons, which the owner approved the same day and which are published. None is proved yet: no evaluation has run with and without them, so the step's last acceptance criterion is open.

The loop

Stage Where What it does
Capture app/states/lessons.py (step 7) Each Execution records its lesson candidates.
Distil app/learning/evidence.py Reads every Execution's candidates where it wrote them and groups what recurred in at least two Executions, with ids and values removed from the description. A refusal the Execution's own permissions caused, and the router's own questions, are not evidence.
app/learning/curator.py, agents/lesson-curator.md The lesson-curator role, outside any Execution, proposes a lesson for each group worth one, or says why not. A proposal naming a Customer case or an Execution is refused.
Review app/learning/proposals.py Each proposal is .scratch/lessons/issues/NN-<slug>.md, Status: needs-triage, with its rule, reason and evidence as counts. A person sets ready-for-agent to approve or wontfix to reject.
Publish app/learning/skill.py Writes an approved lesson into skills/google-cloud-lessons/references/<deliverable>.md and rewrites the index in SKILL.md. Refuses anything not approved.
Prove LESSONS_ENABLED false leaves the lessons Skill out of every role, so the evaluation can run with and without. The comparison goes under the proposal's ## Proof.
Retire app/learning/skill.py Removes a lesson by id; an emptied reference is deleted.

eval/curate_lessons.py runs it: propose, status, publish, retire L-NN.

A Specialist is given the lessons Skill only once it holds a reference for that Specialist's Deliverable (load_roles), so nobody loads an empty Skill. The provider bundle names the Skill (lessons_skill), which makes lessons per Cloud provider. The protocol tells a Specialist with a lessons Skill to read its Deliverable's reference before drafting. Each Execution already records the version of every Skill it loaded.

Differs from the plan

  • A lesson's source is its proposal, not an Execution. The plan said each lesson records its source Execution. An Execution id names a Customer case, and ADR-0004 says a lesson carries no client detail, so the Skill records the proposal number and the proposal records how many Executions the evidence came from.
  • Retiring at each vendor sync is not automated. retire exists; nothing yet compares lessons with a new upstream Skill.
  • Proving is a manual comparison of two evaluation runs. Nothing computes it or writes it into the proposal.

First proposals

Run on 2026-10-09 over the candidates of 21 Executions (eval/results/), which held 14 recurring problems. The curator proposed six and passed over eight:

Id Deliverable Lesson Evidence
L-01 Target architecture A Check for every explicit constraint in a requirement, not one broad Check 14 of 21 Executions, 58 times
L-02 Target architecture Every Element has a Check before submitting 13 of 21, 13 times
L-03 Target architecture Confirm the resource type supports the argument a Check names 3 of 21
L-04 Target architecture Name the block's argument, not the expected value, as the setting 2 of 21
L-05 Deployment guidance Include required arguments in alert-policy blocks 2 of 21
L-06 Target architecture not_applicable is for recommendations, never requirements 2 of 21

The owner approved all six on 2026-10-09 and they were published: five for the target architect, one for the IaC author, which are now the two roles given the lessons Skill. Several restate what instructions or checks already say (L-02 is refused at submission today; L-03 is what the schema review does), which is the kind of thing a reviewer should weigh: a lesson costs tokens on every Execution.

Tests

tests/test_lessons_loop.py (20): grouping, proposals as files, the approval gate, publish and retire, the curator with a scripted model, and which roles are given the Skill.