docs: accept ADR-0004, lessons and overlays arrive as Skills
Accepted by the owner on 2026-10-08 with its six decisions as written: a person approves every lesson; a lesson must be shown to help by evaluation; lessons are per Cloud provider; overlay Skills come only from configured directories; each Execution records the Skills it loaded; and lesson candidates are client data kept out of the repository. - CONTEXT.md gains Lesson candidate, Lesson and Overlay Skill. - Iteration 4 step 9 is complete; steps 10 and 12 name the decisions they must meet and are unblocked. Neither is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -88,6 +88,7 @@ Every step ends with the full suite passing. Steps that change agent behaviour (
|
|||||||
- Retire: at each vendor sync, remove lessons the upstream Skill now covers.
|
- Retire: at each vendor sync, remove lessons the upstream Skill now covers.
|
||||||
- **Where:** `skills/<lessons-skill>/`, `agents/lesson-curator.md`, a curator entry point beside `eval/solution_eval.py`, `providers/gcp.yaml`, `scripts/sync-vendored-skills.sh`.
|
- **Where:** `skills/<lessons-skill>/`, `agents/lesson-curator.md`, a curator entry point beside `eval/solution_eval.py`, `providers/gcp.yaml`, `scripts/sync-vendored-skills.sh`.
|
||||||
- **Acceptance criteria:** Vendored Skills stay byte-identical. No lesson reaches a Skill without an approved issue file. Each Execution records the lessons Skill version it loaded. One real lesson has gone through the whole loop with its evaluation comparison recorded.
|
- **Acceptance criteria:** Vendored Skills stay byte-identical. No lesson reaches a Skill without an approved issue file. Each Execution records the lessons Skill version it loaded. One real lesson has gone through the whole loop with its evaluation comparison recorded.
|
||||||
|
- **From ADR-0004 (accepted 2026-10-08):** the lessons Skill is per Cloud provider and listed in that provider's bundle; a lesson is kept only if the evaluation with and without it holds quality and lowers Revisions, time or tokens; a lesson carries no client detail, and lesson candidates stay out of the repository.
|
||||||
|
|
||||||
## Step 11: IaC patterns Skill
|
## Step 11: IaC patterns Skill
|
||||||
|
|
||||||
@@ -104,6 +105,7 @@ Every step ends with the full suite passing. Steps that change agent behaviour (
|
|||||||
- **Rationale:** Skills are fixed at start-up: `AdkAgentRuntime` loads every Skill once, and `role_skills` is static per provider. Only 2 of the 9 Skills are our own, and both describe tools.
|
- **Rationale:** Skills are fixed at start-up: `AdkAgentRuntime` loads every Skill once, and `role_skills` is static per provider. Only 2 of the 9 Skills are our own, and both describe tools.
|
||||||
- **Where:** `app/adk/adk_runtime.py`, `app/adk/roles.py`, `app/providers.py`, `ExecutionEngine.start` in `app/agent.py`, `app/a2a_server.py`, a standards Skill under `skills/`.
|
- **Where:** `app/adk/adk_runtime.py`, `app/adk/roles.py`, `app/providers.py`, `ExecutionEngine.start` in `app/agent.py`, `app/a2a_server.py`, a standards Skill under `skills/`.
|
||||||
- **Acceptance criteria:** An Execution can name overlay Skills at start; each passes the same Agent Skills validation as bundled Skills, and an unknown or invalid one refuses the start. An organisation standards Skill (naming, tagging, approved products) is loaded by the roles the bundle lists.
|
- **Acceptance criteria:** An Execution can name overlay Skills at start; each passes the same Agent Skills validation as bundled Skills, and an unknown or invalid one refuses the start. An organisation standards Skill (naming, tagging, approved products) is loaded by the roles the bundle lists.
|
||||||
|
- **From ADR-0004 (accepted 2026-10-08):** overlay Skills come only from directories the deployment configures; a caller names one by its Skill name and cannot upload one; the Execution records each overlay it loaded with a content hash. Skills are resolved per Execution, not once at start-up.
|
||||||
|
|
||||||
## Step 13: Trim Specialist context
|
## Step 13: Trim Specialist context
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
**Iteration:** 4, learning and Skills
|
**Iteration:** 4, learning and Skills
|
||||||
**Intent:** Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration.
|
**Intent:** Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration.
|
||||||
**Status:** steps 1 to 8, 11, 13 and 15 to 20 built; case 01 completed on them on 2026-10-07 (`../iter-06-target-architecture-graph/steps/step-11.md`); the other cases have not run, so no criterion is measured across the benchmark; step 9 drafted and awaiting the owner; steps 10, 12 and 14 planned. Steps 1 to 6 are in commit `7ee49ab`; the rest are uncommitted.
|
**Status:** steps 1 to 8, 11, 13 and 15 to 20 built; case 01 completed on them on 2026-10-07 (`../iter-06-target-architecture-graph/steps/step-11.md`); the other cases have not run, so no criterion is measured across the benchmark; step 9 complete (ADR-0004 accepted 2026-10-08), which unblocks steps 10 and 12; steps 10, 12 and 14 planned. Steps 1 to 6 are in commit `7ee49ab`; the rest are uncommitted.
|
||||||
**Changed by iteration 6:** `docs/adr/0005-the-target-architecture-is-an-explicit-graph.md` (accepted 2026-10-07) replaces the free-form `findings.json` of step 1 with a verdict per Check, remove the diagrammer, and add a Phase. Its plan is `../iter-06-target-architecture-graph/`, which lists the effect on each step here and should be built before steps 10 to 14.
|
**Changed by iteration 6:** `docs/adr/0005-the-target-architecture-is-an-explicit-graph.md` (accepted 2026-10-07) replaces the free-form `findings.json` of step 1 with a verdict per Check, remove the diagrammer, and add a Phase. Its plan is `../iter-06-target-architecture-graph/`, which lists the effect on each step here and should be built before steps 10 to 14.
|
||||||
**This directory covers:** steps 1 to 24
|
**This directory covers:** steps 1 to 24
|
||||||
|
|
||||||
@@ -25,7 +25,7 @@
|
|||||||
| 6 | Orchestrator step order | A | S | built; seen working in one stopped run, no completed evaluation |
|
| 6 | Orchestrator step order | A | S | built; seen working in one stopped run, no completed evaluation |
|
||||||
| 7 | Capture lesson candidates | B | M | complete |
|
| 7 | Capture lesson candidates | B | M | complete |
|
||||||
| 8 | Evaluation that can see improvement | B | M | built; thresholds and a judged run outstanding |
|
| 8 | Evaluation that can see improvement | B | M | built; thresholds and a judged run outstanding |
|
||||||
| 9 | ADR: the lessons loop and overlay Skills | B | S | drafted as `docs/adr/0004-lessons-and-overlays-arrive-as-skills.md`, status proposed; awaits the owner |
|
| 9 | ADR: the lessons loop and overlay Skills | B | S | complete: ADR-0004 accepted 2026-10-08 |
|
||||||
| 10 | Lessons Skill and lesson curator | B | L | planned |
|
| 10 | Lessons Skill and lesson curator | B | L | planned |
|
||||||
| 11 | IaC patterns Skill | B | M | built; in case 01 the IaC author fell from 57 requests to 11 with no Revision; other cases unmeasured |
|
| 11 | IaC patterns Skill | B | M | built; in case 01 the IaC author fell from 57 requests to 11 with no Revision; other cases unmeasured |
|
||||||
| 12 | Overlay Skills and an organisation standards Skill | C | M | planned |
|
| 12 | Overlay Skills and an organisation standards Skill | C | M | planned |
|
||||||
|
|||||||
16
.agents/plans/iter-04-learning-and-skills/steps/step-9.md
Normal file
16
.agents/plans/iter-04-learning-and-skills/steps/step-9.md
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
# Step 9 — Implementation Detail
|
||||||
|
|
||||||
|
`docs/adr/0004-lessons-and-overlays-arrive-as-skills.md` was accepted by the owner on 2026-10-08, with its six decisions as written:
|
||||||
|
|
||||||
|
1. No lesson takes effect without a person's approval.
|
||||||
|
2. A lesson is kept only if the evaluation, run with and without it, holds quality and lowers Revisions, time or tokens.
|
||||||
|
3. Lessons are per Cloud provider, listed in that provider's bundle.
|
||||||
|
4. Overlay Skills come only from directories the deployment configures; a caller names one and cannot upload one; an unknown name refuses the Execution at the start.
|
||||||
|
5. Every Execution records the Skills it loaded and a content hash of each.
|
||||||
|
6. Lesson candidates are client data: stored with the Execution's records, never in the repository, and a lesson carries no client detail.
|
||||||
|
|
||||||
|
`CONTEXT.md` gained **Lesson candidate**, **Lesson** and **Overlay Skill**. Steps 10 and 12 in `DETAIL.md` were updated to name the decisions they must meet.
|
||||||
|
|
||||||
|
Already in place from earlier steps: lesson candidates are captured per Execution (step 7) under a gitignored folder, and each Execution records the version of every Skill it loaded.
|
||||||
|
|
||||||
|
Not built: the curator and the lessons Skill (step 10), and overlay Skills chosen per Execution (step 12).
|
||||||
12
CONTEXT.md
12
CONTEXT.md
@@ -66,6 +66,18 @@ _Avoid_: prompt template, instructions, playbook
|
|||||||
The hyperscaler a solution architecture targets, such as Google Cloud. Each Cloud provider brings its own Skills, plus the MCP servers it hosts for discovering a user's current environment and grounding recommendations in its documentation.
|
The hyperscaler a solution architecture targets, such as Google Cloud. Each Cloud provider brings its own Skills, plus the MCP servers it hosts for discovering a user's current environment and grounding recommendations in its documentation.
|
||||||
_Avoid_: hyperscaler, cloud, platform, target
|
_Avoid_: hyperscaler, cloud, platform, target
|
||||||
|
|
||||||
|
**Lesson candidate**:
|
||||||
|
Something an Execution got wrong or was corrected on, recorded as it happened: a Revision and its reason, a tool refusal, a user's request for changes. It is raw material and changes nothing by itself.
|
||||||
|
_Avoid_: learning, memory, feedback record
|
||||||
|
|
||||||
|
**Lesson**:
|
||||||
|
A rule distilled from Lesson candidates across Executions, approved by a person and shown by evaluation to help, then published in a Skill. It carries no client detail.
|
||||||
|
_Avoid_: memory, fine-tuning, learned behaviour
|
||||||
|
|
||||||
|
**Overlay Skill**:
|
||||||
|
A Skill that adds an organisation's standards or one engagement's constraints to what the Cloud provider's Skills say. It is named when an Execution starts and comes only from a place the deployment has configured.
|
||||||
|
_Avoid_: custom prompt, override, plugin
|
||||||
|
|
||||||
### Outputs
|
### Outputs
|
||||||
|
|
||||||
**Deliverable**:
|
**Deliverable**:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
status: proposed
|
status: accepted
|
||||||
---
|
---
|
||||||
|
|
||||||
# Lessons and overlays arrive as Skills
|
# Lessons and overlays arrive as Skills
|
||||||
@@ -11,9 +11,9 @@ An Execution starts clean ([CONTEXT.md](../../CONTEXT.md)) unless it is a Change
|
|||||||
|
|
||||||
Vendored Skills stay unmodified ([ADR-0002](0002-skills-are-the-portable-unit-of-behaviour.md)); lessons and overlays sit beside them.
|
Vendored Skills stay unmodified ([ADR-0002](0002-skills-are-the-portable-unit-of-behaviour.md)); lessons and overlays sit beside them.
|
||||||
|
|
||||||
## Decisions to confirm
|
## Decisions
|
||||||
|
|
||||||
This ADR is proposed. Accepting it means agreeing to each of these:
|
Accepted by the owner on 2026-10-08, agreeing to each of these:
|
||||||
|
|
||||||
1. **No lesson takes effect without a person's approval.** The curator proposes; it never edits a Skill that agents load. We prefer slower learning to an agent that rewrites its own instructions.
|
1. **No lesson takes effect without a person's approval.** The curator proposes; it never edits a Skill that agents load. We prefer slower learning to an agent that rewrites its own instructions.
|
||||||
2. **A lesson must be shown to help.** It is kept only if the evaluation, run with and without it, holds quality and lowers Revisions, time or tokens.
|
2. **A lesson must be shown to help.** It is kept only if the evaluation, run with and without it, holds quality and lowers Revisions, time or tokens.
|
||||||
@@ -33,5 +33,5 @@ This ADR is proposed. Accepting it means agreeing to each of these:
|
|||||||
|
|
||||||
- Learning is slow by design: a lesson reaches agents only after capture, proposal, review and an evaluation run.
|
- Learning is slow by design: a lesson reaches agents only after capture, proposal, review and an evaluation run.
|
||||||
- The evaluation must be able to see a difference a Skill makes. Its structural checks cannot, so the judged checks and the thresholds on Revisions, time and tokens become part of the loop.
|
- The evaluation must be able to see a difference a Skill makes. Its structural checks cannot, so the judged checks and the thresholds on Revisions, time and tokens become part of the loop.
|
||||||
- The glossary gains **Lesson candidate**, **Lesson** and **Overlay Skill** once this is accepted.
|
- The glossary gains **Lesson candidate**, **Lesson** and **Overlay Skill**.
|
||||||
- Skills can no longer be loaded once at start-up for all Executions: the runtime must resolve a role's Skills per Execution.
|
- Skills can no longer be loaded once at start-up for all Executions: the runtime must resolve a role's Skills per Execution.
|
||||||
|
|||||||
Reference in New Issue
Block a user