Iteration 4: - Overlay Skills (step 12): an Execution names an organisation's standards Skill from the folders the deployment configures; recorded with its version. An example standards Skill and an evaluation case that uses it. - Lessons (step 10): a prove command that compares a run with lessons against one without, and triage of evidence an existing lesson already covers. Iteration 5: - Step 8: the references of a Specialist's Skills its task needs are named in its context as read_first. - Step 14: a remote MCP server's reply that instructs the agent is withheld; the user's own text is flagged as a risk and nothing more. - Step 15: in a Change request, items only reworded are listed apart from real changes. - Step 19: each Execution's design recommendations are judged when it ends and the scores kept in its record. The evaluation takes its rubric from the same place. Each question was calibrated against live Jev; none has run in a case. Steps 6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.9 KiB
Plan Map
Iteration: 4, learning and Skills
Intent: Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration.
Status: steps 1 to 8, 11, 13 and 15 to 20 built; case 01 completed on them on 2026-10-07 (../iter-06-target-architecture-graph/steps/step-11.md); the other cases have not run, so no criterion is measured across the benchmark; step 9 complete (ADR-0004 accepted 2026-10-08), which unblocks steps 10 and 12; steps 10, 12 and 14 planned. Steps 1 to 6 are in commit 7ee49ab; the rest are uncommitted.
Changed by iteration 6: docs/adr/0005-the-target-architecture-is-an-explicit-graph.md (accepted 2026-10-07) replaces the free-form findings.json of step 1 with a verdict per Check, remove the diagrammer, and add a Phase. Its plan is ../iter-06-target-architecture-graph/, which lists the effect on each step here and should be built before steps 10 to 14.
This directory covers: steps 1 to 27
Sources of truth
- Review report (2026-10-06): https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. Each step's
Report refnames its enhancement number there. The report uses pre-iteration-3 paths. iter-02-solution-architect-agent/steps/step-8.md: the evaluation figures quoted as evidence.docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md: every step must keep behaviour in Skills and vendored Skills unmodified.CONTEXT.md: "every Execution starts clean". Learning arrives as a reviewed Skill change, never as memory carried between Executions.
Status
| Step | What | Group | Size | Status |
|---|---|---|---|---|
| 1 | Structured validator findings | A | S | replaced by iteration 6 step 9 (verdicts per Check) |
| 2 | Carry the draft through needs_input |
A | S | complete |
| 3 | Reuse pillar advice across Revisions | A | S | built; no run has revised design recommendations, so unexercised by evaluation |
| 4 | Single source for Deliverable contracts | A | S | complete |
| 5 | Cap parallel Specialists | A | S | complete |
| 6 | Orchestrator step order | A | S | built; seen working in one stopped run, no completed evaluation |
| 7 | Capture lesson candidates | B | M | complete |
| 8 | Evaluation that can see improvement | B | M | complete: thresholds set and four judged runs done on 2026-10-08 |
| 9 | ADR: the lessons loop and overlay Skills | B | S | complete: ADR-0004 accepted 2026-10-08 |
| 10 | Lessons Skill and lesson curator | B | L | the loop is built; six lessons proposed from 21 Executions, approved by the owner and published on 2026-10-09; not yet proved by an evaluation with and without (steps/step-10.md) |
| 11 | IaC patterns Skill | B | M | built; in case 01 the IaC author fell from 57 requests to 11 with no Revision; other cases unmeasured |
| 12 | Overlay Skills and an organisation standards Skill | C | M | built, with an example standards Skill and an evaluation case; not run with real models |
| 13 | Trim Specialist context | C | M | built; measured on case 01 only, together with steps 11 and 17 |
| 14 | Provider-owned contracts | C | M | deferred by its own plan until a second Cloud provider is planned (ADR-0002) |
| 15 | Group Executions by Customer case | added | S | complete |
| 16 | Cache Terraform providers between checks | added | S | complete |
| 17 | A working copy for files and for the graph | added | M | complete; in every run since 2026-10-07 |
| 18 | Provider schema lookup and one pinned provider version | added | S | built; checked against the real provider |
| 19 | A Check the code cannot meet goes back to the Target architecture | added | S | built |
| 20 | A model timeout per role | added | S | built |
| 21 | A Deliverable stands while what it was written from is unchanged | added | S | complete; the description was kept on Check-only Revisions in runs of 2026-10-07 to 09 |
| 22 | Validation reports every gap in one pass | added | S | instructions changed; with conditions listed, case 02 ran with 102 Checks and no Revision |
| 23 | Validation recorded a few verdicts at a time | added | S | built; in case 02 a validator run cut short carried on and submitted in 68 seconds, with no timeout |
| 24 | The design checked against the provider's schema, in code | added | S | built; measured on two passed designs and the known faults; raised nothing in case 02, which had no unimplementable Check |
| 25 | An Execution that ends with warnings keeps its files | added | S | complete |
| 26 | Each pillar's checklist is reviewed and the review is in the Deliverable | added | S | built; a judged run of case 04 scored the checklist check at 100% on every pillar |
| 27 | A user sees a Deliverable before it is first submitted | added | S | built; case 04 passed the check on 2026-10-09, with the rule not needing to refuse |
Size: S under a day, M a few days, L a week or more. Group A is cheap fixes that also create signal. Group B is the learning loop. Group C is adaptability.
Dependency graph
- Steps 1 to 6: no dependencies; any order
- Step 7: depends on step 1
- Step 8: no dependencies
- Step 9: no dependencies
- Step 10: depends on step 7, step 8, step 9
- Step 11: depends on step 8
- Step 12: depends on step 9
- Step 13: depends on step 8
- Step 14: depends on step 4
- Step 15: no dependencies (added on 2026-10-06 at the owner's request)
- Step 16: no dependencies (added on 2026-10-07 at the owner's request)
- Steps 17 to 20: no dependencies (added on 2026-10-07 at the owner's request, from the causes of the 60 minute run recorded in
../iter-06-target-architecture-graph/steps/) - Step 11 uses step 18's pinned provider version; step 19 uses step 18's lookup to find the Checks it reports
Step 8 comes before steps 10, 11 and 13 so each of them can be shown to help: same quality, fewer Revisions, seconds or tokens.