Files
gcp_solution_architecture_a…/.agents/plans/iter-04-learning-and-skills/MAP.md
Jonathan Boniface 58e778ca7a feat: record verdicts in pieces and check the design against the schema
From the case 02 run cancelled at 66 minutes on 2026-10-07:

- The validator records verdicts a few Checks at a time (verdict_put) and
  they are attached on submission. It timed out five times writing them in
  one reply. Verdicts on the same code survive a run cut short; verdicts on
  changed code are dropped. Its model timeout is 300 s.
- A schema review at submission, in code: every setting a Check names is
  looked up on its Element's resource type, and an unknown resource type is
  reported. Two Revisions in that run were for Checks no code could meet.
  On the two designs that passed validation it raises 2 of 96 Checks, both
  one metric name used as a value, so it is returned once and never blocks.
- Reviews run together; the Jev coverage review stays off.

Plans record both steps and the cancelled run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:39:13 +01:00

4.9 KiB

Plan Map

Iteration: 4, learning and Skills Intent: Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration. Status: steps 1 to 8, 11, 13 and 15 to 20 built; case 01 completed on them on 2026-10-07 (../iter-06-target-architecture-graph/steps/step-11.md); the other cases have not run, so no criterion is measured across the benchmark; step 9 drafted and awaiting the owner; steps 10, 12 and 14 planned. Steps 1 to 6 are in commit 7ee49ab; the rest are uncommitted. Changed by iteration 6: docs/adr/0005-the-target-architecture-is-an-explicit-graph.md (accepted 2026-10-07) replaces the free-form findings.json of step 1 with a verdict per Check, remove the diagrammer, and add a Phase. Its plan is ../iter-06-target-architecture-graph/, which lists the effect on each step here and should be built before steps 10 to 14. This directory covers: steps 1 to 24

Sources of truth

  • Review report (2026-10-06): https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. Each step's Report ref names its enhancement number there. The report uses pre-iteration-3 paths.
  • iter-02-solution-architect-agent/steps/step-8.md: the evaluation figures quoted as evidence.
  • docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md: every step must keep behaviour in Skills and vendored Skills unmodified.
  • CONTEXT.md: "every Execution starts clean". Learning arrives as a reviewed Skill change, never as memory carried between Executions.

Status

Step What Group Size Status
1 Structured validator findings A S replaced by iteration 6 step 9 (verdicts per Check)
2 Carry the draft through needs_input A S complete
3 Reuse pillar advice across Revisions A S built; evaluation run outstanding
4 Single source for Deliverable contracts A S complete
5 Cap parallel Specialists A S complete
6 Orchestrator step order A S built; seen working in one stopped run, no completed evaluation
7 Capture lesson candidates B M complete
8 Evaluation that can see improvement B M built; thresholds and a judged run outstanding
9 ADR: the lessons loop and overlay Skills B S drafted as docs/adr/0004-lessons-and-overlays-arrive-as-skills.md, status proposed; awaits the owner
10 Lessons Skill and lesson curator B L planned
11 IaC patterns Skill B M built; in case 01 the IaC author fell from 57 requests to 11 with no Revision; other cases unmeasured
12 Overlay Skills and an organisation standards Skill C M planned
13 Trim Specialist context C M built; measured on case 01 only, together with steps 11 and 17
14 Provider-owned contracts C M planned
15 Group Executions by Customer case added S complete
16 Cache Terraform providers between checks added S complete
17 A working copy for files and for the graph added M built; case 01 ran in 18 minutes against 60, with no model timeout; the architect's lookups then tuned, unrun
18 Provider schema lookup and one pinned provider version added S built; checked against the real provider
19 A Check the code cannot meet goes back to the Target architecture added S built
20 A model timeout per role added S built
21 A Deliverable stands while what it was written from is unchanged added S built; unrun
22 Validation reports every gap in one pass added S instructions changed; unrun
23 Validation recorded a few verdicts at a time added S built; unrun
24 The design checked against the provider's schema, in code added S built; measured on two passed designs and the known faults; unrun in an Execution

Size: S under a day, M a few days, L a week or more. Group A is cheap fixes that also create signal. Group B is the learning loop. Group C is adaptability.

Dependency graph

  • Steps 1 to 6: no dependencies; any order
  • Step 7: depends on step 1
  • Step 8: no dependencies
  • Step 9: no dependencies
  • Step 10: depends on step 7, step 8, step 9
  • Step 11: depends on step 8
  • Step 12: depends on step 9
  • Step 13: depends on step 8
  • Step 14: depends on step 4
  • Step 15: no dependencies (added on 2026-10-06 at the owner's request)
  • Step 16: no dependencies (added on 2026-10-07 at the owner's request)
  • Steps 17 to 20: no dependencies (added on 2026-10-07 at the owner's request, from the causes of the 60 minute run recorded in ../iter-06-target-architecture-graph/steps/)
  • Step 11 uses step 18's pinned provider version; step 19 uses step 18's lookup to find the Checks it reports

Step 8 comes before steps 10, 11 and 13 so each of them can be shown to help: same quality, fewer Revisions, seconds or tokens.