The target architect exhausted its 45 tool calls on a first attempt in three of five runs: 31 schema lookups and 29 graph_put calls for one design. Each time it carried on from its working graph, at a cost of four to ten minutes. - terraform_schema takes resource_types, a list, and outlines every argument and block each accepts, by name, in one reply. - The architect is told to make that one call once the Elements are decided, and to build the graph in a handful of calls of ten to twenty items. Case 04 then completed at 100% for the first time, the architect finishing in one attempt with 11 lookups and 13 graph calls. Plans record the run and correct three step statuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.6 KiB
Plan Map
Iteration: 4, learning and Skills
Intent: Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration.
Status: steps 1 to 8, 11, 13 and 15 to 20 built; case 01 completed on them on 2026-10-07 (../iter-06-target-architecture-graph/steps/step-11.md); the other cases have not run, so no criterion is measured across the benchmark; step 9 complete (ADR-0004 accepted 2026-10-08), which unblocks steps 10 and 12; steps 10, 12 and 14 planned. Steps 1 to 6 are in commit 7ee49ab; the rest are uncommitted.
Changed by iteration 6: docs/adr/0005-the-target-architecture-is-an-explicit-graph.md (accepted 2026-10-07) replaces the free-form findings.json of step 1 with a verdict per Check, remove the diagrammer, and add a Phase. Its plan is ../iter-06-target-architecture-graph/, which lists the effect on each step here and should be built before steps 10 to 14.
This directory covers: steps 1 to 27
Sources of truth
- Review report (2026-10-06): https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. Each step's
Report refnames its enhancement number there. The report uses pre-iteration-3 paths. iter-02-solution-architect-agent/steps/step-8.md: the evaluation figures quoted as evidence.docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md: every step must keep behaviour in Skills and vendored Skills unmodified.CONTEXT.md: "every Execution starts clean". Learning arrives as a reviewed Skill change, never as memory carried between Executions.
Status
| Step | What | Group | Size | Status |
|---|---|---|---|---|
| 1 | Structured validator findings | A | S | replaced by iteration 6 step 9 (verdicts per Check) |
| 2 | Carry the draft through needs_input |
A | S | complete |
| 3 | Reuse pillar advice across Revisions | A | S | built; no run has revised design recommendations, so unexercised by evaluation |
| 4 | Single source for Deliverable contracts | A | S | complete |
| 5 | Cap parallel Specialists | A | S | complete |
| 6 | Orchestrator step order | A | S | built; seen working in one stopped run, no completed evaluation |
| 7 | Capture lesson candidates | B | M | complete |
| 8 | Evaluation that can see improvement | B | M | complete: thresholds set and four judged runs done on 2026-10-08 |
| 9 | ADR: the lessons loop and overlay Skills | B | S | complete: ADR-0004 accepted 2026-10-08 |
| 10 | Lessons Skill and lesson curator | B | L | planned |
| 11 | IaC patterns Skill | B | M | built; in case 01 the IaC author fell from 57 requests to 11 with no Revision; other cases unmeasured |
| 12 | Overlay Skills and an organisation standards Skill | C | M | planned |
| 13 | Trim Specialist context | C | M | built; measured on case 01 only, together with steps 11 and 17 |
| 14 | Provider-owned contracts | C | M | planned |
| 15 | Group Executions by Customer case | added | S | complete |
| 16 | Cache Terraform providers between checks | added | S | complete |
| 17 | A working copy for files and for the graph | added | M | complete; in every run since 2026-10-07 |
| 18 | Provider schema lookup and one pinned provider version | added | S | built; checked against the real provider |
| 19 | A Check the code cannot meet goes back to the Target architecture | added | S | built |
| 20 | A model timeout per role | added | S | built |
| 21 | A Deliverable stands while what it was written from is unchanged | added | S | complete; the description was kept on Check-only Revisions in runs of 2026-10-07 to 09 |
| 22 | Validation reports every gap in one pass | added | S | instructions changed; with conditions listed, case 02 ran with 102 Checks and no Revision |
| 23 | Validation recorded a few verdicts at a time | added | S | built; in case 02 a validator run cut short carried on and submitted in 68 seconds, with no timeout |
| 24 | The design checked against the provider's schema, in code | added | S | built; measured on two passed designs and the known faults; raised nothing in case 02, which had no unimplementable Check |
| 25 | An Execution that ends with warnings keeps its files | added | S | complete |
| 26 | Each pillar's checklist is reviewed and the review is in the Deliverable | added | S | built; a judged run of case 04 scored the checklist check at 100% on every pillar |
| 27 | A user sees a Deliverable before it is first submitted | added | S | built; case 04 passed the check on 2026-10-09, with the rule not needing to refuse |
Size: S under a day, M a few days, L a week or more. Group A is cheap fixes that also create signal. Group B is the learning loop. Group C is adaptability.
Dependency graph
- Steps 1 to 6: no dependencies; any order
- Step 7: depends on step 1
- Step 8: no dependencies
- Step 9: no dependencies
- Step 10: depends on step 7, step 8, step 9
- Step 11: depends on step 8
- Step 12: depends on step 9
- Step 13: depends on step 8
- Step 14: depends on step 4
- Step 15: no dependencies (added on 2026-10-06 at the owner's request)
- Step 16: no dependencies (added on 2026-10-07 at the owner's request)
- Steps 17 to 20: no dependencies (added on 2026-10-07 at the owner's request, from the causes of the 60 minute run recorded in
../iter-06-target-architecture-graph/steps/) - Step 11 uses step 18's pinned provider version; step 19 uses step 18's lookup to find the Checks it reports
Step 8 comes before steps 10, 11 and 13 so each of them can be shown to help: same quality, fewer Revisions, seconds or tokens.