diff --git a/.agents/plans/README.md b/.agents/plans/README.md index 7879b32..14aaa3e 100644 --- a/.agents/plans/README.md +++ b/.agents/plans/README.md @@ -13,7 +13,7 @@ One directory per iteration of this agent. Each holds the plan for that iteratio | 5 | `iter-05-jev-decisions/` | the router, intake, the reviews, pillar selection, Skill routing, the guardrail, meaning-aware change summaries and an assessment of every Execution are built (steps 2–5, 8, 10–12, 14, 15, 19), the newest ran in cases 05 and 06 on 2026-10-09; a lighter model for a first attempt was built, measured and left off (6); context selection and compaction were decided against on the numbers (7, 9); the cloud-read guard and readings as a node are planned (13, 18) | Jev (a hosted decision model) for the agent's closed decisions: checks, routing, context selection. | | 6 | `iter-06-target-architecture-graph/` | complete: all four cases pass with real models, each run twice on 2026-10-08; thresholds set | The Target architecture as one explicit graph of Elements, Relationships and Checks, established before infrastructure code, with validation as a verdict per Check. | | 7 | `iter-07-change-requests/` | complete (2026-10-09): the change case ran with real models and passed | A user restates a request to say how requirements changed; a new Execution starts from a Baseline Execution of the same Customer case and delivers the updated architecture with what changed. | -| 8 | `iter-08-terraform-modules/` | steps 1–5 built on 2026-10-09; no case has run with them; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. | +| 8 | `iter-08-terraform-modules/` | steps 1–5 built and run once each way on 2026-10-09, both at 100%; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. | Numbers record when an iteration was planned, not the order to build in. Build iteration 6 before the remaining steps of iteration 4 (10 to 14) and before iteration 5: it changes the Deliverables, Phases and validation format they build on. Its `MAP.md` lists the effect on each of their steps. diff --git a/.agents/plans/iter-08-terraform-modules/MAP.md b/.agents/plans/iter-08-terraform-modules/MAP.md index b34080f..74fff4f 100644 --- a/.agents/plans/iter-08-terraform-modules/MAP.md +++ b/.agents/plans/iter-08-terraform-modules/MAP.md @@ -2,7 +2,7 @@ **Iteration:** 8, Terraform modules **Intent:** Let generated infrastructure code call a published Terraform module where one replaces several resources that must be wired together, without losing the Requirements chain or opening a way for arbitrary code to be downloaded. -**Status:** steps 1 to 5 built on 2026-10-09; a case run each way is outstanding; steps 6 and 7 wait on those Cloud providers +**Status:** steps 1 to 5 built and run once each way on 2026-10-09; steps 6 and 7 wait on those Cloud providers **This directory covers:** steps 1 to 7 ## The owner's request (2026-10-09) @@ -36,7 +36,7 @@ The owner's decision, 2026-10-09: **plain resources by default, with an opt-in t | 3 | A Skill with the catalogue and the rule for when to use one | built: `skills/google-cloud-iac-modules/`, four references, each validated by a test | | 4 | Checks through a module | built as guidance: `implementation.json` names the module input; the validator confirms what the input sets | | 5 | Opt-in per Execution, plain resources by default | built: `terraform_modules` on a start, `/generate`, A2A metadata and an evaluation case; recorded in `execution.json` | -| 5a | Run a case each way and see what the agents do with it | not run | +| 5a | Run a case each way and see what the agents do with it | run once each way on 2026-10-09, both at 100%; a module was used for private Cloud SQL only | | 6 | Azure: `aztfm` | waits on an Azure provider bundle | | 7 | AWS: `terraform-aws-modules` | waits on an AWS provider bundle | diff --git a/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md b/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md index 854cda1..72f2fa2 100644 --- a/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md +++ b/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md @@ -60,3 +60,30 @@ Reference routing (iteration 5 step 8) does not choose within the modules Skill, - A module version in the catalogue goes stale; nothing flags a newer one. Tests: `tests/tools/test_modules.py` (24), `tests/test_iac_modules.py` (6), `tests/execution/test_terraform_modules.py` (7). + +## Run on 2026-10-09 + +Case 03's request, once each way, side by side (`20261009T210931Z` plain, `20261009T210935Z` opted in as `case-07-approved-modules`). Both completed at 100%. + +| | Plain resources | Modules opted in | +|---|---|---| +| Minutes | 35 | 22 | +| Tokens | 5.66 million | 3.08 million | +| Cost, about | $5.50 | $3.17 | +| Revisions of the Target architecture | 3 | 1 | +| Checks, all passed | 77 | 54 | +| Module calls / resources / lines of Terraform | 0 / 35 / 555 | 2 / 25 / 448 | +| Design quality score | 0.82 | 0.62 | + +**The difference in time and cost is the Revisions, not the modules.** The two runs produced different designs before any code was written: 77 Checks against 54, three Revisions against one. One run each way cannot say modules are cheaper. + +What the opted-in run does show: + +- The IaC author used a module for one thing, private Cloud SQL (`sql-db//modules/postgresql` with `//modules/private_service_access`, 28.3.0), and wrote the network, load balancer, Cloud Run, storage, secrets, IAM and monitoring as plain resources. A VPC with one subnet did not get the network module. +- 14 of 54 Checks are met through module inputs, each recorded as `module.cloud_sql.`; the validator passed all 14 citing the input's value. +- `terraform_module` was called four times by the author and once by the validator. No module was refused, and the code validated. +- For a Check met by leaving an input unset, the validator's evidence is that the module falls back to its default. That rests on the module's documentation. + +The plain run was given neither the Skill nor the tool, and wrote no module. + +The design quality score of 0.62 is below the bar of 0.67. It is scored on the design recommendations, written before any code.