feat: opt-in published Terraform modules from approved sources (iteration 8)

Infrastructure code is plain resources by default. An Execution may opt in
with terraform_modules; the deployment default is TERRAFORM_MODULES, off.

- A module is accepted only from the registry namespaces the provider bundle
  lists (terraform-google-modules), at one exact version. Any other source is
  refused before Terraform runs; until now nothing stopped generated code
  naming any source for terraform init to download.
- terraform_module looks up a module's inputs, outputs and submodules from
  the registry and says when it cannot be used with the pinned provider.
- skills/google-cloud-iac-modules: when a module fits, and four modules with
  examples a test validates against hashicorp/google 8.6.0.
- Opted in, the IaC author and validator get the Skill and the lookup. Not
  opted in, they get neither and a module block is refused in code.
- A Change request follows its Baseline. execution.json records the choice.
- case-07 runs case 03's request with modules opted into.

Azure (aztfm) and AWS (terraform-aws-modules) sources are recorded in the
plan for when those Cloud providers are added. No case has run with this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 22:09:29 +01:00
parent a8ed21b89d
commit ef37f59bad
35 changed files with 1482 additions and 7 deletions

View File

@@ -13,6 +13,7 @@ One directory per iteration of this agent. Each holds the plan for that iteratio
| 5 | `iter-05-jev-decisions/` | the router, intake, the reviews, pillar selection, Skill routing, the guardrail, meaning-aware change summaries and an assessment of every Execution are built (steps 2–5, 8, 10–12, 14, 15, 19), the newest ran in cases 05 and 06 on 2026-10-09; a lighter model for a first attempt was built, measured and left off (6); context selection and compaction were decided against on the numbers (7, 9); the cloud-read guard and readings as a node are planned (13, 18) | Jev (a hosted decision model) for the agent's closed decisions: checks, routing, context selection. |
| 6 | `iter-06-target-architecture-graph/` | complete: all four cases pass with real models, each run twice on 2026-10-08; thresholds set | The Target architecture as one explicit graph of Elements, Relationships and Checks, established before infrastructure code, with validation as a verdict per Check. |
| 7 | `iter-07-change-requests/` | complete (2026-10-09): the change case ran with real models and passed | A user restates a request to say how requirements changed; a new Execution starts from a Baseline Execution of the same Customer case and delivers the updated architecture with what changed. |
| 8 | `iter-08-terraform-modules/` | steps 1–5 built on 2026-10-09; no case has run with them; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. |
Numbers record when an iteration was planned, not the order to build in. Build iteration 6 before the remaining steps of iteration 4 (10 to 14) and before iteration 5: it changes the Deliverables, Phases and validation format they build on. Its `MAP.md` lists the effect on each of their steps.

View File

@@ -0,0 +1,43 @@
# Plan Map
**Iteration:** 8, Terraform modules
**Intent:** Let generated infrastructure code call a published Terraform module where one replaces several resources that must be wired together, without losing the Requirements chain or opening a way for arbitrary code to be downloaded.
**Status:** steps 1 to 5 built on 2026-10-09; a case run each way is outstanding; steps 6 and 7 wait on those Cloud providers
**This directory covers:** steps 1 to 7
## The owner's request (2026-10-09)
Use modules, when necessary, from three organisations:
| Cloud provider | Source | Here |
|---|---|---|
| Google Cloud | https://github.com/terraform-google-modules | built: the `terraform-google-modules` registry namespace is allowed in `providers/gcp.yaml` |
| Azure | https://github.com/aztfm | recorded for step 6: there is no Azure provider bundle |
| AWS | https://github.com/terraform-aws-modules | recorded for step 7: there is no AWS provider bundle |
The owner's decision, 2026-10-09: **plain resources by default, with an opt-in to known approved sources.**
- **Opt-in per Execution.** A caller sets `terraform_modules`; the deployment's default is `TERRAFORM_MODULES`, off. A Change request does as its Baseline did unless told otherwise.
- **Approved sources are the provider bundle's list.** Opted in, a module is still used only from the tested catalogue and only where it replaces three or more resources that depend on each other.
- **Google now.** The tooling names no Cloud provider; the bundle and a Skill do (ADR-0002).
## What was found first
- `terraform init` downloads and evaluates whatever a `source` names. Before this iteration nothing stopped generated code naming any Git URL.
- A module carries its own provider requirements. Of 23 modules in the Google organisation looked at, these require `google` below 8 and cannot be used with the pinned 8.6.0: `lb-http`, `pubsub`, `bigquery`, `kubernetes-engine`, `log-export`, `event-function`, `scheduled-function`, `bastion-host`. So a catalogue has to be tested, whichever rule decides when a module is used.
- Code using two modules validated in 11 seconds against the pinned provider, modules downloaded.
## Status
| Step | What | Status |
|---|---|---|
| 1 | Allow modules only from namespaces the bundle lists, at one exact version | built: `check_module_sources` in `app/tools/modules.py`, run before every validate and plan |
| 2 | A lookup of what a module accepts | built: the `terraform_module` tool, from the public registry, for the IaC author and the validator |
| 3 | A Skill with the catalogue and the rule for when to use one | built: `skills/google-cloud-iac-modules/`, four references, each validated by a test |
| 4 | Checks through a module | built as guidance: `implementation.json` names the module input; the validator confirms what the input sets |
| 5 | Opt-in per Execution, plain resources by default | built: `terraform_modules` on a start, `/generate`, A2A metadata and an evaluation case; recorded in `execution.json` |
| 5a | Run a case each way and see what the agents do with it | not run |
| 6 | Azure: `aztfm` | waits on an Azure provider bundle |
| 7 | AWS: `terraform-aws-modules` | waits on an AWS provider bundle |
Detail: `steps/step-1-to-5.md`.

View File

@@ -0,0 +1,62 @@
# Steps 1 to 5 — Implementation Detail
Built on 2026-10-09. **No Execution has run with it.**
## 1. Which modules
`terraform.modules.namespaces` in the provider bundle lists the registry namespaces a module may come from; for Google Cloud, `terraform-google-modules`. `TerraformRunner.validate` and `.plan` refuse, before Terraform runs:
- a `source` that is not `<allowed namespace>/<name>/<provider>`, optionally with `//path`: so any Git or HTTP URL, a GitHub shorthand, a local path, or another namespace;
- a `version` that is not one exact version;
- a module written in `.tf.json`.
With no namespace listed every module call is refused. The evaluation's own validation uses the same list, and only for a case that opts in.
The source the owner gave is a GitHub organisation; the registry namespace of the same name is what Terraform is pointed at, because the registry serves a fixed archive for a version and `terraform init` needs no Git.
## 2. What a module accepts
`terraform_module(source, version, find)` reads the public Terraform registry: a module's or submodule's inputs (required first, with type and description), its outputs and its submodules. It compares the module's provider requirements with the versions the bundle pins and says "NOT USABLE" when they exclude it. Only allowed namespaces can be looked up. Answers are cached for the life of the process.
It makes one new outbound call, to `registry.terraform.io`, which `terraform init` already reaches.
## 3. The Skill
`skills/google-cloud-iac-modules/`: when to use a module (it is in the table, it replaces three or more dependent resources, and every Check on the Elements it covers can be met through an input), and four references with a validated example each.
| For | Module | Version |
|---|---|---|
| VPC, subnets, firewall rules, Cloud NAT | `network`, `cloud-nat` | 18.3.1, 7.1.0 |
| Cloud SQL for PostgreSQL on a private address | `sql-db//modules/postgresql`, `//modules/private_service_access` | 28.3.0 |
| A Cloud Storage bucket with retention, lifecycle, encryption, grants | `cloud-storage//modules/simple_bucket` | 12.4.1 |
| A key ring, keys and grants | `kms` | 4.2.1 |
`tests/test_iac_modules.py` runs each example through `terraform validate` with the patterns Skill's versions file, and checks that every module call is from an allowed namespace at the version the table gives. It needs the network and is skipped without it.
The Skill is named in the bundle under `terraform.modules.skill`, not among any role's Skills. See step 5 for who is given it.
## 4. Checks through a module
A Check stays worded about the resource. `implementation.json` gives the module and the input that meets it, such as `module.database.ip_configuration.ipv4_enabled`. When no input sets what a Check needs, the module does not fit and the resources are written directly. The validator reads the input's value and confirms with the lookup what it sets, or what its default is.
This is instruction, in the Skills and the two role files. Nothing in code verifies that a module input does what a Check says: that would need a plan against a real project.
## 5. Opt-in per Execution
Infrastructure code is plain resources unless the Execution opts in.
- `ExecutionEngine.start(terraform_modules=...)`: true, false, or unset. Unset, a Change request does as its Baseline did, and any other Execution takes `TERRAFORM_MODULES` (off). Opting in is refused when the provider's bundle lists no sources.
- Callers: `terraform_modules` in the `/generate` body, in A2A message metadata, and on an evaluation case.
- **Opted in:** the roles that carry the `terraform_module` tool (the IaC author and the validator) are given the modules Skill, and their context says `terraform_modules: true`.
- **Not opted in:** those roles get neither the Skill nor the tool, and any `module` block is refused by `terraform_validate` and `terraform_plan` with a message saying to write the resources directly. The refusal is in code (`MODULES_OPTED_IN` in `app/tools/modules.py`, set when a Specialist starts), not only in instructions.
- `execution.json` records `terraform_modules`.
Reference routing (iteration 5 step 8) does not choose within the modules Skill, because it is not among a role's Skills at start-up.
## Known limits
- Modules also bring providers the bundle does not pin (`google-beta`, `random`, `null`), at whatever version their constraints allow.
- The schema review and the target architect's lookups still describe resources. A Check's wording is unaffected by whether a module later creates the resource.
- A module version in the catalogue goes stale; nothing flags a newer one.
Tests: `tests/tools/test_modules.py` (24), `tests/test_iac_modules.py` (6), `tests/execution/test_terraform_modules.py` (7).