diff --git a/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md b/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md index 72f2fa2..d0730d2 100644 --- a/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md +++ b/.agents/plans/iter-08-terraform-modules/steps/step-1-to-5.md @@ -57,7 +57,7 @@ Reference routing (iteration 5 step 8) does not choose within the modules Skill, - Modules also bring providers the bundle does not pin (`google-beta`, `random`, `null`), at whatever version their constraints allow. - The schema review and the target architect's lookups still describe resources. A Check's wording is unaffected by whether a module later creates the resource. -- A module version in the catalogue goes stale; nothing flags a newer one. +- A module version in the catalogue goes stale; `scripts/check-terraform-modules.py` reports it when run, and nothing runs it on a schedule. Tests: `tests/tools/test_modules.py` (24), `tests/test_iac_modules.py` (6), `tests/execution/test_terraform_modules.py` (7). @@ -87,3 +87,8 @@ What the opted-in run does show: The plain run was given neither the Skill nor the tool, and wrote no module. The design quality score of 0.62 is below the bar of 0.67. It is scored on the design recommendations, written before any code. + +## Two follow-ups (2026-10-10) + +- **A Check is met by a value in the code.** The first run showed the validator arguing from a module's default where an input was left unset. The Skill now tells the author to set every input a Check depends on, even when the default already gives the value; where one is still left unset the verdict says so. +- **Stale and newly usable modules are reported.** `scripts/check-terraform-modules.py` reads the registry and prints, for each module the Skill's examples call, the pinned version against the latest and whether the latest still works with the pinned provider; and for each module in `terraform.modules.watch`, whether it can now be used. Today all five are up to date and none of the eight watched modules can be used: each still requires a `google` provider below 8. It changes nothing; it needs running by hand. diff --git a/.scratch/lessons/issues/07-verify-location-constraints.md b/.scratch/lessons/issues/07-verify-location-constraints.md index cdfee78..594407b 100644 --- a/.scratch/lessons/issues/07-verify-location-constraints.md +++ b/.scratch/lessons/issues/07-verify-location-constraints.md @@ -8,11 +8,11 @@ Published: 2026-10-10 ## Rule -When a requirement specifies where resources must be provisioned, such as a designated region, multi-zone placement, or a low-carbon location, create a Check for the `region` or `location` argument on every applicable Element. +When a requirement specifies where resources must be provisioned, such as a designated region or multi-zone placement, create a Check for the `region` or `location` argument on every applicable Element, naming the region the design uses. Do not write a Check about a property of the region that code cannot show, such as its carbon rating. ## Why -Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating compliance or sustainability requirements. +Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating location requirements. A Check about a region's carbon rating cannot be verified from code and sends the design back. ## Evidence @@ -25,3 +25,5 @@ Not yet measured. ## Comments Approved by the owner on 2026-10-10. It covers which region is set, which code can show. It does not settle whether a region is low-carbon, which code cannot show. + +Reworded with the owner's approval on 2026-10-10: "a low-carbon location" led to a Check about the region's carbon score, which could not be verified. diff --git a/app/providers.py b/app/providers.py index b49fff0..1139e47 100644 --- a/app/providers.py +++ b/app/providers.py @@ -78,6 +78,10 @@ class ProviderBundle: # The Skill that says which modules to use and when. It reaches a Specialist # only in an Execution that opted into modules. terraform_modules_skill: str | None = None + # Modules that could not be used with the pinned providers when last looked + # at, as //. scripts/check-terraform-modules.py + # says when one can. + terraform_modules_watch: tuple[str, ...] = () # The Skill approved lessons are published in (ADR-0004); None when the provider # has none. A Specialist is given it once it holds lessons for its Deliverable. lessons_skill: str | None = None @@ -244,5 +248,12 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle: terraform_providers=terraform_providers, terraform_module_namespaces=tuple(module_namespaces), terraform_modules_skill=modules_skill, + terraform_modules_watch=tuple( + str(item) + for item in ((config.get("terraform") or {}).get("modules") or {}).get( + "watch" + ) + or () + ), lessons_skill=lessons_skill, ) diff --git a/providers/gcp.yaml b/providers/gcp.yaml index 5b76e77..ce20c0f 100644 --- a/providers/gcp.yaml +++ b/providers/gcp.yaml @@ -87,6 +87,17 @@ terraform: modules: namespaces: [terraform-google-modules] skill: google-cloud-iac-modules + # Not usable on 2026-10-09: each required a google provider below 8. Run + # scripts/check-terraform-modules.py to see whether that has changed. + watch: + - terraform-google-modules/lb-http/google + - terraform-google-modules/pubsub/google + - terraform-google-modules/bigquery/google + - terraform-google-modules/kubernetes-engine/google + - terraform-google-modules/log-export/google + - terraform-google-modules/event-function/google + - terraform-google-modules/scheduled-function/google + - terraform-google-modules/bastion-host/google mcp_auth: kind: google_adc # Application Default Credentials bearer token; sends X-Goog-User-Project diff --git a/scripts/check-terraform-modules.py b/scripts/check-terraform-modules.py new file mode 100644 index 0000000..3efb2ce --- /dev/null +++ b/scripts/check-terraform-modules.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""Review the published Terraform modules a provider bundle's modules Skill uses. + +For each module the Skill's examples call: the version pinned there, the latest in +the registry, and whether the latest still works with the pinned provider. For each +module the bundle lists under `terraform.modules.watch`, which could not be used +when last looked at: whether its latest version now can. + +It reads the public registry and changes nothing. Raising a pinned version is a +change to the Skill's example, which `tests/test_iac_modules.py` then validates. + +Usage: .venv/bin/python scripts/check-terraform-modules.py [provider] +""" + +from __future__ import annotations + +import asyncio +import re +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +from app.adk.runtime import ToolResult +from app.providers import load_provider +from app.tools.modules import ModuleLookup, admits, module_calls + +_HCL = re.compile(r"^```hcl\n(.*?)^```", re.MULTILINE | re.DOTALL) + + +def _blocked(part: dict, pinned: dict[str, str]) -> list[str]: + """The provider requirements of a module that the pinned versions do not meet.""" + return [ + f"{need.get('name')} {need.get('version')}" + for need in part.get("provider_dependencies") or [] + if need.get("name") in pinned + and need.get("version") + and not admits(need["version"], pinned[need["name"]]) + ] + + +async def main(provider: str) -> int: + bundle = load_provider(provider) + if not bundle.terraform_modules_skill: + print(f"The {provider} bundle names no modules Skill.") + return 0 + pinned = {name: pin["version"] for name, pin in bundle.terraform_providers.items()} + lookup = ModuleLookup(namespaces=bundle.terraform_module_namespaces, pinned=pinned) + skill = bundle.skills[bundle.terraform_modules_skill].path + + used: dict[str, set[str]] = {} + for reference in sorted((skill / "references").glob("*.md")): + code = "\n".join(_HCL.findall(reference.read_text(encoding="utf-8"))) + for call in module_calls({"main.tf": code}): + used.setdefault(call.source.split("//")[0], set()).add(call.version) + + stale = 0 + print(f"Pinned providers: {pinned}\n\nModules the Skill uses:") + for module, versions in sorted(used.items()): + data = await lookup._fetch(module, "") + if isinstance(data, ToolResult): + print(f" {module}: {data.message}") + continue + latest = str(data.get("version", "?")) + blocked = _blocked(data.get("root") or {}, pinned) + note = "up to date" if versions == {latest} else f"latest is {latest}" + if versions != {latest}: + stale += 1 + if blocked: + note += f"; the latest needs {', '.join(blocked)}, so stay on the pin" + print(f" {module} {', '.join(sorted(versions))}: {note}") + + print("\nModules that could not be used when last looked at:") + usable = 0 + for name in bundle.terraform_modules_watch: + data = await lookup._fetch(name, "") + if isinstance(data, ToolResult): + print(f" {name}: {data.message}") + continue + blocked = _blocked(data.get("root") or {}, pinned) + if blocked: + print(f" {name} {data.get('version')}: still needs {', '.join(blocked)}") + else: + usable += 1 + print( + f" {name} {data.get('version')}: NOW USABLE with the pinned provider" + ) + print( + f"\n{stale} pinned module(s) have a newer version; {usable} watched " + "module(s) can now be used." + ) + return 0 + + +if __name__ == "__main__": + sys.exit(asyncio.run(main(sys.argv[1] if len(sys.argv) > 1 else "gcp"))) diff --git a/skills/google-cloud-iac-modules/SKILL.md b/skills/google-cloud-iac-modules/SKILL.md index 677a687..eeb224d 100644 --- a/skills/google-cloud-iac-modules/SKILL.md +++ b/skills/google-cloud-iac-modules/SKILL.md @@ -44,6 +44,7 @@ Not usable with this provider version, so write these directly: the load balance - Keep `terraform/versions.tf` from the patterns Skill unchanged. A module's own provider requirements are met by it. - Pass every environment-specific value from a variable, as for resources. - Before using an input the reference does not show, call `terraform_module` with `find` set to its name. +- Set every input a Check depends on, even when the module's default already gives that value. A Check is verified from a value written in the code; a default left unset can only be argued from the module's documentation, and changes when the module does. - A module takes names, not always ids: read which one an input wants. Pass another module's output where one depends on the other, so Terraform orders them. ## Checks on an Element a module creates @@ -52,4 +53,4 @@ A Check is still about the resource: "the instance has no public IP address". Th - In `implementation.json`, `address` names the module and the input: `module.database.ip_configuration.ipv4_enabled`. - When no input sets what a Check needs, the module does not fit that Element. Write its resources directly; do not mark the Check unimplementable, since the resource type does have the setting. -- When validating, read the input's value in the code and confirm with `terraform_module` what that input sets. An input left at its default is judged by the default the lookup shows. A Check is not verifiable only when neither the code nor the module's documentation shows what the resource will get. +- When validating, read the input's value in the code and confirm with `terraform_module` what that input sets. An input left at its default is judged by the default the lookup shows, and the verdict says the value is a default and not set in the code. A Check is not verifiable only when neither the code nor the module's documentation shows what the resource will get. diff --git a/skills/google-cloud-lessons/references/target_architecture.md b/skills/google-cloud-lessons/references/target_architecture.md index 909a021..a8e714b 100644 --- a/skills/google-cloud-lessons/references/target_architecture.md +++ b/skills/google-cloud-lessons/references/target_architecture.md @@ -36,9 +36,9 @@ Published 2026-10-10, from lesson proposal 06. ## L-07: Verify location constraints -**Rule.** When a requirement specifies where resources must be provisioned, such as a designated region, multi-zone placement, or a low-carbon location, create a Check for the `region` or `location` argument on every applicable Element. +**Rule.** When a requirement specifies where resources must be provisioned, such as a designated region or multi-zone placement, create a Check for the `region` or `location` argument on every applicable Element, naming the region the design uses. Do not write a Check about a property of the region that code cannot show, such as its carbon rating. -**Why.** Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating compliance or sustainability requirements. +**Why.** Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating location requirements. A Check about a region's carbon rating cannot be verified from code and sends the design back. Published 2026-10-10, from lesson proposal 07. diff --git a/tests/execution/test_terraform_modules.py b/tests/execution/test_terraform_modules.py index c09cd25..9348ea0 100644 --- a/tests/execution/test_terraform_modules.py +++ b/tests/execution/test_terraform_modules.py @@ -203,3 +203,12 @@ def test_the_gcp_bundle_names_its_modules_skill_for_the_roles_that_run_terraform assert all( bundle.terraform_modules_skill not in role.skills for role in roles.values() ) + + +def test_the_bundle_lists_modules_to_watch_from_its_approved_namespaces(): + bundle = load_provider("gcp") + + assert bundle.terraform_modules_watch + for module in bundle.terraform_modules_watch: + namespace, _, provider = module.split("/") + assert namespace in bundle.terraform_module_namespaces and provider == "google"