feat: module follow-ups, and L-07 reworded
- A Check met through a module is met by a value in the code: the modules Skill tells the author to set every input a Check depends on. - scripts/check-terraform-modules.py reports catalogued modules with a newer version, and watched modules that can now be used with the pinned provider. - L-07 no longer mentions a low-carbon location, which led to a Check about a region's carbon score that code cannot show. Reworded with the owner's approval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -57,7 +57,7 @@ Reference routing (iteration 5 step 8) does not choose within the modules Skill,
|
||||
|
||||
- Modules also bring providers the bundle does not pin (`google-beta`, `random`, `null`), at whatever version their constraints allow.
|
||||
- The schema review and the target architect's lookups still describe resources. A Check's wording is unaffected by whether a module later creates the resource.
|
||||
- A module version in the catalogue goes stale; nothing flags a newer one.
|
||||
- A module version in the catalogue goes stale; `scripts/check-terraform-modules.py` reports it when run, and nothing runs it on a schedule.
|
||||
|
||||
Tests: `tests/tools/test_modules.py` (24), `tests/test_iac_modules.py` (6), `tests/execution/test_terraform_modules.py` (7).
|
||||
|
||||
@@ -87,3 +87,8 @@ What the opted-in run does show:
|
||||
The plain run was given neither the Skill nor the tool, and wrote no module.
|
||||
|
||||
The design quality score of 0.62 is below the bar of 0.67. It is scored on the design recommendations, written before any code.
|
||||
|
||||
## Two follow-ups (2026-10-10)
|
||||
|
||||
- **A Check is met by a value in the code.** The first run showed the validator arguing from a module's default where an input was left unset. The Skill now tells the author to set every input a Check depends on, even when the default already gives the value; where one is still left unset the verdict says so.
|
||||
- **Stale and newly usable modules are reported.** `scripts/check-terraform-modules.py` reads the registry and prints, for each module the Skill's examples call, the pinned version against the latest and whether the latest still works with the pinned provider; and for each module in `terraform.modules.watch`, whether it can now be used. Today all five are up to date and none of the eight watched modules can be used: each still requires a `google` provider below 8. It changes nothing; it needs running by hand.
|
||||
|
||||
@@ -8,11 +8,11 @@ Published: 2026-10-10
|
||||
|
||||
## Rule
|
||||
|
||||
When a requirement specifies where resources must be provisioned, such as a designated region, multi-zone placement, or a low-carbon location, create a Check for the `region` or `location` argument on every applicable Element.
|
||||
When a requirement specifies where resources must be provisioned, such as a designated region or multi-zone placement, create a Check for the `region` or `location` argument on every applicable Element, naming the region the design uses. Do not write a Check about a property of the region that code cannot show, such as its carbon rating.
|
||||
|
||||
## Why
|
||||
|
||||
Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating compliance or sustainability requirements.
|
||||
Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating location requirements. A Check about a region's carbon rating cannot be verified from code and sends the design back.
|
||||
|
||||
## Evidence
|
||||
|
||||
@@ -25,3 +25,5 @@ Not yet measured.
|
||||
## Comments
|
||||
|
||||
Approved by the owner on 2026-10-10. It covers which region is set, which code can show. It does not settle whether a region is low-carbon, which code cannot show.
|
||||
|
||||
Reworded with the owner's approval on 2026-10-10: "a low-carbon location" led to a Check about the region's carbon score, which could not be verified.
|
||||
|
||||
@@ -78,6 +78,10 @@ class ProviderBundle:
|
||||
# The Skill that says which modules to use and when. It reaches a Specialist
|
||||
# only in an Execution that opted into modules.
|
||||
terraform_modules_skill: str | None = None
|
||||
# Modules that could not be used with the pinned providers when last looked
|
||||
# at, as <namespace>/<name>/<provider>. scripts/check-terraform-modules.py
|
||||
# says when one can.
|
||||
terraform_modules_watch: tuple[str, ...] = ()
|
||||
# The Skill approved lessons are published in (ADR-0004); None when the provider
|
||||
# has none. A Specialist is given it once it holds lessons for its Deliverable.
|
||||
lessons_skill: str | None = None
|
||||
@@ -244,5 +248,12 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
|
||||
terraform_providers=terraform_providers,
|
||||
terraform_module_namespaces=tuple(module_namespaces),
|
||||
terraform_modules_skill=modules_skill,
|
||||
terraform_modules_watch=tuple(
|
||||
str(item)
|
||||
for item in ((config.get("terraform") or {}).get("modules") or {}).get(
|
||||
"watch"
|
||||
)
|
||||
or ()
|
||||
),
|
||||
lessons_skill=lessons_skill,
|
||||
)
|
||||
|
||||
@@ -87,6 +87,17 @@ terraform:
|
||||
modules:
|
||||
namespaces: [terraform-google-modules]
|
||||
skill: google-cloud-iac-modules
|
||||
# Not usable on 2026-10-09: each required a google provider below 8. Run
|
||||
# scripts/check-terraform-modules.py to see whether that has changed.
|
||||
watch:
|
||||
- terraform-google-modules/lb-http/google
|
||||
- terraform-google-modules/pubsub/google
|
||||
- terraform-google-modules/bigquery/google
|
||||
- terraform-google-modules/kubernetes-engine/google
|
||||
- terraform-google-modules/log-export/google
|
||||
- terraform-google-modules/event-function/google
|
||||
- terraform-google-modules/scheduled-function/google
|
||||
- terraform-google-modules/bastion-host/google
|
||||
|
||||
mcp_auth:
|
||||
kind: google_adc # Application Default Credentials bearer token; sends X-Goog-User-Project
|
||||
|
||||
96
scripts/check-terraform-modules.py
Normal file
96
scripts/check-terraform-modules.py
Normal file
@@ -0,0 +1,96 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Review the published Terraform modules a provider bundle's modules Skill uses.
|
||||
|
||||
For each module the Skill's examples call: the version pinned there, the latest in
|
||||
the registry, and whether the latest still works with the pinned provider. For each
|
||||
module the bundle lists under `terraform.modules.watch`, which could not be used
|
||||
when last looked at: whether its latest version now can.
|
||||
|
||||
It reads the public registry and changes nothing. Raising a pinned version is a
|
||||
change to the Skill's example, which `tests/test_iac_modules.py` then validates.
|
||||
|
||||
Usage: .venv/bin/python scripts/check-terraform-modules.py [provider]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
from app.adk.runtime import ToolResult
|
||||
from app.providers import load_provider
|
||||
from app.tools.modules import ModuleLookup, admits, module_calls
|
||||
|
||||
_HCL = re.compile(r"^```hcl\n(.*?)^```", re.MULTILINE | re.DOTALL)
|
||||
|
||||
|
||||
def _blocked(part: dict, pinned: dict[str, str]) -> list[str]:
|
||||
"""The provider requirements of a module that the pinned versions do not meet."""
|
||||
return [
|
||||
f"{need.get('name')} {need.get('version')}"
|
||||
for need in part.get("provider_dependencies") or []
|
||||
if need.get("name") in pinned
|
||||
and need.get("version")
|
||||
and not admits(need["version"], pinned[need["name"]])
|
||||
]
|
||||
|
||||
|
||||
async def main(provider: str) -> int:
|
||||
bundle = load_provider(provider)
|
||||
if not bundle.terraform_modules_skill:
|
||||
print(f"The {provider} bundle names no modules Skill.")
|
||||
return 0
|
||||
pinned = {name: pin["version"] for name, pin in bundle.terraform_providers.items()}
|
||||
lookup = ModuleLookup(namespaces=bundle.terraform_module_namespaces, pinned=pinned)
|
||||
skill = bundle.skills[bundle.terraform_modules_skill].path
|
||||
|
||||
used: dict[str, set[str]] = {}
|
||||
for reference in sorted((skill / "references").glob("*.md")):
|
||||
code = "\n".join(_HCL.findall(reference.read_text(encoding="utf-8")))
|
||||
for call in module_calls({"main.tf": code}):
|
||||
used.setdefault(call.source.split("//")[0], set()).add(call.version)
|
||||
|
||||
stale = 0
|
||||
print(f"Pinned providers: {pinned}\n\nModules the Skill uses:")
|
||||
for module, versions in sorted(used.items()):
|
||||
data = await lookup._fetch(module, "")
|
||||
if isinstance(data, ToolResult):
|
||||
print(f" {module}: {data.message}")
|
||||
continue
|
||||
latest = str(data.get("version", "?"))
|
||||
blocked = _blocked(data.get("root") or {}, pinned)
|
||||
note = "up to date" if versions == {latest} else f"latest is {latest}"
|
||||
if versions != {latest}:
|
||||
stale += 1
|
||||
if blocked:
|
||||
note += f"; the latest needs {', '.join(blocked)}, so stay on the pin"
|
||||
print(f" {module} {', '.join(sorted(versions))}: {note}")
|
||||
|
||||
print("\nModules that could not be used when last looked at:")
|
||||
usable = 0
|
||||
for name in bundle.terraform_modules_watch:
|
||||
data = await lookup._fetch(name, "")
|
||||
if isinstance(data, ToolResult):
|
||||
print(f" {name}: {data.message}")
|
||||
continue
|
||||
blocked = _blocked(data.get("root") or {}, pinned)
|
||||
if blocked:
|
||||
print(f" {name} {data.get('version')}: still needs {', '.join(blocked)}")
|
||||
else:
|
||||
usable += 1
|
||||
print(
|
||||
f" {name} {data.get('version')}: NOW USABLE with the pinned provider"
|
||||
)
|
||||
print(
|
||||
f"\n{stale} pinned module(s) have a newer version; {usable} watched "
|
||||
"module(s) can now be used."
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(asyncio.run(main(sys.argv[1] if len(sys.argv) > 1 else "gcp")))
|
||||
@@ -44,6 +44,7 @@ Not usable with this provider version, so write these directly: the load balance
|
||||
- Keep `terraform/versions.tf` from the patterns Skill unchanged. A module's own provider requirements are met by it.
|
||||
- Pass every environment-specific value from a variable, as for resources.
|
||||
- Before using an input the reference does not show, call `terraform_module` with `find` set to its name.
|
||||
- Set every input a Check depends on, even when the module's default already gives that value. A Check is verified from a value written in the code; a default left unset can only be argued from the module's documentation, and changes when the module does.
|
||||
- A module takes names, not always ids: read which one an input wants. Pass another module's output where one depends on the other, so Terraform orders them.
|
||||
|
||||
## Checks on an Element a module creates
|
||||
@@ -52,4 +53,4 @@ A Check is still about the resource: "the instance has no public IP address". Th
|
||||
|
||||
- In `implementation.json`, `address` names the module and the input: `module.database.ip_configuration.ipv4_enabled`.
|
||||
- When no input sets what a Check needs, the module does not fit that Element. Write its resources directly; do not mark the Check unimplementable, since the resource type does have the setting.
|
||||
- When validating, read the input's value in the code and confirm with `terraform_module` what that input sets. An input left at its default is judged by the default the lookup shows. A Check is not verifiable only when neither the code nor the module's documentation shows what the resource will get.
|
||||
- When validating, read the input's value in the code and confirm with `terraform_module` what that input sets. An input left at its default is judged by the default the lookup shows, and the verdict says the value is a default and not set in the code. A Check is not verifiable only when neither the code nor the module's documentation shows what the resource will get.
|
||||
|
||||
@@ -36,9 +36,9 @@ Published 2026-10-10, from lesson proposal 06.
|
||||
|
||||
## L-07: Verify location constraints
|
||||
|
||||
**Rule.** When a requirement specifies where resources must be provisioned, such as a designated region, multi-zone placement, or a low-carbon location, create a Check for the `region` or `location` argument on every applicable Element.
|
||||
**Rule.** When a requirement specifies where resources must be provisioned, such as a designated region or multi-zone placement, create a Check for the `region` or `location` argument on every applicable Element, naming the region the design uses. Do not write a Check about a property of the region that code cannot show, such as its carbon rating.
|
||||
|
||||
**Why.** Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating compliance or sustainability requirements.
|
||||
**Why.** Validation frequently catches missing location constraints. Without this Check, resources may be provisioned in default regions, violating location requirements. A Check about a region's carbon rating cannot be verified from code and sends the design back.
|
||||
|
||||
Published 2026-10-10, from lesson proposal 07.
|
||||
|
||||
|
||||
@@ -203,3 +203,12 @@ def test_the_gcp_bundle_names_its_modules_skill_for_the_roles_that_run_terraform
|
||||
assert all(
|
||||
bundle.terraform_modules_skill not in role.skills for role in roles.values()
|
||||
)
|
||||
|
||||
|
||||
def test_the_bundle_lists_modules_to_watch_from_its_approved_namespaces():
|
||||
bundle = load_provider("gcp")
|
||||
|
||||
assert bundle.terraform_modules_watch
|
||||
for module in bundle.terraform_modules_watch:
|
||||
namespace, _, provider = module.split("/")
|
||||
assert namespace in bundle.terraform_module_namespaces and provider == "google"
|
||||
|
||||
Reference in New Issue
Block a user