Files
gcp_solution_architecture_a…/.agents/plans/iter-04-learning-and-skills/steps/step-18.md
Jonathan Boniface db28d02f56 fix: one schema lookup for a whole design, and larger graph batches
The target architect exhausted its 45 tool calls on a first attempt in
three of five runs: 31 schema lookups and 29 graph_put calls for one
design. Each time it carried on from its working graph, at a cost of four
to ten minutes.

- terraform_schema takes resource_types, a list, and outlines every
  argument and block each accepts, by name, in one reply.
- The architect is told to make that one call once the Elements are
  decided, and to build the graph in a handful of calls of ten to twenty
  items.

Case 04 then completed at 100% for the first time, the architect
finishing in one attempt with 11 lookups and 13 graph calls.

Plans record the run and correct three step statuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:34:47 +01:00

3.5 KiB

Step 18 — Implementation Detail

Built on 2026-10-07.

  1. providers/gcp.yaml pins terraform.required_providers.google to hashicorp/google 8.6.0. load_provider requires a source and an exact version.
  2. TerraformRunner.schema(resource_type, block, find) writes a versions file for the pinned providers in a scratch directory, runs terraform init -backend=false and terraform providers schema -json once per process, and keeps the resource schemas in memory. Measured here: 0.6 seconds and a 9.7 MB reply for the Google provider, from the plugin cache.
  3. With resource_type alone it lists the arguments code may set (read-only ones are left out) and outlines every block. With block it gives one block's arguments with descriptions. With find it says where a setting sits, or that the resource type has none. An unknown resource type gets close matches.
  4. The tool is terraform_schema, tier general, given to the target architect, the IaC author and the validator.

Checked against the real provider on 2026-10-07:

  • find="startup_cpu_boost" on google_cloud_run_v2_job: "has no argument or block named like 'startup_cpu_boost'. It cannot be set on this resource type."
  • the same on google_cloud_run_v2_service: template.containers.resources.startup_cpu_boost (bool, optional).

Not done: generated code is not forced onto the pinned version. An agent that writes its own required_providers with another version is validated against that version. The patterns Skill gives it the pinned versions.tf to use.

Tests: tests/tools/test_local.py, "schema lookup", on a stand-in schema; tests/test_providers.py for the pin.

After the run of 2026-10-07 16:59 UTC: the target architect made 39 lookups for 8 distinct resource types and exhausted its budget. find now takes several names separated by commas, the tool's description says the plain lookup lists every argument and block, and agents/target-architect.md asks for one lookup per distinct resource type (budget 45). An outline longer than 8,000 characters is cut with a note to use block or find; 7 of the provider's 1,370 resource types are that long, google_container_cluster the longest.

After the runs of 2026-10-08: the target architect still exhausted its 45 tool calls on its first attempt in three of five runs. In the last it made 31 schema lookups and 29 graph_put calls for one design. Each exhausted attempt cost four to ten minutes before the next one carried on from the working graph. Changed on 2026-10-09:

  • terraform_schema takes resource_types, a list (or several names in resource_type), and returns an outline of each: every argument and block it accepts, by name, required ones starred, with no descriptions. Up to 20 types a call, 4,000 characters a type. The 11 resource types of that design come back in one reply of 14,600 characters.
  • agents/target-architect.md asks for that one call once the Elements are decided, and for graph_put in a handful of calls of ten to twenty items each, where it said "a few items per call".
  • The schema review at submission (step 24) already checks every setting a Check names, so a lookup the architect skips is still caught.

Run on 2026-10-09 (case 04, eval/results/20261009T083440Z/): the target architect finished its first attempt in 314 seconds without exhausting its tool calls, making 11 schema lookups and 13 graph_put calls over the whole run, against 31 and 29 the run before. It still looked types up more than once, so the instruction is only partly followed.