feat: load the env file at start-up, trim schema lookups, remove legacy samples

Start-up
- Export the env file's variables for other libraries (the Cloud provider's
  client) into the process environment; the app's own settings and secrets
  are never exported, and a value already in the environment wins.
- Use GOOGLE_GENAI_USE_ENTERPRISE in place of the deprecated
  GOOGLE_GENAI_USE_VERTEXAI.

Target architect
- Look each distinct resource type up once instead of once per setting: in
  the run of 2026-10-07 it made 39 lookups for 8 types and exhausted its
  budget. terraform_schema's find takes several names; budget 30 to 45.
- A cut-off schema outline says so.

Removed, as leftovers of the original pipeline that nothing read
- terraform/ (no longer passed terraform validate), scripts/validate.sh,
  tests/test_package.py and the CI steps that ran on them.
- The sample outputs under docs/ and requirements.yaml.
- CI now pins Terraform 1.14.3, as the container image does.

Plans record the case 01 run: completed at 100% in 18 minutes with no
Revision, against 60 minutes and warnings before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 20:06:58 +01:00
parent 5bb22d984e
commit f1f6f1b83a
34 changed files with 161 additions and 402 deletions

View File

@@ -2,7 +2,7 @@
**Iteration:** 6, the Target architecture graph
**Intent:** Establish the Target architecture as one explicit graph of Elements, Relationships and Checks before any infrastructure code is written, and derive and check every later Deliverable from it, so that the outcome is understood before code exists and validation has a fixed bar.
**Status:** steps 1 to 10 built and covered by the deterministic suite; nothing has run against a real model; step 11's evaluation run is on hold by the owner
**Status:** steps 1 to 10 built and covered by the deterministic suite; case 01 completed at 100% with no Revision on 2026-10-07 (`steps/step-11.md`); cases 02 to 04 have not run on this build
**This directory covers:** steps 1 to 11
## Order of work
@@ -56,7 +56,7 @@ flowchart TD
| 8 | Deployment guidance implements the graph | M | complete |
| 9 | Validation gives a verdict per Check | M | complete |
| 10 | Revision routing: the code or the graph | S | complete; routing unproven with a real model |
| 11 | Evaluation: prove the loop converges | M | checks built; evaluation run outstanding (on hold by the owner) |
| 11 | Evaluation: prove the loop converges | M | case 01 passes (100%, no Revision); cases 02 to 04 and the thresholds outstanding |
Size: S under a day, M a few days.

View File

@@ -56,3 +56,26 @@ Changes this points to, not yet made:
- Have the validator distinguish "the code does not do this" from "this cannot be done to this resource".
- Check a proposed addition against the Element it names before it becomes a Check.
- Raise or tune the model timeout for the target architect.
## Run of 2026-10-07, 16:59 UTC (`eval/results/20261007T165920Z/`)
Case 01 again, after iteration 4 steps 11, 13 and 17 to 20 and the router of iteration 5. Not judged.
| | Run of 12:59 | This run |
|---|---|---|
| Status | completed with warnings | completed |
| Score | 73% | 100% (11 of 11 checks) |
| Seconds | 3,627 | 1,091 |
| Tokens | 4.25 million | 2.38 million |
| Revisions | Revision limit reached | none |
| Model timeouts | target architect, both attempts | none |
| `terraform validate` failures | 13 | 0 |
| IaC author | 57 requests, 2.35 million tokens | 11 requests, 271 thousand tokens |
| Target architect | timed out twice | 55 requests, 1.15 million tokens |
The graph had 13 Elements, 17 Relationships and 42 Checks. The code met all 42 on the first validation; no Check was unimplementable or not verifiable, and nothing was proposed. The Orchestrator was the router: 11 turns, no model chose an action, and no decision needed Jev.
What went wrong: the target architect exhausted its 30 tool calls on its first attempt. It made 39 `terraform_schema` calls for a graph with 8 distinct resource types, one per setting, as its instructions told it to. Its second attempt started from the working graph and submitted in 92 seconds. Changed after the run: it is told to look up each distinct resource type once, `find` takes several names in one call, and its budget is 45. That change has not been run.
Acceptance criteria: met for case 01. Still open: cases 02 to 04 have not run on this build, so "all benchmark cases" and the thresholds are unset.