fix: one schema lookup for a whole design, and larger graph batches

The target architect exhausted its 45 tool calls on a first attempt in
three of five runs: 31 schema lookups and 29 graph_put calls for one
design. Each time it carried on from its working graph, at a cost of four
to ten minutes.

- terraform_schema takes resource_types, a list, and outlines every
  argument and block each accepts, by name, in one reply.
- The architect is told to make that one call once the Elements are
  decided, and to build the graph in a handful of calls of ten to twenty
  items.

Case 04 then completed at 100% for the first time, the architect
finishing in one attempt with 11 lookups and 13 graph calls.

Plans record the run and correct three step statuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 10:34:47 +01:00
parent 654370bb46
commit db28d02f56
8 changed files with 139 additions and 15 deletions

View File

@@ -210,3 +210,13 @@ To verify two changes made after the batches began: the pillar checklist review
- **`specialists_ask_before_submitting` failed again**, for the Target architecture. The rule opened once the user had answered any question about the Deliverable; the target architect ran out of tool calls, the Orchestrator asked the user what to do, and that answer opened it. Fixed after the run: the rule now opens only when the user has been shown a draft of that Deliverable (`drafts_shown`). Not rerun.
- **`requirements_name_no_products` failed** for the first time in any run: the requirements named Secret Manager. This is what the requirements review of iteration 5 step 4 looks for; it is off by default and was off here.
## Run of 2026-10-09, 08:34 UTC (`eval/results/20261009T083440Z/`): case 04
To verify the batched schema lookup (iteration 4 step 18) and the corrected confirm-first rule (step 27). Completed, 100% (12 of 12 checks), 36 minutes, 4.25 million tokens, one Revision of the Target architecture for an addition validation proposed (an alert on database storage); the description was kept and the code fixed in 40 seconds.
- The target architect did not exhaust its tool calls: one first attempt, then its draft confirmed by the user.
- `specialists_ask_before_submitting` passed: every Specialist put its draft to the user before submitting. The rule refused nothing, so it was not needed in this run.
- `requirements_name_no_products` passed.
This is the first run of case 04 at 100%.