fix: one schema lookup for a whole design, and larger graph batches
The target architect exhausted its 45 tool calls on a first attempt in three of five runs: 31 schema lookups and 29 graph_put calls for one design. Each time it carried on from its working graph, at a cost of four to ten minutes. - terraform_schema takes resource_types, a list, and outlines every argument and block each accepts, by name, in one reply. - The architect is told to make that one call once the Elements are decided, and to build the graph in a handful of calls of ten to twenty items. Case 04 then completed at 100% for the first time, the architect finishing in one attempt with 11 lookups and 13 graph calls. Plans record the run and correct three step statuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -210,3 +210,13 @@ To verify two changes made after the batches began: the pillar checklist review
|
||||
- **`specialists_ask_before_submitting` failed again**, for the Target architecture. The rule opened once the user had answered any question about the Deliverable; the target architect ran out of tool calls, the Orchestrator asked the user what to do, and that answer opened it. Fixed after the run: the rule now opens only when the user has been shown a draft of that Deliverable (`drafts_shown`). Not rerun.
|
||||
- **`requirements_name_no_products` failed** for the first time in any run: the requirements named Secret Manager. This is what the requirements review of iteration 5 step 4 looks for; it is off by default and was off here.
|
||||
|
||||
## Run of 2026-10-09, 08:34 UTC (`eval/results/20261009T083440Z/`): case 04
|
||||
|
||||
To verify the batched schema lookup (iteration 4 step 18) and the corrected confirm-first rule (step 27). Completed, 100% (12 of 12 checks), 36 minutes, 4.25 million tokens, one Revision of the Target architecture for an addition validation proposed (an alert on database storage); the description was kept and the code fixed in 40 seconds.
|
||||
|
||||
- The target architect did not exhaust its tool calls: one first attempt, then its draft confirmed by the user.
|
||||
- `specialists_ask_before_submitting` passed: every Specialist put its draft to the user before submitting. The rule refused nothing, so it was not needed in this run.
|
||||
- `requirements_name_no_products` passed.
|
||||
|
||||
This is the first run of case 04 at 100%.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user