chore: raise the validator's budget and record the runs of cases 02, 03 and 04
- Validator budget 15 to 30: recording verdicts in pieces for 100 Checks used all 15 calls in one attempt. - Plans record three runs on 2026-10-08: case 02 completed at 100% with no Revision in 24 minutes; case 03 completed at 100% with two Revisions of the design in 40 minutes; case 04 (interactive) ended with warnings at 75%, on two router faults: a reply that echoes the question is read as a change request, and the router loops at the Revision limit. Neither is fixed here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,6 @@ Built on 2026-10-08.
|
||||
4. The validator's model timeout is 300 seconds (`models.timeout_seconds` in `providers/gcp.yaml`); it had the default 180.
|
||||
5. `agents/validator.md` tells it to record as it goes, one Element's Checks at a time, and to submit with no attachments.
|
||||
|
||||
**Unrun:** no evaluation since.
|
||||
Run on 2026-10-08 (case 02): no model timeout. The validator used all 15 of its tool calls on its first attempt, having recorded verdicts; the second attempt read them and submitted in 68 seconds. Its budget is now 30 (`agents/validator.md`).
|
||||
|
||||
Tests: `tests/execution/test_validation_in_pieces.py`, "verdicts a few at a time" (7).
|
||||
|
||||
Reference in New Issue
Block a user