chore: raise the validator's budget and record the runs of cases 02, 03 and 04

- Validator budget 15 to 30: recording verdicts in pieces for 100 Checks
  used all 15 calls in one attempt.
- Plans record three runs on 2026-10-08:
  case 02 completed at 100% with no Revision in 24 minutes;
  case 03 completed at 100% with two Revisions of the design in 40 minutes;
  case 04 (interactive) ended with warnings at 75%, on two router faults:
  a reply that echoes the question is read as a change request, and the
  router loops at the Revision limit. Neither is fixed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-08 13:16:36 +01:00
parent 58e778ca7a
commit 4e7a16cedd
8 changed files with 45 additions and 9 deletions

View File

@@ -8,6 +8,6 @@ Built on 2026-10-08.
4. The validator's model timeout is 300 seconds (`models.timeout_seconds` in `providers/gcp.yaml`); it had the default 180.
5. `agents/validator.md` tells it to record as it goes, one Element's Checks at a time, and to submit with no attachments.
**Unrun:** no evaluation since.
Run on 2026-10-08 (case 02): no model timeout. The validator used all 15 of its tool calls on its first attempt, having recorded verdicts; the second attempt read them and submitted in 68 seconds. Its budget is now 30 (`agents/validator.md`).
Tests: `tests/execution/test_validation_in_pieces.py`, "verdicts a few at a time" (7).