From the case 02 run cancelled at 66 minutes on 2026-10-07: - The validator records verdicts a few Checks at a time (verdict_put) and they are attached on submission. It timed out five times writing them in one reply. Verdicts on the same code survive a run cut short; verdicts on changed code are dropped. Its model timeout is 300 s. - A schema review at submission, in code: every setting a Check names is looked up on its Element's resource type, and an unknown resource type is reported. Two Revisions in that run were for Checks no code could meet. On the two designs that passed validation it raises 2 of 96 Checks, both one metric name used as a value, so it is returned once and never blocks. - Reviews run together; the Jev coverage review stays off. Plans record both steps and the cancelled run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.1 KiB
1.1 KiB
Step 23 — Implementation Detail
Built on 2026-10-08.
verdict_putrecords verdicts a few Checks per call, replacing any already recorded for the same Check, and takes the dry-run's result and proposed additions. Each reply says how many Checks have a verdict and names those left.verdict_readreturns what is recorded.submit_deliverableattaches the recorded verdicts asverdicts.jsonwhen none is passed. Passing the file still works.- The verdicts are kept in the Deliverable's working memory with a mark of the code they were given on (a hash of the submitted deployment guidance's files). A later attempt on the same code carries on from them; on changed code they are dropped before any tool runs.
- The validator's model timeout is 300 seconds (
models.timeout_secondsinproviders/gcp.yaml); it had the default 180. agents/validator.mdtells it to record as it goes, one Element's Checks at a time, and to submit with no attachments.
Unrun: no evaluation since.
Tests: tests/execution/test_validation_in_pieces.py, "verdicts a few at a time" (7).