Files
gcp_solution_architecture_a…/.agents/plans/iter-04-learning-and-skills/steps/step-23.md
Jonathan Boniface 58e778ca7a feat: record verdicts in pieces and check the design against the schema
From the case 02 run cancelled at 66 minutes on 2026-10-07:

- The validator records verdicts a few Checks at a time (verdict_put) and
  they are attached on submission. It timed out five times writing them in
  one reply. Verdicts on the same code survive a run cut short; verdicts on
  changed code are dropped. Its model timeout is 300 s.
- A schema review at submission, in code: every setting a Check names is
  looked up on its Element's resource type, and an unknown resource type is
  reported. Two Revisions in that run were for Checks no code could meet.
  On the two designs that passed validation it raises 2 of 96 Checks, both
  one metric name used as a value, so it is returned once and never blocks.
- Reviews run together; the Jev coverage review stays off.

Plans record both steps and the cancelled run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:39:13 +01:00

1.1 KiB

Step 23 — Implementation Detail

Built on 2026-10-08.

  1. verdict_put records verdicts a few Checks per call, replacing any already recorded for the same Check, and takes the dry-run's result and proposed additions. Each reply says how many Checks have a verdict and names those left. verdict_read returns what is recorded.
  2. submit_deliverable attaches the recorded verdicts as verdicts.json when none is passed. Passing the file still works.
  3. The verdicts are kept in the Deliverable's working memory with a mark of the code they were given on (a hash of the submitted deployment guidance's files). A later attempt on the same code carries on from them; on changed code they are dropped before any tool runs.
  4. The validator's model timeout is 300 seconds (models.timeout_seconds in providers/gcp.yaml); it had the default 180.
  5. agents/validator.md tells it to record as it goes, one Element's Checks at a time, and to submit with no attachments.

Unrun: no evaluation since.

Tests: tests/execution/test_validation_in_pieces.py, "verdicts a few at a time" (7).