feat: record verdicts in pieces and check the design against the schema
From the case 02 run cancelled at 66 minutes on 2026-10-07: - The validator records verdicts a few Checks at a time (verdict_put) and they are attached on submission. It timed out five times writing them in one reply. Verdicts on the same code survive a run cut short; verdicts on changed code are dropped. Its model timeout is 300 s. - A schema review at submission, in code: every setting a Check names is looked up on its Element's resource type, and an unknown resource type is reported. Two Revisions in that run were for Checks no code could meet. On the two designs that passed validation it raises 2 of 96 Checks, both one metric name used as a value, so it is returned once and never blocks. - Reviews run together; the Jev coverage review stays off. Plans record both steps and the cancelled run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -42,7 +42,7 @@ The files are your working copy: the files you wrote with `write_file`, or, when
|
||||
|
||||
## Reporting verdicts
|
||||
|
||||
Give a verdict on every Check in the Target architecture and attach them as `verdicts.json`, in the format the requirements-chain Skill gives. The dry-run's result goes in its `dry_run` entry: `pass` when `terraform_validate` passes, `fail` with the error when it does not, and `not_run` when the tool could not run.
|
||||
Give a verdict on every Check in the Target architecture. Record them with `verdict_put`, a few Checks per call; they are attached as `verdicts.json`, in the format the requirements-chain Skill gives, when you submit. The dry-run's result goes in its `dry_run` entry: `pass` when `terraform_validate` passes, `fail` with the error when it does not, and `not_run` when the tool could not run.
|
||||
|
||||
A Check fails when the code could meet it and does not. When `terraform_schema` shows the resource type has no such setting, no code could meet it: the verdict is `not_verifiable`, with the schema's answer as evidence.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user