From the case 02 run cancelled at 66 minutes on 2026-10-07: - The validator records verdicts a few Checks at a time (verdict_put) and they are attached on submission. It timed out five times writing them in one reply. Verdicts on the same code survive a run cut short; verdicts on changed code are dropped. Its model timeout is 300 s. - A schema review at submission, in code: every setting a Check names is looked up on its Element's resource type, and an unknown resource type is reported. Two Revisions in that run were for Checks no code could meet. On the two designs that passed validation it raises 2 of 96 Checks, both one metric name used as a value, so it is returned once and never blocks. - Reviews run together; the Jev coverage review stays off. Plans record both steps and the cancelled run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.8 KiB
1.8 KiB
Step 24 — Implementation Detail
Built on 2026-10-08 as a second SubmissionReview (the seam from iteration 5 step 4), with no model and no Jev.
named_settingsreads the settings a Check's statement names: anything in backticks that reads as a name or dotted path, and bare snake_case names and dotted paths. Quoted text, upper-case values and paths such asroles/x.yare skipped.SchemaReviewloads the pinned provider's schema once (iteration 4 step 18). For each Check on an Element it looks each named setting up anywhere in that Element's resource type; a path is judged by the setting at its end, and a resource type named in a statement is not a setting. It also reports an Element whose resource type the provider does not have, with close matches.Reviewsruns several reviews as one. The assembly installs the schema review whenever the bundle pins a Terraform provider, and the coverage review after it when that is switched on.- As with any review, what it raises is returned to the target architect once per run and the next submission is accepted.
Measured on 2026-10-08 against the real provider:
| Design | Checks | Raised |
|---|---|---|
| Case 01, passed validation | 42 | 0 |
| Case 02, passed validation | 54 | 2, both the same metric name used as a value (oldest_unacked_message_age) |
Known faults added: startup_cpu_boost on a Cloud Run job, ssl_policy on a backend service, a misspelt resource type |
3 | all 3 |
So 2 false alarms in 96 Checks, which is why it is a review and not a rule. It does not catch a Check that names no setting at all, such as "deny read roles" on a service account; that still reaches the IaC author and comes back by the route of step 19.
Unrun in an Execution.
Tests: tests/execution/test_validation_in_pieces.py, "the design against the provider's schema" (6).