feat: keep an unchanged description, list conditions, review coverage with Jev

After case 02 (completed at 100%, but with two Revisions of the Target
architecture for gaps validation found one pass at a time):

- A Deliverable stands while what it was written from is unchanged. The
  architecture description is written from the Elements and Relationships,
  so a Revision that only adds or rewords Checks no longer runs the describer.
- Non-functional requirements and recommendations list their conditions,
  one entry per separate thing that must be true; submission requires them.
- A coverage review at submission (JEV_COVERAGE_REVIEW, off by default):
  for each condition, Jev chooses which Check verifies it, or none. Asked as
  one broad yes-or-no per requirement it could not tell a complete design
  from one with known gaps; asked as a choice per condition it raised all
  four known gaps of case 02 and nothing on the complete graph. An opt-in
  live test (LIVE_JEV=1) repeats that.
- The validator is told to report every gap in its first pass.

Plans record the case 02 run and both forms of the review with figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 20:58:38 +01:00
parent f1f6f1b83a
commit 5adc911a82
29 changed files with 1687 additions and 16 deletions

View File

@@ -2,7 +2,7 @@
**Iteration:** 6, the Target architecture graph
**Intent:** Establish the Target architecture as one explicit graph of Elements, Relationships and Checks before any infrastructure code is written, and derive and check every later Deliverable from it, so that the outcome is understood before code exists and validation has a fixed bar.
**Status:** steps 1 to 10 built and covered by the deterministic suite; case 01 completed at 100% with no Revision on 2026-10-07 (`steps/step-11.md`); cases 02 to 04 have not run on this build
**Status:** steps 1 to 10 built and covered by the deterministic suite; cases 01 and 02 completed at 100% on 2026-10-07 (`steps/step-11.md`), case 02 after two Revisions of the Target architecture; cases 03 and 04 have not run on this build
**This directory covers:** steps 1 to 11
## Order of work
@@ -56,7 +56,7 @@ flowchart TD
| 8 | Deployment guidance implements the graph | M | complete |
| 9 | Validation gives a verdict per Check | M | complete |
| 10 | Revision routing: the code or the graph | S | complete; routing unproven with a real model |
| 11 | Evaluation: prove the loop converges | M | case 01 passes (100%, no Revision); cases 02 to 04 and the thresholds outstanding |
| 11 | Evaluation: prove the loop converges | M | cases 01 and 02 pass at 100%; cases 03 and 04 and the thresholds outstanding |
Size: S under a day, M a few days.

View File

@@ -79,3 +79,17 @@ What went wrong: the target architect exhausted its 30 tool calls on its first a
Acceptance criteria: met for case 01. Still open: cases 02 to 04 have not run on this build, so "all benchmark cases" and the thresholds are unset.
## Run of 2026-10-07, 19:07 UTC (`eval/results/20261007T190702Z/`)
Case 02, on commit `f1f6f1b`. Completed, 100% (11 of 11 checks), no warnings: 1,781 seconds and 4.78 million tokens; 904 seconds to the first validation.
- The graph ended with 20 Elements, 23 Relationships and 54 Checks. The code passed every Check in all three validations; no Check failed, was not verifiable or unimplementable.
- The Target architecture was revised twice, both for proposed additions: 1 after the first validation, 3 after the second, 0 after the third. The set of gaps did not shrink between the first two passes, so the loop converged without the criterion "never grows" holding.
- The router asked Jev whether each gap was material: 0.92 and 0.88, both above the 0.75 threshold. This is the first use of Jev inside an Execution.
- The target architect made 17 schema lookups in its first attempt (39 in case 01) and did not exhaust its budget.
- By role: IaC author 56 requests and 1.32 million tokens; target architect 57 and 1.20 million; describer 24 and 942 thousand, rewriting the description on each Revision.
Changed after the run, not yet run: iteration 4 steps 21 and 22.
Acceptance criteria: cases 01 and 02 complete at 1.0. Cases 03 and 04 have not run on this build.