Files
gcp_solution_architecture_a…/.agents/plans/iter-05-jev-decisions/steps/step-15.md
Jonathan Boniface e5e1647ff1
Some checks failed
validation / verify (push) Failing after 13s
feat: prove the lessons, check overlays in evaluation, and settle the cost steps
- Schema review: a dotted path is judged only when it starts at a name the
  resource type has, so a database flag or a service name is no longer read
  as a missing setting. Lesson L-04, which worked around that, is retired.
- Skill routing: in a Change request the question says what changed.
- Evaluation: a case may list code_standards the infrastructure code must
  meet; case 06 checks its overlay's regions, naming and labels.
- Model tiers (iteration 5 step 6): a lighter model for a role's first
  attempt, named in the provider bundle. Measured on case 03 and left off:
  Flash saved five cents on the IaC author and took twice the requests.
- Steps 7 and 9 decided against on the numbers, recorded in the plans.
- Plans: the runs of cases 05 and 06, and the first proof of the lessons on
  case 03 (no Revision, 22 minutes against 28, quality held).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:27:09 +01:00

1.9 KiB

Step 15 — Implementation Detail

Built on 2026-10-09. Not run in a Change request with real models.

settle_meaning in app/tools/meaning.py: for each Check, Element, Relationship and recommendation whose text differs from the Baseline's, one yes or no with the old and the new: does the new text change what must be true? An item at or below JEV_REWORDED_THRESHOLD (0.25) moves from changed to reworded in the change summary, and leaves unrequested if it was there. Every probability is recorded under meaning, by id. With no decision an item stays changed.

It runs when the Execution ends, on the summary written to changes.json. The summary shown at each Approval, and the evaluation's change checks, still compare text exactly.

Measured against live Jev on nine pairs:

Pair P(changed)
A Check reworded with the argument's name 0.12
"at least 2" to "2 or more" 0.22
A recommendation reworded 0.10
An Element renamed, same product and type 0.30
A Check moved to another Element 0.41
"only" dropped from a grant 0.61
A value changed, 30 to 2555 0.96
A condition added 0.95
The product changed 0.90

At 0.25 the three rewordings are read as such and no real change is. The dropped "only", the case the plan names, is caught at 0.61 but not by much.

Differs from the plan: requirements are not asked about. They carry their own change and reason, and an unchanged one must keep its text exactly (iteration 7 step 3). The Baseline's verdict is not carried forward for a reworded Check: validation still gives a verdict on every Check.

Tests: tests/execution/test_change_requests.py, "changed, or only reworded?" (2).

Run on 2026-10-09

Ran in case 05's Change request. The Specialists changed no existing item's text: Checks and recommendations were added and removed by id, so there was nothing to settle and nothing listed as reworded.