feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)
Iteration 4: - Overlay Skills (step 12): an Execution names an organisation's standards Skill from the folders the deployment configures; recorded with its version. An example standards Skill and an evaluation case that uses it. - Lessons (step 10): a prove command that compares a run with lessons against one without, and triage of evidence an existing lesson already covers. Iteration 5: - Step 8: the references of a Specialist's Skills its task needs are named in its context as read_first. - Step 14: a remote MCP server's reply that instructs the agent is withheld; the user's own text is flagged as a risk and nothing more. - Step 15: in a Change request, items only reworded are listed apart from real changes. - Step 19: each Execution's design recommendations are judged when it ends and the scores kept in its record. The evaluation takes its rubric from the same place. Each question was calibrated against live Jev; none has run in a case. Steps 6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
27
.agents/plans/iter-05-jev-decisions/steps/step-15.md
Normal file
27
.agents/plans/iter-05-jev-decisions/steps/step-15.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# Step 15 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09. **Not run in a Change request with real models.**
|
||||
|
||||
`settle_meaning` in `app/tools/meaning.py`: for each Check, Element, Relationship and recommendation whose text differs from the Baseline's, one yes or no with the old and the new: does the new text change what must be true? An item at or below `JEV_REWORDED_THRESHOLD` (0.25) moves from `changed` to `reworded` in the change summary, and leaves `unrequested` if it was there. Every probability is recorded under `meaning`, by id. With no decision an item stays changed.
|
||||
|
||||
It runs when the Execution ends, on the summary written to `changes.json`. The summary shown at each Approval, and the evaluation's change checks, still compare text exactly.
|
||||
|
||||
Measured against live Jev on nine pairs:
|
||||
|
||||
| Pair | P(changed) |
|
||||
|---|---|
|
||||
| A Check reworded with the argument's name | 0.12 |
|
||||
| "at least 2" to "2 or more" | 0.22 |
|
||||
| A recommendation reworded | 0.10 |
|
||||
| An Element renamed, same product and type | 0.30 |
|
||||
| A Check moved to another Element | 0.41 |
|
||||
| "only" dropped from a grant | 0.61 |
|
||||
| A value changed, 30 to 2555 | 0.96 |
|
||||
| A condition added | 0.95 |
|
||||
| The product changed | 0.90 |
|
||||
|
||||
At 0.25 the three rewordings are read as such and no real change is. The dropped "only", the case the plan names, is caught at 0.61 but not by much.
|
||||
|
||||
Differs from the plan: requirements are not asked about. They carry their own `change` and reason, and an `unchanged` one must keep its text exactly (iteration 7 step 3). The Baseline's verdict is not carried forward for a reworded Check: validation still gives a verdict on every Check.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py`, "changed, or only reworded?" (2).
|
||||
Reference in New Issue
Block a user