feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)

Iteration 4:
- Overlay Skills (step 12): an Execution names an organisation's standards
  Skill from the folders the deployment configures; recorded with its version.
  An example standards Skill and an evaluation case that uses it.
- Lessons (step 10): a prove command that compares a run with lessons against
  one without, and triage of evidence an existing lesson already covers.

Iteration 5:
- Step 8: the references of a Specialist's Skills its task needs are named in
  its context as read_first.
- Step 14: a remote MCP server's reply that instructs the agent is withheld;
  the user's own text is flagged as a risk and nothing more.
- Step 15: in a Change request, items only reworded are listed apart from
  real changes.
- Step 19: each Execution's design recommendations are judged when it ends and
  the scores kept in its record. The evaluation takes its rubric from the same
  place.

Each question was calibrated against live Jev; none has run in a case. Steps
6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 17:45:56 +01:00
parent 844987587c
commit 567aede5ca
38 changed files with 1956 additions and 58 deletions

View File

@@ -713,3 +713,52 @@ async def test_a_first_consultation_is_given_no_hint(tmp_path):
await engine.start("Ingest webhooks", pre_approved=True)
assert seen["pillars_to_consult"] is None and decider.asked == []
# --- changed, or only reworded? ---------------------------------------------------------
async def test_an_item_that_was_only_reworded_is_listed_apart_from_real_changes():
from app.adk.fake_runtime import FakeDecider
from app.adk.runtime import Answer
from app.tools.meaning import settle_meaning
from app.workflows.changes import change_summary
def revise(graph):
graph["checks"][0]["statement"] = "The service runs at least three instances."
graph["checks"][1]["statement"] = "This service alone uses the identity."
before, after = versions(revise)
summary = change_summary(before, after)
assert summary["checks"]["changed"] == ["K-1", "K-2"]
decider = FakeDecider({"K-1": Answer(0.95, 0.9), "K-2": Answer(0.1, 0.8)})
settled = await settle_meaning(decider, summary, before, after, scope="v2")
assert settled["checks"]["changed"] == ["K-1"]
assert settled["checks"]["reworded"] == ["K-2"]
assert settled["meaning"] == {"K-1": 0.95, "K-2": 0.1}
# K-2 was outside the reach of what changed; reworded, it is no longer unrequested.
assert settled["unrequested"] == {}
_, questions, scope = decider.asked[0]
assert scope == "v2"
assert "The identity is used by this service only." in questions["K-2"].instructions
assert "This service alone uses the identity." in questions["K-2"].instructions
# The summary given in is not altered.
assert summary["checks"]["changed"] == ["K-1", "K-2"]
async def test_without_a_decision_everything_that_differs_stays_changed():
from app.tools.meaning import settle_meaning
from app.workflows.changes import change_summary
def revise(graph):
graph["checks"][1]["statement"] = "This service alone uses the identity."
before, after = versions(revise)
summary = change_summary(before, after)
settled = await settle_meaning(NoDecider(), summary, before, after)
assert settled["checks"]["changed"] == ["K-2"]
assert "reworded" not in settled["checks"] and "meaning" not in settled