Iteration 4: - Overlay Skills (step 12): an Execution names an organisation's standards Skill from the folders the deployment configures; recorded with its version. An example standards Skill and an evaluation case that uses it. - Lessons (step 10): a prove command that compares a run with lessons against one without, and triage of evidence an existing lesson already covers. Iteration 5: - Step 8: the references of a Specialist's Skills its task needs are named in its context as read_first. - Step 14: a remote MCP server's reply that instructs the agent is withheld; the user's own text is flagged as a risk and nothing more. - Step 15: in a Change request, items only reworded are listed apart from real changes. - Step 19: each Execution's design recommendations are judged when it ends and the scores kept in its record. The evaluation takes its rubric from the same place. Each question was calibrated against live Jev; none has run in a case. Steps 6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
112 lines
4.1 KiB
Python
112 lines
4.1 KiB
Python
"""Changed, or only reworded? (ADR-0007, iteration 5 step 15)
|
|
|
|
A Specialist revising from a Baseline rewords things it was not asked to change.
|
|
Compared as text, every rewording is a change; ignoring text would miss a dropped
|
|
restriction, such as "only" removed from a Check. For each item whose text differs
|
|
from the Baseline's, this asks the decider whether the new text changes what must be
|
|
true. An item it is confident did not is listed as reworded, not changed.
|
|
|
|
Without a decision an item stays changed: that is the safe reading.
|
|
|
|
Measured against live Jev on 2026-10-09 with nine pairs: three rewordings scored 0.10
|
|
to 0.22 and a renamed Element 0.30; five real changes scored 0.41 (a Check moved to
|
|
another Element), 0.61 (a dropped "only"), and 0.90 to 0.96. At 0.25 no real change
|
|
was read as a rewording.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import copy
|
|
import json
|
|
import logging
|
|
from collections.abc import Mapping
|
|
from typing import Any
|
|
|
|
from app.adk.runtime import Decider, Question
|
|
from app.workflows.chain import RECOMMENDATIONS_FILE
|
|
from app.workflows.changes import CHANGED, _graph, _items
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
REWORDED = "reworded"
|
|
_TEXT_OF = {
|
|
"checks": ("statement", "applies_to", "source"),
|
|
"elements": ("name", "product", "resource_type", "component"),
|
|
"relationships": ("from", "kind", "to"),
|
|
"recommendations": ("text", "components"),
|
|
}
|
|
|
|
|
|
def _described(item: Mapping[str, Any], part: str) -> str:
|
|
return json.dumps(
|
|
{name: item.get(name) for name in _TEXT_OF[part] if item.get(name) is not None},
|
|
ensure_ascii=False,
|
|
)
|
|
|
|
|
|
def _by_id(deliverables: Mapping[str, Any] | None, part: str) -> dict[str, dict]:
|
|
items = (
|
|
_items(deliverables, "design_recommendations", RECOMMENDATIONS_FILE)
|
|
if part == "recommendations"
|
|
else _graph(deliverables)[part]
|
|
)
|
|
return {str(item.get("id")): item for item in items}
|
|
|
|
|
|
async def settle_meaning(
|
|
decider: Decider,
|
|
summary: Mapping[str, Any],
|
|
baseline: Mapping[str, Any] | None,
|
|
current: Mapping[str, Any] | None,
|
|
*,
|
|
unchanged_at: float = 0.25,
|
|
scope: str = "",
|
|
) -> dict[str, Any]:
|
|
"""The change summary with items that were only reworded listed as such.
|
|
|
|
An item moves from `changed` to `reworded` when the probability that its new
|
|
text changes what must be true is at or below `unchanged_at`. Each decision is
|
|
recorded under `meaning`, by id.
|
|
"""
|
|
settled = copy.deepcopy(dict(summary))
|
|
questions: dict[str, Question] = {}
|
|
where: dict[str, str] = {}
|
|
for part in _TEXT_OF:
|
|
old, new = _by_id(baseline, part), _by_id(current, part)
|
|
for identifier in settled.get(part, {}).get(CHANGED, []):
|
|
if identifier not in old or identifier not in new:
|
|
continue
|
|
questions[identifier] = Question(
|
|
"noul",
|
|
f"Before: {_described(old[identifier], part)}\n"
|
|
f"After: {_described(new[identifier], part)}\n"
|
|
"Does the new text change what must be true of the system, as "
|
|
"opposed to saying the same thing in other words?",
|
|
)
|
|
where[identifier] = part
|
|
if not questions:
|
|
return settled
|
|
answers = await decider.decide(
|
|
"Two versions of one item of a cloud solution architecture.",
|
|
questions,
|
|
scope=scope,
|
|
)
|
|
if not answers:
|
|
return settled
|
|
meaning: dict[str, float] = {}
|
|
for identifier, part in where.items():
|
|
answer = answers.get(identifier)
|
|
if answer is None or not isinstance(answer.value, (int, float)):
|
|
continue
|
|
meaning[identifier] = round(float(answer.value), 3)
|
|
if answer.value <= unchanged_at:
|
|
settled[part][CHANGED].remove(identifier)
|
|
settled[part].setdefault(REWORDED, []).append(identifier)
|
|
unrequested = settled.get("unrequested", {}).get(part)
|
|
if unrequested and identifier in unrequested:
|
|
unrequested.remove(identifier)
|
|
if not unrequested:
|
|
del settled["unrequested"][part]
|
|
settled["meaning"] = meaning
|
|
return settled
|