Files
gcp_solution_architecture_a…/app/tools/meaning.py
Jonathan Boniface 567aede5ca feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)
Iteration 4:
- Overlay Skills (step 12): an Execution names an organisation's standards
  Skill from the folders the deployment configures; recorded with its version.
  An example standards Skill and an evaluation case that uses it.
- Lessons (step 10): a prove command that compares a run with lessons against
  one without, and triage of evidence an existing lesson already covers.

Iteration 5:
- Step 8: the references of a Specialist's Skills its task needs are named in
  its context as read_first.
- Step 14: a remote MCP server's reply that instructs the agent is withheld;
  the user's own text is flagged as a risk and nothing more.
- Step 15: in a Change request, items only reworded are listed apart from
  real changes.
- Step 19: each Execution's design recommendations are judged when it ends and
  the scores kept in its record. The evaluation takes its rubric from the same
  place.

Each question was calibrated against live Jev; none has run in a case. Steps
6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:45:56 +01:00

112 lines
4.1 KiB
Python

"""Changed, or only reworded? (ADR-0007, iteration 5 step 15)
A Specialist revising from a Baseline rewords things it was not asked to change.
Compared as text, every rewording is a change; ignoring text would miss a dropped
restriction, such as "only" removed from a Check. For each item whose text differs
from the Baseline's, this asks the decider whether the new text changes what must be
true. An item it is confident did not is listed as reworded, not changed.
Without a decision an item stays changed: that is the safe reading.
Measured against live Jev on 2026-10-09 with nine pairs: three rewordings scored 0.10
to 0.22 and a renamed Element 0.30; five real changes scored 0.41 (a Check moved to
another Element), 0.61 (a dropped "only"), and 0.90 to 0.96. At 0.25 no real change
was read as a rewording.
"""
from __future__ import annotations
import copy
import json
import logging
from collections.abc import Mapping
from typing import Any
from app.adk.runtime import Decider, Question
from app.workflows.chain import RECOMMENDATIONS_FILE
from app.workflows.changes import CHANGED, _graph, _items
logger = logging.getLogger(__name__)
REWORDED = "reworded"
_TEXT_OF = {
"checks": ("statement", "applies_to", "source"),
"elements": ("name", "product", "resource_type", "component"),
"relationships": ("from", "kind", "to"),
"recommendations": ("text", "components"),
}
def _described(item: Mapping[str, Any], part: str) -> str:
return json.dumps(
{name: item.get(name) for name in _TEXT_OF[part] if item.get(name) is not None},
ensure_ascii=False,
)
def _by_id(deliverables: Mapping[str, Any] | None, part: str) -> dict[str, dict]:
items = (
_items(deliverables, "design_recommendations", RECOMMENDATIONS_FILE)
if part == "recommendations"
else _graph(deliverables)[part]
)
return {str(item.get("id")): item for item in items}
async def settle_meaning(
decider: Decider,
summary: Mapping[str, Any],
baseline: Mapping[str, Any] | None,
current: Mapping[str, Any] | None,
*,
unchanged_at: float = 0.25,
scope: str = "",
) -> dict[str, Any]:
"""The change summary with items that were only reworded listed as such.
An item moves from `changed` to `reworded` when the probability that its new
text changes what must be true is at or below `unchanged_at`. Each decision is
recorded under `meaning`, by id.
"""
settled = copy.deepcopy(dict(summary))
questions: dict[str, Question] = {}
where: dict[str, str] = {}
for part in _TEXT_OF:
old, new = _by_id(baseline, part), _by_id(current, part)
for identifier in settled.get(part, {}).get(CHANGED, []):
if identifier not in old or identifier not in new:
continue
questions[identifier] = Question(
"noul",
f"Before: {_described(old[identifier], part)}\n"
f"After: {_described(new[identifier], part)}\n"
"Does the new text change what must be true of the system, as "
"opposed to saying the same thing in other words?",
)
where[identifier] = part
if not questions:
return settled
answers = await decider.decide(
"Two versions of one item of a cloud solution architecture.",
questions,
scope=scope,
)
if not answers:
return settled
meaning: dict[str, float] = {}
for identifier, part in where.items():
answer = answers.get(identifier)
if answer is None or not isinstance(answer.value, (int, float)):
continue
meaning[identifier] = round(float(answer.value), 3)
if answer.value <= unchanged_at:
settled[part][CHANGED].remove(identifier)
settled[part].setdefault(REWORDED, []).append(identifier)
unrequested = settled.get("unrequested", {}).get(part)
if unrequested and identifier in unrequested:
unrequested.remove(identifier)
if not unrequested:
del settled["unrequested"][part]
settled["meaning"] = meaning
return settled