F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
Some checks failed
check agents / check (push) Failing after 14s
Some checks failed
check agents / check (push) Failing after 14s
- Critic 0.3.0: the hooks look up every disposition citation in the verified Discovery package (app/evidence.py). An unresolved citation fails K-02 and rejects the disposition; whether the records support the basis stays the model's judgement. - Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref is the primary record), ADR 0013 addendum. - Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering ALZ-ASM-02). - Scaffold: builds every prompt before deleting anything. - Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15). - Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -35,3 +35,8 @@ Facts for this Feature's individual-agents pattern, for coding agents. Repositor
|
||||
- **The critic's recalculation rules (critic 0.1.6):** the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (`RECALC_RULES`), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (run `exec_1779863642d6`, K-04).
|
||||
- **Critic 0.2.0 (ADR 0011):** K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its `app/` and `tests/` when regenerating. Run `exec_4d88027a4d26` failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50).
|
||||
- **Run `exec_2cd7a953335b` (critic 0.2.0):** code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule counted `represents` edges (ADR 0012; risk 0.1.1, disposition 0.1.2).
|
||||
- **Critic 0.3.0 (ADR 0013):** K-02's citation lookup is in code (`app/evidence.py`, against the verified package); the model judges only support. Run `exec_1f518b04d6e4` failed K-02 on lookups that resolve.
|
||||
- **Run `exec_d170329cba65` (critic 0.3.0, 2026-11-06):** every citation resolved in code; K-04 to K-12 passed except K-10 (unverified). Failed K-03: the profiler had its own dependents rule without ADR 0012's `represents` exclusion and listed CI0010001 to CI0010008 as dependents (profiler 0.1.1). Failed K-02 on the model's support judgement, partly for missing rate-card evidence, which the prompt says isn't Discovery evidence.
|
||||
- **Run `exec_659b13e5f947` (profiler 0.1.1):** K-03 passed (no dependents). Failed only K-02: the model judged the one `evidence_ref` alone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum).
|
||||
- **The scaffold builds every prompt before it deletes anything** (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
|
||||
- **Run `exec_49e355141382` (critic 0.3.1, 2026-11-06): verdict pass**, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in run `exec_659b13e5f947` on the same package: the disposition agent applies the adjustments differently between runs.
|
||||
|
||||
@@ -1,3 +1,17 @@
|
||||
## [0.3.1] - 2026-11-06
|
||||
|
||||
### Changed
|
||||
|
||||
- K-02 judges support on a disposition's citations taken together (ADR 0013, addendum of 2026-11-06): `evidence_ref` is the primary record and `evidence_refs` the rest. A disposition covers a whole application, so it is never failed because its primary record alone covers one asset. Run `exec_659b13e5f947` failed K-02 on the one `evidence_ref` (`ALZ-DSC-03/CI-001`) although it found that the 19 citations together supported the basis.
|
||||
|
||||
## [0.3.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
|
||||
- K-02's lookup in code (ADR 0013, `app/evidence.py`): every disposition's `evidence_ref` and `evidence_refs` must resolve to a record in the verified Discovery package (by agent and record key). A citation that doesn't resolve fails K-02 and rejects the disposition. Whether the cited records support the rule-trace basis stays the model's judgement.
|
||||
- The prompt says that a resolved citation is valid, and that the rate card is cited by `rate_id` in the TCO line items, never as Discovery evidence. Run `exec_1f518b04d6e4` failed K-02 on both: it said `ALZ-DSC-05/CI-002` and `CI-010` don't resolve (they do), and that the rate card isn't a Discovery record.
|
||||
- Tests on Discovery run `exec_58ba6a71d00b`'s real package and the records of runs `exec_4d88027a4d26` and `exec_1f518b04d6e4` (15 in all).
|
||||
|
||||
## [0.2.1] - 2026-11-05
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
name: "cp1-assessment-critic-agent"
|
||||
display_name: "CP1 - Assessment Critic & Sign-off Router Agent"
|
||||
description: "Independently validates the Assessment: coverage, evidence, dependents, recalculated scores, confidence and TCO, the interim and advisory rules, and routes confidence below 0.80 to disposition sign-off."
|
||||
version: "0.2.1"
|
||||
version: "0.3.1"
|
||||
framework: "adk"
|
||||
tags:
|
||||
- "cp1"
|
||||
|
||||
@@ -10,6 +10,8 @@ This module checks only what follows from the producers' own records by arithmet
|
||||
- K-07: the disposition agent's sign-off queue is exactly its dispositions with confidence below 0.80. The critic's
|
||||
own queue is those, less the dispositions it rejects.
|
||||
- K-11: the records passed the input contract on arrival (schema and 7R enum), so they validate.
|
||||
- K-02, its lookup only (ADR 0013, `app/evidence.py`): a citation that doesn't resolve to a package record fails
|
||||
K-02 and rejects the disposition. Whether the records support the basis stays the model's.
|
||||
|
||||
The rules are ALZ-ASM-03's and ALZ-ASM-04's (their prompts' Scoring and Tail, Confidence, and TCO rules). Standard
|
||||
library only.
|
||||
@@ -238,6 +240,14 @@ def apply(reply: Dict[str, Any], found: Dict[str, Any]) -> Dict[str, Any]:
|
||||
rejected.append({"application_id": app, "check_id": cid, "named_dependent": None,
|
||||
"reason": "; ".join(f"{x['record']}: expected {x['expected']}, recorded {x['recorded']}"
|
||||
for x in found[cid][app])})
|
||||
for app, problems in (found.get("K-02") or {}).items():
|
||||
if not any(r.get("application_id") == app and r.get("check_id") == "K-02" for r in rejected):
|
||||
rejected.append({"application_id": app, "check_id": "K-02", "named_dependent": None,
|
||||
"reason": "; ".join(f"{x['recorded']}: {x['reason']}" for x in problems)})
|
||||
if found.get("K-02"):
|
||||
model = checks.get("K-02") or {}
|
||||
checks["K-02"] = {"check_id": "K-02", "result": "fail",
|
||||
"details": [x for v in found["K-02"].values() for x in v] + list(model.get("details") or [])}
|
||||
rejected_apps = {r.get("application_id") for r in rejected}
|
||||
queue = sorted(a for a, c in found["confidence"].items() if c is not None and float(c) < SIGNOFF_BELOW
|
||||
and a not in rejected_apps)
|
||||
@@ -262,6 +272,6 @@ def apply(reply: Dict[str, Any], found: Dict[str, Any]) -> Dict[str, Any]:
|
||||
summary = (f"Verdict {verdict}, set in code from the checks (ADR 0011): "
|
||||
+ (f"{', '.join(failed)} failed" if failed else "no check failed")
|
||||
+ (f"; {', '.join(unverified)} unverified" if unverified else "")
|
||||
+ ". K-04 to K-07 and K-11 are checked in code; the other checks are the critic model's.")
|
||||
+ ". K-04 to K-07, K-11 and K-02's lookup are checked in code; the other checks are the critic model's.")
|
||||
return {**reply, "verdict": verdict, "checks": ordered, "rejected_dispositions": rejected,
|
||||
"signoff_queue": queue, "unverified_claims": notes, "summary": summary}
|
||||
|
||||
@@ -14,7 +14,7 @@ import contextvars
|
||||
import logging
|
||||
from typing import Any, Optional
|
||||
|
||||
from app import arithmetic
|
||||
from app import arithmetic, evidence
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
_found: contextvars.ContextVar[Optional[dict]] = contextvars.ContextVar("critic_checked_in_code", default=None)
|
||||
@@ -25,6 +25,7 @@ def pre_process(payload: dict) -> Optional[Any]:
|
||||
_found.set(None) # the planner refused: the critic returns fail on K-01 alone, and nothing is checked here
|
||||
return None
|
||||
found = arithmetic.check(payload)
|
||||
found["K-02"] = evidence.resolve(payload) # the lookup only (ADR 0013)
|
||||
_found.set(found)
|
||||
payload["checked_in_code"] = {
|
||||
"note": "K-04, K-05, K-06, K-07 and K-11 are checked in code from the producers' records (ADR 0011), and "
|
||||
@@ -35,6 +36,8 @@ def pre_process(payload: dict) -> Optional[Any]:
|
||||
"K-06": found["K-06"] or "every TCO model adds up against the rate card",
|
||||
"K-07": found["K-07"] or "the disposition agent routed exactly its dispositions below 0.80",
|
||||
"K-11": found["K-11"] or "every record passed the input contract; every category is in the 7R enum",
|
||||
"K-02 lookup": found["K-02"] or ("every disposition's evidence_ref resolves to a record in the Discovery "
|
||||
"package; judge only whether the cited records support the rule-trace basis"),
|
||||
}
|
||||
return payload
|
||||
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""K-02's lookup, in code (F02 ADR 0013): does every disposition's evidence_ref resolve to a record in the package?
|
||||
|
||||
A citation is `discovery:<discovery_execution_id>/<agent_id>/<record_key>`. It resolves when the execution id is the
|
||||
run's, the agent is one whose records Assessment cites, and that agent's part of the Discovery package holds a record
|
||||
with that key in one of its id fields. A part's own field name (`itsm_signals`, say) also resolves: it cites the list,
|
||||
for example to show that it is empty.
|
||||
|
||||
Whether the cited records *support* the rule-trace basis is the critic model's judgement, not this module's.
|
||||
|
||||
The package is found the way the intake finds it (F02 ADR 0008, `cp1-assessment-intake-agent/app/intake.py`): the
|
||||
intake has already verified it, so it is read here, not checked again. Standard library only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, Iterator, List, Optional, Set
|
||||
|
||||
FILE_SEPARATOR = "\n\n---\n\n" # the platform's join between parsed files, as in the intake
|
||||
# The Discovery agents Assessment cites, and their part of the package (F01 ADR 0020).
|
||||
PART_OF = {"ALZ-DSC-03": "inventory", "ALZ-DSC-04": "cmdb", "ALZ-DSC-05": "classification",
|
||||
"ALZ-DSC-06": "dependencies", "ALZ-DSC-07": "network"}
|
||||
ID_FIELDS = ("canonical_id", "drift_id", "ci_ref", "edge_id", "id", "finding_id", "rule_id", "range_id", "flow_id")
|
||||
|
||||
|
||||
def _decode(value: Any, depth: int = 0) -> Any:
|
||||
while isinstance(value, str) and depth < 3:
|
||||
try:
|
||||
value, depth = json.loads(value), depth + 1
|
||||
except ValueError:
|
||||
return value
|
||||
return value
|
||||
|
||||
|
||||
def _packages(value: Any, depth: int = 0) -> Iterator[Dict[str, Any]]:
|
||||
value = _decode(value)
|
||||
if depth > 6:
|
||||
return
|
||||
if isinstance(value, dict):
|
||||
if "package_version" in value and isinstance(value.get("manifest"), dict):
|
||||
yield value
|
||||
return
|
||||
for item in value.values():
|
||||
yield from _packages(item, depth + 1)
|
||||
elif isinstance(value, list):
|
||||
for item in value:
|
||||
yield from _packages(item, depth + 1)
|
||||
|
||||
|
||||
def package(documents: Optional[str]) -> Optional[Dict[str, Any]]:
|
||||
found = [p for doc in (documents or "").split(FILE_SEPARATOR) for p in _packages(doc.strip())]
|
||||
return found[0] if len(found) == 1 else None
|
||||
|
||||
|
||||
def index(pkg: Dict[str, Any]) -> Dict[str, Set[str]]:
|
||||
"""Each cited agent's record keys: the id fields of every record in its part, and the part's own field names."""
|
||||
keys: Dict[str, Set[str]] = {}
|
||||
for agent, part in PART_OF.items():
|
||||
found: Set[str] = set()
|
||||
body = pkg.get(part) or {}
|
||||
for field, value in (body.items() if isinstance(body, dict) else []):
|
||||
found.add(field)
|
||||
for record in value if isinstance(value, list) else []:
|
||||
if isinstance(record, dict):
|
||||
found.update(str(record[f]) for f in ID_FIELDS if record.get(f) not in (None, ""))
|
||||
keys[agent] = found
|
||||
return keys
|
||||
|
||||
|
||||
def unresolved(ref: Any, execution_id: str, keys: Dict[str, Set[str]]) -> Optional[str]:
|
||||
"""Why `ref` doesn't resolve, or None when it does."""
|
||||
text = str(ref or "")
|
||||
if not text.startswith("discovery:"):
|
||||
return "not a discovery: citation"
|
||||
parts = text[len("discovery:"):].split("/", 2)
|
||||
if len(parts) != 3:
|
||||
return "not discovery:<execution_id>/<agent_id>/<record_key>"
|
||||
run, agent, key = parts
|
||||
if run != execution_id:
|
||||
return f"from {run}, not this Discovery run {execution_id}"
|
||||
if agent not in keys:
|
||||
return f"{agent} is not a Discovery agent whose records Assessment cites ({', '.join(PART_OF)})"
|
||||
if key not in keys[agent]:
|
||||
return f"no record {key} in {agent}'s part ({PART_OF[agent]}) of the package"
|
||||
return None
|
||||
|
||||
|
||||
def resolve(payload: Dict[str, Any]) -> Dict[str, List[dict]]:
|
||||
"""K-02 lookup details per application: empty when every disposition's citations resolve."""
|
||||
execution_id = payload.get("discovery_execution_id") or ""
|
||||
pkg = package(payload.get("discovery_documents"))
|
||||
out: Dict[str, List[dict]] = {}
|
||||
for d in payload.get("dispositions") or []:
|
||||
app = d.get("application_id", "?")
|
||||
refs = [d.get("evidence_ref")] + list(d.get("evidence_refs") or [])
|
||||
if pkg is None:
|
||||
out[app] = [{"record": app, "expected": "one Discovery package among discovery_documents",
|
||||
"recorded": "none, or more than one", "reason": "the citations can't be looked up"}]
|
||||
continue
|
||||
keys = index(pkg)
|
||||
problems = []
|
||||
for ref in dict.fromkeys(r for r in refs if r):
|
||||
why = unresolved(ref, execution_id, keys)
|
||||
if why:
|
||||
problems.append({"record": app, "expected": "a citation that resolves to a record in the package",
|
||||
"recorded": str(ref), "reason": why})
|
||||
if problems:
|
||||
out[app] = problems
|
||||
return out
|
||||
@@ -11,5 +11,6 @@ The decisions are recorded once, for all six agents, in the Feature's [docs/adr/
|
||||
- [ADR 0010](../../../docs/adr/0010.md)
|
||||
- [ADR 0011](../../../docs/adr/0011.md)
|
||||
- [ADR 0012](../../../docs/adr/0012.md)
|
||||
- [ADR 0013](../../../docs/adr/0013.md)
|
||||
|
||||
A change that affects only this agent gets its own record here, numbered `06-NNNN`.
|
||||
|
||||
@@ -43,6 +43,13 @@ were called (ADR 0011). Code sets those checks, the sign-off queue and the verdi
|
||||
spend your effort on the checks only you can make: coverage, evidence, dependents, advisory security, telemetry,
|
||||
tool calls and model independence. Where you read a code check differently, say why in its details.
|
||||
|
||||
`checked_in_code` also says whether every disposition's `evidence_ref` resolves to a record in the Discovery package
|
||||
(K-02's lookup, ADR 0013). A citation that resolves is a valid citation: for K-02, judge only whether the cited records
|
||||
support the rule-trace basis. Judge a disposition's citations together: `evidence_ref` is the primary record and
|
||||
`evidence_refs` the rest, and the basis is supported when the records cited across both support it. A disposition
|
||||
covers a whole application, so never fail it because its primary record alone covers only one asset. The rate card is cited by `rate_id` in the TCO line items (K-06), never as Discovery
|
||||
evidence, so a basis that rests on the rate card is not unsupported for lacking a `discovery:` citation.
|
||||
|
||||
If `preconditions.failed` is not empty, the planner refused to start: the later steps ran on empty inputs. Return
|
||||
`verdict: fail`, with a K-01 detail naming each failed precondition, and run no other check.
|
||||
|
||||
@@ -171,7 +178,7 @@ gravest failure available to you. You reject and explain; you never rescore.
|
||||
| Check | Passes when | Fails | On failure |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **K-01** Coverage | Every planned application has a disposition or a stated gap | Any silent drop | Run `fail` |
|
||||
| **K-02** Evidence | Every `evidence_ref` is `discovery:{plan.discovery_execution_id}/<agent_id>/<record_key>` and resolves to a supplied record that supports the rule-trace basis | Missing, other run, unresolvable or unsupportive | Reject disposition |
|
||||
| **K-02** Evidence | Every citation of a disposition (`evidence_ref`, its primary record, and each of `evidence_refs`) is `discovery:{plan.discovery_execution_id}/<agent_id>/<record_key>` and resolves to a supplied record; the citations taken together support the rule-trace basis | Missing, other run, unresolvable, or the citations together are unsupportive | Reject disposition |
|
||||
| **K-03** Dependents | No `retire` with dependents (lifted from ALZ-DSC-06 edges, unmapped sources included) or with an asset whose `retire_constraint` is not `none`; dependents stated in the scoring note | Dependent ignored | Reject, name dependent |
|
||||
| **K-04** Risk recalculation | Every ALZ-ASM-03 score, `partial` flag and tail rule recalculates | Any mismatch | Run `fail` |
|
||||
| **K-05** Confidence recalculation | Every confidence recalculates from 0.90, P1–P6 and C1, and interim categories ≤ 0.79 | Mismatch or cap breached | Reject disposition |
|
||||
|
||||
@@ -0,0 +1,314 @@
|
||||
{
|
||||
"source": "Assessment run exec_1f518b04d6e4 (RUN-ASM-AZURE-009), crucible-kab-dev, 2026-11-06: the disposition step's output, and the critic model's reply (it failed K-02, saying the ALZ-DSC-05 citations don't resolve)",
|
||||
"dispositions": [
|
||||
{
|
||||
"application_id": "APP-001",
|
||||
"disposition": "rehost",
|
||||
"interim_rule": false,
|
||||
"rule_trace": [
|
||||
{
|
||||
"category": "retire",
|
||||
"matched": false,
|
||||
"basis": "No external dependent application remains after represents edges and internal APP-001 edges are excluded, but CI-002, CI-009 and CI-010 have retire_constraint cannot_retire_without_mitigation, and out-of-support status is unknown for every asset."
|
||||
},
|
||||
{
|
||||
"category": "repurchase",
|
||||
"matched": false,
|
||||
"basis": "The application consists of cloud-native managed resources rather than a recorded firewall, appliance or packaged-software product; little customisation and out-of-support or extended-support status are also not recorded."
|
||||
},
|
||||
{
|
||||
"category": "replatform",
|
||||
"matched": false,
|
||||
"basis": "The rate card contains managed equivalents for the recorded services, but the required out-of-support or extended-support condition is unknown."
|
||||
},
|
||||
{
|
||||
"category": "refactor",
|
||||
"matched": false,
|
||||
"basis": "Recorded asset criticality is Low or Medium rather than high or critical, and no unsupported-runtime blocker without a rate-card equivalent is recorded."
|
||||
},
|
||||
{
|
||||
"category": "retain",
|
||||
"matched": false,
|
||||
"basis": "No appliance without an equivalent is recorded, and the two data stores are unclassified rather than subject to a recorded constraint that rate-card targets cannot host."
|
||||
},
|
||||
{
|
||||
"category": "relocate",
|
||||
"matched": false,
|
||||
"basis": "All assets have cloud hosting rather than virtual hosting, and the rate card does not list a hosted hypervisor service."
|
||||
},
|
||||
{
|
||||
"category": "rehost",
|
||||
"matched": true,
|
||||
"basis": "Selected as the final fallback because no earlier category matches and rate-card mappings exist for all ten assets; platform support is unknown and is reflected in confidence rather than inferred."
|
||||
}
|
||||
],
|
||||
"confidence": 0.6,
|
||||
"confidence_workings": {
|
||||
"base": 0.9,
|
||||
"adjustments": [
|
||||
{
|
||||
"rule": "P1 telemetry_unavailable",
|
||||
"delta": -0.1,
|
||||
"basis": "Disposition is rehost and runtime sizing telemetry is unavailable."
|
||||
},
|
||||
{
|
||||
"rule": "P2 unknown_input",
|
||||
"delta": -0.1,
|
||||
"basis": "The platform-support signal needed by the rehost rule is unknown because support_status and end_of_support are absent for all ten assets."
|
||||
},
|
||||
{
|
||||
"rule": "P3 partial_risk_score",
|
||||
"delta": -0.1,
|
||||
"basis": "ALZ-ASM-03 marked the application risk score partial, with eol, data and change factors missing."
|
||||
}
|
||||
],
|
||||
"cap": null,
|
||||
"result": 0.6
|
||||
},
|
||||
"signoff_required": true,
|
||||
"evidence_ref": "discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-001",
|
||||
"evidence_refs": [
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-001",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-002",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-003",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-004",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-005",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-006",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-007",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-008",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-009",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-010",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-05/CI-002",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-05/CI-010",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-001",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-002",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-003",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-004",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-005",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-006",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-06/E-007"
|
||||
],
|
||||
"tco_model": {
|
||||
"currency": "USD",
|
||||
"current_annual": null,
|
||||
"target_annual": 11622,
|
||||
"annual_delta": null,
|
||||
"line_items": [
|
||||
{
|
||||
"canonical_id": "CI-001",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-01",
|
||||
"target": 9600
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-002",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-02",
|
||||
"target": 1500
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-003",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-07",
|
||||
"target": 0
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-004",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-03",
|
||||
"target": 60
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-005",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-03",
|
||||
"target": 60
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-006",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-08",
|
||||
"target": 0
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-007",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-08",
|
||||
"target": 0
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-008",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-05",
|
||||
"target": 12
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-009",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-06",
|
||||
"target": 90
|
||||
},
|
||||
{
|
||||
"canonical_id": "CI-010",
|
||||
"current": null,
|
||||
"rate_id": "RC-AZ-04",
|
||||
"target": 300
|
||||
}
|
||||
],
|
||||
"missing_inputs": [
|
||||
"CI-001.facts.annual_cost",
|
||||
"CI-002.facts.annual_cost",
|
||||
"CI-003.facts.annual_cost",
|
||||
"CI-004.facts.annual_cost",
|
||||
"CI-005.facts.annual_cost",
|
||||
"CI-006.facts.annual_cost",
|
||||
"CI-007.facts.annual_cost",
|
||||
"CI-008.facts.annual_cost",
|
||||
"CI-009.facts.annual_cost",
|
||||
"CI-010.facts.annual_cost"
|
||||
],
|
||||
"indicative": true
|
||||
},
|
||||
"licensing_note": "All mapped rate-card items state \"No licence\"; no inventory cost breakdown records current licensing cost.",
|
||||
"scoring_note": "No external dependent application remains under the prescribed lifting rule because represents edges are excluded and the remaining edges targeting application assets are internal to APP-001; four unmapped upstream dependencies remain. This is not an interim disposition. Advisory security findings SP-001 through SP-005 were recorded but did not change the disposition or confidence. Discovery also reported a prompt-injection string in CMDB data; it was treated only as data and was not followed."
|
||||
}
|
||||
],
|
||||
"reply": {
|
||||
"verdict": "fail",
|
||||
"discovery_execution_id": "exec_58ba6a71d00b",
|
||||
"checks": [
|
||||
{
|
||||
"check_id": "K-01",
|
||||
"result": "pass",
|
||||
"details": [
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "A disposition or stated gap for every planned application",
|
||||
"recorded": "Disposition present for APP-001; no coverage gaps declared",
|
||||
"reason": "Coverage complete for the single planned application."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-02",
|
||||
"result": "fail",
|
||||
"details": [
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "Only resolvable discovery:{discovery_execution_id}/{agent_id}/{record_key} references that support the rule-trace basis",
|
||||
"recorded": "evidence_refs include discovery:exec_58ba6a71d00b/ALZ-DSC-05/CI-002 and discovery:exec_58ba6a71d00b/ALZ-DSC-05/CI-010",
|
||||
"reason": "These classification references do not resolve to supplied Discovery records under the required record key rule because classification records are supplied as stores and no CI-keyed Discovery artifact was separately supplied."
|
||||
},
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "Only resolvable discovery:{discovery_execution_id}/{agent_id}/{record_key} references that support the rule-trace basis",
|
||||
"recorded": "confidence_workings cites unknown platform-support condition via inventory records only",
|
||||
"reason": "The recorded rule trace states that rate-card mappings exist for all ten assets, but no discovery-form evidence references to the rate card exist because the rate card is not a Discovery record; the disposition basis is therefore only partially evidenced by the cited Discovery references."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-03",
|
||||
"result": "pass",
|
||||
"details": [
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "No retire with dependents or retire_constraint != none; dependents stated correctly if referenced",
|
||||
"recorded": "Disposition is rehost, not retire",
|
||||
"reason": "K-03 retirement prohibition is not breached. Application-level dependents from dependency lifting recalculate to none after excluding internal owners and represents edges."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-04",
|
||||
"result": "pass",
|
||||
"details": []
|
||||
},
|
||||
{
|
||||
"check_id": "K-05",
|
||||
"result": "pass",
|
||||
"details": []
|
||||
},
|
||||
{
|
||||
"check_id": "K-06",
|
||||
"result": "pass",
|
||||
"details": []
|
||||
},
|
||||
{
|
||||
"check_id": "K-07",
|
||||
"result": "pass",
|
||||
"details": []
|
||||
},
|
||||
{
|
||||
"check_id": "K-08",
|
||||
"result": "pass",
|
||||
"details": [
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "Security findings listed only as advisory and not used in confidence adjustments or rule-trace basis",
|
||||
"recorded": "scoring_note says findings SP-001 through SP-005 were recorded but did not change disposition or confidence; confidence adjustments are P1, P2, P3 only",
|
||||
"reason": "Advisory security rule held."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-09",
|
||||
"result": "pass",
|
||||
"details": [
|
||||
{
|
||||
"record": "APP-001",
|
||||
"expected": "telemetry none; sizing unavailable; performance_baseline unavailable",
|
||||
"recorded": "telemetry=none; sizing=unavailable; performance_baseline=unavailable",
|
||||
"reason": "Telemetry honesty holds."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-10",
|
||||
"result": "unverified",
|
||||
"details": [
|
||||
{
|
||||
"record": "call_log",
|
||||
"expected": "No MCP call log reaches this agent in this pattern",
|
||||
"recorded": "No call log supplied to this agent",
|
||||
"reason": "Per ADR 0006 this claim cannot be completed here and is reported unverified only."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"check_id": "K-11",
|
||||
"result": "pass",
|
||||
"details": []
|
||||
},
|
||||
{
|
||||
"check_id": "K-12",
|
||||
"result": "pass",
|
||||
"details": [
|
||||
{
|
||||
"record": "workflow_models",
|
||||
"expected": "critic_model not in executor_models",
|
||||
"recorded": "critic_model=gpt-5.4; executor_models=[gpt-5.6-sol]",
|
||||
"reason": "Model independence holds."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"rejected_dispositions": [
|
||||
{
|
||||
"application_id": "APP-001",
|
||||
"check_id": "K-02",
|
||||
"reason": "Disposition evidence references are not fully resolvable/supportive under the required Discovery citation rules; classification references use unresolved CI-style keys and the stated rate-mapping basis is not grounded by the cited Discovery records.",
|
||||
"named_dependent": null
|
||||
}
|
||||
],
|
||||
"signoff_queue": [],
|
||||
"unverified_claims": [
|
||||
"K-10: no MCP/tool call log was supplied to this agent, so tool-call absence cannot be independently completed here."
|
||||
],
|
||||
"critic_model": "gpt-5.4",
|
||||
"executor_models": [
|
||||
"gpt-5.6-sol"
|
||||
],
|
||||
"summary": "Verdict fail, set in code from the checks (ADR 0011): K-02 failed. K-04 to K-07 and K-11 are checked in code; the other checks are the critic model's."
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -118,3 +118,49 @@ def test_the_model_sees_the_code_results():
|
||||
business_logic.pre_process(inputs)
|
||||
assert inputs["checked_in_code"]["K-04"] == "every risk score recalculates"
|
||||
business_logic.post_process(copy.deepcopy(REPLY))
|
||||
|
||||
|
||||
# --- K-02's lookup (ADR 0013), on Discovery run exec_58ba6a71d00b's real package --------------------------------
|
||||
|
||||
from app import evidence # noqa: E402
|
||||
|
||||
RUN9 = json.loads((ROOT / "tests" / "fixtures" / "run-exec_1f518b04d6e4.json").read_text())
|
||||
|
||||
|
||||
def test_run_009s_citations_resolve_including_the_classification_records():
|
||||
"""The critic model failed K-02 in run exec_1f518b04d6e4, saying ALZ-DSC-05/CI-002 and CI-010 don't resolve."""
|
||||
payload = {**INPUTS, "dispositions": RUN9["dispositions"]}
|
||||
assert evidence.resolve(payload) == {}
|
||||
refs = {r for d in RUN9["dispositions"] for r in d["evidence_refs"]}
|
||||
assert "discovery:exec_58ba6a71d00b/ALZ-DSC-05/CI-002" in refs
|
||||
|
||||
|
||||
def test_with_every_citation_resolved_the_models_support_judgement_stands():
|
||||
inputs = {**copy.deepcopy(INPUTS), "dispositions": copy.deepcopy(RUN9["dispositions"])}
|
||||
out = run(inputs, RUN9["reply"])
|
||||
assert result(out, "K-02") == next(c for c in RUN9["reply"]["checks"] if c["check_id"] == "K-02")
|
||||
assert out["verdict"] == "fail" # the model's K-02 is a judgement, kept
|
||||
|
||||
|
||||
def test_a_citation_that_does_not_resolve_fails_k02_in_code():
|
||||
inputs = copy.deepcopy(INPUTS)
|
||||
d = inputs["dispositions"][0]
|
||||
d["evidence_refs"] = d["evidence_refs"] + ["discovery:exec_58ba6a71d00b/ALZ-DSC-03/CI-999",
|
||||
"discovery:exec_000000000000/ALZ-DSC-03/CI-001",
|
||||
"discovery:exec_58ba6a71d00b/ALZ-DSC-99/CI-001"]
|
||||
out = run(inputs)
|
||||
reasons = [x["reason"] for x in result(out, "K-02")["details"]]
|
||||
assert result(out, "K-02")["result"] == "fail" and out["verdict"] == "fail" and len(reasons) == 3
|
||||
assert any("no record CI-999" in r for r in reasons) and any("not this Discovery run" in r for r in reasons)
|
||||
assert [r["check_id"] for r in out["rejected_dispositions"]] == ["K-02"] and out["signoff_queue"] == []
|
||||
|
||||
|
||||
def test_a_part_cited_as_a_whole_resolves():
|
||||
keys = evidence.index(evidence.package(INPUTS["discovery_documents"]))
|
||||
assert evidence.unresolved("discovery:exec_58ba6a71d00b/ALZ-DSC-04/itsm_signals", "exec_58ba6a71d00b", keys) is None
|
||||
|
||||
|
||||
def test_no_package_fails_k02():
|
||||
inputs = copy.deepcopy(INPUTS)
|
||||
inputs["discovery_documents"] = "no package here"
|
||||
assert result(run(inputs), "K-02")["result"] == "fail"
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
## [0.1.1] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
|
||||
- A `represents` edge is not a dependency (ADR 0012), in the profile's `dependents`, `depends_on` and `dependency_exposure` too. Run `exec_d170329cba65` listed the eight CMDB records matched to APP-001's own assets (`unmapped:cmdb:CI0010001` to `CI0010008`) as its downstream dependents, while the risk scorer, using the same rule with the exclusion, found none. The critic failed K-03.
|
||||
|
||||
## [0.1.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
name: "cp1-cloud-readiness-profiler-agent"
|
||||
display_name: "CP1 - Cloud Readiness Profiler Agent"
|
||||
description: "Builds one metadata-only runtime profile per planned application with seven readiness signals, states that no telemetry exists, and assigns no disposition."
|
||||
version: "0.1.0"
|
||||
version: "0.1.1"
|
||||
framework: "adk"
|
||||
tags:
|
||||
- "cp1"
|
||||
|
||||
@@ -5,5 +5,6 @@ The decisions are recorded once, for all six agents, in the Feature's [docs/adr/
|
||||
- [ADR 0001](../../../docs/adr/0001.md)
|
||||
- [ADR 0002](../../../docs/adr/0002.md)
|
||||
- [ADR 0007](../../../docs/adr/0007.md)
|
||||
- [ADR 0012](../../../docs/adr/0012.md)
|
||||
|
||||
A change that affects only this agent gets its own record here, numbered `02-NNNN`.
|
||||
|
||||
@@ -125,12 +125,13 @@ ALZ-DSC-06 records dependencies between assets (`edges[].source` → `edges[].ta
|
||||
|
||||
```python
|
||||
owner(ci) = the application whose assets include ci, or f"unmapped:<ci>" if none
|
||||
dependents(app) = { owner(e.source) FOR e IN edges IF e.target IN app.assets AND owner(e.source) != app }
|
||||
depends_on(app) = { owner(e.target) FOR e IN edges IF e.source IN app.assets AND owner(e.target) != app }
|
||||
dependency(e) = e.type != 'represents' # a CMDB record matched to its asset is that asset (F02 ADR 0012)
|
||||
dependents(app) = { owner(e.source) FOR e IN edges IF dependency(e) AND e.target IN app.assets AND owner(e.source) != app }
|
||||
depends_on(app) = { owner(e.target) FOR e IN edges IF dependency(e) AND e.source IN app.assets AND owner(e.target) != app }
|
||||
```
|
||||
|
||||
An `unmapped:` dependent (for example a CMDB-only host outside the inventory's applications) still
|
||||
counts, so a dependency is never lost because its source has no application.
|
||||
counts, so a dependency is never lost because its source has no application. A `represents` edge (`type: represents`) is not a dependency: it joins a CMDB record to the inventory asset it stands for, so it is the same asset, and it is left out of both sets.
|
||||
|
||||
`hosting` is `'virtual'` or `'physical'` only when an inventory record states it (asset class
|
||||
`hypervisor`, a recorded platform field, or a cloud source); otherwise `'unknown'`. Never infer it
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
| Status | Accepted |
|
||||
| Date | 2026-11-05 |
|
||||
| Decided by | Product owner, 2026-11-05: "lets solve these first" (the run's faults are fixed before it is repeated); the rule follows Discovery's F01 ADR 0019 |
|
||||
| Requirements changed | ALZ-ASM-03 and ALZ-ASM-04: the Application Dependents rule leaves out `represents` edges. ALZ-ASM-04 and ALZ-ASM-06 use ALZ-ASM-03's rule |
|
||||
| Requirements changed | ALZ-ASM-02, ALZ-ASM-03 and ALZ-ASM-04: the Application Dependents rule leaves out `represents` edges. ALZ-ASM-04 and ALZ-ASM-06 use ALZ-ASM-03's rule |
|
||||
|
||||
## Context
|
||||
|
||||
@@ -25,6 +25,9 @@
|
||||
left out of both an application's dependents and its dependencies.
|
||||
2. **ALZ-ASM-04 and ALZ-ASM-06 carry ALZ-ASM-03's rule,** copied into their prompts by the scaffold, so all three lift
|
||||
dependents the same way.
|
||||
3. **ALZ-ASM-02 leaves them out too** (added 2026-11-06). The profiler has its own copy of the rule, for the
|
||||
profile's `dependents`, `depends_on` and `dependency_exposure`, and was missed at first: in run `exec_d170329cba65`
|
||||
it listed APP-001's eight `represents` edges as downstream dependents, and the critic failed K-03.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# 0013. K-02's lookup runs in code; whether the evidence supports the basis stays the model's
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | Accepted |
|
||||
| Date | 2026-11-06 |
|
||||
| Decided by | Product owner, 2026-11-06: "go ahead with the split" |
|
||||
| Requirements changed | ALZ-ASM-06 K-02: the citation lookup is in code, the support judgement in the model. Extends [ADR 0011](0011.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- K-02 passes when every `evidence_ref` "is `discovery:{discovery_execution_id}/<agent_id>/<record_key>` and resolves to
|
||||
a supplied record that supports the rule-trace basis". The first half is a lookup; the second is a judgement.
|
||||
- In run `exec_1f518b04d6e4`, every other check passed, and the critic model failed K-02 on two grounds:
|
||||
- `ALZ-DSC-05/CI-002` and `ALZ-DSC-05/CI-010` "do not resolve". They do: the package's classification records carry
|
||||
those `canonical_id`s, and the prompt says classification records are cited by `canonical_id`;
|
||||
- the rate-card mappings have no Discovery citation. The rate card isn't Discovery evidence: its items are cited by
|
||||
`rate_id` in the TCO line items, which K-06 checks in code.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **The critic's hooks look up every citation** of every disposition (`evidence_ref` and `evidence_refs`) in the
|
||||
Discovery package the intake verified (`app/evidence.py`). A citation resolves when:
|
||||
- its execution id is the run's `discovery_execution_id`;
|
||||
- its agent is one Assessment cites: ALZ-DSC-03 (inventory), 04 (CMDB and ITSM), 05 (classification),
|
||||
06 (dependencies) or 07 (network);
|
||||
- that agent's part of the package has a record with that key in an id field (`canonical_id`, `drift_id`, `ci_ref`,
|
||||
`edge_id`, a node `id`, or a finding, rule, range or flow id), or the key is the part's own field name (a list
|
||||
cited as a whole, for example to show it's empty).
|
||||
2. **A citation that doesn't resolve fails K-02** and rejects the disposition, in code.
|
||||
3. **When every citation resolves,** the model's K-02 stands as it wrote it: it judges whether the cited records
|
||||
support the rule-trace basis. The prompt says a resolved citation is valid, and that the rate card is cited by
|
||||
`rate_id`, never as Discovery evidence.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A wrong lookup by the model no longer fails a run;** a citation that really is wrong always does.
|
||||
- **The model can still fail K-02 on support,** which is the judgement K-02 exists for.
|
||||
- **The package is read twice,** by the intake (to verify it) and the critic (to look up citations). The finding
|
||||
logic is the intake's, copied, not shared, since the two are separate images.
|
||||
|
||||
## Addendum, 2026-11-06: the citations are judged together
|
||||
|
||||
Decided by the product owner, 2026-11-06 ("Lets go with A recomenadation"), after run `exec_659b13e5f947`.
|
||||
|
||||
- **What happened:** every citation resolved, and the critic model said the broader `evidence_refs` helped, but it
|
||||
failed K-02 because the one required `evidence_ref` (`ALZ-DSC-03/CI-001`) didn't support the basis on its own. K-02's
|
||||
row says "Every `evidence_ref` … supports the rule-trace basis", and the model read it as the single field.
|
||||
- **Decision:** K-02 judges support on a disposition's citations taken together. `evidence_ref` is the primary record
|
||||
and `evidence_refs` the rest. A disposition covers a whole application, so it is never failed because its primary
|
||||
record alone covers only one asset. K-02's row in ALZ-ASM-06 is rewritten to say so (critic 0.3.1).
|
||||
- **Not chosen:** keeping the single-record test and making the disposition's `evidence_ref` the package's application
|
||||
record. That would need the package's `applications` part in the lookup and a change to ALZ-ASM-04.
|
||||
- **Requirements changed:** ALZ-ASM-06's K-02 row.
|
||||
|
||||
@@ -25,5 +25,6 @@ Each agent's `docs/adr/README.md` lists the records that apply to its own story.
|
||||
| [0010](0010.md) | Model independence (K-12) is checked when the workflow is built, not judged by the critic | Accepted | ASM-06 |
|
||||
| [0011](0011.md) | The critic's arithmetic, routing and schema checks run in code; the model judges the rest | Accepted | ASM-06 |
|
||||
| [0012](0012.md) | A `represents` edge is not a dependency | Accepted | ASM-03, 04, 06 |
|
||||
| [0013](0013.md) | K-02's lookup runs in code; whether the evidence supports the basis stays the model's | Accepted | ASM-06 |
|
||||
|
||||
New records copy [template.md](template.md) and take the next number.
|
||||
|
||||
@@ -189,7 +189,7 @@ AGENTS = [
|
||||
output=obj({"discovery_execution_id": S, "profiles": arr(PROFILE),
|
||||
"gaps": arr(obj({"application_id": S, "reason": S})), "findings_note": S},
|
||||
["discovery_execution_id", "profiles", "gaps", "findings_note"], title="Readiness profiles"),
|
||||
rewrites=[]),
|
||||
rewrites=REPRESENTS_REWRITES), # run exec_d170329cba65: the profiler listed eight represents edges as dependents
|
||||
dict(n="03", key="cp1-migration-risk-scorer-agent", ado=3214461, display="CP1 - Migration Risk Scorer Agent",
|
||||
desc="Scores each planned application's migration risk from four weighted factors, names missing factors, "
|
||||
"and flags the high-risk tail by score or hard trigger.",
|
||||
@@ -295,6 +295,15 @@ AGENTS = [
|
||||
"`executor_models`. Report `pass` when `critic_model` is not in `executor_models`. Read nothing else for "
|
||||
"K-12: Discovery is a separate run, and the models of its agents or its critic never fail K-12 | "
|
||||
"`critic_model` is in `executor_models` | Run `fail` |"),
|
||||
# Run exec_659b13e5f947 judged the one required evidence_ref alone, though the 19 citations together supported
|
||||
# the basis: an application's disposition can't rest on one asset's record (ADR 0013, 2026-11-06 addendum).
|
||||
("| **K-02** Evidence | Every `evidence_ref` is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` "
|
||||
"and resolves to a supplied record that supports the rule-trace basis | Missing, other run, unresolvable or "
|
||||
"unsupportive | Reject disposition |",
|
||||
"| **K-02** Evidence | Every citation of a disposition (`evidence_ref`, its primary record, and each of "
|
||||
"`evidence_refs`) is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` and resolves to a "
|
||||
"supplied record; the citations taken together support the rule-trace basis | Missing, other run, "
|
||||
"unresolvable, or the citations together are unsupportive | Reject disposition |"),
|
||||
("- **Scope**: This Assessment run's outputs, the cited Discovery records, the rate card and the call log",
|
||||
"- **Scope**: This Assessment run's outputs, the cited Discovery records and the rate card"),
|
||||
]),
|
||||
@@ -359,6 +368,13 @@ were called (ADR 0011). Code sets those checks, the sign-off queue and the verdi
|
||||
spend your effort on the checks only you can make: coverage, evidence, dependents, advisory security, telemetry,
|
||||
tool calls and model independence. Where you read a code check differently, say why in its details.
|
||||
|
||||
`checked_in_code` also says whether every disposition's `evidence_ref` resolves to a record in the Discovery package
|
||||
(K-02's lookup, ADR 0013). A citation that resolves is a valid citation: for K-02, judge only whether the cited records
|
||||
support the rule-trace basis. Judge a disposition's citations together: `evidence_ref` is the primary record and
|
||||
`evidence_refs` the rest, and the basis is supported when the records cited across both support it. A disposition
|
||||
covers a whole application, so never fail it because its primary record alone covers only one asset. The rate card is cited by `rate_id` in the TCO line items (K-06), never as Discovery
|
||||
evidence, so a basis that rests on the rate card is not unsupported for lacking a `discovery:` citation.
|
||||
|
||||
If `preconditions.failed` is not empty, the planner refused to start: the later steps ran on empty inputs. Return
|
||||
`verdict: fail`, with a K-01 detail naming each failed precondition, and run no other check.
|
||||
"""
|
||||
@@ -701,9 +717,9 @@ cases (it needs `kyndryl-agent-runtime[eval]`). Documentation for people is in [
|
||||
"""
|
||||
|
||||
|
||||
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007"], "03": ["0001", "0002", "0012"],
|
||||
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007", "0012"], "03": ["0001", "0002", "0012"],
|
||||
"05": ["0001", "0002", "0007"], "04": ["0001", "0002", "0003", "0009", "0012"],
|
||||
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012"]}
|
||||
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012", "0013"]}
|
||||
|
||||
USER_GUIDE = """# User guide: {display}
|
||||
|
||||
@@ -802,6 +818,13 @@ def main():
|
||||
missing = [a["n"] for a in AGENTS if not list(STORIES.glob(f"ALZ-ASM-{a['n']}-*"))] if STORIES.is_dir() else ["all"]
|
||||
if missing:
|
||||
sys.exit(f"no story files for ALZ-ASM-{', '.join(missing)} in {STORIES}: nothing was changed")
|
||||
# Every prompt is built before anything is deleted, so a rewrite that no longer matches its story stops the run here
|
||||
# (2026-11-06: one stopped after the critic's folder was deleted).
|
||||
try:
|
||||
for a in AGENTS:
|
||||
prompt_md(a)
|
||||
except AssertionError as e:
|
||||
sys.exit(f"a rewrite doesn't match its story {e}: nothing was changed")
|
||||
# An agent with hooks keeps its hand-written app/ and tests/ (ADR 0011): everything else is regenerated.
|
||||
import tempfile
|
||||
kept = Path(tempfile.mkdtemp(prefix="scaffold-kept-"))
|
||||
|
||||
Reference in New Issue
Block a user