F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
Some checks failed
check agents / check (push) Failing after 14s

- Critic 0.3.0: the hooks look up every disposition citation in the verified
  Discovery package (app/evidence.py). An unresolved citation fails K-02 and
  rejects the disposition; whether the records support the basis stays the
  model's judgement.
- Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref
  is the primary record), ADR 0013 addendum.
- Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering
  ALZ-ASM-02).
- Scaffold: builds every prompt before deleting anything.
- Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15).
- Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 15:59:01 +00:00
parent 041e199248
commit 3b3ca64de0
19 changed files with 613 additions and 14 deletions

View File

@@ -189,7 +189,7 @@ AGENTS = [
output=obj({"discovery_execution_id": S, "profiles": arr(PROFILE),
"gaps": arr(obj({"application_id": S, "reason": S})), "findings_note": S},
["discovery_execution_id", "profiles", "gaps", "findings_note"], title="Readiness profiles"),
rewrites=[]),
rewrites=REPRESENTS_REWRITES), # run exec_d170329cba65: the profiler listed eight represents edges as dependents
dict(n="03", key="cp1-migration-risk-scorer-agent", ado=3214461, display="CP1 - Migration Risk Scorer Agent",
desc="Scores each planned application's migration risk from four weighted factors, names missing factors, "
"and flags the high-risk tail by score or hard trigger.",
@@ -295,6 +295,15 @@ AGENTS = [
"`executor_models`. Report `pass` when `critic_model` is not in `executor_models`. Read nothing else for "
"K-12: Discovery is a separate run, and the models of its agents or its critic never fail K-12 | "
"`critic_model` is in `executor_models` | Run `fail` |"),
# Run exec_659b13e5f947 judged the one required evidence_ref alone, though the 19 citations together supported
# the basis: an application's disposition can't rest on one asset's record (ADR 0013, 2026-11-06 addendum).
("| **K-02** Evidence | Every `evidence_ref` is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` "
"and resolves to a supplied record that supports the rule-trace basis | Missing, other run, unresolvable or "
"unsupportive | Reject disposition |",
"| **K-02** Evidence | Every citation of a disposition (`evidence_ref`, its primary record, and each of "
"`evidence_refs`) is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` and resolves to a "
"supplied record; the citations taken together support the rule-trace basis | Missing, other run, "
"unresolvable, or the citations together are unsupportive | Reject disposition |"),
("- **Scope**: This Assessment run's outputs, the cited Discovery records, the rate card and the call log",
"- **Scope**: This Assessment run's outputs, the cited Discovery records and the rate card"),
]),
@@ -359,6 +368,13 @@ were called (ADR 0011). Code sets those checks, the sign-off queue and the verdi
spend your effort on the checks only you can make: coverage, evidence, dependents, advisory security, telemetry,
tool calls and model independence. Where you read a code check differently, say why in its details.
`checked_in_code` also says whether every disposition's `evidence_ref` resolves to a record in the Discovery package
(K-02's lookup, ADR 0013). A citation that resolves is a valid citation: for K-02, judge only whether the cited records
support the rule-trace basis. Judge a disposition's citations together: `evidence_ref` is the primary record and
`evidence_refs` the rest, and the basis is supported when the records cited across both support it. A disposition
covers a whole application, so never fail it because its primary record alone covers only one asset. The rate card is cited by `rate_id` in the TCO line items (K-06), never as Discovery
evidence, so a basis that rests on the rate card is not unsupported for lacking a `discovery:` citation.
If `preconditions.failed` is not empty, the planner refused to start: the later steps ran on empty inputs. Return
`verdict: fail`, with a K-01 detail naming each failed precondition, and run no other check.
"""
@@ -701,9 +717,9 @@ cases (it needs `kyndryl-agent-runtime[eval]`). Documentation for people is in [
"""
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007"], "03": ["0001", "0002", "0012"],
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007", "0012"], "03": ["0001", "0002", "0012"],
"05": ["0001", "0002", "0007"], "04": ["0001", "0002", "0003", "0009", "0012"],
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012"]}
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012", "0013"]}
USER_GUIDE = """# User guide: {display}
@@ -802,6 +818,13 @@ def main():
missing = [a["n"] for a in AGENTS if not list(STORIES.glob(f"ALZ-ASM-{a['n']}-*"))] if STORIES.is_dir() else ["all"]
if missing:
sys.exit(f"no story files for ALZ-ASM-{', '.join(missing)} in {STORIES}: nothing was changed")
# Every prompt is built before anything is deleted, so a rewrite that no longer matches its story stops the run here
# (2026-11-06: one stopped after the critic's folder was deleted).
try:
for a in AGENTS:
prompt_md(a)
except AssertionError as e:
sys.exit(f"a rewrite doesn't match its story {e}: nothing was changed")
# An agent with hooks keeps its hand-written app/ and tests/ (ADR 0011): everything else is regenerated.
import tempfile
kept = Path(tempfile.mkdtemp(prefix="scaffold-kept-"))