F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
Some checks failed
check agents / check (push) Failing after 14s
Some checks failed
check agents / check (push) Failing after 14s
- Critic 0.3.0: the hooks look up every disposition citation in the verified Discovery package (app/evidence.py). An unresolved citation fails K-02 and rejects the disposition; whether the records support the basis stays the model's judgement. - Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref is the primary record), ADR 0013 addendum. - Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering ALZ-ASM-02). - Scaffold: builds every prompt before deleting anything. - Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15). - Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -189,7 +189,7 @@ AGENTS = [
|
||||
output=obj({"discovery_execution_id": S, "profiles": arr(PROFILE),
|
||||
"gaps": arr(obj({"application_id": S, "reason": S})), "findings_note": S},
|
||||
["discovery_execution_id", "profiles", "gaps", "findings_note"], title="Readiness profiles"),
|
||||
rewrites=[]),
|
||||
rewrites=REPRESENTS_REWRITES), # run exec_d170329cba65: the profiler listed eight represents edges as dependents
|
||||
dict(n="03", key="cp1-migration-risk-scorer-agent", ado=3214461, display="CP1 - Migration Risk Scorer Agent",
|
||||
desc="Scores each planned application's migration risk from four weighted factors, names missing factors, "
|
||||
"and flags the high-risk tail by score or hard trigger.",
|
||||
@@ -295,6 +295,15 @@ AGENTS = [
|
||||
"`executor_models`. Report `pass` when `critic_model` is not in `executor_models`. Read nothing else for "
|
||||
"K-12: Discovery is a separate run, and the models of its agents or its critic never fail K-12 | "
|
||||
"`critic_model` is in `executor_models` | Run `fail` |"),
|
||||
# Run exec_659b13e5f947 judged the one required evidence_ref alone, though the 19 citations together supported
|
||||
# the basis: an application's disposition can't rest on one asset's record (ADR 0013, 2026-11-06 addendum).
|
||||
("| **K-02** Evidence | Every `evidence_ref` is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` "
|
||||
"and resolves to a supplied record that supports the rule-trace basis | Missing, other run, unresolvable or "
|
||||
"unsupportive | Reject disposition |",
|
||||
"| **K-02** Evidence | Every citation of a disposition (`evidence_ref`, its primary record, and each of "
|
||||
"`evidence_refs`) is `discovery:{plan.discovery_execution_id}/{agent_id}/{record_key}` and resolves to a "
|
||||
"supplied record; the citations taken together support the rule-trace basis | Missing, other run, "
|
||||
"unresolvable, or the citations together are unsupportive | Reject disposition |"),
|
||||
("- **Scope**: This Assessment run's outputs, the cited Discovery records, the rate card and the call log",
|
||||
"- **Scope**: This Assessment run's outputs, the cited Discovery records and the rate card"),
|
||||
]),
|
||||
@@ -359,6 +368,13 @@ were called (ADR 0011). Code sets those checks, the sign-off queue and the verdi
|
||||
spend your effort on the checks only you can make: coverage, evidence, dependents, advisory security, telemetry,
|
||||
tool calls and model independence. Where you read a code check differently, say why in its details.
|
||||
|
||||
`checked_in_code` also says whether every disposition's `evidence_ref` resolves to a record in the Discovery package
|
||||
(K-02's lookup, ADR 0013). A citation that resolves is a valid citation: for K-02, judge only whether the cited records
|
||||
support the rule-trace basis. Judge a disposition's citations together: `evidence_ref` is the primary record and
|
||||
`evidence_refs` the rest, and the basis is supported when the records cited across both support it. A disposition
|
||||
covers a whole application, so never fail it because its primary record alone covers only one asset. The rate card is cited by `rate_id` in the TCO line items (K-06), never as Discovery
|
||||
evidence, so a basis that rests on the rate card is not unsupported for lacking a `discovery:` citation.
|
||||
|
||||
If `preconditions.failed` is not empty, the planner refused to start: the later steps ran on empty inputs. Return
|
||||
`verdict: fail`, with a K-01 detail naming each failed precondition, and run no other check.
|
||||
"""
|
||||
@@ -701,9 +717,9 @@ cases (it needs `kyndryl-agent-runtime[eval]`). Documentation for people is in [
|
||||
"""
|
||||
|
||||
|
||||
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007"], "03": ["0001", "0002", "0012"],
|
||||
ADRS_FOR = {"01": ["0001", "0002", "0003", "0008", "0009"], "02": ["0001", "0002", "0007", "0012"], "03": ["0001", "0002", "0012"],
|
||||
"05": ["0001", "0002", "0007"], "04": ["0001", "0002", "0003", "0009", "0012"],
|
||||
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012"]}
|
||||
"06": ["0001", "0002", "0004", "0005", "0006", "0009", "0010", "0011", "0012", "0013"]}
|
||||
|
||||
USER_GUIDE = """# User guide: {display}
|
||||
|
||||
@@ -802,6 +818,13 @@ def main():
|
||||
missing = [a["n"] for a in AGENTS if not list(STORIES.glob(f"ALZ-ASM-{a['n']}-*"))] if STORIES.is_dir() else ["all"]
|
||||
if missing:
|
||||
sys.exit(f"no story files for ALZ-ASM-{', '.join(missing)} in {STORIES}: nothing was changed")
|
||||
# Every prompt is built before anything is deleted, so a rewrite that no longer matches its story stops the run here
|
||||
# (2026-11-06: one stopped after the critic's folder was deleted).
|
||||
try:
|
||||
for a in AGENTS:
|
||||
prompt_md(a)
|
||||
except AssertionError as e:
|
||||
sys.exit(f"a rewrite doesn't match its story {e}: nothing was changed")
|
||||
# An agent with hooks keeps its hand-written app/ and tests/ (ADR 0011): everything else is regenerated.
|
||||
import tempfile
|
||||
kept = Path(tempfile.mkdtemp(prefix="scaffold-kept-"))
|
||||
|
||||
Reference in New Issue
Block a user