Some checks failed
check agents / check (push) Failing after 14s
- Critic 0.3.0: the hooks look up every disposition citation in the verified Discovery package (app/evidence.py). An unresolved citation fails K-02 and rejects the disposition; whether the records support the basis stays the model's judgement. - Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref is the primary record), ADR 0013 addendum. - Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering ALZ-ASM-02). - Scaffold: builds every prompt before deleting anything. - Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15). - Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.6 KiB
3.6 KiB
0013. K-02's lookup runs in code; whether the evidence supports the basis stays the model's
| Status | Accepted |
| Date | 2026-11-06 |
| Decided by | Product owner, 2026-11-06: "go ahead with the split" |
| Requirements changed | ALZ-ASM-06 K-02: the citation lookup is in code, the support judgement in the model. Extends ADR 0011 |
Context
- K-02 passes when every
evidence_ref"isdiscovery:{discovery_execution_id}/<agent_id>/<record_key>and resolves to a supplied record that supports the rule-trace basis". The first half is a lookup; the second is a judgement. - In run
exec_1f518b04d6e4, every other check passed, and the critic model failed K-02 on two grounds:ALZ-DSC-05/CI-002andALZ-DSC-05/CI-010"do not resolve". They do: the package's classification records carry thosecanonical_ids, and the prompt says classification records are cited bycanonical_id;- the rate-card mappings have no Discovery citation. The rate card isn't Discovery evidence: its items are cited by
rate_idin the TCO line items, which K-06 checks in code.
Decision
- The critic's hooks look up every citation of every disposition (
evidence_refandevidence_refs) in the Discovery package the intake verified (app/evidence.py). A citation resolves when:- its execution id is the run's
discovery_execution_id; - its agent is one Assessment cites: ALZ-DSC-03 (inventory), 04 (CMDB and ITSM), 05 (classification), 06 (dependencies) or 07 (network);
- that agent's part of the package has a record with that key in an id field (
canonical_id,drift_id,ci_ref,edge_id, a nodeid, or a finding, rule, range or flow id), or the key is the part's own field name (a list cited as a whole, for example to show it's empty).
- its execution id is the run's
- A citation that doesn't resolve fails K-02 and rejects the disposition, in code.
- When every citation resolves, the model's K-02 stands as it wrote it: it judges whether the cited records
support the rule-trace basis. The prompt says a resolved citation is valid, and that the rate card is cited by
rate_id, never as Discovery evidence.
Consequences
- A wrong lookup by the model no longer fails a run; a citation that really is wrong always does.
- The model can still fail K-02 on support, which is the judgement K-02 exists for.
- The package is read twice, by the intake (to verify it) and the critic (to look up citations). The finding logic is the intake's, copied, not shared, since the two are separate images.
Addendum, 2026-11-06: the citations are judged together
Decided by the product owner, 2026-11-06 ("Lets go with A recomenadation"), after run exec_659b13e5f947.
- What happened: every citation resolved, and the critic model said the broader
evidence_refshelped, but it failed K-02 because the one requiredevidence_ref(ALZ-DSC-03/CI-001) didn't support the basis on its own. K-02's row says "Everyevidence_ref… supports the rule-trace basis", and the model read it as the single field. - Decision: K-02 judges support on a disposition's citations taken together.
evidence_refis the primary record andevidence_refsthe rest. A disposition covers a whole application, so it is never failed because its primary record alone covers only one asset. K-02's row in ALZ-ASM-06 is rewritten to say so (critic 0.3.1). - Not chosen: keeping the single-record test and making the disposition's
evidence_refthe package's application record. That would need the package'sapplicationspart in the lookup and a change to ALZ-ASM-04. - Requirements changed: ALZ-ASM-06's K-02 row.