F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
Some checks failed
check agents / check (push) Failing after 14s

- Critic 0.3.0: the hooks look up every disposition citation in the verified
  Discovery package (app/evidence.py). An unresolved citation fails K-02 and
  rejects the disposition; whether the records support the basis stays the
  model's judgement.
- Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref
  is the primary record), ADR 0013 addendum.
- Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering
  ALZ-ASM-02).
- Scaffold: builds every prompt before deleting anything.
- Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15).
- Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 15:59:01 +00:00
parent 041e199248
commit 3b3ca64de0
19 changed files with 613 additions and 14 deletions

View File

@@ -0,0 +1,55 @@
# 0013. K-02's lookup runs in code; whether the evidence supports the basis stays the model's
| | |
|---|---|
| Status | Accepted |
| Date | 2026-11-06 |
| Decided by | Product owner, 2026-11-06: "go ahead with the split" |
| Requirements changed | ALZ-ASM-06 K-02: the citation lookup is in code, the support judgement in the model. Extends [ADR 0011](0011.md) |
## Context
- K-02 passes when every `evidence_ref` "is `discovery:{discovery_execution_id}/<agent_id>/<record_key>` and resolves to
a supplied record that supports the rule-trace basis". The first half is a lookup; the second is a judgement.
- In run `exec_1f518b04d6e4`, every other check passed, and the critic model failed K-02 on two grounds:
- `ALZ-DSC-05/CI-002` and `ALZ-DSC-05/CI-010` "do not resolve". They do: the package's classification records carry
those `canonical_id`s, and the prompt says classification records are cited by `canonical_id`;
- the rate-card mappings have no Discovery citation. The rate card isn't Discovery evidence: its items are cited by
`rate_id` in the TCO line items, which K-06 checks in code.
## Decision
1. **The critic's hooks look up every citation** of every disposition (`evidence_ref` and `evidence_refs`) in the
Discovery package the intake verified (`app/evidence.py`). A citation resolves when:
- its execution id is the run's `discovery_execution_id`;
- its agent is one Assessment cites: ALZ-DSC-03 (inventory), 04 (CMDB and ITSM), 05 (classification),
06 (dependencies) or 07 (network);
- that agent's part of the package has a record with that key in an id field (`canonical_id`, `drift_id`, `ci_ref`,
`edge_id`, a node `id`, or a finding, rule, range or flow id), or the key is the part's own field name (a list
cited as a whole, for example to show it's empty).
2. **A citation that doesn't resolve fails K-02** and rejects the disposition, in code.
3. **When every citation resolves,** the model's K-02 stands as it wrote it: it judges whether the cited records
support the rule-trace basis. The prompt says a resolved citation is valid, and that the rate card is cited by
`rate_id`, never as Discovery evidence.
## Consequences
- **A wrong lookup by the model no longer fails a run;** a citation that really is wrong always does.
- **The model can still fail K-02 on support,** which is the judgement K-02 exists for.
- **The package is read twice,** by the intake (to verify it) and the critic (to look up citations). The finding
logic is the intake's, copied, not shared, since the two are separate images.
## Addendum, 2026-11-06: the citations are judged together
Decided by the product owner, 2026-11-06 ("Lets go with A recomenadation"), after run `exec_659b13e5f947`.
- **What happened:** every citation resolved, and the critic model said the broader `evidence_refs` helped, but it
failed K-02 because the one required `evidence_ref` (`ALZ-DSC-03/CI-001`) didn't support the basis on its own. K-02's
row says "Every `evidence_ref` … supports the rule-trace basis", and the model read it as the single field.
- **Decision:** K-02 judges support on a disposition's citations taken together. `evidence_ref` is the primary record
and `evidence_refs` the rest. A disposition covers a whole application, so it is never failed because its primary
record alone covers only one asset. K-02's row in ALZ-ASM-06 is rewritten to say so (critic 0.3.1).
- **Not chosen:** keeping the single-record test and making the disposition's `evidence_ref` the package's application
record. That would need the package's `applications` part in the lookup and a change to ALZ-ASM-04.
- **Requirements changed:** ALZ-ASM-06's K-02 row.