F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
Some checks failed
check agents / check (push) Failing after 14s

- Critic 0.3.0: the hooks look up every disposition citation in the verified
  Discovery package (app/evidence.py). An unresolved citation fails K-02 and
  rejects the disposition; whether the records support the basis stays the
  model's judgement.
- Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref
  is the primary record), ADR 0013 addendum.
- Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering
  ALZ-ASM-02).
- Scaffold: builds every prompt before deleting anything.
- Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15).
- Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 15:59:01 +00:00
parent 041e199248
commit 3b3ca64de0
19 changed files with 613 additions and 14 deletions

View File

@@ -5,7 +5,7 @@
| Status | Accepted |
| Date | 2026-11-05 |
| Decided by | Product owner, 2026-11-05: "lets solve these first" (the run's faults are fixed before it is repeated); the rule follows Discovery's F01 ADR 0019 |
| Requirements changed | ALZ-ASM-03 and ALZ-ASM-04: the Application Dependents rule leaves out `represents` edges. ALZ-ASM-04 and ALZ-ASM-06 use ALZ-ASM-03's rule |
| Requirements changed | ALZ-ASM-02, ALZ-ASM-03 and ALZ-ASM-04: the Application Dependents rule leaves out `represents` edges. ALZ-ASM-04 and ALZ-ASM-06 use ALZ-ASM-03's rule |
## Context
@@ -25,6 +25,9 @@
left out of both an application's dependents and its dependencies.
2. **ALZ-ASM-04 and ALZ-ASM-06 carry ALZ-ASM-03's rule,** copied into their prompts by the scaffold, so all three lift
dependents the same way.
3. **ALZ-ASM-02 leaves them out too** (added 2026-11-06). The profiler has its own copy of the rule, for the
profile's `dependents`, `depends_on` and `dependency_exposure`, and was missed at first: in run `exec_d170329cba65`
it listed APP-001's eight `represents` edges as downstream dependents, and the critic failed K-03.
## Consequences

View File

@@ -0,0 +1,55 @@
# 0013. K-02's lookup runs in code; whether the evidence supports the basis stays the model's
| | |
|---|---|
| Status | Accepted |
| Date | 2026-11-06 |
| Decided by | Product owner, 2026-11-06: "go ahead with the split" |
| Requirements changed | ALZ-ASM-06 K-02: the citation lookup is in code, the support judgement in the model. Extends [ADR 0011](0011.md) |
## Context
- K-02 passes when every `evidence_ref` "is `discovery:{discovery_execution_id}/<agent_id>/<record_key>` and resolves to
a supplied record that supports the rule-trace basis". The first half is a lookup; the second is a judgement.
- In run `exec_1f518b04d6e4`, every other check passed, and the critic model failed K-02 on two grounds:
- `ALZ-DSC-05/CI-002` and `ALZ-DSC-05/CI-010` "do not resolve". They do: the package's classification records carry
those `canonical_id`s, and the prompt says classification records are cited by `canonical_id`;
- the rate-card mappings have no Discovery citation. The rate card isn't Discovery evidence: its items are cited by
`rate_id` in the TCO line items, which K-06 checks in code.
## Decision
1. **The critic's hooks look up every citation** of every disposition (`evidence_ref` and `evidence_refs`) in the
Discovery package the intake verified (`app/evidence.py`). A citation resolves when:
- its execution id is the run's `discovery_execution_id`;
- its agent is one Assessment cites: ALZ-DSC-03 (inventory), 04 (CMDB and ITSM), 05 (classification),
06 (dependencies) or 07 (network);
- that agent's part of the package has a record with that key in an id field (`canonical_id`, `drift_id`, `ci_ref`,
`edge_id`, a node `id`, or a finding, rule, range or flow id), or the key is the part's own field name (a list
cited as a whole, for example to show it's empty).
2. **A citation that doesn't resolve fails K-02** and rejects the disposition, in code.
3. **When every citation resolves,** the model's K-02 stands as it wrote it: it judges whether the cited records
support the rule-trace basis. The prompt says a resolved citation is valid, and that the rate card is cited by
`rate_id`, never as Discovery evidence.
## Consequences
- **A wrong lookup by the model no longer fails a run;** a citation that really is wrong always does.
- **The model can still fail K-02 on support,** which is the judgement K-02 exists for.
- **The package is read twice,** by the intake (to verify it) and the critic (to look up citations). The finding
logic is the intake's, copied, not shared, since the two are separate images.
## Addendum, 2026-11-06: the citations are judged together
Decided by the product owner, 2026-11-06 ("Lets go with A recomenadation"), after run `exec_659b13e5f947`.
- **What happened:** every citation resolved, and the critic model said the broader `evidence_refs` helped, but it
failed K-02 because the one required `evidence_ref` (`ALZ-DSC-03/CI-001`) didn't support the basis on its own. K-02's
row says "Every `evidence_ref` … supports the rule-trace basis", and the model read it as the single field.
- **Decision:** K-02 judges support on a disposition's citations taken together. `evidence_ref` is the primary record
and `evidence_refs` the rest. A disposition covers a whole application, so it is never failed because its primary
record alone covers only one asset. K-02's row in ALZ-ASM-06 is rewritten to say so (critic 0.3.1).
- **Not chosen:** keeping the single-record test and making the disposition's `evidence_ref` the package's application
record. That would need the package's `applications` part in the lookup and a change to ALZ-ASM-04.
- **Requirements changed:** ALZ-ASM-06's K-02 row.

View File

@@ -25,5 +25,6 @@ Each agent's `docs/adr/README.md` lists the records that apply to its own story.
| [0010](0010.md) | Model independence (K-12) is checked when the workflow is built, not judged by the critic | Accepted | ASM-06 |
| [0011](0011.md) | The critic's arithmetic, routing and schema checks run in code; the model judges the rest | Accepted | ASM-06 |
| [0012](0012.md) | A `represents` edge is not a dependency | Accepted | ASM-03, 04, 06 |
| [0013](0013.md) | K-02's lookup runs in code; whether the evidence supports the basis stays the model's | Accepted | ASM-06 |
New records copy [template.md](template.md) and take the next number.