Files
cp1-agents/individual-agents/ALZ-F02: Assessment & Disposition Workflow Agent/docs/adr/0010.md
Jonathan Boniface e22d037ab0 ALZ-F02 Assessment: individual agents, intake, rate card input, critic fixes
- Six declarative agents generated from the v1.1 stories (scaffold_from_stories.py, ADR 0001),
  and a 9-step workflow export (build_workflow.py).
- Code-only intake (ADR 0008) verifies the one Discovery package in code before any agent.
- The rate card is the run input rate_card_document; no upload step (ADR 0009).
- Critic 0.1.6 after live runs on crucible-kab-dev:
  - K-12 model independence checked at build time against each agent's deployed model
    (ADR 0010); the critic reports it from its two inputs;
  - receives in_cloud_estate for K-10; an unverified K-10 never fails the run alone;
  - K-08 allows the scoring note to list findings as advisory, as ALZ-ASM-04 must;
  - carries ALZ-ASM-03's and ALZ-ASM-04's rule sections, copied by the scaffold, so it can
    recalculate risk, confidence and TCO.
- The scaffold refuses to regenerate unless every story file is present.
- Case azure-crucible-kab-dev-assessment with an illustrative rate card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

37 lines
2.2 KiB
Markdown

# 0010. Model independence (K-12) is checked when the workflow is built, not judged by the critic
| | |
|---|---|
| Status | Accepted |
| Date | 2026-11-05 |
| Decided by | Product owner, 2026-11-05: "go with the recommendation" (check K-12 at build time; the critic reports it) |
| Requirements changed | ALZ-ASM-06 K-12: the check runs in `scripts/build_workflow.py`; the critic reports its result. Refines [ADR 0005](0005.md) |
## Context
- K-12 asks whether the critic runs on a different model from the agents it checks (PR-07). The workflow passes the
critic both names as static inputs (ADR 0005), so the answer is fixed before any run starts.
- The critic judged it anyway, and failed it on Discovery's models in two runs out of three:
- `exec_4be193061c41`: a Discovery critic note that Discovery's network agent runs on `gpt-5.4`.
- `exec_0b0afb70b8b9`: a Discovery critic note that Discovery's critic was validated on `gpt-5.4`. Every other
check passed; the prompt (critic 0.1.2) already said that models named in Discovery's documents don't count.
- Discovery is a separate run with its own critic. Its models have no bearing on this run's independence.
## Decision
1. **`build_workflow.py` checks K-12** before writing the export. It reads each agent's deployed model
(`LITELLM_MODEL` in its `kubernetes/configmap.yaml`) and refuses to build unless:
- the critic is deployed on the static `critic_model`;
- every other step's agent is deployed on a model in `executor_models`;
- `critic_model` is not in `executor_models`.
2. **The critic reports K-12** as `pass` when `critic_model` is not in `executor_models`, and reads nothing else for
it. It has no code: the agent stays declarative (ADR 0001).
## Consequences
- **A workflow with a critic on an executor's model can't be built,** so it never reaches KAB.
- **The names the critic is given are checked against the deployments,** not just typed in. A model changed in a
configmap without a rebuild of the workflow isn't caught: rebuild and update the workflow after any model change.
- **The critic can still misreport K-12,** being a model. If it does, the report contradicts its own inputs, which
is visible in the check's details.