- Six declarative agents generated from the v1.1 stories (scaffold_from_stories.py, ADR 0001),
and a 9-step workflow export (build_workflow.py).
- Code-only intake (ADR 0008) verifies the one Discovery package in code before any agent.
- The rate card is the run input rate_card_document; no upload step (ADR 0009).
- Critic 0.1.6 after live runs on crucible-kab-dev:
- K-12 model independence checked at build time against each agent's deployed model
(ADR 0010); the critic reports it from its two inputs;
- receives in_cloud_estate for K-10; an unverified K-10 never fails the run alone;
- K-08 allows the scoring note to list findings as advisory, as ALZ-ASM-04 must;
- carries ALZ-ASM-03's and ALZ-ASM-04's rule sections, copied by the scaffold, so it can
recalculate risk, confidence and TCO.
- The scaffold refuses to regenerate unless every story file is present.
- Case azure-crucible-kab-dev-assessment with an illustrative rate card.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
37 lines
2.2 KiB
Markdown
37 lines
2.2 KiB
Markdown
# 0010. Model independence (K-12) is checked when the workflow is built, not judged by the critic
|
|
|
|
| | |
|
|
|---|---|
|
|
| Status | Accepted |
|
|
| Date | 2026-11-05 |
|
|
| Decided by | Product owner, 2026-11-05: "go with the recommendation" (check K-12 at build time; the critic reports it) |
|
|
| Requirements changed | ALZ-ASM-06 K-12: the check runs in `scripts/build_workflow.py`; the critic reports its result. Refines [ADR 0005](0005.md) |
|
|
|
|
## Context
|
|
|
|
- K-12 asks whether the critic runs on a different model from the agents it checks (PR-07). The workflow passes the
|
|
critic both names as static inputs (ADR 0005), so the answer is fixed before any run starts.
|
|
- The critic judged it anyway, and failed it on Discovery's models in two runs out of three:
|
|
- `exec_4be193061c41`: a Discovery critic note that Discovery's network agent runs on `gpt-5.4`.
|
|
- `exec_0b0afb70b8b9`: a Discovery critic note that Discovery's critic was validated on `gpt-5.4`. Every other
|
|
check passed; the prompt (critic 0.1.2) already said that models named in Discovery's documents don't count.
|
|
- Discovery is a separate run with its own critic. Its models have no bearing on this run's independence.
|
|
|
|
## Decision
|
|
|
|
1. **`build_workflow.py` checks K-12** before writing the export. It reads each agent's deployed model
|
|
(`LITELLM_MODEL` in its `kubernetes/configmap.yaml`) and refuses to build unless:
|
|
- the critic is deployed on the static `critic_model`;
|
|
- every other step's agent is deployed on a model in `executor_models`;
|
|
- `critic_model` is not in `executor_models`.
|
|
2. **The critic reports K-12** as `pass` when `critic_model` is not in `executor_models`, and reads nothing else for
|
|
it. It has no code: the agent stays declarative (ADR 0001).
|
|
|
|
## Consequences
|
|
|
|
- **A workflow with a critic on an executor's model can't be built,** so it never reaches KAB.
|
|
- **The names the critic is given are checked against the deployments,** not just typed in. A model changed in a
|
|
configmap without a rebuild of the workflow isn't caught: rebuild and update the workflow after any model change.
|
|
- **The critic can still misreport K-12,** being a model. If it does, the report contradicts its own inputs, which
|
|
is visible in the check's details.
|