- Six declarative agents generated from the v1.1 stories (scaffold_from_stories.py, ADR 0001),
and a 9-step workflow export (build_workflow.py).
- Code-only intake (ADR 0008) verifies the one Discovery package in code before any agent.
- The rate card is the run input rate_card_document; no upload step (ADR 0009).
- Critic 0.1.6 after live runs on crucible-kab-dev:
- K-12 model independence checked at build time against each agent's deployed model
(ADR 0010); the critic reports it from its two inputs;
- receives in_cloud_estate for K-10; an unverified K-10 never fails the run alone;
- K-08 allows the scoring note to list findings as advisory, as ALZ-ASM-04 must;
- carries ALZ-ASM-03's and ALZ-ASM-04's rule sections, copied by the scaffold, so it can
recalculate risk, confidence and TCO.
- The scaffold refuses to regenerate unless every story file is present.
- Case azure-crucible-kab-dev-assessment with an illustrative rate card.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
22 lines
1.1 KiB
YAML
22 lines
1.1 KiB
YAML
# Eval cases for CP1 - Assessment Intake Agent.
|
|
schema_version: '1.0'
|
|
agent: cp1-assessment-intake-agent
|
|
defaults:
|
|
timeout: 300
|
|
cases:
|
|
- id: a-checked-package-is-verified
|
|
description: 'One complete package from the named Discovery run, every part matching its sha256: verified.'
|
|
input:
|
|
json:
|
|
engagement_id: ENG-EVAL-001
|
|
run_id: RUN-EVAL-01
|
|
discovery_execution_id: exec_eval00000001
|
|
discovery_documents: '{"engagement_id": "ENG-EVAL-001", "run_id": "RUN-D-01", "package_version": "1.0", "manifest": {"package_version": "1.0", "engagement_id": "ENG-EVAL-001", "run_id": "RUN-D-01",
|
|
"execution_id": "exec_eval00000001", "created_at": "2026-11-05T00:00:00Z", "critic_verdict": "pass", "parts": {"inventory": {"sha256": "sha256:3f3684e17a90f649f856c3c1e9dd7e7b497bcae30c373952781a8f1585a247b2"},
|
|
"critic": {"sha256": "sha256:f5d6f98b22d346a32b0be68e2561b17913ff64e9b15a7c1e12f41b7c697ae1ac"}}, "absent_parts": [], "complete": true, "problems": [], "scorecard": {}}, "inventory": {"records":
|
|
[{"canonical_id": "CI-001"}]}, "critic": {"verdict": "pass"}}'
|
|
checks:
|
|
- schema: true
|
|
- field: verified
|
|
equals: true
|