Some checks failed
check agents / check (push) Failing after 14s
- Report (cp1-assessment-report-agent 0.2.0, step s7): a model writes the Assessment report from facts worked out in code (register, gaps, scorecard, rule traces, checks, and what each application is from the Discovery package); code sets the verdict and renders the pages. The disposition_signoff review point moves from the critic to the report. - Package (cp1-assessment-package-agent 0.1.1, step s8): after sign-off, every step's output in one package, with a register, a gaps register, the sign-off record, lineage and a sha256 per part, in code. Page step s9. - scripts/kab_register_agents.py: registers deployed agents in KAB's agent catalog (an in-place workflow update doesn't). - Runs exec_1b1257380df7 (014) and exec_374d5d577077 (015) completed on crucible-kab-dev: critic pass, report figures all in its facts, package complete with every digest recomputing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
26 lines
1.5 KiB
YAML
26 lines
1.5 KiB
YAML
# Eval cases for CP1 - Assessment Report Agent.
|
|
schema_version: '1.0'
|
|
agent: cp1-assessment-report-agent
|
|
defaults:
|
|
timeout: 600
|
|
cases:
|
|
- id: a-passing-run-gets-a-report-with-its-pages
|
|
description: 'One application, critic pass: a report in Markdown, the verdict from the facts, and both pages rendered in code.'
|
|
input:
|
|
json:
|
|
engagement_id: ENG-EVAL-001
|
|
run_id: RUN-EVAL-01
|
|
discovery_execution_id: exec_eval00000001
|
|
intake_verified: true
|
|
applications: [{"application_id": "APP-001", "name": "Eval app", "assets": ["discovery:exec_eval00000001/ALZ-DSC-03/CI-001"]}]
|
|
scores: [{"application_id": "APP-001", "score": 0.1, "partial": false, "missing_factors": [], "high_risk": false}]
|
|
posture_status: not_applicable
|
|
security_findings: []
|
|
dispositions: [{"application_id": "APP-001", "disposition": "rehost", "confidence": 0.7, "signoff_required": true, "evidence_ref": "discovery:exec_eval00000001/ALZ-DSC-03/CI-001", "rule_trace": [{"category": "rehost", "matched": true, "basis": "No earlier category matched."}], "confidence_workings": {"base": 0.9, "adjustments": [{"rule": "P1 telemetry_unavailable", "delta": -0.1, "basis": "No telemetry."}, {"rule": "P3 partial_risk_score", "delta": -0.1, "basis": "Partial."}], "cap": null, "result": 0.7}, "tco_model": {"currency": "USD", "current_annual": null, "target_annual": 100, "indicative": true}}]
|
|
verdict: pass
|
|
signoff_queue: [APP-001]
|
|
checks:
|
|
- schema: true
|
|
- field: verdict
|
|
equals: pass
|