- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
22 lines
618 B
YAML
22 lines
618 B
YAML
# Eval cases for CP1 - Discovery Report Agent. No model or MCP tool is involved.
|
|
schema_version: "1.0"
|
|
agent: "cp1-discovery-report-agent"
|
|
defaults:
|
|
timeout: 60
|
|
cases:
|
|
- id: "fail-verdict-is-marked"
|
|
description: "A failed run says so at the top of the report."
|
|
input:
|
|
json:
|
|
engagement_id: "ENG-EVAL-001"
|
|
run_id: "RUN-EVAL-01"
|
|
verdict: "fail"
|
|
checks:
|
|
- schema: true
|
|
- field: "verdict"
|
|
equals: "fail"
|
|
- field: "report_markdown"
|
|
contains: "didn't pass the critic"
|
|
- field: "report_html"
|
|
contains: "didn't pass the critic"
|