ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# Eval cases for CP1 - Discovery Report Agent. No model or MCP tool is involved.
|
||||
schema_version: "1.0"
|
||||
agent: "cp1-discovery-report-agent"
|
||||
defaults:
|
||||
timeout: 60
|
||||
cases:
|
||||
- id: "fail-verdict-is-marked"
|
||||
description: "A failed run says so at the top of the report."
|
||||
input:
|
||||
json:
|
||||
engagement_id: "ENG-EVAL-001"
|
||||
run_id: "RUN-EVAL-01"
|
||||
verdict: "fail"
|
||||
checks:
|
||||
- schema: true
|
||||
- field: "verdict"
|
||||
equals: "fail"
|
||||
- field: "report_markdown"
|
||||
contains: "didn't pass the critic"
|
||||
- field: "report_html"
|
||||
contains: "didn't pass the critic"
|
||||
Reference in New Issue
Block a user