- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7.4 KiB
Discovery output value: plan
| Date | 2026-11-05 |
| Status | Phases 0 and 1 live (ADR 0020); the rest is planned |
| Decided by | Product owner, 2026-11-05: go ahead, with Discovery and Assessment kept as separate workflows; combining them is decided later |
1. Purpose and boundaries
Make the live Discovery workflow's output useful to the people and workflow that consume it, using ideas from the
original gcp_solution_architecture_agent.
- Consumers: the Engagement Lead at the review point, and the Assessment workflow (ALZ-F02), assuming F02 is still required. No other workflow or pattern consumes Discovery's output.
- Separate streams: the solution architecture agent (
outcome-pattern/) and the sub-agent pattern are independent. No hand-over is built to them; only ideas are borrowed from the original agent, never its code or engine. - The owner's order holds: Azure working (done 2026-11-05), then the package step, then Discovery meets every F02 requirement.
- Discovery and Assessment stay separate workflows. Whether to also offer one combined run is decided later.
2. What "meaningful value" means
| Consumer | The output must answer |
|---|---|
| Engagement Lead | What's there, what's wrong, what's missing, and what must be decided |
| F02 Assessment | For each application: its assets, dependencies, support status, data class, compliance, cost and owner, each with evidence |
The value scorecard (cp1-discovery-package-agent/app/scorecard.py) measures this for every run, in code: F02
fields filled, declared or missing; F02's run-level checks; decisions left for a person; findings with an owner and
an action. Cases also carry limits on tokens, model calls and agent seconds.
Baseline, case azure-crucible-kab-dev-cmdb, run exec_14956d524a8b, 2026-11-05 (46 of 46 checks):
| Measure | Value |
|---|---|
| F02 fields filled | 24% |
| F02 fields filled or declared | 27% |
| Missing everywhere | application, owner, support status, end of support, compliance scope, annual cost |
| F02 run checks | 6 of 7 (no applications list) |
| Decisions for a person | 9 |
| Findings with an owner / an action | 0% / 100% |
| Cost | about 106k tokens, 10 model calls, 108 agent seconds |
| Package size | about 100 KB |
3. Borrowed from the original
| Idea | In the original | In Discovery |
|---|---|---|
Field meanings: ask, decide, not_applicable, provenance, a catalogue, a check command |
ADR-0008, app/scope.py |
The run context (phase 2) |
| An explicit graph, and an id chain code won't break | ADR-0005, the requirements chain | Application nodes in the ADR 0019 graph (phase 3); evidence on every finding (phase 4) |
| A deliverable manifest; customer questions | execution.json, customer-questions.md |
The package manifest (phase 1); decisions needed (phase 4) |
| A Baseline and its changes | ADR-0006, changes.json |
Re-discovery against the last approved run (phase 5) |
| An evaluation harness | solution_eval, judge calibration, limits |
Value bars and limits (phase 0); a judge (phase 6) |
| Lessons and overlays | ADR-0004 | Phase 7 |
Left behind: its own server, database, LangGraph and interrupt() (they conflict with KAB); one model for every
role (Discovery's critic stays on a different model); Jev and unscrubbed free-text search (data would leave the
gateways).
4. Phases
Sizes are relative. "F02-only" marks work that only F02 justifies.
| # | Phase | Size | What | Done when | Status |
|---|---|---|---|---|---|
| 0 | Value baseline | S | The scorecard; value: bars and limits: in both Azure cases; scripts/collect_run_usage.py |
Both cases have a baseline | built (ADR 0020) |
| 1 | Discovery package | M, F02-only | Step s13, cp1-discovery-package-agent: every part unchanged, a manifest with sha256 per part, completeness and the scorecard |
An F02 run starts from one artifact id | live (ADR 0020): both Azure cases pass with complete packages; F02's own change to verify the manifest is next |
| 2 | Run context field meanings | M | A catalogue for context.yaml (value, ask, decide, not_applicable; essential, important, optional) covering compliance scope, criticality, owner, business services; check_context.py; provenance per fact; assumptions shown apart |
A decide gives a visible assumption; an unset essential field stops the run before it scans |
todo. ADR 0016 (Proposed) needs the owner's review first: it carries the file to solution architecture |
| 3 | Close F02's data gaps | L, mostly F02-only | Applications as graph nodes (CMDB business service, then tags, then resource group or naming, then model-proposed and judged in code); CMDB owners onto records; end-of-life rules in code over provider_attributes; cost through fixed-shape queries on the existing discovery servers; per-application summaries for size; the repositories source |
The Azure case reaches the agreed F02 bar | applications and owners built (ADR 0021): CMDB→tag→container→judged model, owners from the CMDB or an owner tag; expected 46% filled and 7 of 7 run checks. End-of-life, cost and per-application summaries still todo |
| 4 | A report that leads to decisions | M | First page: decisions needed, top risks, open questions. Each finding has a severity, an owner (else "unowned"), an action and its consumer; code refuses a finding without evidence or an action | 100% of findings with an owner and an action | todo |
| 5 | Re-discovery against a Baseline | M | A run may name an earlier approved execution; a code-only diff step gives added, removed and changed assets, edges and drift | Identical runs give an empty diff; a planted change shows | todo |
| 6 | Evaluation harness | M | A report judge on a different model, calibrated on 10–20 labelled sections; runs with and without a feature | Each phase's effect is measured | todo |
| 7 | Lessons and overlays | M | Repeated critic findings become approved rules; customer standards arrive as overlays | — | later |
Phases 0, 2, 4, 5 and 6 are worth doing whatever happens to F02; 1 and 3 are justified by F02.
5. Decisions for the owner
- Is F02 still required? It gates the rest of phase 3.
- ADR 0016: the owner will go through its fields one by one before phase 2 starts: run-context-field-review.md.
Application groupingdecided 2026-11-05: CMDB, then tag, then resource group, then a judged model proposal, with model groups reviewed (ADR 0021).- Cost and end-of-life sources: cost queries on the existing servers (needs Cost Management Reader or the equivalent) or an uploaded file; end-of-life from a table in code or the provider's lifecycle data.
- The value bar that counts as "Discovery meets F02".
- Later: a combined Discovery and Assessment run, beside the separate workflows.
6. Risks
| Risk | Mitigation |
|---|---|
| The package outgrows KAB's 5 MB inline limit | Per-application summaries (phase 3), or a platform zip step |
| Wrong application groups mislead F02's scoring | Checks in code, the critic, and review of model-proposed groups |
| Cost and lifecycle need permissions beyond Reader | Update docs/customer-cloud-access.md; a missing permission is a declared gap |
| Borrowing turns into porting | Each borrowing gets its own ADR; ideas only |
| Model cost grows with new steps | The limits in each case; code-only steps wherever possible |