Files
cp1-agents/individual-agents/ALZ-F01: Discovery Workflow Agent/docs/discovery-value-plan.md
Jonathan Boniface 9e503885cc ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part
  hashed (sha256 over canonical JSON), with a manifest and a value scorecard.
- Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021):
  CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them.
- Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum
  narrower than its producer's.
- Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic
  agents; the report and Well-Architected review agents; ADRs 0012-0021.
- Workflow cases with value bars and run limits; run evaluation and usage collection scripts;
  the run context template and the field-by-field review for ADR 0016.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

7.4 KiB
Raw Permalink Blame History

Discovery output value: plan

Date 2026-11-05
Status Phases 0 and 1 live (ADR 0020); the rest is planned
Decided by Product owner, 2026-11-05: go ahead, with Discovery and Assessment kept as separate workflows; combining them is decided later

1. Purpose and boundaries

Make the live Discovery workflow's output useful to the people and workflow that consume it, using ideas from the original gcp_solution_architecture_agent.

  • Consumers: the Engagement Lead at the review point, and the Assessment workflow (ALZ-F02), assuming F02 is still required. No other workflow or pattern consumes Discovery's output.
  • Separate streams: the solution architecture agent (outcome-pattern/) and the sub-agent pattern are independent. No hand-over is built to them; only ideas are borrowed from the original agent, never its code or engine.
  • The owner's order holds: Azure working (done 2026-11-05), then the package step, then Discovery meets every F02 requirement.
  • Discovery and Assessment stay separate workflows. Whether to also offer one combined run is decided later.

2. What "meaningful value" means

Consumer The output must answer
Engagement Lead What's there, what's wrong, what's missing, and what must be decided
F02 Assessment For each application: its assets, dependencies, support status, data class, compliance, cost and owner, each with evidence

The value scorecard (cp1-discovery-package-agent/app/scorecard.py) measures this for every run, in code: F02 fields filled, declared or missing; F02's run-level checks; decisions left for a person; findings with an owner and an action. Cases also carry limits on tokens, model calls and agent seconds.

Baseline, case azure-crucible-kab-dev-cmdb, run exec_14956d524a8b, 2026-11-05 (46 of 46 checks):

Measure Value
F02 fields filled 24%
F02 fields filled or declared 27%
Missing everywhere application, owner, support status, end of support, compliance scope, annual cost
F02 run checks 6 of 7 (no applications list)
Decisions for a person 9
Findings with an owner / an action 0% / 100%
Cost about 106k tokens, 10 model calls, 108 agent seconds
Package size about 100 KB

3. Borrowed from the original

Idea In the original In Discovery
Field meanings: ask, decide, not_applicable, provenance, a catalogue, a check command ADR-0008, app/scope.py The run context (phase 2)
An explicit graph, and an id chain code won't break ADR-0005, the requirements chain Application nodes in the ADR 0019 graph (phase 3); evidence on every finding (phase 4)
A deliverable manifest; customer questions execution.json, customer-questions.md The package manifest (phase 1); decisions needed (phase 4)
A Baseline and its changes ADR-0006, changes.json Re-discovery against the last approved run (phase 5)
An evaluation harness solution_eval, judge calibration, limits Value bars and limits (phase 0); a judge (phase 6)
Lessons and overlays ADR-0004 Phase 7

Left behind: its own server, database, LangGraph and interrupt() (they conflict with KAB); one model for every role (Discovery's critic stays on a different model); Jev and unscrubbed free-text search (data would leave the gateways).

4. Phases

Sizes are relative. "F02-only" marks work that only F02 justifies.

# Phase Size What Done when Status
0 Value baseline S The scorecard; value: bars and limits: in both Azure cases; scripts/collect_run_usage.py Both cases have a baseline built (ADR 0020)
1 Discovery package M, F02-only Step s13, cp1-discovery-package-agent: every part unchanged, a manifest with sha256 per part, completeness and the scorecard An F02 run starts from one artifact id live (ADR 0020): both Azure cases pass with complete packages; F02's own change to verify the manifest is next
2 Run context field meanings M A catalogue for context.yaml (value, ask, decide, not_applicable; essential, important, optional) covering compliance scope, criticality, owner, business services; check_context.py; provenance per fact; assumptions shown apart A decide gives a visible assumption; an unset essential field stops the run before it scans todo. ADR 0016 (Proposed) needs the owner's review first: it carries the file to solution architecture
3 Close F02's data gaps L, mostly F02-only Applications as graph nodes (CMDB business service, then tags, then resource group or naming, then model-proposed and judged in code); CMDB owners onto records; end-of-life rules in code over provider_attributes; cost through fixed-shape queries on the existing discovery servers; per-application summaries for size; the repositories source The Azure case reaches the agreed F02 bar applications and owners built (ADR 0021): CMDB→tag→container→judged model, owners from the CMDB or an owner tag; expected 46% filled and 7 of 7 run checks. End-of-life, cost and per-application summaries still todo
4 A report that leads to decisions M First page: decisions needed, top risks, open questions. Each finding has a severity, an owner (else "unowned"), an action and its consumer; code refuses a finding without evidence or an action 100% of findings with an owner and an action todo
5 Re-discovery against a Baseline M A run may name an earlier approved execution; a code-only diff step gives added, removed and changed assets, edges and drift Identical runs give an empty diff; a planted change shows todo
6 Evaluation harness M A report judge on a different model, calibrated on 10–20 labelled sections; runs with and without a feature Each phase's effect is measured todo
7 Lessons and overlays M Repeated critic findings become approved rules; customer standards arrive as overlays — later

Phases 0, 2, 4, 5 and 6 are worth doing whatever happens to F02; 1 and 3 are justified by F02.

5. Decisions for the owner

  1. Is F02 still required? It gates the rest of phase 3.
  2. ADR 0016: the owner will go through its fields one by one before phase 2 starts: run-context-field-review.md.
  3. Application grouping decided 2026-11-05: CMDB, then tag, then resource group, then a judged model proposal, with model groups reviewed (ADR 0021).
  4. Cost and end-of-life sources: cost queries on the existing servers (needs Cost Management Reader or the equivalent) or an uploaded file; end-of-life from a table in code or the provider's lifecycle data.
  5. The value bar that counts as "Discovery meets F02".
  6. Later: a combined Discovery and Assessment run, beside the separate workflows.

6. Risks

Risk Mitigation
The package outgrows KAB's 5 MB inline limit Per-application summaries (phase 3), or a platform zip step
Wrong application groups mislead F02's scoring Checks in code, the critic, and review of model-proposed groups
Cost and lifecycle need permissions beyond Reader Update docs/customer-cloud-access.md; a missing permission is a declared gap
Borrowing turns into porting Each borrowing gets its own ADR; ideas only
Model cost grows with new steps The limits in each case; code-only steps wherever possible