Files
cp1-agents/individual-agents/ALZ-F01: Discovery Workflow Agent/docs/adr/0020.md
Jonathan Boniface 9e503885cc ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part
  hashed (sha256 over canonical JSON), with a manifest and a value scorecard.
- Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021):
  CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them.
- Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum
  narrower than its producer's.
- Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic
  agents; the report and Well-Architected review agents; ADRs 0012-0021.
- Workflow cases with value bars and run limits; run evaluation and usage collection scripts;
  the run context template and the field-by-field review for ADR 0016.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

76 lines
5.9 KiB
Markdown

# 0020. One Discovery package for the Assessment workflow, with a manifest and a value scorecard, built in code
| | |
|---|---|
| Status | Accepted |
| Date | 2026-11-05 |
| Decided by | Product owner, 2026-11-05: "lets go ahead with the enhancements when these are not combined and then later that can be decided"; the solution architecture agents are a separate stream, and Discovery's outputs are used only by the Assessment workflow. Plan: [../discovery-value-plan.md](../discovery-value-plan.md), phases 0 and 1 |
| Requirements changed | ALZ-F02 ASM-01 (Discovery hand-off): F02 selects one package artifact rather than one per Discovery step (F02 ADR 0002). New step `s13`, agent `cp1-discovery-package-agent` |
## Context
- **F02 reads Discovery by hand.** Its Artifacts step needs each Discovery step's artifact id typed in, and every F02
agent receives all of them as one text, joined by `---` (engineering hub review
`reports/cp1/discovery-to-assessment-requirements-review-2026-11-04.md`, section 1).
- **Nothing measured whether a run was useful.** A case either passed its checks or it didn't. Nothing showed how
much of what F02 needs a run gives, how many decisions it leaves for a person, or what it cost.
- **KAB can't write a .zip.** It can expand a selected zip, but no step type creates one. Every step's output is kept
as a JSON artifact, `<execution_id>_<step_id>`.
- **Discovery and Assessment stay separate workflows** for now; combining them is a later decision.
- **The idea is borrowed, not the code.** The original `gcp_solution_architecture_agent` writes a deliverable folder
with an `execution.json` and measures its runs with limits per case. Those ideas are used here; nothing is ported,
and no hand-over to the solution architecture stream is built.
## Decision
1. **A package step, `s13`, answered in code** (`cp1-discovery-package-agent`, `app/package.py`, no model, no tools):
- It runs after the report's review point, beside the two page steps, so a package exists only for a run the
Engagement Lead let through.
- Its output holds every step's output fields, **copied unchanged**, as named parts: `plan`, `scan`, `inventory`,
`cmdb`, `classification`, `network`, `dependencies`, `critic`, `well_architected`, `report`. Parts are named by
meaning, because KAB renames step ids on import. The report part keeps `counts` and `report_markdown`, not the
HTML.
- Its `manifest` holds the engagement, run, execution id, provider, time, critic verdict, each agent's version
(a static input set when the workflow is built), the planner's source gaps, a `sha256` and item count per part,
the parts absent, `complete`, `problems` and the scorecard.
- `complete` is true only when the required parts (plan, inventory, classification, network, dependencies,
critic, report) are present and the critic passed. The CMDB and Well-Architected parts are optional.
- Each `sha256` is over the part's canonical JSON (`sort_keys`, `separators=(",", ":")`, UTF-8), so a consumer
can recompute it before trusting the part.
2. **A value scorecard** (`app/scorecard.py`), in the manifest and in `scripts/evaluate_workflow_run.py`, from the
same code:
- `f02_fields`: each F02 field (application, owner, criticality, environment, support status, end of support,
compliance scope, annual cost, data class), per record, as `filled`, `declared_gap`, `missing` or
`not_applicable`;
- `f02_run`: the run-level facts F02's ASM-01 checks;
- `decisions`: what the run leaves for a person, by kind;
- `findings`: how many findings name an owner and an action.
`filled` and `filled_or_declared` are both reported: a declared gap is honest, but it isn't value.
3. **Cases carry a value bar and limits.** `value:` (for example `f02_filled_min`) holds the baseline so a change
can't lower it; `limits:` bound tokens, model calls and agent seconds, read from the agents' logs by
`scripts/collect_run_usage.py`. Both fail as major.
4. **F02 is unchanged by this ADR.** Its Artifacts step can select the package today. Teaching F02 to verify the
manifest in code is F02's own change, with its own ADR.
## Consequences
- **First live runs** (2026-11-05, both after the Engagement Lead continued): `exec_14956d524a8b`
(`azure-crucible-kab-dev-cmdb`, 46 of 46 checks) and `exec_d1c242609f82` (`azure-crucible-kab-dev`, 19 of 19).
Both packages are complete, and every part's `sha256` recomputes.
- **Baseline:** 24% of F02's fields filled in both cases; 27% filled or declared with a CMDB, 51% without one (the
CMDB gap declares owners and applications); 6 of 7 run checks met (no applications list); 9 decisions with a CMDB;
findings 0% with an owner, 100% with an action. Owners are in the CMDB export but never reach the inventory
records, and nothing reads cost, lifecycle or compliance. These are phases 3 and 4 of the plan.
- **Size:** about 100 KB with a CMDB and 78 KB without, against KAB's 5 MB inline limit. A large estate will need
per-application summaries (plan phase 3) or a platform zip step.
- **The review record is implied, not explicit.** The package proves the run continued past review, but not who
approved it or their comments. That still needs the platform to expose review records to later steps.
- **Every Discovery output field now has a reader:** the context report's "unread" fields reach F02 through the
package, except the run's own ids.
- **The package input schema has no enums**, so the build-time contract check can never refuse a producer's value
at this step. A new field needs a row in `INPUTS`, its wiring in `build_workflow.py`, and a test checks the two
match.
- **ADR 0016 (Proposed) needs the owner's review:** it carries one run context file through to solution
architecture, which the decision of 2026-11-05 to keep the streams independent rules out. Plan phase 2 borrows
only the scope spec's field meanings, within F01.