- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
76 lines
5.9 KiB
Markdown
76 lines
5.9 KiB
Markdown
# 0020. One Discovery package for the Assessment workflow, with a manifest and a value scorecard, built in code
|
|
|
|
| | |
|
|
|---|---|
|
|
| Status | Accepted |
|
|
| Date | 2026-11-05 |
|
|
| Decided by | Product owner, 2026-11-05: "lets go ahead with the enhancements when these are not combined and then later that can be decided"; the solution architecture agents are a separate stream, and Discovery's outputs are used only by the Assessment workflow. Plan: [../discovery-value-plan.md](../discovery-value-plan.md), phases 0 and 1 |
|
|
| Requirements changed | ALZ-F02 ASM-01 (Discovery hand-off): F02 selects one package artifact rather than one per Discovery step (F02 ADR 0002). New step `s13`, agent `cp1-discovery-package-agent` |
|
|
|
|
## Context
|
|
|
|
- **F02 reads Discovery by hand.** Its Artifacts step needs each Discovery step's artifact id typed in, and every F02
|
|
agent receives all of them as one text, joined by `---` (engineering hub review
|
|
`reports/cp1/discovery-to-assessment-requirements-review-2026-11-04.md`, section 1).
|
|
- **Nothing measured whether a run was useful.** A case either passed its checks or it didn't. Nothing showed how
|
|
much of what F02 needs a run gives, how many decisions it leaves for a person, or what it cost.
|
|
- **KAB can't write a .zip.** It can expand a selected zip, but no step type creates one. Every step's output is kept
|
|
as a JSON artifact, `<execution_id>_<step_id>`.
|
|
- **Discovery and Assessment stay separate workflows** for now; combining them is a later decision.
|
|
- **The idea is borrowed, not the code.** The original `gcp_solution_architecture_agent` writes a deliverable folder
|
|
with an `execution.json` and measures its runs with limits per case. Those ideas are used here; nothing is ported,
|
|
and no hand-over to the solution architecture stream is built.
|
|
|
|
## Decision
|
|
|
|
1. **A package step, `s13`, answered in code** (`cp1-discovery-package-agent`, `app/package.py`, no model, no tools):
|
|
- It runs after the report's review point, beside the two page steps, so a package exists only for a run the
|
|
Engagement Lead let through.
|
|
- Its output holds every step's output fields, **copied unchanged**, as named parts: `plan`, `scan`, `inventory`,
|
|
`cmdb`, `classification`, `network`, `dependencies`, `critic`, `well_architected`, `report`. Parts are named by
|
|
meaning, because KAB renames step ids on import. The report part keeps `counts` and `report_markdown`, not the
|
|
HTML.
|
|
- Its `manifest` holds the engagement, run, execution id, provider, time, critic verdict, each agent's version
|
|
(a static input set when the workflow is built), the planner's source gaps, a `sha256` and item count per part,
|
|
the parts absent, `complete`, `problems` and the scorecard.
|
|
- `complete` is true only when the required parts (plan, inventory, classification, network, dependencies,
|
|
critic, report) are present and the critic passed. The CMDB and Well-Architected parts are optional.
|
|
- Each `sha256` is over the part's canonical JSON (`sort_keys`, `separators=(",", ":")`, UTF-8), so a consumer
|
|
can recompute it before trusting the part.
|
|
2. **A value scorecard** (`app/scorecard.py`), in the manifest and in `scripts/evaluate_workflow_run.py`, from the
|
|
same code:
|
|
- `f02_fields`: each F02 field (application, owner, criticality, environment, support status, end of support,
|
|
compliance scope, annual cost, data class), per record, as `filled`, `declared_gap`, `missing` or
|
|
`not_applicable`;
|
|
- `f02_run`: the run-level facts F02's ASM-01 checks;
|
|
- `decisions`: what the run leaves for a person, by kind;
|
|
- `findings`: how many findings name an owner and an action.
|
|
`filled` and `filled_or_declared` are both reported: a declared gap is honest, but it isn't value.
|
|
3. **Cases carry a value bar and limits.** `value:` (for example `f02_filled_min`) holds the baseline so a change
|
|
can't lower it; `limits:` bound tokens, model calls and agent seconds, read from the agents' logs by
|
|
`scripts/collect_run_usage.py`. Both fail as major.
|
|
4. **F02 is unchanged by this ADR.** Its Artifacts step can select the package today. Teaching F02 to verify the
|
|
manifest in code is F02's own change, with its own ADR.
|
|
|
|
## Consequences
|
|
|
|
- **First live runs** (2026-11-05, both after the Engagement Lead continued): `exec_14956d524a8b`
|
|
(`azure-crucible-kab-dev-cmdb`, 46 of 46 checks) and `exec_d1c242609f82` (`azure-crucible-kab-dev`, 19 of 19).
|
|
Both packages are complete, and every part's `sha256` recomputes.
|
|
- **Baseline:** 24% of F02's fields filled in both cases; 27% filled or declared with a CMDB, 51% without one (the
|
|
CMDB gap declares owners and applications); 6 of 7 run checks met (no applications list); 9 decisions with a CMDB;
|
|
findings 0% with an owner, 100% with an action. Owners are in the CMDB export but never reach the inventory
|
|
records, and nothing reads cost, lifecycle or compliance. These are phases 3 and 4 of the plan.
|
|
- **Size:** about 100 KB with a CMDB and 78 KB without, against KAB's 5 MB inline limit. A large estate will need
|
|
per-application summaries (plan phase 3) or a platform zip step.
|
|
- **The review record is implied, not explicit.** The package proves the run continued past review, but not who
|
|
approved it or their comments. That still needs the platform to expose review records to later steps.
|
|
- **Every Discovery output field now has a reader:** the context report's "unread" fields reach F02 through the
|
|
package, except the run's own ids.
|
|
- **The package input schema has no enums**, so the build-time contract check can never refuse a producer's value
|
|
at this step. A new field needs a row in `INPUTS`, its wiring in `build_workflow.py`, and a test checks the two
|
|
match.
|
|
- **ADR 0016 (Proposed) needs the owner's review:** it carries one run context file through to solution
|
|
architecture, which the decision of 2026-11-05 to keep the streams independent rules out. Plan phase 2 borrows
|
|
only the scope spec's field meanings, within F01.
|