Files
cp1-agents/individual-agents/ALZ-F02: Assessment & Disposition Workflow Agent/docs/assessment-outputs-proposal.md
Jonathan Boniface 66c32745aa
Some checks failed
check agents / check (push) Failing after 14s
F02: Assessment report and package (ADR 0014, 0015); register agents in KAB
- Report (cp1-assessment-report-agent 0.2.0, step s7): a model writes the
  Assessment report from facts worked out in code (register, gaps, scorecard,
  rule traces, checks, and what each application is from the Discovery
  package); code sets the verdict and renders the pages. The
  disposition_signoff review point moves from the critic to the report.
- Package (cp1-assessment-package-agent 0.1.1, step s8): after sign-off, every
  step's output in one package, with a register, a gaps register, the
  sign-off record, lineage and a sha256 per part, in code. Page step s9.
- scripts/kab_register_agents.py: registers deployed agents in KAB's agent
  catalog (an in-place workflow update doesn't).
- Runs exec_1b1257380df7 (014) and exec_374d5d577077 (015) completed on
  crucible-kab-dev: critic pass, report figures all in its facts, package
  complete with every digest recomputing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 20:19:10 +00:00

7.8 KiB
Raw Blame History

Assessment outputs: proposal

Status Decisions 1, 2, 4 and 6 accepted 2026-11-08 (ADR 0014). Decision 3 decided 2026-11-08: a model writes the report from the data, before sign-off (ADR 0015). Decision 5: recorded in the ADR, not on the board
Asked by Product owner, 2026-11-06: "propose outputs for the assessment workflow as currently there are not any specific"
Becomes An ADR (0014) once decided; Discovery's equivalent is F01 ADR 0020 (package) and ADR 0012 (report page)

Why

The Feature promises an "Assessment deliverable" besides the per-application records, but no step makes one:

  • The workflow ends at the critic's review point (s6, disposition_signoff). What it leaves is seven separate step artifacts (<exec>_s0 … _s6), each in its own shape, plus a review decision that lives only in KAB's activity log.
  • A person has nothing to read: the Engagement Lead signs off from the critic's raw JSON.
  • The next Features have nothing to select. Strategy (F03) needs "signed-off dispositions and risk scores"; Business Case (F04) needs the TCO models, dispositions and Discovery's cost facts; Target Architecture (F05) and Wave Planning (F06) need signed-off dispositions. Each would have to find and join seven artifacts by hand, which is the problem Discovery's package solved for F02 (F01 ADR 0020).
  • What Assessment couldn't do is never stated in one place. In run exec_49e355141382 APP-001's risk is a partial score (end of life, data and change factors missing) and its confidence 0.50. Nobody is told which Discovery data would raise it.

Proposal: three outputs, after the sign-off review point

All three run only after the Engagement Lead continues the run at disposition_signoff, so they exist only for assessments a person let through (as in Discovery). The critic's review stops being the final step.

s0 intake → s1 planner → s2 profiler ┐
                         s3 risk     ├→ s5 disposition → s6 critic ⏸ disposition_signoff
                         s4 security ┘                              ├→ s7 report (model + code) → s8 page
                                                                    └→ s9 package (code only)

Output 1: the Assessment package (machine hand-off, code only)

New agent cp1-assessment-package-agent, no model and no tools, like cp1-discovery-package-agent. Its output is one artifact that F03 to F06 select, with every step's fields copied unchanged into named parts:

Part From Holds
manifest all engagement, run, execution id; lineage: Discovery execution id, the Discovery package's sha256 (as the intake verified it), rate_card_id and its sha256; each agent's version; critic_model and executor_models; critic verdict; a sha256 and item count per part; complete; problems
register s1–s6, joined in code one row per application, the view the next Features read: disposition, confidence, signoff_required, sign-off status, risk score, partial and missing_factors, high-risk tail, current and target annual cost, posture summary (advisory), evidence_ref, and whether the critic rejected it
plan s1 preconditions, applications, application source, in_cloud_estate, coverage gaps
profiles s2 readiness profiles and gaps
risk s3 scores with factors, high-risk tail
security s4 status, posture score, controls, findings, pillars (always advisory)
dispositions s5 dispositions with rule trace, confidence workings, TCO model, licensing and scoring notes
critic s6 checks, rejected dispositions, sign-off queue, unverified claims, summary
gaps s1–s5, in code a data-improvement register: per application, each missing input (risk factor, TCO input, readiness signal, telemetry) with the Discovery source or field that would fill it, and what it costs today (for example "−0.10 confidence: P1"). Discovery's source gaps are carried through
scorecard in code dispositions per 7R category; share above and below 0.80; share of partial risk scores; rejected count; advisory findings by severity; TCO coverage (applications with both current and target cost)

complete is true only when every required part is present, the critic's verdict is pass and the run was continued at the review point. Each part's sha256 is computed the same way as Discovery's (canonical JSON), so F03 can verify it the way F02's intake verifies Discovery's package (ADR 0008).

Output 2: the Assessment report (for people)

Correction, 2026-11-08: cp1-discovery-report-agent calls no model: its app/report.py renders the whole report in code. The precedent given below for a model writing prose was wrong, so decision 3 is open again.

New agent cp1-assessment-report-agent (model plus hooks, like cp1-discovery-report-agent), then a format_for_ui page step. Every figure is filled in by code from the step outputs (ADR 0011); the model writes only the prose around them. Sections:

  1. Summary: applications assessed, the verdict, how many need sign-off and why.
  2. Dispositions: one table row per application (7R, confidence, rule that matched, sign-off), and the 7R mix.
  3. Sign-off queue: each low-confidence or interim-rule item with the deductions that lowered it.
  4. Risk: scores, the high-risk tail, and which scores are partial.
  5. Cost: current against target annual cost per application, with indicative and missing inputs named.
  6. Security posture (advisory): posture score and the top findings; says plainly they changed no disposition.
  7. What would raise confidence: the gaps register in words, ordered by effect.
  8. Assurance: the critic's checks, what is checked in code, what is unverified (K-10) and why.
  9. Lineage: the Discovery run, the rate card, agent versions and models.

Output 3: the sign-off record

KAB keeps a review decision in its activity log, not on an artifact (see .agents/CONTEXT.md), so "signed-off dispositions" can't be read from the run today. Proposed: continuing the run at disposition_signoff signs off the whole queue as it stands. The package records each application as signed_off (it was in the queue) or not_required. The next workflow records the approval as a run input, assessment_approval_ref, the same way F02 takes discovery_approval_ref (ADR 0002). Overriding single dispositions (accepting one, changing another) is left for later.

Decisions for the product owner

  1. The package (Output 1): build it as proposed? Recommended: it's what F03 to F06 need, and it repeats a pattern that already works for Discovery.
  2. The register and gaps parts: include them? Recommended. They're the only parts that aren't copies; both are worked out in code, so the figures can't drift.
  3. The report (Output 2): a model writing prose around code-filled figures, or code only (a fixed template, no model, cheaper and identical every run, but less readable)? Recommended: model plus code, as Discovery does.
  4. Sign-off (Output 3): whole-queue sign-off on continue now, per-application overrides later? Recommended. The alternative is a sign-off input file (one decision per application) on a rerun, which needs a design of its own.
  5. The board: the package and report agents are new agents. Add them as User Stories under ALZ-F02 (for example ALZ-ASM-07 and ALZ-ASM-08), or record them in this repository only, in an ADR?
  6. Order: package first (unblocks F03), then report, then sign-off? Recommended.

Not in this proposal

  • How F03 reads the package (F03 has no agents yet).
  • Telemetry, end-of-life or cost sources that would fill the gaps (Discovery value plan, phases 3 to 7).
  • Changes to the six existing agents' outputs: everything above is assembled from what they already produce.