Some checks failed
check agents / check (push) Failing after 14s
- Report (cp1-assessment-report-agent 0.2.0, step s7): a model writes the Assessment report from facts worked out in code (register, gaps, scorecard, rule traces, checks, and what each application is from the Discovery package); code sets the verdict and renders the pages. The disposition_signoff review point moves from the critic to the report. - Package (cp1-assessment-package-agent 0.1.1, step s8): after sign-off, every step's output in one package, with a register, a gaps register, the sign-off record, lineage and a sha256 per part, in code. Page step s9. - scripts/kab_register_agents.py: registers deployed agents in KAB's agent catalog (an in-place workflow update doesn't). - Runs exec_1b1257380df7 (014) and exec_374d5d577077 (015) completed on crucible-kab-dev: critic pass, report figures all in its facts, package complete with every digest recomputing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4.2 KiB
4.2 KiB
0015. A model writes the Assessment report from facts made in code, and the Engagement Lead signs off from it
| Status | Accepted |
| Date | 2026-11-08 |
| Decided by | Product owner, 2026-11-08: "in terms of the report the model should genereate it based on the data collected and should get before sign off" (proposal decision 3, reopened after the correction that Discovery's report calls no model) |
| Requirements changed | ALZ-ASM-06: disposition_signoff moves from the critic's step to the report's. New step s7, agent cp1-assessment-report-agent (no story); the package becomes s8 (ADR 0014); new page step s9 |
Context
- The Engagement Lead signed off from the critic's raw JSON: nothing readable showed the dispositions, why they were chosen, what lowered their confidence, or what signing off commits to.
- The proposal's precedent for a model writing the report was wrong: Discovery's report (
cp1-discovery-report-agent) is rendered entirely in code. The product owner chose a model anyway, writing from the data collected, before sign-off. - A model writing figures is the failure ADR 0011 removed from the critic (it summed 0.90 − 0.10 − 0.20 − 0.10 to 0.60).
Decision
- The report step,
s7, after the critic, carries thedisposition_signoffreview point. The critic's step no longer pauses. The review shows the report as a page marked as a draft for sign-off. - Code works out the facts; the model writes from them.
pre_process(app/facts.py) replaces the step outputs with facts: the register, gaps and scorecard from the package's own module (copied; a test keeps the copies identical), each disposition's rule trace and confidence workings, the critic's checks, the top ten security findings and the lineage. The model writes the report in Markdown, in the sectionsprompt.mdsets, and is told to take every figure from the facts and to calculate none. - Code sets what must not be the model's:
post_processsets the verdict, ids and sign-off queue from the facts, and renders the HTML (app/markdown.py, which escapes the model's text before adding markup):report_html, the draft shown at the review point, andfinal_html. - After the review point: the package (
s8) holds the report's Markdown as itsreportpart, and aformat_for_uistep (s9) makesfinal_htmlthe run's page, as Discovery does (F01 ADR 0012).
Consequences
- The Engagement Lead reads a report, not JSON, at the point where continuing signs off the queue.
- The model can still misquote a figure. The facts are correct, and the prompt forbids new figures, but nothing
checks the prose against them. If a run shows a misquote, the next step is a check in
post_processthat every number in the report appears in the facts. - Each run costs one more model call, about 13k characters in.
- The report is not checked by the critic: it is written after it, from the critic's own results.
Addendum, 2026-11-08: the report says what each application is
Asked for by the product owner after run exec_1b1257380df7: "it doesnt breakd down what the application is within
the report. This is necessary to understand what it has identified".
- Why it was missing: the Assessment steps' outputs name an application's assets by id only, and the report was written from them alone.
- Decision: the report step also receives
discovery_documents(the parse step's text,p1.parsed_content). Code (app/discovery.py) reads the Discovery package there and adds to the facts, per application: how Discovery identified it and why it asked for a review, its business service and owner, every asset (name, resource type, environment, location, criticality, services, size, state, CMDB class and support group, data class, support and cost facts), its data stores, and its dependencies withoutrepresentsedges (ADR 0012). The model writes a new section,## The application, from them, and never receives the package itself. - Consequences: the facts grow from about 13k to 22k characters for one application. The package (ADR 0014) is unchanged: it already names the Discovery package it rests on, which holds these records.