F02: Assessment report and package (ADR 0014, 0015); register agents in KAB
Some checks failed
check agents / check (push) Failing after 14s

- Report (cp1-assessment-report-agent 0.2.0, step s7): a model writes the
  Assessment report from facts worked out in code (register, gaps, scorecard,
  rule traces, checks, and what each application is from the Discovery
  package); code sets the verdict and renders the pages. The
  disposition_signoff review point moves from the critic to the report.
- Package (cp1-assessment-package-agent 0.1.1, step s8): after sign-off, every
  step's output in one package, with a register, a gaps register, the
  sign-off record, lineage and a sha256 per part, in code. Page step s9.
- scripts/kab_register_agents.py: registers deployed agents in KAB's agent
  catalog (an in-place workflow update doesn't).
- Runs exec_1b1257380df7 (014) and exec_374d5d577077 (015) completed on
  crucible-kab-dev: critic pass, report figures all in its facts, package
  complete with every digest recomputing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-08 20:19:10 +00:00
parent 3b3ca64de0
commit 66c32745aa
58 changed files with 7106 additions and 26 deletions

View File

@@ -27,7 +27,7 @@ Facts for this Feature's individual-agents pattern, for coding agents. Repositor
- **Static inputs are applied after run inputs.** So the `a1` step leaves `artifact_ids` to the run (`from_workflow`) unless `--artifact-id` fixes them in the export.
- **Step outputs are registered as project artifacts only at the end of a run, and only when it has a `project_id`.** The artifact name is `{execution_id}_{step_id}`.
- **Review decisions go to the activity log** (`record_review_decision`), not onto artifacts. That's why the approval reference is a run input (ADR 0002).
- **Models served by the cluster's LiteLLM:** `gpt-5.6-sol`, `gpt-5.4`, `gpt-4o` (checked 2026-11-01).
- **Models served by the cluster's LiteLLM:** `gpt-5.6-sol`, `gpt-5.4`, `gpt-4.1`, `gpt-4o`, and the embedding model `text-embedding-3-small` (checked 2026-11-08 through KAB's `GET /api/v1/config/models`, which asks LiteLLM; `gpt-4.1` is new since 2026-11-01).
- **No upload step** (ADR 0009). KAB hands every run input to every step, and `parse_assessment_files` reads a `file` input before the `files` wired to it, so a rate card uploaded as `file` was parsed by `p1` in place of the Discovery package (run `exec_20d55cfe6fd7`). The rate card is the run input `rate_card_document`, its JSON as text.
- **Run `exec_4be193061c41` (2026-11-05):** intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds an `az account get-access-token` token (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares only `critic_model` with `executor_models` and gets `in_cloud_estate` for K-10.
- **Never pass the scaffold an option:** `scaffold_from_stories.py --help` once took `--help` as the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there.
@@ -40,3 +40,10 @@ Facts for this Feature's individual-agents pattern, for coding agents. Repositor
- **Run `exec_659b13e5f947` (profiler 0.1.1):** K-03 passed (no dependents). Failed only K-02: the model judged the one `evidence_ref` alone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum).
- **The scaffold builds every prompt before it deletes anything** (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
- **Run `exec_49e355141382` (critic 0.3.1, 2026-11-06): verdict pass**, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in run `exec_659b13e5f947` on the same package: the disposition agent applies the adjustments differently between runs.
- **The Assessment package (ADR 0014, 2026-11-08):** step `s8`, `cp1-assessment-package-agent`, code only, after `disposition_signoff`. It is hand-written like the intake; the scaffold doesn't touch it. `INPUTS` in `app/package.py` and `PACKAGE_INPUTS` in `scripts/build_workflow.py` must agree (a test checks). Tested on run `exec_49e355141382`'s step outputs.
- **The Assessment report (ADR 0015, 2026-11-08):** step `s7`, `cp1-assessment-report-agent`, carries the review point. The model writes from facts made in code; `app/package.py` there is a copy of the package agent's (a test fails if they differ). `scripts/package_schemas.py` writes both agents' input schemas from `INPUTS`.
- **A new agent must be in KAB's agent catalog** before a workflow that uses it validates ("Agent '...' not found in this environment"); `--update` doesn't register agents. The API is `POST /api/v1/agent-catalog/discover-and-register` with `{"url": ...}` (Editor), which reads the agent card. A code-only agent needs no MCP key Secret.
- **Run `exec_1b1257380df7` (RUN-ASM-AZURE-014, 2026-11-08), first on the 12-step workflow:** critic pass; the report (model, about 9k tokens) quoted only figures in its facts (19 of 19 checked); the Engagement Lead continued at the report; the package (step 11 in KAB's numbering) was complete, every part's sha256 recomputed, APP-001 `signed_off`. Run `exec_52f93ebb2dee` (013, old workflow) was stopped.
- **Claude models in Foundry (2026-11-08):** `claude-sonnet-5-5` and `claude-haiku-5-5` are deployed on `crucible-kab-eng-dev-ai` (the resource behind the gateway's models) but not mapped in LiteLLM, so agents can't call them. Mapping one is a LiteLLM admin step (KAB's `AKS/LLM_configuration_Foundry_LiteLLM.md`); the product owner chose not to do it for now.
- **Report 0.2.0 (ADR 0015 addendum):** the report also reads `p1.parsed_content` and has a `## The application` section (assets, owner, how Discovery identified it, data stores, dependencies), extracted in code by `app/discovery.py`.
- **Run `exec_374d5d577077` (RUN-ASM-AZURE-015, report 0.2.0, 2026-11-08):** critic pass; the report's new `## The application` section lists all ten assets with their types, CMDB class and support group, the grouping basis, and the seven dependencies; 17 of 17 figures and every id in the report are in its facts. Report agent: about 13k tokens.