Some checks failed
check agents / check (push) Failing after 14s
- Report (cp1-assessment-report-agent 0.2.0, step s7): a model writes the Assessment report from facts worked out in code (register, gaps, scorecard, rule traces, checks, and what each application is from the Discovery package); code sets the verdict and renders the pages. The disposition_signoff review point moves from the critic to the report. - Package (cp1-assessment-package-agent 0.1.1, step s8): after sign-off, every step's output in one package, with a register, a gaps register, the sign-off record, lineage and a sha256 per part, in code. Page step s9. - scripts/kab_register_agents.py: registers deployed agents in KAB's agent catalog (an in-place workflow update doesn't). - Runs exec_1b1257380df7 (014) and exec_374d5d577077 (015) completed on crucible-kab-dev: critic pass, report figures all in its facts, package complete with every digest recomputing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9.1 KiB
9.1 KiB
Context: individual agents, ALZ-F02 Assessment & Disposition
Facts for this Feature's individual-agents pattern, for coding agents. Repository-wide facts are in ../../.agents/CONTEXT.md. Per-agent context is in <agent>/.agents/CONTEXT.md.
Pattern
- Six declarative agents, one per ALZ-ASM story, plus the code-only intake (
s0, ADR 0008). Only ALZ-ASM-02 and ALZ-ASM-05 declare MCP tools; the other four and the intake have none. - The workflow (
workflows/cp1-alz-assessment-individual.json) is the orchestration. Step order comes fromparallelGroupandexecutionLevel, and data fromfrom_steps. The step map isSTEPSinscripts/build_workflow.py, and the scaffold reads it for each agent's context file. - Order:
a1the Discovery package artifact,p1parse, thens0intake (checks the Discovery package in code; a refusal fails the run), thens1planner, thens2profiler,s3risk ands4security posture in parallel, thens5disposition, thens6critic.s6has a human approval, which isdisposition_signoff(ADR 0004). - Nothing persists between steps except their replies, as in F01 (its ADR 0010).
Sources of truth
| What | Where |
|---|---|
| Requirements | ADO Feature 3214442 and stories 3214459–3214464. The v1.1 story files are in the engineering hub workspace, at drafts/cp1/agentic-landing-zone/stories/F02-assessment/ |
| Decisions on the board | Feature 3214442 "Decisions" table (2026-10-30 and 2026-11-02) |
| Changes to requirements | docs/adr/ (this pattern only) |
| MCP servers | ../../mcp-servers/ (existing, shared, unchanged) |
| Discovery's output shapes | ../ALZ-F01: Discovery Workflow Agent/cp1-*/schemas/output.json |
Platform facts (verified in modernization-platform @ c89aedcb9)
select_artifactshas exactly one output,artifacts: per-file records with base64contentwhile the selection fits the inline allowance (8 MiB live). Beyond that, a record carries only afile_path(migration 024;artifacts_activities.py).parse_assessment_filesaccepts those records under any input key (fileshere). It returnsparsed_content, the texts joined by\n\n---\n\nwith no names, andfile_metadata, which has the names in the same order (data_file_processing.pyparse_files).- A record with no inline content is skipped silently by the parser. An oversized Discovery output therefore arrives as a missing document, and the agents report it.
- Static inputs are applied after run inputs. So the
a1step leavesartifact_idsto the run (from_workflow) unless--artifact-idfixes them in the export. - Step outputs are registered as project artifacts only at the end of a run, and only when it has a
project_id. The artifact name is{execution_id}_{step_id}. - Review decisions go to the activity log (
record_review_decision), not onto artifacts. That's why the approval reference is a run input (ADR 0002). - Models served by the cluster's LiteLLM:
gpt-5.6-sol,gpt-5.4,gpt-4.1,gpt-4o, and the embedding modeltext-embedding-3-small(checked 2026-11-08 through KAB'sGET /api/v1/config/models, which asks LiteLLM;gpt-4.1is new since 2026-11-01). - No upload step (ADR 0009). KAB hands every run input to every step, and
parse_assessment_filesreads afileinput before thefileswired to it, so a rate card uploaded asfilewas parsed byp1in place of the Discovery package (runexec_20d55cfe6fd7). The rate card is the run inputrate_card_document, its JSON as text. - Run
exec_4be193061c41(2026-11-05): intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds anaz account get-access-tokentoken (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares onlycritic_modelwithexecutor_modelsand getsin_cloud_estatefor K-10. - Never pass the scaffold an option:
scaffold_from_stories.py --helponce took--helpas the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there. - K-12 at build time (ADR 0010):
build_workflow.pyrefuses a critic deployed on an executor's model, reading each agent'sLITELLM_MODEL; critic 0.1.4 reports K-12 from its two inputs. Runexec_0b0afb70b8b9(first with live Azure evidence: posture 0.69, 10 findings) failed only on K-12. - The critic's recalculation rules (critic 0.1.6): the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (
RECALC_RULES), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (runexec_1779863642d6, K-04). - Critic 0.2.0 (ADR 0011): K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its
app/andtests/when regenerating. Runexec_4d88027a4d26failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50). - Run
exec_2cd7a953335b(critic 0.2.0): code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule countedrepresentsedges (ADR 0012; risk 0.1.1, disposition 0.1.2). - Critic 0.3.0 (ADR 0013): K-02's citation lookup is in code (
app/evidence.py, against the verified package); the model judges only support. Runexec_1f518b04d6e4failed K-02 on lookups that resolve. - Run
exec_d170329cba65(critic 0.3.0, 2026-11-06): every citation resolved in code; K-04 to K-12 passed except K-10 (unverified). Failed K-03: the profiler had its own dependents rule without ADR 0012'srepresentsexclusion and listed CI0010001 to CI0010008 as dependents (profiler 0.1.1). Failed K-02 on the model's support judgement, partly for missing rate-card evidence, which the prompt says isn't Discovery evidence. - Run
exec_659b13e5f947(profiler 0.1.1): K-03 passed (no dependents). Failed only K-02: the model judged the oneevidence_refalone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum). - The scaffold builds every prompt before it deletes anything (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
- Run
exec_49e355141382(critic 0.3.1, 2026-11-06): verdict pass, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in runexec_659b13e5f947on the same package: the disposition agent applies the adjustments differently between runs. - The Assessment package (ADR 0014, 2026-11-08): step
s8,cp1-assessment-package-agent, code only, afterdisposition_signoff. It is hand-written like the intake; the scaffold doesn't touch it.INPUTSinapp/package.pyandPACKAGE_INPUTSinscripts/build_workflow.pymust agree (a test checks). Tested on runexec_49e355141382's step outputs. - The Assessment report (ADR 0015, 2026-11-08): step
s7,cp1-assessment-report-agent, carries the review point. The model writes from facts made in code;app/package.pythere is a copy of the package agent's (a test fails if they differ).scripts/package_schemas.pywrites both agents' input schemas fromINPUTS. - A new agent must be in KAB's agent catalog before a workflow that uses it validates ("Agent '...' not found in this environment");
--updatedoesn't register agents. The API isPOST /api/v1/agent-catalog/discover-and-registerwith{"url": ...}(Editor), which reads the agent card. A code-only agent needs no MCP key Secret. - Run
exec_1b1257380df7(RUN-ASM-AZURE-014, 2026-11-08), first on the 12-step workflow: critic pass; the report (model, about 9k tokens) quoted only figures in its facts (19 of 19 checked); the Engagement Lead continued at the report; the package (step 11 in KAB's numbering) was complete, every part's sha256 recomputed, APP-001signed_off. Runexec_52f93ebb2dee(013, old workflow) was stopped. - Claude models in Foundry (2026-11-08):
claude-sonnet-5-5andclaude-haiku-5-5are deployed oncrucible-kab-eng-dev-ai(the resource behind the gateway's models) but not mapped in LiteLLM, so agents can't call them. Mapping one is a LiteLLM admin step (KAB'sAKS/LLM_configuration_Foundry_LiteLLM.md); the product owner chose not to do it for now. - Report 0.2.0 (ADR 0015 addendum): the report also reads
p1.parsed_contentand has a## The applicationsection (assets, owner, how Discovery identified it, data stores, dependencies), extracted in code byapp/discovery.py. - Run
exec_374d5d577077(RUN-ASM-AZURE-015, report 0.2.0, 2026-11-08): critic pass; the report's new## The applicationsection lists all ten assets with their types, CMDB class and support group, the grouping basis, and the seven dependencies; 17 of 17 figures and every id in the report are in its facts. Report agent: about 13k tokens.