Some checks failed
check agents / check (push) Failing after 14s
- Critic 0.3.0: the hooks look up every disposition citation in the verified Discovery package (app/evidence.py). An unresolved citation fails K-02 and rejects the disposition; whether the records support the basis stays the model's judgement. - Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref is the primary record), ADR 0013 addendum. - Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering ALZ-ASM-02). - Scaffold: builds every prompt before deleting anything. - Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15). - Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6.6 KiB
6.6 KiB
Context: individual agents, ALZ-F02 Assessment & Disposition
Facts for this Feature's individual-agents pattern, for coding agents. Repository-wide facts are in ../../.agents/CONTEXT.md. Per-agent context is in <agent>/.agents/CONTEXT.md.
Pattern
- Six declarative agents, one per ALZ-ASM story, plus the code-only intake (
s0, ADR 0008). Only ALZ-ASM-02 and ALZ-ASM-05 declare MCP tools; the other four and the intake have none. - The workflow (
workflows/cp1-alz-assessment-individual.json) is the orchestration. Step order comes fromparallelGroupandexecutionLevel, and data fromfrom_steps. The step map isSTEPSinscripts/build_workflow.py, and the scaffold reads it for each agent's context file. - Order:
a1the Discovery package artifact,p1parse, thens0intake (checks the Discovery package in code; a refusal fails the run), thens1planner, thens2profiler,s3risk ands4security posture in parallel, thens5disposition, thens6critic.s6has a human approval, which isdisposition_signoff(ADR 0004). - Nothing persists between steps except their replies, as in F01 (its ADR 0010).
Sources of truth
| What | Where |
|---|---|
| Requirements | ADO Feature 3214442 and stories 3214459–3214464. The v1.1 story files are in the engineering hub workspace, at drafts/cp1/agentic-landing-zone/stories/F02-assessment/ |
| Decisions on the board | Feature 3214442 "Decisions" table (2026-10-30 and 2026-11-02) |
| Changes to requirements | docs/adr/ (this pattern only) |
| MCP servers | ../../mcp-servers/ (existing, shared, unchanged) |
| Discovery's output shapes | ../ALZ-F01: Discovery Workflow Agent/cp1-*/schemas/output.json |
Platform facts (verified in modernization-platform @ c89aedcb9)
select_artifactshas exactly one output,artifacts: per-file records with base64contentwhile the selection fits the inline allowance (8 MiB live). Beyond that, a record carries only afile_path(migration 024;artifacts_activities.py).parse_assessment_filesaccepts those records under any input key (fileshere). It returnsparsed_content, the texts joined by\n\n---\n\nwith no names, andfile_metadata, which has the names in the same order (data_file_processing.pyparse_files).- A record with no inline content is skipped silently by the parser. An oversized Discovery output therefore arrives as a missing document, and the agents report it.
- Static inputs are applied after run inputs. So the
a1step leavesartifact_idsto the run (from_workflow) unless--artifact-idfixes them in the export. - Step outputs are registered as project artifacts only at the end of a run, and only when it has a
project_id. The artifact name is{execution_id}_{step_id}. - Review decisions go to the activity log (
record_review_decision), not onto artifacts. That's why the approval reference is a run input (ADR 0002). - Models served by the cluster's LiteLLM:
gpt-5.6-sol,gpt-5.4,gpt-4o(checked 2026-11-01). - No upload step (ADR 0009). KAB hands every run input to every step, and
parse_assessment_filesreads afileinput before thefileswired to it, so a rate card uploaded asfilewas parsed byp1in place of the Discovery package (runexec_20d55cfe6fd7). The rate card is the run inputrate_card_document, its JSON as text. - Run
exec_4be193061c41(2026-11-05): intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds anaz account get-access-tokentoken (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares onlycritic_modelwithexecutor_modelsand getsin_cloud_estatefor K-10. - Never pass the scaffold an option:
scaffold_from_stories.py --helponce took--helpas the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there. - K-12 at build time (ADR 0010):
build_workflow.pyrefuses a critic deployed on an executor's model, reading each agent'sLITELLM_MODEL; critic 0.1.4 reports K-12 from its two inputs. Runexec_0b0afb70b8b9(first with live Azure evidence: posture 0.69, 10 findings) failed only on K-12. - The critic's recalculation rules (critic 0.1.6): the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (
RECALC_RULES), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (runexec_1779863642d6, K-04). - Critic 0.2.0 (ADR 0011): K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its
app/andtests/when regenerating. Runexec_4d88027a4d26failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50). - Run
exec_2cd7a953335b(critic 0.2.0): code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule countedrepresentsedges (ADR 0012; risk 0.1.1, disposition 0.1.2). - Critic 0.3.0 (ADR 0013): K-02's citation lookup is in code (
app/evidence.py, against the verified package); the model judges only support. Runexec_1f518b04d6e4failed K-02 on lookups that resolve. - Run
exec_d170329cba65(critic 0.3.0, 2026-11-06): every citation resolved in code; K-04 to K-12 passed except K-10 (unverified). Failed K-03: the profiler had its own dependents rule without ADR 0012'srepresentsexclusion and listed CI0010001 to CI0010008 as dependents (profiler 0.1.1). Failed K-02 on the model's support judgement, partly for missing rate-card evidence, which the prompt says isn't Discovery evidence. - Run
exec_659b13e5f947(profiler 0.1.1): K-03 passed (no dependents). Failed only K-02: the model judged the oneevidence_refalone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum). - The scaffold builds every prompt before it deletes anything (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
- Run
exec_49e355141382(critic 0.3.1, 2026-11-06): verdict pass, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in runexec_659b13e5f947on the same package: the disposition agent applies the adjustments differently between runs.