Files
cp1-agents/individual-agents/ALZ-F02: Assessment & Disposition Workflow Agent/.agents/CONTEXT.md
Jonathan Boniface 041e199248
Some checks failed
check agents / check (push) Failing after 12s
ALZ-F02 Assessment: critic code checks (ADR 0011), represents edges (ADR 0012)
- Critic 0.2.0 / 0.2.1: K-04 (risk), K-05 (confidence), K-06 (TCO), K-07 (routing) and K-11
  (schema and 7R enum) run in code in the critic's hooks (app/arithmetic.py,
  app/business_logic.py); post_process sets those checks, the rejected dispositions, the
  sign-off queue and the verdict. The model keeps the judgement checks; a different reading of
  a code check is kept in unverified_claims. K-07 checks the disposition agent's own queue.
  Tests on the records of run exec_4d88027a4d26, whose model sums were wrong.
- A represents edge is not a dependency (ADR 0012): risk scorer 0.1.1 and disposition 0.1.2
  leave it out of dependents and dependencies; the disposition and critic prompts carry the
  risk scorer's rule. Run exec_1f518b04d6e4: APP-001's risk 1.00 -> 0.00, out of the tail.
- The scaffold declares the critic's hooks, copies app/ in its Dockerfile, and keeps its
  app/ and tests/ when regenerating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 13:20:40 +00:00

5.2 KiB
Raw Blame History

Context: individual agents, ALZ-F02 Assessment & Disposition

Facts for this Feature's individual-agents pattern, for coding agents. Repository-wide facts are in ../../.agents/CONTEXT.md. Per-agent context is in <agent>/.agents/CONTEXT.md.

Pattern

  • Six declarative agents, one per ALZ-ASM story, plus the code-only intake (s0, ADR 0008). Only ALZ-ASM-02 and ALZ-ASM-05 declare MCP tools; the other four and the intake have none.
  • The workflow (workflows/cp1-alz-assessment-individual.json) is the orchestration. Step order comes from parallelGroup and executionLevel, and data from from_steps. The step map is STEPS in scripts/build_workflow.py, and the scaffold reads it for each agent's context file.
  • Order: a1 the Discovery package artifact, p1 parse, then s0 intake (checks the Discovery package in code; a refusal fails the run), then s1 planner, then s2 profiler, s3 risk and s4 security posture in parallel, then s5 disposition, then s6 critic. s6 has a human approval, which is disposition_signoff (ADR 0004).
  • Nothing persists between steps except their replies, as in F01 (its ADR 0010).

Sources of truth

What Where
Requirements ADO Feature 3214442 and stories 3214459–3214464. The v1.1 story files are in the engineering hub workspace, at drafts/cp1/agentic-landing-zone/stories/F02-assessment/
Decisions on the board Feature 3214442 "Decisions" table (2026-10-30 and 2026-11-02)
Changes to requirements docs/adr/ (this pattern only)
MCP servers ../../mcp-servers/ (existing, shared, unchanged)
Discovery's output shapes ../ALZ-F01: Discovery Workflow Agent/cp1-*/schemas/output.json

Platform facts (verified in modernization-platform @ c89aedcb9)

  • select_artifacts has exactly one output, artifacts: per-file records with base64 content while the selection fits the inline allowance (8 MiB live). Beyond that, a record carries only a file_path (migration 024; artifacts_activities.py).
  • parse_assessment_files accepts those records under any input key (files here). It returns parsed_content, the texts joined by \n\n---\n\n with no names, and file_metadata, which has the names in the same order (data_file_processing.py parse_files).
  • A record with no inline content is skipped silently by the parser. An oversized Discovery output therefore arrives as a missing document, and the agents report it.
  • Static inputs are applied after run inputs. So the a1 step leaves artifact_ids to the run (from_workflow) unless --artifact-id fixes them in the export.
  • Step outputs are registered as project artifacts only at the end of a run, and only when it has a project_id. The artifact name is {execution_id}_{step_id}.
  • Review decisions go to the activity log (record_review_decision), not onto artifacts. That's why the approval reference is a run input (ADR 0002).
  • Models served by the cluster's LiteLLM: gpt-5.6-sol, gpt-5.4, gpt-4o (checked 2026-11-01).
  • No upload step (ADR 0009). KAB hands every run input to every step, and parse_assessment_files reads a file input before the files wired to it, so a rate card uploaded as file was parsed by p1 in place of the Discovery package (run exec_20d55cfe6fd7). The rate card is the run input rate_card_document, its JSON as text.
  • Run exec_4be193061c41 (2026-11-05): intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds an az account get-access-token token (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares only critic_model with executor_models and gets in_cloud_estate for K-10.
  • Never pass the scaffold an option: scaffold_from_stories.py --help once took --help as the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there.
  • K-12 at build time (ADR 0010): build_workflow.py refuses a critic deployed on an executor's model, reading each agent's LITELLM_MODEL; critic 0.1.4 reports K-12 from its two inputs. Run exec_0b0afb70b8b9 (first with live Azure evidence: posture 0.69, 10 findings) failed only on K-12.
  • The critic's recalculation rules (critic 0.1.6): the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (RECALC_RULES), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (run exec_1779863642d6, K-04).
  • Critic 0.2.0 (ADR 0011): K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its app/ and tests/ when regenerating. Run exec_4d88027a4d26 failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50).
  • Run exec_2cd7a953335b (critic 0.2.0): code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule counted represents edges (ADR 0012; risk 0.1.1, disposition 0.1.2).