Files
cp1-agents/individual-agents/ALZ-F02: Assessment & Disposition Workflow Agent/.agents/CONTEXT.md
Jonathan Boniface 3b3ca64de0
Some checks failed
check agents / check (push) Failing after 14s
F02: K-02 lookup in code (ADR 0013), represents rule in the profiler; first passing run
- Critic 0.3.0: the hooks look up every disposition citation in the verified
  Discovery package (app/evidence.py). An unresolved citation fails K-02 and
  rejects the disposition; whether the records support the basis stays the
  model's judgement.
- Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref
  is the primary record), ADR 0013 addendum.
- Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering
  ALZ-ASM-02).
- Scaffold: builds every prompt before deleting anything.
- Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15).
- Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 15:59:01 +00:00

6.6 KiB
Raw Blame History

Context: individual agents, ALZ-F02 Assessment & Disposition

Facts for this Feature's individual-agents pattern, for coding agents. Repository-wide facts are in ../../.agents/CONTEXT.md. Per-agent context is in <agent>/.agents/CONTEXT.md.

Pattern

  • Six declarative agents, one per ALZ-ASM story, plus the code-only intake (s0, ADR 0008). Only ALZ-ASM-02 and ALZ-ASM-05 declare MCP tools; the other four and the intake have none.
  • The workflow (workflows/cp1-alz-assessment-individual.json) is the orchestration. Step order comes from parallelGroup and executionLevel, and data from from_steps. The step map is STEPS in scripts/build_workflow.py, and the scaffold reads it for each agent's context file.
  • Order: a1 the Discovery package artifact, p1 parse, then s0 intake (checks the Discovery package in code; a refusal fails the run), then s1 planner, then s2 profiler, s3 risk and s4 security posture in parallel, then s5 disposition, then s6 critic. s6 has a human approval, which is disposition_signoff (ADR 0004).
  • Nothing persists between steps except their replies, as in F01 (its ADR 0010).

Sources of truth

What Where
Requirements ADO Feature 3214442 and stories 3214459–3214464. The v1.1 story files are in the engineering hub workspace, at drafts/cp1/agentic-landing-zone/stories/F02-assessment/
Decisions on the board Feature 3214442 "Decisions" table (2026-10-30 and 2026-11-02)
Changes to requirements docs/adr/ (this pattern only)
MCP servers ../../mcp-servers/ (existing, shared, unchanged)
Discovery's output shapes ../ALZ-F01: Discovery Workflow Agent/cp1-*/schemas/output.json

Platform facts (verified in modernization-platform @ c89aedcb9)

  • select_artifacts has exactly one output, artifacts: per-file records with base64 content while the selection fits the inline allowance (8 MiB live). Beyond that, a record carries only a file_path (migration 024; artifacts_activities.py).
  • parse_assessment_files accepts those records under any input key (files here). It returns parsed_content, the texts joined by \n\n---\n\n with no names, and file_metadata, which has the names in the same order (data_file_processing.py parse_files).
  • A record with no inline content is skipped silently by the parser. An oversized Discovery output therefore arrives as a missing document, and the agents report it.
  • Static inputs are applied after run inputs. So the a1 step leaves artifact_ids to the run (from_workflow) unless --artifact-id fixes them in the export.
  • Step outputs are registered as project artifacts only at the end of a run, and only when it has a project_id. The artifact name is {execution_id}_{step_id}.
  • Review decisions go to the activity log (record_review_decision), not onto artifacts. That's why the approval reference is a run input (ADR 0002).
  • Models served by the cluster's LiteLLM: gpt-5.6-sol, gpt-5.4, gpt-4o (checked 2026-11-01).
  • No upload step (ADR 0009). KAB hands every run input to every step, and parse_assessment_files reads a file input before the files wired to it, so a rate card uploaded as file was parsed by p1 in place of the Discovery package (run exec_20d55cfe6fd7). The rate card is the run input rate_card_document, its JSON as text.
  • Run exec_4be193061c41 (2026-11-05): intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds an az account get-access-token token (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares only critic_model with executor_models and gets in_cloud_estate for K-10.
  • Never pass the scaffold an option: scaffold_from_stories.py --help once took --help as the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there.
  • K-12 at build time (ADR 0010): build_workflow.py refuses a critic deployed on an executor's model, reading each agent's LITELLM_MODEL; critic 0.1.4 reports K-12 from its two inputs. Run exec_0b0afb70b8b9 (first with live Azure evidence: posture 0.69, 10 findings) failed only on K-12.
  • The critic's recalculation rules (critic 0.1.6): the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (RECALC_RULES), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (run exec_1779863642d6, K-04).
  • Critic 0.2.0 (ADR 0011): K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its app/ and tests/ when regenerating. Run exec_4d88027a4d26 failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50).
  • Run exec_2cd7a953335b (critic 0.2.0): code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule counted represents edges (ADR 0012; risk 0.1.1, disposition 0.1.2).
  • Critic 0.3.0 (ADR 0013): K-02's citation lookup is in code (app/evidence.py, against the verified package); the model judges only support. Run exec_1f518b04d6e4 failed K-02 on lookups that resolve.
  • Run exec_d170329cba65 (critic 0.3.0, 2026-11-06): every citation resolved in code; K-04 to K-12 passed except K-10 (unverified). Failed K-03: the profiler had its own dependents rule without ADR 0012's represents exclusion and listed CI0010001 to CI0010008 as dependents (profiler 0.1.1). Failed K-02 on the model's support judgement, partly for missing rate-card evidence, which the prompt says isn't Discovery evidence.
  • Run exec_659b13e5f947 (profiler 0.1.1): K-03 passed (no dependents). Failed only K-02: the model judged the one evidence_ref alone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum).
  • The scaffold builds every prompt before it deletes anything (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
  • Run exec_49e355141382 (critic 0.3.1, 2026-11-06): verdict pass, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in run exec_659b13e5f947 on the same package: the disposition agent applies the adjustments differently between runs.