Some checks failed
check agents / check (push) Failing after 14s
- Critic 0.3.0: the hooks look up every disposition citation in the verified Discovery package (app/evidence.py). An unresolved citation fails K-02 and rejects the disposition; whether the records support the basis stays the model's judgement. - Critic 0.3.1: K-02 judges a disposition's citations together (evidence_ref is the primary record), ADR 0013 addendum. - Profiler 0.1.1: represents edges are not dependencies (ADR 0012, now covering ALZ-ASM-02). - Scaffold: builds every prompt before deleting anything. - Tests on Discovery run exec_58ba6a71d00b's package and runs 007 and 009 (15). - Run exec_49e355141382 (RUN-ASM-AZURE-012) passed on crucible-kab-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
43 lines
6.6 KiB
Markdown
43 lines
6.6 KiB
Markdown
# Context: individual agents, ALZ-F02 Assessment & Disposition
|
||
|
||
Facts for this Feature's individual-agents pattern, for coding agents. Repository-wide facts are in `../../.agents/CONTEXT.md`. Per-agent context is in `<agent>/.agents/CONTEXT.md`.
|
||
|
||
## Pattern
|
||
|
||
- **Six declarative agents, one per ALZ-ASM story,** plus the code-only intake (`s0`, ADR 0008). Only ALZ-ASM-02 and ALZ-ASM-05 declare MCP tools; the other four and the intake have none.
|
||
- **The workflow (`workflows/cp1-alz-assessment-individual.json`) is the orchestration.** Step order comes from `parallelGroup` and `executionLevel`, and data from `from_steps`. The step map is `STEPS` in `scripts/build_workflow.py`, and the scaffold reads it for each agent's context file.
|
||
- **Order:** `a1` the Discovery package artifact, `p1` parse, then `s0` intake (checks the Discovery package in code; a refusal fails the run), then `s1` planner, then `s2` profiler, `s3` risk and `s4` security posture in parallel, then `s5` disposition, then `s6` critic. `s6` has a human approval, which is `disposition_signoff` (ADR 0004).
|
||
- **Nothing persists between steps** except their replies, as in F01 (its ADR 0010).
|
||
|
||
## Sources of truth
|
||
|
||
| What | Where |
|
||
|---|---|
|
||
| Requirements | ADO Feature 3214442 and stories 3214459–3214464. The v1.1 story files are in the engineering hub workspace, at `drafts/cp1/agentic-landing-zone/stories/F02-assessment/` |
|
||
| Decisions on the board | Feature 3214442 "Decisions" table (2026-10-30 and 2026-11-02) |
|
||
| Changes to requirements | `docs/adr/` (this pattern only) |
|
||
| MCP servers | `../../mcp-servers/` (existing, shared, unchanged) |
|
||
| Discovery's output shapes | `../ALZ-F01: Discovery Workflow Agent/cp1-*/schemas/output.json` |
|
||
|
||
## Platform facts (verified in `modernization-platform` @ `c89aedcb9`)
|
||
|
||
- **`select_artifacts` has exactly one output, `artifacts`:** per-file records with base64 `content` while the selection fits the inline allowance (8 MiB live). Beyond that, a record carries only a `file_path` (migration 024; `artifacts_activities.py`).
|
||
- **`parse_assessment_files` accepts those records** under any input key (`files` here). It returns `parsed_content`, the texts joined by `\n\n---\n\n` with no names, and `file_metadata`, which has the names in the same order (`data_file_processing.py` `parse_files`).
|
||
- **A record with no inline content is skipped silently by the parser.** An oversized Discovery output therefore arrives as a missing document, and the agents report it.
|
||
- **Static inputs are applied after run inputs.** So the `a1` step leaves `artifact_ids` to the run (`from_workflow`) unless `--artifact-id` fixes them in the export.
|
||
- **Step outputs are registered as project artifacts only at the end of a run, and only when it has a `project_id`.** The artifact name is `{execution_id}_{step_id}`.
|
||
- **Review decisions go to the activity log** (`record_review_decision`), not onto artifacts. That's why the approval reference is a run input (ADR 0002).
|
||
- **Models served by the cluster's LiteLLM:** `gpt-5.6-sol`, `gpt-5.4`, `gpt-4o` (checked 2026-11-01).
|
||
- **No upload step** (ADR 0009). KAB hands every run input to every step, and `parse_assessment_files` reads a `file` input before the `files` wired to it, so a rate card uploaded as `file` was parsed by `p1` in place of the Discovery package (run `exec_20d55cfe6fd7`). The rate card is the run input `rate_card_document`, its JSON as text.
|
||
- **Run `exec_4be193061c41` (2026-11-05):** intake verified the package; every step ran. Two faults: (1) every Azure tool call from ALZ-ASM-02 and ALZ-ASM-05 got HTTP 401 from Azure, because the project's Azure Discovery connection holds an `az account get-access-token` token (about an hour); refresh it, or store a service principal, before a run. (2) The critic failed K-12 on a Discovery agent's model read from the package; critic 0.1.2 compares only `critic_model` with `executor_models` and gets `in_cloud_estate` for K-10.
|
||
- **Never pass the scaffold an option:** `scaffold_from_stories.py --help` once took `--help` as the stories folder and deleted the six agent folders (restored from the deployed images and the scaffold). It now refuses unless every story file is there.
|
||
- **K-12 at build time (ADR 0010):** `build_workflow.py` refuses a critic deployed on an executor's model, reading each agent's `LITELLM_MODEL`; critic 0.1.4 reports K-12 from its two inputs. Run `exec_0b0afb70b8b9` (first with live Azure evidence: posture 0.69, 10 findings) failed only on K-12.
|
||
- **The critic's recalculation rules (critic 0.1.6):** the scaffold copies ALZ-ASM-03's and ALZ-ASM-04's rule sections into the critic's prompt (`RECALC_RULES`), so K-04 to K-06 recalculate with the producers' own rules. Before 0.1.6 the critic had none (run `exec_1779863642d6`, K-04).
|
||
- **Critic 0.2.0 (ADR 0011):** K-04 to K-07, K-11 and the verdict are set in code by the critic's hooks; the scaffold keeps its `app/` and `tests/` when regenerating. Run `exec_4d88027a4d26` failed on the model's own sums (0.70 for 1.00; 0.60 for 0.50).
|
||
- **Run `exec_2cd7a953335b` (critic 0.2.0):** code checks ran; K-04 to K-06 passed. Two faults fixed: K-07 compared the wrong queue (critic 0.2.1), and the dependents rule counted `represents` edges (ADR 0012; risk 0.1.1, disposition 0.1.2).
|
||
- **Critic 0.3.0 (ADR 0013):** K-02's citation lookup is in code (`app/evidence.py`, against the verified package); the model judges only support. Run `exec_1f518b04d6e4` failed K-02 on lookups that resolve.
|
||
- **Run `exec_d170329cba65` (critic 0.3.0, 2026-11-06):** every citation resolved in code; K-04 to K-12 passed except K-10 (unverified). Failed K-03: the profiler had its own dependents rule without ADR 0012's `represents` exclusion and listed CI0010001 to CI0010008 as dependents (profiler 0.1.1). Failed K-02 on the model's support judgement, partly for missing rate-card evidence, which the prompt says isn't Discovery evidence.
|
||
- **Run `exec_659b13e5f947` (profiler 0.1.1):** K-03 passed (no dependents). Failed only K-02: the model judged the one `evidence_ref` alone. Critic 0.3.1 judges a disposition's citations together (ADR 0013 addendum).
|
||
- **The scaffold builds every prompt before it deletes anything** (2026-11-06): a rewrite that no longer matched its story stopped it after the critic's folder was deleted (restored from a backup).
|
||
- **Run `exec_49e355141382` (critic 0.3.1, 2026-11-06): verdict pass**, the first. K-01 to K-12 pass; K-10 unverified (no call log). APP-001: no dependents, risk 0.00 (partial: eol, data, change missing), posture 0.69 with 5 advisory findings, rehost at confidence 0.50, target cost USD 11,622 a year, routed to sign-off. Confidence was 0.60 in run `exec_659b13e5f947` on the same package: the disposition agent applies the adjustments differently between runs.
|