Files
cp1-agents/individual-agents/ALZ-F01: Discovery Workflow Agent/.agents/PLAN.md
Jonathan Boniface 9e503885cc ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part
  hashed (sha256 over canonical JSON), with a manifest and a value scorecard.
- Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021):
  CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them.
- Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum
  narrower than its producer's.
- Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic
  agents; the report and Well-Architected review agents; ADRs 0012-0021.
- Workflow cases with value bars and run limits; run evaluation and usage collection scripts;
  the run context template and the field-by-field review for ADR 0016.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

42 lines
8.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation plan: individual agents, ALZ-F01 Discovery
For coding agents. Update the status in the same change as the work.
Status values: `done`, `blocked (<on whom>)`, `todo`.
| # | Step | Verified by | Status |
|---|---|---|---|
| 1 | Generate the eight agent folders from the v1.1 stories, cp4-agents style, without `alz-run` or the write tools | `scripts/check_agents.py` (cp4's rules) 0 problems; the runtime parses every `agent.yaml`; every schema is valid | done |
| 2 | Eval cases per agent, Meridian estate, `timeout: 330` | `python -m kyndryl_agent_runtime.eval --validate` | done (not yet run live) |
| 3 | Workflow export: 8 steps, `from_steps` hand-offs, critic approval | `build_workflow.py` asserts the fields on both sides; same keys as cp4's export | done |
| 4 | Deploy profile and kustomize render | `render_k8s.py` + `kustomize build` (3 objects each) | done |
| 5 | Docs: README, user guides, ADRs 0001–0010; agent context | Review | done |
| 6 | Register the MCP servers. Done in the CP1 tenant, under the nine logical names, not by fixing the default tenant | Gateway reports 15 healthy servers | done (2026-11-01) |
| 7 | Build and push the eight images | `podman push` | done (2026-10-29, on the pinned runtime 0.35.7) |
| 8 | Store each `MCP_AGENT_API_KEY`, using gateway keys from `scripts/kab_bootstrap_discovery.py`. The keys expire after 30 days. Whether the agents also need creating in KAB for the workflow is unconfirmed | Secrets exist; no file holds a key; each pod resolves its MCP tools at startup | done (2026-11-01) |
| 9 | `kubectl diff -k`, then `apply -k`, for each agent | Pods ready; agent cards served | done (2026-10-29): all 8 Ready, cards served; requests fail at startup until step 8 |
| 9a | Uploads and artifacts (ADR 0011): `source_documents` / `reference_documents` on the planner and inventory (0.2.0, regenerated), front steps in `build_workflow.py`, `scripts/kab_import_workflow.py` | cp4 check 0 problems; runtime parse; eval validation; builder asserts wiring | done (2026-10-29) |
| 9b | Move into `kab-agents-and-workflows/cp1-agents` (`individual-agents/<Feature>/`); shared scripts, MCP server and deploy profile at the repository root; `repo: cp1-agents-individual` kept per agent (Deployment selector) | `check_agents.py` 0 problems; `build_workflow.py` output identical; `kubectl diff -k` empty for all 8 | done (2026-10-29) |
| 9c | Timeouts: every workflow step 20 minutes (`AGENT_STEP_TIMEOUT_MINUTES`, shared); `AGENT_MAX_DURATION_SECONDS` 1140 in each ConfigMap and in the scaffold | Export rebuilt; `kubectl diff -k` shows only that line for all 8 | done (2026-11-01): applied to all 8, and running |
| 10 | Run each agent's eval cases, then the workflow on `ENG-MERIDIAN-001` | All cases pass; the critic step waits for approval | done (2026-11-01): evals 8 of 8; workflow run `exec_fbf1d3af1290` scored 27 of 29 on `workflows/cases/meridian-baseline`. Five findings to fix are in the engineering hub report `reports/cp1/discovery-run-quality-2026-11-01.md` |
| 10a | Fix the five findings from the first run (`reports/cp1/discovery-run-quality-2026-11-01.md`): planner 0.4.0, scan, network, dependency and critic 0.2.0, sample server 0.1.2 | Workflow case `meridian-baseline`, now 33 checks; run `exec_fbf1d3af1290` scores 26 of 33 against it | done (2026-11-01): `RUN-007` scored 36 of 36 with critic verdict `pass`. Agent evals not rerun since these changes |
| 10b | Report step (ADR 0012): `cp1-discovery-report-agent`, rendered in code; the approval gate moves to it; case now 41 checks; Markdown for the record, HTML for the customer | `scripts/render_discovery_report.py` wrote the `RUN-007` report | done (2026-11-01): deployed; `wf_06af245b-dc0e-4bf5-90ff-ab74a4a4330f`; `RUN-008` scored 41 of 41 with verdict `pass`. Publishing to KAB not yet tried |
| 10c | Missing sources: `source_gaps` from the planner, respected by CMDB, classification, critic and report; case `snapshot-missing-sources` | `RUN-GCP-002` passed 8 of 8 on `wf_795e1918-e240-4854-b31e-282a1ab655b8` | done (2026-11-01); Meridian `RUN-009` 41 of 42; inventory 0.3.1 deployed, then `RUN-010` |
| 10d | Report narrative: a model writes a marked executive summary and recommendations from the structured findings only. Code keeps every finding; a check confirms each cited id exists and that source gaps and the verdict are stated; the Engagement Lead approves it at the gate. Test prompt injection on it. Start after the sub-agent pattern's first live run, so the patterns compare cleanly. Update ADR 0012 | Cited ids checked in code; injection test passes | todo |
| 11 | Live provider (ADR 0013): `provider` input, live tools on 5 agents, Live provider prompt sections (`scripts/add_live_provider.py`); then build, deploy and run on GCP `delry-cp1-prj-1239-a31894bc` | `check_agents.py` 0 problems; workflow export rebuilt; live GCP run | done (2026-11-03): deployed; live runs on GCP and, from 2026-11-04, Azure |
| 11a | First live GCP run `exec_0f8629066e13` (verdict `pass`, approved by the user). Fixes: dependencies keep out-of-scope targets (0.4.2), planner never scopes projects or subscriptions (0.7.3), critic checks dropped dependencies and structural scope entries and runs on `gpt-5.4` (0.4.3) | Rerun on GCP | done (2026-11-03): deployed; `RUN-GCP-LIVE-004` (`exec_42fc8affdbdd`) held all four fixes |
| 11b | Well-Architected review (ADR 0014): new code-only agent `cp1-well-architected-review-agent` 0.1.0, workflow steps `s11` and `s12`; richer report (0.6.0) from `provider_attributes` (scan 0.4.0, inventory 0.5.0) and `topology_summary` (network 0.4.0) | `tests/test_waf.py` 6 passed; report rendered from `RUN-GCP-LIVE-002` data; `check_agents.py` 10 folders, 0 problems; export 13 steps | done (2026-11-03): deployed with its MCP key; `exec_42fc8affdbdd` completed all 13 steps |
| 12 | Value plan phase 0 (ADR 0020, `docs/discovery-value-plan.md`): the value scorecard (`cp1-discovery-package-agent/app/scorecard.py`) in `evaluate_workflow_run.py`; `value:` and `limits:` in both Azure cases (`azure-crucible-kab-dev` gains its `case.yaml`); `scripts/collect_run_usage.py` | Baseline on `exec_14956d524a8b`: 24% filled, 27% filled or declared, 6 of 7 run checks; usage 106k tokens, 10 calls, 108 s, within limits | done (2026-11-05) |
| 12a | Value plan phase 1 (ADR 0020): `cp1-discovery-package-agent` 0.1.0, step `s13` after the report's review point, 14 steps; the context report gains its 2026-11-05 wiring section | `tests/test_package.py` 8 passed; `check_context_doc.py` matches; `check_agents.py` 11 folders, 0 problems; `exec_14956d524a8b` 46 of 46 and `exec_d1c242609f82` 19 of 19 | done (2026-11-05): deployed, registered in the agent catalog (`discover-and-register`), workflow `wf_05565d2d` updated to 14 steps |
| 12b | F02 verifies the package's manifest in code (F02's own ADR), and selects one artifact | An F02 run from one package id | todo (F02) |
| 12c | Value plan phases 2–7 | `docs/discovery-value-plan.md` | todo; phase 2 waits on the owner's review of ADR 0016 |
| 13 | Applications and asset owners (ADR 0021, value plan phase 3, first part): CMDB 0.7.0 (owner, business service, support group per CI), dependency mapping 0.6.0 (`app/applications.py`, sub-agent `propose_application_groups`), report 0.9.0 (Applications section), package 0.2.0 (`applications` part, scorecard 2); both Azure cases gain application checks and raised value bars | dependency 20, report 8, package 9, CMDB 16 tests passed; `check_agents.py` 0 problems; `check_context_doc.py` matches | done (2026-11-05): deployed; `exec_58ba6a71d00b` 56 of 56 (46% filled, 7 of 7 run checks, 33% of findings with an owner) and `exec_925194d79c47` 21 of 21 |
| 14 | Compare with `sub-agents/ALZ-F01: Discovery Workflow Agent/` and record which pattern is adopted | An ADR in both patterns | todo (product owner) |
## Open questions (owner)
- ~~**Blocks the critic:** `claude-sonnet-5-5` isn't served on the cluster's LiteLLM.~~ Resolved 2026-11-03: the critic runs on `gpt-5.4` (ADR 0008, Accepted).
- **Platform team:** does the executor externalise large step outputs between steps? (ADR 0010)
- **Product owner:** confirm the Proposed ADRs: 0003, 0004, 0005, 0006, 0007, 0010, and review ADR 0016 field by field (it carries the run context to solution architecture, which is a separate stream).