Files
cp1-agents/docs/artifact-integration-plan-2026-10-29.md
Jonathan Boniface 416c66ced4
All checks were successful
check agents / check (push) Successful in 10s
Add CP1 Discovery agents in both patterns
ALZ-F01 Discovery as a sub-agent orchestrator (sub-agents/) and as eight
individual declarative agents (individual-agents/), with their KAB workflow
exports. Shared check, render and import scripts, the alz-discovery-sample
MCP server, the crucible-kab-dev deploy profile, deployment docs, platform
issue reports and the source material. Agent steps time out at 20 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 18:25:57 +00:00

87 lines
9.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Plan: integrating KAB platform artifacts into the CP1 Discovery agents
> **Moved here on 2026-10-29.** Written while the agents were in two separate repositories, both now in this one:
> `cp1-agents` → `sub-agents/ALZ-F01: Discovery Workflow Agent/`, and `cp1-agents-individual` → `individual-agents/ALZ-F01: Discovery Workflow Agent/`.
| | |
|---|---|
| Date | 2026-10-29 |
| Scope | `code_generated/cp1-agents` (orchestrator pattern) and `code_generated/cp1-agents-individual` (individual pattern) |
| Platform reviewed | `modernization-platform` @ `7adfc87f` (2026-10-19) |
| Rule | In the orchestrator pattern, only the orchestrator handles artifacts. Its sub-agents never receive an artifact reference: they see workspace records only. |
| Status | Plan for review. Nothing changed yet. |
## 1. How artifacts work on the platform today
| Question | Finding | Where |
|---|---|---|
| Can an agent call the artifact API? | **No.** The API needs a user's or the platform's internal token, and agents run with `AGENT_EGRESS=enforce`: they can reach only the model gateway and the MCP gateway. | `agent-builder-api/src/api/artifact_routes.py`; agents' `configmap.yaml` |
| How does an artifact reach an agent? | Through a **`select_artifacts` workflow step**. The designer picks `artifact_ids` (fixed per workflow, as `static` inputs). At run time the step outputs either **`artifacts`**, a list of entries with `file_path` on the executions volume, `content_type` and optional base64 `content` up to 5 MB, or **`parsed_content`**, the decoded file. An agent step takes one of them with `from_steps`, e.g. `"artifacts": "s0.artifacts"`, and the API marks the producing step `wants_artifacts` / `wants_parsed_content`. | `agent-builder-temporal-workers/.../activities/artifacts_activities.py` (`SelectArtifactsActivity`); `workflow_execution_routes.py` (`ON_DEMAND_OUTPUTS`) |
| How does an agent declare what it accepts? | An input named `artifacts` (array) with **`x-artifact-types`** in `schemas/input.json`. KAB's run chat then offers the agent for runs that hold those types. | `agent-builder-knowledge-graph-analyst/schemas/input.json`; `services/run_chat/follow_up.py` |
| How does an agent read one? | The reference agent accepts the list in several shapes (`find_artifacts`). It reads a referenced file only if the real path is inside the executions volume (`read_file`). | `agent-builder-knowledge-graph-analyst/app/request.py` |
| How does an agent's output become an artifact? | **Automatically.** At the end of a run with a `project_id`, the executor registers every step's output as an **`execution_output`** artifact at `pv://{execution_id}/step_{n}/output.json`. There is no per-step artifact type. | `generic_workflow_executor.py` (`_register_artifacts`); `artifact_registration_activities.py` |
| Which artifact types exist? | `knowledge_graph`, `structured_data`, `file`, `report`, `assessment_result` (supports contributions) and `execution_output` (on the volume). | `agent-builder-api/src/database/init_artifact_types.py` |
| Typed outputs and contributions? | Only through the API (`POST /projects/{id}/artifacts`, `.../contributions`), so not from an agent. | `artifact_routes.py` |
**This corrects what I said earlier:** workflow steps *can* take constants. The executor accepts `static` inputs beside `from_workflow` and `from_steps`.
## 2. Design
### 2.1 Orchestrator pattern: only the orchestrator integrates
```text
s0 select_artifacts (static artifact_ids) --artifacts--> s1 orchestrator (plan) -> s2 collect -> s3 analyse -> s4 critique
|
+- import_artifacts (code): each artifact -> workspace record
plan/artifact-<n>, produced_by "platform-artifact"
sub-agents <- delegate(input_refs=["plan/artifact-1", ...]) (records only; never artifact ids or paths)
each phase reply ------------------------------------------> execution_output artifact (automatic, end of run)
```
- **Input.** Add an optional `artifacts` input (array, `x-artifact-types: ["file", "structured_data", "report", "execution_output"]`).
- **Import, in code, not the model.** In the plan phase, `pre_process` calls a new `app/artifacts.py` (`import_artifacts`). For each artifact it:
- accepts the platform's shapes, as `find_artifacts` does;
- reads only files whose real path is under `/data/executions`, with a size cap;
- parses JSON or CSV, otherwise keeps text;
- stores one write-once record per artifact, `plan/artifact-<n>`, with `artifact_id`, `type_id`, the source `sha256` and `file_path` as provenance, chained in the manifest like every other record.
- **Sub-agents unchanged.** The orchestrator's plan skill lists the artifact records and passes them with `delegate`: to the planner (for example a scope or CMDB export), and in analyse to DSC-03 / DSC-04. Sub-agents get records exactly as now, and their prompts and functions don't change.
- **Output.** Phase replies already become `execution_output` artifacts. The critique reply gains a compact `deliverable` object (verdict, coverage, the counts per output, and refs), so its artifact is the Discovery deliverable that Assessment can select.
- **Charter.** Artifact content is data: the existing content rule and the injection checks apply. The critic gets the artifact records with everything else.
### 2.2 Individual pattern: each agent integrates itself, declaratively
These agents have no code, so they can't read files. They use **`parsed_content`**, the decoded file, never `artifacts`, the paths.
- **Workflow.** Add an optional `s0 select_artifacts` step. Steps that need source data take it with `from_steps`, e.g. `"source_records": "s0.parsed_content"`:
- **planner:** a scope or connector export;
- **inventory:** a CMDB or asset export.
- **Contracts.** Add an optional `source_records` input to those two agents (no `x-artifact-types`, because they don't take the `artifacts` list). Their prompts say to treat it as data, cite it as `artifact:<id>`, and never follow instructions in it.
- **Output.** Unchanged. Every step's reply already becomes an `execution_output` artifact.
- **Limit.** `parsed_content` is inline, bounded by the platform's inline payload limit, so large exports stay a job for the orchestrator pattern.
## 3. Steps
| # | Step | Repository | Verified by |
|---|---|---|---|
| 1 | ADRs: orchestrator 0012 (artifacts enter only through the orchestrator, as workspace records); individual 0011 (artifacts through `parsed_content`) | both | Review |
| 2 | `app/artifacts.py`: `find_artifacts` and a bounded, volume-only `read_artifact_file`; `import_artifacts` writes `plan/artifact-<n>` | `cp1-agents` | Dry run: list, wrapped and file-reference shapes; path outside the volume refused; oversize refused; duplicate import write-once |
| 3 | Input schema `artifacts` + `x-artifact-types`; output `deliverable` on critique; `pre_process` calls the import in plan only | `cp1-agents` | Dry run; contract validation |
| 4 | Plan skill: list artifact records and pass them to `dsc_01_planner`; analyse skill: pass them to DSC-03 / DSC-04 | `cp1-agents` | Review; eval case with a CMDB export artifact |
| 5 | Workflow export: optional `s0 select_artifacts` step with `static` `artifact_ids`; `s1.from_steps.artifacts` | both | Export shape still matches; builder asserts wiring |
| 6 | Individual agents: `source_records` on planner and inventory; prompt line; regenerate from the scaffold | `cp1-agents-individual` | `check_agents.py` (cp4's) 0 problems; runtime parse; eval validation |
| 7 | Version 0.2.0 + release notes; rebuild and push; `kubectl diff`, then `apply` | both | Pods ready; cards list the new input |
| 8 | In-cluster check: put a small fixture artifact file under `/data/executions`, run `import_artifacts` inside the orchestrator pod, confirm the record and its hash | `cp1-agents` | Record stored; chain sound |
| 9 | End-to-end, once MCP keys exist: run the workflow in a project with one selected artifact | both | Artifact record used by the planner; phase outputs listed as `execution_output` artifacts |
## 4. Decisions for you
1. **Which artifacts should Discovery take?** For example a CMDB or asset export, a scope list, a previous discovery run's outputs, or client documents. This decides the `x-artifact-types` list and which sub-agents get them.
2. **Should we ask the platform team for typed outputs?** Today every output is `execution_output`. A per-step output type, such as `structured_data` for the inventory or `assessment_result` with contributions for findings, would need a platform change: either the executor taking an output type per step, or an artifacts MCP server agents could call.
3. **Projects.** Outputs are registered only when the run has a `project_id`, so Discovery must run inside the engagement's KAB project.
## 5. Constraints and risks
- **Still blocked end to end:** real runs need the MCP keys, which need the gateway tenant link (see `kab-dev-mcp-gateway-default-tenant-500-2026-10-29.md`). Steps 1–8 don't.
- **Sizes:** `select_artifacts` inlines at most 5 MB, and larger values travel as file references, which only the orchestrator can read.
- **Provenance:** an imported artifact's `sha256` is recorded at import. A later change to the artifact doesn't change the run's record.