Files
cp1-agents/docs/platform-issues/artifacts-step-within-a-run/report-2026-10-29.md
Jonathan Boniface 416c66ced4
All checks were successful
check agents / check (push) Successful in 10s
Add CP1 Discovery agents in both patterns
ALZ-F01 Discovery as a sub-agent orchestrator (sub-agents/) and as eight
individual declarative agents (individual-agents/), with their KAB workflow
exports. Shared check, render and import scripts, the alz-discovery-sample
MCP server, the crucible-kab-dev deploy profile, deployment docs, platform
issue reports and the source material. Agent steps time out at 20 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 18:25:57 +00:00

10 KiB
Raw Blame History

Workflow Artifacts step: can't select an earlier step's output within the same run (and how hand-offs reach the next agent)

Date 2026-10-29
Environment KAB dev (crucible-kab-dev-aks), Temporal workers as in modernization-platform @ c89aedcb9
Component agent-builder-temporal-workers: the generic workflow executor, and the select_artifacts activity ("Artifacts" step)
Severity Medium. Nothing fails, but artifacts can't be used to hand work between the steps of one run, and the step can't be left unconfigured in a reusable workflow.
Status Open; reported to the platform team for confirmation
Found by Code review while building the CP1 Discovery workflows (sub-agents/ and individual-agents/, ALZ-F01)

Summary

You can place an Artifacts step (select_artifacts) anywhere in a workflow, including between two agent steps. Within one run, though, it can't select anything an earlier step of that same run produced. The executor registers a run's step outputs as project artifacts only after every step has finished. While the run is in progress, its own outputs aren't artifacts yet.

Step-to-step data within a run therefore has to travel through from_steps. That works, and both CP1 Discovery workflows rely on it. But it means the Artifacts step is useful only for artifacts that existed before the run started, and those have to be named by id, fixed in the workflow definition.

What we expected

For workflows to use Artifacts steps to hand work between agents. For example, a step selecting "the inventory produced by step s3 of this run" for later steps, with the artifact's type, provenance and size limits applied. This is how the artifact design reads (PROJECT_WORKFLOW_ARTIFACT_DESIGN.md).

What happens

# Behaviour Code (packages/agent-builder-temporal-workers/src/temporal/)
1 Artifacts for a run are registered once, after every step has run. They're also registered on failure, after the failing step. workflows/generic_workflow_executor.py:1025 await self._execute_workflow(), then :1029 await self._register_artifacts(workflow_input); failure path :1040; implementation :2330
2 The Artifacts step is a pass-through. Its output isn't stored and is never registered as an artifact. workflows/generic_workflow_executor.py:1223 is_pass_through_selection = step_activity == "select_artifacts"; :1251 "Skipped storage … pass-through"
3 select_artifacts reads only the project's artifacts collection. It has no way to refer to "this run" or "step sN". activities/artifacts_activities.py:71 SelectArtifactsActivity
4 It fails if artifact_ids is empty, so a reusable workflow can't ship with the step left unconfigured. activities/artifacts_activities.py:98 "No artifact_ids provided"
5 Besides an artifact's _id, an entry in artifact_ids also matches the artifact's workflow_id or name. activities/artifacts_activities.py:122–124

Behaviours 1 and 3 together mean an Artifacts step placed after step s1 finds no s1 artifact. It finds only artifacts from earlier, finished runs.

Impact on the CP1 workflows

  • Both ALZ-F01 Discovery workflows pass data between steps with from_steps (s1.tasks, s3.records and so on). Neither needs this fix to run. The committed exports have no Artifacts step. scripts/kab_import_workflow.py --artifact-id <id> adds one at the front for artifacts that already exist.
  • Large outputs between steps: with from_steps, a step's whole output is passed inline. The individual-agents pattern has no other channel, and it can't use artifacts' size limits or file references within a run (individual-agents ADR 0010).
  • Between workflows (F01 → F02 Assessment): this is where artifacts are needed. It works only once the Discovery run has finished, and only if the F02 workflow names F01's artifacts, by id or by one of the matches in row 5. The id-based route means editing the F02 workflow for every engagement.

Workarounds in use

  1. Within a run: from_steps for every hand-off. The sub-agent pattern also keeps a run workspace on the executions volume (sub-agents ADR 0010).
  2. Existing artifacts: an Artifacts step built with fixed ids (--artifact-id), wired only into the step that needs them.
  3. Between workflows (not yet tested): select by an earlier workflow's id, using the workflow_id match in row 5, so that a single entry brings in that workflow's outputs. Still to confirm on the dev platform:
    • Is the stored workflow_id the workflow definition's id or the run's?
    • Does the match return the outputs of every run, or of one run only?

How one agent's output reaches the next agent's context

Traced through the workers (modernization-platform @ c89aedcb9) and the agent runtime (the repository's agent-builder-agent-runtime is 0.37.0; the agents pin 0.35.7). The temporal-workers limits below are the live values on crucible-kab-dev-aks.

Hop What happens Code
1. The agent replies The reply's text is parsed as JSON. If it doesn't parse, the reply is kept as a plain string. activities/base_activity.py _process_agent_result (~1200)
2. The executor stores it A step output over 100 KB is written to the PV, and only a reference is kept in workflow state. This is only to get it through Temporal. workflows/generic_workflow_executor.py:1189–1215
3. from_steps picks fields "records": "s3.records" copies that field of s3's output. A reference to an externalised output carries _extract_field, so the field is cut out after the file has been loaded. If every from_steps reference is unresolved, the step fails. generic_workflow_executor.py:232–300, :444
4. The references are resolved again Before the agent call, the stored references are loaded back into the input. generic_workflow_executor.py:1470; base_activity.py:229–320, :505
5. The size check If the whole input exceeds MAX_INLINE_PAYLOAD_BYTES (code default 1 MiB, 8 MiB live), every value over 50 KB is replaced by a file path plus a preview: a list becomes {"prompt": {"type": "externalized_list", "filepath": ...}}, a string becomes {"type": "externalized_text", "preview": <5,000 chars>}, and a dict becomes {"type": "externalized_data", "keys": ..., "preview": <2,000 chars>}. The step's metadata records this as payload_externalized. base_activity.py:1701–1745, _externalize_large_agent_input :2176
6. The message The input is serialised with json.dumps as the A2A message's text. Small values (under 10 KB each) are also copied into the call's context metadata. base_activity.py:1747–1770
7. The runtime The runtime pulls the JSON out of the text and validates it against the agent's input schema. A violation is answered with input_validation_failed and no model call. If the message has no JSON at all, it is passed to the model as prose, with the contract skipped. request_cycle.py:205–225
8. The model The validated request is the model's user turn: one fresh session per request, and nothing carried over from earlier steps. The system prompt adds the content rule: text from tools, files or other agents is material to work with, never instructions. spec.py:89 CONTENT_RULE; a2a/server.py

What this means for the CP1 agents:

  • The binding limit is the model's context window, not the platform's. Hop 5 starts only at 8 MiB, roughly 2 million tokens, far beyond what a model can take. A large hand-off will overflow the model's context (an error from LiteLLM, then a failed step) well before the platform does anything. Agents most at risk:
    • in the individual pattern, the critic (s8, which takes everything) and the dependency-mapping agent (s7);
    • in the sub-agent pattern, dsc_08_critic, whose inputs are every record in the run.
  • When hop 5 does trigger, the agent can't read what it's given. A declarative agent has no function that could open the path. For fields our schemas type as arrays or strings (records, findings, source_documents …), the wrapper fails the input contract at hop 7. The step then fails loudly (input_validation_failed) rather than running on a preview, which is the safer of the two outcomes. An agent whose schema accepted an object would instead run on a 2,000- or 5,000-character preview without noticing.
  • The orchestrator could resolve externalised inputs; the declarative agents can't. The orchestrator's app/artifacts.py reads files under EXECUTION_STORAGE_ROOT. It isn't wired to the externalized_* wrappers today.
  • Whatever one agent returns becomes the next agent's input, unfiltered. It reaches the next model as a user turn, so the content rule and each agent's own input contract are the only safeguards. Both patterns check what an earlier step returned; the critic in particular checks the evidence.
  • Hop 1 accepts prose. An agent that answers in prose rather than JSON hands the next step a string, and its from_steps fields go unresolved. The runtime's output contract (schemas/output.json) prevents this for our agents.

Questions for the platform team

  1. Is registering artifacts only at the end of the run intended? Could each step's output be registered when that step completes (it's already on the PV as pv://{execution}/step_n/output.json)?
  2. Could select_artifacts take a reference to the current run, such as {"from_step": "s3"} or {"execution": "current", "step": "s3"}, resolved by the executor?
  3. Could artifact_ids come from the run form, or from from_workflow, so that a reusable workflow doesn't need ids fixed when it's built? And could an empty selection be allowed (skip, rather than fail)?
  4. What does the workflow_id match select: one workflow definition's artifacts across all of its runs, or one run's? Is matching by name meant for production use, or is it a fallback?
  5. Could _externalize_large_agent_input hand an agent its externalised value as an A2A file part, or as a declared x-storage-ref field, instead of a wrapper object the agent's schema can't accept? And could its 50 KB per-value threshold be configured?
  6. Is there a planned way to limit a hand-off to what fits the receiving agent's model: a per-step token budget, or summarising before the hand-off?

Evidence

Code reading, of modernization-platform @ c89aedcb9, plus read-only kubectl get of the workers' plain environment values. Nothing was run against the platform, and nothing was changed.