diff --git a/.agents/plans/README.md b/.agents/plans/README.md index 05f5e5a..156dcd7 100644 --- a/.agents/plans/README.md +++ b/.agents/plans/README.md @@ -7,7 +7,7 @@ One directory per iteration. Each holds the plan for that iteration and, for fin | # | Directory | Status | What it is | | --- | --- | --- | --- | | 1 | `iter-01-execution-engine/` | steps 1 to 6 built on 2026-10-13; 7 planned | The Execution engine: brought from the Reference, given a Workflow, and proved on a toy Workflow and on the Reference's own. | -| 2 | `iter-02-orchestration-engine/` | F1 proposed; F2 and F4 built on 2026-10-20 | The Orchestration engine: routes each request to the agent for its work and relays everything back. | +| 2 | `iter-02-orchestration-engine/` | F1 proposed; F2 and F4 built on 2026-10-20; K1, K2 and K3's object storage (ADR-0004) built on 2026-10-21 | The Orchestration engine: routes each request to the agent for its work and relays everything back. | The plan across both repositories is `../tom_analysis/IMPLEMENTATION_PLAN.md`. diff --git a/.agents/plans/iter-02-orchestration-engine/MAP.md b/.agents/plans/iter-02-orchestration-engine/MAP.md index ee8e3de..bb5f9b6 100644 --- a/.agents/plans/iter-02-orchestration-engine/MAP.md +++ b/.agents/plans/iter-02-orchestration-engine/MAP.md @@ -2,7 +2,7 @@ **Iteration:** 2, the Orchestration engine **Intent:** The service every request comes through. It decides which agent the work belongs to, dispatches it, and relays every question and Approval. First behind it: the operating model agent (`../tom_analysis`). -**Status:** F1 proposed; F2 and F4 built 2026-10-20 +**Status:** F1 proposed; F2 and F4 built 2026-10-20; K1, K2 and object storage (K3) built 2026-10-21 ## Decided by the owner, 2026-10-20 @@ -13,6 +13,7 @@ - `../tom_analysis/IMPLEMENTATION_PLAN.md`, sections 2 and 4, and steps F1 to F5 in section 10. - `docs/adr/0003-one-orchestration-engine-above-the-agents.md`. +- `docs/adr/0004-the-orchestration-engine-keeps-what-agents-deliver.md`. - `../tom_analysis/.agents/plans/iter-02-platform-deployment/`: how an agent runs on the local KAB platform. ## Status @@ -22,5 +23,8 @@ | F1 | ADR-0003: one Orchestration engine above the agents | Accepted by the owner | proposed | | F2 | The service: an A2A server, a configured list of agents, a request passed through to one agent with every prompt relayed; its image; on the local platform | An Execution of the operating model agent runs through it, Approvals included, with the agent untouched | built (`steps/step-2.md`); the platform workflow through it is step 4 of `../tom_analysis` iteration 2 | | F3 | The decision: readings in code, the Jev question, thresholds, the fallbacks, the record | A labelled set of messages is routed correctly; low confidence asks; no Jev key still routes what is named | planned | -| F4 | "Both": architecture, then its files as `sources` to the operating model agent | One request yields both sets of Deliverables under one Customer case | built against scripted agents (`steps/step-4.md`), ahead of F3 at the owner's request; not yet run with real models | +| F4 | "Both": architecture, then its files as `sources` to the operating model agent | One request yields both sets of Deliverables under one Customer case | built (`steps/step-4.md`), ahead of F3 at the owner's request; run with real models 2026-10-21: the solution architecture agent's work handed to the operating model agent, both completed in 74.9 minutes, kept in MinIO (`../solution_architecture_agent/.agents/plans/iter-01-on-the-engine/steps/step-2.md`) | | F5 | Later messages to the agent that owns the work | A change to the architecture is followed by a change to the operating model | planned | +| K1 | What agents deliver, kept and served at signed links; a platform step given one document of links (ADR-0004, phase 1) | A platform step's output links to each delivered file, and each link serves it | built (`steps/step-K1.md`); the owner's run through the platform to follow | +| K2 | `sources_from`: a request names kept work instead of carrying its files; the sequence moved onto it | An operating model started from the latest architecture of a Customer case | built (`steps/step-K2.md`); the owner's run through the platform to follow | +| K3 | Object storage shared with the platform, the platform's sign-in, retention, each file as a platform artifact | A kept guide is listed among the project's artifacts | object storage built (`steps/step-K3.md`); retention, the platform's sign-in and platform artifacts not chosen for now, the last needing the platform team | diff --git a/.agents/plans/iter-02-orchestration-engine/steps/step-K1.md b/.agents/plans/iter-02-orchestration-engine/steps/step-K1.md new file mode 100644 index 0000000..2d14ec9 --- /dev/null +++ b/.agents/plans/iter-02-orchestration-engine/steps/step-K1.md @@ -0,0 +1,29 @@ +# Step K1: what agents deliver, kept and linked (ADR-0004, phase 1) + +Built 2026-10-21 on `feat/delivery-store`, the owner having chosen option 3 of three ways to make agents' outputs available in the KAB platform. + +## What was built + +- **`store.py`:** every completed agent Execution relayed is kept under `/tenants//cases//executions//`: its files as the agent named them, and `record.json` (work, how it ended, the request, tenant, Customer case, project, the guide's name, the files, when). Written once; a second write is refused. File names that leave the Execution, and tenants or cases that are not slugs, are refused. `latest` is the newest Execution of a work that ended `completed`. +- **`links.py`:** a link is the path with `exp` and `sig`, an HMAC of both under `LINK_SECRET`; `LINK_TTL_SECONDS`, 30 days by default, 0 for never. +- **`delivery.py`:** keeping for the relay, with the tenant from the platform's `tenant_id` (`local` otherwise) and the Customer case (`unassigned` otherwise), and the links to a record, its guide, each file, the latest of its work and its Customer case. A failure to keep is logged and never fails the caller's task. +- **`serving.py`:** `GET /tenants/{tenant}/cases/{case}`, `…/executions/{id}`, `…/executions/{id}/files/{name}` and `…/latest/{work}`. Each checks its link's signature against the path it was made for, or answers 403; a record is served with fresh links to its files. +- **The relay:** a completed agent's files are kept, whatever the caller asked for. `deliver: links` gives the caller one artifact, `result.json`: the status, the work named, the Customer case, and links to the last agent's record, guide, files, latest and case, with each agent's under `results`. A KAB workflow step (it sends `workflow_id` and `step_number`) is given links unless it asks for `files`; any other caller, files. The guide is the file the agent sent as markdown, as the Execution engine sends its final Deliverable. `sources` given as data, as a platform step maps an earlier output, are sent to the agent as text. +- **The card:** `deliver` and `sources` among the inputs, and an `outputSchema` for the document of links. +- **Deployment:** `STORE_DIR=/data/store` and a `/data` volume in the image; in the dev kit, the `orchestration-data` volume, `PUBLIC_URL=http://localhost:10030`, and `LINK_SECRET` from `ORCHESTRATION_LINK_SECRET` in the kit's `.env`, which is not committed. + +## What differed from the plan + +- The tests of option 1 (`test_document.py`, on `main`) are replaced by `test_links.py`: a platform step is given links, not the files' content. +- A file's link carries the name as it was quoted, and is checked against the request's raw path. + +## Verified + +- 25 tests in `test_store.py` and `test_links.py`: keeping, media types, never changing what was kept, refusing names and slugs, `latest`, signing, expiry; a platform step given links that serve what was delivered; the guide; unsigned and changed links refused; files still kept when files are asked for; a sequence's links, and a stopped sequence's; `latest` and the case listing; sources as data sent as text; the card. +- 577 tests pass; `ruff` and `mypy` clean. +- Deployed on the local platform: healthy, the store on its volume, the secret and the public URL set, the card declaring `deliver` and the document. An unsigned link is refused. + +## Not done + +- No run through the platform yet: the owner runs it from a workflow. +- Phase 2, `sources_from` (a request naming kept work instead of carrying files); phase 3, object storage, the platform's sign-in, retention, and each file as a platform artifact. diff --git a/.agents/plans/iter-02-orchestration-engine/steps/step-K2.md b/.agents/plans/iter-02-orchestration-engine/steps/step-K2.md new file mode 100644 index 0000000..4708768 --- /dev/null +++ b/.agents/plans/iter-02-orchestration-engine/steps/step-K2.md @@ -0,0 +1,25 @@ +# Step K2: building on kept work by reference (ADR-0004, phase 2) + +Built 2026-10-21 on `feat/sources-from`, from `feat/delivery-store`. + +## What was built + +- **`sources_from`:** a request names work kept earlier instead of carrying its files. It may be: + - a link from an earlier document of links, its `record` or its `latest`: what a platform step maps from an earlier step's output. The signature is checked; the link must be of the request's tenant when the request names one, and of its Customer case when it names one; + - `latest/`: the newest completed Execution of that work in the request's Customer case; + - an Execution's id in the request's Customer case. +- **What the agent is sent:** the kept files as `sources`, with the Orchestration engine's record that the work completed (`execution.json`: status, Customer case, Execution, the request, the work). A request naming no Customer case takes the kept work's. +- **Nothing starts, and the caller is told why,** when the reference resolves to nothing, to work that did not end `completed`, to another tenant or Customer case, when the link was changed, when nothing is kept here, or when `sources` is given as well. The reply is a completed task with `{"status": "not_started", "reason": ...}`. +- **The sequence on the same footing:** an agent after another is given what was kept of the one before (`Delivery.sources`), as a request naming it in `sources_from` would be. Without a store, it is given the files relayed, as before. +- **`Store.files`, `Delivery.resolve`, `Delivery.sources`;** the record `execution.json` moved to `delivery.py`. +- **The card** offers `sources_from`. + +## Verified + +- 12 tests in `test_sources_from.py`: each of the three forms; a `latest` link followed to the newest work; a changed link, another tenant, another Customer case, nothing kept, work that ended with warnings, and `sources` with `sources_from` each starting nothing; a sequence's next agent given what was kept; the card. +- 589 tests pass; `ruff` and `mypy` clean. +- Deployed on the local platform; the card offers `sources_from`. No agent was run: the owner runs it through the platform. + +## Using it from the platform + +A second workflow step on the Orchestration engine with `work: operating_model` and `sources_from` mapped from the first step's `record`; or, in a later workflow, `sources_from: latest/solution_architecture` with the Customer case. The operating model agent's own fields (who operates it, business outcomes) go in that step's `scope`. diff --git a/.agents/plans/iter-02-orchestration-engine/steps/step-K3.md b/.agents/plans/iter-02-orchestration-engine/steps/step-K3.md new file mode 100644 index 0000000..1d92927 --- /dev/null +++ b/.agents/plans/iter-02-orchestration-engine/steps/step-K3.md @@ -0,0 +1,30 @@ +# Step K3: what is kept, in object storage (ADR-0004, phase 3, first part) + +Built 2026-10-21 on `feat/object-store`, from `feat/sources-from`. + +## Scope, as the owner chose it + +Phase 3 was planned as object storage shared with the platform, the platform's sign-in, retention, and each kept file as a platform artifact. Looking at the platform first changed it: + +- **The platform's S3 and GCS artifact backends are not implemented** (`agent-builder-api/src/storage/router.py`: "S3 backend not yet implemented"). It serves artifacts from MongoDB, Azure Blob, Neo4j and its execution volume only. +- **The only artifact route the internal service token may call,** `POST /api/v1/executions/{id}/artifacts`, registers a step's whole output (`step_N/output.json`); `POST /projects/{id}/artifacts` needs a person's sign-in. + +So each kept file as a platform artifact is the platform team's change, and the platform's sign-in would help API callers only. The owner chose object storage alone, for now. + +## What was built + +- **`store.py`:** the store's rules (names, write-once, records, `latest`) over a `Blobs` layer with two kinds: `VolumeBlobs`, a folder, and `S3Blobs`, a prefix in an S3-compatible bucket, made if missing. The layout is the same in both: `tenants//cases//executions//{record.json, files/…}`. The record is written last, so an Execution is kept once its record is. A delivered file named `record.json` is not taken for one. +- **Settings:** `STORE_BACKEND` (`volume` or `s3`), `S3_BUCKET`, `S3_PREFIX`, `S3_ENDPOINT_URL`, `S3_REGION`; credentials are the standard `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. `store_from_settings` makes the store. +- **Packaging:** the `s3` extra (`boto3`); the image installs it. `moto` in the dev extra; mypy told the AWS SDK has no types. +- **Local deployment:** the dev kit's MinIO (`agent-builder-minio`, started alone from the `observability` profile), bucket `orchestration`, its root credentials from the kit's `.env`. + +## Verified + +- Every store test runs on both kinds of storage, S3 through moto in the process; and the object layout under its prefix, a delivered `record.json`, and the settings choosing the storage. +- 606 tests pass; `ruff` and `mypy` clean. +- Deployed: healthy, with the `orchestration` bucket made and reachable on MinIO. No agent was run. + +## Not done + +- Retention, the platform's sign-in, and each kept file as a platform artifact (a proposal to the platform team) remain, not chosen for now. +- Nothing kept on the volume before is moved: nothing had been kept. diff --git a/Dockerfile.orchestration b/Dockerfile.orchestration index d723f52..f9a8437 100644 --- a/Dockerfile.orchestration +++ b/Dockerfile.orchestration @@ -4,7 +4,9 @@ # podman build -f Dockerfile.orchestration -t orchestration-engine . # # Which agents it routes to is configuration, mounted at /app/agents.yaml -# (AGENTS_FILE); deploy/agents.local.yaml is the local KAB platform's. +# (AGENTS_FILE); deploy/agents.local.yaml is the local KAB platform's. What agents +# deliver is kept in /data/store (ADR-0004), or in an S3-compatible bucket with +# STORE_BACKEND=s3; give it a volume or a bucket, a PUBLIC_URL and a LINK_SECRET. FROM python:3.12-slim AS builder @@ -12,7 +14,7 @@ WORKDIR /build COPY pyproject.toml ./ COPY execution_engine/ ./execution_engine/ COPY orchestration/ ./orchestration/ -RUN pip install --no-cache-dir --prefix=/install . +RUN pip install --no-cache-dir --prefix=/install ".[s3]" FROM python:3.12-slim @@ -21,7 +23,8 @@ ENV PYTHONUNBUFFERED=1 \ PYTHONDONTWRITEBYTECODE=1 \ A2A_HOST=0.0.0.0 \ A2A_PORT=8080 \ - AGENTS_FILE=/app/agents.yaml + AGENTS_FILE=/app/agents.yaml \ + STORE_DIR=/data/store WORKDIR /app @@ -30,8 +33,10 @@ RUN groupadd -g 10001 appgroup && \ COPY --from=builder /install /usr/local COPY deploy/agents.local.yaml /app/agents.yaml +RUN mkdir -p /data/store && chown -R appuser:appgroup /data USER appuser +VOLUME ["/data"] EXPOSE 8080 CMD ["python", "-m", "orchestration.main"] diff --git a/docs/adr/0004-the-orchestration-engine-keeps-what-agents-deliver.md b/docs/adr/0004-the-orchestration-engine-keeps-what-agents-deliver.md new file mode 100644 index 0000000..6da9809 --- /dev/null +++ b/docs/adr/0004-the-orchestration-engine-keeps-what-agents-deliver.md @@ -0,0 +1,33 @@ +--- +status: accepted +--- + +# The Orchestration engine keeps what agents deliver + +An agent on the Execution engine delivers its work as named files: a guide, the data files behind it, infrastructure code. The KAB platform takes one output per workflow step and keeps it as one artifact; a set of named files reaches it as one unnamed string. Nothing delivered had an address of its own, so a guide could not be shared or cited, and a later request could not build on earlier work without the caller carrying its files. + +We decide that **the Orchestration engine keeps every file an agent delivers through it, and serves each at a stable address.** + +- **What is kept:** the files of every agent's Execution that completes, as the agent delivered them, with a record: the work, the status the agent ended with, the request, when, the Customer case, and the tenant and project a platform step names. +- **Where:** a store of its own, by tenant, Customer case and Execution. Files are copied in, not referred to in an agent's container, so they outlive any agent's deployment. +- **Never changed.** An Execution's files are written once. A Change request is a new Execution. Only `latest` moves: the newest completed Execution of a kind of work in a Customer case. +- **Addresses:** `/tenants/{tenant}/cases/{case}` lists a Customer case; `…/executions/{id}` is one Execution's record with links to its files; `…/executions/{id}/files/{path}` is a file with its media type; `…/latest/{work}` is the record of the newest completed Execution of that work. +- **Read only by signed link.** Every address carries an expiring signature the Orchestration engine made, because the files name customers, people and suppliers. The base of every link is a setting, as the address a browser uses is not the one inside the platform's network. +- **A caller that takes one output per step is given links.** A KAB workflow step, or any caller that asks (`deliver: links`), is given one JSON document of links to what was kept. A caller that asks for files is given them one by one, as before; they are kept either way. + +This keeps ADR-0003's rule that the Orchestration engine produces nothing: it keeps what agents produced. It replaces ADR-0003's consequence that nothing outlives a restart for delivered work; tasks in flight are still held in memory. + +Accepted by the owner on 2026-10-21, as option 3 of three ways to make agents' outputs available in the platform. + +## Considered Options + +- **One JSON document with every file's content (option 1):** rejected. Nothing has an address, a large architecture can pass the platform's inline limit, and earlier work is reusable only by wiring steps on a canvas. +- **The platform keeps each named artifact (option 2):** not rejected, and not ours to make: a change to how the platform's worker treats every agent. Kept as the platform team's to consider; a shared object store later lets each kept file become a platform artifact without moving it. +- **Links into each agent's own deliverables folder:** rejected. Those folders are private to a container and go with it. + +## Consequences + +- The Orchestration engine has a volume, a public base URL and a signing secret. Without a secret set, one is made at start-up and links stop working on a restart. +- Links expire (`LINK_TTL_SECONDS`, 30 days by default; 0 for never). A record fetched with a valid link carries fresh links to its files. +- A request may name earlier work by reference rather than carry its files (`sources_from`): the next step, with the sequence moved onto it. +- Object storage, the platform's own sign-in and retention follow, in that order. diff --git a/orchestration/card.py b/orchestration/card.py index 730fb17..9489362 100644 --- a/orchestration/card.py +++ b/orchestration/card.py @@ -81,6 +81,42 @@ def build_card( ) +_LINK = {"type": ["string", "null"], "format": "uri"} +# The document of links a caller asking for links is given (ADR-0004). +OUTPUT_SCHEMA: dict[str, Any] = { + "type": "object", + "properties": { + "status": { + "type": "string", + "description": "completed, completed_with_warnings, or stopped when a " + "sequence did not run to its end.", + }, + "work": {"type": "string", "description": "The work that was done."}, + "customer_case": {"type": "string"}, + "execution_id": { + "type": "string", + "description": "The last agent's Execution.", + }, + "record": {**_LINK, "description": "The last Execution's record and links."}, + "guide": {**_LINK, "description": "The last agent's guide."}, + "files": { + "type": "object", + "description": "A signed link to each file the last agent delivered.", + }, + "latest": { + **_LINK, + "description": "The newest completed Execution of this work.", + }, + "case": {**_LINK, "description": "Every Execution kept for the Customer case."}, + "results": { + "type": "object", + "description": "Each agent's Execution, how it ended and its links, by its work.", + }, + "not_started": {"type": "array", "items": {"type": "string"}}, + }, +} + + def input_contract( agents: Sequence[Agent], sequences: Sequence[AgentSequence] = () ) -> dict[str, Any]: @@ -115,6 +151,30 @@ def input_contract( "uiFieldType": "textarea", "description": "The spec to start from, as a YAML or JSON document.", }, + "sources": { + "type": "object", + "description": ( + "Files to start from, by name, such as an earlier step's " + "`files`: another agent's delivered work." + ), + }, + "sources_from": { + "type": "string", + "description": ( + "Work kept earlier to start from instead of its files: an " + "earlier step's `record` or `latest` link, `latest/`, " + "or an Execution's id in this Customer case." + ), + }, + "deliver": { + "type": "string", + "enum": ["links", "files"], + "description": ( + "links: one document of signed links to what the agents " + "delivered, kept here (a platform step is given this). " + "files: each file as the agent sent it." + ), + }, "pre_approved": { "type": "boolean", "default": True, @@ -126,6 +186,7 @@ def input_contract( }, "required": [WORK, "customer_case"], }, + "outputSchema": OUTPUT_SCHEMA, } diff --git a/orchestration/config.py b/orchestration/config.py index 0201664..920cd25 100644 --- a/orchestration/config.py +++ b/orchestration/config.py @@ -26,3 +26,22 @@ class OrchestrationSettings(BaseSettings): # How long to wait to connect to an agent. Reading waits as long as the # agent's work takes. CONNECT_TIMEOUT_SECONDS: float = 10 + + # What agents deliver is kept and served at signed links (ADR-0004): in a folder + # (`volume`, STORE_DIR; unset, nothing is kept) or in an S3-compatible bucket + # (`s3`). The bucket's credentials are the standard AWS_ACCESS_KEY_ID and + # AWS_SECRET_ACCESS_KEY. + STORE_BACKEND: str = "volume" + STORE_DIR: str | None = None + S3_BUCKET: str | None = None + S3_PREFIX: str = "" + S3_ENDPOINT_URL: str | None = None # MinIO, or another S3-compatible service + S3_REGION: str = "us-east-1" + # The base of every link: the address a browser reaches this service at, which + # is not its address inside the platform's network. Unset: A2A_PUBLIC_URL. + PUBLIC_URL: str | None = None + # The secret links are signed with. Unset: one is made at start-up, and links + # stop working when the service restarts. + LINK_SECRET: str | None = None + # How long a link works; 0 for always. + LINK_TTL_SECONDS: int = 30 * 24 * 3600 diff --git a/orchestration/delivery.py b/orchestration/delivery.py new file mode 100644 index 0000000..ecdca58 --- /dev/null +++ b/orchestration/delivery.py @@ -0,0 +1,186 @@ +"""Keeping what an agent delivered and linking to it (ADR-0004): the store, the +signer and the address a caller reaches them at, together.""" + +from __future__ import annotations + +import json +import logging +import re +from collections.abc import Mapping +from typing import Any +from urllib.parse import parse_qs, quote, unquote, urlsplit + +from orchestration.links import Signer +from orchestration.store import Store + +logger = logging.getLogger(__name__) + +LOCAL_TENANT = "local" # what is kept for a caller that names no tenant +UNASSIGNED = "unassigned" # for a request that names no Customer case +# The Orchestration engine's own record that an Execution completed, sent with its +# files to the agent that builds on them. +RECORD = "execution.json" +_NOT_SLUG = re.compile(r"[^A-Za-z0-9._-]+") +_KEPT_PATH = re.compile(r"^/tenants/([^/]+)/cases/([^/]+)/(executions|latest)/([^/]+)$") + + +def _slug(value: Any, default: str) -> str: + text = _NOT_SLUG.sub("-", str(value or "")).strip("-.") + return text or default + + +class Delivery: + def __init__(self, store: Store, signer: Signer, public_url: str) -> None: + self.store = store + self.signer = signer + self.public_url = public_url.rstrip("/") + + @staticmethod + def place(fields: Mapping[str, Any]) -> tuple[str, str]: + """The tenant and Customer case what a request delivers is kept under.""" + return ( + _slug(fields.get("tenant_id"), LOCAL_TENANT), + _slug(fields.get("customer_case"), UNASSIGNED), + ) + + def keep( + self, + fields: Mapping[str, Any], + *, + work: str, + execution_id: str, + status: str, + request: str, + files: Mapping[str, str], + guide: str | None, + ) -> dict[str, Any] | None: + """Keep an agent's Execution; None, logged, when it cannot be kept, so a + caller still gets the agent's answer.""" + tenant, case = self.place(fields) + try: + return self.store.keep( + tenant=tenant, + case=case, + execution_id=execution_id, + work=work, + status=status, + request=request, + files=files, + guide=guide, + project_id=fields.get("project_id"), + ) + except (OSError, ValueError): + logger.exception("Could not keep %s Execution %s", work, execution_id) + return None + + # --- building on what was kept (phase 2) --------------------------------------- + + def resolve(self, reference: str, fields: Mapping[str, Any]) -> dict[str, Any]: + """The kept Execution a request names in `sources_from`: a link from an + earlier document, `latest/`, or an Execution's id in the request's + Customer case. Raises ValueError, saying why, for anything else.""" + tenant, case = self.place(fields) + named_case = fields.get("customer_case") + if reference.startswith(("http://", "https://", "/")): + url = urlsplit(reference) + query = parse_qs(url.query) + exp, sig = (query.get(key, [None])[0] for key in ("exp", "sig")) + if not self.signer.verify(url.path, exp, sig): + raise ValueError("The link in sources_from is not valid.") + match = _KEPT_PATH.match(url.path) + if not match: + raise ValueError("The link in sources_from is not to kept work.") + tenant_of, case, kind, ident = match.groups() + if fields.get("tenant_id") and tenant_of != tenant: + raise ValueError("The link in sources_from is of another tenant.") + tenant = tenant_of + if named_case and _slug(named_case, UNASSIGNED) != case: + raise ValueError( + f"The work in sources_from is of the Customer case {case}, " + f"not {named_case}." + ) + ident = unquote(ident) + record = ( + self.store.record(tenant, case, ident) + if kind == "executions" + else self.store.latest(tenant, case, ident) + ) + elif reference.startswith("latest/"): + work = reference.removeprefix("latest/") + record = self.store.latest(tenant, case, work) + if record is None: + raise ValueError( + f"No {work} work has completed in the Customer case {case}." + ) + else: + record = self.store.record(tenant, case, reference) + if record is None: + raise ValueError( + f"Nothing is kept for {reference!r} in the Customer case {case}." + ) + if record["status"] != "completed": + raise ValueError( + f"The {record['work']} work {record['execution_id']} ended " + f"{record['status']}; only work that completed is built on." + ) + return record + + def sources(self, record: Mapping[str, Any]) -> dict[str, str]: + """A kept Execution as an agent's sources: its files, with the record that + it completed.""" + files = self.store.files( + record["tenant"], record["customer_case"], record["execution_id"] + ) + completion = { + "status": record["status"], + "customer_case": record["customer_case"] + if record["customer_case"] != UNASSIGNED + else None, + "execution_id": record["execution_id"], + "workflow_request": record["request"], + "work": record["work"], + } + files[RECORD] = json.dumps({k: v for k, v in completion.items() if v}) + return files + + # --- addresses ------------------------------------------------------------- + + @staticmethod + def case_path(tenant: str, case: str) -> str: + return f"/tenants/{tenant}/cases/{case}" + + def execution_path(self, tenant: str, case: str, execution_id: str) -> str: + return f"{self.case_path(tenant, case)}/executions/{execution_id}" + + def file_path(self, tenant: str, case: str, execution_id: str, name: str) -> str: + return f"{self.execution_path(tenant, case, execution_id)}/files/{quote(name)}" + + def latest_path(self, tenant: str, case: str, work: str) -> str: + return f"{self.case_path(tenant, case)}/latest/{quote(work)}" + + def link(self, path: str) -> str: + return self.signer.link(self.public_url, path) + + def links(self, record: Mapping[str, Any]) -> dict[str, Any]: + """Signed links to a kept Execution: its record, its guide, each file, the + latest of its work and its Customer case.""" + tenant, case = record["tenant"], record["customer_case"] + execution_id = record["execution_id"] + files = { + name: self.link(self.file_path(tenant, case, execution_id, name)) + for name in record["files"] + } + return { + "record": self.link(self.execution_path(tenant, case, execution_id)), + "guide": files.get(record.get("guide") or ""), + "files": files, + "latest": self.link(self.latest_path(tenant, case, record["work"])), + "case": self.link(self.case_path(tenant, case)), + } + + def shown(self, record: Mapping[str, Any]) -> dict[str, Any]: + """A record as it is served: what was kept, with links, without internals.""" + return { + **{k: v for k, v in record.items() if k != "kept_ns"}, + "links": self.links(record), + } diff --git a/orchestration/links.py b/orchestration/links.py new file mode 100644 index 0000000..a22dafc --- /dev/null +++ b/orchestration/links.py @@ -0,0 +1,47 @@ +"""Signed links to what the Orchestration engine keeps (ADR-0004). + +A link is a path with `exp` and `sig` in its query: `sig` is an HMAC of the path and +`exp` under the engine's secret. `exp` 0 never expires. +""" + +from __future__ import annotations + +import hashlib +import hmac +import time +from collections.abc import Callable +from urllib.parse import urlencode + + +class Signer: + def __init__( + self, + secret: str, + ttl_seconds: int, + now: Callable[[], float] = time.time, + ) -> None: + self._secret = secret.encode() + self._ttl = ttl_seconds + self._now = now + + def _signature(self, path: str, exp: int) -> str: + return hmac.new( + self._secret, f"{path}\n{exp}".encode(), hashlib.sha256 + ).hexdigest() + + def sign(self, path: str) -> dict[str, str]: + """The query that makes `path` readable.""" + exp = int(self._now()) + self._ttl if self._ttl > 0 else 0 + return {"exp": str(exp), "sig": self._signature(path, exp)} + + def verify(self, path: str, exp: str | None = None, sig: str | None = None) -> bool: + if not exp or not sig or not exp.isdigit(): + return False + expires = int(exp) + if expires and expires < self._now(): + return False + return hmac.compare_digest(sig, self._signature(path, expires)) + + def link(self, base_url: str, path: str) -> str: + """`path` under `base_url`, signed.""" + return f"{base_url.rstrip('/')}{path}?{urlencode(self.sign(path))}" diff --git a/orchestration/main.py b/orchestration/main.py index 6edcc11..71e05a4 100644 --- a/orchestration/main.py +++ b/orchestration/main.py @@ -3,6 +3,7 @@ from __future__ import annotations import logging +import secrets from collections.abc import AsyncIterator, Sequence from contextlib import asynccontextmanager @@ -24,7 +25,13 @@ from orchestration.agents import Agent, load_routes from orchestration.agents import Sequence as AgentSequence from orchestration.card import A2A_PATH, build_card, card_document from orchestration.config import OrchestrationSettings +from orchestration.delivery import Delivery +from orchestration.links import Signer from orchestration.relay import AgentClients, RelayExecutor +from orchestration.serving import kept_routes +from orchestration.store import Store, store_from_settings + +logger = logging.getLogger(__name__) LEGACY_CARD_PATH = "/.well-known/agent.json" @@ -36,11 +43,16 @@ def create_app( httpx_client: httpx.AsyncClient | None = None, clients: AgentClients | None = None, sequences: Sequence[AgentSequence] = (), + store: Store | None = None, + signer: Signer | None = None, + public_url: str | None = None, ) -> Starlette: """The server. `httpx_client` is how it reaches the agents; one is made (and closed) here when none is given. `clients` replaces how agents are - called altogether, for tests.""" + called altogether, for tests. `store`, `signer` and `public_url` replace what + the settings make of them.""" settings = settings or OrchestrationSettings() + delivery = _delivery(settings, store, signer, public_url) own_client = httpx_client is None client = httpx_client or httpx.AsyncClient( timeout=httpx.Timeout(None, connect=settings.CONNECT_TIMEOUT_SECONDS) @@ -53,6 +65,7 @@ def create_app( clients or AgentClients(client), settings.HEARTBEAT_SECONDS, sequences, + delivery, ), task_store=InMemoryTaskStore(), agent_card=card, @@ -93,12 +106,42 @@ def create_app( Route("/health", health, methods=["GET"]), Route(AGENT_CARD_WELL_KNOWN_PATH, agent_card, methods=["GET"]), Route(LEGACY_CARD_PATH, agent_card, methods=["GET"]), + *(kept_routes(delivery) if delivery else []), *jsonrpc, ], lifespan=lifespan, ) +def _delivery( + settings: OrchestrationSettings, + store: Store | None, + signer: Signer | None, + public_url: str | None, +) -> Delivery | None: + """Where what agents deliver is kept, and how it is linked to; None when + nothing is to be kept.""" + if store is None: + store = store_from_settings(settings) + if store is None: + return None + if signer is None: + secret = settings.LINK_SECRET + if not secret: + logger.warning( + "LINK_SECRET is not set: links made now stop working on a restart" + ) + secret = secrets.token_hex(32) + signer = Signer(secret, settings.LINK_TTL_SECONDS) + base = ( + public_url + or settings.PUBLIC_URL + or settings.A2A_PUBLIC_URL + or f"http://{settings.A2A_HOST}:{settings.A2A_PORT}" + ) + return Delivery(store, signer, base) + + @click.command() @click.option("--host", default=None, help="Host to bind (default A2A_HOST)") @click.option("--port", default=None, type=int, help="Port (default A2A_PORT)") diff --git a/orchestration/relay.py b/orchestration/relay.py index 0cb53d9..6427220 100644 --- a/orchestration/relay.py +++ b/orchestration/relay.py @@ -44,6 +44,7 @@ from google.protobuf import json_format from orchestration.agents import Agent from orchestration.agents import Sequence as AgentSequence +from orchestration.delivery import RECORD, UNASSIGNED, Delivery from orchestration.routing import WORK, route logger = logging.getLogger(__name__) @@ -61,9 +62,16 @@ _PAUSES = {TaskState.TASK_STATE_INPUT_REQUIRED, TaskState.TASK_STATE_AUTH_REQUIR # What every agent of a sequence is sent of the caller's fields. The rest go to the # first agent only, or to the agent whose work names them: `operating_model: {...}`. _CARRIED = ("customer_case", "pre_approved", "request") -# The Orchestration engine's own record that an agent's Execution completed, sent -# with its files to the next agent. -RECORD = "execution.json" +# Where a request names work kept earlier to build on (ADR-0004, phase 2). +SOURCES_FROM = "sources_from" +# How a caller is given what agents delivered (ADR-0004): one document of links to +# what was kept, or each file as the agent sent it. +DELIVER, LINKS, FILES = "deliver", "links", "files" +# The one artifact a caller given links receives. +DOCUMENT = "result.json" +# What a KAB workflow step sends with every call, by which it is known: it takes +# one output per step, so it is given links unless it asks otherwise. +_PLATFORM_STEP = ("workflow_id", "step_number") @dataclass @@ -71,9 +79,15 @@ class Plan: """What one of our tasks runs: the agents, in turn, and the caller's fields.""" steps: tuple[Agent, ...] - fields: dict[str, Any] # the request's metadata, but for `work` + fields: dict[str, Any] # the request's metadata, but for `work` and `deliver` text: str # the request, as the agents are sent it other_parts: list[Part] = field(default_factory=list) # data and files sent with it + work: str = "" # the work the caller named, an agent's or a sequence's + deliver: str = FILES + # Each agent that completed, by its work: its Execution, how it ended, links. + results: dict[str, dict[str, Any]] = field(default_factory=dict) + # What was kept of each, by its work, for the agent after it to build on. + kept: dict[str, dict[str, Any]] = field(default_factory=dict) def metadata(self, step: int, sources: Mapping[str, str] | None) -> dict[str, Any]: """The metadata for the agent at `step`.""" @@ -87,6 +101,8 @@ class Plan: sent.update(own) if sources is not None: sent["sources"] = dict(sources) + if isinstance(sent.get("sources"), Mapping): + sent["sources"] = _as_files(sent["sources"]) return sent @@ -130,7 +146,9 @@ class RelayExecutor(AgentExecutor): clients: AgentClients, heartbeat_seconds: float, sequences: Sequence[AgentSequence] = (), + delivery: Delivery | None = None, ) -> None: + self._delivery = delivery self._agents = list(agents) self._sequences = list(sequences) self._clients = clients @@ -174,7 +192,7 @@ class RelayExecutor(AgentExecutor): ) ) return - fields = {key: value for key, value in options.items() if key != WORK} + fields = {k: v for k, v in options.items() if k not in (WORK, DELIVER)} texts = [part.text for part in parts if part.HasField("text")] # A KAB step sends its fields as metadata and, as the text, all of them # again as one JSON document. The agents are sent the request as their text. @@ -183,9 +201,39 @@ class RelayExecutor(AgentExecutor): fields=fields, text=str(fields["request"]) if fields.get("request") else "\n".join(texts), other_parts=[part for part in parts if not part.HasField("text")], + work=str(options.get(WORK) or decision.agents[0].work), + deliver=_deliver(options), ) + if SOURCES_FROM in fields: + try: + self._build_on(fields) + except ValueError as exc: + await updater.complete( + updater.new_agent_message( + [ + new_text_part(f"Nothing was started: {exc}"), + new_data_part( + {"status": "not_started", "reason": str(exc)} + ), + ] + ) + ) + return await self._run(task_id, plan, 0, self._opening(plan, 0, None), updater) + def _build_on(self, fields: dict[str, Any]) -> None: + """Make the work a request names in `sources_from` its sources, and its + Customer case the request's when it names none.""" + reference = str(fields.pop(SOURCES_FROM) or "") + if self._delivery is None: + raise ValueError("Nothing is kept here to build on.") + if fields.get("sources"): + raise ValueError("Give sources or sources_from, not both.") + record = self._delivery.resolve(reference, fields) + if not fields.get("customer_case") and record["customer_case"] != UNASSIGNED: + fields["customer_case"] = record["customer_case"] + fields["sources"] = self._delivery.sources(record) + async def _resume( self, task_id: str, @@ -257,6 +305,8 @@ class RelayExecutor(AgentExecutor): heartbeat = asyncio.create_task(self._beat(agent, updater)) relayed: set[str] = set() # artifact ids already passed on files: dict[str, str] = {} # what it delivered, by name, for the next agent + guide: list[str] = [] # the file it sent as its document, if any + relay_files = plan.deliver == FILES try: client = await self._clients.for_agent(agent) async for event in client.send_message(SendMessageRequest(message=message)): @@ -264,11 +314,19 @@ class RelayExecutor(AgentExecutor): if event.HasField("task"): self._hold(task_id, plan, step, event.task) for artifact in event.task.artifacts: - await self._artifact(agent, artifact, relayed, files, updater) + await self._artifact( + agent, artifact, relayed, files, guide, updater, relay_files + ) status = event.task.status elif event.HasField("artifact_update"): await self._artifact( - agent, event.artifact_update.artifact, relayed, files, updater + agent, + event.artifact_update.artifact, + relayed, + files, + guide, + updater, + relay_files, ) elif event.HasField("status_update"): status = event.status_update.status @@ -282,11 +340,11 @@ class RelayExecutor(AgentExecutor): return if status is not None and status.state in _ENDS_A_TURN: heartbeat.cancel() - waiting = self._waiting.get(task_id) - following = step + 1 < len(plan.steps) - if status.state == TaskState.TASK_STATE_COMPLETED and following: - await self._next( - task_id, plan, step, status, files, waiting, updater + if status.state == TaskState.TASK_STATE_COMPLETED: + waiting = self._waiting.get(task_id) + self._kept(plan, agent, waiting, status, files, guide) + await self._completed( + task_id, plan, step, status, files, updater ) else: await self._end_turn(task_id, status, updater) @@ -325,57 +383,98 @@ class RelayExecutor(AgentExecutor): finally: heartbeat.cancel() - async def _next( + def _kept( + self, + plan: Plan, + agent: Agent, + waiting: Waiting | None, + status: TaskStatus, + files: Mapping[str, str], + guide: list[str], + ) -> None: + """Keep what the agent delivered and note it in the plan's results.""" + execution_id = waiting.task_id if waiting else "" + ended = _ended(status) + result: dict[str, Any] = {"execution_id": execution_id, "status": ended} + record = ( + self._delivery.keep( + plan.fields, + work=agent.work, + execution_id=execution_id or str(uuid.uuid4()), + status=ended, + request=plan.text, + files=files, + guide=guide[0] if guide else None, + ) + if self._delivery is not None + else None + ) + if record is not None and self._delivery is not None: + plan.kept[agent.work] = record + result.update(self._delivery.links(record)) + else: + result["files"] = {name: None for name in sorted(files)} + plan.results[agent.work] = result + + async def _completed( self, task_id: str, plan: Plan, step: int, status: TaskStatus, files: dict[str, str], - waiting: Waiting | None, updater: TaskUpdater, ) -> None: """The agent at `step` completed: start the next with its files, unless it - ended with warnings, when nothing after it starts.""" + ended with warnings, when nothing after it starts; or, after the last, + finish.""" agent = plan.steps[step] parts = list(status.message.parts) if status.HasField("message") else [] - said = next( - (data for data in get_data_parts(parts) if isinstance(data, Mapping)), {} - ) - if said.get("status", "completed") != "completed": - self._waiting.pop(task_id, None) + ended = _ended(status) + if step + 1 == len(plan.steps): + await self._finish(task_id, plan, parts, ended, updater) + return + if ended != "completed": not_started = [later.work for later in plan.steps[step + 1 :]] - await updater.complete( - updater.new_agent_message( - [ - new_text_part( - f"The {agent.work} agent ended with warnings, so " - f"{', '.join(not_started)} was not started. Settle them " - "and ask again." - ), - new_data_part( - { - "status": "stopped", - "after": agent.work, - "not_started": not_started, - } - ), - *parts, # what the agent itself said - ] - ) + await self._finish( + task_id, + plan, + [ + new_text_part( + f"The {agent.work} agent ended with warnings, so " + f"{', '.join(not_started)} was not started. Settle them " + "and ask again." + ), + new_data_part( + { + "status": "stopped", + "after": agent.work, + "not_started": not_started, + } + ), + *parts, # what the agent itself said + ], + "stopped", + updater, + not_started, ) return - record = { - "status": "completed", - "customer_case": plan.fields.get("customer_case"), - "execution_id": waiting.task_id if waiting else None, - "workflow_request": plan.text, - "work": agent.work, - } - sources = { - **files, - RECORD: json.dumps({k: v for k, v in record.items() if v is not None}), - } + kept = plan.kept.get(agent.work) + if kept is not None and self._delivery is not None: + # What was kept, as a request naming it in sources_from would have it. + sources = self._delivery.sources(kept) + else: + record = { + "status": "completed", + "customer_case": plan.fields.get("customer_case"), + "execution_id": plan.results[agent.work]["execution_id"] or None, + "workflow_request": plan.text, + "work": agent.work, + } + sources = { + **files, + RECORD: json.dumps({k: v for k, v in record.items() if v is not None}), + } following = plan.steps[step + 1] await updater.update_status( TaskState.TASK_STATE_WORKING, @@ -392,6 +491,48 @@ class RelayExecutor(AgentExecutor): task_id, plan, step + 1, self._opening(plan, step + 1, sources), updater ) + async def _finish( + self, + task_id: str, + plan: Plan, + parts: list[Part], + ended: str, + updater: TaskUpdater, + not_started: list[str] | None = None, + ) -> None: + """Complete our task; a caller given links has one document of them.""" + self._waiting.pop(task_id, None) + if plan.deliver == LINKS: + document = self._document(plan, ended, not_started) + await updater.add_artifact( + [new_text_part(json.dumps(document), media_type="application/json")], + name=DOCUMENT, + ) + await updater.complete(updater.new_agent_message(parts)) + + def _document( + self, plan: Plan, ended: str, not_started: list[str] | None + ) -> dict[str, Any]: + """One document of links to what each agent delivered; at its top level, + the last agent's.""" + last = list(plan.results.values())[-1] if plan.results else {} + _, case = Delivery.place(plan.fields) + document: dict[str, Any] = { + "status": ended, + "work": plan.work, + "customer_case": case, + "execution_id": last.get("execution_id"), + "record": last.get("record"), + "guide": last.get("guide"), + "files": last.get("files", {}), + "latest": last.get("latest"), + "case": last.get("case"), + "results": plan.results, + } + if not_started: + document["not_started"] = not_started + return document + def _hold(self, task_id: str, plan: Plan, step: int, task: Task) -> None: if task.id: self._waiting[task_id] = Waiting(plan, step, task.id, task.context_id) @@ -422,7 +563,9 @@ class RelayExecutor(AgentExecutor): artifact: Artifact, relayed: set[str], files: dict[str, str], + guide: list[str], updater: TaskUpdater, + relay: bool, ) -> None: if artifact.artifact_id in relayed: return @@ -430,6 +573,13 @@ class RelayExecutor(AgentExecutor): text = "".join(part.text for part in artifact.parts if part.HasField("text")) if artifact.name and text: files[artifact.name] = text + # The Execution engine sends its final Deliverable as markdown. + if not guide and any( + part.media_type == "text/markdown" for part in artifact.parts + ): + guide.append(artifact.name) + if not relay: + return metadata = ( json_format.MessageToDict(artifact.metadata) if artifact.HasField("metadata") @@ -453,6 +603,34 @@ class RelayExecutor(AgentExecutor): ) +def _deliver(options: Mapping[str, Any]) -> str: + """How the caller is given what agents delivered: as it asked, or links for a + KAB workflow step, which takes one output per step, and files for anyone else.""" + asked = options.get(DELIVER) + if asked in (LINKS, FILES): + return str(asked) + return LINKS if all(key in options for key in _PLATFORM_STEP) else FILES + + +def _ended(status: TaskStatus) -> str: + """How an agent says its work ended, from its last word: `completed` unless it + says otherwise, such as `completed_with_warnings`.""" + parts = list(status.message.parts) if status.HasField("message") else [] + said = next( + (data for data in get_data_parts(parts) if isinstance(data, Mapping)), {} + ) + return str(said.get("status", "completed")) + + +def _as_files(sources: Mapping[str, Any]) -> dict[str, str]: + """Sources as files by name: what was parsed on the way, as a platform step + does with an earlier step's output, is sent on as JSON text.""" + return { + str(name): content if isinstance(content, str) else json.dumps(content) + for name, content in sources.items() + } + + def _options(context: RequestContext) -> dict[str, Any]: """The request's metadata: the message's, then the request's own.""" options: dict[str, Any] = {} diff --git a/orchestration/serving.py b/orchestration/serving.py new file mode 100644 index 0000000..9777a97 --- /dev/null +++ b/orchestration/serving.py @@ -0,0 +1,76 @@ +"""Serving what was kept at signed links (ADR-0004). Every route checks the link's +signature against the path it was made for; anything else is refused.""" + +from __future__ import annotations + +from starlette.requests import Request +from starlette.responses import JSONResponse, Response +from starlette.routing import Route + +from orchestration.delivery import Delivery + + +def kept_routes(delivery: Delivery) -> list[Route]: + store = delivery.store + + def signed(request: Request) -> bool: + # The path as the link carried it: a name is signed as it was quoted. + raw = request.scope.get("raw_path") + path = raw.decode("latin-1") if isinstance(raw, bytes) else request.url.path + return delivery.signer.verify( + path, request.query_params.get("exp"), request.query_params.get("sig") + ) + + def refused() -> JSONResponse: + return JSONResponse({"error": "This link is not valid."}, status_code=403) + + def missing() -> JSONResponse: + return JSONResponse({"error": "Nothing is kept here."}, status_code=404) + + async def case(request: Request) -> Response: + if not signed(request): + return refused() + tenant, name = request.path_params["tenant"], request.path_params["case"] + return JSONResponse( + { + "tenant": tenant, + "customer_case": name, + "executions": [delivery.shown(r) for r in store.case(tenant, name)], + } + ) + + async def execution(request: Request) -> Response: + if not signed(request): + return refused() + p = request.path_params + record = store.record(p["tenant"], p["case"], p["execution_id"]) + return JSONResponse(delivery.shown(record)) if record else missing() + + async def file(request: Request) -> Response: + if not signed(request): + return refused() + p = request.path_params + found = store.file(p["tenant"], p["case"], p["execution_id"], p["name"]) + if found is None: + return missing() + content, media_type = found + return Response(content, media_type=media_type) + + async def latest(request: Request) -> Response: + if not signed(request): + return refused() + p = request.path_params + record = store.latest(p["tenant"], p["case"], p["work"]) + return JSONResponse(delivery.shown(record)) if record else missing() + + case_path = "/tenants/{tenant}/cases/{case}" + return [ + Route(case_path, case, methods=["GET"]), + Route(f"{case_path}/executions/{{execution_id}}", execution, methods=["GET"]), + Route( + f"{case_path}/executions/{{execution_id}}/files/{{name:path}}", + file, + methods=["GET"], + ), + Route(f"{case_path}/latest/{{work}}", latest, methods=["GET"]), + ] diff --git a/orchestration/store.py b/orchestration/store.py new file mode 100644 index 0000000..1e6ea01 --- /dev/null +++ b/orchestration/store.py @@ -0,0 +1,311 @@ +"""What agents delivered, kept by tenant, Customer case and Execution (ADR-0004). + + tenants//cases//executions//record.json + /files/ + +An Execution is written once. Its files are what the agent delivered, by the names +it gave them; its record says what it was. The same layout is kept in a folder or +under a prefix in an S3-compatible bucket. +""" + +from __future__ import annotations + +import datetime as dt +import json +import re +import time +from collections.abc import Mapping +from pathlib import Path, PurePosixPath +from typing import TYPE_CHECKING, Any, Protocol + +if TYPE_CHECKING: + from orchestration.config import OrchestrationSettings + +_SLUG = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$") +_MEDIA_TYPES = { + ".md": "text/markdown; charset=utf-8", + ".json": "application/json", + ".yaml": "application/yaml", + ".yml": "application/yaml", + ".html": "text/html; charset=utf-8", + ".csv": "text/csv; charset=utf-8", +} +_TEXT = "text/plain; charset=utf-8" + + +def slug(value: str, what: str) -> str: + """`value`, when it can name a folder: letters, digits, dot, dash, underscore.""" + if not _SLUG.match(value or "") or value in {".", ".."}: + raise ValueError(f"{what} {value!r} must be letters, digits, '.', '-' or '_'") + return value + + +def file_name(name: str) -> PurePosixPath: + """A delivered file's name as a path inside its Execution, or ValueError.""" + path = PurePosixPath(name) + if ( + not name + or "\\" in name + or path.is_absolute() + or any(part in {"", ".", ".."} for part in name.split("/")) + ): + raise ValueError(f"{name!r} is not a file name inside an Execution") + return path + + +def media_type(name: str) -> str: + return _MEDIA_TYPES.get(PurePosixPath(name).suffix.lower(), _TEXT) + + +class Blobs(Protocol): + """Where a store's keys are kept: a folder, or a bucket.""" + + def put(self, key: str, data: bytes, content_type: str) -> None: ... + def get(self, key: str) -> bytes | None: ... + def exists(self, key: str) -> bool: ... + def keys(self, prefix: str) -> list[str]: ... + + +class VolumeBlobs: + """Keys as files under a folder.""" + + def __init__(self, root: str | Path) -> None: + self.root = Path(root) + + def _path(self, key: str) -> Path: + path = (self.root / key).resolve() + if self.root.resolve() not in path.parents: + raise ValueError(f"{key!r} is outside the store") + return path + + def put(self, key: str, data: bytes, content_type: str) -> None: + path = self._path(key) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(data) + + def get(self, key: str) -> bytes | None: + path = self._path(key) + return path.read_bytes() if path.is_file() else None + + def exists(self, key: str) -> bool: + return self._path(key).is_file() + + def keys(self, prefix: str) -> list[str]: + folder = self.root / prefix + if not folder.is_dir(): + return [] + return [ + path.relative_to(self.root).as_posix() + for path in folder.rglob("*") + if path.is_file() + ] + + +class S3Blobs: + """Keys as objects under a prefix in an S3-compatible bucket (MinIO locally).""" + + def __init__( + self, client: Any, bucket: str, prefix: str = "", create: bool = False + ) -> None: + self._client = client + self.bucket = bucket + self.prefix = prefix.lstrip("/") + if create: + self._ensure_bucket() + + def _ensure_bucket(self) -> None: + from botocore.exceptions import ClientError + + try: + self._client.head_bucket(Bucket=self.bucket) + except ClientError: + self._client.create_bucket(Bucket=self.bucket) + + def put(self, key: str, data: bytes, content_type: str) -> None: + self._client.put_object( + Bucket=self.bucket, + Key=self.prefix + key, + Body=data, + ContentType=content_type, + ) + + def get(self, key: str) -> bytes | None: + from botocore.exceptions import ClientError + + try: + response = self._client.get_object( + Bucket=self.bucket, Key=self.prefix + key + ) + except ClientError as exc: + if exc.response.get("Error", {}).get("Code") in {"NoSuchKey", "404"}: + return None + raise + data: bytes = response["Body"].read() + return data + + def exists(self, key: str) -> bool: + from botocore.exceptions import ClientError + + try: + self._client.head_object(Bucket=self.bucket, Key=self.prefix + key) + except ClientError as exc: + if exc.response.get("Error", {}).get("Code") in {"NoSuchKey", "404"}: + return False + raise + return True + + def keys(self, prefix: str) -> list[str]: + found: list[str] = [] + pages = self._client.get_paginator("list_objects_v2").paginate( + Bucket=self.bucket, Prefix=self.prefix + prefix + ) + for page in pages: + for item in page.get("Contents", []): + found.append(str(item["Key"])[len(self.prefix) :]) + return found + + +class Store: + def __init__(self, blobs: Blobs | str | Path) -> None: + self.blobs: Blobs = ( + VolumeBlobs(blobs) if isinstance(blobs, (str, Path)) else blobs + ) + + @staticmethod + def _case(tenant: str, case: str) -> str: + return f"tenants/{slug(tenant, 'Tenant')}/cases/{slug(case, 'Case')}" + + def _execution(self, tenant: str, case: str, execution_id: str) -> str: + return ( + f"{self._case(tenant, case)}/executions/{slug(execution_id, 'Execution')}" + ) + + def keep( + self, + *, + tenant: str, + case: str, + execution_id: str, + work: str, + status: str, + request: str, + files: Mapping[str, str], + guide: str | None = None, + project_id: str | None = None, + ) -> dict[str, Any]: + """Keep one Execution's files and record it. Written once: an Execution + already kept is refused.""" + folder = self._execution(tenant, case, execution_id) + paths = {name: file_name(name) for name in files} + if self.blobs.exists(f"{folder}/record.json"): + raise ValueError(f"Execution {execution_id} is already kept") + for name, path in paths.items(): + self.blobs.put( + f"{folder}/files/{path}", files[name].encode("utf-8"), media_type(name) + ) + record = { + "tenant": tenant, + "customer_case": case, + "execution_id": execution_id, + "work": work, + "status": status, + "request": request, + "project_id": project_id, + "guide": guide if guide in files else None, + "files": sorted(files), + "kept_at": dt.datetime.now(dt.UTC).isoformat(), + "kept_ns": time.time_ns(), + } + # The record last: an Execution is kept once its record is. + self.blobs.put( + f"{folder}/record.json", + json.dumps(record, indent=2).encode("utf-8"), + "application/json", + ) + return record + + def record( + self, tenant: str, case: str, execution_id: str + ) -> dict[str, Any] | None: + try: + data = self.blobs.get( + f"{self._execution(tenant, case, execution_id)}/record.json" + ) + except ValueError: + return None + if data is None: + return None + record: dict[str, Any] = json.loads(data) + return record + + def file( + self, tenant: str, case: str, execution_id: str, name: str + ) -> tuple[bytes, str] | None: + """A kept file's content and media type, or None.""" + try: + key = ( + f"{self._execution(tenant, case, execution_id)}/files/{file_name(name)}" + ) + data = self.blobs.get(key) + except ValueError: + return None + return None if data is None else (data, media_type(name)) + + def files(self, tenant: str, case: str, execution_id: str) -> dict[str, str]: + """Every file kept for an Execution, by name.""" + record = self.record(tenant, case, execution_id) or {} + found: dict[str, str] = {} + for name in record.get("files", []): + kept = self.file(tenant, case, execution_id, name) + if kept is not None: + found[name] = kept[0].decode("utf-8") + return found + + def case(self, tenant: str, case: str) -> list[dict[str, Any]]: + """The Customer case's Executions, newest first.""" + try: + prefix = f"{self._case(tenant, case)}/executions/" + except ValueError: + return [] + records = [] + for key in self.blobs.keys(prefix): + # /record.json; a delivered file of that name is under files/. + parts = key[len(prefix) :].split("/") + if len(parts) == 2 and parts[1] == "record.json": + data = self.blobs.get(key) + if data is not None: + records.append(json.loads(data)) + return sorted(records, key=lambda r: r.get("kept_ns", 0), reverse=True) + + def latest(self, tenant: str, case: str, work: str) -> dict[str, Any] | None: + """The newest Execution of `work` in the Customer case that completed.""" + return next( + ( + record + for record in self.case(tenant, case) + if record["work"] == work and record["status"] == "completed" + ), + None, + ) + + +def store_from_settings(settings: OrchestrationSettings) -> Store | None: + """The store the settings name; None when nothing is to be kept.""" + if settings.STORE_BACKEND == "s3": + import boto3 + + if not settings.S3_BUCKET: + raise ValueError("STORE_BACKEND=s3 needs S3_BUCKET") + client = boto3.client( + "s3", + endpoint_url=settings.S3_ENDPOINT_URL or None, + region_name=settings.S3_REGION, + ) + return Store( + S3Blobs(client, settings.S3_BUCKET, settings.S3_PREFIX, create=True) + ) + if settings.STORE_BACKEND != "volume": + raise ValueError( + f"STORE_BACKEND must be volume or s3, not {settings.STORE_BACKEND!r}" + ) + return Store(VolumeBlobs(settings.STORE_DIR)) if settings.STORE_DIR else None diff --git a/pyproject.toml b/pyproject.toml index bb8c7e1..1c6e2f9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -30,7 +30,11 @@ dependencies = [ ] [project.optional-dependencies] +# The Orchestration engine's store on S3-compatible object storage (ADR-0004). +s3 = ["boto3>=1.34"] dev = [ + "boto3>=1.34", + "moto[s3]>=5.0", "pytest>=8.0.0", "pytest-asyncio>=0.23.0", "ruff>=0.6.0", @@ -59,3 +63,8 @@ cache-dir = ".cache/ruff" [tool.mypy] cache_dir = ".cache/mypy" + +[[tool.mypy.overrides]] +# The AWS SDK and its test double ship without type information. +module = ["boto3", "boto3.*", "botocore", "botocore.*", "moto", "moto.*"] +ignore_missing_imports = true diff --git a/tests/orchestration/test_document.py b/tests/orchestration/test_document.py deleted file mode 100644 index f27b661..0000000 --- a/tests/orchestration/test_document.py +++ /dev/null @@ -1,162 +0,0 @@ -"""Delivering one JSON document to a caller that takes one output per step, as a KAB -workflow step does, instead of each agent's files one by one.""" - -from __future__ import annotations - -import json -from typing import Any - -import httpx - -from orchestration.main import create_app -from tests.orchestration.test_sequence import ( - ARCHITECTURE, - BOTH, - OPERATIONS, - SETTINGS, - ScriptedAgents, - call, - finished, - struct, -) - -# What a KAB workflow step sends with every call. -PLATFORM = {"workflow_id": "wf-1", "step_number": 1, "task_id": "t-1"} -GRAPH = '{"elements": []}' - - -def document(task: dict) -> dict[str, Any]: - (artifact,) = task["artifacts"] - assert artifact["name"] == "result.json" - return json.loads("".join(part["text"] for part in artifact["parts"])) - - -async def test_a_platform_step_is_given_one_document_of_the_agents_files(): - agents = ScriptedAgents( - {"architecture": [finished("a1", {"graph.json": GRAPH, "guide.md": "# Guide"})]} - ) - task, _ = await call( - agents, - [{"text": "A landing zone"}], - metadata={**PLATFORM, "work": "architecture", "customer_case": "acme"}, - ) - - assert task["status"]["state"] == "TASK_STATE_COMPLETED" - result = document(task) - assert result["status"] == "completed" - assert result["work"] == "architecture" - assert result["customer_case"] == "acme" - # Every file as the agent delivered it, ready to be the next step's sources. - assert result["files"]["graph.json"] == GRAPH - assert result["files"]["guide.md"] == "# Guide" - assert json.loads(result["files"]["execution.json"]) == { - "status": "completed", - "customer_case": "acme", - "execution_id": "a1", - "workflow_request": "A landing zone", - "work": "architecture", - } - assert result["results"]["architecture"]["status"] == "completed" - assert result["results"]["architecture"]["execution_id"] == "a1" - assert set(result["results"]["architecture"]["files"]) == {"graph.json", "guide.md"} - - -async def test_a_caller_that_asks_for_files_gets_them_one_by_one(): - agents = ScriptedAgents({"architecture": [finished("a1", {"graph.json": GRAPH})]}) - task, _ = await call( - agents, - [{"text": "A landing zone"}], - metadata={**PLATFORM, "work": "architecture", "deliver": "files"}, - ) - - assert [a["name"] for a in task["artifacts"]] == ["graph.json"] - (sent,) = agents.received["architecture"] - assert "deliver" not in struct(sent) - - -async def test_any_caller_can_ask_for_the_document(): - agents = ScriptedAgents({"architecture": [finished("a1", {"graph.json": GRAPH})]}) - task, _ = await call( - agents, - [{"text": "A landing zone"}], - metadata={"work": "architecture", "deliver": "document"}, - ) - - assert document(task)["files"]["graph.json"] == GRAPH - - -async def test_a_sequence_is_one_document_with_each_agents_results(): - agents = ScriptedAgents( - { - "architecture": [finished("a1", {"graph.json": GRAPH})], - "operations": [finished("o1", {"model.json": "{}"})], - } - ) - task, _ = await call( - agents, - [{"text": "A landing zone"}], - metadata={**PLATFORM, "work": "architecture_and_operations"}, - ) - - result = document(task) - assert result["status"] == "completed" - assert set(result["results"]) == {"architecture", "operations"} - assert set(result["results"]["architecture"]["files"]) == {"graph.json"} - # The top-level files are the last agent's. - assert set(result["files"]) == {"model.json", "execution.json"} - assert json.loads(result["files"]["execution.json"])["work"] == "operations" - - -async def test_a_sequence_stopped_by_warnings_still_delivers_what_was_done(): - agents = ScriptedAgents( - { - "architecture": [ - finished("a1", {"graph.json": GRAPH}, status="completed_with_warnings") - ], - "operations": [], - } - ) - task, _ = await call( - agents, - [{"text": "A landing zone"}], - metadata={**PLATFORM, "work": "architecture_and_operations"}, - ) - - result = document(task) - assert result["status"] == "stopped" - assert result["not_started"] == ["operations"] - assert result["results"]["architecture"]["status"] == "completed_with_warnings" - assert result["files"]["graph.json"] == GRAPH - - -async def test_sources_given_as_data_reach_the_agent_as_files(): - # A platform step maps an earlier step's output into `sources`; whatever was - # parsed on the way is sent on as text, the form an agent reads files in. - agents = ScriptedAgents({"operations": [finished("o1", {"model.json": "{}"})]}) - await call( - agents, - [{"text": "Operate it"}], - metadata={ - "work": "operations", - "sources": {"graph.json": {"elements": []}, "guide.md": "# Guide"}, - }, - ) - - (sent,) = agents.received["operations"] - sources = struct(sent)["sources"] - assert json.loads(sources["graph.json"]) == {"elements": []} - assert sources["guide.md"] == "# Guide" - - -async def test_the_card_declares_the_document_and_the_choice_of_delivery(): - app = create_app([ARCHITECTURE, OPERATIONS], SETTINGS, sequences=[BOTH]) - async with httpx.AsyncClient( - transport=httpx.ASGITransport(app=app), base_url="http://orchestration" - ) as caller: - skill = (await caller.get("/.well-known/agent-card.json")).json()["skills"][0] - - inputs = skill["inputSchema"]["properties"] - assert inputs["deliver"]["enum"] == ["document", "files"] - assert inputs["sources"]["type"] == "object" - outputs = skill["outputSchema"]["properties"] - assert {"status", "work", "customer_case", "files", "results"} <= set(outputs) diff --git a/tests/orchestration/test_links.py b/tests/orchestration/test_links.py new file mode 100644 index 0000000..1b5dc55 --- /dev/null +++ b/tests/orchestration/test_links.py @@ -0,0 +1,344 @@ +"""Delivered files kept and served at signed links (ADR-0004); a caller that takes +one output per step, as a KAB workflow step does, is given one document of links.""" + +from __future__ import annotations + +import json +import uuid +from typing import Any +from urllib.parse import urlsplit + +import httpx + +from orchestration.links import Signer +from orchestration.main import create_app +from orchestration.store import Store +from tests.orchestration.test_sequence import ( + ARCHITECTURE, + BOTH, + OPERATIONS, + SETTINGS, + ScriptedAgents, + finished, + struct, +) + +BASE = "http://orchestration" +# What a KAB workflow step sends with every call. +PLATFORM = { + "workflow_id": "wf-1", + "step_number": 1, + "tenant_id": "t1", + "project_id": "p1", +} +GRAPH = '{"elements": []}' + + +class Harness: + """The Orchestration engine in front of scripted agents, with a store and the + caller's client, both in this process.""" + + def __init__(self, tmp_path: Any, agents: ScriptedAgents) -> None: + self.agents = agents + self.store = Store(tmp_path / "store") + self.app = create_app( + [ARCHITECTURE, OPERATIONS], + SETTINGS, + clients=agents, # type: ignore[arg-type] + sequences=[BOTH], + store=self.store, + signer=Signer("secret", ttl_seconds=3600), + public_url=BASE, + ) + + def client(self) -> httpx.AsyncClient: + return httpx.AsyncClient( + transport=httpx.ASGITransport(app=self.app), base_url=BASE + ) + + async def send(self, text: str, metadata: dict[str, Any]) -> dict[str, Any]: + message = { + "messageId": str(uuid.uuid4()), + "role": "ROLE_USER", + "parts": [{"text": text}], + "metadata": metadata, + } + async with self.client() as caller: + response = await caller.post( + "/a2a", + json={ + "jsonrpc": "2.0", + "id": 1, + "method": "SendMessage", + "params": {"message": message}, + }, + headers={"A2A-Version": "1.0"}, + ) + body = response.json() + assert "error" not in body, body + task: dict[str, Any] = body["result"]["task"] + return task + + async def get(self, link: str) -> httpx.Response: + url = urlsplit(link) + async with self.client() as caller: + return await caller.get( + f"{url.path}?{url.query}" if url.query else url.path + ) + + +def document(task: dict[str, Any]) -> dict[str, Any]: + (artifact,) = task["artifacts"] + assert artifact["name"] == "result.json" + loaded: dict[str, Any] = json.loads("".join(p["text"] for p in artifact["parts"])) + return loaded + + +async def test_a_platform_step_is_given_links_to_what_was_kept(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents( + { + "architecture": [ + finished("a1", {"graph.json": GRAPH, "guide.md": "# Guide"}) + ] + } + ), + ) + + task = await harness.send( + "A landing zone", {**PLATFORM, "work": "architecture", "customer_case": "acme"} + ) + + assert task["status"]["state"] == "TASK_STATE_COMPLETED" + result = document(task) + assert result["status"] == "completed" + assert result["work"] == "architecture" + assert result["customer_case"] == "acme" + assert result["execution_id"] == "a1" + assert set(result["files"]) == {"graph.json", "guide.md"} + assert result["record"].startswith(f"{BASE}/tenants/t1/cases/acme/executions/a1?") + # Each link serves what the agent delivered, with its media type. + guide = await harness.get(result["files"]["guide.md"]) + assert guide.status_code == 200 + assert guide.text == "# Guide" + assert guide.headers["content-type"].startswith("text/markdown") + assert (await harness.get(result["files"]["graph.json"])).json() == {"elements": []} + # The record names the platform's project and links to every file. + record = (await harness.get(result["record"])).json() + assert record["project_id"] == "p1" + assert record["request"] == "A landing zone" + assert set(record["links"]["files"]) == {"graph.json", "guide.md"} + + +async def test_the_guide_is_the_markdown_the_agent_sent_as_its_document(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents( + { + "architecture": [ + finished( + "a1", + {"graph.json": GRAPH, "notes.md": "n", "guide.md": "# Guide"}, + guide="guide.md", + ) + ] + } + ), + ) + + result = document( + await harness.send("A landing zone", {**PLATFORM, "work": "architecture"}) + ) + + assert result["guide"] == result["files"]["guide.md"] + + +async def test_a_link_that_is_not_signed_or_was_changed_is_refused(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents({"architecture": [finished("a1", {"guide.md": "# G"})]}), + ) + result = document( + await harness.send( + "A landing zone", + {**PLATFORM, "work": "architecture", "customer_case": "acme"}, + ) + ) + link = result["files"]["guide.md"] + + assert (await harness.get(link.split("?")[0])).status_code == 403 + assert (await harness.get(link.replace("guide.md", "other.md"))).status_code == 403 + assert (await harness.get(link.replace("/t1/", "/t2/"))).status_code == 403 + + +async def test_a_caller_that_asks_for_files_gets_them_and_they_are_kept(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents({"architecture": [finished("a1", {"graph.json": GRAPH})]}), + ) + + task = await harness.send( + "A landing zone", + { + **PLATFORM, + "work": "architecture", + "customer_case": "acme", + "deliver": "files", + }, + ) + + assert [a["name"] for a in task["artifacts"]] == ["graph.json"] + assert harness.store.record("t1", "acme", "a1")["files"] == ["graph.json"] + (sent,) = harness.agents.received["architecture"] + assert "deliver" not in struct(sent) + + +async def test_without_a_tenant_or_case_what_is_kept_is_still_found(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents({"architecture": [finished("a1", {"graph.json": GRAPH})]}), + ) + + result = document( + await harness.send( + "A landing zone", {"work": "architecture", "deliver": "links"} + ) + ) + + assert "/tenants/local/cases/unassigned/executions/a1" in result["record"] + assert (await harness.get(result["files"]["graph.json"])).status_code == 200 + + +async def test_a_sequence_is_one_document_with_each_agents_links(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents( + { + "architecture": [finished("a1", {"graph.json": GRAPH})], + "operations": [finished("o1", {"model.json": "{}"})], + } + ), + ) + + result = document( + await harness.send( + "A landing zone", + { + **PLATFORM, + "work": "architecture_and_operations", + "customer_case": "acme", + }, + ) + ) + + assert result["status"] == "completed" + assert result["work"] == "architecture_and_operations" + assert set(result["results"]) == {"architecture", "operations"} + assert set(result["results"]["architecture"]["files"]) == {"graph.json"} + # The top level is the last agent's. + assert result["execution_id"] == "o1" + assert set(result["files"]) == {"model.json"} + assert [r["work"] for r in harness.store.case("t1", "acme")] == [ + "operations", + "architecture", + ] + + +async def test_a_sequence_stopped_by_warnings_links_what_was_done(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents( + { + "architecture": [ + finished( + "a1", {"graph.json": GRAPH}, status="completed_with_warnings" + ) + ], + "operations": [], + } + ), + ) + + result = document( + await harness.send( + "A landing zone", {**PLATFORM, "work": "architecture_and_operations"} + ) + ) + + assert result["status"] == "stopped" + assert result["not_started"] == ["operations"] + assert result["results"]["architecture"]["status"] == "completed_with_warnings" + assert set(result["files"]) == {"graph.json"} + + +async def test_latest_is_the_newest_completed_execution_of_the_work(tmp_path): + harness = Harness( + tmp_path, + ScriptedAgents( + { + "architecture": [ + finished("a1", {"graph.json": "1"}), + finished("a2", {"graph.json": "2"}), + ] + } + ), + ) + first = document( + await harness.send( + "v1", {**PLATFORM, "work": "architecture", "customer_case": "acme"} + ) + ) + await harness.send( + "v2", {**PLATFORM, "work": "architecture", "customer_case": "acme"} + ) + + latest = (await harness.get(first["latest"])).json() + + assert latest["execution_id"] == "a2" + assert (await harness.get(latest["links"]["files"]["graph.json"])).text == "2" + case = (await harness.get(first["case"])).json() + assert [r["execution_id"] for r in case["executions"]] == ["a2", "a1"] + + +async def test_sources_given_as_data_reach_the_agent_as_files(tmp_path): + # A platform step maps an earlier step's output into `sources`; whatever was + # parsed on the way is sent on as text, the form an agent reads files in. + harness = Harness( + tmp_path, ScriptedAgents({"operations": [finished("o1", {"model.json": "{}"})]}) + ) + + await harness.send( + "Operate it", + { + "work": "operations", + "sources": {"graph.json": {"elements": []}, "g.md": "# G"}, + }, + ) + + (sent,) = harness.agents.received["operations"] + sources = struct(sent)["sources"] + assert json.loads(sources["graph.json"]) == {"elements": []} + assert sources["g.md"] == "# G" + + +async def test_the_card_declares_the_links_document_and_the_choice_of_delivery( + tmp_path, +): + harness = Harness(tmp_path, ScriptedAgents({})) + async with harness.client() as caller: + skill = (await caller.get("/.well-known/agent-card.json")).json()["skills"][0] + + inputs = skill["inputSchema"]["properties"] + assert inputs["deliver"]["enum"] == ["links", "files"] + assert inputs["sources"]["type"] == "object" + outputs = set(skill["outputSchema"]["properties"]) + assert { + "status", + "work", + "customer_case", + "record", + "guide", + "files", + "results", + } <= outputs diff --git a/tests/orchestration/test_sequence.py b/tests/orchestration/test_sequence.py index 91c5168..bbe8a67 100644 --- a/tests/orchestration/test_sequence.py +++ b/tests/orchestration/test_sequence.py @@ -38,8 +38,13 @@ SETTINGS = OrchestrationSettings(_env_file=None, HEARTBEAT_SECONDS=3600) # type def finished( - task_id: str, files: dict[str, str], status: str = "completed" + task_id: str, + files: dict[str, str], + status: str = "completed", + guide: str | None = None, ) -> StreamResponse: + """A completed task with `files`; `guide` is sent as markdown, as the Execution + engine sends its final Deliverable.""" message = Message( message_id=str(uuid.uuid4()), role=Role.ROLE_AGENT, @@ -57,7 +62,11 @@ def finished( Artifact( artifact_id=f"{task_id}-{name}", name=name, - parts=[new_text_part(text)], + parts=[ + new_text_part(text, media_type="text/markdown") + if name == guide + else new_text_part(text) + ], ) for name, text in files.items() ], diff --git a/tests/orchestration/test_sources_from.py b/tests/orchestration/test_sources_from.py new file mode 100644 index 0000000..1647f30 --- /dev/null +++ b/tests/orchestration/test_sources_from.py @@ -0,0 +1,258 @@ +"""A request names work kept earlier instead of carrying its files (ADR-0004, +phase 2): `sources_from` is a link from an earlier document, `latest/`, or +an Execution's id in the request's Customer case.""" + +from __future__ import annotations + +import json +from typing import Any + +from tests.orchestration.test_links import PLATFORM, Harness, document +from tests.orchestration.test_sequence import ScriptedAgents, finished, struct + +GRAPH = '{"elements": []}' + + +def designed(**more: Any) -> ScriptedAgents: + """An architecture agent that has designed once, and an operations agent ready.""" + turns: dict[str, list[Any]] = { + "architecture": [finished("a1", {"graph.json": GRAPH, "guide.md": "# Guide"})], + "operations": [finished("o1", {"model.json": "{}"})], + } + turns.update(more) + return ScriptedAgents(turns) + + +async def design(harness: Harness, case: str = "acme") -> dict[str, Any]: + return document( + await harness.send( + "A landing zone", + {**PLATFORM, "work": "architecture", "customer_case": case}, + ) + ) + + +def not_started(task: dict[str, Any]) -> str: + assert task["status"]["state"] == "TASK_STATE_COMPLETED" + parts = task["status"]["message"]["parts"] + data = next(p["data"] for p in parts if "data" in p) + assert data["status"] == "not_started" + return " ".join(p.get("text", "") for p in parts) + + +def sent_sources(harness: Harness) -> dict[str, Any]: + (message,) = harness.agents.received["operations"] + metadata = struct(message) + assert "sources_from" not in metadata + return metadata + + +async def test_the_latest_work_of_the_case_is_the_agents_sources(tmp_path): + harness = Harness(tmp_path, designed()) + await design(harness) + + await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "acme", + "sources_from": "latest/architecture", + }, + ) + + sent = sent_sources(harness) + assert sent["sources"]["graph.json"] == GRAPH + assert sent["sources"]["guide.md"] == "# Guide" + assert json.loads(sent["sources"]["execution.json"]) == { + "status": "completed", + "customer_case": "acme", + "execution_id": "a1", + "workflow_request": "A landing zone", + "work": "architecture", + } + + +async def test_a_link_from_an_earlier_document_names_the_work_and_its_case(tmp_path): + # A platform step maps an earlier step's `record` into `sources_from`. + harness = Harness(tmp_path, designed()) + earlier = await design(harness) + + await harness.send( + "Operate it", + {**PLATFORM, "work": "operations", "sources_from": earlier["record"]}, + ) + + sent = sent_sources(harness) + assert sent["customer_case"] == "acme" + assert sent["sources"]["graph.json"] == GRAPH + + +async def test_a_latest_link_is_followed_to_the_newest_completed_work(tmp_path): + harness = Harness( + tmp_path, + designed( + architecture=[ + finished("a1", {"graph.json": "1"}), + finished("a2", {"graph.json": "2"}), + ] + ), + ) + first = await design(harness) + await design(harness) + + await harness.send( + "Operate it", + {**PLATFORM, "work": "operations", "sources_from": first["latest"]}, + ) + + assert sent_sources(harness)["sources"]["graph.json"] == "2" + + +async def test_an_execution_of_the_case_is_named_by_its_id(tmp_path): + harness = Harness(tmp_path, designed()) + await design(harness) + + await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "acme", + "sources_from": "a1", + }, + ) + + assert sent_sources(harness)["sources"]["graph.json"] == GRAPH + + +async def test_a_link_that_was_changed_starts_nothing(tmp_path): + harness = Harness(tmp_path, designed()) + earlier = await design(harness) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "sources_from": earlier["record"].replace("/a1?", "/a2?"), + }, + ) + + assert "not valid" in not_started(task) + assert harness.agents.received["operations"] == [] + + +async def test_a_link_of_another_tenant_starts_nothing(tmp_path): + harness = Harness(tmp_path, designed()) + earlier = await design(harness) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "tenant_id": "t2", + "work": "operations", + "sources_from": earlier["record"], + }, + ) + + assert "tenant" in not_started(task) + assert harness.agents.received["operations"] == [] + + +async def test_work_of_another_customer_case_starts_nothing(tmp_path): + harness = Harness(tmp_path, designed()) + earlier = await design(harness) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "other", + "sources_from": earlier["record"], + }, + ) + + assert "acme" in not_started(task) + + +async def test_nothing_kept_to_build_on_starts_nothing(tmp_path): + harness = Harness(tmp_path, designed()) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "acme", + "sources_from": "latest/architecture", + }, + ) + + assert "architecture" in not_started(task) + assert harness.agents.received["operations"] == [] + + +async def test_work_that_ended_with_warnings_is_not_built_on(tmp_path): + harness = Harness( + tmp_path, + designed( + architecture=[ + finished("a1", {"graph.json": GRAPH}, status="completed_with_warnings") + ] + ), + ) + await design(harness) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "acme", + "sources_from": "a1", + }, + ) + + assert "completed_with_warnings" in not_started(task) + + +async def test_sources_and_sources_from_together_start_nothing(tmp_path): + harness = Harness(tmp_path, designed()) + await design(harness) + + task = await harness.send( + "Operate it", + { + **PLATFORM, + "work": "operations", + "customer_case": "acme", + "sources_from": "a1", + "sources": {"x.json": "{}"}, + }, + ) + + assert "sources" in not_started(task) + + +async def test_a_sequence_gives_the_next_agent_what_was_kept(tmp_path): + harness = Harness(tmp_path, designed()) + + await harness.send( + "A landing zone", + {**PLATFORM, "work": "architecture_and_operations", "customer_case": "acme"}, + ) + + sources = sent_sources(harness)["sources"] + assert sources["graph.json"] == GRAPH + assert json.loads(sources["execution.json"])["execution_id"] == "a1" + + +async def test_the_card_offers_sources_from(tmp_path): + harness = Harness(tmp_path, ScriptedAgents({})) + async with harness.client() as caller: + skill = (await caller.get("/.well-known/agent-card.json")).json()["skills"][0] + + assert skill["inputSchema"]["properties"]["sources_from"]["type"] == "string" diff --git a/tests/orchestration/test_store.py b/tests/orchestration/test_store.py new file mode 100644 index 0000000..e4a313f --- /dev/null +++ b/tests/orchestration/test_store.py @@ -0,0 +1,193 @@ +"""The store of what agents delivered, and the signed links to it (ADR-0004).""" + +from __future__ import annotations + +from collections.abc import Iterator +from typing import Any + +import boto3 +import pytest +from moto import mock_aws + +from orchestration.config import OrchestrationSettings +from orchestration.links import Signer +from orchestration.store import S3Blobs, Store, VolumeBlobs, store_from_settings + + +@pytest.fixture(params=["volume", "s3"]) +def make_store(request: Any, tmp_path: Any) -> Iterator[Any]: + """A store on each kind of storage: a folder, and S3 (moto in this process).""" + if request.param == "volume": + yield lambda: Store(VolumeBlobs(tmp_path)) + return + with mock_aws(): + client = boto3.client("s3", region_name="us-east-1") + yield lambda: Store( + S3Blobs(client, "kept", prefix="orchestration/", create=True) + ) + + +FILES = { + "guide.md": "# Guide", + "graph.json": '{"elements": []}', + "terraform/main.tf": 'resource "x" "y" {}', +} + + +def kept(store: Store, execution_id: str = "a1", **overrides: object) -> dict: + values: dict = { + "tenant": "t1", + "case": "acme", + "execution_id": execution_id, + "work": "architecture", + "status": "completed", + "request": "A landing zone", + "files": FILES, + "guide": "guide.md", + "project_id": "p1", + } + values.update(overrides) + return store.keep(**values) + + +def test_an_execution_is_kept_with_its_record(make_store): + store = make_store() + + record = kept(store) + + assert record["work"] == "architecture" + assert record["status"] == "completed" + assert record["customer_case"] == "acme" + assert record["project_id"] == "p1" + assert record["files"] == sorted(FILES) + assert store.record("t1", "acme", "a1") == record + + +def test_a_file_comes_back_with_its_media_type(make_store): + store = make_store() + kept(store) + + assert store.file("t1", "acme", "a1", "guide.md") == ( + b"# Guide", + "text/markdown; charset=utf-8", + ) + assert store.file("t1", "acme", "a1", "graph.json")[1] == "application/json" + assert ( + store.file("t1", "acme", "a1", "terraform/main.tf")[0] + == FILES["terraform/main.tf"].encode() + ) + assert store.file("t1", "acme", "a1", "missing.md") is None + + +def test_what_is_kept_is_never_changed(make_store): + store = make_store() + kept(store) + + with pytest.raises(ValueError, match="a1"): + kept(store, files={"guide.md": "# Another"}) + + +@pytest.mark.parametrize( + "name", ["../escape.md", "/etc/passwd", "a/../../b", "a\\b", ""] +) +def test_a_file_name_outside_the_execution_is_refused(make_store, name): + store = make_store() + + with pytest.raises(ValueError): + kept(store, files={name: "x"}) + assert store.file("t1", "acme", "a1", name) is None + + +@pytest.mark.parametrize("slug", ["..", "a/b", "", "a b"]) +def test_a_tenant_or_case_that_is_not_a_slug_is_refused(make_store, slug): + with pytest.raises(ValueError): + kept(make_store(), case=slug) + + +def test_latest_is_the_newest_completed_execution_of_the_work(make_store): + store = make_store() + kept(store, "a1") + kept(store, "a2") + kept(store, "a3", status="completed_with_warnings") + kept(store, "o1", work="operations") + + assert store.latest("t1", "acme", "architecture")["execution_id"] == "a2" + assert store.latest("t1", "acme", "nothing") is None + assert [r["execution_id"] for r in store.case("t1", "acme")] == [ + "o1", + "a3", + "a2", + "a1", + ] + + +def test_a_delivered_file_named_like_a_record_is_not_taken_for_one(make_store): + store = make_store() + kept(store, files={"record.json": '{"not": "a record"}'}) + + assert [r["execution_id"] for r in store.case("t1", "acme")] == ["a1"] + + +def test_object_storage_keeps_the_same_layout_under_its_prefix(): + with mock_aws(): + client = boto3.client("s3", region_name="us-east-1") + kept(Store(S3Blobs(client, "kept", prefix="orchestration/", create=True))) + + keys = sorted( + item["Key"] for item in client.list_objects_v2(Bucket="kept")["Contents"] + ) + + base = "orchestration/tenants/t1/cases/acme/executions/a1" + assert keys == [ + f"{base}/files/graph.json", + f"{base}/files/guide.md", + f"{base}/files/terraform/main.tf", + f"{base}/record.json", + ] + + +def test_the_settings_choose_the_storage(tmp_path, monkeypatch): + assert store_from_settings(OrchestrationSettings(_env_file=None)) is None # type: ignore[call-arg] + volume = store_from_settings( + OrchestrationSettings(_env_file=None, STORE_DIR=str(tmp_path)) # type: ignore[call-arg] + ) + assert isinstance(volume, Store) and isinstance(volume.blobs, VolumeBlobs) + monkeypatch.setenv("AWS_ACCESS_KEY_ID", "test") + monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "test") + with mock_aws(): + s3 = store_from_settings( + OrchestrationSettings( # type: ignore[call-arg] + _env_file=None, + STORE_BACKEND="s3", + S3_BUCKET="kept", + S3_REGION="us-east-1", + ) + ) + assert isinstance(s3, Store) and isinstance(s3.blobs, S3Blobs) + kept(s3) + assert s3.record("t1", "acme", "a1") is not None + + +def test_a_signed_path_is_accepted_and_a_changed_one_is_not(): + signer = Signer("secret", ttl_seconds=60, now=lambda: 1000) + + query = signer.sign("/tenants/t1/cases/acme") + + assert signer.verify("/tenants/t1/cases/acme", **query) + assert not signer.verify("/tenants/t1/cases/other", **query) + assert not signer.verify("/tenants/t1/cases/acme", exp=query["exp"], sig="0" * 64) + assert not Signer("other", ttl_seconds=60, now=lambda: 1000).verify( + "/tenants/t1/cases/acme", **query + ) + + +def test_a_link_expires_unless_its_lifetime_is_unlimited(): + clock = {"now": 1000} + expiring = Signer("secret", ttl_seconds=60, now=lambda: clock["now"]) + forever = Signer("secret", ttl_seconds=0, now=lambda: clock["now"]) + short, lasting = expiring.sign("/x"), forever.sign("/x") + + clock["now"] = 1061 + + assert not expiring.verify("/x", **short) + assert forever.verify("/x", **lasting)