4 Commits

Author SHA1 Message Date
5ea4fa4065 Merge pull request 'feat: keep what agents deliver, build on it by reference, in object storage (ADR-0004)' (#1) from docs/handoff-run into main
Reviewed-on: #1
2026-10-23 17:29:50 +00:00
fcac4e425c docs: record the sequence run with real models
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:04:26 +01:00
83df77005a feat: keep delivered work in S3-compatible object storage (ADR-0004 phase 3)
The store's rules now sit on a Blobs layer: a folder, as before, or a
prefix in an S3-compatible bucket (MinIO locally), chosen by STORE_BACKEND.
The layout is the same in both; the record is written last.

- S3_BUCKET, S3_PREFIX, S3_ENDPOINT_URL, S3_REGION; AWS credentials
- an s3 extra (boto3), installed in the image; moto for tests
- every store test runs on both kinds of storage

Retention, the platform's sign-in and each kept file as a platform
artifact are left: the last needs the platform's S3 backend, which it does
not yet have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:08:39 +01:00
1ec939f4fd feat: build on kept work by reference with sources_from (ADR-0004 phase 2)
A request names work the Orchestration engine kept instead of carrying its
files: a record or latest link from an earlier document of links (checked,
and of the request's tenant and Customer case), latest/<work>, or an
Execution's id. The kept files and the record that the work completed are
the agent's sources. Nothing starts, and the caller is told why, when the
reference does not resolve to completed work it may use.

A sequence's next agent is given what was kept of the one before, the same
way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:00:26 +01:00
14 changed files with 743 additions and 75 deletions

View File

@@ -7,7 +7,7 @@ One directory per iteration. Each holds the plan for that iteration and, for fin
| # | Directory | Status | What it is |
| --- | --- | --- | --- |
| 1 | `iter-01-execution-engine/` | steps 1 to 6 built on 2026-10-13; 7 planned | The Execution engine: brought from the Reference, given a Workflow, and proved on a toy Workflow and on the Reference's own. |
| 2 | `iter-02-orchestration-engine/` | F1 proposed; F2 and F4 built on 2026-10-20; K1 (ADR-0004) built on 2026-10-21 | The Orchestration engine: routes each request to the agent for its work and relays everything back. |
| 2 | `iter-02-orchestration-engine/` | F1 proposed; F2 and F4 built on 2026-10-20; K1, K2 and K3's object storage (ADR-0004) built on 2026-10-21 | The Orchestration engine: routes each request to the agent for its work and relays everything back. |
The plan across both repositories is `../tom_analysis/IMPLEMENTATION_PLAN.md`.

View File

@@ -2,7 +2,7 @@
**Iteration:** 2, the Orchestration engine
**Intent:** The service every request comes through. It decides which agent the work belongs to, dispatches it, and relays every question and Approval. First behind it: the operating model agent (`../tom_analysis`).
**Status:** F1 proposed; F2 and F4 built 2026-10-20; K1 built 2026-10-21
**Status:** F1 proposed; F2 and F4 built 2026-10-20; K1, K2 and object storage (K3) built 2026-10-21
## Decided by the owner, 2026-10-20
@@ -23,8 +23,8 @@
| F1 | ADR-0003: one Orchestration engine above the agents | Accepted by the owner | proposed |
| F2 | The service: an A2A server, a configured list of agents, a request passed through to one agent with every prompt relayed; its image; on the local platform | An Execution of the operating model agent runs through it, Approvals included, with the agent untouched | built (`steps/step-2.md`); the platform workflow through it is step 4 of `../tom_analysis` iteration 2 |
| F3 | The decision: readings in code, the Jev question, thresholds, the fallbacks, the record | A labelled set of messages is routed correctly; low confidence asks; no Jev key still routes what is named | planned |
| F4 | "Both": architecture, then its files as `sources` to the operating model agent | One request yields both sets of Deliverables under one Customer case | built against scripted agents (`steps/step-4.md`), ahead of F3 at the owner's request; not yet run with real models |
| F4 | "Both": architecture, then its files as `sources` to the operating model agent | One request yields both sets of Deliverables under one Customer case | built (`steps/step-4.md`), ahead of F3 at the owner's request; run with real models 2026-10-21: the solution architecture agent's work handed to the operating model agent, both completed in 74.9 minutes, kept in MinIO (`../solution_architecture_agent/.agents/plans/iter-01-on-the-engine/steps/step-2.md`) |
| F5 | Later messages to the agent that owns the work | A change to the architecture is followed by a change to the operating model | planned |
| K1 | What agents deliver, kept and served at signed links; a platform step given one document of links (ADR-0004, phase 1) | A platform step's output links to each delivered file, and each link serves it | built (`steps/step-K1.md`); the owner's run through the platform to follow |
| K2 | `sources_from`: a request names kept work instead of carrying its files; the sequence moved onto it | An operating model started from the latest architecture of a Customer case | planned |
| K3 | Object storage shared with the platform, the platform's sign-in, retention, each file as a platform artifact | A kept guide is listed among the project's artifacts | planned |
| K2 | `sources_from`: a request names kept work instead of carrying its files; the sequence moved onto it | An operating model started from the latest architecture of a Customer case | built (`steps/step-K2.md`); the owner's run through the platform to follow |
| K3 | Object storage shared with the platform, the platform's sign-in, retention, each file as a platform artifact | A kept guide is listed among the project's artifacts | object storage built (`steps/step-K3.md`); retention, the platform's sign-in and platform artifacts not chosen for now, the last needing the platform team |

View File

@@ -0,0 +1,25 @@
# Step K2: building on kept work by reference (ADR-0004, phase 2)
Built 2026-10-21 on `feat/sources-from`, from `feat/delivery-store`.
## What was built
- **`sources_from`:** a request names work kept earlier instead of carrying its files. It may be:
- a link from an earlier document of links, its `record` or its `latest`: what a platform step maps from an earlier step's output. The signature is checked; the link must be of the request's tenant when the request names one, and of its Customer case when it names one;
- `latest/<work>`: the newest completed Execution of that work in the request's Customer case;
- an Execution's id in the request's Customer case.
- **What the agent is sent:** the kept files as `sources`, with the Orchestration engine's record that the work completed (`execution.json`: status, Customer case, Execution, the request, the work). A request naming no Customer case takes the kept work's.
- **Nothing starts, and the caller is told why,** when the reference resolves to nothing, to work that did not end `completed`, to another tenant or Customer case, when the link was changed, when nothing is kept here, or when `sources` is given as well. The reply is a completed task with `{"status": "not_started", "reason": ...}`.
- **The sequence on the same footing:** an agent after another is given what was kept of the one before (`Delivery.sources`), as a request naming it in `sources_from` would be. Without a store, it is given the files relayed, as before.
- **`Store.files`, `Delivery.resolve`, `Delivery.sources`;** the record `execution.json` moved to `delivery.py`.
- **The card** offers `sources_from`.
## Verified
- 12 tests in `test_sources_from.py`: each of the three forms; a `latest` link followed to the newest work; a changed link, another tenant, another Customer case, nothing kept, work that ended with warnings, and `sources` with `sources_from` each starting nothing; a sequence's next agent given what was kept; the card.
- 589 tests pass; `ruff` and `mypy` clean.
- Deployed on the local platform; the card offers `sources_from`. No agent was run: the owner runs it through the platform.
## Using it from the platform
A second workflow step on the Orchestration engine with `work: operating_model` and `sources_from` mapped from the first step's `record`; or, in a later workflow, `sources_from: latest/solution_architecture` with the Customer case. The operating model agent's own fields (who operates it, business outcomes) go in that step's `scope`.

View File

@@ -0,0 +1,30 @@
# Step K3: what is kept, in object storage (ADR-0004, phase 3, first part)
Built 2026-10-21 on `feat/object-store`, from `feat/sources-from`.
## Scope, as the owner chose it
Phase 3 was planned as object storage shared with the platform, the platform's sign-in, retention, and each kept file as a platform artifact. Looking at the platform first changed it:
- **The platform's S3 and GCS artifact backends are not implemented** (`agent-builder-api/src/storage/router.py`: "S3 backend not yet implemented"). It serves artifacts from MongoDB, Azure Blob, Neo4j and its execution volume only.
- **The only artifact route the internal service token may call,** `POST /api/v1/executions/{id}/artifacts`, registers a step's whole output (`step_N/output.json`); `POST /projects/{id}/artifacts` needs a person's sign-in.
So each kept file as a platform artifact is the platform team's change, and the platform's sign-in would help API callers only. The owner chose object storage alone, for now.
## What was built
- **`store.py`:** the store's rules (names, write-once, records, `latest`) over a `Blobs` layer with two kinds: `VolumeBlobs`, a folder, and `S3Blobs`, a prefix in an S3-compatible bucket, made if missing. The layout is the same in both: `tenants/<tenant>/cases/<case>/executions/<id>/{record.json, files/…}`. The record is written last, so an Execution is kept once its record is. A delivered file named `record.json` is not taken for one.
- **Settings:** `STORE_BACKEND` (`volume` or `s3`), `S3_BUCKET`, `S3_PREFIX`, `S3_ENDPOINT_URL`, `S3_REGION`; credentials are the standard `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. `store_from_settings` makes the store.
- **Packaging:** the `s3` extra (`boto3`); the image installs it. `moto` in the dev extra; mypy told the AWS SDK has no types.
- **Local deployment:** the dev kit's MinIO (`agent-builder-minio`, started alone from the `observability` profile), bucket `orchestration`, its root credentials from the kit's `.env`.
## Verified
- Every store test runs on both kinds of storage, S3 through moto in the process; and the object layout under its prefix, a delivered `record.json`, and the settings choosing the storage.
- 606 tests pass; `ruff` and `mypy` clean.
- Deployed: healthy, with the `orchestration` bucket made and reachable on MinIO. No agent was run.
## Not done
- Retention, the platform's sign-in, and each kept file as a platform artifact (a proposal to the platform team) remain, not chosen for now.
- Nothing kept on the volume before is moved: nothing had been kept.

View File

@@ -5,8 +5,8 @@
#
# Which agents it routes to is configuration, mounted at /app/agents.yaml
# (AGENTS_FILE); deploy/agents.local.yaml is the local KAB platform's. What agents
# deliver is kept in /data/store (ADR-0004); give it a volume, a PUBLIC_URL and a
# LINK_SECRET.
# deliver is kept in /data/store (ADR-0004), or in an S3-compatible bucket with
# STORE_BACKEND=s3; give it a volume or a bucket, a PUBLIC_URL and a LINK_SECRET.
FROM python:3.12-slim AS builder
@@ -14,7 +14,7 @@ WORKDIR /build
COPY pyproject.toml ./
COPY execution_engine/ ./execution_engine/
COPY orchestration/ ./orchestration/
RUN pip install --no-cache-dir --prefix=/install .
RUN pip install --no-cache-dir --prefix=/install ".[s3]"
FROM python:3.12-slim

View File

@@ -158,6 +158,14 @@ def input_contract(
"`files`: another agent's delivered work."
),
},
"sources_from": {
"type": "string",
"description": (
"Work kept earlier to start from instead of its files: an "
"earlier step's `record` or `latest` link, `latest/<work>`, "
"or an Execution's id in this Customer case."
),
},
"deliver": {
"type": "string",
"enum": ["links", "files"],

View File

@@ -27,9 +27,16 @@ class OrchestrationSettings(BaseSettings):
# agent's work takes.
CONNECT_TIMEOUT_SECONDS: float = 10
# What agents deliver is kept here and served at signed links (ADR-0004).
# Unset: nothing is kept, and a caller asking for links gets none.
# What agents deliver is kept and served at signed links (ADR-0004): in a folder
# (`volume`, STORE_DIR; unset, nothing is kept) or in an S3-compatible bucket
# (`s3`). The bucket's credentials are the standard AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY.
STORE_BACKEND: str = "volume"
STORE_DIR: str | None = None
S3_BUCKET: str | None = None
S3_PREFIX: str = ""
S3_ENDPOINT_URL: str | None = None # MinIO, or another S3-compatible service
S3_REGION: str = "us-east-1"
# The base of every link: the address a browser reaches this service at, which
# is not its address inside the platform's network. Unset: A2A_PUBLIC_URL.
PUBLIC_URL: str | None = None

View File

@@ -3,11 +3,12 @@ signer and the address a caller reaches them at, together."""
from __future__ import annotations
import json
import logging
import re
from collections.abc import Mapping
from typing import Any
from urllib.parse import quote
from urllib.parse import parse_qs, quote, unquote, urlsplit
from orchestration.links import Signer
from orchestration.store import Store
@@ -16,7 +17,11 @@ logger = logging.getLogger(__name__)
LOCAL_TENANT = "local" # what is kept for a caller that names no tenant
UNASSIGNED = "unassigned" # for a request that names no Customer case
# The Orchestration engine's own record that an Execution completed, sent with its
# files to the agent that builds on them.
RECORD = "execution.json"
_NOT_SLUG = re.compile(r"[^A-Za-z0-9._-]+")
_KEPT_PATH = re.compile(r"^/tenants/([^/]+)/cases/([^/]+)/(executions|latest)/([^/]+)$")
def _slug(value: Any, default: str) -> str:
@@ -68,6 +73,76 @@ class Delivery:
logger.exception("Could not keep %s Execution %s", work, execution_id)
return None
# --- building on what was kept (phase 2) ---------------------------------------
def resolve(self, reference: str, fields: Mapping[str, Any]) -> dict[str, Any]:
"""The kept Execution a request names in `sources_from`: a link from an
earlier document, `latest/<work>`, or an Execution's id in the request's
Customer case. Raises ValueError, saying why, for anything else."""
tenant, case = self.place(fields)
named_case = fields.get("customer_case")
if reference.startswith(("http://", "https://", "/")):
url = urlsplit(reference)
query = parse_qs(url.query)
exp, sig = (query.get(key, [None])[0] for key in ("exp", "sig"))
if not self.signer.verify(url.path, exp, sig):
raise ValueError("The link in sources_from is not valid.")
match = _KEPT_PATH.match(url.path)
if not match:
raise ValueError("The link in sources_from is not to kept work.")
tenant_of, case, kind, ident = match.groups()
if fields.get("tenant_id") and tenant_of != tenant:
raise ValueError("The link in sources_from is of another tenant.")
tenant = tenant_of
if named_case and _slug(named_case, UNASSIGNED) != case:
raise ValueError(
f"The work in sources_from is of the Customer case {case}, "
f"not {named_case}."
)
ident = unquote(ident)
record = (
self.store.record(tenant, case, ident)
if kind == "executions"
else self.store.latest(tenant, case, ident)
)
elif reference.startswith("latest/"):
work = reference.removeprefix("latest/")
record = self.store.latest(tenant, case, work)
if record is None:
raise ValueError(
f"No {work} work has completed in the Customer case {case}."
)
else:
record = self.store.record(tenant, case, reference)
if record is None:
raise ValueError(
f"Nothing is kept for {reference!r} in the Customer case {case}."
)
if record["status"] != "completed":
raise ValueError(
f"The {record['work']} work {record['execution_id']} ended "
f"{record['status']}; only work that completed is built on."
)
return record
def sources(self, record: Mapping[str, Any]) -> dict[str, str]:
"""A kept Execution as an agent's sources: its files, with the record that
it completed."""
files = self.store.files(
record["tenant"], record["customer_case"], record["execution_id"]
)
completion = {
"status": record["status"],
"customer_case": record["customer_case"]
if record["customer_case"] != UNASSIGNED
else None,
"execution_id": record["execution_id"],
"workflow_request": record["request"],
"work": record["work"],
}
files[RECORD] = json.dumps({k: v for k, v in completion.items() if v})
return files
# --- addresses -------------------------------------------------------------
@staticmethod

View File

@@ -29,7 +29,7 @@ from orchestration.delivery import Delivery
from orchestration.links import Signer
from orchestration.relay import AgentClients, RelayExecutor
from orchestration.serving import kept_routes
from orchestration.store import Store
from orchestration.store import Store, store_from_settings
logger = logging.getLogger(__name__)
@@ -121,8 +121,8 @@ def _delivery(
) -> Delivery | None:
"""Where what agents deliver is kept, and how it is linked to; None when
nothing is to be kept."""
if store is None and settings.STORE_DIR:
store = Store(settings.STORE_DIR)
if store is None:
store = store_from_settings(settings)
if store is None:
return None
if signer is None:

View File

@@ -44,7 +44,7 @@ from google.protobuf import json_format
from orchestration.agents import Agent
from orchestration.agents import Sequence as AgentSequence
from orchestration.delivery import Delivery
from orchestration.delivery import RECORD, UNASSIGNED, Delivery
from orchestration.routing import WORK, route
logger = logging.getLogger(__name__)
@@ -62,9 +62,8 @@ _PAUSES = {TaskState.TASK_STATE_INPUT_REQUIRED, TaskState.TASK_STATE_AUTH_REQUIR
# What every agent of a sequence is sent of the caller's fields. The rest go to the
# first agent only, or to the agent whose work names them: `operating_model: {...}`.
_CARRIED = ("customer_case", "pre_approved", "request")
# The Orchestration engine's own record that an agent's Execution completed, sent
# with its files to the next agent.
RECORD = "execution.json"
# Where a request names work kept earlier to build on (ADR-0004, phase 2).
SOURCES_FROM = "sources_from"
# How a caller is given what agents delivered (ADR-0004): one document of links to
# what was kept, or each file as the agent sent it.
DELIVER, LINKS, FILES = "deliver", "links", "files"
@@ -87,6 +86,8 @@ class Plan:
deliver: str = FILES
# Each agent that completed, by its work: its Execution, how it ended, links.
results: dict[str, dict[str, Any]] = field(default_factory=dict)
# What was kept of each, by its work, for the agent after it to build on.
kept: dict[str, dict[str, Any]] = field(default_factory=dict)
def metadata(self, step: int, sources: Mapping[str, str] | None) -> dict[str, Any]:
"""The metadata for the agent at `step`."""
@@ -203,8 +204,36 @@ class RelayExecutor(AgentExecutor):
work=str(options.get(WORK) or decision.agents[0].work),
deliver=_deliver(options),
)
if SOURCES_FROM in fields:
try:
self._build_on(fields)
except ValueError as exc:
await updater.complete(
updater.new_agent_message(
[
new_text_part(f"Nothing was started: {exc}"),
new_data_part(
{"status": "not_started", "reason": str(exc)}
),
]
)
)
return
await self._run(task_id, plan, 0, self._opening(plan, 0, None), updater)
def _build_on(self, fields: dict[str, Any]) -> None:
"""Make the work a request names in `sources_from` its sources, and its
Customer case the request's when it names none."""
reference = str(fields.pop(SOURCES_FROM) or "")
if self._delivery is None:
raise ValueError("Nothing is kept here to build on.")
if fields.get("sources"):
raise ValueError("Give sources or sources_from, not both.")
record = self._delivery.resolve(reference, fields)
if not fields.get("customer_case") and record["customer_case"] != UNASSIGNED:
fields["customer_case"] = record["customer_case"]
fields["sources"] = self._delivery.sources(record)
async def _resume(
self,
task_id: str,
@@ -381,6 +410,7 @@ class RelayExecutor(AgentExecutor):
else None
)
if record is not None and self._delivery is not None:
plan.kept[agent.work] = record
result.update(self._delivery.links(record))
else:
result["files"] = {name: None for name in sorted(files)}
@@ -429,17 +459,22 @@ class RelayExecutor(AgentExecutor):
not_started,
)
return
record = {
"status": "completed",
"customer_case": plan.fields.get("customer_case"),
"execution_id": plan.results[agent.work]["execution_id"] or None,
"workflow_request": plan.text,
"work": agent.work,
}
sources = {
**files,
RECORD: json.dumps({k: v for k, v in record.items() if v is not None}),
}
kept = plan.kept.get(agent.work)
if kept is not None and self._delivery is not None:
# What was kept, as a request naming it in sources_from would have it.
sources = self._delivery.sources(kept)
else:
record = {
"status": "completed",
"customer_case": plan.fields.get("customer_case"),
"execution_id": plan.results[agent.work]["execution_id"] or None,
"workflow_request": plan.text,
"work": agent.work,
}
sources = {
**files,
RECORD: json.dumps({k: v for k, v in record.items() if v is not None}),
}
following = plan.steps[step + 1]
await updater.update_status(
TaskState.TASK_STATE_WORKING,

View File

@@ -1,10 +1,11 @@
"""What agents delivered, kept by tenant, Customer case and Execution (ADR-0004).
<root>/tenants/<tenant>/cases/<case>/executions/<id>/record.json
/files/<name as delivered>
tenants/<tenant>/cases/<case>/executions/<id>/record.json
/files/<name as delivered>
An Execution is written once. Its files are what the agent delivered, by the names
it gave them; its record says what it was.
it gave them; its record says what it was. The same layout is kept in a folder or
under a prefix in an S3-compatible bucket.
"""
from __future__ import annotations
@@ -15,7 +16,10 @@ import re
import time
from collections.abc import Mapping
from pathlib import Path, PurePosixPath
from typing import Any
from typing import TYPE_CHECKING, Any, Protocol
if TYPE_CHECKING:
from orchestration.config import OrchestrationSettings
_SLUG = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
_MEDIA_TYPES = {
@@ -53,21 +57,128 @@ def media_type(name: str) -> str:
return _MEDIA_TYPES.get(PurePosixPath(name).suffix.lower(), _TEXT)
class Store:
class Blobs(Protocol):
"""Where a store's keys are kept: a folder, or a bucket."""
def put(self, key: str, data: bytes, content_type: str) -> None: ...
def get(self, key: str) -> bytes | None: ...
def exists(self, key: str) -> bool: ...
def keys(self, prefix: str) -> list[str]: ...
class VolumeBlobs:
"""Keys as files under a folder."""
def __init__(self, root: str | Path) -> None:
self.root = Path(root)
def _case(self, tenant: str, case: str) -> Path:
return (
self.root
/ "tenants"
/ slug(tenant, "Tenant")
/ "cases"
/ slug(case, "Case")
def _path(self, key: str) -> Path:
path = (self.root / key).resolve()
if self.root.resolve() not in path.parents:
raise ValueError(f"{key!r} is outside the store")
return path
def put(self, key: str, data: bytes, content_type: str) -> None:
path = self._path(key)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
def get(self, key: str) -> bytes | None:
path = self._path(key)
return path.read_bytes() if path.is_file() else None
def exists(self, key: str) -> bool:
return self._path(key).is_file()
def keys(self, prefix: str) -> list[str]:
folder = self.root / prefix
if not folder.is_dir():
return []
return [
path.relative_to(self.root).as_posix()
for path in folder.rglob("*")
if path.is_file()
]
class S3Blobs:
"""Keys as objects under a prefix in an S3-compatible bucket (MinIO locally)."""
def __init__(
self, client: Any, bucket: str, prefix: str = "", create: bool = False
) -> None:
self._client = client
self.bucket = bucket
self.prefix = prefix.lstrip("/")
if create:
self._ensure_bucket()
def _ensure_bucket(self) -> None:
from botocore.exceptions import ClientError
try:
self._client.head_bucket(Bucket=self.bucket)
except ClientError:
self._client.create_bucket(Bucket=self.bucket)
def put(self, key: str, data: bytes, content_type: str) -> None:
self._client.put_object(
Bucket=self.bucket,
Key=self.prefix + key,
Body=data,
ContentType=content_type,
)
def _execution(self, tenant: str, case: str, execution_id: str) -> Path:
return self._case(tenant, case) / "executions" / slug(execution_id, "Execution")
def get(self, key: str) -> bytes | None:
from botocore.exceptions import ClientError
try:
response = self._client.get_object(
Bucket=self.bucket, Key=self.prefix + key
)
except ClientError as exc:
if exc.response.get("Error", {}).get("Code") in {"NoSuchKey", "404"}:
return None
raise
data: bytes = response["Body"].read()
return data
def exists(self, key: str) -> bool:
from botocore.exceptions import ClientError
try:
self._client.head_object(Bucket=self.bucket, Key=self.prefix + key)
except ClientError as exc:
if exc.response.get("Error", {}).get("Code") in {"NoSuchKey", "404"}:
return False
raise
return True
def keys(self, prefix: str) -> list[str]:
found: list[str] = []
pages = self._client.get_paginator("list_objects_v2").paginate(
Bucket=self.bucket, Prefix=self.prefix + prefix
)
for page in pages:
for item in page.get("Contents", []):
found.append(str(item["Key"])[len(self.prefix) :])
return found
class Store:
def __init__(self, blobs: Blobs | str | Path) -> None:
self.blobs: Blobs = (
VolumeBlobs(blobs) if isinstance(blobs, (str, Path)) else blobs
)
@staticmethod
def _case(tenant: str, case: str) -> str:
return f"tenants/{slug(tenant, 'Tenant')}/cases/{slug(case, 'Case')}"
def _execution(self, tenant: str, case: str, execution_id: str) -> str:
return (
f"{self._case(tenant, case)}/executions/{slug(execution_id, 'Execution')}"
)
def keep(
self,
@@ -86,12 +197,12 @@ class Store:
already kept is refused."""
folder = self._execution(tenant, case, execution_id)
paths = {name: file_name(name) for name in files}
if folder.exists():
if self.blobs.exists(f"{folder}/record.json"):
raise ValueError(f"Execution {execution_id} is already kept")
for name, path in paths.items():
target = folder / "files" / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(files[name], encoding="utf-8")
self.blobs.put(
f"{folder}/files/{path}", files[name].encode("utf-8"), media_type(name)
)
record = {
"tenant": tenant,
"customer_case": case,
@@ -105,20 +216,26 @@ class Store:
"kept_at": dt.datetime.now(dt.UTC).isoformat(),
"kept_ns": time.time_ns(),
}
folder.mkdir(parents=True, exist_ok=True)
(folder / "record.json").write_text(json.dumps(record, indent=2), "utf-8")
# The record last: an Execution is kept once its record is.
self.blobs.put(
f"{folder}/record.json",
json.dumps(record, indent=2).encode("utf-8"),
"application/json",
)
return record
def record(
self, tenant: str, case: str, execution_id: str
) -> dict[str, Any] | None:
try:
path = self._execution(tenant, case, execution_id) / "record.json"
data = self.blobs.get(
f"{self._execution(tenant, case, execution_id)}/record.json"
)
except ValueError:
return None
if not path.is_file():
if data is None:
return None
record: dict[str, Any] = json.loads(path.read_text("utf-8"))
record: dict[str, Any] = json.loads(data)
return record
def file(
@@ -126,23 +243,38 @@ class Store:
) -> tuple[bytes, str] | None:
"""A kept file's content and media type, or None."""
try:
folder = self._execution(tenant, case, execution_id) / "files"
target = folder / file_name(name)
key = (
f"{self._execution(tenant, case, execution_id)}/files/{file_name(name)}"
)
data = self.blobs.get(key)
except ValueError:
return None
if not target.is_file() or folder.resolve() not in target.resolve().parents:
return None
return target.read_bytes(), media_type(name)
return None if data is None else (data, media_type(name))
def files(self, tenant: str, case: str, execution_id: str) -> dict[str, str]:
"""Every file kept for an Execution, by name."""
record = self.record(tenant, case, execution_id) or {}
found: dict[str, str] = {}
for name in record.get("files", []):
kept = self.file(tenant, case, execution_id, name)
if kept is not None:
found[name] = kept[0].decode("utf-8")
return found
def case(self, tenant: str, case: str) -> list[dict[str, Any]]:
"""The Customer case's Executions, newest first."""
try:
folder = self._case(tenant, case) / "executions"
prefix = f"{self._case(tenant, case)}/executions/"
except ValueError:
return []
records = [
json.loads(path.read_text("utf-8")) for path in folder.glob("*/record.json")
]
records = []
for key in self.blobs.keys(prefix):
# <id>/record.json; a delivered file of that name is under files/.
parts = key[len(prefix) :].split("/")
if len(parts) == 2 and parts[1] == "record.json":
data = self.blobs.get(key)
if data is not None:
records.append(json.loads(data))
return sorted(records, key=lambda r: r.get("kept_ns", 0), reverse=True)
def latest(self, tenant: str, case: str, work: str) -> dict[str, Any] | None:
@@ -155,3 +287,25 @@ class Store:
),
None,
)
def store_from_settings(settings: OrchestrationSettings) -> Store | None:
"""The store the settings name; None when nothing is to be kept."""
if settings.STORE_BACKEND == "s3":
import boto3
if not settings.S3_BUCKET:
raise ValueError("STORE_BACKEND=s3 needs S3_BUCKET")
client = boto3.client(
"s3",
endpoint_url=settings.S3_ENDPOINT_URL or None,
region_name=settings.S3_REGION,
)
return Store(
S3Blobs(client, settings.S3_BUCKET, settings.S3_PREFIX, create=True)
)
if settings.STORE_BACKEND != "volume":
raise ValueError(
f"STORE_BACKEND must be volume or s3, not {settings.STORE_BACKEND!r}"
)
return Store(VolumeBlobs(settings.STORE_DIR)) if settings.STORE_DIR else None

View File

@@ -30,7 +30,11 @@ dependencies = [
]
[project.optional-dependencies]
# The Orchestration engine's store on S3-compatible object storage (ADR-0004).
s3 = ["boto3>=1.34"]
dev = [
"boto3>=1.34",
"moto[s3]>=5.0",
"pytest>=8.0.0",
"pytest-asyncio>=0.23.0",
"ruff>=0.6.0",
@@ -59,3 +63,8 @@ cache-dir = ".cache/ruff"
[tool.mypy]
cache_dir = ".cache/mypy"
[[tool.mypy.overrides]]
# The AWS SDK and its test double ship without type information.
module = ["boto3", "boto3.*", "botocore", "botocore.*", "moto", "moto.*"]
ignore_missing_imports = true

View File

@@ -0,0 +1,258 @@
"""A request names work kept earlier instead of carrying its files (ADR-0004,
phase 2): `sources_from` is a link from an earlier document, `latest/<work>`, or
an Execution's id in the request's Customer case."""
from __future__ import annotations
import json
from typing import Any
from tests.orchestration.test_links import PLATFORM, Harness, document
from tests.orchestration.test_sequence import ScriptedAgents, finished, struct
GRAPH = '{"elements": []}'
def designed(**more: Any) -> ScriptedAgents:
"""An architecture agent that has designed once, and an operations agent ready."""
turns: dict[str, list[Any]] = {
"architecture": [finished("a1", {"graph.json": GRAPH, "guide.md": "# Guide"})],
"operations": [finished("o1", {"model.json": "{}"})],
}
turns.update(more)
return ScriptedAgents(turns)
async def design(harness: Harness, case: str = "acme") -> dict[str, Any]:
return document(
await harness.send(
"A landing zone",
{**PLATFORM, "work": "architecture", "customer_case": case},
)
)
def not_started(task: dict[str, Any]) -> str:
assert task["status"]["state"] == "TASK_STATE_COMPLETED"
parts = task["status"]["message"]["parts"]
data = next(p["data"] for p in parts if "data" in p)
assert data["status"] == "not_started"
return " ".join(p.get("text", "") for p in parts)
def sent_sources(harness: Harness) -> dict[str, Any]:
(message,) = harness.agents.received["operations"]
metadata = struct(message)
assert "sources_from" not in metadata
return metadata
async def test_the_latest_work_of_the_case_is_the_agents_sources(tmp_path):
harness = Harness(tmp_path, designed())
await design(harness)
await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "acme",
"sources_from": "latest/architecture",
},
)
sent = sent_sources(harness)
assert sent["sources"]["graph.json"] == GRAPH
assert sent["sources"]["guide.md"] == "# Guide"
assert json.loads(sent["sources"]["execution.json"]) == {
"status": "completed",
"customer_case": "acme",
"execution_id": "a1",
"workflow_request": "A landing zone",
"work": "architecture",
}
async def test_a_link_from_an_earlier_document_names_the_work_and_its_case(tmp_path):
# A platform step maps an earlier step's `record` into `sources_from`.
harness = Harness(tmp_path, designed())
earlier = await design(harness)
await harness.send(
"Operate it",
{**PLATFORM, "work": "operations", "sources_from": earlier["record"]},
)
sent = sent_sources(harness)
assert sent["customer_case"] == "acme"
assert sent["sources"]["graph.json"] == GRAPH
async def test_a_latest_link_is_followed_to_the_newest_completed_work(tmp_path):
harness = Harness(
tmp_path,
designed(
architecture=[
finished("a1", {"graph.json": "1"}),
finished("a2", {"graph.json": "2"}),
]
),
)
first = await design(harness)
await design(harness)
await harness.send(
"Operate it",
{**PLATFORM, "work": "operations", "sources_from": first["latest"]},
)
assert sent_sources(harness)["sources"]["graph.json"] == "2"
async def test_an_execution_of_the_case_is_named_by_its_id(tmp_path):
harness = Harness(tmp_path, designed())
await design(harness)
await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "acme",
"sources_from": "a1",
},
)
assert sent_sources(harness)["sources"]["graph.json"] == GRAPH
async def test_a_link_that_was_changed_starts_nothing(tmp_path):
harness = Harness(tmp_path, designed())
earlier = await design(harness)
task = await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"sources_from": earlier["record"].replace("/a1?", "/a2?"),
},
)
assert "not valid" in not_started(task)
assert harness.agents.received["operations"] == []
async def test_a_link_of_another_tenant_starts_nothing(tmp_path):
harness = Harness(tmp_path, designed())
earlier = await design(harness)
task = await harness.send(
"Operate it",
{
**PLATFORM,
"tenant_id": "t2",
"work": "operations",
"sources_from": earlier["record"],
},
)
assert "tenant" in not_started(task)
assert harness.agents.received["operations"] == []
async def test_work_of_another_customer_case_starts_nothing(tmp_path):
harness = Harness(tmp_path, designed())
earlier = await design(harness)
task = await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "other",
"sources_from": earlier["record"],
},
)
assert "acme" in not_started(task)
async def test_nothing_kept_to_build_on_starts_nothing(tmp_path):
harness = Harness(tmp_path, designed())
task = await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "acme",
"sources_from": "latest/architecture",
},
)
assert "architecture" in not_started(task)
assert harness.agents.received["operations"] == []
async def test_work_that_ended_with_warnings_is_not_built_on(tmp_path):
harness = Harness(
tmp_path,
designed(
architecture=[
finished("a1", {"graph.json": GRAPH}, status="completed_with_warnings")
]
),
)
await design(harness)
task = await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "acme",
"sources_from": "a1",
},
)
assert "completed_with_warnings" in not_started(task)
async def test_sources_and_sources_from_together_start_nothing(tmp_path):
harness = Harness(tmp_path, designed())
await design(harness)
task = await harness.send(
"Operate it",
{
**PLATFORM,
"work": "operations",
"customer_case": "acme",
"sources_from": "a1",
"sources": {"x.json": "{}"},
},
)
assert "sources" in not_started(task)
async def test_a_sequence_gives_the_next_agent_what_was_kept(tmp_path):
harness = Harness(tmp_path, designed())
await harness.send(
"A landing zone",
{**PLATFORM, "work": "architecture_and_operations", "customer_case": "acme"},
)
sources = sent_sources(harness)["sources"]
assert sources["graph.json"] == GRAPH
assert json.loads(sources["execution.json"])["execution_id"] == "a1"
async def test_the_card_offers_sources_from(tmp_path):
harness = Harness(tmp_path, ScriptedAgents({}))
async with harness.client() as caller:
skill = (await caller.get("/.well-known/agent-card.json")).json()["skills"][0]
assert skill["inputSchema"]["properties"]["sources_from"]["type"] == "string"

View File

@@ -2,10 +2,30 @@
from __future__ import annotations
import pytest
from collections.abc import Iterator
from typing import Any
import boto3
import pytest
from moto import mock_aws
from orchestration.config import OrchestrationSettings
from orchestration.links import Signer
from orchestration.store import Store
from orchestration.store import S3Blobs, Store, VolumeBlobs, store_from_settings
@pytest.fixture(params=["volume", "s3"])
def make_store(request: Any, tmp_path: Any) -> Iterator[Any]:
"""A store on each kind of storage: a folder, and S3 (moto in this process)."""
if request.param == "volume":
yield lambda: Store(VolumeBlobs(tmp_path))
return
with mock_aws():
client = boto3.client("s3", region_name="us-east-1")
yield lambda: Store(
S3Blobs(client, "kept", prefix="orchestration/", create=True)
)
FILES = {
"guide.md": "# Guide",
@@ -30,8 +50,8 @@ def kept(store: Store, execution_id: str = "a1", **overrides: object) -> dict:
return store.keep(**values)
def test_an_execution_is_kept_with_its_record(tmp_path):
store = Store(tmp_path)
def test_an_execution_is_kept_with_its_record(make_store):
store = make_store()
record = kept(store)
@@ -43,8 +63,8 @@ def test_an_execution_is_kept_with_its_record(tmp_path):
assert store.record("t1", "acme", "a1") == record
def test_a_file_comes_back_with_its_media_type(tmp_path):
store = Store(tmp_path)
def test_a_file_comes_back_with_its_media_type(make_store):
store = make_store()
kept(store)
assert store.file("t1", "acme", "a1", "guide.md") == (
@@ -59,8 +79,8 @@ def test_a_file_comes_back_with_its_media_type(tmp_path):
assert store.file("t1", "acme", "a1", "missing.md") is None
def test_what_is_kept_is_never_changed(tmp_path):
store = Store(tmp_path)
def test_what_is_kept_is_never_changed(make_store):
store = make_store()
kept(store)
with pytest.raises(ValueError, match="a1"):
@@ -70,8 +90,8 @@ def test_what_is_kept_is_never_changed(tmp_path):
@pytest.mark.parametrize(
"name", ["../escape.md", "/etc/passwd", "a/../../b", "a\\b", ""]
)
def test_a_file_name_outside_the_execution_is_refused(tmp_path, name):
store = Store(tmp_path)
def test_a_file_name_outside_the_execution_is_refused(make_store, name):
store = make_store()
with pytest.raises(ValueError):
kept(store, files={name: "x"})
@@ -79,13 +99,13 @@ def test_a_file_name_outside_the_execution_is_refused(tmp_path, name):
@pytest.mark.parametrize("slug", ["..", "a/b", "", "a b"])
def test_a_tenant_or_case_that_is_not_a_slug_is_refused(tmp_path, slug):
def test_a_tenant_or_case_that_is_not_a_slug_is_refused(make_store, slug):
with pytest.raises(ValueError):
kept(Store(tmp_path), case=slug)
kept(make_store(), case=slug)
def test_latest_is_the_newest_completed_execution_of_the_work(tmp_path):
store = Store(tmp_path)
def test_latest_is_the_newest_completed_execution_of_the_work(make_store):
store = make_store()
kept(store, "a1")
kept(store, "a2")
kept(store, "a3", status="completed_with_warnings")
@@ -101,6 +121,53 @@ def test_latest_is_the_newest_completed_execution_of_the_work(tmp_path):
]
def test_a_delivered_file_named_like_a_record_is_not_taken_for_one(make_store):
store = make_store()
kept(store, files={"record.json": '{"not": "a record"}'})
assert [r["execution_id"] for r in store.case("t1", "acme")] == ["a1"]
def test_object_storage_keeps_the_same_layout_under_its_prefix():
with mock_aws():
client = boto3.client("s3", region_name="us-east-1")
kept(Store(S3Blobs(client, "kept", prefix="orchestration/", create=True)))
keys = sorted(
item["Key"] for item in client.list_objects_v2(Bucket="kept")["Contents"]
)
base = "orchestration/tenants/t1/cases/acme/executions/a1"
assert keys == [
f"{base}/files/graph.json",
f"{base}/files/guide.md",
f"{base}/files/terraform/main.tf",
f"{base}/record.json",
]
def test_the_settings_choose_the_storage(tmp_path, monkeypatch):
assert store_from_settings(OrchestrationSettings(_env_file=None)) is None # type: ignore[call-arg]
volume = store_from_settings(
OrchestrationSettings(_env_file=None, STORE_DIR=str(tmp_path)) # type: ignore[call-arg]
)
assert isinstance(volume, Store) and isinstance(volume.blobs, VolumeBlobs)
monkeypatch.setenv("AWS_ACCESS_KEY_ID", "test")
monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "test")
with mock_aws():
s3 = store_from_settings(
OrchestrationSettings( # type: ignore[call-arg]
_env_file=None,
STORE_BACKEND="s3",
S3_BUCKET="kept",
S3_REGION="us-east-1",
)
)
assert isinstance(s3, Store) and isinstance(s3.blobs, S3Blobs)
kept(s3)
assert s3.record("t1", "acme", "a1") is not None
def test_a_signed_path_is_accepted_and_a_changed_one_is_not():
signer = Signer("secret", ttl_seconds=60, now=lambda: 1000)