feat: add the hierarchical Execution engine on LangGraph
Increment 4 of .scratch/solution-architect-agent/spec.md, test-first. - app/agents/runtime.py: framework-neutral AgentRuntime interface (RoleSpec, AgentTask, ToolSpec, outcomes); fake_runtime.py replays scripted tool calls through the real tools with step budgets. - app/execution/graph.py: Orchestrator turn fans out to parallel Specialist tasks (Send), pauses with interrupt() for questions, Approvals and cloud-read permission, and finalizes or ends with warnings. Only the Orchestrator turn writes the Ledger. - app/execution/tools.py: Orchestrator tools, Specialist submit and needs_input, and the cloud-read permission gate. - app/execution/engine.py: ExecutionEngine.start/resume returning an Interaction; the single interface for A2A, /generate and eval. - Recorders (in-memory, DatabaseManager), configurable checkpointer (memory, sqlite, postgres), engine settings, langgraph checkpoint dependencies. - 19 new tests; full suite 121 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -37,6 +37,14 @@ ADK_ENABLE_ARTIFACT_STORE=true
|
||||
CLOUD_PROVIDER=gcp
|
||||
MCP_ENABLE_PREVIEW=true
|
||||
|
||||
# Hierarchical Execution Engine
|
||||
ORCHESTRATOR_BUDGET=40
|
||||
SPECIALIST_BUDGET=15
|
||||
MAX_REVISIONS=3
|
||||
MAX_ORCHESTRATOR_TURNS=100
|
||||
CHECKPOINTER=sqlite
|
||||
CHECKPOINT_DB_PATH=checkpoints.db
|
||||
|
||||
# Google Developer Knowledge MCP Integration
|
||||
DEVELOPER_KNOWLEDGE_MCP_URL=https://developerknowledge.googleapis.com/mcp
|
||||
DEVELOPER_KNOWLEDGE_MCP_ENABLED=true
|
||||
|
||||
@@ -569,3 +569,44 @@ Slowest tests: eval benchmark 7.3 s, then 3–4.5 s each for the orchestrator, e
|
||||
- `solution_guide`: the template's 8 top-level sections.
|
||||
- `architecture_diagram`: attachment `architecture.mmd`.
|
||||
- `technical_decomposition` and `architecture_description`: non-empty content only.
|
||||
|
||||
### 2026-09-15 · Increment 4: Execution graph
|
||||
|
||||
**Done (test-first)**
|
||||
- `app/agents/runtime.py`: the `AgentRuntime` protocol, `RoleSpec`, `AgentTask`, `ToolSpec`/`ToolResult`, and the outcomes `Submitted | NeedsInput | BudgetExhausted | Stopped | TurnEnded`.
|
||||
- `app/agents/fake_runtime.py`: `FakeAgentRuntime`, which replays scripted tool calls per role through the real `ToolSpec`s, enforces `RoleSpec.budget`, and records every run (task, tools offered, tool messages, outcome).
|
||||
- `app/execution/state.py`: `ExecutionState` (JSON-compatible). `outcomes` uses an append reducer, so parallel Specialist tasks never write the Ledger.
|
||||
- `app/execution/tools.py`:
|
||||
- `OrchestratorToolbox`: `ready_work`, `delegate`, `ask_user`, `request_approval`, `reopen`, `read_deliverable`, `finish`.
|
||||
- `specialist_tools`: `submit_deliverable` (checks the contract and refuses unsafe attachment paths) and `needs_input`, plus the role's extra tools.
|
||||
- `gate_cloud_read`: allow / refuse / ask permission.
|
||||
- `app/execution/graph.py`: `orchestrator_turn` → `Send` × N `specialist_task` → `orchestrator_turn`; `await_user` (`interrupt`); `finalize`; `end_with_warnings`.
|
||||
- `app/execution/engine.py`: **`ExecutionEngine.start(request, session_id, pre_approved, allow_cloud_read)` / `resume(execution_id, reply)` / `interaction(execution_id)` → `Interaction(status, prompt, state)`**. This is the one interface for A2A, `/generate` and the eval harness. The tests use only it.
|
||||
- `app/execution/recorder.py`: the `ExecutionRecorder` protocol with `InMemoryRecorder` and `DatabaseRecorder` (the existing tables via `asyncio.to_thread`).
|
||||
- `app/execution/checkpointing.py`: `open_checkpointer(settings)` for `memory | sqlite | postgres`.
|
||||
- Settings `ORCHESTRATOR_BUDGET`, `SPECIALIST_BUDGET`, `MAX_REVISIONS`, `MAX_ORCHESTRATOR_TURNS`, `CHECKPOINTER` (default `sqlite`) and `CHECKPOINT_DB_PATH`, plus `.env.example`. `requirements.txt`: `langgraph>=1.2`, `langgraph-checkpoint-sqlite`, `langgraph-checkpoint-postgres`, `psycopg[binary]`.
|
||||
- Tests:
|
||||
- `tests/execution/test_engine.py`: 17 tests covering the pre-approved happy path with written files, parallel start, dependency context, refusal messages, incomplete submissions, Approval pause/resume, free text never approving, changes → Revision → re-approval, one voice, Revision after a validation failure, pre-approved escalation, Orchestrator and Specialist budgets, cloud-read ask/grant/deny, clean Executions, and resuming from a durable SQLite checkpoint.
|
||||
- `tests/execution/test_recording.py`: 2 tests covering `DatabaseRecorder` against a temp SQLite database, and the checkpointer factory.
|
||||
- Full suite: **121 passed**. `ruff` and `mypy` are clean on `app/agents` and `app/execution`.
|
||||
- **Checking the tests bite.** I broke each of these in the code on purpose, and each break failed at least one test:
|
||||
- no cloud-read gate
|
||||
- permission never asked
|
||||
- Specialist questions dropped
|
||||
- submission contract unchecked
|
||||
|
||||
**Added to or changed from the spec text**
|
||||
- **Engine and file names.** There's a new `app/execution/engine.py` facade, and tests live in `test_engine.py`/`test_recording.py`, not `test_graph.py`. The graph is an internal detail behind the engine.
|
||||
- **`Stopped` outcome.** An agent that ends without a turn-ending tool is `Stopped`. Tools end a run by returning `ToolResult(outcome=...)`, so ending can depend on the result: pre-approved `request_approval` doesn't end the turn, and refusals never do. Engine-owned tools use `tier="execution"`.
|
||||
- **Prompt and reply shapes.**
|
||||
- `Interaction.prompt` is `{"questions": [{"from": <kind|orchestrator>, "question"}]}`, `{"approval": {"phase", "summary", "deliverables"}}` or `{"permission": {"tier", "requested_by"}}`.
|
||||
- `resume` takes `{"reply": str}`, `{"approval": {"phase", "decision": approve|changes, "feedback"}}` or `{"permission": {"cloud_read": bool}}`.
|
||||
- An approval reply that isn't structured goes into the conversation and never grants.
|
||||
- **Single writer.** Specialist outcomes are applied by the next `orchestrator_turn` in graph order, using `outcomes_applied`. A superseded result (stale attempt) is discarded and noted to the Orchestrator.
|
||||
- **Specialist problems as questions.** A Specialist that runs out of budget or stops is released and relayed as a question from its Deliverable, keeping one voice. A permission `NeedsInput` becomes a single permission prompt, not a question.
|
||||
- **Pre-approved mode.** `allow_cloud_read=None` becomes `False`. An escalation, or an Orchestrator that can't continue, ends as `completed_with_warnings` with no files written.
|
||||
- **Loop guard.** `MAX_ORCHESTRATOR_TURNS` (default 100) bounds loops, and the LangGraph `recursion_limit` is raised to 10 000.
|
||||
- **Conversation.** It records the Orchestrator's rendered prompts as well as user replies.
|
||||
- **Session name.** `DatabaseRecorder` session rows use `agent_name="solution-architect"`.
|
||||
- **Deferred to increment 5:** the design advisor's pillar fan-out tool, which needs the `pillar-advisor` role file.
|
||||
- **Not tested:** `AsyncPostgresSaver` against a real PostgreSQL (none runs locally). The import and wiring are in place.
|
||||
|
||||
1
app/agents/__init__.py
Normal file
1
app/agents/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Agent runtimes: the framework-neutral interface agents run behind, and its implementations."""
|
||||
89
app/agents/fake_runtime.py
Normal file
89
app/agents/fake_runtime.py
Normal file
@@ -0,0 +1,89 @@
|
||||
"""A scripted AgentRuntime for deterministic tests and evaluation dry runs.
|
||||
|
||||
Each role has a queue of turns; a turn is a list of tool calls, or a function of the
|
||||
task that returns one. Calls go through the real ToolSpecs, so every rule the tools
|
||||
enforce is exercised exactly as a model would meet it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import deque
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import (
|
||||
AgentTask,
|
||||
BudgetExhausted,
|
||||
Outcome,
|
||||
RoleSpec,
|
||||
Stopped,
|
||||
ToolSpec,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Call:
|
||||
tool: str
|
||||
args: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
Turn = Sequence[Call] | Callable[[AgentTask], Sequence[Call]]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RunRecord:
|
||||
role: str
|
||||
task: AgentTask
|
||||
tools: tuple[str, ...]
|
||||
results: tuple[tuple[str, str], ...] # (tool name, message)
|
||||
outcome: Outcome
|
||||
|
||||
|
||||
class ScriptExhausted(AssertionError):
|
||||
"""A role was run more times than the test scripted."""
|
||||
|
||||
|
||||
class FakeAgentRuntime:
|
||||
def __init__(self, turns: Mapping[str, Sequence[Turn]]) -> None:
|
||||
self._turns = {role: deque(role_turns) for role, role_turns in turns.items()}
|
||||
self.runs: list[RunRecord] = []
|
||||
|
||||
def remaining(self, role: str) -> int:
|
||||
return len(self._turns.get(role, ()))
|
||||
|
||||
async def run(
|
||||
self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]
|
||||
) -> Outcome:
|
||||
queue = self._turns.get(role.name)
|
||||
if not queue:
|
||||
raise ScriptExhausted(f"No scripted turn left for role {role.name!r}")
|
||||
turn = queue.popleft()
|
||||
calls = turn(task) if callable(turn) else turn
|
||||
by_name = {tool.name: tool for tool in tools}
|
||||
|
||||
results: list[tuple[str, str]] = []
|
||||
outcome: Outcome = Stopped(
|
||||
"The script ended without a tool that ends the turn."
|
||||
)
|
||||
for index, call in enumerate(calls):
|
||||
if index >= role.budget:
|
||||
outcome = BudgetExhausted(
|
||||
f"{role.name} used all {role.budget} tool calls before finishing."
|
||||
)
|
||||
break
|
||||
if call.tool not in by_name:
|
||||
raise ValueError(
|
||||
f"{role.name} called {call.tool!r}, which it was not offered: "
|
||||
f"{sorted(by_name)}"
|
||||
)
|
||||
result = await by_name[call.tool].handler(**call.args)
|
||||
results.append((call.tool, result.message))
|
||||
if result.outcome is not None:
|
||||
outcome = result.outcome
|
||||
break
|
||||
|
||||
self.runs.append(
|
||||
RunRecord(role.name, task, tuple(by_name), tuple(results), outcome)
|
||||
)
|
||||
return outcome
|
||||
103
app/agents/runtime.py
Normal file
103
app/agents/runtime.py
Normal file
@@ -0,0 +1,103 @@
|
||||
"""The agent-runtime interface (ADR-0003).
|
||||
|
||||
Everything an Orchestrator or Specialist needs crosses this seam as plain data:
|
||||
a RoleSpec, an AgentTask and ToolSpecs in; an Outcome back. No framework types.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RoleSpec:
|
||||
"""One agent role, parsed from agents/<role>.md."""
|
||||
|
||||
name: str
|
||||
instructions: str
|
||||
budget: int
|
||||
skills: tuple[str, ...] = ()
|
||||
tool_tiers: tuple[str, ...] = ()
|
||||
mcp_servers: tuple[str, ...] = ()
|
||||
model: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentTask:
|
||||
"""What one run of an agent is asked to do, with the context it may use."""
|
||||
|
||||
role: str
|
||||
prompt: str
|
||||
context: Mapping[str, Any] = field(default_factory=dict)
|
||||
deliverable: str | None = None
|
||||
|
||||
|
||||
# --- outcomes ----------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Submitted:
|
||||
"""A Specialist's contract-complete Deliverable."""
|
||||
|
||||
content: str
|
||||
attachments: Mapping[str, str] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NeedsInput:
|
||||
"""The agent can't continue without the user. `permission` names a tool tier to grant."""
|
||||
|
||||
questions: tuple[str, ...]
|
||||
permission: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BudgetExhausted:
|
||||
summary: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Stopped:
|
||||
"""The agent stopped without calling a tool that ends its turn."""
|
||||
|
||||
message: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TurnEnded:
|
||||
"""The Orchestrator ended its turn by calling `tool`; the tool recorded its effect."""
|
||||
|
||||
tool: str
|
||||
|
||||
|
||||
Outcome = Submitted | NeedsInput | BudgetExhausted | Stopped | TurnEnded
|
||||
|
||||
|
||||
# --- tools -------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ToolResult:
|
||||
"""What a tool returns to the agent. A non-None `outcome` ends the agent's run."""
|
||||
|
||||
message: str
|
||||
outcome: Outcome | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ToolSpec:
|
||||
name: str
|
||||
description: str
|
||||
parameters: Mapping[str, Any] # JSON schema for the keyword arguments
|
||||
handler: Callable[..., Awaitable[ToolResult]]
|
||||
tier: str = "general" # general | cloud_read | execution
|
||||
|
||||
|
||||
class AgentRuntime(Protocol):
|
||||
async def run(
|
||||
self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]
|
||||
) -> Outcome:
|
||||
"""Run one ReAct loop until a tool ends the turn, the budget is spent, or it stops."""
|
||||
...
|
||||
@@ -53,6 +53,14 @@ class Settings(BaseSettings):
|
||||
CLOUD_PROVIDER: str = "gcp"
|
||||
MCP_ENABLE_PREVIEW: bool = True
|
||||
|
||||
# Hierarchical Execution Engine (LangGraph state, ReAct agents)
|
||||
ORCHESTRATOR_BUDGET: int = 40
|
||||
SPECIALIST_BUDGET: int = 15
|
||||
MAX_REVISIONS: int = 3
|
||||
MAX_ORCHESTRATOR_TURNS: int = 100
|
||||
CHECKPOINTER: str = "sqlite" # memory | sqlite | postgres
|
||||
CHECKPOINT_DB_PATH: str = "checkpoints.db"
|
||||
|
||||
# Google Developer Knowledge MCP Server Settings
|
||||
DEVELOPER_KNOWLEDGE_MCP_URL: str = "https://developerknowledge.googleapis.com/mcp"
|
||||
DEVELOPER_KNOWLEDGE_MCP_ENABLED: bool = True
|
||||
|
||||
38
app/execution/checkpointing.py
Normal file
38
app/execution/checkpointing.py
Normal file
@@ -0,0 +1,38 @@
|
||||
"""The configured LangGraph checkpointer that makes paused Executions resumable."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
|
||||
from app.config import Settings
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def open_checkpointer(settings: Settings) -> AsyncIterator[BaseCheckpointSaver]:
|
||||
"""Yield a checkpointer per CHECKPOINTER: memory | sqlite | postgres."""
|
||||
kind = settings.CHECKPOINTER
|
||||
if kind == "memory":
|
||||
yield InMemorySaver()
|
||||
elif kind == "sqlite":
|
||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
|
||||
async with AsyncSqliteSaver.from_conn_string(
|
||||
settings.CHECKPOINT_DB_PATH
|
||||
) as saver:
|
||||
yield saver
|
||||
elif kind == "postgres":
|
||||
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver
|
||||
|
||||
async with AsyncPostgresSaver.from_conn_string(
|
||||
settings.get_database_url()
|
||||
) as saver:
|
||||
await saver.setup()
|
||||
yield saver
|
||||
else:
|
||||
raise ValueError(
|
||||
f"CHECKPOINTER must be memory, sqlite or postgres, not {kind!r}"
|
||||
)
|
||||
145
app/execution/engine.py
Normal file
145
app/execution/engine.py
Normal file
@@ -0,0 +1,145 @@
|
||||
"""ExecutionEngine: start and resume Executions.
|
||||
|
||||
The one interface the A2A server, /generate and the evaluation harness use. Each
|
||||
call runs the Execution until it needs the user or finishes, and returns an
|
||||
Interaction describing which.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||
from langgraph.types import Command
|
||||
|
||||
from app.agents.runtime import AgentRuntime, RoleSpec
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH
|
||||
from app.execution.graph import ToolFactory, build_execution_graph
|
||||
from app.execution.ledger import Ledger
|
||||
from app.execution.recorder import ExecutionRecorder
|
||||
from app.execution.tools import ORCHESTRATOR
|
||||
|
||||
_RECURSION_LIMIT = 10_000 # loops are bounded by max_orchestrator_turns instead
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Interaction:
|
||||
execution_id: str
|
||||
status: str # awaiting_user | completed | completed_with_warnings
|
||||
prompt: Mapping[str, Any] | None # questions | approval | permission
|
||||
state: Mapping[str, Any]
|
||||
|
||||
|
||||
class ExecutionNotAwaitingUser(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class ExecutionEngine:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
recorder: ExecutionRecorder,
|
||||
checkpointer: BaseCheckpointSaver,
|
||||
output_root: Path,
|
||||
max_revisions: int = 3,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> None:
|
||||
required = {
|
||||
ORCHESTRATOR,
|
||||
*(spec.specialist for spec in DEPENDENCY_GRAPH.values()),
|
||||
}
|
||||
missing = sorted(required - set(roles))
|
||||
if missing:
|
||||
raise ValueError(f"Missing roles: {', '.join(missing)}")
|
||||
self._max_revisions = max_revisions
|
||||
self._graph = build_execution_graph(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder,
|
||||
output_root=output_root,
|
||||
max_orchestrator_turns=max_orchestrator_turns,
|
||||
tool_factory=tool_factory,
|
||||
).compile(checkpointer=checkpointer)
|
||||
|
||||
async def start(
|
||||
self,
|
||||
workflow_request: str,
|
||||
*,
|
||||
session_id: str | None = None,
|
||||
pre_approved: bool = False,
|
||||
allow_cloud_read: bool | None = None,
|
||||
) -> Interaction:
|
||||
execution_id = str(uuid.uuid4())
|
||||
if pre_approved and allow_cloud_read is None:
|
||||
allow_cloud_read = False
|
||||
initial: dict[str, Any] = {
|
||||
"execution_id": execution_id,
|
||||
"session_id": session_id or str(uuid.uuid4()),
|
||||
"workflow_request": workflow_request,
|
||||
"pre_approved": pre_approved,
|
||||
"allow_cloud_read": allow_cloud_read,
|
||||
"ledger": Ledger(
|
||||
pre_approved=pre_approved, max_revisions=self._max_revisions
|
||||
).to_dict(),
|
||||
"deliverables": {},
|
||||
"conversation": [{"from": "user", "text": workflow_request}],
|
||||
"notes": [],
|
||||
"pending_questions": [],
|
||||
"awaiting_questions": [],
|
||||
"answers": {},
|
||||
"pending_permission": None,
|
||||
"prompt": None,
|
||||
"outcomes": [],
|
||||
"outcomes_applied": 0,
|
||||
"events_recorded": 0,
|
||||
"turns": 0,
|
||||
"status": "running",
|
||||
"warnings": [],
|
||||
}
|
||||
await self._graph.ainvoke(initial, self._config(execution_id))
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
|
||||
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
|
||||
current = await self.interaction(execution_id)
|
||||
if current.status != "awaiting_user":
|
||||
raise ExecutionNotAwaitingUser(
|
||||
f"Execution {execution_id} is {current.status}, not awaiting the user"
|
||||
)
|
||||
await self._graph.ainvoke(
|
||||
Command(resume=dict(reply)), self._config(execution_id)
|
||||
)
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def interaction(self, execution_id: str) -> Interaction:
|
||||
snapshot = await self._graph.aget_state(self._config(execution_id))
|
||||
if not snapshot.values:
|
||||
raise KeyError(f"Unknown Execution {execution_id}")
|
||||
if snapshot.interrupts:
|
||||
return Interaction(
|
||||
execution_id,
|
||||
"awaiting_user",
|
||||
snapshot.interrupts[0].value,
|
||||
snapshot.values,
|
||||
)
|
||||
return Interaction(
|
||||
execution_id,
|
||||
snapshot.values.get("status", "running"),
|
||||
None,
|
||||
snapshot.values,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _config(execution_id: str) -> RunnableConfig:
|
||||
return {
|
||||
"configurable": {"thread_id": execution_id},
|
||||
"recursion_limit": _RECURSION_LIMIT,
|
||||
}
|
||||
478
app/execution/graph.py
Normal file
478
app/execution/graph.py
Normal file
@@ -0,0 +1,478 @@
|
||||
"""The LangGraph that holds an Execution (ADR-0003).
|
||||
|
||||
START -> orchestrator_turn
|
||||
orchestrator_turn -> specialist_task x N (Send, parallel) -> orchestrator_turn
|
||||
orchestrator_turn -> await_user (interrupt) -> orchestrator_turn
|
||||
orchestrator_turn -> finalize -> END
|
||||
orchestrator_turn -> end_with_warnings -> END
|
||||
|
||||
Only orchestrator_turn and await_user change the Ledger; parallel Specialist tasks
|
||||
append outcomes that the next Orchestrator turn applies in order.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from langgraph.graph import END, START, StateGraph
|
||||
from langgraph.types import Command, Send, interrupt
|
||||
|
||||
from app.agents.runtime import (
|
||||
AgentRuntime,
|
||||
AgentTask,
|
||||
BudgetExhausted,
|
||||
NeedsInput,
|
||||
Outcome,
|
||||
RoleSpec,
|
||||
Stopped,
|
||||
Submitted,
|
||||
ToolSpec,
|
||||
TurnEnded,
|
||||
)
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH, PHASE_ORDER, DeliverableKind
|
||||
from app.execution.ledger import Ledger, LedgerRefusal
|
||||
from app.execution.recorder import ExecutionRecorder
|
||||
from app.execution.state import ExecutionState
|
||||
from app.execution.tools import ORCHESTRATOR, OrchestratorToolbox, specialist_tools
|
||||
|
||||
ToolFactory = Callable[[RoleSpec, str | None], Sequence[ToolSpec]]
|
||||
|
||||
NO_USER_REPLY = (
|
||||
"No user is available. State your assumption, record it in the Deliverable under "
|
||||
"'Assumptions', and proceed."
|
||||
)
|
||||
GUIDE_FILENAME = "solution-architecture-guide.md"
|
||||
|
||||
|
||||
def _no_tools(role: RoleSpec, deliverable: str | None) -> Sequence[ToolSpec]:
|
||||
return ()
|
||||
|
||||
|
||||
def build_execution_graph(
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
recorder: ExecutionRecorder,
|
||||
output_root: Path,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> StateGraph:
|
||||
tool_factory = tool_factory or _no_tools
|
||||
|
||||
async def record_progress(
|
||||
state: ExecutionState, ledger: Ledger, status: str | None = None
|
||||
) -> int:
|
||||
recorded = state.get("events_recorded", 0)
|
||||
await recorder.record_events(state["execution_id"], ledger.events[recorded:])
|
||||
if status:
|
||||
await recorder.record_status(
|
||||
state["execution_id"],
|
||||
session_id=state["session_id"],
|
||||
workflow_request=state["workflow_request"],
|
||||
status=status,
|
||||
current_phase=_current_phase(ledger),
|
||||
details={
|
||||
"revisions": sum(
|
||||
ledger.revisions(kind) for kind in DEPENDENCY_GRAPH
|
||||
),
|
||||
"warnings": state.get("warnings", []),
|
||||
},
|
||||
)
|
||||
return len(ledger.events)
|
||||
|
||||
async def orchestrator_turn(state: ExecutionState) -> Command:
|
||||
ledger = Ledger.from_dict(state["ledger"])
|
||||
deliverables = dict(state.get("deliverables", {}))
|
||||
pending_questions = list(state.get("pending_questions", []))
|
||||
notes = list(state.get("notes", []))
|
||||
pending_permission = state.get("pending_permission")
|
||||
warnings = list(state.get("warnings", []))
|
||||
|
||||
outcomes = state.get("outcomes", [])
|
||||
for record in outcomes[state.get("outcomes_applied", 0) :]:
|
||||
kind = DeliverableKind(record["kind"])
|
||||
try:
|
||||
if record["type"] == "submitted":
|
||||
ledger.submit(kind, record["attempt"], actor=record["role"])
|
||||
deliverables[kind.value] = {
|
||||
"content": record["content"],
|
||||
"attachments": record["attachments"],
|
||||
}
|
||||
await recorder.record_submission(
|
||||
state["execution_id"],
|
||||
kind.value,
|
||||
record["content"],
|
||||
record["attachments"],
|
||||
)
|
||||
continue
|
||||
ledger.release(kind, record["attempt"], reason=record["type"])
|
||||
except LedgerRefusal as exc:
|
||||
notes.append(f"Discarded a superseded result for {kind}: {exc}")
|
||||
continue
|
||||
if record["type"] == "needs_input" and record.get("permission"):
|
||||
if state.get("allow_cloud_read") is None:
|
||||
requested = (pending_permission or {}).get("requested_by", [])
|
||||
pending_permission = {
|
||||
"tier": record["permission"],
|
||||
"requested_by": [*requested, kind.value],
|
||||
}
|
||||
continue
|
||||
questions = record.get("questions") or [
|
||||
(
|
||||
f"The {record['role']} stopped: {record['summary']} "
|
||||
"How should it proceed?"
|
||||
)
|
||||
]
|
||||
pending_questions.append({"kind": kind.value, "questions": questions})
|
||||
|
||||
turns = state.get("turns", 0) + 1
|
||||
update: dict[str, Any] = {
|
||||
"deliverables": deliverables,
|
||||
"pending_questions": pending_questions,
|
||||
"pending_permission": pending_permission,
|
||||
"outcomes_applied": len(outcomes),
|
||||
"turns": turns,
|
||||
}
|
||||
|
||||
def finish_with(goto: str, **extra: Any) -> Command:
|
||||
return Command(
|
||||
update={**update, "ledger": ledger.to_dict(), "notes": notes, **extra},
|
||||
goto=goto,
|
||||
)
|
||||
|
||||
if pending_permission and not state.get("pre_approved"):
|
||||
update["events_recorded"] = await record_progress(
|
||||
state, ledger, "awaiting_user"
|
||||
)
|
||||
return finish_with("await_user", prompt={"permission": pending_permission})
|
||||
|
||||
if turns > max_orchestrator_turns:
|
||||
warnings.append(
|
||||
f"Stopped after {max_orchestrator_turns} Orchestrator turns without "
|
||||
"finishing."
|
||||
)
|
||||
return finish_with("end_with_warnings", warnings=warnings)
|
||||
|
||||
toolbox = OrchestratorToolbox(ledger, deliverables, pending_questions)
|
||||
task = AgentTask(
|
||||
role=ORCHESTRATOR,
|
||||
prompt="Decide the next step for this Execution using your tools.",
|
||||
context=_orchestrator_context(state, ledger, pending_questions, notes),
|
||||
)
|
||||
outcome = await runtime.run(roles[ORCHESTRATOR], task, toolbox.tools())
|
||||
notes = []
|
||||
pre_approved = bool(state.get("pre_approved"))
|
||||
conversation = list(state.get("conversation", []))
|
||||
|
||||
def ask(prompt: dict[str, Any], **extra: Any) -> Command:
|
||||
conversation.append({"from": ORCHESTRATOR, "text": _render_prompt(prompt)})
|
||||
return finish_with(
|
||||
"await_user", prompt=prompt, conversation=conversation, **extra
|
||||
)
|
||||
|
||||
if isinstance(outcome, TurnEnded) and outcome.tool == "delegate":
|
||||
sends = [
|
||||
Send(
|
||||
"specialist_task",
|
||||
_specialist_input(state, ledger, deliverables, kind, attempt),
|
||||
)
|
||||
for kind, attempt in toolbox.started.items()
|
||||
]
|
||||
update["events_recorded"] = await record_progress(state, ledger)
|
||||
return Command(
|
||||
update={**update, "ledger": ledger.to_dict(), "notes": notes},
|
||||
goto=sends,
|
||||
)
|
||||
|
||||
if isinstance(outcome, TurnEnded) and outcome.tool == "finish":
|
||||
update["events_recorded"] = await record_progress(state, ledger)
|
||||
return finish_with("finalize")
|
||||
|
||||
if isinstance(outcome, TurnEnded) and outcome.tool == "escalate":
|
||||
warnings.append(toolbox.escalation or "Escalated without detail.")
|
||||
update["events_recorded"] = await record_progress(state, ledger)
|
||||
return finish_with("end_with_warnings", warnings=warnings)
|
||||
|
||||
if isinstance(outcome, TurnEnded) and outcome.tool == "request_approval":
|
||||
update["events_recorded"] = await record_progress(
|
||||
state, ledger, None if pre_approved else "awaiting_user"
|
||||
)
|
||||
return ask({"approval": toolbox.approval})
|
||||
|
||||
if isinstance(outcome, TurnEnded) and outcome.tool == "ask_user":
|
||||
update["events_recorded"] = await record_progress(
|
||||
state, ledger, None if pre_approved else "awaiting_user"
|
||||
)
|
||||
return ask(
|
||||
{"questions": toolbox.questions},
|
||||
pending_questions=[],
|
||||
awaiting_questions=pending_questions,
|
||||
)
|
||||
|
||||
message = _describe_unfinished(outcome)
|
||||
if pre_approved:
|
||||
warnings.append(f"The Orchestrator could not continue: {message}")
|
||||
update["events_recorded"] = await record_progress(state, ledger)
|
||||
return finish_with("end_with_warnings", warnings=warnings)
|
||||
update["events_recorded"] = await record_progress(
|
||||
state, ledger, "awaiting_user"
|
||||
)
|
||||
return ask({"questions": [{"from": ORCHESTRATOR, "question": message}]})
|
||||
|
||||
async def specialist_task(task_input: dict[str, Any]) -> dict[str, Any]:
|
||||
role = roles[task_input["role"]]
|
||||
kind = DeliverableKind(task_input["kind"])
|
||||
tools = specialist_tools(
|
||||
kind,
|
||||
extra=tool_factory(role, kind.value),
|
||||
allow_cloud_read=task_input["allow_cloud_read"],
|
||||
)
|
||||
task = AgentTask(
|
||||
role=role.name,
|
||||
prompt=f"Produce the {kind} Deliverable.",
|
||||
context=task_input["context"],
|
||||
deliverable=kind.value,
|
||||
)
|
||||
outcome = await runtime.run(role, task, tools)
|
||||
return {
|
||||
"outcomes": [
|
||||
_outcome_record(role.name, kind, task_input["attempt"], outcome)
|
||||
]
|
||||
}
|
||||
|
||||
async def await_user(state: ExecutionState) -> Command:
|
||||
prompt = state["prompt"] or {}
|
||||
if state.get("pre_approved"):
|
||||
reply = _automatic_reply(prompt)
|
||||
else:
|
||||
reply = interrupt(prompt)
|
||||
|
||||
ledger = Ledger.from_dict(state["ledger"])
|
||||
conversation = list(state.get("conversation", []))
|
||||
update: dict[str, Any] = {"prompt": None}
|
||||
|
||||
if "approval" in prompt:
|
||||
phase = prompt["approval"]["phase"]
|
||||
decision = reply.get("approval") if isinstance(reply, Mapping) else None
|
||||
if (
|
||||
isinstance(decision, Mapping)
|
||||
and decision.get("phase") == phase
|
||||
and decision.get("decision") in {"approve", "changes"}
|
||||
):
|
||||
phase_id = next(p for p in PHASE_ORDER if p.value == phase)
|
||||
if decision["decision"] == "approve":
|
||||
ledger.approve(phase_id, actor="user")
|
||||
text = f"Approved {phase}."
|
||||
else:
|
||||
feedback = str(decision.get("feedback", ""))
|
||||
ledger.request_changes(phase_id, feedback, actor="user")
|
||||
text = f"Requested changes to {phase}: {feedback}"
|
||||
conversation.append({"from": "user", "text": text})
|
||||
else:
|
||||
conversation.append({"from": "user", "text": _reply_text(reply)})
|
||||
elif "permission" in prompt:
|
||||
tier = prompt["permission"]["tier"]
|
||||
permission = reply.get("permission") if isinstance(reply, Mapping) else None
|
||||
grant = permission.get(tier) if isinstance(permission, Mapping) else None
|
||||
if isinstance(grant, bool):
|
||||
update["allow_cloud_read"] = grant
|
||||
update["pending_permission"] = None
|
||||
verdict = "granted" if grant else "denied"
|
||||
conversation.append(
|
||||
{
|
||||
"from": "user",
|
||||
"text": f"Permission to use {tier} tools {verdict}.",
|
||||
}
|
||||
)
|
||||
else:
|
||||
conversation.append({"from": "user", "text": _reply_text(reply)})
|
||||
else:
|
||||
text = _reply_text(reply)
|
||||
conversation.append({"from": "user", "text": text})
|
||||
answers = {
|
||||
kind: list(items) for kind, items in state.get("answers", {}).items()
|
||||
}
|
||||
for asked in state.get("awaiting_questions", []):
|
||||
answers.setdefault(asked["kind"], []).append(
|
||||
{"questions": list(asked["questions"]), "reply": text}
|
||||
)
|
||||
update.update(answers=answers, awaiting_questions=[])
|
||||
|
||||
update["events_recorded"] = await record_progress(state, ledger)
|
||||
update.update(ledger=ledger.to_dict(), conversation=conversation)
|
||||
return Command(update=update, goto="orchestrator_turn")
|
||||
|
||||
async def finalize(state: ExecutionState) -> dict[str, Any]:
|
||||
ledger = Ledger.from_dict(state["ledger"])
|
||||
_write_outputs(
|
||||
output_root / "deliverables" / "executions" / state["execution_id"],
|
||||
state.get("deliverables", {}),
|
||||
)
|
||||
await record_progress(state, ledger, "completed")
|
||||
return {"status": "completed"}
|
||||
|
||||
async def end_with_warnings(state: ExecutionState) -> dict[str, Any]:
|
||||
ledger = Ledger.from_dict(state["ledger"])
|
||||
await record_progress(state, ledger, "completed_with_warnings")
|
||||
return {"status": "completed_with_warnings"}
|
||||
|
||||
graph = StateGraph(ExecutionState)
|
||||
graph.add_node("orchestrator_turn", orchestrator_turn)
|
||||
# Specialist tasks receive their Send payload, not the Execution state.
|
||||
graph.add_node("specialist_task", cast(Any, specialist_task))
|
||||
graph.add_node("await_user", await_user)
|
||||
graph.add_node("finalize", finalize)
|
||||
graph.add_node("end_with_warnings", end_with_warnings)
|
||||
graph.add_edge(START, "orchestrator_turn")
|
||||
graph.add_edge("specialist_task", "orchestrator_turn")
|
||||
graph.add_edge("finalize", END)
|
||||
graph.add_edge("end_with_warnings", END)
|
||||
return graph
|
||||
|
||||
|
||||
# --- helpers ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _current_phase(ledger: Ledger) -> str:
|
||||
for phase in PHASE_ORDER:
|
||||
if ledger.phase_status(phase).value != "approved":
|
||||
return phase.value
|
||||
return PHASE_ORDER[-1].value
|
||||
|
||||
|
||||
def _orchestrator_context(
|
||||
state: ExecutionState,
|
||||
ledger: Ledger,
|
||||
pending_questions: Sequence[Mapping[str, Any]],
|
||||
notes: Sequence[str],
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"workflow_request": state["workflow_request"],
|
||||
"pre_approved": bool(state.get("pre_approved")),
|
||||
"allow_cloud_read": state.get("allow_cloud_read"),
|
||||
"deliverables": {
|
||||
kind.value: {
|
||||
"phase": spec.phase.value,
|
||||
"status": ledger.status(kind).value,
|
||||
"revisions": ledger.revisions(kind),
|
||||
"feedback": list(ledger.feedback(kind)),
|
||||
}
|
||||
for kind, spec in DEPENDENCY_GRAPH.items()
|
||||
},
|
||||
"phases": {
|
||||
phase.value: ledger.phase_status(phase).value for phase in PHASE_ORDER
|
||||
},
|
||||
"ready_work": [kind.value for kind in ledger.ready_work()],
|
||||
"phases_ready_for_approval": [
|
||||
p.value for p in ledger.phases_ready_for_approval()
|
||||
],
|
||||
"pending_questions": list(pending_questions),
|
||||
"conversation": list(state.get("conversation", []))[-20:],
|
||||
"notes": list(notes),
|
||||
}
|
||||
|
||||
|
||||
def _specialist_input(
|
||||
state: ExecutionState,
|
||||
ledger: Ledger,
|
||||
deliverables: Mapping[str, Mapping[str, Any]],
|
||||
kind: DeliverableKind,
|
||||
attempt: int,
|
||||
) -> dict[str, Any]:
|
||||
spec = DEPENDENCY_GRAPH[kind]
|
||||
return {
|
||||
"role": spec.specialist,
|
||||
"kind": kind.value,
|
||||
"attempt": attempt,
|
||||
"allow_cloud_read": state.get("allow_cloud_read"),
|
||||
"context": {
|
||||
"workflow_request": state["workflow_request"],
|
||||
"pre_approved": bool(state.get("pre_approved")),
|
||||
"dependencies": {
|
||||
dependency.value: dict(deliverables[dependency.value])
|
||||
for dependency in spec.depends_on
|
||||
},
|
||||
"feedback": list(ledger.feedback(kind)),
|
||||
"answers": list(state.get("answers", {}).get(kind.value, [])),
|
||||
"attempt": attempt,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _outcome_record(
|
||||
role: str, kind: DeliverableKind, attempt: int, outcome: Outcome
|
||||
) -> dict[str, Any]:
|
||||
base = {"role": role, "kind": kind.value, "attempt": attempt}
|
||||
if isinstance(outcome, Submitted):
|
||||
return {
|
||||
**base,
|
||||
"type": "submitted",
|
||||
"content": outcome.content,
|
||||
"attachments": dict(outcome.attachments),
|
||||
}
|
||||
if isinstance(outcome, NeedsInput):
|
||||
return {
|
||||
**base,
|
||||
"type": "needs_input",
|
||||
"questions": list(outcome.questions),
|
||||
"permission": outcome.permission,
|
||||
}
|
||||
if isinstance(outcome, BudgetExhausted):
|
||||
return {**base, "type": "budget_exhausted", "summary": outcome.summary}
|
||||
return {**base, "type": "stopped", "summary": _describe_unfinished(outcome)}
|
||||
|
||||
|
||||
def _describe_unfinished(outcome: Outcome) -> str:
|
||||
if isinstance(outcome, BudgetExhausted):
|
||||
return f"I paused: {outcome.summary} How should I proceed?"
|
||||
if isinstance(outcome, Stopped):
|
||||
return outcome.message
|
||||
if isinstance(outcome, NeedsInput):
|
||||
return " ".join(outcome.questions)
|
||||
return f"Ended with an unexpected outcome: {type(outcome).__name__}."
|
||||
|
||||
|
||||
def _automatic_reply(prompt: Mapping[str, Any]) -> dict[str, Any]:
|
||||
if "approval" in prompt:
|
||||
return {
|
||||
"approval": {"phase": prompt["approval"]["phase"], "decision": "approve"}
|
||||
}
|
||||
if "permission" in prompt:
|
||||
return {"permission": {prompt["permission"]["tier"]: False}}
|
||||
return {"reply": NO_USER_REPLY}
|
||||
|
||||
|
||||
def _reply_text(reply: object) -> str:
|
||||
if isinstance(reply, str):
|
||||
return reply
|
||||
if isinstance(reply, Mapping) and isinstance(reply.get("reply"), str):
|
||||
return reply["reply"]
|
||||
return str(reply)
|
||||
|
||||
|
||||
def _render_prompt(prompt: Mapping[str, Any]) -> str:
|
||||
if "approval" in prompt:
|
||||
approval = prompt["approval"]
|
||||
return f"Please approve {approval['phase']}: {approval['summary']}"
|
||||
if "permission" in prompt:
|
||||
return f"May I use {prompt['permission']['tier']} tools?"
|
||||
return "\n".join(item["question"] for item in prompt.get("questions", []))
|
||||
|
||||
|
||||
def _write_outputs(
|
||||
out_dir: Path, deliverables: Mapping[str, Mapping[str, Any]]
|
||||
) -> None:
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
guide = deliverables.get(DeliverableKind.SOLUTION_GUIDE.value)
|
||||
if guide:
|
||||
(out_dir / GUIDE_FILENAME).write_text(guide["content"], encoding="utf-8")
|
||||
for kind, deliverable in deliverables.items():
|
||||
path = out_dir / "deliverables" / f"{kind}.md"
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(deliverable["content"], encoding="utf-8")
|
||||
for name, content in deliverable.get("attachments", {}).items():
|
||||
attachment = out_dir / name
|
||||
attachment.parent.mkdir(parents=True, exist_ok=True)
|
||||
attachment.write_text(content, encoding="utf-8")
|
||||
156
app/execution/recorder.py
Normal file
156
app/execution/recorder.py
Normal file
@@ -0,0 +1,156 @@
|
||||
"""Recording an Execution's progress for audit and reporting.
|
||||
|
||||
Two adapters sit behind ExecutionRecorder: InMemoryRecorder for tests, and
|
||||
DatabaseRecorder, which reuses the existing DatabaseManager tables until the
|
||||
Execution store (review candidate 03) replaces them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any, Protocol
|
||||
|
||||
from app.execution.ledger import LedgerEvent
|
||||
|
||||
SESSION_AGENT_NAME = "solution-architect"
|
||||
|
||||
|
||||
class ExecutionRecorder(Protocol):
|
||||
async def record_events(
|
||||
self, execution_id: str, events: Sequence[LedgerEvent]
|
||||
) -> None: ...
|
||||
|
||||
async def record_submission(
|
||||
self,
|
||||
execution_id: str,
|
||||
kind: str,
|
||||
content: str,
|
||||
attachments: Mapping[str, str],
|
||||
) -> None: ...
|
||||
|
||||
async def record_status(
|
||||
self,
|
||||
execution_id: str,
|
||||
*,
|
||||
session_id: str,
|
||||
workflow_request: str,
|
||||
status: str,
|
||||
current_phase: str,
|
||||
details: Mapping[str, Any],
|
||||
) -> None: ...
|
||||
|
||||
|
||||
class InMemoryRecorder:
|
||||
def __init__(self) -> None:
|
||||
self.events: dict[str, list[LedgerEvent]] = defaultdict(list)
|
||||
self.submissions: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||
self.statuses: dict[str, list[dict[str, Any]]] = defaultdict(list)
|
||||
|
||||
async def record_events(
|
||||
self, execution_id: str, events: Sequence[LedgerEvent]
|
||||
) -> None:
|
||||
self.events[execution_id].extend(events)
|
||||
|
||||
async def record_submission(
|
||||
self,
|
||||
execution_id: str,
|
||||
kind: str,
|
||||
content: str,
|
||||
attachments: Mapping[str, str],
|
||||
) -> None:
|
||||
self.submissions[execution_id].append(
|
||||
{"kind": kind, "content": content, "attachments": dict(attachments)}
|
||||
)
|
||||
|
||||
async def record_status(
|
||||
self,
|
||||
execution_id: str,
|
||||
*,
|
||||
session_id: str,
|
||||
workflow_request: str,
|
||||
status: str,
|
||||
current_phase: str,
|
||||
details: Mapping[str, Any],
|
||||
) -> None:
|
||||
self.statuses[execution_id].append(
|
||||
{
|
||||
"session_id": session_id,
|
||||
"workflow_request": workflow_request,
|
||||
"status": status,
|
||||
"current_phase": current_phase,
|
||||
"details": dict(details),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class DatabaseRecorder:
|
||||
"""Writes to DatabaseManager (blocking) off the event loop."""
|
||||
|
||||
def __init__(self, db_manager: Any = None) -> None:
|
||||
if db_manager is None:
|
||||
from app.database import get_db_manager
|
||||
|
||||
db_manager = get_db_manager()
|
||||
self._db = db_manager
|
||||
|
||||
async def record_events(
|
||||
self, execution_id: str, events: Sequence[LedgerEvent]
|
||||
) -> None:
|
||||
for event in events:
|
||||
await asyncio.to_thread(
|
||||
self._db.record_orchestrator_log,
|
||||
log_id=str(uuid.uuid4()),
|
||||
execution_id=execution_id,
|
||||
iteration=event.sequence,
|
||||
review_status=event.type,
|
||||
reviewer_agent=event.actor,
|
||||
feedback=f"{event.subject}: {event.detail}".rstrip(": "),
|
||||
)
|
||||
|
||||
async def record_submission(
|
||||
self,
|
||||
execution_id: str,
|
||||
kind: str,
|
||||
content: str,
|
||||
attachments: Mapping[str, str],
|
||||
) -> None:
|
||||
files = {f"deliverables/{kind}.md": content, **attachments}
|
||||
for file_path, file_content in files.items():
|
||||
await asyncio.to_thread(
|
||||
self._db.save_artifact,
|
||||
artifact_id=str(uuid.uuid4()),
|
||||
execution_id=execution_id,
|
||||
artifact_type=kind,
|
||||
file_path=file_path,
|
||||
content=file_content,
|
||||
)
|
||||
|
||||
async def record_status(
|
||||
self,
|
||||
execution_id: str,
|
||||
*,
|
||||
session_id: str,
|
||||
workflow_request: str,
|
||||
status: str,
|
||||
current_phase: str,
|
||||
details: Mapping[str, Any],
|
||||
) -> None:
|
||||
await asyncio.to_thread(
|
||||
self._db.save_workflow_execution,
|
||||
execution_id=execution_id,
|
||||
session_id=session_id,
|
||||
status=status,
|
||||
current_phase=current_phase,
|
||||
loop_count=int(details.get("revisions", 0)),
|
||||
request_summary=workflow_request,
|
||||
results=dict(details),
|
||||
)
|
||||
await asyncio.to_thread(
|
||||
self._db.save_session,
|
||||
session_id=session_id,
|
||||
agent_name=SESSION_AGENT_NAME,
|
||||
state_data={"latest_execution_id": execution_id, "status": status},
|
||||
)
|
||||
35
app/execution/state.py
Normal file
35
app/execution/state.py
Normal file
@@ -0,0 +1,35 @@
|
||||
"""The checkpointed state of one Execution. Plain, JSON-compatible data only."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import operator
|
||||
from typing import Annotated, Any, TypedDict
|
||||
|
||||
|
||||
class ExecutionState(TypedDict, total=False):
|
||||
execution_id: str
|
||||
session_id: str
|
||||
workflow_request: str
|
||||
pre_approved: bool
|
||||
allow_cloud_read: bool | None
|
||||
|
||||
ledger: dict[str, Any] # Ledger.to_dict()
|
||||
deliverables: dict[str, dict[str, Any]] # kind -> {content, attachments}
|
||||
|
||||
conversation: list[dict[str, str]] # {"from": "user" | "orchestrator", "text"}
|
||||
notes: list[str] # facts for the Orchestrator's next turn
|
||||
pending_questions: list[dict[str, Any]] # {kind, questions} not yet asked
|
||||
awaiting_questions: list[dict[str, Any]] # {kind, questions} asked, not answered
|
||||
answers: dict[str, list[dict[str, Any]]] # kind -> [{questions, reply}]
|
||||
pending_permission: dict[str, Any] | None # {tier, requested_by}
|
||||
prompt: dict[str, Any] | None # what the user is being asked
|
||||
|
||||
# Specialist outcomes, appended concurrently by parallel Specialist tasks and
|
||||
# applied to the Ledger by the next Orchestrator turn.
|
||||
outcomes: Annotated[list[dict[str, Any]], operator.add]
|
||||
outcomes_applied: int
|
||||
events_recorded: int
|
||||
|
||||
turns: int
|
||||
status: str # running | completed | completed_with_warnings
|
||||
warnings: list[str]
|
||||
318
app/execution/tools.py
Normal file
318
app/execution/tools.py
Normal file
@@ -0,0 +1,318 @@
|
||||
"""Execution tools: how agents act on an Execution.
|
||||
|
||||
Tools enforce the Ledger's rules at the moment an agent acts, and return refusals as
|
||||
messages so a ReAct agent can recover. A tool ends the agent's turn by returning an
|
||||
outcome.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping, Sequence
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import (
|
||||
NeedsInput,
|
||||
Submitted,
|
||||
ToolResult,
|
||||
ToolSpec,
|
||||
TurnEnded,
|
||||
)
|
||||
from app.execution.deliverables import (
|
||||
DEPENDENCY_GRAPH,
|
||||
DeliverableKind,
|
||||
PhaseId,
|
||||
check_submission,
|
||||
deliverables_in,
|
||||
)
|
||||
from app.execution.ledger import Ledger, LedgerRefusal, PhaseStatus
|
||||
|
||||
ORCHESTRATOR = "orchestrator"
|
||||
CLOUD_READ = "cloud_read"
|
||||
|
||||
_NO_ARGS: Mapping[str, Any] = {"type": "object", "properties": {}}
|
||||
|
||||
|
||||
def _schema(properties: Mapping[str, Any], required: Sequence[str]) -> dict[str, Any]:
|
||||
return {
|
||||
"type": "object",
|
||||
"properties": dict(properties),
|
||||
"required": list(required),
|
||||
}
|
||||
|
||||
|
||||
def _refused(reason: object) -> ToolResult:
|
||||
return ToolResult(f"Refused: {reason}")
|
||||
|
||||
|
||||
class OrchestratorToolbox:
|
||||
"""The Orchestrator's tools, bound to a working copy of the Ledger for one turn."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
ledger: Ledger,
|
||||
deliverables: Mapping[str, Mapping[str, Any]],
|
||||
pending_questions: Sequence[Mapping[str, Any]],
|
||||
) -> None:
|
||||
self.ledger = ledger
|
||||
self._deliverables = deliverables
|
||||
self._pending_questions = pending_questions
|
||||
self.started: dict[DeliverableKind, int] = {}
|
||||
self.questions: list[dict[str, str]] = []
|
||||
self.approval: dict[str, Any] | None = None
|
||||
self.escalation: str | None = None
|
||||
|
||||
def tools(self) -> list[ToolSpec]:
|
||||
return [
|
||||
ToolSpec(
|
||||
"ready_work",
|
||||
"List Deliverables ready to delegate and Phases ready for Approval.",
|
||||
_NO_ARGS,
|
||||
self.ready_work,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"delegate",
|
||||
"Start Specialists on ready Deliverables, in parallel. Ends your turn.",
|
||||
_schema(
|
||||
{"kinds": {"type": "array", "items": {"type": "string"}}},
|
||||
["kinds"],
|
||||
),
|
||||
self.delegate,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"ask_user",
|
||||
"Ask the user your questions together with any waiting Specialist "
|
||||
"questions. Ends your turn.",
|
||||
_schema(
|
||||
{"questions": {"type": "array", "items": {"type": "string"}}}, []
|
||||
),
|
||||
self.ask_user,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"request_approval",
|
||||
"Ask the user to approve a Phase whose Deliverables are all "
|
||||
"submitted. Ends your turn unless the Execution is pre-approved.",
|
||||
_schema(
|
||||
{"phase": {"type": "string"}, "summary": {"type": "string"}},
|
||||
["phase", "summary"],
|
||||
),
|
||||
self.request_approval,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"reopen",
|
||||
"Revise a submitted Deliverable; everything depending on it returns "
|
||||
"to draft.",
|
||||
_schema(
|
||||
{"kind": {"type": "string"}, "reason": {"type": "string"}},
|
||||
["kind", "reason"],
|
||||
),
|
||||
self.reopen,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"read_deliverable",
|
||||
"Read a submitted Deliverable's content.",
|
||||
_schema({"kind": {"type": "string"}}, ["kind"]),
|
||||
self.read_deliverable,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"finish",
|
||||
"Complete the Execution once packaging is approved. Ends your turn.",
|
||||
_NO_ARGS,
|
||||
self.finish,
|
||||
tier="execution",
|
||||
),
|
||||
]
|
||||
|
||||
async def ready_work(self) -> ToolResult:
|
||||
ready = ", ".join(self.ledger.ready_work()) or "none"
|
||||
phases = ", ".join(self.ledger.phases_ready_for_approval()) or "none"
|
||||
return ToolResult(
|
||||
f"Ready to delegate: {ready}. Phases ready for approval: {phases}."
|
||||
)
|
||||
|
||||
async def delegate(self, kinds: Sequence[str]) -> ToolResult:
|
||||
try:
|
||||
parsed = [DeliverableKind(kind) for kind in kinds]
|
||||
if not parsed:
|
||||
raise LedgerRefusal("name at least one Deliverable")
|
||||
started = self.ledger.start(parsed, actor=ORCHESTRATOR)
|
||||
except (ValueError, LedgerRefusal) as exc:
|
||||
return _refused(exc)
|
||||
self.started.update(started)
|
||||
return ToolResult(f"Started: {', '.join(started)}.", TurnEnded("delegate"))
|
||||
|
||||
async def ask_user(self, questions: Sequence[str] = ()) -> ToolResult:
|
||||
waiting = sorted(
|
||||
self._pending_questions,
|
||||
key=lambda item: list(DEPENDENCY_GRAPH).index(
|
||||
DeliverableKind(item["kind"])
|
||||
),
|
||||
)
|
||||
combined = [
|
||||
{"from": item["kind"], "question": question}
|
||||
for item in waiting
|
||||
for question in item["questions"]
|
||||
]
|
||||
combined += [{"from": ORCHESTRATOR, "question": q} for q in questions if q]
|
||||
if not combined:
|
||||
return _refused("there is nothing to ask the user")
|
||||
self.questions = combined
|
||||
return ToolResult("Asked the user.", TurnEnded("ask_user"))
|
||||
|
||||
async def request_approval(self, phase: str, summary: str) -> ToolResult:
|
||||
try:
|
||||
phase_id = PhaseId(phase)
|
||||
except ValueError as exc:
|
||||
return _refused(exc)
|
||||
if self.ledger.phase_status(phase_id) is not PhaseStatus.AWAITING_APPROVAL:
|
||||
try:
|
||||
self.ledger.request_approval(phase_id, actor=ORCHESTRATOR)
|
||||
except LedgerRefusal as exc:
|
||||
return _refused(exc)
|
||||
if self.ledger.phase_status(phase_id) is PhaseStatus.APPROVED:
|
||||
return ToolResult(
|
||||
f"Phase {phase_id} approved (pre-approved Execution)."
|
||||
)
|
||||
self.approval = {
|
||||
"phase": phase_id.value,
|
||||
"summary": summary,
|
||||
"deliverables": [kind.value for kind in deliverables_in(phase_id)],
|
||||
}
|
||||
return ToolResult(
|
||||
f"Asked the user to approve {phase_id}.", TurnEnded("request_approval")
|
||||
)
|
||||
|
||||
async def reopen(self, kind: str, reason: str) -> ToolResult:
|
||||
try:
|
||||
result = self.ledger.reopen(
|
||||
DeliverableKind(kind), reason, actor=ORCHESTRATOR
|
||||
)
|
||||
except (ValueError, LedgerRefusal) as exc:
|
||||
return _refused(exc)
|
||||
if result.escalation:
|
||||
if self.ledger.pre_approved:
|
||||
self.escalation = result.escalation
|
||||
return ToolResult(result.escalation, TurnEnded("escalate"))
|
||||
return ToolResult(
|
||||
f"{result.escalation} Use ask_user to put this to the user."
|
||||
)
|
||||
withdrawn = ", ".join(result.withdrawn_approvals) or "none"
|
||||
return ToolResult(
|
||||
f"Reopened {kind}. Returned to draft: {', '.join(result.reset)}. "
|
||||
f"Approvals withdrawn: {withdrawn}."
|
||||
)
|
||||
|
||||
async def read_deliverable(self, kind: str) -> ToolResult:
|
||||
deliverable = self._deliverables.get(kind)
|
||||
if deliverable is None:
|
||||
return ToolResult(f"{kind} has not been submitted.")
|
||||
return ToolResult(deliverable["content"])
|
||||
|
||||
async def finish(self) -> ToolResult:
|
||||
if not self.ledger.can_finish():
|
||||
return _refused("packaging has not been approved")
|
||||
return ToolResult("Finishing the Execution.", TurnEnded("finish"))
|
||||
|
||||
|
||||
def _safe_relative(name: str) -> bool:
|
||||
path = PurePosixPath(name)
|
||||
return bool(name) and not path.is_absolute() and ".." not in path.parts
|
||||
|
||||
|
||||
def specialist_tools(
|
||||
kind: DeliverableKind,
|
||||
extra: Sequence[ToolSpec],
|
||||
allow_cloud_read: bool | None,
|
||||
) -> list[ToolSpec]:
|
||||
"""A Specialist's tools: submit, ask, and its role's tools with the cloud-read gate."""
|
||||
|
||||
async def submit_deliverable(
|
||||
content: str, attachments: Mapping[str, str] | None = None
|
||||
) -> ToolResult:
|
||||
attachments = dict(attachments or {})
|
||||
unsafe = [name for name in attachments if not _safe_relative(name)]
|
||||
if unsafe:
|
||||
return _refused(
|
||||
"attachment names must be relative paths without '..': "
|
||||
+ ", ".join(unsafe)
|
||||
)
|
||||
missing = check_submission(kind, content, attachments)
|
||||
if missing:
|
||||
return ToolResult(
|
||||
"Not submitted. The Deliverable is missing: " + "; ".join(missing)
|
||||
)
|
||||
return ToolResult(f"Submitted {kind}.", Submitted(content, attachments))
|
||||
|
||||
async def needs_input(questions: Sequence[str]) -> ToolResult:
|
||||
questions = tuple(q for q in questions if q)
|
||||
if not questions:
|
||||
return _refused("ask at least one question")
|
||||
return ToolResult("Questions sent to the user.", NeedsInput(questions))
|
||||
|
||||
return [
|
||||
ToolSpec(
|
||||
"submit_deliverable",
|
||||
f"Submit the {kind} Deliverable once it meets its contract. Ends your turn.",
|
||||
_schema(
|
||||
{
|
||||
"content": {"type": "string"},
|
||||
"attachments": {
|
||||
"type": "object",
|
||||
"additionalProperties": {"type": "string"},
|
||||
},
|
||||
},
|
||||
["content"],
|
||||
),
|
||||
submit_deliverable,
|
||||
tier="execution",
|
||||
),
|
||||
ToolSpec(
|
||||
"needs_input",
|
||||
"Ask the user questions you can't proceed without. Ends your turn.",
|
||||
_schema(
|
||||
{"questions": {"type": "array", "items": {"type": "string"}}},
|
||||
["questions"],
|
||||
),
|
||||
needs_input,
|
||||
tier="execution",
|
||||
),
|
||||
*(gate_cloud_read(tool, allow_cloud_read) for tool in extra),
|
||||
]
|
||||
|
||||
|
||||
def gate_cloud_read(tool: ToolSpec, allow_cloud_read: bool | None) -> ToolSpec:
|
||||
"""Apply the Execution's cloud-read permission to a tool in that tier."""
|
||||
if tool.tier != CLOUD_READ or allow_cloud_read is True:
|
||||
return tool
|
||||
|
||||
if allow_cloud_read is False:
|
||||
|
||||
async def refuse(**_: Any) -> ToolResult:
|
||||
return _refused(
|
||||
"reading live cloud resources is not permitted for this Execution. "
|
||||
"Work from what the user has told you and record the gap."
|
||||
)
|
||||
|
||||
handler = refuse
|
||||
else:
|
||||
|
||||
async def ask_permission(**_: Any) -> ToolResult:
|
||||
return ToolResult(
|
||||
"Permission to read live cloud resources is needed first.",
|
||||
NeedsInput(
|
||||
(
|
||||
f"May I read live resources in your cloud environment? ({tool.name})",
|
||||
),
|
||||
permission=CLOUD_READ,
|
||||
),
|
||||
)
|
||||
|
||||
handler = ask_permission
|
||||
|
||||
return ToolSpec(tool.name, tool.description, tool.parameters, handler, tool.tier)
|
||||
@@ -1,7 +1,10 @@
|
||||
langchain>=0.2.0
|
||||
langchain-core>=0.2.0
|
||||
langchain-openai>=0.1.0
|
||||
langgraph>=0.1.0
|
||||
langgraph>=1.2
|
||||
langgraph-checkpoint-sqlite>=3.1
|
||||
langgraph-checkpoint-postgres>=3.1
|
||||
psycopg[binary]>=3.2
|
||||
starlette>=0.37.0
|
||||
uvicorn>=0.30.0
|
||||
pydantic>=2.7.0
|
||||
|
||||
628
tests/execution/test_engine.py
Normal file
628
tests/execution/test_engine.py
Normal file
@@ -0,0 +1,628 @@
|
||||
"""The Execution engine: LangGraph state, Specialist fan-out, interrupts and recording.
|
||||
|
||||
Driven entirely through ExecutionEngine.start/resume with a scripted agent runtime.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
|
||||
from app.agents.fake_runtime import Call, FakeAgentRuntime
|
||||
from app.agents.runtime import RoleSpec, ToolResult, ToolSpec
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||
from app.execution.engine import ExecutionEngine
|
||||
from app.execution.recorder import InMemoryRecorder
|
||||
|
||||
K = DeliverableKind
|
||||
P = PhaseId
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
|
||||
ROLES = {
|
||||
"orchestrator": RoleSpec("orchestrator", "Coordinate the Execution.", budget=40),
|
||||
**{
|
||||
spec.specialist: RoleSpec(spec.specialist, f"Produce {spec.kind}.", budget=15)
|
||||
for spec in DEPENDENCY_GRAPH.values()
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# --- scripting helpers ---------------------------------------------------------------
|
||||
|
||||
|
||||
def delegate(*kinds: DeliverableKind) -> Call:
|
||||
return Call("delegate", {"kinds": [kind.value for kind in kinds]})
|
||||
|
||||
|
||||
def request_approval(phase: PhaseId) -> Call:
|
||||
return Call("request_approval", {"phase": phase.value, "summary": f"{phase} done"})
|
||||
|
||||
|
||||
def ask_user(*questions: str) -> Call:
|
||||
return Call("ask_user", {"questions": list(questions)})
|
||||
|
||||
|
||||
def reopen(kind: DeliverableKind, reason: str) -> Call:
|
||||
return Call("reopen", {"kind": kind.value, "reason": reason})
|
||||
|
||||
|
||||
FINISH = Call("finish")
|
||||
|
||||
|
||||
def submission(kind: DeliverableKind, note: str = "") -> Call:
|
||||
spec = DEPENDENCY_GRAPH[kind]
|
||||
body = "\n".join(f"## {section}\n{note or 'Details.'}" for section in spec.sections)
|
||||
content = f"# {kind}\n{note or 'Summary.'}\n{body}\n"
|
||||
attachments = {name: f"# {name}\n" for name in spec.attachments}
|
||||
return Call("submit_deliverable", {"content": content, "attachments": attachments})
|
||||
|
||||
|
||||
def submits(kind: DeliverableKind) -> list[Call]:
|
||||
return [submission(kind)]
|
||||
|
||||
|
||||
def specialist(kind: DeliverableKind) -> str:
|
||||
return DEPENDENCY_GRAPH[kind].specialist
|
||||
|
||||
|
||||
def make_engine(tmp_path: Path, runtime, **overrides) -> ExecutionEngine:
|
||||
options = {
|
||||
"runtime": runtime,
|
||||
"roles": ROLES,
|
||||
"recorder": InMemoryRecorder(),
|
||||
"checkpointer": InMemorySaver(),
|
||||
"output_root": tmp_path,
|
||||
}
|
||||
options.update(overrides)
|
||||
return ExecutionEngine(**options)
|
||||
|
||||
|
||||
def happy_path_turns(**extra_specialist_turns) -> dict:
|
||||
turns = {
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY), delegate(K.PRODUCT_SELECTION)],
|
||||
[delegate(K.ARCHITECTURE_DIAGRAM, K.DESIGN_RECOMMENDATIONS)],
|
||||
[delegate(K.ARCHITECTURE_DESCRIPTION, K.DEPLOYMENT_GUIDANCE)],
|
||||
[request_approval(P.SOLUTION_ARCHITECTURE), delegate(K.VALIDATION_RESULTS)],
|
||||
[request_approval(P.SOLUTION_VALIDATION), delegate(K.SOLUTION_GUIDE)],
|
||||
[request_approval(P.PACKAGING), FINISH],
|
||||
],
|
||||
**{spec.specialist: [submits(kind)] for kind, spec in DEPENDENCY_GRAPH.items()},
|
||||
}
|
||||
turns.update(extra_specialist_turns)
|
||||
return turns
|
||||
|
||||
|
||||
# --- happy path and parallel start -----------------------------------------------------
|
||||
|
||||
|
||||
async def test_pre_approved_execution_completes_and_writes_deliverables(tmp_path):
|
||||
runtime = FakeAgentRuntime(happy_path_turns())
|
||||
recorder = InMemoryRecorder()
|
||||
engine = make_engine(tmp_path, runtime, recorder=recorder)
|
||||
|
||||
result = await engine.start("Event-driven ingestion", pre_approved=True)
|
||||
|
||||
assert result.status == "completed"
|
||||
assert result.prompt is None
|
||||
ledger = result.state["ledger"]
|
||||
assert set(ledger["phases"].values()) == {"approved"}
|
||||
assert {d["status"] for d in ledger["deliverables"].values()} == {"submitted"}
|
||||
|
||||
out = tmp_path / "deliverables" / "executions" / result.execution_id
|
||||
assert (out / "solution-architecture-guide.md").is_file()
|
||||
for kind in K:
|
||||
assert (out / "deliverables" / f"{kind}.md").is_file(), kind
|
||||
assert (out / "terraform" / "main.tf").is_file()
|
||||
assert (out / "architecture.mmd").is_file()
|
||||
|
||||
assert recorder.statuses[result.execution_id][-1]["status"] == "completed"
|
||||
assert {s["kind"] for s in recorder.submissions[result.execution_id]} == set(K)
|
||||
recorded_types = [e.type for e in recorder.events[result.execution_id]]
|
||||
assert recorded_types.count("approved") == 4
|
||||
|
||||
|
||||
async def test_independent_specialists_run_together_between_orchestrator_turns(
|
||||
tmp_path,
|
||||
):
|
||||
runtime = FakeAgentRuntime(happy_path_turns())
|
||||
|
||||
await make_engine(tmp_path, runtime).start("Ingestion", pre_approved=True)
|
||||
|
||||
roles = [run.role for run in runtime.runs]
|
||||
fourth_turn = [i for i, role in enumerate(roles) if role == "orchestrator"][3]
|
||||
assert set(roles[fourth_turn + 1 : fourth_turn + 3]) == {
|
||||
specialist(K.ARCHITECTURE_DIAGRAM),
|
||||
specialist(K.DESIGN_RECOMMENDATIONS),
|
||||
}
|
||||
assert roles[fourth_turn + 3] == "orchestrator"
|
||||
|
||||
|
||||
async def test_specialist_receives_its_dependencies_content(tmp_path):
|
||||
runtime = FakeAgentRuntime(happy_path_turns())
|
||||
|
||||
await make_engine(tmp_path, runtime).start("Ingestion", pre_approved=True)
|
||||
|
||||
describer_run = next(
|
||||
r for r in runtime.runs if r.role == specialist(K.ARCHITECTURE_DESCRIPTION)
|
||||
)
|
||||
dependencies = describer_run.task.context["dependencies"]
|
||||
assert set(dependencies) == {"architecture_diagram", "product_selection"}
|
||||
assert "architecture.mmd" in dependencies["architecture_diagram"]["attachments"]
|
||||
assert describer_run.task.deliverable == "architecture_description"
|
||||
|
||||
|
||||
# --- refusals reach the agent ------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_orchestrator_cannot_delegate_work_whose_dependencies_are_unmet(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.PRODUCT_SELECTION), delegate(K.REQUIREMENTS)],
|
||||
[ask_user("Anything else?")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
}
|
||||
)
|
||||
|
||||
result = await make_engine(tmp_path, runtime).start("Ingestion")
|
||||
|
||||
first_turn = runtime.runs[0]
|
||||
assert first_turn.results[0][0] == "delegate"
|
||||
assert "Refused" in first_turn.results[0][1]
|
||||
assert "product_selection" in first_turn.results[0][1]
|
||||
assert result.status == "awaiting_user"
|
||||
|
||||
|
||||
async def test_incomplete_submission_is_returned_to_the_specialist(tmp_path):
|
||||
incomplete = Call(
|
||||
"submit_deliverable", {"content": "## Functional requirements\nx"}
|
||||
)
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [[delegate(K.REQUIREMENTS)], [ask_user("Next?")]],
|
||||
specialist(K.REQUIREMENTS): [[incomplete, submission(K.REQUIREMENTS)]],
|
||||
}
|
||||
)
|
||||
|
||||
await make_engine(tmp_path, runtime).start("Ingestion")
|
||||
|
||||
analyst = runtime.runs[1]
|
||||
assert "Non-functional requirements" in analyst.results[0][1]
|
||||
assert analyst.results[1] == ("submit_deliverable", "Submitted requirements.")
|
||||
|
||||
|
||||
# --- interactive approvals ----------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_approval_pauses_the_execution_until_the_user_decides(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
[delegate(K.PRODUCT_SELECTION)],
|
||||
[ask_user("Pause here")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
specialist(K.TECHNICAL_DECOMPOSITION): [submits(K.TECHNICAL_DECOMPOSITION)],
|
||||
specialist(K.PRODUCT_SELECTION): [submits(K.PRODUCT_SELECTION)],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime)
|
||||
|
||||
paused = await engine.start("Ingestion")
|
||||
|
||||
assert paused.status == "awaiting_user"
|
||||
assert paused.prompt["approval"]["phase"] == "requirements_discovery"
|
||||
assert set(paused.prompt["approval"]["deliverables"]) == {
|
||||
"requirements",
|
||||
"technical_decomposition",
|
||||
}
|
||||
assert (
|
||||
paused.state["ledger"]["phases"]["requirements_discovery"]
|
||||
== "awaiting_approval"
|
||||
)
|
||||
|
||||
resumed = await engine.resume(
|
||||
paused.execution_id,
|
||||
{"approval": {"phase": "requirements_discovery", "decision": "approve"}},
|
||||
)
|
||||
|
||||
assert resumed.state["ledger"]["phases"]["requirements_discovery"] == "approved"
|
||||
assert (
|
||||
resumed.state["ledger"]["deliverables"]["product_selection"]["status"]
|
||||
== "submitted"
|
||||
)
|
||||
assert resumed.prompt == {
|
||||
"questions": [{"from": "orchestrator", "question": "Pause here"}]
|
||||
}
|
||||
|
||||
|
||||
async def test_free_text_reply_never_grants_an_approval(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
specialist(K.TECHNICAL_DECOMPOSITION): [submits(K.TECHNICAL_DECOMPOSITION)],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime)
|
||||
paused = await engine.start("Ingestion")
|
||||
|
||||
again = await engine.resume(paused.execution_id, {"reply": "approve, looks good"})
|
||||
|
||||
assert (
|
||||
again.state["ledger"]["phases"]["requirements_discovery"] == "awaiting_approval"
|
||||
)
|
||||
assert again.prompt["approval"]["phase"] == "requirements_discovery"
|
||||
assert again.state["conversation"][-2:] == [
|
||||
{"from": "user", "text": "approve, looks good"},
|
||||
{
|
||||
"from": "orchestrator",
|
||||
"text": "Please approve requirements_discovery: requirements_discovery done",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
async def test_requested_changes_lead_to_a_revision_and_a_new_approval(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
[
|
||||
reopen(K.REQUIREMENTS, "Add the RTO target"),
|
||||
delegate(K.REQUIREMENTS),
|
||||
],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [
|
||||
submits(K.REQUIREMENTS),
|
||||
submits(K.REQUIREMENTS),
|
||||
],
|
||||
specialist(K.TECHNICAL_DECOMPOSITION): [
|
||||
submits(K.TECHNICAL_DECOMPOSITION),
|
||||
submits(K.TECHNICAL_DECOMPOSITION),
|
||||
],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime)
|
||||
paused = await engine.start("Ingestion")
|
||||
|
||||
second = await engine.resume(
|
||||
paused.execution_id,
|
||||
{
|
||||
"approval": {
|
||||
"phase": "requirements_discovery",
|
||||
"decision": "changes",
|
||||
"feedback": "Add the RTO target",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
assert second.prompt["approval"]["phase"] == "requirements_discovery"
|
||||
ledger = second.state["ledger"]
|
||||
assert ledger["deliverables"]["requirements"]["revisions"] == 1
|
||||
second_analyst_run = [
|
||||
r for r in runtime.runs if r.role == specialist(K.REQUIREMENTS)
|
||||
][1]
|
||||
assert second_analyst_run.task.context["feedback"] == ["Add the RTO target"]
|
||||
assert "changes_requested" in [e["type"] for e in ledger["events"]]
|
||||
|
||||
|
||||
# --- one voice ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_questions_from_parallel_specialists_reach_the_user_in_one_prompt(
|
||||
tmp_path,
|
||||
):
|
||||
def diagrammer_turn(task):
|
||||
if task.context["answers"]:
|
||||
return submits(K.ARCHITECTURE_DIAGRAM)
|
||||
return [Call("needs_input", {"questions": ["Single region or multi-region?"]})]
|
||||
|
||||
def advisor_turn(task):
|
||||
if task.context["answers"]:
|
||||
return submits(K.DESIGN_RECOMMENDATIONS)
|
||||
return [Call("needs_input", {"questions": ["What is the RTO?"]})]
|
||||
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
[delegate(K.PRODUCT_SELECTION)],
|
||||
[delegate(K.ARCHITECTURE_DIAGRAM, K.DESIGN_RECOMMENDATIONS)],
|
||||
[ask_user()],
|
||||
[delegate(K.ARCHITECTURE_DIAGRAM, K.DESIGN_RECOMMENDATIONS)],
|
||||
[ask_user("Continue?")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
specialist(K.TECHNICAL_DECOMPOSITION): [submits(K.TECHNICAL_DECOMPOSITION)],
|
||||
specialist(K.PRODUCT_SELECTION): [submits(K.PRODUCT_SELECTION)],
|
||||
specialist(K.ARCHITECTURE_DIAGRAM): [diagrammer_turn, diagrammer_turn],
|
||||
specialist(K.DESIGN_RECOMMENDATIONS): [advisor_turn, advisor_turn],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime)
|
||||
approval = await engine.start("Ingestion")
|
||||
first = await engine.resume(
|
||||
approval.execution_id,
|
||||
{"approval": {"phase": "requirements_discovery", "decision": "approve"}},
|
||||
)
|
||||
|
||||
assert first.prompt == {
|
||||
"questions": [
|
||||
{
|
||||
"from": "architecture_diagram",
|
||||
"question": "Single region or multi-region?",
|
||||
},
|
||||
{"from": "design_recommendations", "question": "What is the RTO?"},
|
||||
]
|
||||
}
|
||||
|
||||
second = await engine.resume(
|
||||
first.execution_id, {"reply": "Single region, RTO 1 hour"}
|
||||
)
|
||||
|
||||
deliverables = second.state["ledger"]["deliverables"]
|
||||
assert deliverables["architecture_diagram"]["status"] == "submitted"
|
||||
assert deliverables["design_recommendations"]["status"] == "submitted"
|
||||
advisor_retry = [
|
||||
r for r in runtime.runs if r.role == specialist(K.DESIGN_RECOMMENDATIONS)
|
||||
][1]
|
||||
assert advisor_retry.task.context["answers"] == [
|
||||
{"questions": ["What is the RTO?"], "reply": "Single region, RTO 1 hour"}
|
||||
]
|
||||
|
||||
|
||||
# --- revision after validation ------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_validation_failure_revises_deployment_guidance_and_reapproves(tmp_path):
|
||||
turns = happy_path_turns()
|
||||
turns["orchestrator"] = [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY), delegate(K.PRODUCT_SELECTION)],
|
||||
[delegate(K.ARCHITECTURE_DIAGRAM, K.DESIGN_RECOMMENDATIONS)],
|
||||
[delegate(K.ARCHITECTURE_DESCRIPTION, K.DEPLOYMENT_GUIDANCE)],
|
||||
[request_approval(P.SOLUTION_ARCHITECTURE), delegate(K.VALIDATION_RESULTS)],
|
||||
[
|
||||
reopen(K.DEPLOYMENT_GUIDANCE, "terraform plan: missing IAM binding"),
|
||||
delegate(K.DEPLOYMENT_GUIDANCE),
|
||||
],
|
||||
[request_approval(P.SOLUTION_ARCHITECTURE), delegate(K.VALIDATION_RESULTS)],
|
||||
[request_approval(P.SOLUTION_VALIDATION), delegate(K.SOLUTION_GUIDE)],
|
||||
[request_approval(P.PACKAGING), FINISH],
|
||||
]
|
||||
turns[specialist(K.DEPLOYMENT_GUIDANCE)] = [
|
||||
submits(K.DEPLOYMENT_GUIDANCE),
|
||||
submits(K.DEPLOYMENT_GUIDANCE),
|
||||
]
|
||||
turns[specialist(K.VALIDATION_RESULTS)] = [
|
||||
[submission(K.VALIDATION_RESULTS, "FAILED: missing IAM binding")],
|
||||
submits(K.VALIDATION_RESULTS),
|
||||
]
|
||||
runtime = FakeAgentRuntime(turns)
|
||||
|
||||
result = await make_engine(tmp_path, runtime).start("Ingestion", pre_approved=True)
|
||||
|
||||
assert result.status == "completed"
|
||||
events = result.state["ledger"]["events"]
|
||||
withdrawn = [e["subject"] for e in events if e["type"] == "approval_withdrawn"]
|
||||
assert withdrawn == ["solution_architecture"]
|
||||
requests = [e["subject"] for e in events if e["type"] == "approval_requested"]
|
||||
assert requests.count("solution_architecture") == 2
|
||||
iac_runs = [r for r in runtime.runs if r.role == specialist(K.DEPLOYMENT_GUIDANCE)]
|
||||
assert iac_runs[1].task.context["feedback"] == [
|
||||
"terraform plan: missing IAM binding"
|
||||
]
|
||||
|
||||
|
||||
async def test_pre_approved_escalation_ends_with_warnings(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[reopen(K.REQUIREMENTS, "not good enough")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
}
|
||||
)
|
||||
|
||||
result = await make_engine(tmp_path, runtime, max_revisions=0).start(
|
||||
"Ingestion", pre_approved=True
|
||||
)
|
||||
|
||||
assert result.status == "completed_with_warnings"
|
||||
assert "revision limit" in result.state["warnings"][0]
|
||||
assert not (tmp_path / "deliverables").exists()
|
||||
|
||||
|
||||
# --- budgets -------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_orchestrator_budget_exhaustion_hands_control_to_the_user(tmp_path):
|
||||
roles = {**ROLES, "orchestrator": RoleSpec("orchestrator", "", budget=2)}
|
||||
wandering = [Call("ready_work"), Call("ready_work"), Call("ready_work")]
|
||||
runtime = FakeAgentRuntime({"orchestrator": [wandering]})
|
||||
|
||||
result = await make_engine(tmp_path, runtime, roles=roles).start("Ingestion")
|
||||
|
||||
assert result.status == "awaiting_user"
|
||||
question = result.prompt["questions"][0]
|
||||
assert question["from"] == "orchestrator"
|
||||
assert "2 tool calls" in question["question"]
|
||||
|
||||
|
||||
async def test_specialist_budget_exhaustion_is_relayed_as_a_question(tmp_path):
|
||||
roles = {
|
||||
**ROLES,
|
||||
specialist(K.REQUIREMENTS): RoleSpec("requirements-analyst", "", budget=1),
|
||||
}
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [[delegate(K.REQUIREMENTS)], [ask_user()]],
|
||||
specialist(K.REQUIREMENTS): [
|
||||
[
|
||||
Call("submit_deliverable", {"content": "## Current state\nx"}),
|
||||
submission(K.REQUIREMENTS),
|
||||
]
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = await make_engine(tmp_path, runtime, roles=roles).start("Ingestion")
|
||||
|
||||
assert result.prompt["questions"][0]["from"] == "requirements"
|
||||
assert result.state["ledger"]["deliverables"]["requirements"]["status"] == "pending"
|
||||
|
||||
|
||||
# --- cloud read permission -------------------------------------------------------------------
|
||||
|
||||
|
||||
def cloud_read_factory(calls: list[str]):
|
||||
async def list_projects() -> ToolResult:
|
||||
calls.append("list_projects")
|
||||
return ToolResult("projects: demo-project")
|
||||
|
||||
tool = ToolSpec(
|
||||
"list_projects",
|
||||
"List projects.",
|
||||
{"type": "object", "properties": {}},
|
||||
list_projects,
|
||||
tier="cloud_read",
|
||||
)
|
||||
|
||||
def factory(role: RoleSpec, deliverable: str | None):
|
||||
return [tool] if role.name == "requirements-analyst" else []
|
||||
|
||||
return factory
|
||||
|
||||
|
||||
async def test_cloud_read_asks_permission_once_then_runs_the_tool(tmp_path):
|
||||
calls: list[str] = []
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[ask_user("Next?")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [
|
||||
[Call("list_projects"), submission(K.REQUIREMENTS)],
|
||||
[Call("list_projects"), submission(K.REQUIREMENTS)],
|
||||
],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime, tool_factory=cloud_read_factory(calls))
|
||||
|
||||
asked = await engine.start("Ingestion")
|
||||
|
||||
assert asked.prompt == {
|
||||
"permission": {"tier": "cloud_read", "requested_by": ["requirements"]}
|
||||
}
|
||||
assert calls == []
|
||||
|
||||
granted = await engine.resume(
|
||||
asked.execution_id, {"permission": {"cloud_read": True}}
|
||||
)
|
||||
|
||||
assert calls == ["list_projects"]
|
||||
assert granted.state["allow_cloud_read"] is True
|
||||
assert (
|
||||
granted.state["ledger"]["deliverables"]["requirements"]["status"] == "submitted"
|
||||
)
|
||||
|
||||
|
||||
async def test_denied_cloud_read_is_refused_to_the_agent(tmp_path):
|
||||
calls: list[str] = []
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [[delegate(K.REQUIREMENTS)], [ask_user("Next?")]],
|
||||
specialist(K.REQUIREMENTS): [
|
||||
[Call("list_projects"), submission(K.REQUIREMENTS)]
|
||||
],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime, tool_factory=cloud_read_factory(calls))
|
||||
|
||||
await engine.start("Ingestion", allow_cloud_read=False)
|
||||
|
||||
analyst = runtime.runs[1]
|
||||
assert "not permitted" in analyst.results[0][1]
|
||||
assert calls == []
|
||||
|
||||
|
||||
# --- isolation and durability -------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_executions_in_one_session_share_no_state(tmp_path):
|
||||
runtime = FakeAgentRuntime(
|
||||
{
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[ask_user("?")],
|
||||
[ask_user("fresh?")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
}
|
||||
)
|
||||
engine = make_engine(tmp_path, runtime)
|
||||
|
||||
first = await engine.start("First", session_id="s-1")
|
||||
second = await engine.start("Second", session_id="s-1")
|
||||
|
||||
assert first.execution_id != second.execution_id
|
||||
assert second.state["deliverables"] == {}
|
||||
assert second.state["ledger"]["deliverables"]["requirements"]["status"] == "pending"
|
||||
assert second.state["workflow_request"] == "Second"
|
||||
assert second.state["session_id"] == "s-1"
|
||||
|
||||
|
||||
async def test_paused_execution_resumes_from_a_durable_checkpoint(tmp_path):
|
||||
turns = {
|
||||
"orchestrator": [
|
||||
[delegate(K.REQUIREMENTS)],
|
||||
[delegate(K.TECHNICAL_DECOMPOSITION)],
|
||||
[request_approval(P.REQUIREMENTS_DISCOVERY)],
|
||||
[ask_user("Resumed")],
|
||||
],
|
||||
specialist(K.REQUIREMENTS): [submits(K.REQUIREMENTS)],
|
||||
specialist(K.TECHNICAL_DECOMPOSITION): [submits(K.TECHNICAL_DECOMPOSITION)],
|
||||
}
|
||||
runtime = FakeAgentRuntime(turns)
|
||||
db_path = str(tmp_path / "checkpoints.db")
|
||||
|
||||
async with AsyncSqliteSaver.from_conn_string(db_path) as saver:
|
||||
paused = await make_engine(tmp_path, runtime, checkpointer=saver).start(
|
||||
"Ingestion"
|
||||
)
|
||||
assert paused.prompt["approval"]["phase"] == "requirements_discovery"
|
||||
|
||||
async with AsyncSqliteSaver.from_conn_string(db_path) as saver:
|
||||
resumed = await make_engine(tmp_path, runtime, checkpointer=saver).resume(
|
||||
paused.execution_id,
|
||||
{"approval": {"phase": "requirements_discovery", "decision": "approve"}},
|
||||
)
|
||||
|
||||
assert resumed.prompt == {
|
||||
"questions": [{"from": "orchestrator", "question": "Resumed"}]
|
||||
}
|
||||
assert resumed.state["ledger"]["phases"]["requirements_discovery"] == "approved"
|
||||
91
tests/execution/test_recording.py
Normal file
91
tests/execution/test_recording.py
Normal file
@@ -0,0 +1,91 @@
|
||||
"""The database recorder and the configured checkpointer."""
|
||||
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
|
||||
from app.config import get_settings
|
||||
from app.database import DatabaseManager
|
||||
from app.execution.checkpointing import open_checkpointer
|
||||
from app.execution.ledger import LedgerEvent
|
||||
from app.execution.recorder import DatabaseRecorder
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sqlite_db(tmp_path, monkeypatch):
|
||||
path = tmp_path / "records.db"
|
||||
monkeypatch.setattr(get_settings(), "SQLITE_FALLBACK_DB", str(path))
|
||||
return DatabaseManager(db_url="sqlite:///unused"), path
|
||||
|
||||
|
||||
async def test_database_recorder_writes_events_submissions_and_status(sqlite_db):
|
||||
manager, path = sqlite_db
|
||||
recorder = DatabaseRecorder(manager)
|
||||
|
||||
await recorder.record_events(
|
||||
"exec-1",
|
||||
[
|
||||
LedgerEvent(
|
||||
1, "submitted", "requirements", "requirements-analyst", "attempt 1"
|
||||
)
|
||||
],
|
||||
)
|
||||
await recorder.record_submission(
|
||||
"exec-1", "deployment_guidance", "# Guide", {"terraform/main.tf": "resource {}"}
|
||||
)
|
||||
await recorder.record_status(
|
||||
"exec-1",
|
||||
session_id="session-1",
|
||||
workflow_request="Ingestion",
|
||||
status="awaiting_user",
|
||||
current_phase="solution_architecture",
|
||||
details={"revisions": 2},
|
||||
)
|
||||
|
||||
with sqlite3.connect(path) as conn:
|
||||
logs = conn.execute(
|
||||
"SELECT iteration, review_status, reviewer_agent, feedback "
|
||||
"FROM orchestrator_review_logs WHERE execution_id = 'exec-1'"
|
||||
).fetchall()
|
||||
artifacts = conn.execute(
|
||||
"SELECT artifact_type, file_path FROM adk_artifacts WHERE execution_id = 'exec-1' "
|
||||
"ORDER BY file_path"
|
||||
).fetchall()
|
||||
execution = conn.execute(
|
||||
"SELECT status, current_phase, loop_count FROM workflow_executions "
|
||||
"WHERE execution_id = 'exec-1'"
|
||||
).fetchone()
|
||||
session = conn.execute(
|
||||
"SELECT agent_name, state_data FROM adk_sessions WHERE session_id = 'session-1'"
|
||||
).fetchone()
|
||||
|
||||
assert logs == [(1, "submitted", "requirements-analyst", "requirements: attempt 1")]
|
||||
assert artifacts == [
|
||||
("deployment_guidance", "deliverables/deployment_guidance.md"),
|
||||
("deployment_guidance", "terraform/main.tf"),
|
||||
]
|
||||
assert execution == ("awaiting_user", "solution_architecture", 2)
|
||||
assert session[0] == "solution-architect"
|
||||
assert "exec-1" in session[1]
|
||||
|
||||
|
||||
async def test_checkpointer_follows_settings(tmp_path, monkeypatch):
|
||||
settings = get_settings()
|
||||
|
||||
monkeypatch.setattr(settings, "CHECKPOINTER", "memory")
|
||||
async with open_checkpointer(settings) as saver:
|
||||
assert isinstance(saver, InMemorySaver)
|
||||
|
||||
monkeypatch.setattr(settings, "CHECKPOINTER", "sqlite")
|
||||
monkeypatch.setattr(settings, "CHECKPOINT_DB_PATH", str(tmp_path / "cp.db"))
|
||||
async with open_checkpointer(settings) as saver:
|
||||
assert isinstance(saver, AsyncSqliteSaver)
|
||||
|
||||
monkeypatch.setattr(settings, "CHECKPOINTER", "redis")
|
||||
with pytest.raises(ValueError, match="redis"):
|
||||
async with open_checkpointer(settings):
|
||||
pass
|
||||
Reference in New Issue
Block a user