# Spec: Solution architect agent A portable, hierarchical ReAct agent that replaces the upstream [`google-cloud-solution-architecture`](https://github.com/google/skills/blob/main/skills/cloud/google-cloud-solution-architecture/SKILL.md) skill. An Orchestrator reasons over a code-declared dependency graph of Deliverables and delegates to Specialists. All agents run vendored Agent Skills through a framework's own skill runtime. Users grant Approvals at Phase boundaries. - Glossary: [`CONTEXT.md`](../../CONTEXT.md). Use its terms: Execution, Session, Phase, Approval, Revision, Orchestrator, Specialist, Skill, Cloud provider, Deliverable, Dependency, Solution guide. - Decisions: [ADR-0002](../../docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md) (Skills-first, hybrid, hierarchical and ReAct, vendoring, Cloud provider as configuration) and [ADR-0003](../../docs/adr/0003-langgraph-holds-state-adk-agents-reason.md) (LangGraph state with ADK agents). - Supersedes: [`.scratch/phase-pipeline/spec.md`](../phase-pipeline/spec.md). Its decision log (D1 to D46) records how this design was reached. - Branch: `feat/solution-architect-agent`, pushed to the Gitea remote. A human opens the PR. ## Problem The current agent is a linear chain of five template functions. They ignore the Workflow request (design always picks Cloud Run and Pub/Sub). They run under a fake "ADK" layer that never touches `google-adk`. The skill prompts they build are thrown away. The agent can't ask the user anything, can't take an Approval, can't run the upstream Skill, and can't target another framework or Cloud provider without a rewrite. ## Goals - **Upstream parity.** It runs the upstream Skill's workflow: requirements discovery with strict separation from solutioning, grounded product selection with alternatives, diagram, description, WAF-pillar design recommendations, IaC deployment guidance, dry-run validation, and packaging into the upstream template. - **Hierarchical and ReAct.** Work follows Dependencies, not a fixed order, and independent work runs in parallel. - **Interactive by default**, with code-enforced, recorded Approvals at Phase boundaries. There is also a pre-approved mode for batch and evaluation runs. - **Discovery through the Cloud provider's own remote MCP servers.** Current state is discovered through remote MCP servers hosted by the Cloud provider, such as Google Cloud's Resource Manager, Asset Inventory, Compute and GKE servers. Grounding uses the provider's knowledge MCP server. No hand-written cloud REST scanners. - **Portable.** - Behaviour lives in Skills and framework-neutral role files. - Framework types stay behind one agent-runtime interface. - Everything specific to a Cloud provider lives in a provider bundle. - **Deterministic tests** check every rule the code owns, without a model. ## Non-goals (v1) - **Runtime validation.** Upstream Task 3.2, verifying after the user has deployed, is out of scope. - **Changing cloud resources.** No `apply` or deploy tool, and ADK's `run_skill_script` is off. - **A second Cloud provider** or a second agent-runtime implementation (Deep Agents). The design leaves room for both; neither is built. - **Architecture review candidates 02 to 06**, beyond what this spec needs: tools get tier wrappers, not full ports; `DatabaseManager` is reused as is; no typed Deliverable records. - **Human approval inside a Phase enforced by code.** That stays conversational and driven by the Skill (see [Risks](#risks)). - **Acting as the end user in their cloud (OAuth).** Remote MCP calls use the agent's own identity (a service account or Application Default Credentials) in v1. - **Routing MCP calls through the platform's MCP gateway** (`mcp_gateway_client.py` in the base agents). Remote MCP servers are called directly (D46). If a gateway is needed later, it becomes a new `mcp_auth.kind: gateway` implementation in `app/tools/mcp.py`, and nothing else changes. ## Decisions These are carried from the decision log. The numbers refer to it. | Area | Decision | |---|---| | Sequencing (D19, D23) | **Hybrid.** Code owns Dependencies, Phase boundaries and Approvals, and the Skill owns the work inside a Deliverable. The agent is hierarchical and ReAct, not linear. | | Engine (D24) | **LangGraph + ADK.** A LangGraph `StateGraph` holds Execution state, checkpoints and `interrupt()`. The Orchestrator and Specialists are ADK `LlmAgent`s with `SkillToolset`, behind an `AgentRuntime` interface. | | Phases (D25) | **Upstream's 4 Phases**, as groupings of Deliverables. Discovery of current state is done through tools (remote MCP servers, D45). | | Approvals (D26, D32) | Code enforces Approval at Phase boundaries. A Dependency inside a Phase is met on *submitted*, and one across Phases only on Phase *approved*. Every user question is an `interrupt()`. | | Completion (D27) | Agents finish by calling a submit tool that is checked against the Deliverable's contract. | | Transport (D28) | **A2A** for interactive Executions. `/generate` is kept for pre-approved ones. | | Hierarchy (D29) | **An Orchestrator over one Specialist per Deliverable.** The design advisor fans out to 6 pillar advisors. | | Dependency graph (D30) | **Declared in code.** The ReAct Orchestrator picks work through tools that refuse work whose Dependencies aren't met. | | Budgets (D31) | **Per-agent ReAct step budgets.** Hitting one hands control to the user. | | Voice (D33) | **Only the Orchestrator talks to the user.** Specialists ask through `needs_input`. | | Specialist memory (D34) | **Stateless per delegation.** Only the LangGraph checkpoint persists. | | Revision (D35) | **Reopening a Deliverable** returns it and everything that depends on it to draft and withdraws affected Approvals. Revisions are capped, and past the cap the user decides. | | Models (D36) | **Configurable per role via ADK LiteLLM**, with a Gemini default. | | Dependency graph content (D38) | See [Dependency graph](#dependency-graph). Runtime validation is out of scope. | | Skill slicing (D39) | **A shared `SkillToolset` plus `agents/.md`.** Pillar advisors are scoped to their own WAF Skill. | | Vendoring (D21, D40) | **Upstream solution-architecture + 6 WAF Skills**, unmodified, pinned to `81a31a6`, in `skills/vendor/google/`. | | Tool tiers (D41) | **General tools** (local checks, public documentation) run freely. **Reading live cloud resources** needs permission once per Execution. **Changing cloud resources** is never exposed. | | Discovery (D45) | **The Cloud provider's own remote MCP servers**, declared in the provider bundle, provide discovery and grounding. Every tool that reads the user's cloud is `cloud_read`. Only tools on an explicit read-only allowlist are exposed. `GCPEnvironmentScanner` is retired. | | MCP routing (D46) | **Call remote MCP servers directly**, with no platform MCP gateway for now. | | Outputs (D42) | **Deliverables live in state and the database.** Files go to `deliverables/executions/{id}/` only after Phase 4 Approval. | | Tests (D43) | **A fake `AgentRuntime`** for deterministic tests. The real-model evaluation runs pre-approved with scripted answers. | | Migration (D44) | **Build alongside the current code.** One cleanup increment follows once the evaluation passes. | | Carried over (D9, D14) | **Every Execution starts clean**, and a Session only groups Executions. The Execution graph and agent runtime are built once at startup. | ## Design ### Module layout New code only. The current modules stay untouched until [increment 9](#build-increments). ``` agents/ # framework-neutral role files (one per role) orchestrator.md requirements-analyst.md decomposer.md product-selector.md diagrammer.md describer.md design-advisor.md pillar-advisor.md iac-author.md validator.md packager.md providers/ gcp.yaml # Cloud provider bundle skills/ vendor/google/ # unmodified upstream Skills + VENDOR.md + LICENSE google-cloud-solution-architecture/ google-cloud-waf-{security,reliability,cost-optimization,operational-excellence,performance-optimization,sustainability}/ cloud-discovery/SKILL.md # our tooling Skills (Agent Skills spec compliant) iac-validation/SKILL.md app/ execution/ deliverables.py # DeliverableKind, PhaseId, DEPENDENCY_GRAPH, contracts ledger.py # Ledger: Deliverable/Phase states, acceptance, Revision (pure, no I/O) graph.py # LangGraph StateGraph: orchestrator turn, Specialist fan-out, interrupts tools.py # Orchestrator and Specialist execution tools bound to the Ledger state.py # ExecutionState TypedDict recorder.py # records submissions, Approvals, Revisions, final files agents/ runtime.py # AgentRuntime protocol, RoleSpec, AgentTask, Outcome types roles.py # loads agents/*.md into RoleSpec adk_runtime.py # ADK LlmAgent + SkillToolset + LiteLlm implementation fake_runtime.py # scripted implementation for tests (under app/ so eval can reuse it) tools/ general/ # mermaid check, terraform fmt/validate (local) cloud_read/ # terraform plan (local CLI, reads live state) mcp.py # McpServerSpec, read-only allowlist, tier + permission metadata providers.py # loads providers/.yaml a2a/ server.py # aiohttp A2A server on :8000, agent_card.json executor.py # maps A2A tasks <-> Executions and interrupts scripts/sync-vendored-skills.sh ``` ### Deliverables and the dependency graph `app/execution/deliverables.py` declares the graph as data. It is the only place Phases, Deliverables and Dependencies are listed. | Phase | Deliverable kind | Specialist role | Depends on | Tools (tier) | |---|---|---|---|---| | 1 `requirements_discovery` | `requirements` | `requirements-analyst` | (none) | discovery MCP servers (cloud_read) | | 1 | `technical_decomposition` | `decomposer` | `requirements` | (none) | | 2 `solution_architecture` | `product_selection` | `product-selector` | `technical_decomposition` | knowledge MCP (general) | | 2 | `architecture_diagram` | `diagrammer` | `product_selection` | mermaid check (general) | | 2 | `architecture_description` | `describer` | `architecture_diagram`, `product_selection` | knowledge MCP (general) | | 2 | `design_recommendations` | `design-advisor` → 6 × `pillar-advisor` | `product_selection`, `requirements` | knowledge MCP (general) | | 2 | `deployment_guidance` | `iac-author` | `product_selection`, `design_recommendations` | terraform fmt/validate (general) | | 3 `solution_validation` | `validation_results` | `validator` | `deployment_guidance` | terraform validate (general), terraform plan and policy MCP servers: IAM, Network Intelligence (cloud_read) | | 4 `packaging` | `solution_guide` | `packager` | every Phase 3 Deliverable | (none) | **Deliverable contract.** Each kind declares its required content in `deliverables.py`: - `sections`: required markdown headings. - `attachments`: named code blobs, for example `deployment_guidance` requires `terraform/main.tf` and `terraform/variables.tf`. `submit_deliverable` checks the contract. A failing submission returns what's missing to the Specialist, and nothing is recorded. ### The Ledger: acceptance and Revision rules `app/execution/ledger.py` is a pure module. It holds every rule code owns, and the tests exercise it heavily. - **Deliverable states:** `pending → in_progress → submitted`. Only a Revision goes backwards. - **Phase states:** `open → awaiting_approval → approved`. A Phase becomes `awaiting_approval` when the Orchestrator calls `request_approval` with all its Deliverables `submitted`. - **When a Dependency is met:** - **Same Phase:** the Dependency is `submitted`. - **Across Phases:** the Dependency's Phase is `approved`. - **Ready work:** Deliverables that are `pending` and have every Dependency met. - **`reopen(kind, reason)`:** - The Deliverable goes to `pending` with `reason` appended to its feedback. - Everything that depends on it, directly or not, goes to `pending`. - Every affected Phase that was `approved` or `awaiting_approval` goes back to `open`. - The Deliverable's `revisions` count goes up. At the cap (`max_revisions`, default 3) `reopen` refuses and returns an escalation the Orchestrator must put to the user. - **Pre-approved mode:** `request_approval` grants immediately and records `approved_by = "pre-approved"`. - **Records:** every transition appends an event (`submitted`, `approval_requested`, `approved`, `changes_requested`, `reopened`, `escalated`) for the recorder. ### Execution graph (LangGraph) `app/execution/graph.py` compiles once at startup with the configured checkpointer: `SqliteSaver` locally and in tests, `PostgresSaver` in production. The Execution id is the LangGraph `thread_id`. **State (`ExecutionState`):** - `execution_id`, `session_id`, `workflow_request`, `mode: "interactive" | "pre_approved"`, `allow_cloud_read: bool | None` - `ledger` (serialisable), `deliverables: {kind: {content, attachments, feedback[], revisions}}` - `conversation[]` (user ↔ Orchestrator turns only), `pending_questions[]`, `pending_delegations[]` - `budget_exhausted: {role, summary} | None` **Nodes and edges:** ``` START → orchestrator_turn orchestrator_turn ──(delegations)──────────> specialist_task × N (LangGraph Send, parallel) orchestrator_turn ──(questions/approval)───> await_user (interrupt) orchestrator_turn ──(finish)───────────────> finalize → END specialist_task ───────────────────────────> orchestrator_turn (outcomes merged into ledger) await_user ────────────────────────────────> orchestrator_turn (resume value = user reply / approval decision) ``` - **`orchestrator_turn`:** runs one Orchestrator ReAct turn through `AgentRuntime`, with Orchestrator tools bound to a working copy of the Ledger. The turn ends when the agent does one of these, and the node routes on it: - calls `delegate`, `ask_user`, `request_approval` or `finish` - runs out of budget - **`specialist_task`:** runs one Specialist delegation. Its task holds the role, the Deliverable kind, the Dependency contents, feedback, and the answers to its earlier `needs_input` questions. The outcome is one of: - `Submitted(content, attachments)`, applied to the Ledger - `NeedsInput(questions)`, added to `pending_questions` with the Deliverable back to `pending` - `BudgetExhausted(summary)` - **`await_user`:** `interrupt({"questions": [...]} | {"approval": {"phase": ..., "summary": ...}} | {"permission": "cloud_read"} | {"escalation": ...})`. In pre-approved mode it doesn't pause: - **Approvals:** granted automatically. - **`cloud_read` permission:** taken from the request. - **Questions:** answered with `"No user is available. State your assumption, record it in the Deliverable under 'Assumptions', and proceed."` - **Escalations:** end the Execution as `completed_with_warnings`. - **`finalize`:** runs after Phase 4 is `approved`. It writes files to `deliverables/executions/{execution_id}/`: the Solution guide, every Deliverable as markdown, and attachments under `terraform/`. It then records completion. **Orchestrator tools** (`app/execution/tools.py`): | Tool | Effect | |---|---| | `ready_work()` | Lists ready Deliverables and Phases ready for Approval. | | `delegate(kinds: list[str])` | Refuses any kind that isn't ready and sets it `in_progress`. Ends the turn and fans out. | | `ask_user(questions: list[str])` | Combines Specialist questions with its own. Ends the turn and interrupts. | | `request_approval(phase: str)` | Refuses unless every Deliverable in the Phase is submitted. Ends the turn and interrupts. | | `reopen(kind: str, reason: str)` | Applies the Revision rule, or returns an escalation. | | `read_deliverable(kind: str)` | Returns the current content, so the Orchestrator can summarise it for Approval. | | `finish()` | Refuses unless Phase 4 is approved. | **Specialist tools:** `submit_deliverable(content, attachments)` (checks the contract), `needs_input(questions)`, the `SkillToolset`, and the role's tier tools. Cloud-read tools check `allow_cloud_read`. If it's `None`, the tool returns `NeedsInput(["May I read live resources in your Google Cloud project to …?"])` so the Orchestrator asks once. If it's `False`, it returns a refusal the agent must work around. **Design advisor fan-out.** The `design-advisor` role runs a nested fan-out itself: it sends one task to `pillar-advisor` for each WAF pillar, in parallel, through `AgentRuntime`, and combines the results into its submission. Pillar results are not Deliverables and don't appear in the Ledger. ### Agent runtime interface `app/agents/runtime.py`: ```python class AgentRuntime(Protocol): async def run(self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]) -> Outcome: ... @dataclass(frozen=True) class RoleSpec: # parsed from agents/.md name: str instructions: str # markdown body skills: tuple[str, ...] # Skill names from the provider bundle tool_tiers: tuple[str, ...] # subset of ("general", "cloud_read") mcp_servers: tuple[str, ...] # server ids from the provider bundle this role may use model: str | None # overrides provider default budget: int # max tool calls per run Outcome = Submitted | NeedsInput | BudgetExhausted | TurnEnded # TurnEnded carries the Orchestrator's ending tool call ``` - **`ToolSpec`:** framework-neutral (name, description, JSON schema, async callable). `adk_runtime.py` builds each agent from these pieces: - `ToolSpec` → ADK `FunctionTool`. - The role's MCP servers → ADK `McpToolset` over streamable HTTP, with `tool_filter` set to the server's read-only allowlist. - `SkillToolset(skills=[load_skill_from_dir(...)])` with only the role's Skills. - `model` wrapped in `LiteLlm`. - **Per run:** a fresh `InMemorySessionService` (D34). - **Callbacks:** a `before_tool_callback` counts tool calls against `budget` and enforces the `cloud_read` permission for *every* tool in that tier, MCP tools included. - **No framework types** cross this interface. **Role file format** (`agents/.md`): ```markdown --- role: diagrammer deliverable: architecture_diagram upstream_task: "Phase 2, Task 2.2: Generate an architecture diagram" skills: [google-cloud-solution-architecture] tool_tiers: [general] mcp_servers: [] budget: 15 --- You produce the architecture diagram Deliverable. Load the `google-cloud-solution-architecture` Skill and follow its Task 2.2 ... Submit with `submit_deliverable` once the diagram passes the Mermaid check and the user has confirmed it through `needs_input`. ``` `pillar-advisor.md` is parameterised by pillar, and its `skills` value is resolved per task to the one matching `google-cloud-waf-`. ### Cloud provider bundle `providers/gcp.yaml`, loaded by `app/providers.py`: ```yaml name: gcp skills_dirs: [skills/vendor/google, skills] output_template: skills/vendor/google/google-cloud-solution-architecture/assets/output-template.md models: default: gemini/gemini-2.5-pro roles: { pillar-advisor: gemini/gemini-2.5-flash, diagrammer: gemini/gemini-2.5-flash } tools: general: [mermaid_check, terraform_fmt, terraform_validate] cloud_read: [terraform_plan] mcp_auth: kind: google_adc # service account / ADC bearer token; sends X-Goog-User-Project mcp_servers: # remote MCP servers hosted by the Cloud provider (streamable HTTP) developer_knowledge: url: https://developerknowledge.googleapis.com/mcp tier: general # public documentation, no customer data allow: [search_documents, get_documents, answer_query] resource_manager: { url: https://cloudresourcemanager.googleapis.com/mcp, tier: cloud_read, allow: [] } asset_inventory: { url: https://cloudasset.googleapis.com/mcp, tier: cloud_read, allow: [...], preview: true } compute: { url: https://compute.googleapis.com/mcp, tier: cloud_read, allow: [...] } gke: { url: https://container.googleapis.com/mcp, tier: cloud_read, allow: [...] } cloud_run: { url: https://run.googleapis.com/mcp, tier: cloud_read, allow: [...] } cloud_sql: { url: https://sqladmin.googleapis.com/mcp, tier: cloud_read, allow: [...] } storage: { url: https://storage.googleapis.com/storage/mcp, tier: cloud_read, allow: [...] } pubsub: { url: https://pubsub.googleapis.com/mcp, tier: cloud_read, allow: [...] } iam: { url: https://iam.googleapis.com/mcp, tier: cloud_read, allow: [...] } monitoring: { url: https://monitoring.googleapis.com/mcp, tier: cloud_read, allow: [...] } network_intelligence: { url: https://networkmanagement.googleapis.com/mcp, tier: cloud_read, allow: [...], preview: true } role_mcp_servers: requirements-analyst: [resource_manager, asset_inventory, compute, gke, cloud_run, cloud_sql, storage, pubsub, iam, monitoring] product-selector: [developer_knowledge] describer: [developer_knowledge] design-advisor: [developer_knowledge] pillar-advisor: [developer_knowledge] validator: [iam, network_intelligence] ``` **Loading the bundle.** - Settings select the bundle with `CLOUD_PROVIDER=gcp`. Core modules don't name a provider. - `role_mcp_servers` overrides the `mcp_servers` list in role files, so role files stay neutral across providers. - The `allow` lists are written once in increment 6: run MCP `tools/list` on each server and include only tools whose `readOnlyHint` annotation is true *and* whose names are reviewed as read-only. - Servers marked `preview: true` can be switched off with `MCP_ENABLE_PREVIEW=false`. **Other Cloud providers** reuse the same shape: - **AWS:** the managed, generally available AWS MCP Server. - **Azure:** Azure MCP Server 2.0 runs as a *self-hosted* remote server, so its `url` points at the team's own deployment. ### Tools - **General, local** (`app/tools/general/`): - `mermaid_check`: moved from `validation_tools.validate_mermaid_diagram`. - `terraform_fmt` and `terraform_validate`: run the `terraform` CLI in a temp dir with `-backend=false`. There are no credentials, and the tool fails clearly if `terraform` is missing. - **Cloud read, local** (`app/tools/cloud_read/`): `terraform_plan` needs ADC and `-lock=false`, and never runs `apply`. - **Remote MCP servers** (`app/tools/mcp.py`): the Cloud provider's own hosted servers, declared in the provider bundle and used for both discovery and grounding. They replace `GCPEnvironmentScanner` (hand-written REST calls) and the hand-rolled JSON-RPC `DeveloperKnowledgeMCPClient`, including its offline fallback knowledge base and the `data`/`documents` bug from review candidate 02. Both are deleted in increment 9. - `McpServerSpec(id, url, tier, allow, preview)` is framework-neutral. The ADK runtime maps each one to `McpToolset(StreamableHTTPConnectionParams(url, headers=auth), tool_filter=allow)`. - **Read-only in depth.** Tools outside `allow` are never offered to any agent. At startup, a server whose `tools/list` reports an allowed tool with `readOnlyHint` not true fails to load, loudly. Operators should also run the agent's identity with only viewer roles, plus Google Cloud's control that blocks read-write MCP tool use and an IAM deny policy (`docs/operations/mcp-access.md`, written in increment 6). - **Auth:** `google_adc` gets a bearer token from Application Default Credentials (`google.auth.default`, refreshed per call) and sends `X-Goog-User-Project`. Follows `google-adk-base-agent/app/google_mcp_client.py`, without the `gcloud` subprocess fallback. - **Failure:** an unreachable, disabled or unauthorised server returns a structured tool error such as `"compute MCP unavailable: API not enabled in project X"`. The agent records the gap under 'Assumptions / discovery gaps' and doesn't fail the Execution. - **Our tooling Skills** (`skills/cloud-discovery`, `skills/iac-validation`): short Agent Skills-compliant guides for agents. - `cloud-discovery`: which discovery servers answer which current-state questions (inventory first through Asset Inventory and Resource Manager, then per-product detail), and how to summarise current state. - `iac-validation`: when and how to run the Terraform checks. ### Interaction **A2A** (`app/a2a/`), using `a2a-sdk>=0.3` on `aiohttp` :8000, with `agent_card.json` and `/.well-known/agent.json`: - **Task ↔ Execution:** an A2A task is an Execution (`task.id` = `execution_id`), and `context_id` is the Session. - **Status updates:** `working` while nodes run. - **`input-required`** at every `interrupt()`. The interrupt payload goes out as a `DataPart`, with a text rendering for plain clients. - **User reply:** a text part becomes an answer to pending questions. A `DataPart` with `{"approval": {"phase", "decision": "approve" | "changes", "feedback"}}` or `{"permission": {"cloud_read": bool}}` resumes an Approval or permission interrupt. **Approvals are never taken from free text**, which keeps them explicit and recorded. - **Final state:** `completed` after `finalize`, and `failed` on unhandled errors. **`POST /generate`** (existing Starlette app, pre-approved mode): - **Request:** `{request, session_id?, allow_cloud_read?: false}`. - **Response:** today's keys (`execution_id`, `session_id`, `status`, `validation_passed`, `total_loop_iterations`, `current_phase`, `artifacts{8 keys}`, `active_skills`), mapped from the final state. - `artifacts` maps today's 8 keys to the closest Deliverables. `source_discovery_doc` and `source_mermaid_diagram` become the current-state section of `requirements`. - `total_loop_iterations` = total Revisions + 1. - `active_skills` = the Skills loaded during the Execution. The route switches to the new graph in increment 7, behind `AGENT_ENGINE=hierarchical` (default `legacy` until increment 9). ### Recording `app/execution/recorder.py` reuses `DatabaseManager` tables until candidate 03: - `workflow_executions`: one row per Execution, updated at each interrupt and at completion. - `adk_artifacts`: one row per Deliverable submission. Rows for earlier Revisions stay as history. - `orchestrator_review_logs`: one row per Ledger event. `review_status` holds the event type, `reviewer_agent` holds the role or `user`, and `feedback` holds the reason or feedback. - `adk_sessions`: the latest Execution summary for the Session (D9). ### Settings Add to `app/config.py` and `.env.example`: - `CLOUD_PROVIDER=gcp` - `AGENT_ENGINE=legacy|hierarchical` - `ORCHESTRATOR_BUDGET=40`, `SPECIALIST_BUDGET=15`, `MAX_REVISIONS=3` - `CHECKPOINTER=sqlite|postgres`, `CHECKPOINT_DB_PATH=./checkpoints.db` - `A2A_HOST=0.0.0.0`, `A2A_PORT=8000` - `MCP_ENABLE_PREVIEW=true`, `MCP_QUOTA_PROJECT` (defaults to the ADC project) Model API keys follow LiteLLM conventions (`GEMINI_API_KEY` / Vertex ADC). ### Dependencies Add to `requirements.txt`: - `google-adk>=1.25` (Skills are Experimental: pin the exact version in the lock file you use) - `litellm~=1.74` - `a2a-sdk>=0.3` - `aiohttp>=3.10` - `langgraph-checkpoint-sqlite` - `langgraph-checkpoint-postgres` Keep `langgraph` and upgrade it to a version with `interrupt()` and `Send`. The `terraform` CLI is added to the Dockerfile image. ## Testing **Deterministic**, with `FakeAgentRuntime`, which replays scripted `Outcome`s and tool calls per role and task. There are no model calls and no network. - **`tests/execution/test_ledger.py`** (pure): - Ready work respects both thresholds: same-Phase `submitted` and cross-Phase `approved`. - Each refusal case: `delegate` on unready work, `request_approval` with unsubmitted Deliverables, `finish` before Phase 4 is approved. - `reopen` cascades to everything that depends on the Deliverable and withdraws approved Phases. The cap produces an escalation. - Pre-approved `request_approval` grants and records `pre-approved`. - **`tests/execution/test_graph.py`**, with the fake runtime and `MemorySaver`: - **Happy path, pre-approved:** every Deliverable submitted, all 4 Phases approved, files written under `deliverables/executions/{id}/` (tmp dir). - **Parallel start:** after `product_selection` is submitted, one Orchestrator turn can delegate `architecture_diagram` and `design_recommendations` together, and both run. - **One voice:** questions from two parallel Specialists produce a single interrupt holding both. - **Interactive pause and resume:** an Approval interrupt stops the graph. Resuming with `approve` continues, and resuming with `changes` plus feedback reopens. - **Validation failure:** reopening `deployment_guidance` withdraws Phase 2's Approval, re-delegates the IaC author with the findings, and asks for Phase 2 Approval again. - **Budgets:** an exhausted budget hands control to the user with a summary. - **Cloud read:** a tool with `allow_cloud_read=None` produces one permission question. `False` makes the tool refuse. - **Clean Executions:** two Executions in one Session share no state. - **`tests/agents/test_roles.py`:** every `agents/*.md` parses, and every role maps to a Deliverable kind (or `orchestrator`/`pillar-advisor`), with Skills that exist in the provider bundle. - **`tests/test_vendored_skills.py`:** - Vendored Skills match `VENDOR.md`'s commit (by checksum). - Every Skill under `skills/` passes Agent Skills frontmatter rules: kebab-case `name` matching its directory, `description` ≤ 1024 chars. - **`tests/agents/test_adk_runtime.py`:** builds an ADK agent for one role with a stub model. It confirms that the `SkillToolset` holds only that role's Skills, that the tool list matches the role's tiers and MCP servers, and that each `McpToolset` has the server's `tool_filter`. It does not call a model. - **`tests/tools/test_mcp.py`:** uses a local fake MCP server over streamable HTTP, started in the test. - Tools outside `allow` are never exposed. - An allowed tool whose `readOnlyHint` isn't true makes the server fail to load. - A `cloud_read` MCP tool call without permission is blocked by `before_tool_callback`. - An unreachable server returns a structured discovery gap and doesn't raise. - `google_adc` headers include `X-Goog-User-Project`. - **`tests/test_provider_bundle.py`:** every `mcp_servers` entry has `tier` and a non-empty `allow`. `developer_knowledge` is the only `general` server. Every `role_mcp_servers` id exists. - **`tests/a2a/test_executor.py`:** A2A task → `input-required` on interrupt → `DataPart` approval resumes, using the fake runtime. A free-text "approve" does **not** grant. - **`tests/test_agent_api.py`:** the `/generate` response keys are unchanged under both `AGENT_ENGINE` values. **Evaluation** (`eval/`): - **Harness:** runs `benchmark_cases.json` in pre-approved mode with the real `AdkRuntime`. Each case adds `scripted_answers: {question_pattern: answer}`, and the pre-approved `await_user` uses these answers before falling back to assumptions. - **Scoring:** existing rubric metrics, plus checks that the Solution guide has every `output-template.md` heading and that the product selection names at least one alternative with pros and cons. - **Not in the default test run.** It lives under `eval/` and is marked. ## Build increments These are ordered, and each leaves the suite green. They are called *increments* so they don't clash with the glossary's **Phase**. 1. **Environment and baseline.** - Create a venv and install dependencies, plus `terraform`. - Run the current suite and record the baseline under `## Comments`. - Create the branch. 2. **Vendored Skills and provider bundle.** - Add `skills/vendor/google/` at `81a31a6` with `VENDOR.md` and the licence, plus the sync script. - Add `providers/gcp.yaml` and `app/providers.py`. - Add the vendoring and frontmatter tests. 3. **Deliverables and Ledger,** test-first: `deliverables.py`, `ledger.py`, `test_ledger.py`. 4. **Execution graph,** test-first: - `runtime.py` (the interface and outcome types), `fake_runtime.py`. - `state.py`, `tools.py`, `graph.py`, `recorder.py`. - Checkpointer settings. - `test_graph.py`. 5. **ADK runtime and role files:** `adk_runtime.py`, `roles.py`, all `agents/*.md`, `test_roles.py`, `test_adk_runtime.py`. 6. **Tools and remote MCP servers:** - General and cloud-read local tools (`terraform` in a temp dir). - `app/tools/mcp.py`, `google_adc` auth and the `McpToolset` mapping. - **Allowlists:** run `tools/list` against each Google Cloud MCP server in a sandbox project, review the read-only tools, and write the `allow` lists into `providers/gcp.yaml`, recording the reviewed tool lists and date under `## Comments`. - `docs/operations/mcp-access.md`: the APIs to enable, viewer roles, and blocking read-write MCP use with IAM deny. - The `skills/cloud-discovery` and `skills/iac-validation` Skills. - The tests in `test_mcp.py` and `test_provider_bundle.py`. 7. **Interfaces:** - The A2A server and executor, `agent_card.json`, `test_executor.py`. - `/generate` behind `AGENT_ENGINE`, and the Dockerfile exposing :8000 and :8080. 8. **Evaluation:** scripted answers, template-heading scoring, and a real-model eval run with results recorded under `## Comments`. Proceed to 9 only when the pass rate meets the current harness threshold (≥ 80% per case). 9. **Cleanup** (carried over from the superseded spec): - **Delete:** `app/tools/gcp_scanner.py`, `app/tools/mcp_developer_knowledge.py` (replaced by remote MCP servers), `app/adk/`, `app/nodes/`, `app/states/`, `app/workflows/gcp_architecture_graph.py`, `app/skills/` (the loader and the 5 non-compliant Skills), `tests/test_graph_workflow.py`, the ADK and orchestrator tests in `tests/test_adk_architecture.py` (keeping the DB test as `tests/test_database.py`), and `test_adk_mcp_function_tools`. - **Fix:** remove the unreachable duplicate `return` in `routes.py`. Rename `ADK_MAX_LOOP_ITERATIONS` → removed, and delete `ADK_ENABLE_ARTIFACT_STORE`. - **Update:** - Make `AGENT_ENGINE=hierarchical` the only engine. - Derive `app/card.py` / `agent_card.json` capabilities from `DEPENDENCY_GRAPH` and the role files. - Rewrite `workflow.yaml` phases to upstream's 4 with their Deliverables. - Rewrite the README around the Orchestrator, Specialists, Skills and A2A. - **Stop** the writes to shared paths (`deliverables/{as-is,target,validation,guides}/`, `docs/*.md`, `terraform/main.tf`). - Push the branch. ## Acceptance - [ ] **Upstream parity:** a pre-approved evaluation run produces a Solution guide containing every heading of the upstream `output-template.md`, and all benchmark cases pass the threshold. - [ ] **Single source for the workflow:** Phases, Deliverables and Dependencies are listed only in `app/execution/deliverables.py`, and card and manifest data is derived from it. - [ ] **No framework types outside `app/agents/adk_runtime.py`:** `rg -n "google\.adk" app | grep -v adk_runtime.py` is empty. After increment 9, `rg -n "app\.adk|OrchestratorLoopAgent|compat" app eval tests` is empty. - [ ] **Vendored Skills unchanged:** they match the `81a31a6` checksums, and every Skill under `skills/` passes the frontmatter rules. - [ ] **Approvals only by structured decision:** no path grants an Approval from free text (covered by tests). - [ ] **Nothing changes cloud resources:** no tool can run `terraform apply`, a deploy or a script (`run_skill_script` absent from the tool lists). Cloud-read tools, MCP included, are unreachable without permission. - [ ] **Discovery uses the provider's remote MCP servers:** current state comes from the Cloud provider's hosted MCP servers. No module under `app/` calls a cloud provider's REST API directly (`rg -n "googleapis.com/(compute|storage|run|sqladmin|pubsub)/v" app` is empty). Every exposed MCP tool is on a reviewed read-only allowlist. - [ ] **Deterministic suite:** every test above passes with no network access. - [ ] **`/generate` contract:** response keys are unchanged. ## Risks - **Specialists submitting without asking (D32).** A Specialist might submit without getting the user's conversational OK. Mitigation: role files require confirmation through `needs_input` before `submit_deliverable`, and the evaluation checks that submissions follow a question. If it's still observed, fall back to Approval per Deliverable (the Q31 alternative). - **ADK Skills are Experimental.** The `SkillToolset` API may change. It's confined to `adk_runtime.py`, and Deep Agents is the planned second implementation. - **Parallel Specialists and rate limits.** Fan-out multiplies concurrent model calls. Cap concurrency per Execution (default 4) in `graph.py`. - **Upstream Skill drift.** Upstream edits may assume a different workflow. Re-sync only through the script, and review the diff against `agents/*.md` and `deliverables.py`. - **Two frameworks in one process.** LangGraph and ADK both manage async state. Keep ADK sessions in memory per run, and never put ADK objects in graph state. - **Remote MCP server availability and drift.** Several servers are Preview (Asset Inventory, Network Intelligence, Billing), each needs its product API enabled in the user's project, and tool sets can change. Mitigations: - allowlists are reviewed, not discovered at run time - a changed `readOnlyHint` fails the server loudly - preview servers can be switched off - an unavailable server becomes a recorded discovery gap, not a failure - **Uneven hosting across Cloud providers.** AWS hosts a managed MCP server. Azure's general MCP server is self-hosted. The bundle's `url` covers both, but "hosted by the Cloud provider" isn't uniform. ## Comments ### 2026-09-15 · Increment 1: environment and baseline **Environment** - Branch `feat/solution-architect-agent` (already created and pushed, commit `4fefe0a`). - `.venv/` (gitignored), Python 3.12.3. Installed `requirements.txt` and `requirements-dev.txt` with no errors. - Resolved versions: - `google-adk` 2.9.0 (the spec's floor is `>=1.25`; `google.adk.skills` and `SkillToolset` import fine, and the toolset also exposes `ListSkillsTool`, `SearchSkillsTool`, `SkillRegistry` and `RunSkillScriptTool`) - `langgraph` 1.2.11, `langgraph-checkpoint` 4.2.0 - `langchain-core` 1.6.3 - `starlette` 1.6.0 - `pytest` 9.1.1, `pytest-asyncio` 1.4.0 - CLIs already on the machine: `terraform` 1.14.3 (`/usr/bin/terraform`), `gcloud`, `psql`. - The real `google-adk` is now installed, but `app.adk.compat.HAS_NATIVE_ADK` is still `False`, because compat also imports `google.adk.workflows` and `google.adk.evaluation`, which fail. The stubs stay in use, so baseline behaviour is unchanged. - No local PostgreSQL is running, so every `DatabaseManager` call logs a connection-refused warning and falls back to `gcp_agent_local.db`. **Baseline** - Command: `.venv/bin/python -m pytest tests eval/test_eval_harness.py -rA`. - Result: **30 passed, 0 failed, 1 warning, ~37 s**. The warning is a Starlette `TestClient` deprecation (`anyio.abc.BlockingPortal`). | File | Passed | |---|---| | `tests/test_adk_architecture.py` | 8/8 | | `tests/test_agent_api.py` | 4/4 | | `tests/test_artifacts.py` | 5/5 | | `tests/test_graph_workflow.py` | 1/1 | | `tests/test_mcp_developer_knowledge.py` | 5/5 | | `tests/test_package.py` | 2/2 | | `tests/test_skill_loader.py` | 3/3 | | `eval/test_eval_harness.py` | 2/2 | Slowest tests: eval benchmark 7.3 s, then 3–4.5 s each for the orchestrator, evaluator, workflow, graph, session API and runner tests. **Side effects found (fix in increment 9 or earlier if they get in the way)** - **New Execution folders.** Each suite run creates 6 new untracked `deliverables/executions//` directories. The shared-path copies are byte-identical because the templates don't vary. I removed the 12 from two runs with `git clean -fd deliverables/executions/`. - **Tracked bytecode.** 30 `__pycache__/*.pyc` files are tracked despite `.gitignore`, so every run dirties the tree (for example `eval/__pycache__/*.pyc`). Candidate fix: `git rm -r --cached '*/__pycache__'`. ### 2026-09-15 · Increment 2: vendored Skills and provider bundle **Done** - `scripts/sync-vendored-skills.sh []` downloads `google/skills` at a full commit sha. It replaces `skills/vendor/google/` wholesale with the 7 Skills plus the upstream `LICENSE` (there's no `NOTICE` upstream), then writes `SHA256SUMS` and `VENDOR.md`. I ran it at `81a31a6e4300425598a69fd96c8e843c947ae308` and diffed the result against the upstream tarball: identical. - `app/agent_skills.py`: - `read_skill` checks one Skill against the Agent Skills specification, and `discover_skills` indexes a set of directories. - It enforces the name rules (1–64 characters, lowercase, single hyphens, matches the directory) and the description rules (1–1024 characters). It also checks `compatibility` (≤ 500 characters), `metadata` (string → string), and that `license`/`allowed-tools` are strings. - It rejects unknown fields, rejects duplicate names across directories, and only looks at immediate child directories. - `app/providers.py`: `load_provider(name, root)` returns a `ProviderBundle` (skills index, output template, `ModelConfig.for_role`, tool tiers, `mcp_auth`, `McpServerSpec`s, `role_mcp_servers`, `mcp_servers_for_role(role, include_preview)`). It validates tiers, https URLs, the `allow` list and role → server references. - `providers/gcp.yaml` holds the bundle from the spec, with **every MCP `allow` list empty** (so no tools are exposed) until the increment 6 review. - Settings: `CLOUD_PROVIDER=gcp` and `MCP_ENABLE_PREVIEW=true` in `app/config.py` and `.env.example`. - Tests: - `tests/test_agent_skills.py`: 17 tests. - `tests/test_vendored_skills.py`: 3 tests. A test run with a vendored file edited showed the checksum test failing, as intended. - `tests/test_providers.py`: 15 tests. - Full suite: **65 passed** (30 baseline + 35 new), same single warning. `ruff check`, `ruff format` and `mypy` are clean on the new modules. **Deviations from the spec text** - **Checksums file.** Checksums live in `skills/vendor/google/SHA256SUMS`, which `VENDOR.md` references, not inside `VENDOR.md` itself. It's easier to diff and works with `sha256sum -c`. - **Where `McpServerSpec` lives.** It's in `app/providers.py`, because it's bundle configuration. `app/tools/mcp.py` (increment 6) will import it and add the runtime behaviour: `readOnlyHint` checks, auth and the ADK mapping. - **Test file name.** The spec named a `tests/test_provider_bundle.py`; this increment uses `tests/test_providers.py`. Increment 6 adds its assertions there, including non-empty `allow` lists once they're reviewed. - **Strict frontmatter.** The validator rejects unknown fields. As a result the 5 legacy `app/skills/*/SKILL.md` fail (the custom `phase:` key), which confirms ADR-0002's consequence. They're out of the bundle's `skills_dirs` and get deleted in increment 9. **Still to check** - The default model ids (`gemini/gemini-2.5-pro`, `gemini/gemini-2.5-flash`) come from the spec and haven't been checked against models currently available. Confirm them in increment 5, when LiteLLM is wired. ### 2026-09-15 · Increment 3: Deliverables and Ledger **Done (test-first)** - `app/execution/deliverables.py`: `PhaseId`, `PHASE_ORDER`, `DeliverableKind`, `DeliverableSpec`, `DEPENDENCY_GRAPH` (the table above), `deliverables_in`, and `check_submission(kind, content, attachments) -> missing`. - `app/execution/ledger.py`: a pure `Ledger`. - Queries: `ready_work`, `phases_ready_for_approval`, `dependencies_met`, `can_finish`. - Work: `start`, `submit`, `release`. - Approvals: `request_approval`, `approve`, `request_changes`. - Revisions: `reopen`. - Persistence: `to_dict` / `from_dict`. - Tests: - `tests/execution/test_deliverables.py`: 12 tests. - `tests/execution/test_ledger.py`: 25 tests. - Full suite: **102 passed**. `ruff` and `mypy` are clean. - **Checking the tests bite.** I broke each of these in the code on purpose, and in each case at least one test failed: - cross-Phase dependencies accepting *submitted* - no cascade on reopen - the revision cap off by one - pre-approved mode not granting approval A fifth mutation, `release` not bumping the attempt, survived. It showed the extra increments were dead code, so I removed them. Attempts now change only in `start`, and a test covers a late submission from a superseded attempt. **Added to or tightened from the spec text** - **Attempts.** `start` returns an attempt number per Deliverable, and `submit`/`release` must present the current one. This protects against late results from Specialists running in parallel after their work was released or reset by a Revision. The spec had no such token. - **More event types.** Beyond the spec's list: `started`, `released`, `reset` (a dependent that went back to pending) and `approval_withdrawn`. - **Return values.** `reopen` returns `ReopenResult(reset, withdrawn_approvals, escalation)`. `reset` lists only Deliverables whose status actually changed. Invalid requests raise `LedgerRefusal` and change nothing, and at the cap `reopen` records `escalated` and changes nothing. `override_cap=True` covers the user deciding to revise past the cap. - **Changes on approval.** `request_changes(phase, feedback)` returns the Phase to `open` and records the feedback, but doesn't reopen Deliverables itself. The Orchestrator reopens the specific ones with `reopen`. - **Dependent feedback.** A dependent that is reset gets a feedback note (`Dependency X was revised: `), so its next Specialist run knows why. - **Single source for progress.** The Ledger owns each Deliverable's status, attempts, revisions and feedback. The Execution state (increment 4) holds only content and attachments, not the `feedback[]`/`revisions` fields the spec's state sketch showed. - **Contracts,** matched against headings after stripping numbering, case and trailing punctuation, by exact match or a space-separated suffix. That way "4.1. Google Cloud products and features mapping" satisfies "Products and features mapping", but "Non-functional requirements" doesn't satisfy "Functional requirements". - `requirements`: functional and non-functional requirements, current state, dependencies. - `product_selection`: products and features mapping. - `design_recommendations`: the 6 pillar headings. - `deployment_guidance`: prerequisites and step-by-step instructions, plus attachments `terraform/main.tf` and `terraform/variables.tf`. - `validation_results`: deployment dry-run, architecture and policy analysis. - `solution_guide`: the template's 8 top-level sections. - `architecture_diagram`: attachment `architecture.mmd`. - `technical_decomposition` and `architecture_description`: non-empty content only. ### 2026-09-15 · Increment 4: Execution graph **Done (test-first)** - `app/agents/runtime.py`: the `AgentRuntime` protocol, `RoleSpec`, `AgentTask`, `ToolSpec`/`ToolResult`, and the outcomes `Submitted | NeedsInput | BudgetExhausted | Stopped | TurnEnded`. - `app/agents/fake_runtime.py`: `FakeAgentRuntime`, which replays scripted tool calls per role through the real `ToolSpec`s, enforces `RoleSpec.budget`, and records every run (task, tools offered, tool messages, outcome). - `app/execution/state.py`: `ExecutionState` (JSON-compatible). `outcomes` uses an append reducer, so parallel Specialist tasks never write the Ledger. - `app/execution/tools.py`: - `OrchestratorToolbox`: `ready_work`, `delegate`, `ask_user`, `request_approval`, `reopen`, `read_deliverable`, `finish`. - `specialist_tools`: `submit_deliverable` (checks the contract and refuses unsafe attachment paths) and `needs_input`, plus the role's extra tools. - `gate_cloud_read`: allow / refuse / ask permission. - `app/execution/graph.py`: `orchestrator_turn` → `Send` × N `specialist_task` → `orchestrator_turn`; `await_user` (`interrupt`); `finalize`; `end_with_warnings`. - `app/execution/engine.py`: **`ExecutionEngine.start(request, session_id, pre_approved, allow_cloud_read)` / `resume(execution_id, reply)` / `interaction(execution_id)` → `Interaction(status, prompt, state)`**. This is the one interface for A2A, `/generate` and the eval harness. The tests use only it. - `app/execution/recorder.py`: the `ExecutionRecorder` protocol with `InMemoryRecorder` and `DatabaseRecorder` (the existing tables via `asyncio.to_thread`). - `app/execution/checkpointing.py`: `open_checkpointer(settings)` for `memory | sqlite | postgres`. - Settings `ORCHESTRATOR_BUDGET`, `SPECIALIST_BUDGET`, `MAX_REVISIONS`, `MAX_ORCHESTRATOR_TURNS`, `CHECKPOINTER` (default `sqlite`) and `CHECKPOINT_DB_PATH`, plus `.env.example`. `requirements.txt`: `langgraph>=1.2`, `langgraph-checkpoint-sqlite`, `langgraph-checkpoint-postgres`, `psycopg[binary]`. - Tests: - `tests/execution/test_engine.py`: 17 tests covering the pre-approved happy path with written files, parallel start, dependency context, refusal messages, incomplete submissions, Approval pause/resume, free text never approving, changes → Revision → re-approval, one voice, Revision after a validation failure, pre-approved escalation, Orchestrator and Specialist budgets, cloud-read ask/grant/deny, clean Executions, and resuming from a durable SQLite checkpoint. - `tests/execution/test_recording.py`: 2 tests covering `DatabaseRecorder` against a temp SQLite database, and the checkpointer factory. - Full suite: **121 passed**. `ruff` and `mypy` are clean on `app/agents` and `app/execution`. - **Checking the tests bite.** I broke each of these in the code on purpose, and each break failed at least one test: - no cloud-read gate - permission never asked - Specialist questions dropped - submission contract unchecked **Added to or changed from the spec text** - **Engine and file names.** There's a new `app/execution/engine.py` facade, and tests live in `test_engine.py`/`test_recording.py`, not `test_graph.py`. The graph is an internal detail behind the engine. - **`Stopped` outcome.** An agent that ends without a turn-ending tool is `Stopped`. Tools end a run by returning `ToolResult(outcome=...)`, so ending can depend on the result: pre-approved `request_approval` doesn't end the turn, and refusals never do. Engine-owned tools use `tier="execution"`. - **Prompt and reply shapes.** - `Interaction.prompt` is `{"questions": [{"from": , "question"}]}`, `{"approval": {"phase", "summary", "deliverables"}}` or `{"permission": {"tier", "requested_by"}}`. - `resume` takes `{"reply": str}`, `{"approval": {"phase", "decision": approve|changes, "feedback"}}` or `{"permission": {"cloud_read": bool}}`. - An approval reply that isn't structured goes into the conversation and never grants. - **Single writer.** Specialist outcomes are applied by the next `orchestrator_turn` in graph order, using `outcomes_applied`. A superseded result (stale attempt) is discarded and noted to the Orchestrator. - **Specialist problems as questions.** A Specialist that runs out of budget or stops is released and relayed as a question from its Deliverable, keeping one voice. A permission `NeedsInput` becomes a single permission prompt, not a question. - **Pre-approved mode.** `allow_cloud_read=None` becomes `False`. An escalation, or an Orchestrator that can't continue, ends as `completed_with_warnings` with no files written. - **Loop guard.** `MAX_ORCHESTRATOR_TURNS` (default 100) bounds loops, and the LangGraph `recursion_limit` is raised to 10 000. - **Conversation.** It records the Orchestrator's rendered prompts as well as user replies. - **Session name.** `DatabaseRecorder` session rows use `agent_name="solution-architect"`. - **Deferred to increment 5:** the design advisor's pillar fan-out tool, which needs the `pillar-advisor` role file. - **Not tested:** `AsyncPostgresSaver` against a real PostgreSQL (none runs locally). The import and wiring are in place. ### 2026-09-16 · Increment 5: ADK runtime and role files **Done (test-first)** - **Role files** `agents/*.md` (11): `orchestrator`, the 9 Specialists and `pillar-advisor`, plus `agents/partials/specialist-protocol.md`, which every Specialist includes. Each Specialist file names its Deliverable, the Skill task it covers, its constraints and its required headings. The shared protocol covers context fields, questions through `needs_input`, confirming a draft with the user before submitting (or recording assumptions when pre-approved), and resubmitting until the contract passes. - `app/agents/roles.py`: `load_roles(agents_dir, bundle, orchestrator_budget, specialist_budget, include_preview_mcp)`. It validates the file name against the role, the deliverable against `DEPENDENCY_GRAPH`, the tool tiers, the budget, the includes, a non-empty body, unknown fields, and missing or unexpected roles. - `app/agents/adk_runtime.py`: `AdkAgentRuntime(skill_dirs, model_factory)`, the only module importing `google.adk`. - **Per run:** a fresh `LlmAgent`, an in-memory session and a `Runner`. - **Tools:** each `ToolSpec` is adapted to a `BaseTool` with a JSON-schema declaration, and a `SkillToolset` holding only the role's Skills is filtered to `list_skills`/`load_skill`/`load_skill_resource`. - **Instructions:** passed as a callable, so role text is never state-templated. - **Stopping:** a `before_tool_callback` budget counts every tool call, including Skill tools. Tool calls within a run are serialised, the first outcome ends the run (later calls in the same batch are refused), and a final text with no outcome becomes `Stopped(text)`. - **Model:** `default_model_factory` wraps `role.model` in `LiteLlm`. - `app/execution/pillars.py`: the `consult_pillar_advisors(brief, pillars?)` tool, which runs one `pillar-advisor` per pillar in parallel through `AgentRuntime`, each scoped to its pillar Skill with a `submit_advice` tool. One failed advisor is reported without losing the others. `app/execution/toolkit.py`: `make_tool_factory(runtime, roles, pillar_skills)` gives the tool to the design advisor only; increment 6 extends it. - `app/execution/deliverables.py`: `inputs_of(kind)` gives a Specialist every Deliverable of earlier Phases plus everything it depends on within its own Phase. Before this, the packager's context lacked `architecture_description`, which nothing depends on. - `providers/gcp.yaml` and `app/providers.py`: `role_skills` and `pillar_skills`, both validated against the Skills index. Models are now `gemini/gemini-pro-latest` by default and `gemini/gemini-flash-latest` for the pillar advisor and diagrammer. - `requirements.txt`: `google-adk[extensions]==2.9.0`, pinned exactly because Skills are Experimental. Its extras bring LiteLLM (installed 1.101.0). - Tests: - `tests/agents/test_roles.py`: 14 tests. - `tests/agents/test_adk_runtime.py`: 12 tests with a scripted `BaseLlm`, no network. - `tests/agents/test_adk_engine.py`: 1 test running `ExecutionEngine` on the ADK runtime with the real role files and bundle. A Specialist question reaches the user and the answer reaches the next run. - `tests/execution/test_pillars.py`: 6 tests. - Plus provider and deliverable tests. - Full suite: **158 passed**. `ruff` and `mypy` are clean. The 5 warnings are ADK and Starlette deprecations. - **Checking the tests bite.** I broke each of these in the code on purpose, and each break failed at least one test: - every Skill loadable by every role - no budget - no early stop after an outcome - templated instructions - both turn-ended guards removed at once Either guard alone suffices under ADK 2.9's sequential callback order; both are kept as defence in depth. **Deviations from the spec text** - **Role frontmatter is provider-neutral.** Role files carry only `role`, `deliverable`, `tool_tiers`, `budget` and `includes`. Skills (`role_skills`, `pillar_skills`), MCP servers and models come from the provider bundle, so no role file names a Google Cloud Skill (a test checks this). `upstream_task` was dropped: role bodies describe the Skill task in neutral words. - **Model ids.** They're LiteLLM `-latest` aliases, not the spec's `gemini-2.5-*` ids, which are older than LiteLLM's current Gemini 3.x entries. - **LiteLLM version.** `litellm` comes from `google-adk[extensions]` (ADK requires ≥ 1.84), not `litellm~=1.74`. - **Pillar fan-out** is a tool on the design advisor, not code inside the runtime. **Deferred** - **To increment 6:** mapping `RoleSpec.mcp_servers` to ADK MCP toolsets (`McpToolset` needs the `mcp` extra, not installed) and supplying local tools by `tool_tiers`. Both are declared and loaded now, but the runtime doesn't use them yet. - **Not yet run against a real model:** no API key was used, so role-file quality is unproven until increment 8's evaluation. - **Model API errors propagate** from `AgentRuntime.run` and fail the graph step, leaving the Execution at its last checkpoint. Decide in increment 7 whether to retry, surface as a question, or fail the A2A task.