Increment 7 of .scratch/solution-architect-agent/spec.md. - app/assembly.py: composition root building the engine from settings and the provider bundle (roles, ADK runtime, MCP catalog, Terraform tools, checkpointer, recorder). - app/a2a: a2a-sdk 1.1 JSON-RPC server (v0.3 compatible) mapping tasks to Executions; questions, Approvals and permissions pause as input-required; Approvals accepted only as structured data; completed tasks return the guide and code as artifacts; agent card derived from the dependency graph. - POST /generate runs a pre-approved Execution with the legacy keys. - Specialist exceptions are relayed to the user; Orchestrator exceptions fail the task. Engine accepts a caller-chosen execution id. - Dockerfile installs checksum-verified Terraform and ships agents, providers and skills; add mcp and a2a-sdk requirements. - 17 new tests; full suite 206 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
69 KiB
Spec: Solution architect agent
A portable, hierarchical ReAct agent that replaces the upstream google-cloud-solution-architecture skill. An Orchestrator reasons over a code-declared dependency graph of Deliverables and delegates to Specialists. All agents run vendored Agent Skills through a framework's own skill runtime. Users grant Approvals at Phase boundaries.
- Glossary:
CONTEXT.md. Use its terms: Execution, Session, Phase, Approval, Revision, Orchestrator, Specialist, Skill, Cloud provider, Deliverable, Dependency, Solution guide. - Decisions: ADR-0002 (Skills-first, hybrid, hierarchical and ReAct, vendoring, Cloud provider as configuration) and ADR-0003 (LangGraph state with ADK agents).
- Supersedes:
.scratch/phase-pipeline/spec.md. Its decision log (D1 to D46) records how this design was reached. - Branch:
feat/solution-architect-agent, pushed to the Gitea remote. A human opens the PR.
Problem
The current agent is a linear chain of five template functions. They ignore the Workflow request (design always picks Cloud Run and Pub/Sub). They run under a fake "ADK" layer that never touches google-adk. The skill prompts they build are thrown away. The agent can't ask the user anything, can't take an Approval, can't run the upstream Skill, and can't target another framework or Cloud provider without a rewrite.
Goals
- Upstream parity. It runs the upstream Skill's workflow: requirements discovery with strict separation from solutioning, grounded product selection with alternatives, diagram, description, WAF-pillar design recommendations, IaC deployment guidance, dry-run validation, and packaging into the upstream template.
- Hierarchical and ReAct. Work follows Dependencies, not a fixed order, and independent work runs in parallel.
- Interactive by default, with code-enforced, recorded Approvals at Phase boundaries. There is also a pre-approved mode for batch and evaluation runs.
- Discovery through the Cloud provider's own remote MCP servers. Current state is discovered through remote MCP servers hosted by the Cloud provider, such as Google Cloud's Resource Manager, Asset Inventory, Compute and GKE servers. Grounding uses the provider's knowledge MCP server. No hand-written cloud REST scanners.
- Portable.
- Behaviour lives in Skills and framework-neutral role files.
- Framework types stay behind one agent-runtime interface.
- Everything specific to a Cloud provider lives in a provider bundle.
- Deterministic tests check every rule the code owns, without a model.
Non-goals (v1)
- Runtime validation. Upstream Task 3.2, verifying after the user has deployed, is out of scope.
- Changing cloud resources. No
applyor deploy tool, and ADK'srun_skill_scriptis off. - A second Cloud provider or a second agent-runtime implementation (Deep Agents). The design leaves room for both; neither is built.
- Architecture review candidates 02 to 06, beyond what this spec needs: tools get tier wrappers, not full ports;
DatabaseManageris reused as is; no typed Deliverable records. - Human approval inside a Phase enforced by code. That stays conversational and driven by the Skill (see Risks).
- Acting as the end user in their cloud (OAuth). Remote MCP calls use the agent's own identity (a service account or Application Default Credentials) in v1.
- Routing MCP calls through the platform's MCP gateway (
mcp_gateway_client.pyin the base agents). Remote MCP servers are called directly (D46). If a gateway is needed later, it becomes a newmcp_auth.kind: gatewayimplementation inapp/tools/mcp.py, and nothing else changes.
Decisions
These are carried from the decision log. The numbers refer to it.
| Area | Decision |
|---|---|
| Sequencing (D19, D23) | Hybrid. Code owns Dependencies, Phase boundaries and Approvals, and the Skill owns the work inside a Deliverable. The agent is hierarchical and ReAct, not linear. |
| Engine (D24) | LangGraph + ADK. A LangGraph StateGraph holds Execution state, checkpoints and interrupt(). The Orchestrator and Specialists are ADK LlmAgents with SkillToolset, behind an AgentRuntime interface. |
| Phases (D25) | Upstream's 4 Phases, as groupings of Deliverables. Discovery of current state is done through tools (remote MCP servers, D45). |
| Approvals (D26, D32) | Code enforces Approval at Phase boundaries. A Dependency inside a Phase is met on submitted, and one across Phases only on Phase approved. Every user question is an interrupt(). |
| Completion (D27) | Agents finish by calling a submit tool that is checked against the Deliverable's contract. |
| Transport (D28) | A2A for interactive Executions. /generate is kept for pre-approved ones. |
| Hierarchy (D29) | An Orchestrator over one Specialist per Deliverable. The design advisor fans out to 6 pillar advisors. |
| Dependency graph (D30) | Declared in code. The ReAct Orchestrator picks work through tools that refuse work whose Dependencies aren't met. |
| Budgets (D31) | Per-agent ReAct step budgets. Hitting one hands control to the user. |
| Voice (D33) | Only the Orchestrator talks to the user. Specialists ask through needs_input. |
| Specialist memory (D34) | Stateless per delegation. Only the LangGraph checkpoint persists. |
| Revision (D35) | Reopening a Deliverable returns it and everything that depends on it to draft and withdraws affected Approvals. Revisions are capped, and past the cap the user decides. |
| Models (D36) | Configurable per role via ADK LiteLLM, with a Gemini default. |
| Dependency graph content (D38) | See Dependency graph. Runtime validation is out of scope. |
| Skill slicing (D39) | A shared SkillToolset plus agents/<role>.md. Pillar advisors are scoped to their own WAF Skill. |
| Vendoring (D21, D40) | Upstream solution-architecture + 6 WAF Skills, unmodified, pinned to 81a31a6, in skills/vendor/google/. |
| Tool tiers (D41) | General tools (local checks, public documentation) run freely. Reading live cloud resources needs permission once per Execution. Changing cloud resources is never exposed. |
| Discovery (D45) | The Cloud provider's own remote MCP servers, declared in the provider bundle, provide discovery and grounding. Every tool that reads the user's cloud is cloud_read. Only tools on an explicit read-only allowlist are exposed. GCPEnvironmentScanner is retired. |
| MCP routing (D46) | Call remote MCP servers directly, with no platform MCP gateway for now. |
| Outputs (D42) | Deliverables live in state and the database. Files go to deliverables/executions/{id}/ only after Phase 4 Approval. |
| Tests (D43) | A fake AgentRuntime for deterministic tests. The real-model evaluation runs pre-approved with scripted answers. |
| Migration (D44) | Build alongside the current code. One cleanup increment follows once the evaluation passes. |
| Carried over (D9, D14) | Every Execution starts clean, and a Session only groups Executions. The Execution graph and agent runtime are built once at startup. |
Design
Module layout
New code only. The current modules stay untouched until increment 9.
agents/ # framework-neutral role files (one per role)
orchestrator.md
requirements-analyst.md decomposer.md product-selector.md diagrammer.md
describer.md design-advisor.md pillar-advisor.md iac-author.md
validator.md packager.md
providers/
gcp.yaml # Cloud provider bundle
skills/
vendor/google/ # unmodified upstream Skills + VENDOR.md + LICENSE
google-cloud-solution-architecture/
google-cloud-waf-{security,reliability,cost-optimization,operational-excellence,performance-optimization,sustainability}/
cloud-discovery/SKILL.md # our tooling Skills (Agent Skills spec compliant)
iac-validation/SKILL.md
app/
execution/
deliverables.py # DeliverableKind, PhaseId, DEPENDENCY_GRAPH, contracts
ledger.py # Ledger: Deliverable/Phase states, acceptance, Revision (pure, no I/O)
graph.py # LangGraph StateGraph: orchestrator turn, Specialist fan-out, interrupts
tools.py # Orchestrator and Specialist execution tools bound to the Ledger
state.py # ExecutionState TypedDict
recorder.py # records submissions, Approvals, Revisions, final files
agents/
runtime.py # AgentRuntime protocol, RoleSpec, AgentTask, Outcome types
roles.py # loads agents/*.md into RoleSpec
adk_runtime.py # ADK LlmAgent + SkillToolset + LiteLlm implementation
fake_runtime.py # scripted implementation for tests (under app/ so eval can reuse it)
tools/
general/ # mermaid check, terraform fmt/validate (local)
cloud_read/ # terraform plan (local CLI, reads live state)
mcp.py # McpServerSpec, read-only allowlist, tier + permission metadata
providers.py # loads providers/<name>.yaml
a2a/
server.py # aiohttp A2A server on :8000, agent_card.json
executor.py # maps A2A tasks <-> Executions and interrupts
scripts/sync-vendored-skills.sh
Deliverables and the dependency graph
app/execution/deliverables.py declares the graph as data. It is the only place Phases, Deliverables and Dependencies are listed.
| Phase | Deliverable kind | Specialist role | Depends on | Tools (tier) |
|---|---|---|---|---|
1 requirements_discovery |
requirements |
requirements-analyst |
(none) | discovery MCP servers (cloud_read) |
| 1 | technical_decomposition |
decomposer |
requirements |
(none) |
2 solution_architecture |
product_selection |
product-selector |
technical_decomposition |
knowledge MCP (general) |
| 2 | architecture_diagram |
diagrammer |
product_selection |
mermaid check (general) |
| 2 | architecture_description |
describer |
architecture_diagram, product_selection |
knowledge MCP (general) |
| 2 | design_recommendations |
design-advisor → 6 × pillar-advisor |
product_selection, requirements |
knowledge MCP (general) |
| 2 | deployment_guidance |
iac-author |
product_selection, design_recommendations |
terraform fmt/validate (general) |
3 solution_validation |
validation_results |
validator |
deployment_guidance |
terraform validate (general), terraform plan and policy MCP servers: IAM, Network Intelligence (cloud_read) |
4 packaging |
solution_guide |
packager |
every Phase 3 Deliverable | (none) |
Deliverable contract. Each kind declares its required content in deliverables.py:
sections: required markdown headings.attachments: named code blobs, for exampledeployment_guidancerequiresterraform/main.tfandterraform/variables.tf.
submit_deliverable checks the contract. A failing submission returns what's missing to the Specialist, and nothing is recorded.
The Ledger: acceptance and Revision rules
app/execution/ledger.py is a pure module. It holds every rule code owns, and the tests exercise it heavily.
- Deliverable states:
pending → in_progress → submitted. Only a Revision goes backwards. - Phase states:
open → awaiting_approval → approved. A Phase becomesawaiting_approvalwhen the Orchestrator callsrequest_approvalwith all its Deliverablessubmitted. - When a Dependency is met:
- Same Phase: the Dependency is
submitted. - Across Phases: the Dependency's Phase is
approved.
- Same Phase: the Dependency is
- Ready work: Deliverables that are
pendingand have every Dependency met. reopen(kind, reason):- The Deliverable goes to
pendingwithreasonappended to its feedback. - Everything that depends on it, directly or not, goes to
pending. - Every affected Phase that was
approvedorawaiting_approvalgoes back toopen. - The Deliverable's
revisionscount goes up. At the cap (max_revisions, default 3)reopenrefuses and returns an escalation the Orchestrator must put to the user.
- The Deliverable goes to
- Pre-approved mode:
request_approvalgrants immediately and recordsapproved_by = "pre-approved". - Records: every transition appends an event (
submitted,approval_requested,approved,changes_requested,reopened,escalated) for the recorder.
Execution graph (LangGraph)
app/execution/graph.py compiles once at startup with the configured checkpointer: SqliteSaver locally and in tests, PostgresSaver in production. The Execution id is the LangGraph thread_id.
State (ExecutionState):
execution_id,session_id,workflow_request,mode: "interactive" | "pre_approved",allow_cloud_read: bool | Noneledger(serialisable),deliverables: {kind: {content, attachments, feedback[], revisions}}conversation[](user ↔ Orchestrator turns only),pending_questions[],pending_delegations[]budget_exhausted: {role, summary} | None
Nodes and edges:
START → orchestrator_turn
orchestrator_turn ──(delegations)──────────> specialist_task × N (LangGraph Send, parallel)
orchestrator_turn ──(questions/approval)───> await_user (interrupt)
orchestrator_turn ──(finish)───────────────> finalize → END
specialist_task ───────────────────────────> orchestrator_turn (outcomes merged into ledger)
await_user ────────────────────────────────> orchestrator_turn (resume value = user reply / approval decision)
orchestrator_turn: runs one Orchestrator ReAct turn throughAgentRuntime, with Orchestrator tools bound to a working copy of the Ledger. The turn ends when the agent does one of these, and the node routes on it:- calls
delegate,ask_user,request_approvalorfinish - runs out of budget
- calls
specialist_task: runs one Specialist delegation. Its task holds the role, the Deliverable kind, the Dependency contents, feedback, and the answers to its earlierneeds_inputquestions. The outcome is one of:Submitted(content, attachments), applied to the LedgerNeedsInput(questions), added topending_questionswith the Deliverable back topendingBudgetExhausted(summary)
await_user:interrupt({"questions": [...]} | {"approval": {"phase": ..., "summary": ...}} | {"permission": "cloud_read"} | {"escalation": ...}). In pre-approved mode it doesn't pause:- Approvals: granted automatically.
cloud_readpermission: taken from the request.- Questions: answered with
"No user is available. State your assumption, record it in the Deliverable under 'Assumptions', and proceed." - Escalations: end the Execution as
completed_with_warnings.
finalize: runs after Phase 4 isapproved. It writes files todeliverables/executions/{execution_id}/: the Solution guide, every Deliverable as markdown, and attachments underterraform/. It then records completion.
Orchestrator tools (app/execution/tools.py):
| Tool | Effect |
|---|---|
ready_work() |
Lists ready Deliverables and Phases ready for Approval. |
delegate(kinds: list[str]) |
Refuses any kind that isn't ready and sets it in_progress. Ends the turn and fans out. |
ask_user(questions: list[str]) |
Combines Specialist questions with its own. Ends the turn and interrupts. |
request_approval(phase: str) |
Refuses unless every Deliverable in the Phase is submitted. Ends the turn and interrupts. |
reopen(kind: str, reason: str) |
Applies the Revision rule, or returns an escalation. |
read_deliverable(kind: str) |
Returns the current content, so the Orchestrator can summarise it for Approval. |
finish() |
Refuses unless Phase 4 is approved. |
Specialist tools: submit_deliverable(content, attachments) (checks the contract), needs_input(questions), the SkillToolset, and the role's tier tools. Cloud-read tools check allow_cloud_read. If it's None, the tool returns NeedsInput(["May I read live resources in your Google Cloud project to …?"]) so the Orchestrator asks once. If it's False, it returns a refusal the agent must work around.
Design advisor fan-out. The design-advisor role runs a nested fan-out itself: it sends one task to pillar-advisor for each WAF pillar, in parallel, through AgentRuntime, and combines the results into its submission. Pillar results are not Deliverables and don't appear in the Ledger.
Agent runtime interface
app/agents/runtime.py:
class AgentRuntime(Protocol):
async def run(self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]) -> Outcome: ...
@dataclass(frozen=True)
class RoleSpec: # parsed from agents/<role>.md
name: str
instructions: str # markdown body
skills: tuple[str, ...] # Skill names from the provider bundle
tool_tiers: tuple[str, ...] # subset of ("general", "cloud_read")
mcp_servers: tuple[str, ...] # server ids from the provider bundle this role may use
model: str | None # overrides provider default
budget: int # max tool calls per run
Outcome = Submitted | NeedsInput | BudgetExhausted | TurnEnded # TurnEnded carries the Orchestrator's ending tool call
ToolSpec: framework-neutral (name, description, JSON schema, async callable).adk_runtime.pybuilds each agent from these pieces:ToolSpec→ ADKFunctionTool.- The role's MCP servers → ADK
McpToolsetover streamable HTTP, withtool_filterset to the server's read-only allowlist. SkillToolset(skills=[load_skill_from_dir(...)])with only the role's Skills.modelwrapped inLiteLlm.
- Per run: a fresh
InMemorySessionService(D34). - Callbacks: a
before_tool_callbackcounts tool calls againstbudgetand enforces thecloud_readpermission for every tool in that tier, MCP tools included. - No framework types cross this interface.
Role file format (agents/<role>.md):
---
role: diagrammer
deliverable: architecture_diagram
upstream_task: "Phase 2, Task 2.2: Generate an architecture diagram"
skills: [google-cloud-solution-architecture]
tool_tiers: [general]
mcp_servers: []
budget: 15
---
You produce the architecture diagram Deliverable. Load the `google-cloud-solution-architecture` Skill and follow its Task 2.2 ...
Submit with `submit_deliverable` once the diagram passes the Mermaid check and the user has confirmed it through `needs_input`.
pillar-advisor.md is parameterised by pillar, and its skills value is resolved per task to the one matching google-cloud-waf-<pillar>.
Cloud provider bundle
providers/gcp.yaml, loaded by app/providers.py:
name: gcp
skills_dirs: [skills/vendor/google, skills]
output_template: skills/vendor/google/google-cloud-solution-architecture/assets/output-template.md
models:
default: gemini/gemini-2.5-pro
roles: { pillar-advisor: gemini/gemini-2.5-flash, diagrammer: gemini/gemini-2.5-flash }
tools:
general: [mermaid_check, terraform_fmt, terraform_validate]
cloud_read: [terraform_plan]
mcp_auth:
kind: google_adc # service account / ADC bearer token; sends X-Goog-User-Project
mcp_servers: # remote MCP servers hosted by the Cloud provider (streamable HTTP)
developer_knowledge:
url: https://developerknowledge.googleapis.com/mcp
tier: general # public documentation, no customer data
allow: [search_documents, get_documents, answer_query]
resource_manager: { url: https://cloudresourcemanager.googleapis.com/mcp, tier: cloud_read, allow: [<read-only tools>] }
asset_inventory: { url: https://cloudasset.googleapis.com/mcp, tier: cloud_read, allow: [...], preview: true }
compute: { url: https://compute.googleapis.com/mcp, tier: cloud_read, allow: [...] }
gke: { url: https://container.googleapis.com/mcp, tier: cloud_read, allow: [...] }
cloud_run: { url: https://run.googleapis.com/mcp, tier: cloud_read, allow: [...] }
cloud_sql: { url: https://sqladmin.googleapis.com/mcp, tier: cloud_read, allow: [...] }
storage: { url: https://storage.googleapis.com/storage/mcp, tier: cloud_read, allow: [...] }
pubsub: { url: https://pubsub.googleapis.com/mcp, tier: cloud_read, allow: [...] }
iam: { url: https://iam.googleapis.com/mcp, tier: cloud_read, allow: [...] }
monitoring: { url: https://monitoring.googleapis.com/mcp, tier: cloud_read, allow: [...] }
network_intelligence: { url: https://networkmanagement.googleapis.com/mcp, tier: cloud_read, allow: [...], preview: true }
role_mcp_servers:
requirements-analyst: [resource_manager, asset_inventory, compute, gke, cloud_run, cloud_sql, storage, pubsub, iam, monitoring]
product-selector: [developer_knowledge]
describer: [developer_knowledge]
design-advisor: [developer_knowledge]
pillar-advisor: [developer_knowledge]
validator: [iam, network_intelligence]
Loading the bundle.
- Settings select the bundle with
CLOUD_PROVIDER=gcp. Core modules don't name a provider. role_mcp_serversoverrides themcp_serverslist in role files, so role files stay neutral across providers.- The
allowlists are written once in increment 6: run MCPtools/liston each server and include only tools whosereadOnlyHintannotation is true and whose names are reviewed as read-only. - Servers marked
preview: truecan be switched off withMCP_ENABLE_PREVIEW=false.
Other Cloud providers reuse the same shape:
- AWS: the managed, generally available AWS MCP Server.
- Azure: Azure MCP Server 2.0 runs as a self-hosted remote server, so its
urlpoints at the team's own deployment.
Tools
- General, local (
app/tools/general/):mermaid_check: moved fromvalidation_tools.validate_mermaid_diagram.terraform_fmtandterraform_validate: run theterraformCLI in a temp dir with-backend=false. There are no credentials, and the tool fails clearly ifterraformis missing.
- Cloud read, local (
app/tools/cloud_read/):terraform_planneeds ADC and-lock=false, and never runsapply. - Remote MCP servers (
app/tools/mcp.py): the Cloud provider's own hosted servers, declared in the provider bundle and used for both discovery and grounding. They replaceGCPEnvironmentScanner(hand-written REST calls) and the hand-rolled JSON-RPCDeveloperKnowledgeMCPClient, including its offline fallback knowledge base and thedata/documentsbug from review candidate 02. Both are deleted in increment 9.McpServerSpec(id, url, tier, allow, preview)is framework-neutral. The ADK runtime maps each one toMcpToolset(StreamableHTTPConnectionParams(url, headers=auth), tool_filter=allow).- Read-only in depth. Tools outside
alloware never offered to any agent. At startup, a server whosetools/listreports an allowed tool withreadOnlyHintnot true fails to load, loudly. Operators should also run the agent's identity with only viewer roles, plus Google Cloud's control that blocks read-write MCP tool use and an IAM deny policy (docs/operations/mcp-access.md, written in increment 6). - Auth:
google_adcgets a bearer token from Application Default Credentials (google.auth.default, refreshed per call) and sendsX-Goog-User-Project. Followsgoogle-adk-base-agent/app/google_mcp_client.py, without thegcloudsubprocess fallback. - Failure: an unreachable, disabled or unauthorised server returns a structured tool error such as
"compute MCP unavailable: API not enabled in project X". The agent records the gap under 'Assumptions / discovery gaps' and doesn't fail the Execution.
- Our tooling Skills (
skills/cloud-discovery,skills/iac-validation): short Agent Skills-compliant guides for agents.cloud-discovery: which discovery servers answer which current-state questions (inventory first through Asset Inventory and Resource Manager, then per-product detail), and how to summarise current state.iac-validation: when and how to run the Terraform checks.
Interaction
A2A (app/a2a/), using a2a-sdk>=0.3 on aiohttp :8000, with agent_card.json and /.well-known/agent.json:
- Task ↔ Execution: an A2A task is an Execution (
task.id=execution_id), andcontext_idis the Session. - Status updates:
workingwhile nodes run. input-requiredat everyinterrupt(). The interrupt payload goes out as aDataPart, with a text rendering for plain clients.- User reply: a text part becomes an answer to pending questions. A
DataPartwith{"approval": {"phase", "decision": "approve" | "changes", "feedback"}}or{"permission": {"cloud_read": bool}}resumes an Approval or permission interrupt. Approvals are never taken from free text, which keeps them explicit and recorded. - Final state:
completedafterfinalize, andfailedon unhandled errors.
POST /generate (existing Starlette app, pre-approved mode):
- Request:
{request, session_id?, allow_cloud_read?: false}. - Response: today's keys (
execution_id,session_id,status,validation_passed,total_loop_iterations,current_phase,artifacts{8 keys},active_skills), mapped from the final state.artifactsmaps today's 8 keys to the closest Deliverables.source_discovery_docandsource_mermaid_diagrambecome the current-state section ofrequirements.total_loop_iterations= total Revisions + 1.active_skills= the Skills loaded during the Execution.
The route switches to the new graph in increment 7, behind AGENT_ENGINE=hierarchical (default legacy until increment 9).
Recording
app/execution/recorder.py reuses DatabaseManager tables until candidate 03:
workflow_executions: one row per Execution, updated at each interrupt and at completion.adk_artifacts: one row per Deliverable submission. Rows for earlier Revisions stay as history.orchestrator_review_logs: one row per Ledger event.review_statusholds the event type,reviewer_agentholds the role oruser, andfeedbackholds the reason or feedback.adk_sessions: the latest Execution summary for the Session (D9).
Settings
Add to app/config.py and .env.example:
CLOUD_PROVIDER=gcpAGENT_ENGINE=legacy|hierarchicalORCHESTRATOR_BUDGET=40,SPECIALIST_BUDGET=15,MAX_REVISIONS=3CHECKPOINTER=sqlite|postgres,CHECKPOINT_DB_PATH=./checkpoints.dbA2A_HOST=0.0.0.0,A2A_PORT=8000MCP_ENABLE_PREVIEW=true,MCP_QUOTA_PROJECT(defaults to the ADC project)
Model API keys follow LiteLLM conventions (GEMINI_API_KEY / Vertex ADC).
Dependencies
Add to requirements.txt:
google-adk>=1.25(Skills are Experimental: pin the exact version in the lock file you use)litellm~=1.74a2a-sdk>=0.3aiohttp>=3.10langgraph-checkpoint-sqlitelanggraph-checkpoint-postgres
Keep langgraph and upgrade it to a version with interrupt() and Send. The terraform CLI is added to the Dockerfile image.
Testing
Deterministic, with FakeAgentRuntime, which replays scripted Outcomes and tool calls per role and task. There are no model calls and no network.
tests/execution/test_ledger.py(pure):- Ready work respects both thresholds: same-Phase
submittedand cross-Phaseapproved. - Each refusal case:
delegateon unready work,request_approvalwith unsubmitted Deliverables,finishbefore Phase 4 is approved. reopencascades to everything that depends on the Deliverable and withdraws approved Phases. The cap produces an escalation.- Pre-approved
request_approvalgrants and recordspre-approved.
- Ready work respects both thresholds: same-Phase
tests/execution/test_graph.py, with the fake runtime andMemorySaver:- Happy path, pre-approved: every Deliverable submitted, all 4 Phases approved, files written under
deliverables/executions/{id}/(tmp dir). - Parallel start: after
product_selectionis submitted, one Orchestrator turn can delegatearchitecture_diagramanddesign_recommendationstogether, and both run. - One voice: questions from two parallel Specialists produce a single interrupt holding both.
- Interactive pause and resume: an Approval interrupt stops the graph. Resuming with
approvecontinues, and resuming withchangesplus feedback reopens. - Validation failure: reopening
deployment_guidancewithdraws Phase 2's Approval, re-delegates the IaC author with the findings, and asks for Phase 2 Approval again. - Budgets: an exhausted budget hands control to the user with a summary.
- Cloud read: a tool with
allow_cloud_read=Noneproduces one permission question.Falsemakes the tool refuse. - Clean Executions: two Executions in one Session share no state.
- Happy path, pre-approved: every Deliverable submitted, all 4 Phases approved, files written under
tests/agents/test_roles.py: everyagents/*.mdparses, and every role maps to a Deliverable kind (ororchestrator/pillar-advisor), with Skills that exist in the provider bundle.tests/test_vendored_skills.py:- Vendored Skills match
VENDOR.md's commit (by checksum). - Every Skill under
skills/passes Agent Skills frontmatter rules: kebab-casenamematching its directory,description≤ 1024 chars.
- Vendored Skills match
tests/agents/test_adk_runtime.py: builds an ADK agent for one role with a stub model. It confirms that theSkillToolsetholds only that role's Skills, that the tool list matches the role's tiers and MCP servers, and that eachMcpToolsethas the server'stool_filter. It does not call a model.tests/tools/test_mcp.py: uses a local fake MCP server over streamable HTTP, started in the test.- Tools outside
alloware never exposed. - An allowed tool whose
readOnlyHintisn't true makes the server fail to load. - A
cloud_readMCP tool call without permission is blocked bybefore_tool_callback. - An unreachable server returns a structured discovery gap and doesn't raise.
google_adcheaders includeX-Goog-User-Project.
- Tools outside
tests/test_provider_bundle.py: everymcp_serversentry hastierand a non-emptyallow.developer_knowledgeis the onlygeneralserver. Everyrole_mcp_serversid exists.tests/a2a/test_executor.py: A2A task →input-requiredon interrupt →DataPartapproval resumes, using the fake runtime. A free-text "approve" does not grant.tests/test_agent_api.py: the/generateresponse keys are unchanged under bothAGENT_ENGINEvalues.
Evaluation (eval/):
- Harness: runs
benchmark_cases.jsonin pre-approved mode with the realAdkRuntime. Each case addsscripted_answers: {question_pattern: answer}, and the pre-approvedawait_useruses these answers before falling back to assumptions. - Scoring: existing rubric metrics, plus checks that the Solution guide has every
output-template.mdheading and that the product selection names at least one alternative with pros and cons. - Not in the default test run. It lives under
eval/and is marked.
Build increments
These are ordered, and each leaves the suite green. They are called increments so they don't clash with the glossary's Phase.
- Environment and baseline.
- Create a venv and install dependencies, plus
terraform. - Run the current suite and record the baseline under
## Comments. - Create the branch.
- Create a venv and install dependencies, plus
- Vendored Skills and provider bundle.
- Add
skills/vendor/google/at81a31a6withVENDOR.mdand the licence, plus the sync script. - Add
providers/gcp.yamlandapp/providers.py. - Add the vendoring and frontmatter tests.
- Add
- Deliverables and Ledger, test-first:
deliverables.py,ledger.py,test_ledger.py. - Execution graph, test-first:
runtime.py(the interface and outcome types),fake_runtime.py.state.py,tools.py,graph.py,recorder.py.- Checkpointer settings.
test_graph.py.
- ADK runtime and role files:
adk_runtime.py,roles.py, allagents/*.md,test_roles.py,test_adk_runtime.py. - Tools and remote MCP servers:
- General and cloud-read local tools (
terraformin a temp dir). app/tools/mcp.py,google_adcauth and theMcpToolsetmapping.- Allowlists: run
tools/listagainst each Google Cloud MCP server in a sandbox project, review the read-only tools, and write theallowlists intoproviders/gcp.yaml, recording the reviewed tool lists and date under## Comments. docs/operations/mcp-access.md: the APIs to enable, viewer roles, and blocking read-write MCP use with IAM deny.- The
skills/cloud-discoveryandskills/iac-validationSkills. - The tests in
test_mcp.pyandtest_provider_bundle.py.
- General and cloud-read local tools (
- Interfaces:
- The A2A server and executor,
agent_card.json,test_executor.py. /generatebehindAGENT_ENGINE, and the Dockerfile exposing :8000 and :8080.
- The A2A server and executor,
- Evaluation: scripted answers, template-heading scoring, and a real-model eval run with results recorded under
## Comments. Proceed to 9 only when the pass rate meets the current harness threshold (≥ 80% per case). - Cleanup (carried over from the superseded spec):
- Delete:
app/tools/gcp_scanner.py,app/tools/mcp_developer_knowledge.py(replaced by remote MCP servers),app/adk/,app/nodes/,app/states/,app/workflows/gcp_architecture_graph.py,app/skills/(the loader and the 5 non-compliant Skills),tests/test_graph_workflow.py, the ADK and orchestrator tests intests/test_adk_architecture.py(keeping the DB test astests/test_database.py), andtest_adk_mcp_function_tools. - Fix: remove the unreachable duplicate
returninroutes.py. RenameADK_MAX_LOOP_ITERATIONS→ removed, and deleteADK_ENABLE_ARTIFACT_STORE. - Update:
- Make
AGENT_ENGINE=hierarchicalthe only engine. - Derive
app/card.py/agent_card.jsoncapabilities fromDEPENDENCY_GRAPHand the role files. - Rewrite
workflow.yamlphases to upstream's 4 with their Deliverables. - Rewrite the README around the Orchestrator, Specialists, Skills and A2A.
- Make
- Stop the writes to shared paths (
deliverables/{as-is,target,validation,guides}/,docs/*.md,terraform/main.tf). - Push the branch.
- Delete:
Acceptance
- Upstream parity: a pre-approved evaluation run produces a Solution guide containing every heading of the upstream
output-template.md, and all benchmark cases pass the threshold. - Single source for the workflow: Phases, Deliverables and Dependencies are listed only in
app/execution/deliverables.py, and card and manifest data is derived from it. - No framework types outside
app/agents/adk_runtime.py:rg -n "google\.adk" app | grep -v adk_runtime.pyis empty. After increment 9,rg -n "app\.adk|OrchestratorLoopAgent|compat" app eval testsis empty. - Vendored Skills unchanged: they match the
81a31a6checksums, and every Skill underskills/passes the frontmatter rules. - Approvals only by structured decision: no path grants an Approval from free text (covered by tests).
- Nothing changes cloud resources: no tool can run
terraform apply, a deploy or a script (run_skill_scriptabsent from the tool lists). Cloud-read tools, MCP included, are unreachable without permission. - Discovery uses the provider's remote MCP servers: current state comes from the Cloud provider's hosted MCP servers. No module under
app/calls a cloud provider's REST API directly (rg -n "googleapis.com/(compute|storage|run|sqladmin|pubsub)/v" appis empty). Every exposed MCP tool is on a reviewed read-only allowlist. - Deterministic suite: every test above passes with no network access.
/generatecontract: response keys are unchanged.
Risks
- Specialists submitting without asking (D32). A Specialist might submit without getting the user's conversational OK. Mitigation: role files require confirmation through
needs_inputbeforesubmit_deliverable, and the evaluation checks that submissions follow a question. If it's still observed, fall back to Approval per Deliverable (the Q31 alternative). - ADK Skills are Experimental. The
SkillToolsetAPI may change. It's confined toadk_runtime.py, and Deep Agents is the planned second implementation. - Parallel Specialists and rate limits. Fan-out multiplies concurrent model calls. Cap concurrency per Execution (default 4) in
graph.py. - Upstream Skill drift. Upstream edits may assume a different workflow. Re-sync only through the script, and review the diff against
agents/*.mdanddeliverables.py. - Two frameworks in one process. LangGraph and ADK both manage async state. Keep ADK sessions in memory per run, and never put ADK objects in graph state.
- Remote MCP server availability and drift. Several servers are Preview (Asset Inventory, Network Intelligence, Billing), each needs its product API enabled in the user's project, and tool sets can change. Mitigations:
- allowlists are reviewed, not discovered at run time
- a changed
readOnlyHintfails the server loudly - preview servers can be switched off
- an unavailable server becomes a recorded discovery gap, not a failure
- Uneven hosting across Cloud providers. AWS hosts a managed MCP server. Azure's general MCP server is self-hosted. The bundle's
urlcovers both, but "hosted by the Cloud provider" isn't uniform.
Comments
2026-09-15 · Increment 1: environment and baseline
Environment
- Branch
feat/solution-architect-agent(already created and pushed, commit4fefe0a). .venv/(gitignored), Python 3.12.3. Installedrequirements.txtandrequirements-dev.txtwith no errors.- Resolved versions:
google-adk2.9.0 (the spec's floor is>=1.25;google.adk.skillsandSkillToolsetimport fine, and the toolset also exposesListSkillsTool,SearchSkillsTool,SkillRegistryandRunSkillScriptTool)langgraph1.2.11,langgraph-checkpoint4.2.0langchain-core1.6.3starlette1.6.0pytest9.1.1,pytest-asyncio1.4.0
- CLIs already on the machine:
terraform1.14.3 (/usr/bin/terraform),gcloud,psql. - The real
google-adkis now installed, butapp.adk.compat.HAS_NATIVE_ADKis stillFalse, because compat also importsgoogle.adk.workflowsandgoogle.adk.evaluation, which fail. The stubs stay in use, so baseline behaviour is unchanged. - No local PostgreSQL is running, so every
DatabaseManagercall logs a connection-refused warning and falls back togcp_agent_local.db.
Baseline
- Command:
.venv/bin/python -m pytest tests eval/test_eval_harness.py -rA. - Result: 30 passed, 0 failed, 1 warning, ~37 s. The warning is a Starlette
TestClientdeprecation (anyio.abc.BlockingPortal).
| File | Passed |
|---|---|
tests/test_adk_architecture.py |
8/8 |
tests/test_agent_api.py |
4/4 |
tests/test_artifacts.py |
5/5 |
tests/test_graph_workflow.py |
1/1 |
tests/test_mcp_developer_knowledge.py |
5/5 |
tests/test_package.py |
2/2 |
tests/test_skill_loader.py |
3/3 |
eval/test_eval_harness.py |
2/2 |
Slowest tests: eval benchmark 7.3 s, then 3–4.5 s each for the orchestrator, evaluator, workflow, graph, session API and runner tests.
Side effects found (fix in increment 9 or earlier if they get in the way)
- New Execution folders. Each suite run creates 6 new untracked
deliverables/executions/<uuid>/directories. The shared-path copies are byte-identical because the templates don't vary. I removed the 12 from two runs withgit clean -fd deliverables/executions/. - Tracked bytecode. 30
__pycache__/*.pycfiles are tracked despite.gitignore, so every run dirties the tree (for exampleeval/__pycache__/*.pyc). Candidate fix:git rm -r --cached '*/__pycache__'.
2026-09-15 · Increment 2: vendored Skills and provider bundle
Done
scripts/sync-vendored-skills.sh [<sha>]downloadsgoogle/skillsat a full commit sha. It replacesskills/vendor/google/wholesale with the 7 Skills plus the upstreamLICENSE(there's noNOTICEupstream), then writesSHA256SUMSandVENDOR.md. I ran it at81a31a6e4300425598a69fd96c8e843c947ae308and diffed the result against the upstream tarball: identical.app/agent_skills.py:read_skillchecks one Skill against the Agent Skills specification, anddiscover_skillsindexes a set of directories.- It enforces the name rules (1–64 characters, lowercase, single hyphens, matches the directory) and the description rules (1–1024 characters). It also checks
compatibility(≤ 500 characters),metadata(string → string), and thatlicense/allowed-toolsare strings. - It rejects unknown fields, rejects duplicate names across directories, and only looks at immediate child directories.
app/providers.py:load_provider(name, root)returns aProviderBundle(skills index, output template,ModelConfig.for_role, tool tiers,mcp_auth,McpServerSpecs,role_mcp_servers,mcp_servers_for_role(role, include_preview)). It validates tiers, https URLs, theallowlist and role → server references.providers/gcp.yamlholds the bundle from the spec, with every MCPallowlist empty (so no tools are exposed) until the increment 6 review.- Settings:
CLOUD_PROVIDER=gcpandMCP_ENABLE_PREVIEW=trueinapp/config.pyand.env.example. - Tests:
tests/test_agent_skills.py: 17 tests.tests/test_vendored_skills.py: 3 tests. A test run with a vendored file edited showed the checksum test failing, as intended.tests/test_providers.py: 15 tests.- Full suite: 65 passed (30 baseline + 35 new), same single warning.
ruff check,ruff formatandmypyare clean on the new modules.
Deviations from the spec text
- Checksums file. Checksums live in
skills/vendor/google/SHA256SUMS, whichVENDOR.mdreferences, not insideVENDOR.mditself. It's easier to diff and works withsha256sum -c. - Where
McpServerSpeclives. It's inapp/providers.py, because it's bundle configuration.app/tools/mcp.py(increment 6) will import it and add the runtime behaviour:readOnlyHintchecks, auth and the ADK mapping. - Test file name. The spec named a
tests/test_provider_bundle.py; this increment usestests/test_providers.py. Increment 6 adds its assertions there, including non-emptyallowlists once they're reviewed. - Strict frontmatter. The validator rejects unknown fields. As a result the 5 legacy
app/skills/*/SKILL.mdfail (the customphase:key), which confirms ADR-0002's consequence. They're out of the bundle'sskills_dirsand get deleted in increment 9.
Still to check
- The default model ids (
gemini/gemini-2.5-pro,gemini/gemini-2.5-flash) come from the spec and haven't been checked against models currently available. Confirm them in increment 5, when LiteLLM is wired.
2026-09-15 · Increment 3: Deliverables and Ledger
Done (test-first)
-
app/execution/deliverables.py:PhaseId,PHASE_ORDER,DeliverableKind,DeliverableSpec,DEPENDENCY_GRAPH(the table above),deliverables_in, andcheck_submission(kind, content, attachments) -> missing. -
app/execution/ledger.py: a pureLedger.- Queries:
ready_work,phases_ready_for_approval,dependencies_met,can_finish. - Work:
start,submit,release. - Approvals:
request_approval,approve,request_changes. - Revisions:
reopen. - Persistence:
to_dict/from_dict.
- Queries:
-
Tests:
tests/execution/test_deliverables.py: 12 tests.tests/execution/test_ledger.py: 25 tests.- Full suite: 102 passed.
ruffandmypyare clean.
-
Checking the tests bite. I broke each of these in the code on purpose, and in each case at least one test failed:
- cross-Phase dependencies accepting submitted
- no cascade on reopen
- the revision cap off by one
- pre-approved mode not granting approval
A fifth mutation,
releasenot bumping the attempt, survived. It showed the extra increments were dead code, so I removed them. Attempts now change only instart, and a test covers a late submission from a superseded attempt.
Added to or tightened from the spec text
- Attempts.
startreturns an attempt number per Deliverable, andsubmit/releasemust present the current one. This protects against late results from Specialists running in parallel after their work was released or reset by a Revision. The spec had no such token. - More event types. Beyond the spec's list:
started,released,reset(a dependent that went back to pending) andapproval_withdrawn. - Return values.
reopenreturnsReopenResult(reset, withdrawn_approvals, escalation).resetlists only Deliverables whose status actually changed. Invalid requests raiseLedgerRefusaland change nothing, and at the capreopenrecordsescalatedand changes nothing.override_cap=Truecovers the user deciding to revise past the cap. - Changes on approval.
request_changes(phase, feedback)returns the Phase toopenand records the feedback, but doesn't reopen Deliverables itself. The Orchestrator reopens the specific ones withreopen. - Dependent feedback. A dependent that is reset gets a feedback note (
Dependency X was revised: <reason>), so its next Specialist run knows why. - Single source for progress. The Ledger owns each Deliverable's status, attempts, revisions and feedback. The Execution state (increment 4) holds only content and attachments, not the
feedback[]/revisionsfields the spec's state sketch showed. - Contracts, matched against headings after stripping numbering, case and trailing punctuation, by exact match or a space-separated suffix. That way "4.1. Google Cloud products and features mapping" satisfies "Products and features mapping", but "Non-functional requirements" doesn't satisfy "Functional requirements".
requirements: functional and non-functional requirements, current state, dependencies.product_selection: products and features mapping.design_recommendations: the 6 pillar headings.deployment_guidance: prerequisites and step-by-step instructions, plus attachmentsterraform/main.tfandterraform/variables.tf.validation_results: deployment dry-run, architecture and policy analysis.solution_guide: the template's 8 top-level sections.architecture_diagram: attachmentarchitecture.mmd.technical_decompositionandarchitecture_description: non-empty content only.
2026-09-15 · Increment 4: Execution graph
Done (test-first)
app/agents/runtime.py: theAgentRuntimeprotocol,RoleSpec,AgentTask,ToolSpec/ToolResult, and the outcomesSubmitted | NeedsInput | BudgetExhausted | Stopped | TurnEnded.app/agents/fake_runtime.py:FakeAgentRuntime, which replays scripted tool calls per role through the realToolSpecs, enforcesRoleSpec.budget, and records every run (task, tools offered, tool messages, outcome).app/execution/state.py:ExecutionState(JSON-compatible).outcomesuses an append reducer, so parallel Specialist tasks never write the Ledger.app/execution/tools.py:OrchestratorToolbox:ready_work,delegate,ask_user,request_approval,reopen,read_deliverable,finish.specialist_tools:submit_deliverable(checks the contract and refuses unsafe attachment paths) andneeds_input, plus the role's extra tools.gate_cloud_read: allow / refuse / ask permission.
app/execution/graph.py:orchestrator_turn→Send× Nspecialist_task→orchestrator_turn;await_user(interrupt);finalize;end_with_warnings.app/execution/engine.py:ExecutionEngine.start(request, session_id, pre_approved, allow_cloud_read)/resume(execution_id, reply)/interaction(execution_id)→Interaction(status, prompt, state). This is the one interface for A2A,/generateand the eval harness. The tests use only it.app/execution/recorder.py: theExecutionRecorderprotocol withInMemoryRecorderandDatabaseRecorder(the existing tables viaasyncio.to_thread).app/execution/checkpointing.py:open_checkpointer(settings)formemory | sqlite | postgres.- Settings
ORCHESTRATOR_BUDGET,SPECIALIST_BUDGET,MAX_REVISIONS,MAX_ORCHESTRATOR_TURNS,CHECKPOINTER(defaultsqlite) andCHECKPOINT_DB_PATH, plus.env.example.requirements.txt:langgraph>=1.2,langgraph-checkpoint-sqlite,langgraph-checkpoint-postgres,psycopg[binary]. - Tests:
tests/execution/test_engine.py: 17 tests covering the pre-approved happy path with written files, parallel start, dependency context, refusal messages, incomplete submissions, Approval pause/resume, free text never approving, changes → Revision → re-approval, one voice, Revision after a validation failure, pre-approved escalation, Orchestrator and Specialist budgets, cloud-read ask/grant/deny, clean Executions, and resuming from a durable SQLite checkpoint.tests/execution/test_recording.py: 2 tests coveringDatabaseRecorderagainst a temp SQLite database, and the checkpointer factory.- Full suite: 121 passed.
ruffandmypyare clean onapp/agentsandapp/execution.
- Checking the tests bite. I broke each of these in the code on purpose, and each break failed at least one test:
- no cloud-read gate
- permission never asked
- Specialist questions dropped
- submission contract unchecked
Added to or changed from the spec text
- Engine and file names. There's a new
app/execution/engine.pyfacade, and tests live intest_engine.py/test_recording.py, nottest_graph.py. The graph is an internal detail behind the engine. Stoppedoutcome. An agent that ends without a turn-ending tool isStopped. Tools end a run by returningToolResult(outcome=...), so ending can depend on the result: pre-approvedrequest_approvaldoesn't end the turn, and refusals never do. Engine-owned tools usetier="execution".- Prompt and reply shapes.
Interaction.promptis{"questions": [{"from": <kind|orchestrator>, "question"}]},{"approval": {"phase", "summary", "deliverables"}}or{"permission": {"tier", "requested_by"}}.resumetakes{"reply": str},{"approval": {"phase", "decision": approve|changes, "feedback"}}or{"permission": {"cloud_read": bool}}.- An approval reply that isn't structured goes into the conversation and never grants.
- Single writer. Specialist outcomes are applied by the next
orchestrator_turnin graph order, usingoutcomes_applied. A superseded result (stale attempt) is discarded and noted to the Orchestrator. - Specialist problems as questions. A Specialist that runs out of budget or stops is released and relayed as a question from its Deliverable, keeping one voice. A permission
NeedsInputbecomes a single permission prompt, not a question. - Pre-approved mode.
allow_cloud_read=NonebecomesFalse. An escalation, or an Orchestrator that can't continue, ends ascompleted_with_warningswith no files written. - Loop guard.
MAX_ORCHESTRATOR_TURNS(default 100) bounds loops, and the LangGraphrecursion_limitis raised to 10 000. - Conversation. It records the Orchestrator's rendered prompts as well as user replies.
- Session name.
DatabaseRecordersession rows useagent_name="solution-architect". - Deferred to increment 5: the design advisor's pillar fan-out tool, which needs the
pillar-advisorrole file. - Not tested:
AsyncPostgresSaveragainst a real PostgreSQL (none runs locally). The import and wiring are in place.
2026-09-16 · Increment 5: ADK runtime and role files
Done (test-first)
-
Role files
agents/*.md(11):orchestrator, the 9 Specialists andpillar-advisor, plusagents/partials/specialist-protocol.md, which every Specialist includes. Each Specialist file names its Deliverable, the Skill task it covers, its constraints and its required headings. The shared protocol covers context fields, questions throughneeds_input, confirming a draft with the user before submitting (or recording assumptions when pre-approved), and resubmitting until the contract passes. -
app/agents/roles.py:load_roles(agents_dir, bundle, orchestrator_budget, specialist_budget, include_preview_mcp). It validates the file name against the role, the deliverable againstDEPENDENCY_GRAPH, the tool tiers, the budget, the includes, a non-empty body, unknown fields, and missing or unexpected roles. -
app/agents/adk_runtime.py:AdkAgentRuntime(skill_dirs, model_factory), the only module importinggoogle.adk.- Per run: a fresh
LlmAgent, an in-memory session and aRunner. - Tools: each
ToolSpecis adapted to aBaseToolwith a JSON-schema declaration, and aSkillToolsetholding only the role's Skills is filtered tolist_skills/load_skill/load_skill_resource. - Instructions: passed as a callable, so role text is never state-templated.
- Stopping: a
before_tool_callbackbudget counts every tool call, including Skill tools. Tool calls within a run are serialised, the first outcome ends the run (later calls in the same batch are refused), and a final text with no outcome becomesStopped(text). - Model:
default_model_factorywrapsrole.modelinLiteLlm.
- Per run: a fresh
-
app/execution/pillars.py: theconsult_pillar_advisors(brief, pillars?)tool, which runs onepillar-advisorper pillar in parallel throughAgentRuntime, each scoped to its pillar Skill with asubmit_advicetool. One failed advisor is reported without losing the others.app/execution/toolkit.py:make_tool_factory(runtime, roles, pillar_skills)gives the tool to the design advisor only; increment 6 extends it. -
app/execution/deliverables.py:inputs_of(kind)gives a Specialist every Deliverable of earlier Phases plus everything it depends on within its own Phase. Before this, the packager's context lackedarchitecture_description, which nothing depends on. -
providers/gcp.yamlandapp/providers.py:role_skillsandpillar_skills, both validated against the Skills index. Models are nowgemini/gemini-pro-latestby default andgemini/gemini-flash-latestfor the pillar advisor and diagrammer. -
requirements.txt:google-adk[extensions]==2.9.0, pinned exactly because Skills are Experimental. Its extras bring LiteLLM (installed 1.101.0). -
Tests:
tests/agents/test_roles.py: 14 tests.tests/agents/test_adk_runtime.py: 12 tests with a scriptedBaseLlm, no network.tests/agents/test_adk_engine.py: 1 test runningExecutionEngineon the ADK runtime with the real role files and bundle. A Specialist question reaches the user and the answer reaches the next run.tests/execution/test_pillars.py: 6 tests.- Plus provider and deliverable tests.
- Full suite: 158 passed.
ruffandmypyare clean. The 5 warnings are ADK and Starlette deprecations.
-
Checking the tests bite. I broke each of these in the code on purpose, and each break failed at least one test:
- every Skill loadable by every role
- no budget
- no early stop after an outcome
- templated instructions
- both turn-ended guards removed at once
Either guard alone suffices under ADK 2.9's sequential callback order; both are kept as defence in depth.
Deviations from the spec text
- Role frontmatter is provider-neutral. Role files carry only
role,deliverable,tool_tiers,budgetandincludes. Skills (role_skills,pillar_skills), MCP servers and models come from the provider bundle, so no role file names a Google Cloud Skill (a test checks this).upstream_taskwas dropped: role bodies describe the Skill task in neutral words. - Model ids. They're LiteLLM
-latestaliases, not the spec'sgemini-2.5-*ids, which are older than LiteLLM's current Gemini 3.x entries. - LiteLLM version.
litellmcomes fromgoogle-adk[extensions](ADK requires ≥ 1.84), notlitellm~=1.74. - Pillar fan-out is a tool on the design advisor, not code inside the runtime.
Deferred
- To increment 6: mapping
RoleSpec.mcp_serversto ADK MCP toolsets (McpToolsetneeds themcpextra, not installed) and supplying local tools bytool_tiers. Both are declared and loaded now, but the runtime doesn't use them yet. - Not yet run against a real model: no API key was used, so role-file quality is unproven until increment 8's evaluation.
- Model API errors propagate from
AgentRuntime.runand fail the graph step, leaving the Execution at its last checkpoint. Decide in increment 7 whether to retry, surface as a question, or fail the A2A task.
2026-09-16 · Increment 6: tools and remote MCP servers
Done (test-first)
app/tools/mcp.py:- Client.
McpClientconnects over streamable HTTP with the MCP Python SDK (mcp1.30), one session per operation. - Catalog.
McpCatalog.load(servers, auth)turns each server's allowlisted tools into framework-neutralToolSpecs named<server>__<tool>, with the server's tier. Loading raisesMcpConfigErrorwhen an allowlisted tool isn'treadOnlyHint: true. An allowlisted tool the server no longer offers is recorded ingaps. An unreachable server becomes a<server>__unavailabletool that returns a discovery gap, and a failed call returns a discovery gap instead of raising. Results are truncated at 20k characters. Servers with an empty allowlist are never contacted. - Auth.
NoAuth,StaticHeadersAuth, andGoogleAdcAuth(ADC bearer token refreshed off the event loop, plusX-Goog-User-Project), chosen byauth_from_config.
- Client.
app/tools/local.py:- Mermaid.
mermaid_checkdoes a structural check (diagram type, brackets per line, quotes respected); it doesn't render. - Terraform.
TerraformRunnerrunsfmt,validate(init -backend=false) andplan(-lock=false -input=false) in a temporary copy, and never buildsapply/destroy. It refuses unsafe file names and variable names, and handles a missing binary and timeouts. - Registry.
LOCAL_TOOLSandlocal_tools(names, runner)build tools by name with their tier.
- Mermaid.
app/execution/toolkit.py:make_tool_factory(runtime, roles, pillar_skills, local_tools=, catalog=)gives each role its named local tools and its bundle MCP servers' tools. Pillar advisors get general-tier MCP tools (Developer Knowledge) inside the consultation. Cloud-read gating still happens per Execution inspecialist_tools, so MCP tools are gated exactly like local ones.- Allowlists reviewed 2026-09-16 against the live
tools/listof all 12 Google Cloud MCP servers.tools/listworks without credentials; no tool was called. The snapshot is atproviders/reviews/gcp-mcp-tools.json, made withscripts/list-mcp-tools.py gcp --json --no-auth.- Allowed: inventory and configuration metadata only, every tool read-only. That's 42 tools across the 12 servers, listed in
providers/gcp.yaml. - Excluded although read-only: Cloud SQL
execute_sql_readonly, Storageread_object/read_text/list_objects, GKEget_k8s_resource/describe_k8s_resource(can return Secrets), Asset Inventoryget_iam_policy_analysis_status, and all write tools. - Tests check every allowed tool against the snapshot and keep the risky tools excluded.
- Allowed: inventory and configuration metadata only, every tool read-only. That's 42 tools across the 12 servers, listed in
docs/operations/mcp-access.mdcovers the APIs to enable, a dedicated service account (roles/mcp.toolUserplus viewer roles), the IAM deny policy onmcp.googleapis.com/tools.callwheremcp.googleapis.com/tool.isReadOnlyis false, ADC and the quota project, and the allowlist review procedure.- Tooling Skills
skills/cloud-discoveryandskills/iac-validation(both pass the Agent Skills validator), added torole_skillsfor the requirements analyst, IaC author and validator. - Settings
MCP_QUOTA_PROJECT,MCP_TIMEOUT_SECONDS,TERRAFORM_BINARY,TERRAFORM_TIMEOUT_SECONDS,TF_PLUGIN_CACHE_DIRinapp/config.pyand.env.example.pytest.ini:asyncio_mode = auto, so async tests and fixtures need no marks. - Tests:
tests/tools/test_mcp.py: 13 tests against a real fake MCP server (FastMCP over streamable HTTP on its own thread).tests/tools/test_local.py: 13 tests, running the realterraformCLI with a provider-free configuration, so no network.- Role, provider and tool-factory additions.
- Full suite: 189 passed.
ruffandmypyare clean on the new modules.
- Checking the tests bite. I broke each of these in the code on purpose, and each break failed at least one test:
- read-only check off
- allowlist ignored
- no truncation
Deviations from the spec text
- Where MCP tools are built. MCP tools are
ToolSpecs built by our own client, not ADKMcpToolsetinadk_runtime.py. This keeps the framework boundary: a Deep Agents runtime, the fake runtime and the cloud-read gate all use the same tools. As a result, thebefore_tool_callbackneeds no MCP-specific permission check. - Tools named per role. Role files name their local tools (
tools: [terraform_validate, terraform_plan]), not tool tiers; a tool's tier comes from its definition. Tiers alone would have given the requirements analystterraform_planand the diagrammer the Terraform tools. - File layout. Local tools live in one module,
app/tools/local.py, notgeneral/andcloud_read/directories. - Test file names. Allowlist tests are in
tests/test_providers.py, nottest_provider_bundle.py.
Open
- Expired credentials. The Application Default Credentials on the dev machine have expired (
invalid_grant), so no authenticated MCP tool call has been made. Rungcloud auth application-default loginbefore increment 8's evaluation. - ADK warns about Gemini via LiteLLM. The warning recommends native
Geminimodels forgemini/*. Decide in increment 8 whetherdefault_model_factoryshould use ADK's native Gemini class for Gemini ids and keep LiteLLM for other providers. - Legacy package import.
app/tools/__init__.pystill imports the legacy modules (LangChain tools, scanner), soapp.tools.localandapp.tools.mcppull them in until increment 9.
Live check (2026-09-16, after gcloud auth application-default login and enabling developerknowledge.googleapis.com in shining-hydra-367716)
- Authentication. ADC authentication works and sends
X-Goog-User-Project: shining-hydra-367716. - Calls succeed. All three allowlisted Developer Knowledge tools succeed through
McpCatalog:search_documents,get_documentsandanswer_query. - API propagation. A call just after the API was enabled returned 403 and became a discovery gap, which confirms the gap path against the real endpoint. The gap message shows only
403 Forbidden, because the SDK doesn't expose the response body; Google's reason (for example "API has not been used… or it is disabled") is lost. - Truncation.
search_documentsandget_documentsboth hit the 20k-character limit, cutting their JSON mid-document. Revisit the limit, or summarise, during increment 8's evaluation. - No customer-resource reads yet. No tool that reads project resources has been called. Among the discovery APIs, only
computeandcloudassetare enabled in this project.
2026-09-16 · Increment 7: interfaces
Done (test-first where it could be)
app/assembly.py:open_assembly(settings, runtime=, recorder=, output_root=)is the composition root. It loads the provider bundle and roles (budgets and preview MCP servers from settings), the ADK runtime by default, theMcpCatalogfor the servers the roles use (skipped whenMCP_ENABLED=false), the Terraform runner and local tools, the tool factory, the configured checkpointer and aDatabaseRecorder, and yieldsAssembly(engine, bundle, roles, mcp_gaps).app/a2a/executor.py:SolutionArchitectExecutormaps A2A tasks onto Executions: task id =execution_id, context id = Session.- New task: enqueues the Task, then
engine.start.pre_approvedandallow_cloud_readcome from message or request metadata. - Follow-up message:
engine.resume. A data part withapprovalorpermissionis the structured reply; anything else is{"reply": text}, so free text never approves. - Input required: a text rendering of the prompt plus a
{"prompt": ...}data part. - Completed: artifacts for the solution guide and every attachment (Terraform files,
architecture.mmd), then a completion message.completed_with_warningslists the warnings. - Engine exception: the task fails with the error text.
- Cancel: cancels the A2A task only; the checkpoint remains.
- New task: enqueues the Task, then
app/a2a/card.py: the agent card is built from settings andPHASE_ORDER/deliverables_in, with a JSON-RPC interface at/a2a.app/a2a/server.py:create_app(settings, assembly_factory)is a Starlette app. The lifespan opens the assembly and mountsa2a-sdkJSON-RPC routes at/a2awith v0.3 compatibility enabled, and serves the card at/.well-known/agent-card.jsonand the legacy/.well-known/agent.json./healthreportsmcp_discovery_gaps.POST /generateruns pre-approved (allow_cloud_readdefaults to false), returns 400 withoutrequestand 500 with the error on failure. Run it withpython -m app.a2a.server(defaultA2A_HOST/A2A_PORT0.0.0.0:8000).app/execution/presentation.py:describe_prompt, whose approval and permission text shows the exact data reply, andto_generate_response. The legacy keys map as follows:source_discovery_doc: the requirements'Current statesectionsource_mermaid_diagram:Nonearchitecture_doc: the architecture descriptionmermaid_diagram:architecture.mmdterraform_code:terraform/main.tfvalidation_passed: statuscompletedtotal_loop_iterations: revisions + 1active_skills: the Skills of the roles whose Deliverables were submitted- also adds
warnings
- Engine changes.
start(execution_id=)accepts a caller-chosen id and refuses a duplicate.- Model errors (the increment 5 open item): an exception in a Specialist run becomes
Stopped("it failed with …"), relayed to the user as a question from that Deliverable. An exception in the Orchestrator propagates, which fails the A2A task or returns 500 from/generate, leaving the Execution at its last checkpoint.
- Settings
MCP_ENABLED,A2A_HOST,A2A_PORTandA2A_PUBLIC_URL. Requirements:mcp>=1.24,<2(missing from increment 6's commit) anda2a-sdk[http-server]>=1.1,<2.pytest.inifiltersa2a-sdk's protobuflabel()deprecation noise. Dockerfile: the builder installs Terraform 1.14.3, verified against HashiCorp's SHA256SUMS. The image copiesagents/,providers/andskills/and exposes 8000 as well as 8080. TheCMDis still the legacy app until increment 9.- Tests:
tests/a2a/test_server.py: 10 tests over JSON-RPC. A question pauses the task and a text reply resumes it. Approval counts only as data. A pre-approved task completes with guide and code artifacts. An engine failure fails the task. A v0.3message/sendclient is accepted./generatekeeps the legacy keys, and returns 400 without a request. Health shows gaps.tests/test_assembly.py: 2 tests.tests/execution/test_presentation.py: 3 tests.- Two engine tests: caller-chosen id, and Specialist failure relayed.
- Full suite: 206 passed.
ruffandmypyare clean.
- Live smoke test.
python -m app.a2a.serverstarted with the real assembly: ADK runtime, live authenticatedtools/liston all 12 Google Cloud MCP servers with no gaps, and memory checkpointer./healthand the agent card responded. No message was sent, so no model was called.
Deviations from the spec text
- Server stack. The A2A server uses the Starlette routes from
a2a-sdk1.1 (A2A 1.0, with v0.3 compatibility), notaiohttp, matchinggoogle-adk-base-agent, which also serves A2A on Starlette. Platform base agents pina2a-sdk>=0.3; the v0.3 compatibility keeps those clients working. - Where
/generatelives./generatefor the new engine is served by the A2A app, not behind anAGENT_ENGINEswitch in the legacy app. The legacy app on :8080 stays unchanged until increment 9 retires it, so there's noAGENT_ENGINEsetting. - No static card file. There's no
agent_card.json; the card is built at startup from settings and the graph. - Test file names. A2A tests are HTTP-level (
tests/a2a/test_server.py), not executor-leveltest_executor.py.
Open
- Card URL. The card advertises
A2A_PUBLIC_URL, orhttp://A2A_HOST:A2A_PORT. SetA2A_PUBLIC_URLin any deployment, because the fallback shows0.0.0.0. - Task store. The A2A task store is in memory, so tasks are lost on restart while Execution checkpoints survive. Use
a2a-sdk's database task store with Postgres in production. - Docker build unverified. No Docker on the dev machine.
- Disabled APIs aren't gaps at startup.
tools/listworks whether or not a product's API is enabled, so a disabled API shows up only when a tool is called, not in/health.