gcp_solution_architecture_agent/.scratch/solution-architect-agent/spec.md

Spec: Solution architect agent

A portable, hierarchical ReAct agent that replaces the upstream google-cloud-solution-architecture skill. An Orchestrator reasons over a code-declared dependency graph of Deliverables and delegates to Specialists. All agents run vendored Agent Skills through a framework's own skill runtime. Users grant Approvals at Phase boundaries.

• Glossary: CONTEXT.md. Use its terms: Execution, Session, Phase, Approval, Revision, Orchestrator, Specialist, Skill, Cloud provider, Deliverable, Dependency, Solution guide.
• Decisions: ADR-0002 (Skills-first, hybrid, hierarchical and ReAct, vendoring, Cloud provider as configuration) and ADR-0003 (LangGraph state with ADK agents).
• Supersedes: .scratch/phase-pipeline/spec.md. Its decision log (D1 to D46) records how this design was reached.
• Branch: feat/solution-architect-agent, pushed to the Gitea remote. A human opens the PR.

Problem

The current agent is a linear chain of five template functions. They ignore the Workflow request (design always picks Cloud Run and Pub/Sub). They run under a fake "ADK" layer that never touches google-adk. The skill prompts they build are thrown away. The agent can't ask the user anything, can't take an Approval, can't run the upstream Skill, and can't target another framework or Cloud provider without a rewrite.

Goals

• Upstream parity. It runs the upstream Skill's workflow: requirements discovery with strict separation from solutioning, grounded product selection with alternatives, diagram, description, WAF-pillar design recommendations, IaC deployment guidance, dry-run validation, and packaging into the upstream template.
• Hierarchical and ReAct. Work follows Dependencies, not a fixed order, and independent work runs in parallel.
• Interactive by default, with code-enforced, recorded Approvals at Phase boundaries. There is also a pre-approved mode for batch and evaluation runs.
• Discovery through the Cloud provider's own remote MCP servers. Current state is discovered through remote MCP servers hosted by the Cloud provider, such as Google Cloud's Resource Manager, Asset Inventory, Compute and GKE servers. Grounding uses the provider's knowledge MCP server. No hand-written cloud REST scanners.
• Portable.
  • Behaviour lives in Skills and framework-neutral role files.
  • Framework types stay behind one agent-runtime interface.
  • Everything specific to a Cloud provider lives in a provider bundle.
• Deterministic tests check every rule the code owns, without a model.

Non-goals (v1)

• Runtime validation. Upstream Task 3.2, verifying after the user has deployed, is out of scope.
• Changing cloud resources. No apply or deploy tool, and ADK's run_skill_script is off.
• A second Cloud provider or a second agent-runtime implementation (Deep Agents). The design leaves room for both; neither is built.
• Architecture review candidates 02 to 06, beyond what this spec needs: tools get tier wrappers, not full ports; DatabaseManager is reused as is; no typed Deliverable records.
• Human approval inside a Phase enforced by code. That stays conversational and driven by the Skill (see Risks).
• Acting as the end user in their cloud (OAuth). Remote MCP calls use the agent's own identity (a service account or Application Default Credentials) in v1.
• Routing MCP calls through the platform's MCP gateway (mcp_gateway_client.py in the base agents). Remote MCP servers are called directly (D46). If a gateway is needed later, it becomes a new mcp_auth.kind: gateway implementation in app/tools/mcp.py, and nothing else changes.

Decisions

These are carried from the decision log. The numbers refer to it.

Area	Decision
Sequencing (D19, D23)	Hybrid. Code owns Dependencies, Phase boundaries and Approvals, and the Skill owns the work inside a Deliverable. The agent is hierarchical and ReAct, not linear.
Engine (D24)	LangGraph + ADK. A LangGraph StateGraph holds Execution state, checkpoints and interrupt(). The Orchestrator and Specialists are ADK LlmAgents with SkillToolset, behind an AgentRuntime interface.
Phases (D25)	Upstream's 4 Phases, as groupings of Deliverables. Discovery of current state is done through tools (remote MCP servers, D45).
Approvals (D26, D32)	Code enforces Approval at Phase boundaries. A Dependency inside a Phase is met on submitted, and one across Phases only on Phase approved. Every user question is an interrupt().
Completion (D27)	Agents finish by calling a submit tool that is checked against the Deliverable's contract.
Transport (D28)	A2A for interactive Executions. /generate is kept for pre-approved ones.
Hierarchy (D29)	An Orchestrator over one Specialist per Deliverable. The design advisor fans out to 6 pillar advisors.
Dependency graph (D30)	Declared in code. The ReAct Orchestrator picks work through tools that refuse work whose Dependencies aren't met.
Budgets (D31)	Per-agent ReAct step budgets. Hitting one hands control to the user.
Voice (D33)	Only the Orchestrator talks to the user. Specialists ask through needs_input.
Specialist memory (D34)	Stateless per delegation. Only the LangGraph checkpoint persists.
Revision (D35)	Reopening a Deliverable returns it and everything that depends on it to draft and withdraws affected Approvals. Revisions are capped, and past the cap the user decides.
Models (D36)	Configurable per role via ADK LiteLLM, with a Gemini default.
Dependency graph content (D38)	See Dependency graph. Runtime validation is out of scope.
Skill slicing (D39)	A shared SkillToolset plus agents/<role>.md. Pillar advisors are scoped to their own WAF Skill.
Vendoring (D21, D40)	Upstream solution-architecture + 6 WAF Skills, unmodified, pinned to 81a31a6, in skills/vendor/google/.
Tool tiers (D41)	General tools (local checks, public documentation) run freely. Reading live cloud resources needs permission once per Execution. Changing cloud resources is never exposed.
Discovery (D45)	The Cloud provider's own remote MCP servers, declared in the provider bundle, provide discovery and grounding. Every tool that reads the user's cloud is cloud_read. Only tools on an explicit read-only allowlist are exposed. GCPEnvironmentScanner is retired.
MCP routing (D46)	Call remote MCP servers directly, with no platform MCP gateway for now.
Outputs (D42)	Deliverables live in state and the database. Files go to deliverables/executions/{id}/ only after Phase 4 Approval.
Tests (D43)	A fake AgentRuntime for deterministic tests. The real-model evaluation runs pre-approved with scripted answers.
Migration (D44)	Build alongside the current code. One cleanup increment follows once the evaluation passes.
Carried over (D9, D14)	Every Execution starts clean, and a Session only groups Executions. The Execution graph and agent runtime are built once at startup.

Design

Module layout

New code only. The current modules stay untouched until increment 9.

agents/                              # framework-neutral role files (one per role)
  orchestrator.md
  requirements-analyst.md  decomposer.md  product-selector.md  diagrammer.md
  describer.md  design-advisor.md  pillar-advisor.md  iac-author.md
  validator.md  packager.md
providers/
  gcp.yaml                           # Cloud provider bundle
skills/
  vendor/google/                     # unmodified upstream Skills + VENDOR.md + LICENSE
    google-cloud-solution-architecture/
    google-cloud-waf-{security,reliability,cost-optimization,operational-excellence,performance-optimization,sustainability}/
  cloud-discovery/SKILL.md           # our tooling Skills (Agent Skills spec compliant)
  iac-validation/SKILL.md
app/
  execution/
    deliverables.py                  # DeliverableKind, PhaseId, DEPENDENCY_GRAPH, contracts
    ledger.py                        # Ledger: Deliverable/Phase states, acceptance, Revision (pure, no I/O)
    graph.py                         # LangGraph StateGraph: orchestrator turn, Specialist fan-out, interrupts
    tools.py                         # Orchestrator and Specialist execution tools bound to the Ledger
    state.py                         # ExecutionState TypedDict
    recorder.py                      # records submissions, Approvals, Revisions, final files
  agents/
    runtime.py                       # AgentRuntime protocol, RoleSpec, AgentTask, Outcome types
    roles.py                         # loads agents/*.md into RoleSpec
    adk_runtime.py                   # ADK LlmAgent + SkillToolset + LiteLlm implementation
    fake_runtime.py                  # scripted implementation for tests (under app/ so eval can reuse it)
  tools/
    general/                         # mermaid check, terraform fmt/validate (local)
    cloud_read/                      # terraform plan (local CLI, reads live state)
    mcp.py                           # McpServerSpec, read-only allowlist, tier + permission metadata
  providers.py                       # loads providers/<name>.yaml
  a2a/
    server.py                        # aiohttp A2A server on :8000, agent_card.json
    executor.py                      # maps A2A tasks <-> Executions and interrupts
scripts/sync-vendored-skills.sh

Deliverables and the dependency graph

app/execution/deliverables.py declares the graph as data. It is the only place Phases, Deliverables and Dependencies are listed.

Phase	Deliverable kind	Specialist role	Depends on	Tools (tier)
1 requirements_discovery	requirements	requirements-analyst	(none)	discovery MCP servers (cloud_read)
1	technical_decomposition	decomposer	requirements	(none)
2 solution_architecture	product_selection	product-selector	technical_decomposition	knowledge MCP (general)
2	architecture_diagram	diagrammer	product_selection	mermaid check (general)
2	architecture_description	describer	architecture_diagram, product_selection	knowledge MCP (general)
2	design_recommendations	design-advisor → 6 × pillar-advisor	product_selection, requirements	knowledge MCP (general)
2	deployment_guidance	iac-author	product_selection, design_recommendations	terraform fmt/validate (general)
3 solution_validation	validation_results	validator	deployment_guidance	terraform validate (general), terraform plan and policy MCP servers: IAM, Network Intelligence (cloud_read)
4 packaging	solution_guide	packager	every Phase 3 Deliverable	(none)

Deliverable contract. Each kind declares its required content in deliverables.py:

• sections: required markdown headings.
• attachments: named code blobs, for example deployment_guidance requires terraform/main.tf and terraform/variables.tf.

submit_deliverable checks the contract. A failing submission returns what's missing to the Specialist, and nothing is recorded.

The Ledger: acceptance and Revision rules

app/execution/ledger.py is a pure module. It holds every rule code owns, and the tests exercise it heavily.

• Deliverable states: pending → in_progress → submitted. Only a Revision goes backwards.
• Phase states: open → awaiting_approval → approved. A Phase becomes awaiting_approval when the Orchestrator calls request_approval with all its Deliverables submitted.
• When a Dependency is met:
  • Same Phase: the Dependency is submitted.
  • Across Phases: the Dependency's Phase is approved.
• Ready work: Deliverables that are pending and have every Dependency met.
• reopen(kind, reason):
  • The Deliverable goes to pending with reason appended to its feedback.
  • Everything that depends on it, directly or not, goes to pending.
  • Every affected Phase that was approved or awaiting_approval goes back to open.
  • The Deliverable's revisions count goes up. At the cap (max_revisions, default 3) reopen refuses and returns an escalation the Orchestrator must put to the user.
• Pre-approved mode: request_approval grants immediately and records approved_by = "pre-approved".
• Records: every transition appends an event (submitted, approval_requested, approved, changes_requested, reopened, escalated) for the recorder.

Execution graph (LangGraph)

app/execution/graph.py compiles once at startup with the configured checkpointer: SqliteSaver locally and in tests, PostgresSaver in production. The Execution id is the LangGraph thread_id.

State (ExecutionState):

• execution_id, session_id, workflow_request, mode: "interactive" | "pre_approved", allow_cloud_read: bool | None
• ledger (serialisable), deliverables: {kind: {content, attachments, feedback[], revisions}}
• conversation[] (user ↔ Orchestrator turns only), pending_questions[], pending_delegations[]
• budget_exhausted: {role, summary} | None

Nodes and edges:

START → orchestrator_turn
orchestrator_turn ──(delegations)──────────> specialist_task × N   (LangGraph Send, parallel)
orchestrator_turn ──(questions/approval)───> await_user            (interrupt)
orchestrator_turn ──(finish)───────────────> finalize → END
specialist_task ───────────────────────────> orchestrator_turn     (outcomes merged into ledger)
await_user ────────────────────────────────> orchestrator_turn     (resume value = user reply / approval decision)

• orchestrator_turn: runs one Orchestrator ReAct turn through AgentRuntime, with Orchestrator tools bound to a working copy of the Ledger. The turn ends when the agent does one of these, and the node routes on it:
  • calls delegate, ask_user, request_approval or finish
  • runs out of budget
• specialist_task: runs one Specialist delegation. Its task holds the role, the Deliverable kind, the Dependency contents, feedback, and the answers to its earlier needs_input questions. The outcome is one of:
  • Submitted(content, attachments), applied to the Ledger
  • NeedsInput(questions), added to pending_questions with the Deliverable back to pending
  • BudgetExhausted(summary)
• await_user: interrupt({"questions": [...]} | {"approval": {"phase": ..., "summary": ...}} | {"permission": "cloud_read"} | {"escalation": ...}). In pre-approved mode it doesn't pause:
  • Approvals: granted automatically.
  • cloud_read permission: taken from the request.
  • Questions: answered with "No user is available. State your assumption, record it in the Deliverable under 'Assumptions', and proceed."
  • Escalations: end the Execution as completed_with_warnings.
• finalize: runs after Phase 4 is approved. It writes files to deliverables/executions/{execution_id}/: the Solution guide, every Deliverable as markdown, and attachments under terraform/. It then records completion.

Orchestrator tools (app/execution/tools.py):

Tool	Effect
ready_work()	Lists ready Deliverables and Phases ready for Approval.
delegate(kinds: list[str])	Refuses any kind that isn't ready and sets it in_progress. Ends the turn and fans out.
ask_user(questions: list[str])	Combines Specialist questions with its own. Ends the turn and interrupts.
request_approval(phase: str)	Refuses unless every Deliverable in the Phase is submitted. Ends the turn and interrupts.
reopen(kind: str, reason: str)	Applies the Revision rule, or returns an escalation.
read_deliverable(kind: str)	Returns the current content, so the Orchestrator can summarise it for Approval.
finish()	Refuses unless Phase 4 is approved.

Specialist tools: submit_deliverable(content, attachments) (checks the contract), needs_input(questions), the SkillToolset, and the role's tier tools. Cloud-read tools check allow_cloud_read. If it's None, the tool returns NeedsInput(["May I read live resources in your Google Cloud project to …?"]) so the Orchestrator asks once. If it's False, it returns a refusal the agent must work around.

Design advisor fan-out. The design-advisor role runs a nested fan-out itself: it sends one task to pillar-advisor for each WAF pillar, in parallel, through AgentRuntime, and combines the results into its submission. Pillar results are not Deliverables and don't appear in the Ledger.

Agent runtime interface

app/agents/runtime.py:

class AgentRuntime(Protocol):
    async def run(self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]) -> Outcome: ...

@dataclass(frozen=True)
class RoleSpec:           # parsed from agents/<role>.md
    name: str
    instructions: str     # markdown body
    skills: tuple[str, ...]           # Skill names from the provider bundle
    tool_tiers: tuple[str, ...]       # subset of ("general", "cloud_read")
    mcp_servers: tuple[str, ...]      # server ids from the provider bundle this role may use
    model: str | None                 # overrides provider default
    budget: int                       # max tool calls per run

Outcome = Submitted | NeedsInput | BudgetExhausted | TurnEnded   # TurnEnded carries the Orchestrator's ending tool call

• ToolSpec: framework-neutral (name, description, JSON schema, async callable). adk_runtime.py builds each agent from these pieces:
  • ToolSpec → ADK FunctionTool.
  • The role's MCP servers → ADK McpToolset over streamable HTTP, with tool_filter set to the server's read-only allowlist.
  • SkillToolset(skills=[load_skill_from_dir(...)]) with only the role's Skills.
  • model wrapped in LiteLlm.
• Per run: a fresh InMemorySessionService (D34).
• Callbacks: a before_tool_callback counts tool calls against budget and enforces the cloud_read permission for every tool in that tier, MCP tools included.
• No framework types cross this interface.

Role file format (agents/<role>.md):

---
role: diagrammer
deliverable: architecture_diagram
upstream_task: "Phase 2, Task 2.2: Generate an architecture diagram"
skills: [google-cloud-solution-architecture]
tool_tiers: [general]
mcp_servers: []
budget: 15
---
You produce the architecture diagram Deliverable. Load the `google-cloud-solution-architecture` Skill and follow its Task 2.2 ...
Submit with `submit_deliverable` once the diagram passes the Mermaid check and the user has confirmed it through `needs_input`.

pillar-advisor.md is parameterised by pillar, and its skills value is resolved per task to the one matching google-cloud-waf-<pillar>.

Cloud provider bundle

providers/gcp.yaml, loaded by app/providers.py:

name: gcp
skills_dirs: [skills/vendor/google, skills]
output_template: skills/vendor/google/google-cloud-solution-architecture/assets/output-template.md
models:
  default: gemini/gemini-2.5-pro
  roles: { pillar-advisor: gemini/gemini-2.5-flash, diagrammer: gemini/gemini-2.5-flash }
tools:
  general: [mermaid_check, terraform_fmt, terraform_validate]
  cloud_read: [terraform_plan]
mcp_auth:
  kind: google_adc                     # service account / ADC bearer token; sends X-Goog-User-Project
mcp_servers:                           # remote MCP servers hosted by the Cloud provider (streamable HTTP)
  developer_knowledge:
    url: https://developerknowledge.googleapis.com/mcp
    tier: general                      # public documentation, no customer data
    allow: [search_documents, get_documents, answer_query]
  resource_manager: { url: https://cloudresourcemanager.googleapis.com/mcp, tier: cloud_read, allow: [<read-only tools>] }
  asset_inventory:  { url: https://cloudasset.googleapis.com/mcp,         tier: cloud_read, allow: [...], preview: true }
  compute:          { url: https://compute.googleapis.com/mcp,            tier: cloud_read, allow: [...] }
  gke:              { url: https://container.googleapis.com/mcp,          tier: cloud_read, allow: [...] }
  cloud_run:        { url: https://run.googleapis.com/mcp,                tier: cloud_read, allow: [...] }
  cloud_sql:        { url: https://sqladmin.googleapis.com/mcp,           tier: cloud_read, allow: [...] }
  storage:          { url: https://storage.googleapis.com/storage/mcp,    tier: cloud_read, allow: [...] }
  pubsub:           { url: https://pubsub.googleapis.com/mcp,             tier: cloud_read, allow: [...] }
  iam:              { url: https://iam.googleapis.com/mcp,                tier: cloud_read, allow: [...] }
  monitoring:       { url: https://monitoring.googleapis.com/mcp,         tier: cloud_read, allow: [...] }
  network_intelligence: { url: https://networkmanagement.googleapis.com/mcp, tier: cloud_read, allow: [...], preview: true }
role_mcp_servers:
  requirements-analyst: [resource_manager, asset_inventory, compute, gke, cloud_run, cloud_sql, storage, pubsub, iam, monitoring]
  product-selector: [developer_knowledge]
  describer: [developer_knowledge]
  design-advisor: [developer_knowledge]
  pillar-advisor: [developer_knowledge]
  validator: [iam, network_intelligence]

Loading the bundle.

• Settings select the bundle with CLOUD_PROVIDER=gcp. Core modules don't name a provider.
• role_mcp_servers overrides the mcp_servers list in role files, so role files stay neutral across providers.
• The allow lists are written once in increment 6: run MCP tools/list on each server and include only tools whose readOnlyHint annotation is true and whose names are reviewed as read-only.
• Servers marked preview: true can be switched off with MCP_ENABLE_PREVIEW=false.

Other Cloud providers reuse the same shape:

• AWS: the managed, generally available AWS MCP Server.
• Azure: Azure MCP Server 2.0 runs as a self-hosted remote server, so its url points at the team's own deployment.

Tools

• General, local (app/tools/general/):
  • mermaid_check: moved from validation_tools.validate_mermaid_diagram.
  • terraform_fmt and terraform_validate: run the terraform CLI in a temp dir with -backend=false. There are no credentials, and the tool fails clearly if terraform is missing.
• Cloud read, local (app/tools/cloud_read/): terraform_plan needs ADC and -lock=false, and never runs apply.
• Remote MCP servers (app/tools/mcp.py): the Cloud provider's own hosted servers, declared in the provider bundle and used for both discovery and grounding. They replace GCPEnvironmentScanner (hand-written REST calls) and the hand-rolled JSON-RPC DeveloperKnowledgeMCPClient, including its offline fallback knowledge base and the data/documents bug from review candidate 02. Both are deleted in increment 9.
  • McpServerSpec(id, url, tier, allow, preview) is framework-neutral. The ADK runtime maps each one to McpToolset(StreamableHTTPConnectionParams(url, headers=auth), tool_filter=allow).
  • Read-only in depth. Tools outside allow are never offered to any agent. At startup, a server whose tools/list reports an allowed tool with readOnlyHint not true fails to load, loudly. Operators should also run the agent's identity with only viewer roles, plus Google Cloud's control that blocks read-write MCP tool use and an IAM deny policy (docs/operations/mcp-access.md, written in increment 6).
  • Auth: google_adc gets a bearer token from Application Default Credentials (google.auth.default, refreshed per call) and sends X-Goog-User-Project. Follows google-adk-base-agent/app/google_mcp_client.py, without the gcloud subprocess fallback.
  • Failure: an unreachable, disabled or unauthorised server returns a structured tool error such as "compute MCP unavailable: API not enabled in project X". The agent records the gap under 'Assumptions / discovery gaps' and doesn't fail the Execution.
• Our tooling Skills (skills/cloud-discovery, skills/iac-validation): short Agent Skills-compliant guides for agents.
  • cloud-discovery: which discovery servers answer which current-state questions (inventory first through Asset Inventory and Resource Manager, then per-product detail), and how to summarise current state.
  • iac-validation: when and how to run the Terraform checks.

Interaction

A2A (app/a2a/), using a2a-sdk>=0.3 on aiohttp :8000, with agent_card.json and /.well-known/agent.json:

• Task ↔ Execution: an A2A task is an Execution (task.id = execution_id), and context_id is the Session.
• Status updates: working while nodes run.
• input-required at every interrupt(). The interrupt payload goes out as a DataPart, with a text rendering for plain clients.
• User reply: a text part becomes an answer to pending questions. A DataPart with {"approval": {"phase", "decision": "approve" | "changes", "feedback"}} or {"permission": {"cloud_read": bool}} resumes an Approval or permission interrupt. Approvals are never taken from free text, which keeps them explicit and recorded.
• Final state: completed after finalize, and failed on unhandled errors.

POST /generate (existing Starlette app, pre-approved mode):

• Request: {request, session_id?, allow_cloud_read?: false}.
• Response: today's keys (execution_id, session_id, status, validation_passed, total_loop_iterations, current_phase, artifacts{8 keys}, active_skills), mapped from the final state.
  • artifacts maps today's 8 keys to the closest Deliverables. source_discovery_doc and source_mermaid_diagram become the current-state section of requirements.
  • total_loop_iterations = total Revisions + 1.
  • active_skills = the Skills loaded during the Execution.

The route switches to the new graph in increment 7, behind AGENT_ENGINE=hierarchical (default legacy until increment 9).

Recording

app/execution/recorder.py reuses DatabaseManager tables until candidate 03:

• workflow_executions: one row per Execution, updated at each interrupt and at completion.
• adk_artifacts: one row per Deliverable submission. Rows for earlier Revisions stay as history.
• orchestrator_review_logs: one row per Ledger event. review_status holds the event type, reviewer_agent holds the role or user, and feedback holds the reason or feedback.
• adk_sessions: the latest Execution summary for the Session (D9).

Settings

Add to app/config.py and .env.example:

• CLOUD_PROVIDER=gcp
• AGENT_ENGINE=legacy|hierarchical
• ORCHESTRATOR_BUDGET=40, SPECIALIST_BUDGET=15, MAX_REVISIONS=3
• CHECKPOINTER=sqlite|postgres, CHECKPOINT_DB_PATH=./checkpoints.db
• A2A_HOST=0.0.0.0, A2A_PORT=8000
• MCP_ENABLE_PREVIEW=true, MCP_QUOTA_PROJECT (defaults to the ADC project)

Model API keys follow LiteLLM conventions (GEMINI_API_KEY / Vertex ADC).

Dependencies

Add to requirements.txt:

• google-adk>=1.25 (Skills are Experimental: pin the exact version in the lock file you use)
• litellm~=1.74
• a2a-sdk>=0.3
• aiohttp>=3.10
• langgraph-checkpoint-sqlite
• langgraph-checkpoint-postgres

Keep langgraph and upgrade it to a version with interrupt() and Send. The terraform CLI is added to the Dockerfile image.

Testing

Deterministic, with FakeAgentRuntime, which replays scripted Outcomes and tool calls per role and task. There are no model calls and no network.

• tests/execution/test_ledger.py (pure):
  • Ready work respects both thresholds: same-Phase submitted and cross-Phase approved.
  • Each refusal case: delegate on unready work, request_approval with unsubmitted Deliverables, finish before Phase 4 is approved.
  • reopen cascades to everything that depends on the Deliverable and withdraws approved Phases. The cap produces an escalation.
  • Pre-approved request_approval grants and records pre-approved.
• tests/execution/test_graph.py, with the fake runtime and MemorySaver:
  • Happy path, pre-approved: every Deliverable submitted, all 4 Phases approved, files written under deliverables/executions/{id}/ (tmp dir).
  • Parallel start: after product_selection is submitted, one Orchestrator turn can delegate architecture_diagram and design_recommendations together, and both run.
  • One voice: questions from two parallel Specialists produce a single interrupt holding both.
  • Interactive pause and resume: an Approval interrupt stops the graph. Resuming with approve continues, and resuming with changes plus feedback reopens.
  • Validation failure: reopening deployment_guidance withdraws Phase 2's Approval, re-delegates the IaC author with the findings, and asks for Phase 2 Approval again.
  • Budgets: an exhausted budget hands control to the user with a summary.
  • Cloud read: a tool with allow_cloud_read=None produces one permission question. False makes the tool refuse.
  • Clean Executions: two Executions in one Session share no state.
• tests/agents/test_roles.py: every agents/*.md parses, and every role maps to a Deliverable kind (or orchestrator/pillar-advisor), with Skills that exist in the provider bundle.
• tests/test_vendored_skills.py:
  • Vendored Skills match VENDOR.md's commit (by checksum).
  • Every Skill under skills/ passes Agent Skills frontmatter rules: kebab-case name matching its directory, description ≤ 1024 chars.
• tests/agents/test_adk_runtime.py: builds an ADK agent for one role with a stub model. It confirms that the SkillToolset holds only that role's Skills, that the tool list matches the role's tiers and MCP servers, and that each McpToolset has the server's tool_filter. It does not call a model.
• tests/tools/test_mcp.py: uses a local fake MCP server over streamable HTTP, started in the test.
  • Tools outside allow are never exposed.
  • An allowed tool whose readOnlyHint isn't true makes the server fail to load.
  • A cloud_read MCP tool call without permission is blocked by before_tool_callback.
  • An unreachable server returns a structured discovery gap and doesn't raise.
  • google_adc headers include X-Goog-User-Project.
• tests/test_provider_bundle.py: every mcp_servers entry has tier and a non-empty allow. developer_knowledge is the only general server. Every role_mcp_servers id exists.
• tests/a2a/test_executor.py: A2A task → input-required on interrupt → DataPart approval resumes, using the fake runtime. A free-text "approve" does not grant.
• tests/test_agent_api.py: the /generate response keys are unchanged under both AGENT_ENGINE values.

Evaluation (eval/):

• Harness: runs benchmark_cases.json in pre-approved mode with the real AdkRuntime. Each case adds scripted_answers: {question_pattern: answer}, and the pre-approved await_user uses these answers before falling back to assumptions.
• Scoring: existing rubric metrics, plus checks that the Solution guide has every output-template.md heading and that the product selection names at least one alternative with pros and cons.
• Not in the default test run. It lives under eval/ and is marked.

Build increments

These are ordered, and each leaves the suite green. They are called increments so they don't clash with the glossary's Phase.

1. Environment and baseline.
   • Create a venv and install dependencies, plus terraform.
   • Run the current suite and record the baseline under ## Comments.
   • Create the branch.
2. Vendored Skills and provider bundle.
   • Add skills/vendor/google/ at 81a31a6 with VENDOR.md and the licence, plus the sync script.
   • Add providers/gcp.yaml and app/providers.py.
   • Add the vendoring and frontmatter tests.
3. Deliverables and Ledger, test-first: deliverables.py, ledger.py, test_ledger.py.
4. Execution graph, test-first:
   • runtime.py (the interface and outcome types), fake_runtime.py.
   • state.py, tools.py, graph.py, recorder.py.
   • Checkpointer settings.
   • test_graph.py.
5. ADK runtime and role files: adk_runtime.py, roles.py, all agents/*.md, test_roles.py, test_adk_runtime.py.
6. Tools and remote MCP servers:
   • General and cloud-read local tools (terraform in a temp dir).
   • app/tools/mcp.py, google_adc auth and the McpToolset mapping.
   • Allowlists: run tools/list against each Google Cloud MCP server in a sandbox project, review the read-only tools, and write the allow lists into providers/gcp.yaml, recording the reviewed tool lists and date under ## Comments.
   • docs/operations/mcp-access.md: the APIs to enable, viewer roles, and blocking read-write MCP use with IAM deny.
   • The skills/cloud-discovery and skills/iac-validation Skills.
   • The tests in test_mcp.py and test_provider_bundle.py.
7. Interfaces:
   • The A2A server and executor, agent_card.json, test_executor.py.
   • /generate behind AGENT_ENGINE, and the Dockerfile exposing :8000 and :8080.
8. Evaluation: scripted answers, template-heading scoring, and a real-model eval run with results recorded under ## Comments. Proceed to 9 only when the pass rate meets the current harness threshold (≥ 80% per case).
9. Cleanup (carried over from the superseded spec):
   • Delete: app/tools/gcp_scanner.py, app/tools/mcp_developer_knowledge.py (replaced by remote MCP servers), app/adk/, app/nodes/, app/states/, app/workflows/gcp_architecture_graph.py, app/skills/ (the loader and the 5 non-compliant Skills), tests/test_graph_workflow.py, the ADK and orchestrator tests in tests/test_adk_architecture.py (keeping the DB test as tests/test_database.py), and test_adk_mcp_function_tools.
   • Fix: remove the unreachable duplicate return in routes.py. Rename ADK_MAX_LOOP_ITERATIONS → removed, and delete ADK_ENABLE_ARTIFACT_STORE.
   • Update:
     • Make AGENT_ENGINE=hierarchical the only engine.
     • Derive app/card.py / agent_card.json capabilities from DEPENDENCY_GRAPH and the role files.
     • Rewrite workflow.yaml phases to upstream's 4 with their Deliverables.
     • Rewrite the README around the Orchestrator, Specialists, Skills and A2A.
   • Stop the writes to shared paths (deliverables/{as-is,target,validation,guides}/, docs/*.md, terraform/main.tf).
   • Push the branch.

Acceptance

• Upstream parity: a pre-approved evaluation run produces a Solution guide containing every heading of the upstream output-template.md, and all benchmark cases pass the threshold.
• Single source for the workflow: Phases, Deliverables and Dependencies are listed only in app/execution/deliverables.py, and card and manifest data is derived from it.
• No framework types outside app/agents/adk_runtime.py: rg -n "google\.adk" app | grep -v adk_runtime.py is empty. After increment 9, rg -n "app\.adk|OrchestratorLoopAgent|compat" app eval tests is empty.
• Vendored Skills unchanged: they match the 81a31a6 checksums, and every Skill under skills/ passes the frontmatter rules.
• Approvals only by structured decision: no path grants an Approval from free text (covered by tests).
• Nothing changes cloud resources: no tool can run terraform apply, a deploy or a script (run_skill_script absent from the tool lists). Cloud-read tools, MCP included, are unreachable without permission.
• Discovery uses the provider's remote MCP servers: current state comes from the Cloud provider's hosted MCP servers. No module under app/ calls a cloud provider's REST API directly (rg -n "googleapis.com/(compute|storage|run|sqladmin|pubsub)/v" app is empty). Every exposed MCP tool is on a reviewed read-only allowlist.
• Deterministic suite: every test above passes with no network access.
• /generate contract: response keys are unchanged.

Risks

• Specialists submitting without asking (D32). A Specialist might submit without getting the user's conversational OK. Mitigation: role files require confirmation through needs_input before submit_deliverable, and the evaluation checks that submissions follow a question. If it's still observed, fall back to Approval per Deliverable (the Q31 alternative).
• ADK Skills are Experimental. The SkillToolset API may change. It's confined to adk_runtime.py, and Deep Agents is the planned second implementation.
• Parallel Specialists and rate limits. Fan-out multiplies concurrent model calls. Cap concurrency per Execution (default 4) in graph.py.
• Upstream Skill drift. Upstream edits may assume a different workflow. Re-sync only through the script, and review the diff against agents/*.md and deliverables.py.
• Two frameworks in one process. LangGraph and ADK both manage async state. Keep ADK sessions in memory per run, and never put ADK objects in graph state.
• Remote MCP server availability and drift. Several servers are Preview (Asset Inventory, Network Intelligence, Billing), each needs its product API enabled in the user's project, and tool sets can change. Mitigations:
  • allowlists are reviewed, not discovered at run time
  • a changed readOnlyHint fails the server loudly
  • preview servers can be switched off
  • an unavailable server becomes a recorded discovery gap, not a failure
• Uneven hosting across Cloud providers. AWS hosts a managed MCP server. Azure's general MCP server is self-hosted. The bundle's url covers both, but "hosted by the Cloud provider" isn't uniform.

Comments