feat: add local tools and read-only remote MCP servers
Increment 6 of .scratch/solution-architect-agent/spec.md, test-first. - app/tools/mcp.py: MCP client and catalog exposing only allowlisted, readOnlyHint tools as framework-neutral ToolSpecs; unreachable servers and failed calls become discovery gaps; ADC auth with quota project. - app/tools/local.py: Mermaid structure check and Terraform fmt, validate and plan in a scratch copy; apply is never constructed. - Tool factory gives roles their named local tools and bundle MCP servers; pillar advisors get general-tier documentation tools. - providers/gcp.yaml: 42 allowlisted tools reviewed against live tools/list of 12 Google Cloud MCP servers (snapshot in providers/reviews/), excluding data-reading and secret-capable tools. - scripts/list-mcp-tools.py, docs/operations/mcp-access.md (IAM deny policy, service account roles), cloud-discovery and iac-validation Skills, MCP and Terraform settings, pytest asyncio auto mode. - 31 new tests; full suite 189 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -652,3 +652,50 @@ Slowest tests: eval benchmark 7.3 s, then 3–4.5 s each for the orchestrator, e
|
||||
- **To increment 6:** mapping `RoleSpec.mcp_servers` to ADK MCP toolsets (`McpToolset` needs the `mcp` extra, not installed) and supplying local tools by `tool_tiers`. Both are declared and loaded now, but the runtime doesn't use them yet.
|
||||
- **Not yet run against a real model:** no API key was used, so role-file quality is unproven until increment 8's evaluation.
|
||||
- **Model API errors propagate** from `AgentRuntime.run` and fail the graph step, leaving the Execution at its last checkpoint. Decide in increment 7 whether to retry, surface as a question, or fail the A2A task.
|
||||
|
||||
### 2026-09-16 · Increment 6: tools and remote MCP servers
|
||||
|
||||
**Done (test-first)**
|
||||
- `app/tools/mcp.py`:
|
||||
- **Client.** `McpClient` connects over streamable HTTP with the MCP Python SDK (`mcp` 1.30), one session per operation.
|
||||
- **Catalog.** `McpCatalog.load(servers, auth)` turns each server's allowlisted tools into framework-neutral `ToolSpec`s named `<server>__<tool>`, with the server's tier. Loading raises `McpConfigError` when an allowlisted tool isn't `readOnlyHint: true`. An allowlisted tool the server no longer offers is recorded in `gaps`. An unreachable server becomes a `<server>__unavailable` tool that returns a discovery gap, and a failed call returns a discovery gap instead of raising. Results are truncated at 20k characters. Servers with an empty allowlist are never contacted.
|
||||
- **Auth.** `NoAuth`, `StaticHeadersAuth`, and `GoogleAdcAuth` (ADC bearer token refreshed off the event loop, plus `X-Goog-User-Project`), chosen by `auth_from_config`.
|
||||
- `app/tools/local.py`:
|
||||
- **Mermaid.** `mermaid_check` does a structural check (diagram type, brackets per line, quotes respected); it doesn't render.
|
||||
- **Terraform.** `TerraformRunner` runs `fmt`, `validate` (`init -backend=false`) and `plan` (`-lock=false -input=false`) in a temporary copy, and never builds `apply`/`destroy`. It refuses unsafe file names and variable names, and handles a missing binary and timeouts.
|
||||
- **Registry.** `LOCAL_TOOLS` and `local_tools(names, runner)` build tools by name with their tier.
|
||||
- `app/execution/toolkit.py`: `make_tool_factory(runtime, roles, pillar_skills, local_tools=, catalog=)` gives each role its named local tools and its bundle MCP servers' tools. Pillar advisors get general-tier MCP tools (Developer Knowledge) inside the consultation. Cloud-read gating still happens per Execution in `specialist_tools`, so MCP tools are gated exactly like local ones.
|
||||
- **Allowlists reviewed 2026-09-16** against the live `tools/list` of all 12 Google Cloud MCP servers. `tools/list` works without credentials; no tool was called. The snapshot is at `providers/reviews/gcp-mcp-tools.json`, made with `scripts/list-mcp-tools.py gcp --json --no-auth`.
|
||||
- **Allowed:** inventory and configuration metadata only, every tool read-only. That's 42 tools across the 12 servers, listed in `providers/gcp.yaml`.
|
||||
- **Excluded although read-only:** Cloud SQL `execute_sql_readonly`, Storage `read_object`/`read_text`/`list_objects`, GKE `get_k8s_resource`/`describe_k8s_resource` (can return Secrets), Asset Inventory `get_iam_policy_analysis_status`, and all write tools.
|
||||
- **Tests** check every allowed tool against the snapshot and keep the risky tools excluded.
|
||||
- `docs/operations/mcp-access.md` covers the APIs to enable, a dedicated service account (`roles/mcp.toolUser` plus viewer roles), the IAM deny policy on `mcp.googleapis.com/tools.call` where `mcp.googleapis.com/tool.isReadOnly` is false, ADC and the quota project, and the allowlist review procedure.
|
||||
- Tooling Skills `skills/cloud-discovery` and `skills/iac-validation` (both pass the Agent Skills validator), added to `role_skills` for the requirements analyst, IaC author and validator.
|
||||
- Settings `MCP_QUOTA_PROJECT`, `MCP_TIMEOUT_SECONDS`, `TERRAFORM_BINARY`, `TERRAFORM_TIMEOUT_SECONDS`, `TF_PLUGIN_CACHE_DIR` in `app/config.py` and `.env.example`. `pytest.ini`: `asyncio_mode = auto`, so async tests and fixtures need no marks.
|
||||
- Tests:
|
||||
- `tests/tools/test_mcp.py`: 13 tests against a real fake MCP server (FastMCP over streamable HTTP on its own thread).
|
||||
- `tests/tools/test_local.py`: 13 tests, running the real `terraform` CLI with a provider-free configuration, so no network.
|
||||
- Role, provider and tool-factory additions.
|
||||
- Full suite: **189 passed**. `ruff` and `mypy` are clean on the new modules.
|
||||
- **Checking the tests bite.** I broke each of these in the code on purpose, and each break failed at least one test:
|
||||
- read-only check off
|
||||
- allowlist ignored
|
||||
- no truncation
|
||||
|
||||
**Deviations from the spec text**
|
||||
- **Where MCP tools are built.** MCP tools are `ToolSpec`s built by our own client, not ADK `McpToolset` in `adk_runtime.py`. This keeps the framework boundary: a Deep Agents runtime, the fake runtime and the cloud-read gate all use the same tools. As a result, the `before_tool_callback` needs no MCP-specific permission check.
|
||||
- **Tools named per role.** Role files name their local tools (`tools: [terraform_validate, terraform_plan]`), not tool tiers; a tool's tier comes from its definition. Tiers alone would have given the requirements analyst `terraform_plan` and the diagrammer the Terraform tools.
|
||||
- **File layout.** Local tools live in one module, `app/tools/local.py`, not `general/` and `cloud_read/` directories.
|
||||
- **Test file names.** Allowlist tests are in `tests/test_providers.py`, not `test_provider_bundle.py`.
|
||||
|
||||
**Open**
|
||||
- **Expired credentials.** The Application Default Credentials on the dev machine have expired (`invalid_grant`), so no authenticated MCP tool call has been made. Run `gcloud auth application-default login` before increment 8's evaluation.
|
||||
- **ADK warns about Gemini via LiteLLM.** The warning recommends native `Gemini` models for `gemini/*`. Decide in increment 8 whether `default_model_factory` should use ADK's native Gemini class for Gemini ids and keep LiteLLM for other providers.
|
||||
- **Legacy package import.** `app/tools/__init__.py` still imports the legacy modules (LangChain tools, scanner), so `app.tools.local` and `app.tools.mcp` pull them in until increment 9.
|
||||
|
||||
**Live check (2026-09-16, after `gcloud auth application-default login` and enabling `developerknowledge.googleapis.com` in `shining-hydra-367716`)**
|
||||
- **Authentication.** ADC authentication works and sends `X-Goog-User-Project: shining-hydra-367716`.
|
||||
- **Calls succeed.** All three allowlisted Developer Knowledge tools succeed through `McpCatalog`: `search_documents`, `get_documents` and `answer_query`.
|
||||
- **API propagation.** A call just after the API was enabled returned 403 and became a discovery gap, which confirms the gap path against the real endpoint. The gap message shows only `403 Forbidden`, because the SDK doesn't expose the response body; Google's reason (for example "API has not been used… or it is disabled") is lost.
|
||||
- **Truncation.** `search_documents` and `get_documents` both hit the 20k-character limit, cutting their JSON mid-document. Revisit the limit, or summarise, during increment 8's evaluation.
|
||||
- **No customer-resource reads yet.** No tool that reads project resources has been called. Among the discovery APIs, only `compute` and `cloudasset` are enabled in this project.
|
||||
|
||||
Reference in New Issue
Block a user