feat: run role files on the ADK runtime with scoped Skills

Increment 5 of .scratch/solution-architect-agent/spec.md, test-first.

- agents/*.md: provider-neutral role files for the Orchestrator, nine
  Specialists and the pillar advisor, with a shared Specialist protocol.
- app/agents/roles.py: load and validate role files; Skills, MCP
  servers and models come from the provider bundle.
- app/agents/adk_runtime.py: AgentRuntime on google-adk 2.9 with a
  per-run LlmAgent, role-scoped SkillToolset (no run_skill_script),
  step budgets, early stop on turn-ending tools, LiteLLM models.
- app/execution/pillars.py + toolkit.py: design advisor fans out to one
  advisor per WAF pillar in parallel.
- Specialists receive every earlier-Phase Deliverable as input.
- Bundle gains role_skills, pillar_skills and -latest Gemini aliases;
  pin google-adk[extensions]==2.9.0.
- 37 new tests; full suite 158 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-16 09:51:02 +01:00
parent d9214c671a
commit f040493583
31 changed files with 1431 additions and 17 deletions

View File

@@ -610,3 +610,45 @@ Slowest tests: eval benchmark 7.3 s, then 3–4.5 s each for the orchestrator, e
- **Session name.** `DatabaseRecorder` session rows use `agent_name="solution-architect"`.
- **Deferred to increment 5:** the design advisor's pillar fan-out tool, which needs the `pillar-advisor` role file.
- **Not tested:** `AsyncPostgresSaver` against a real PostgreSQL (none runs locally). The import and wiring are in place.
### 2026-09-16 · Increment 5: ADK runtime and role files
**Done (test-first)**
- **Role files** `agents/*.md` (11): `orchestrator`, the 9 Specialists and `pillar-advisor`, plus `agents/partials/specialist-protocol.md`, which every Specialist includes. Each Specialist file names its Deliverable, the Skill task it covers, its constraints and its required headings. The shared protocol covers context fields, questions through `needs_input`, confirming a draft with the user before submitting (or recording assumptions when pre-approved), and resubmitting until the contract passes.
- `app/agents/roles.py`: `load_roles(agents_dir, bundle, orchestrator_budget, specialist_budget, include_preview_mcp)`. It validates the file name against the role, the deliverable against `DEPENDENCY_GRAPH`, the tool tiers, the budget, the includes, a non-empty body, unknown fields, and missing or unexpected roles.
- `app/agents/adk_runtime.py`: `AdkAgentRuntime(skill_dirs, model_factory)`, the only module importing `google.adk`.
- **Per run:** a fresh `LlmAgent`, an in-memory session and a `Runner`.
- **Tools:** each `ToolSpec` is adapted to a `BaseTool` with a JSON-schema declaration, and a `SkillToolset` holding only the role's Skills is filtered to `list_skills`/`load_skill`/`load_skill_resource`.
- **Instructions:** passed as a callable, so role text is never state-templated.
- **Stopping:** a `before_tool_callback` budget counts every tool call, including Skill tools. Tool calls within a run are serialised, the first outcome ends the run (later calls in the same batch are refused), and a final text with no outcome becomes `Stopped(text)`.
- **Model:** `default_model_factory` wraps `role.model` in `LiteLlm`.
- `app/execution/pillars.py`: the `consult_pillar_advisors(brief, pillars?)` tool, which runs one `pillar-advisor` per pillar in parallel through `AgentRuntime`, each scoped to its pillar Skill with a `submit_advice` tool. One failed advisor is reported without losing the others. `app/execution/toolkit.py`: `make_tool_factory(runtime, roles, pillar_skills)` gives the tool to the design advisor only; increment 6 extends it.
- `app/execution/deliverables.py`: `inputs_of(kind)` gives a Specialist every Deliverable of earlier Phases plus everything it depends on within its own Phase. Before this, the packager's context lacked `architecture_description`, which nothing depends on.
- `providers/gcp.yaml` and `app/providers.py`: `role_skills` and `pillar_skills`, both validated against the Skills index. Models are now `gemini/gemini-pro-latest` by default and `gemini/gemini-flash-latest` for the pillar advisor and diagrammer.
- `requirements.txt`: `google-adk[extensions]==2.9.0`, pinned exactly because Skills are Experimental. Its extras bring LiteLLM (installed 1.101.0).
- Tests:
- `tests/agents/test_roles.py`: 14 tests.
- `tests/agents/test_adk_runtime.py`: 12 tests with a scripted `BaseLlm`, no network.
- `tests/agents/test_adk_engine.py`: 1 test running `ExecutionEngine` on the ADK runtime with the real role files and bundle. A Specialist question reaches the user and the answer reaches the next run.
- `tests/execution/test_pillars.py`: 6 tests.
- Plus provider and deliverable tests.
- Full suite: **158 passed**. `ruff` and `mypy` are clean. The 5 warnings are ADK and Starlette deprecations.
- **Checking the tests bite.** I broke each of these in the code on purpose, and each break failed at least one test:
- every Skill loadable by every role
- no budget
- no early stop after an outcome
- templated instructions
- both turn-ended guards removed at once
Either guard alone suffices under ADK 2.9's sequential callback order; both are kept as defence in depth.
**Deviations from the spec text**
- **Role frontmatter is provider-neutral.** Role files carry only `role`, `deliverable`, `tool_tiers`, `budget` and `includes`. Skills (`role_skills`, `pillar_skills`), MCP servers and models come from the provider bundle, so no role file names a Google Cloud Skill (a test checks this). `upstream_task` was dropped: role bodies describe the Skill task in neutral words.
- **Model ids.** They're LiteLLM `-latest` aliases, not the spec's `gemini-2.5-*` ids, which are older than LiteLLM's current Gemini 3.x entries.
- **LiteLLM version.** `litellm` comes from `google-adk[extensions]` (ADK requires ≥ 1.84), not `litellm~=1.74`.
- **Pillar fan-out** is a tool on the design advisor, not code inside the runtime.
**Deferred**
- **To increment 6:** mapping `RoleSpec.mcp_servers` to ADK MCP toolsets (`McpToolset` needs the `mcp` extra, not installed) and supplying local tools by `tool_tiers`. Both are declared and loaded now, but the runtime doesn't use them yet.
- **Not yet run against a real model:** no API key was used, so role-file quality is unproven until increment 8's evaluation.
- **Model API errors propagate** from `AgentRuntime.run` and fail the graph step, leaving the Execution at its last checkpoint. Decide in increment 7 whether to retry, surface as a question, or fail the A2A task.