gcp_solution_architecture_agent/.scratch/phase-pipeline/spec.md

Spec: Phase pipeline

Status: superseded by .scratch/solution-architect-agent/spec.md. Don't implement. This file remains the decision log (D1 to D46), and its still-valid cleanup items moved to increment 9 of the new spec.

Collapse the three orchestrations of the five Phases into one deep Phase pipeline module on LangGraph, with a single phase table as the source of truth.

• Origin: architecture review candidate 01 (2026-09-15), settled through a grilling session.
• Glossary: CONTEXT.md. Decision records: ADR-0001 (reopened), ADR-0002.

Reopened by ADR-0002

The agent is meant to replace the upstream google-cloud-solution-architecture skill. It should run Agent Skills through a framework's own skill runtime (ADK SkillToolset, or Deep Agents skills=), stay portable across Cloud providers, and target Google Cloud first. The decisions below were made before that goal was stated.

Decision	Why it's in question
D1 LangGraph StateGraph sequences Phases	If the Skill describes the workflow and a skill runtime executes it, code-owned sequencing duplicates the Skill. The framework choice shrinks to "which skill runtime first". Note that Deep Agents compiles to a LangGraph graph, so the KAB workflow standard can still be met.
D2 Phase bodies stay deterministic templates	A Skill's instructions only matter if a model executes them. Templates bypass the Skill entirely.
D4 No approval gate	The upstream Skill requires explicit user approval of every Deliverable, and permission before running any script.
D6 Phase row binds one Skill per Phase	The upstream Skill is one Skill covering all four Phases, with references/ and assets/. It sits badly with one-Skill-per-Phase, and our app/skills/* don't follow the specification (snake_case names, custom phase: key).
D7 step(state, ctx) with an injected skill_prompt	Skill runtimes use progressive disclosure: the model calls load_skill / load_skill_resource. They don't take prompt injection.
D5, D11 to D13, D15, D16	These depend on who sequences Phases.
Phase list	Upstream has four Phases, with current state folded into requirements discovery. Ours has five, with a separate source_discover.

Settled so far (round 5)

#	Decision
D19	Hybrid sequencing. Code owns Phase boundaries and the Approvals between them. The Skill, run through a skill runtime, owns the work within each Phase.
D20	Interactive by default, with multi-turn Approvals. A pre-approved mode serves batch and eval runs. The /generate one-shot contract (D3) is no longer the primary interface.
D21	Vendor the upstream Skill unmodified and pinned to a commit. Our tooling gets its own Skills. The five app/skills/* are retired.
D22	A Cloud provider is a configured bundle of Skills and tools. No provider interface yet, and no provider prefix on core module names.
D23	Not linear. A hierarchy of agents follows the dependencies between Deliverables, and every agent runs a ReAct loop. This supersedes the linear phase table (D5, D6) and the round 6 questions on the engine (Q23), Approval granularity (Q25) and Phase completion (Q26), which are re-asked once the hierarchy is settled.

Settled in rounds 6 and 7

#	Decision
D24 (Q23)	Engine. LangGraph holds Execution state, checkpoints and interrupt(). The Orchestrator and Specialists are ADK LlmAgents with SkillToolset, behind one agent-runtime interface. Deep Agents is a later second implementation. See ADR-0003.
D25 (Q24)	Upstream's 4 Phases, as groupings of Deliverables. The environment scan becomes a tool the requirements analyst can call.
D26 (Q25)	Approvals. Code enforces and records Approval at Phase boundaries. Within a Phase, approval is conversational and driven by the Skill. Every user question is an interrupt().
D27 (Q26)	Completion. An agent finishes its work by calling a submit tool, and code checks the submission against the Deliverable contract. If anything is missing, the tool tells the agent what.
D28 (Q27)	Transport. A2A for interactive Executions (input-required covers questions and Approvals). /generate is kept for pre-approved Executions.
D29 (Q28)	Hierarchy. An Orchestrator sits above one Specialist per Deliverable. A Specialist fans out only for independent parts, such as the 6 WAF pillar advisors under the design advisor.
D30 (Q29)	Dependency graph declared in code. The ReAct Orchestrator chooses work through tools (ready_work, delegate, request_approval) that refuse work whose dependencies aren't met. Independent work may run in parallel.
D31 (Q30)	Per-agent ReAct step budgets, configurable. Hitting a budget hands control back to the user with a summary.

Settled in round 8

#	Decision
D32 (Q31)	Two thresholds. A Dependency inside a Phase is met once the Deliverable is submitted, and a Specialist submits only after the user's conversational OK. A Dependency across Phases is met only by that Phase's Approval.
D33 (Q32)	One voice. Only the Orchestrator talks to the user. A Specialist asks through needs_input(question), and questions from parallel Specialists are combined.
D34 (Q33)	Stateless Specialists. Each delegation starts fresh from its Dependencies, any feedback and the user's answers. Only the LangGraph checkpoint persists.
D35 (Q34)	Revision replaces the Review round. Reopening a Deliverable returns it and everything that depends on it to draft, and revokes affected Phase Approvals. Revisions per Deliverable are capped, and at the cap the Orchestrator escalates to the user.
D36 (Q35)	Models are configurable per agent role via ADK LiteLLM, with a Gemini default.
D37 (Q36)	New spec at .scratch/solution-architect-agent/spec.md. This spec becomes superseded, and its still-valid parts become a cleanup section there.

Settled in round 9

#	Decision
D38 (Q37)	The dependency graph is as proposed: 9 Deliverables across 4 Phases, deployment guidance depends on design recommendations, and runtime validation is out of scope for v1.
D39 (Q38)	Shared SkillToolset plus a role file per agent (agents/<role>.md, framework-neutral). Pillar advisors are scoped to their own WAF Skill.
D40 (Q39)	Vendor the upstream solution-architecture Skill and the 6 WAF Skills into skills/vendor/google/, pinned to 81a31a6, with VENDOR.md, the Apache-2.0 licence and a sync script.
D41 (Q40)	Tool tiers. Offline tools run freely. Tools that read live cloud resources need permission once per Execution (allow_cloud_read in pre-approved mode). Nothing that changes cloud resources is exposed, and run_skill_script is disabled.
D42 (Q41)	Deliverables live in Execution state and the DB, and are written only to deliverables/executions/{id}/ after Phase 4 Approval. No shared-path copies.
D43 (Q42)	A fake agent runtime for deterministic tests. The real-model eval runs pre-approved with scripted user answers.
D44 (Q43)	Build alongside the current code, then one cleanup increment after the eval passes.
D45	Discovery and grounding use the Cloud provider's own remote MCP servers (user direction, 2026-09-15). The servers are declared in the provider bundle, with reviewed read-only allowlists. All customer-reading MCP tools are cloud_read. GCPEnvironmentScanner and the hand-rolled Developer Knowledge client are retired.
D46	Call remote MCP servers directly, with no platform MCP gateway for now (user decision, 2026-09-15).

The spec needs rewriting around the Orchestrator and Specialists once the open questions are settled. The current design section still describes the superseded linear Phase pipeline.

Unaffected: delete the fake ADK compat layer (D10's deletions), clean Sessions (D9), and build once at startup (D14). The candidate 02 to 05 findings stand. Grounding and scanning ports (candidate 02) become more important, because they are the tooling a Skill calls on.

• Branch: refactor/phase-pipeline, pushed to the Gitea remote; the PR is opened by a human.

Problem

The five Phases are declared in seven places: app/adk/agents.py, app/workflows/gcp_architecture_graph.py, app/nodes/__init__.py, each node's active_skills bookkeeping, app/card.py (twice), and workflow.yaml (which already omits source_discover). They are run by three orchestrators: a LangGraph StateGraph, an "ADK" SequentialAgent inside a LoopAgent, and an "ADK" Workflow. None of these uses the real google-adk package. HAS_NATIVE_ADK is never read, so every "ADK" class is a local stub. The five agent classes are pass-throughs. Every node builds a skill prompt and discards it. The review loop reruns all five Phases without passing validation findings back. And passing a session_id merges a previous Execution's outputs into the next one.

Goals

• One module owns Phase ordering, state merging, Phase contracts, the Review round, Skill binding and active_skills.
• One small interface: async run(workflow_request) -> Execution.
• Tests exercise the pipeline only through that interface.
• Adding or reordering a Phase is a one-row change to the phase table.

Non-goals (other review candidates)

• Grounding and environment scan ports (candidate 02). Steps keep constructing GCPEnvironmentScanner and calling get_mcp_client().
• The Execution store (candidate 03). DatabaseManager stays as it is, dual-dialect branches included.
• The Deliverable layout (candidate 04). The state-key → path mapping moves unchanged, and output paths in workflow.yaml are left alone.
• LangChain @tool pass-throughs in app/tools/ (candidate 05). validate keeps calling .invoke.
• Typed Phase outputs (candidate 06). Outputs stay markdown strings.
• Making any Phase call an LLM. The interface gets ready for it; the step bodies keep their templates.
• A human approval gate, checkpointing or resume.
• Moving the transport to aiohttp A2A on :8000.

Decisions

#	Decision
D1	LangGraph StateGraph orchestrates Phases. The ADK compat layer is deleted. Real ADK may be used inside a step later (ADR-0001).
D2	Target direction: discover, design and package become LLM-driven; source_discover and validate stay deterministic. This work only readies the interface.
D3	Stable: the /generate response shape and the Deliverable paths. /card is derived from the phase table. workflow.yaml is test-checked against it. Agent and log names may change.
D4	No approval gate. There is a single run.
D5	Review round: a conditional edge validate → design while validation fails and rounds < MAX_REVIEW_ROUNDS. Findings reach design as review_feedback.
D6	Phase row: Phase(name, skill, step, requires, produces). The pipeline enforces contracts, maintains active_skills and current_phase, and binds Skills by name.
D7	Step: async step(state, ctx: PhaseContext) -> dict. ctx has skill_prompt and model=None.
D8	The pipeline does no DB or file I/O. The caller records the Execution.
D9	A Session groups Executions. Every Execution starts from clean state.
D10	app/adk/ is removed. app/executions.py records Executions for the route and the eval harness.
D11	Module: app/workflows/gcp_architecture_pipeline.py. Steps stay in app/nodes/. The state schema stays in app/states/.
D12	The phase table is a constructor argument, so tests inject a fake table. Old orchestrator tests are deleted.
D13	The card is derived at import. workflow.yaml gains source_discover, and a test asserts its phase ids match.
D14	The pipeline is built once in the Starlette lifespan and stored on app.state.pipeline.
D15	When Review rounds run out with validation still failing, the pipeline packages anyway: validation_passed=false, status completed_with_warnings.
D16	A contract breach raises PhaseContractError. The Execution is recorded failed, the route returns HTTP 500, and no Deliverables are written.
D17	Blocking I/O in source_discover and design is offloaded with asyncio.to_thread.
D18	Also: remove google-adk from requirements.txt, rewrite the README's ADK sections, delete the unreachable duplicate return in routes.py.

Design

Module: app/workflows/gcp_architecture_pipeline.py

Replaces app/workflows/gcp_architecture_graph.py.

@dataclass(frozen=True)
class PhaseContext:
    skill_prompt: str
    model: Any | None = None          # D2: populated when a Phase becomes LLM-driven

Step = Callable[[Mapping[str, Any], PhaseContext], Awaitable[dict[str, Any]]]

@dataclass(frozen=True)
class Phase:
    name: str                          # e.g. "design"
    skill: str                         # Skill name, e.g. "architecture_design"
    step: Step
    requires: tuple[str, ...]          # state keys that must be present (not None) before the step
    produces: tuple[str, ...]          # state keys the step must return

@dataclass(frozen=True)
class ReviewRound:
    number: int                        # 1-based
    passed: bool
    findings: tuple[str, ...]

@dataclass(frozen=True)
class Execution:
    execution_id: str
    workflow_request: str
    outputs: Mapping[str, Any]         # final state values for every Phase's `produces`
    validation_passed: bool
    review_rounds: tuple[ReviewRound, ...]
    active_skills: tuple[str, ...]
    current_phase: str                 # last Phase run ("package" on success)

class PhaseContractError(Exception):
    phase: str
    missing: tuple[str, ...]
    kind: Literal["requires", "produces"]

class PhasePipeline:
    def __init__(self, phases: Sequence[Phase], skills: SkillLoader, max_review_rounds: int) -> None: ...
    async def run(self, workflow_request: str) -> Execution: ...

PHASES: tuple[Phase, ...] = (...)      # production table, below

Construction invariants. Each of these raises ValueError from __init__:

• phase names are not unique
• a Phase's skill is missing from the SkillLoader
• there is no Phase named design or validate (the Review round needs both)

The StateGraph is compiled once, in __init__.

Graph. Linear edges follow table order. The exception is one conditional edge leaving validate:

• Loop back: if validation_passed is false and the rounds so far are fewer than max_review_rounds, go to design.
• Otherwise: continue to the Phase after validate.

Per-Phase node wrapper. The pipeline generates one per row, and it does the following in order:

1. Check inputs. Every requires key must be present and not None in state. Otherwise raise PhaseContractError(kind="requires").
2. Run the step. Build PhaseContext(skill_prompt=<that Skill formatted for prompt>) and await step(read-only state view, ctx).
3. Check outputs. Every produces key must be in the returned dict. Otherwise raise PhaseContractError(kind="produces").
4. Merge only declared keys. Keys the step returns that aren't in produces are dropped, with a warning logged.
5. Update bookkeeping.
   • Set current_phase to the Phase name.
   • Append the Skill to active_skills if it isn't already there.
   • For design: increment review_round.
   • For validate: append a ReviewRound(review_round, validation_passed, errors) to review_rounds. On failure, set review_feedback = errors.

Key presence. A key is present if it is not None. False, "" and [] all count as present.

run.

• Initial state: {execution_id: uuid4, workflow_request, active_skills: [], review_round: 0, review_rounds: [], review_feedback: []}. Nothing is carried over from a Session (D9).
• Execution: ainvoke the compiled graph, then build the Execution.
• Errors: PhaseContractError propagates to the caller.

State: app/states/state.py

GCPArchitectureState changes:

• Added: execution_id: str, review_round: int, review_rounds: list[ReviewRound], review_feedback: list[str].
• Removed: target_dir. Nothing reads it; /generate still accepts the field and ignores it, as it effectively does today.

Production phase table

Phase	Skill	requires	produces
source_discover	source_discovery	workflow_request	source_discovery_doc, source_mermaid_diagram
discover	requirements_discovery	workflow_request	requirements_doc, product_selection_deferred
design	architecture_design	requirements_doc	architecture_doc, mermaid_diagram, terraform_code, product_selection_deferred
validate	validation_rules	mermaid_diagram, terraform_code	validation_results, validation_passed, errors
package	packaging_guide	source_discovery_doc, source_mermaid_diagram, requirements_doc, architecture_doc, mermaid_diagram, terraform_code, validation_results	solution_guide, status_summary

design doesn't read requirements_doc today. The table declares the dependency the Phase is meant to have (see workflow.yaml). design may also read review_feedback, which is always present.

Steps: app/nodes/*.py

• New signature: each *_node(state, skill_loader) becomes async def *_step(state, ctx) -> dict (D7). Bodies keep their current templates.
• Removed from steps: each step returns only its produces keys. The skill_loader.format_skills_for_prompt(...) call and the active_skills/current_phase bookkeeping are deleted.
• Blocking I/O (D17):
  • source_discover: scan_result = await asyncio.to_thread(scanner.scan_environment). The scanner is also constructed inside the thread, because its __init__ can shell out to gcloud.
  • design: await asyncio.to_thread(mcp_client.search_documents, ...).
• Skills: SkillLoader gains format_skill_for_prompt(name) -> str, which gives the single-Skill form of format_skills_for_prompt.

Recording: app/executions.py

A plain module that replaces ADKAgentRunner, PostgresSessionService, PostgresArtifactRepository and ADKEvaluator's persistence. It uses get_db_manager() for now (candidate 03 replaces that).

• record(execution, session_id, base_dir) -> None:
  • Writes Deliverables using the state-key → path mapping moved verbatim from app/adk/artifacts.py. That covers current paths, per-execution paths and docs/ mirrors.
  • Saves one adk_artifacts row per written file.
  • Saves the session row (agent_name = settings.AGENT_NAME, state_data = the Execution as JSON).
  • Saves a workflow_executions row with status completed or failed from validation_passed, loop_count = len(review_rounds), and the results JSON as today.
  • Writes one orchestrator_review_logs row per ReviewRound: reviewer_agent="validate", review_status APPROVED or NEEDS_REVISION, feedback = the joined findings.
• record_failure(execution_id, session_id, workflow_request, error: PhaseContractError) -> None: saves a workflow_executions row with status failed, current_phase = error.phase and the error details in the results JSON. It writes no Deliverables.
• to_generate_response(execution, session_id) -> dict builds the stable /generate shape:
  • execution_id, session_id, validation_passed, current_phase, active_skills
  • status: success or completed_with_warnings
  • total_loop_iterations: len(review_rounds)
  • artifacts: the same 8 keys as today

HTTP: app/agent.py, app/workflows/routes.py

• Lifespan: runs get_db_manager() as today, then app.state.pipeline = PhasePipeline(PHASES, loaded SkillLoader, settings.MAX_REVIEW_ROUNDS) (D14).
• POST /generate: await request.app.state.pipeline.run(workflow_request), then executions.record(...), then return to_generate_response(...).
  • On PhaseContractError: call executions.record_failure(...) and return HTTP 500 with {"error": str(e), "phase": e.phase, "missing": [...]}.
  • A missing or invalid session_id in the body gets a new uuid, as today.
• GET /sessions/{id}: reads through get_db_manager().get_session. The response shape is unchanged, but agent_name becomes gcp_solution_architecture_agent.
• GET /health: drops "adk_framework".
• POST /validate: unchanged. Delete the unreachable duplicate return.

Card and manifest

• app/card.py: capabilities.phases = [p.name for p in PHASES] and capabilities.local_skills = [p.skill for p in PHASES]. All other fields are unchanged.
• workflow.yaml: add a source_discover phase entry before discover. Renumbering step is left to the implementer, as long as order matches PHASES.

Settings: app/config.py, .env.example

• MAX_REVIEW_ROUNDS: int = 5, which also reads the legacy env name ADK_MAX_LOOP_ITERATIONS (pydantic AliasChoices).
• Delete ADK_MAX_LOOP_ITERATIONS and ADK_ENABLE_ARTIFACT_STORE (the latter is never read).
• Update .env.example to match.

Eval: eval/eval_harness.py

• EvalHarness.run_eval_suite becomes async. EvalHarness builds a PhasePipeline, awaits run(case["workflow_request"]), scores execution.outputs with eval/metrics.evaluate_case_run, and saves with get_db_manager().save_evaluation.
• main() uses asyncio.run.
• This removes the app → eval import that app/adk/evaluation.py created.

Deleted

• app/adk/ (all 9 files)
• app/workflows/gcp_architecture_graph.py
• tests/test_graph_workflow.py
• google-adk from requirements.txt

Behaviour changes (call out in the PR)

1. A session_id no longer seeds an Execution with a previous Execution's state.
2. A failing validation now re-runs only design → validate, not all five Phases.
3. A Phase contract breach returns HTTP 500 and records a failed Execution. Before, the pipeline silently continued with empty strings.
4. /sessions/{id} returns agent_name: "gcp_solution_architecture_agent", not OrchestratorLoopAgent.
5. /health no longer includes adk_framework.
6. total_loop_iterations counts Review rounds, not whole-pipeline passes. Its value is 1 on the happy path, the same as today.
7. workflow.yaml lists source_discover.

Testing

Tests go through PhasePipeline.run() and the HTTP interface only, using pytest-asyncio.

New: tests/test_phase_pipeline.py

These use a fake phase table of tiny async steps and a stub SkillLoader fixture (or a tmp skills dir):

• Phase order: Phases run in table order, and current_phase ends on the last Phase.
• Merging: only declared produces keys are merged, and extra keys are dropped.
• Requires check: a missing requires key raises PhaseContractError(kind="requires", phase=...).
• Produces check: a step that omits a produces key raises PhaseContractError(kind="produces", phase=...).
• Presence: False and "" count as present.
• Review round, passes on round 2: validate fails once and then passes. design runs twice and receives review_feedback on round 2; review_rounds has 2 entries and validation_passed is true.
• Review round, never passes: validate always fails. Rounds stop at max_review_rounds, the Phase after validate still runs, and validation_passed is false.
• Skills: ctx.skill_prompt holds the bound Skill's content, and active_skills lists each Skill once, even across Review rounds.
• Clean runs: two run() calls on one pipeline share no state.
• Construction: duplicate names, an unknown Skill, or a missing design/validate each raise ValueError.

Also new:

• End to end: run() with the real PHASES and app/skills. Every produces key is present, validation_passed is true, there are 5 active_skills and 1 review round.
• Manifest check: the phase ids in workflow.yaml equal [p.name for p in PHASES], and /card phases and skills equal the table.

Changed

• tests/test_agent_api.py: use with TestClient(app) as client, because the lifespan must run so app.state.pipeline exists. Existing assertions stay and guard the stable /generate contract. Add one test for the 500 response on a contract breach, with a pipeline built from a breaching fake table swapped onto app.state.
• tests/test_adk_architecture.py:
  • Delete: test_adk_tools, test_adk_postgres_session_service, test_adk_orchestrator_loop_agent, test_adk_workflow_composition, test_adk_agent_runner, test_adk_evaluator.
  • Keep: test_database_manager_operations, moved to tests/test_database.py.
  • Move: the session endpoint test into test_agent_api.py, asserting the new agent_name.
  • Then delete the file.
• tests/test_mcp_developer_knowledge.py: delete test_adk_mcp_function_tools and its app.adk.tools import.
• eval/test_eval_harness.py: test_eval_harness_benchmark awaits the now-async EvalHarness.run_eval_suite() (@pytest.mark.asyncio), and its assertions are unchanged. test_skill_optimizer_recommendations is untouched.

Known issue, left out of scope: /generate tests write Deliverables into the working tree and the local SQLite file (candidates 03 and 04).

Implementation increments

These are ordered. Each one leaves the suite green and is one or more commits on refactor/phase-pipeline. They are called increments to avoid clashing with the glossary's Phase.

1. Baseline environment.
   • Create a venv and install requirements.txt and requirements-dev.txt.
   • Run the current suite and record the baseline pass/fail list in this spec's ## Comments. No Python environment exists on the dev machine yet.
   • Create the branch.
2. Pipeline core, test-first.
   • Add Phase, PhaseContext, ReviewRound, Execution, PhaseContractError, PhasePipeline and SkillLoader.format_skill_for_prompt.
   • Write every fake-table test in test_phase_pipeline.py first.
   • Production code is untouched apart from these additions.
3. Port the steps and the table.
   • Convert the five nodes to async steps (including D17), define PHASES, extend GCPArchitectureState and add the end-to-end test.
   • Delete gcp_architecture_graph.py and test_graph_workflow.py, and update app/workflows/__init__.py and app/nodes/__init__.py exports.
   • The app/adk/ agents still call the old node names at this point, so either keep thin old-signature shims until increment 5 or do increments 3 and 5 together. Prefer shims, deleted in 5.
4. Recording and HTTP wiring.
   • Add app/executions.py, the lifespan pipeline, the /generate success and 500 paths, /sessions via DatabaseManager, and /health.
   • Rename settings, with the alias, and update .env.example.
   • Update test_agent_api.py (lifespan client, 500 test, session endpoint test).
5. Remove ADK.
   • Move the eval harness onto the pipeline.
   • Delete app/adk/, the old-signature shims, the ADK tests and google-adk from requirements.txt.
   • Move the DB test to tests/test_database.py.
6. Card and manifest. Derive the card from PHASES, add source_discover to workflow.yaml, and add the manifest consistency test.
7. Docs and cleanup.
   • Rewrite the README's ADK sections around the Phase pipeline.
   • Delete the duplicate return in routes.py.
   • Run a final full suite plus ruff and mypy on the touched modules.
   • Push the branch.

Acceptance

• rg -n "app\.adk|google\.adk|OrchestratorLoopAgent|compat" app eval tests returns nothing.
• The phase list appears only in PHASES, and workflow.yaml is test-checked against it.
• Every test listed under Testing exists and passes. The rest of the baseline suite is no worse than increment 1's record.
• The /generate response keys match today's exactly: execution_id, session_id, status, validation_passed, total_loop_iterations, current_phase, artifacts (8 keys) and active_skills.
• The Deliverable paths written by /generate are unchanged.
• The README no longer describes ADK orchestration, and ADR-0001 is linked.