This commit is contained in:
54
.agents/plans/README.md
Normal file
54
.agents/plans/README.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Plans
|
||||
|
||||
One directory per iteration of this agent. Each holds the plan for that iteration and, for finished steps, the record of what was built. Read the row for the iteration you are working in, then its `MAP.md`.
|
||||
|
||||
## Iterations
|
||||
|
||||
| # | Directory | Status | What it is |
|
||||
|---|---|---|---|
|
||||
| 1 | `wf-decompose-65ea49fc2d04/` | complete, superseded | The decomposer's scaffold: a linear four-phase pipeline of template functions. Its code was deleted in iteration 3 and survives only in git history. Generated by the platform decomposer; leave its files as written. |
|
||||
| 2 | `iter-02-solution-architect-agent/` | complete; step 9 was done in iteration 3 | The rebuild as a hierarchical ReAct agent: Orchestrator, one Specialist per Deliverable, Skills, provider bundle, A2A. |
|
||||
| 3 | `iter-03-kab-layout/` | complete (uncommitted) | The container starts the new engine, the engine moved into the KAB workflow-agent folder layout, and the original pipeline was deleted. |
|
||||
| 4 | `iter-04-learning-and-skills/` | planned | The enhancement backlog from the 2026-10-06 review: cheap fixes, the learning loop through Skills, overlay Skills, provider neutrality. |
|
||||
| 5 | `iter-05-jev-decisions/` | planned, gated on open questions | Jev (a hosted decision model) for the agent's closed decisions: checks, routing, context selection. |
|
||||
|
||||
Two designs sit between iterations 1 and 2 and have no plan directory. Commit `a24a44e` wrapped the pipeline in a compatibility layer that imitated Google ADK without using it (deleted in iteration 3). `.scratch/phase-pipeline/spec.md` then designed a code-sequenced Phase pipeline; ADR-0002 reopened it before any of it was built, and its decision log (D1 to D46) explains how iteration 2's design was reached.
|
||||
|
||||
## Files in an iteration
|
||||
|
||||
- `MAP.md`: the intent, each step's status, the dependency graph between steps, and the sources of truth to read.
|
||||
- `DETAIL.md`: per step, the capability, why it exists (with evidence), where the change lands, and the acceptance criteria.
|
||||
- `steps/step-N.md`: written when step N is finished. What was built, what differed from the plan, and how it was verified. A step with no file is not done.
|
||||
|
||||
Steps are numbered from 1 in iterations 2 onward. Iteration 1 numbers from 0.
|
||||
|
||||
## Working in an iteration
|
||||
|
||||
1. Read `MAP.md`, then the `DETAIL.md` entry for your step and every step it depends on.
|
||||
2. Read the sources of truth `MAP.md` names. Plans point at specs and ADRs; they repeat only what those lack.
|
||||
3. Build the step until every acceptance criterion in its `DETAIL.md` entry holds and `.venv/bin/python -m pytest -q`, `.venv/bin/ruff check .` and `.venv/bin/mypy .` all pass.
|
||||
4. Write `steps/step-N.md` and set the step's status in `MAP.md` to `complete`. When the last step closes, update the iteration's row above.
|
||||
|
||||
A new iteration gets the next number, a directory `iter-NN-<slug>/`, and a row above.
|
||||
|
||||
## Module paths changed in iteration 3
|
||||
|
||||
Iteration 2's spec and ADR text name the modules by their old paths. The code is here now:
|
||||
|
||||
| Old path | Current path |
|
||||
|---|---|
|
||||
| `app/execution/deliverables.py` | `app/workflows/deliverables.py` |
|
||||
| `app/execution/graph.py` | `app/nodes/execution_nodes.py` (nodes) and `app/workflows/execution_graph.py` (wiring) |
|
||||
| `app/execution/engine.py`, `app/assembly.py` | `app/agent.py` |
|
||||
| `app/execution/state.py` | `app/states/execution_state.py` |
|
||||
| `app/execution/ledger.py` | `app/states/ledger.py` |
|
||||
| `app/execution/tools.py`, `toolkit.py`, `pillars.py` | `app/tools/execution.py`, `app/tools/toolkit.py`, `app/tools/pillars.py` |
|
||||
| `app/execution/recorder.py`, `checkpointing.py`, `presentation.py` | `app/recorder.py`, `app/checkpointing.py`, `app/presentation.py` |
|
||||
| `app/agents/{runtime,adk_runtime,fake_runtime,roles}.py` | `app/adk/` (same file names) |
|
||||
| `app/agent_skills.py` | `app/skills/agent_skills.py` |
|
||||
| `app/a2a/server.py`, `executor.py`, `card.py` | `app/main.py`, `app/a2a_server.py`, `app/card.py` |
|
||||
| the pre-rebuild `app/agent.py`, `app/card.py`, `app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/`, `app/states/` deleted; in git history up to `e94b249` |
|
||||
| `tests/test_{adk_architecture,agent_api,graph_workflow,mcp_developer_knowledge,skill_loader}.py` | deleted |
|
||||
| `eval/{metrics,optimizer,eval_harness,test_eval_harness}.py`, `eval/datasets/benchmark_cases.json` | deleted |
|
||||
|
||||
`app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/` and `app/states/` therefore mean different code before and after iteration 3. In text written before 2026-10-06, those names refer to the deleted original pipeline.
|
||||
65
.agents/plans/iter-02-solution-architect-agent/DETAIL.md
Normal file
65
.agents/plans/iter-02-solution-architect-agent/DETAIL.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# Plan Detail
|
||||
|
||||
The spec is the full design. Each entry here gives the step's purpose, the rule it must keep, and its acceptance criteria. Paths are current (post iteration 3).
|
||||
|
||||
## Step 1: Environment and baseline
|
||||
|
||||
- **Capability:** A reproducible environment and a recorded test baseline to compare every later step against.
|
||||
- **Rationale:** Later steps claim "the suite stays green"; that needs a starting count.
|
||||
- **Where:** `.venv/`, `requirements.txt`, `requirements-dev.txt`.
|
||||
- **Acceptance criteria:** The existing suite runs and its result is recorded in the spec's comments.
|
||||
|
||||
## Step 2: Vendored Skills and provider bundle
|
||||
|
||||
- **Capability:** Upstream Skills available unmodified, and a Cloud provider expressed as configuration.
|
||||
- **Rationale:** ADR-0002. Upstream changes must arrive as a diff, and a second Cloud provider must be addable without touching the core.
|
||||
- **Where:** `skills/vendor/google/` (pinned to upstream commit `81a31a6`, with `VENDOR.md`, `LICENSE`, `SHA256SUMS`), `scripts/sync-vendored-skills.sh`, `providers/gcp.yaml`, `app/providers.py`, `app/skills/agent_skills.py`.
|
||||
- **Acceptance criteria:** Vendored files match their checksums. Every Skill under `skills/` passes the Agent Skills frontmatter rules. Only `app/providers.py` names a provider.
|
||||
|
||||
## Step 3: Deliverables and Ledger
|
||||
|
||||
- **Capability:** The dependency graph of Deliverables as data, and a pure Ledger that enforces acceptance, Approval and Revision rules.
|
||||
- **Rationale:** ADR-0003. Dependencies are guaranteed by code; ordering is the model's choice.
|
||||
- **Where:** `app/workflows/deliverables.py` (the only place Phases, Deliverables and Dependencies are declared), `app/states/ledger.py`.
|
||||
- **Acceptance criteria:** A Dependency inside a Phase is met on submission; across Phases only on Approval. Reopening a Deliverable returns it and its dependents to draft and withdraws affected Approvals. Revisions are capped.
|
||||
|
||||
## Step 4: Execution graph
|
||||
|
||||
- **Capability:** A checkpointed LangGraph Execution that pauses for the user, behind one `ExecutionEngine` interface.
|
||||
- **Rationale:** ADR-0003. Resumable, recordable progress at every step, with framework types kept out of state.
|
||||
- **Where:** `app/adk/runtime.py` (interface), `app/adk/fake_runtime.py`, `app/states/execution_state.py`, `app/tools/execution.py`, `app/nodes/execution_nodes.py`, `app/workflows/execution_graph.py`, `app/agent.py`, `app/recorder.py`, `app/checkpointing.py`.
|
||||
- **Acceptance criteria:** Only `orchestrator_turn` and `await_user` change the Ledger. No Approval is ever granted from free text. Cloud-read tools are unreachable without permission. All of it is tested with the fake runtime and no model.
|
||||
|
||||
## Step 5: ADK runtime and role files
|
||||
|
||||
- **Capability:** Real ADK `LlmAgent`s running role files with scoped Skills, behind the runtime interface.
|
||||
- **Rationale:** ADR-0003. Each agent runs its own ReAct loop; Skills load on demand through `SkillToolset`.
|
||||
- **Where:** `app/adk/adk_runtime.py` (the only module that imports `google.adk`), `app/adk/roles.py`, `agents/*.md`, `agents/partials/specialist-protocol.md`, `app/tools/pillars.py`.
|
||||
- **Acceptance criteria:** Role files carry no Skill, MCP server or model names; the bundle supplies them. `run_skill_script` is never offered. Each pillar advisor loads only its own pillar Skill.
|
||||
|
||||
## Step 6: Tools and remote MCP servers
|
||||
|
||||
- **Capability:** Local check tools and the Cloud provider's hosted MCP servers, exposed through reviewed read-only allowlists.
|
||||
- **Rationale:** ADR-0002. No hand-written cloud API clients; reading a user's cloud needs their permission.
|
||||
- **Where:** `app/tools/local.py`, `app/tools/mcp.py`, `app/tools/toolkit.py`, `providers/gcp.yaml` (`mcp_servers`, `role_mcp_servers`), `skills/cloud-discovery/`, `skills/iac-validation/`, `docs/operations/mcp-access.md`, `providers/reviews/gcp-mcp-tools.json`.
|
||||
- **Acceptance criteria:** Only allowlisted tools are exposed. No tool can apply, deploy or run a script. An unavailable server becomes a recorded discovery gap, not a failure.
|
||||
|
||||
## Step 7: Interfaces
|
||||
|
||||
- **Capability:** Interactive Executions over A2A, and a pre-approved one-shot `/generate`.
|
||||
- **Rationale:** Spec decision D28.
|
||||
- **Where:** `app/main.py`, `app/a2a_server.py`, `app/card.py`, `app/presentation.py`.
|
||||
- **Acceptance criteria:** Approvals travel as structured data. The agent card is derived from the dependency graph. `/generate` response keys are unchanged from iteration 1.
|
||||
|
||||
## Step 8: Evaluation
|
||||
|
||||
- **Capability:** A real-model evaluation of the whole engine with scripted user answers.
|
||||
- **Rationale:** The gate for cleanup: proceed only when every case scores at least 0.8.
|
||||
- **Where:** `eval/solution_eval.py`, `eval/scoring.py`, `eval/datasets/solution_cases.json`, `tests/eval/`.
|
||||
- **Acceptance criteria:** Each benchmark case completes and reaches `PASS_THRESHOLD` (0.8) on the 8 checks in `eval/scoring.py`.
|
||||
|
||||
## Step 9: Cleanup
|
||||
|
||||
- **Capability:** One code path: the legacy pipeline removed and the new engine the only engine.
|
||||
- **Status:** Done in iteration 3, by a different route. On 2026-10-06 the engine first moved into the KAB folder layout, and the legacy pipeline was then deleted (`iter-03-kab-layout/steps/step-7.md`).
|
||||
- **Not done from the spec's list:** the sample artefacts the pipeline once wrote (`docs/*.md`, root `terraform/`, `deliverables/{as-is,target,validation,guides}/`) are still checked in.
|
||||
38
.agents/plans/iter-02-solution-architect-agent/MAP.md
Normal file
38
.agents/plans/iter-02-solution-architect-agent/MAP.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# Plan Map
|
||||
|
||||
**Iteration:** 2, solution architect agent
|
||||
**Intent:** Replace the linear template pipeline with a portable, hierarchical ReAct agent that runs the upstream `google-cloud-solution-architecture` Skill's workflow. An Orchestrator reasons over a code-declared dependency graph of Deliverables and delegates to one Specialist per Deliverable. Behaviour lives in Agent Skills; the Cloud provider is configuration.
|
||||
**Branch:** `feat/solution-architect-agent`
|
||||
**This directory covers:** steps 1 to 9, the spec's build increments 1 to 9
|
||||
|
||||
## Sources of truth
|
||||
|
||||
- `.scratch/solution-architect-agent/spec.md`: the design, the decisions table, the acceptance list, and a dated `## Comments` entry per finished increment. Its module paths predate iteration 3; translate them with the table in `../README.md`.
|
||||
- `docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md` and `docs/adr/0003-langgraph-holds-state-adk-agents-reason.md`: the two decisions every step serves.
|
||||
- `CONTEXT.md`: the glossary. Use its terms.
|
||||
|
||||
## Status
|
||||
|
||||
| Step | Increment | Status | Commit |
|
||||
|---|---|---|---|
|
||||
| 1 | Environment and baseline | complete | `0851171` |
|
||||
| 2 | Vendored Skills and provider bundle | complete | `2976dfa` |
|
||||
| 3 | Deliverables and Ledger | complete | `c841094` |
|
||||
| 4 | Execution graph | complete | `d9214c6` |
|
||||
| 5 | ADK runtime and role files | complete | `f040493` |
|
||||
| 6 | Tools and remote MCP servers | complete | `ed104e5` |
|
||||
| 7 | Interfaces (A2A, `/generate`) | complete | `ad371b2` |
|
||||
| 8 | Evaluation | complete, uncommitted | none |
|
||||
| 9 | Cleanup | done in iteration 3 | none |
|
||||
|
||||
## Dependency graph
|
||||
|
||||
- Step 1: no dependencies
|
||||
- Step 2: depends on step 1
|
||||
- Step 3: depends on step 1
|
||||
- Step 4: depends on step 3
|
||||
- Step 5: depends on step 2, step 4
|
||||
- Step 6: depends on step 2, step 5
|
||||
- Step 7: depends on step 4, step 5, step 6
|
||||
- Step 8: depends on step 7
|
||||
- Step 9: depends on step 8 passing its threshold
|
||||
@@ -0,0 +1,7 @@
|
||||
# Step 1 — Implementation Detail
|
||||
|
||||
1. Created `.venv/` on Python 3.12.3 and installed both requirements files. `google-adk` resolved to 2.9.0, `langgraph` to 1.2.11.
|
||||
2. Recorded the baseline: 30 tests passed across the iteration 1 suite.
|
||||
3. Found that the iteration 1 "ADK" layer never used `google-adk`: `HAS_NATIVE_ADK` stayed false even with the package installed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-15 · Increment 1".
|
||||
@@ -0,0 +1,7 @@
|
||||
# Step 2 — Implementation Detail
|
||||
|
||||
1. Vendored 7 upstream Skills (solution architecture plus six Well-Architected Framework pillars) at commit `81a31a6`, with checksums verified by `tests/test_vendored_skills.py`.
|
||||
2. Added the provider bundle `providers/gcp.yaml` and its loader, and the Agent Skills frontmatter validator.
|
||||
3. Suite: 65 passed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-15 · Increment 2".
|
||||
@@ -0,0 +1,8 @@
|
||||
# Step 3 — Implementation Detail
|
||||
|
||||
1. Declared 9 Deliverables in 4 Phases as data, with section and attachment contracts checked by `check_submission`.
|
||||
2. Built the Ledger as pure code with no I/O, serialisable into graph state.
|
||||
3. `technical_decomposition` and `architecture_description` were left with a non-empty-content contract only. Iteration 4 step 4 revisits this.
|
||||
4. Suite: 102 passed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-15 · Increment 3".
|
||||
@@ -0,0 +1,9 @@
|
||||
# Step 4 — Implementation Detail
|
||||
|
||||
1. Built the graph test-first: `orchestrator_turn`, parallel `specialist_task` via `Send`, `await_user` via `interrupt`, `finalize`, `end_with_warnings`.
|
||||
2. Added `ExecutionEngine` (`start`, `resume`, `interaction`) as the single interface; the graph is an internal detail behind it.
|
||||
3. Added the `Stopped` outcome, the single-writer rule for Specialist outcomes, and `MAX_ORCHESTRATOR_TURNS` as the loop guard.
|
||||
4. Verified the tests bite by breaking the cloud-read gate, the permission prompt, question relay and the submission contract in turn; each break failed a test.
|
||||
5. Suite: 121 passed.
|
||||
|
||||
Full record, including prompt and reply shapes: spec `## Comments`, "2026-09-15 · Increment 4".
|
||||
@@ -0,0 +1,8 @@
|
||||
# Step 5 — Implementation Detail
|
||||
|
||||
1. Implemented `AdkAgentRuntime`: a fresh `LlmAgent` and in-memory session per run, tool-call budgets, and nudges when an agent stops without ending its turn.
|
||||
2. Wrote the 11 role files and the shared Specialist protocol; the role loader rejects any field the bundle should own.
|
||||
3. Added the design advisor's fan-out to six pillar advisors.
|
||||
4. Suite: 158 passed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-16 · Increment 5".
|
||||
@@ -0,0 +1,9 @@
|
||||
# Step 6 — Implementation Detail
|
||||
|
||||
1. Added local tools `mermaid_check`, `terraform_fmt`, `terraform_validate` and `terraform_plan`, each running on a scratch copy.
|
||||
2. Added the remote MCP catalog with Application Default Credentials and per-server allowlists, reviewed on 2026-09-16 against each server's live tool list.
|
||||
3. Excluded data-content tools even where read-only (SQL queries, object reads, generic Kubernetes reads).
|
||||
4. Wrote the `cloud-discovery` and `iac-validation` Skills.
|
||||
5. Suite: 189 passed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-16 · Increment 6".
|
||||
@@ -0,0 +1,7 @@
|
||||
# Step 7 — Implementation Detail
|
||||
|
||||
1. Added the A2A server (JSON-RPC at `/a2a`, agent card, `/health`) and the executor that maps A2A tasks to Executions and their prompts.
|
||||
2. Kept `/generate` as a pre-approved one-shot Execution.
|
||||
3. Suite: 206 passed.
|
||||
|
||||
Full record: spec `## Comments`, "2026-09-16 · Increment 7".
|
||||
@@ -0,0 +1,22 @@
|
||||
# Step 8 — Implementation Detail
|
||||
|
||||
This step has no entry in the spec's comments; this file is its record.
|
||||
|
||||
1. Added `eval/solution_eval.py`: runs each case as a pre-approved Execution through the full assembly with real models, counting agent runs, outcomes, seconds and tokens per role.
|
||||
2. Added `eval/scoring.py` with 8 checks: completed, guide follows the template, required products selected, alternatives listed, documentation cited, requirements name no products, diagram is valid Mermaid, Terraform validates.
|
||||
3. Added 3 cases in `eval/datasets/solution_cases.json`. Results are written to `eval/results/<timestamp>/` (gitignored).
|
||||
|
||||
Recorded runs:
|
||||
|
||||
| Run | Case | Score | Seconds | Revisions |
|
||||
|---|---|---|---|---|
|
||||
| 2026-09-16 10:57Z | case-01-event-driven-http | 1.0 | 2,932 | design_recommendations × 2 |
|
||||
| 2026-09-17 08:44Z | case-03-internal-web-app | 1.0 | 2,623 | deployment_guidance × 3 |
|
||||
|
||||
No scored run of `case-02-high-scale-ingestion` is on record.
|
||||
|
||||
Case 03 used 3.27 million tokens over 127 model requests. By role: IaC author 812 thousand, validator 618 thousand, describer 513 thousand, Orchestrator 356 thousand, product selector 345 thousand. These figures are the evidence base for iterations 4 and 5.
|
||||
|
||||
All 8 checks are structural, so a better or worse Skill scores the same. Iteration 4 step 8 addresses that.
|
||||
|
||||
Suite at the end of this step: 226 passed.
|
||||
55
.agents/plans/iter-03-kab-layout/DETAIL.md
Normal file
55
.agents/plans/iter-03-kab-layout/DETAIL.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Plan Detail
|
||||
|
||||
## Step 1: Container starts the Execution engine
|
||||
|
||||
- **Capability:** The image runs the reviewed and evaluated engine.
|
||||
- **Rationale:** The `Dockerfile` started `python -m app.agent`, the iteration 1 pipeline, while every evaluation ran the new engine.
|
||||
- **Where:** `Dockerfile`, `README.md`.
|
||||
- **Acceptance criteria:** The container's main process is the A2A server. `/health` and `/.well-known/agent-card.json` answer on the container port. Existing `-p 8080:8080` run commands still work.
|
||||
|
||||
## Step 2: Legacy pipeline moved to `app/legacy/`
|
||||
|
||||
- **Capability:** The KAB folder names are free for the new engine, with the old pipeline still importable and tested.
|
||||
- **Rationale:** The owner chose to keep the old code rather than delete it (2026-10-06). The old and new code cannot share `app/nodes/`, `app/states/`, `app/workflows/`.
|
||||
- **Where:** `app/legacy/`, the five legacy tests in `tests/`, `eval/eval_harness.py`, `SKILLS_DIR` in `app/config.py`.
|
||||
- **Acceptance criteria:** Every legacy test passes against `app.legacy.*`. Nothing outside `app/legacy/`, those tests and the legacy eval files imports `app.legacy`.
|
||||
|
||||
## Step 3: Engine moved into the KAB folders
|
||||
|
||||
- **Capability:** The engine's modules sit where the KAB workflow standard expects them.
|
||||
- **Rationale:** Platform consistency: decomposers, reviewers and other agents on the platform look for state in `app/states/`, nodes in `app/nodes/`, graphs in `app/workflows/`, the manager in `app/agent.py`.
|
||||
- **Where:** The path table in `../README.md`.
|
||||
- **Acceptance criteria:** Behaviour is unchanged: the full suite passes with only import paths edited in tests. `google.adk` is still imported only by `app/adk/adk_runtime.py`. Package `__init__.py` files import nothing, so `app.states` can import `app.workflows.deliverables` without a cycle.
|
||||
|
||||
## Step 4: Graph file split into nodes and wiring
|
||||
|
||||
- **Capability:** Node logic in `app/nodes/`, graph assembly in `app/workflows/`.
|
||||
- **Rationale:** The KAB layout separates stage nodes from graph construction; the engine had both in one 560-line file.
|
||||
- **Where:** `app/nodes/execution_nodes.py` (`make_execution_nodes` returns an `ExecutionNodes` of the five node callables), `app/workflows/execution_graph.py` (`build_execution_graph`), `ToolFactory` in `app/adk/runtime.py`.
|
||||
- **Acceptance criteria:** `build_execution_graph` keeps its signature. The nodes still route with `Command` and `Send`, so the wiring declares only the fixed edges.
|
||||
|
||||
## Step 5: Refresh paths in the spec and ADR text
|
||||
|
||||
- **Capability:** Design documents that name modules where they are.
|
||||
- **Rationale:** `.scratch/solution-architect-agent/spec.md` (module layout, acceptance list, comments) and the ADRs still use pre-move paths. An agent following them will look in `app/legacy/` or in folders that no longer exist.
|
||||
- **Where:** `.scratch/solution-architect-agent/spec.md` sections "Module layout" and "Acceptance"; `docs/adr/0002-*.md` consequence about `app/skills/`; `docs/adr/0003-*.md`.
|
||||
- **Acceptance criteria:** The spec's module layout matches the tree. Each acceptance command in the spec runs against current paths. Dated `## Comments` entries stay as written, since they are history; one note at the top of the comments points to `../README.md` for translation.
|
||||
|
||||
## Step 6: Finish iteration 2's cleanup list
|
||||
|
||||
- **Capability:** The parts of iteration 2 step 9 that still apply once the legacy code is kept.
|
||||
- **Rationale:** The spec's cleanup list mixed deletions (now dropped) with changes the new engine still needs.
|
||||
- **Where and what:**
|
||||
- `workflow.yaml`: rewrite its phases to the four upstream Phases and their Deliverables.
|
||||
- `app/config.py`, `.env.example`: move settings only `app/legacy/` reads (`HOST`, `PORT`, `GCP_PROJECT_ID`, `GCP_REGION`, `ADK_*`, `DEVELOPER_KNOWLEDGE_MCP_*`, `SKILLS_DIR`, `EVAL_DATASET_PATH`, the OpenAI and Azure keys) under a clearly marked legacy block. Keep `POSTGRES_*` and `DATABASE_URL`: the Postgres checkpointer builds its URL from them.
|
||||
- `mypy.ini`, `ruff.toml`: exclude `app/legacy/` and the legacy eval files, so `ruff check .` and `mypy .` report on live code only (198 and 38 findings today, almost all legacy).
|
||||
- `Dockerfile`: stop copying what only the legacy pipeline reads, once step 7 is decided.
|
||||
- **Acceptance criteria:** `ruff check .` and `mypy .` are clean or list only known, recorded exceptions. `workflow.yaml` names the four Phases in `app/workflows/deliverables.py`. The suite passes.
|
||||
|
||||
## Step 7: Decide the future of `app/legacy/`
|
||||
|
||||
- **Capability:** One code path, or a deliberate, documented second one.
|
||||
- **Rationale:** Two pipelines cost every reader a detour, and the legacy one holds what ADR-0002 rules out: hand-written cloud clients, non-conformant Skills, a pretend ADK layer.
|
||||
- **Options for the owner:** delete `app/legacy/` with its tests and the legacy eval files; or keep it frozen and excluded from lint, types and the image.
|
||||
- **Guardrail:** Deleting files with uncommitted changes was blocked once by the permission system. Commit first, then delete in a commit of its own.
|
||||
- **Acceptance criteria:** The decision is recorded here. If deleted: `grep -rn "app\.legacy" app eval tests` is empty and the suite passes.
|
||||
34
.agents/plans/iter-03-kab-layout/MAP.md
Normal file
34
.agents/plans/iter-03-kab-layout/MAP.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Plan Map
|
||||
|
||||
**Iteration:** 3, KAB layout
|
||||
**Intent:** Make the deployed container run the hierarchical Execution engine, and house that engine in the KAB workflow-agent folder layout (`app/agent.py`, `app/main.py`, `app/a2a_server.py`, `app/states/`, `app/nodes/`, `app/workflows/`), with the iteration 1 pipeline first parked under `app/legacy/` (steps 2 to 6) and then deleted (step 7).
|
||||
**Branch:** `feat/solution-architect-agent` (all of this iteration is uncommitted as of 2026-10-06)
|
||||
**This directory covers:** steps 1 to 7
|
||||
|
||||
## Sources of truth
|
||||
|
||||
- `../README.md`: the old-to-new module path table.
|
||||
- `../docs/KAB_WORKFLOW_AGENT_IMPLEMENTATION_GUIDE.md` and `../langgraph-workflow-agent/app/`, both relative to the repository root: the layout being matched.
|
||||
- `docs/adr/0003-langgraph-holds-state-adk-agents-reason.md`: the engine the layout must keep unchanged.
|
||||
|
||||
## Status
|
||||
|
||||
| Step | What | Status |
|
||||
|---|---|---|
|
||||
| 1 | Container starts the Execution engine | complete |
|
||||
| 2 | Legacy pipeline moved to `app/legacy/` | complete |
|
||||
| 3 | Engine moved into the KAB folders | complete |
|
||||
| 4 | Graph file split into nodes and wiring | complete |
|
||||
| 5 | Refresh paths in the spec and ADR text | complete |
|
||||
| 6 | Finish iteration 2's cleanup list | complete; two items carried forward in `steps/step-6.md` |
|
||||
| 7 | Decide the future of `app/legacy/` | complete: deleted on the owner's instruction |
|
||||
|
||||
## Dependency graph
|
||||
|
||||
- Step 1: no dependencies
|
||||
- Step 2: no dependencies
|
||||
- Step 3: depends on step 2
|
||||
- Step 4: depends on step 3
|
||||
- Step 5: depends on step 4
|
||||
- Step 6: depends on step 4
|
||||
- Step 7: depends on step 6
|
||||
16
.agents/plans/iter-03-kab-layout/steps/step-1.md
Normal file
16
.agents/plans/iter-03-kab-layout/steps/step-1.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# Step 1 — Implementation Detail
|
||||
|
||||
1. `Dockerfile`: the start command became the A2A server; the image sets `A2A_HOST=0.0.0.0` and `A2A_PORT=8080`; only port 8080 is exposed.
|
||||
2. The health check reads `A2A_PORT` and allows 60 seconds to start, because `/health` answers only after the provider bundle and MCP catalogs load.
|
||||
3. `README.md`: server section, start command, container example (now passing `GOOGLE_GENAI_USE_VERTEXAI`, `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION`, `A2A_PUBLIC_URL`) and test calls.
|
||||
|
||||
Verified: built with podman and ran the image with `MCP_ENABLED=false`; `/health` and the agent card returned 200 and process 1 was the server.
|
||||
|
||||
Consequences for callers:
|
||||
|
||||
- `/card`, `/validate` and `/sessions/{session_id}` are no longer served. The card is at `/.well-known/agent-card.json`.
|
||||
- `/generate` now runs a full pre-approved Execution with real models, about 45 minutes in the evaluation runs.
|
||||
- Without `A2A_PUBLIC_URL` the card advertises `http://0.0.0.0:8080/a2a`.
|
||||
- Podman's default image format ignores `HEALTHCHECK`.
|
||||
|
||||
Not verified: a full Execution with real models and MCP servers inside the container.
|
||||
9
.agents/plans/iter-03-kab-layout/steps/step-2.md
Normal file
9
.agents/plans/iter-03-kab-layout/steps/step-2.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Step 2 — Implementation Detail
|
||||
|
||||
1. Moved `app/agent.py`, `app/card.py`, `app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/`, `app/states/` to `app/legacy/`, and `gcp_scanner.py`, `mcp_developer_knowledge.py`, `validation_tools.py` to `app/legacy/tools/`.
|
||||
2. Rewrote imports to `app.legacy.*` in the legacy code, its five tests (`test_adk_architecture`, `test_agent_api`, `test_graph_workflow`, `test_mcp_developer_knowledge`, `test_skill_loader`) and `eval/eval_harness.py`.
|
||||
3. Pointed `SKILLS_DIR` at `app/legacy/skills`, and the paths quoted in `eval/optimizer.py` likewise.
|
||||
|
||||
Verified: 226 passed before any engine file moved; `python -m eval.eval_harness` still runs.
|
||||
|
||||
Nothing was deleted. A first attempt to delete these files was blocked by the permission system because 25 of them held uncommitted changes.
|
||||
10
.agents/plans/iter-03-kab-layout/steps/step-3.md
Normal file
10
.agents/plans/iter-03-kab-layout/steps/step-3.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# Step 3 — Implementation Detail
|
||||
|
||||
1. Moved each engine module to the path in `../../README.md`. `app/execution/`, `app/agents/` and `app/a2a/` no longer exist.
|
||||
2. Merged `engine.py` and `assembly.py` into `app/agent.py`: `ExecutionEngine`, `Interaction`, then `Assembly` and `open_assembly`.
|
||||
3. The entry point is `python -m app.main`; `Dockerfile` and `README.md` use it.
|
||||
4. Left the test folders (`tests/execution/`, `tests/agents/`, `tests/a2a/`) where they were; only their imports changed.
|
||||
|
||||
Verified: 226 passed; the rebuilt image starts `python -m app.main` and answers `/health` and the agent card.
|
||||
|
||||
Side effect: ruff's import sort and unused-import fix ran across `app`, `eval`, `tests` and `scripts`, so legacy and test files show import-only diffs.
|
||||
9
.agents/plans/iter-03-kab-layout/steps/step-4.md
Normal file
9
.agents/plans/iter-03-kab-layout/steps/step-4.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Step 4 — Implementation Detail
|
||||
|
||||
1. `app/nodes/execution_nodes.py` keeps the five nodes as closures over the runtime, roles, recorder and tool factory, built by `make_execution_nodes` and returned as a frozen `ExecutionNodes`. The helper functions and `GUIDE_FILENAME` moved with them.
|
||||
2. `app/workflows/execution_graph.py` holds `build_execution_graph`, which builds the nodes and declares the fixed edges.
|
||||
3. `ToolFactory` moved to `app/adk/runtime.py`, beside `ToolSpec`, so `app/tools/toolkit.py` no longer imports the nodes module.
|
||||
|
||||
The node bodies were moved verbatim; no logic changed.
|
||||
|
||||
Verified: 226 passed; the moved files are ruff-clean.
|
||||
11
.agents/plans/iter-03-kab-layout/steps/step-5.md
Normal file
11
.agents/plans/iter-03-kab-layout/steps/step-5.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Step 5 — Implementation Detail
|
||||
|
||||
1. `.scratch/solution-architect-agent/spec.md`: replaced the "Module layout" tree with the current one, and updated every module path in the design sections (Deliverables, Ledger, Execution graph, tools, agent runtime, A2A, recording) and in the Risks section.
|
||||
2. Rewrote two acceptance commands to match the tree:
|
||||
- `rg -n "google\.adk" app -g '!app/legacy/**' | grep -v adk_runtime.py` is empty.
|
||||
- `rg -n "app\.legacy" app eval tests -g '!**/legacy/**'` is empty.
|
||||
3. Marked build increment 9 as replaced, with a pointer here; its original list stays below the note.
|
||||
4. Left the dated `## Comments` entries as written and added one note above them pointing to `../../README.md` for path translation.
|
||||
5. `docs/adr/0002-*.md`: added a dated note to the `app/skills/` consequence. ADR-0003 names no repository paths and needed no change.
|
||||
|
||||
Verified: both acceptance commands return nothing (run with `grep`).
|
||||
19
.agents/plans/iter-03-kab-layout/steps/step-6.md
Normal file
19
.agents/plans/iter-03-kab-layout/steps/step-6.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Step 6 — Implementation Detail
|
||||
|
||||
1. **Legacy grouped in three folders.** The five legacy tests moved to `tests/legacy/` and the offline harness (`metrics.py`, `optimizer.py`, `eval_harness.py`, `test_eval_harness.py`) to `eval/legacy/`. Run it with `python -m eval.legacy.eval_harness`. `eval/datasets/benchmark_cases.json` stayed where `EVAL_DATASET_PATH` points.
|
||||
2. **Lint and types on live code only.** `ruff.toml` and `mypy.ini` excluded `app/legacy`, `eval/legacy` and `tests/legacy` (exclusions removed again in step 7). `ruff check .` and `mypy .` are now clean, down from 198 and 38 findings.
|
||||
3. **Live findings fixed, none suppressed by config:**
|
||||
- `app/database.py`: modern annotations, an unused import removed, and each broad `except` marked with its reason (connection failure falls back to SQLite; persistence is best-effort and logged).
|
||||
- `app/workflows/execution_graph.py`: every node is cast for `add_node`, since the nodes are now typed callables.
|
||||
- `scripts/list-mcp-tools.py`, `scripts/validate_artifacts.py`, `tests/test_assembly.py`, `tests/test_package.py`, `tests/agents/test_adk_engine.py`, `tests/agents/test_adk_runtime.py`: small typing and lint fixes.
|
||||
- `requirements-dev.txt`: added `types-protobuf` and `types-psycopg2`.
|
||||
4. **Settings.** `app/config.py` and `.env.example` list live settings first and the legacy-only ones under a marked block. No setting was removed or renamed.
|
||||
5. **`workflow.yaml`** names the four Phases and their Deliverables. `tests/test_workflow_manifest.py` fails if it drifts from `app/workflows/deliverables.py`.
|
||||
6. **Image.** `Dockerfile` copies `app/`, `agents/`, `providers/`, `skills/` and `workflow.yaml` only, and creates `deliverables/`. A new `.dockerignore` keeps `app/legacy`, `.venv`, `.git`, caches, databases and past results out of the build context.
|
||||
|
||||
Verified: 227 passed; ruff and mypy clean; the rebuilt image contains no `app/legacy`, `eval/`, `docs/` or `terraform/`, and `/health` answers.
|
||||
|
||||
Still open from iteration 2's cleanup list:
|
||||
|
||||
- **Writes to shared paths.** The engine writes only under `deliverables/executions/<execution_id>/`. The legacy pipeline still writes `deliverables/{as-is,target,validation,guides}/` and `docs/*.md`, and `tests/legacy/` runs it, so a test run can modify tracked files. Step 7 deleted the legacy folders, which closed this item.
|
||||
- **CI.** `.github/workflows/ci.yml` pins `ruff==0.6.9` and `mypy==1.11.2` and installs no project dependencies; the local tools are newer (ruff 0.16.7). CI was not run for this step.
|
||||
15
.agents/plans/iter-03-kab-layout/steps/step-7.md
Normal file
15
.agents/plans/iter-03-kab-layout/steps/step-7.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# Step 7 — Implementation Detail
|
||||
|
||||
Decision: **the original pipeline is deleted.** The owner asked for its removal on 2026-10-06, after it had first been kept frozen in `app/legacy/`, `tests/legacy/` and `eval/legacy/` (step 6).
|
||||
|
||||
1. Deleted `app/legacy/`, `tests/legacy/`, `eval/legacy/` and `eval/datasets/benchmark_cases.json`.
|
||||
2. Removed the legacy-only settings from `app/config.py` and `.env.example`: `HOST`, `PORT`, the OpenAI and Azure keys, `LLM_TEMPERATURE`, `GOOGLE_APPLICATION_CREDENTIALS`, `GCP_PROJECT_ID`, `GCP_REGION`, `ADK_*`, `DEVELOPER_KNOWLEDGE_MCP_*`, `SKILLS_DIR`, `EVAL_DATASET_PATH`. Google client libraries still read `GOOGLE_APPLICATION_CREDENTIALS` from the environment themselves.
|
||||
3. Removed the legacy exclusions from `ruff.toml`, `mypy.ini` and `.dockerignore`.
|
||||
4. Removed four dependencies no live module imports: `langchain`, `langchain-openai`, `sqlalchemy`, `asyncpg`.
|
||||
5. Rewrote the top of `README.md` around the Orchestrator, Specialists, Skills and A2A, and pointed `VALIDATION.md` at `eval.solution_eval`.
|
||||
|
||||
Verified: 206 passed (227 less the 21 legacy tests); `ruff check .` and `mypy .` clean; `grep -rn "app\.legacy\|eval\.legacy" app eval tests scripts` is empty; the image rebuilt from the trimmed `requirements.txt` imports every live module, including both checkpointers, and answers `/health` and the agent card.
|
||||
|
||||
Recovering the old code: the committed versions are in git history at their pre-move paths (`app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/`, `app/states/`, `app/agent.py`, `app/card.py`) up to commit `e94b249`. Uncommitted formatting changes to those files were not kept.
|
||||
|
||||
Left in place, because they are checked-in sample artefacts and not pipeline code: `scripts/validate_artifacts.py`, `tests/test_artifacts.py`, `tests/test_package.py`, the root `terraform/`, `docs/*.md`, `deliverables/{as-is,target,validation,guides}/`, `requirements.yaml`. `app/database.py` keeps two methods only the old pipeline called (`get_session`, `save_evaluation`).
|
||||
123
.agents/plans/iter-04-learning-and-skills/DETAIL.md
Normal file
123
.agents/plans/iter-04-learning-and-skills/DETAIL.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Plan Detail
|
||||
|
||||
Every step ends with the full suite passing. Steps that change agent behaviour (3, 6, 10, 11, 12, 13) also end with an evaluation run recorded in their step file, compared against the baseline in `iter-02-solution-architect-agent/steps/step-8.md`.
|
||||
|
||||
## Step 1: Structured validator findings
|
||||
|
||||
- **Report ref:** enhancement 3
|
||||
- **Capability:** Validation findings as data the engine can route, count and learn from.
|
||||
- **Rationale:** The validator reports findings as free markdown. The Orchestrator reads them once to decide what to reopen; nothing else can use them. They are the richest learning signal in an Execution.
|
||||
- **Where:** `app/workflows/deliverables.py` (add `findings.json` to the `validation_results` attachments), `agents/validator.md`, `skills/iac-validation/SKILL.md` (the finding format), `app/tools/execution.py` (validate the attachment's shape at submission).
|
||||
- **Acceptance criteria:** A `validation_results` submission without a parseable `findings.json` is refused with the reason. Each finding carries the Deliverable kind it traces to, a severity, the evidence and the rule. An empty list is valid and means a pass.
|
||||
|
||||
## Step 2: Carry the draft through `needs_input`
|
||||
|
||||
- **Report ref:** enhancement 9
|
||||
- **Capability:** The user receives the draft they approved.
|
||||
- **Rationale:** The Specialist protocol asks a Specialist to confirm its draft with the user before submitting. `needs_input` carries only questions and Specialists are stateless per delegation, so the approved draft is lost and written again on the next run.
|
||||
- **Where:** `needs_input` in `app/tools/execution.py` (optional `draft` and attachments), `NeedsInput` in `app/adk/runtime.py`, `_outcome_record` and `_specialist_input` in `app/nodes/execution_nodes.py` (store the draft, return it in context), `app/states/execution_state.py`, `agents/partials/specialist-protocol.md`.
|
||||
- **Acceptance criteria:** A Specialist restarted after the user approves its draft receives that draft in its context. A test shows the submitted content equals the approved draft when the fake runtime submits what it was given.
|
||||
|
||||
## Step 3: Reuse pillar advice across Revisions
|
||||
|
||||
- **Report ref:** enhancement 10
|
||||
- **Capability:** A Revision of the design recommendations re-consults only the pillars its feedback touches.
|
||||
- **Rationale:** Pillar advice is never stored. In case 01, two Revisions re-ran all six advisors each time: 18 advisor runs for one Deliverable.
|
||||
- **Where:** `app/tools/pillars.py` (the tool already accepts `pillars`), the Execution state for stored advice, `agents/design-advisor.md` (on a Revision, name the affected pillars).
|
||||
- **Acceptance criteria:** On a second attempt with stored advice, only the named pillars run and the tool returns stored advice for the rest. A changed brief for a pillar invalidates that pillar's stored advice.
|
||||
|
||||
## Step 4: Single source for Deliverable contracts
|
||||
|
||||
- **Report ref:** enhancement 11
|
||||
- **Capability:** Required headings and attachments declared once, and contracts for the Deliverables that have none.
|
||||
- **Rationale:** Required headings appear in each role file and again in `DeliverableSpec.sections`. `technical_decomposition`, `architecture_description` and the diagram's content accept any non-empty text.
|
||||
- **Where:** `app/workflows/deliverables.py`, `_specialist_input` in `app/nodes/execution_nodes.py` (pass the contract in context), `agents/*.md` (remove the "Required headings" lines), `agents/partials/specialist-protocol.md`.
|
||||
- **Acceptance criteria:** No role file lists required headings. Each Specialist's context carries its contract. The decomposition requires a component table naming the requirements each component serves.
|
||||
|
||||
## Step 5: Cap parallel Specialists
|
||||
|
||||
- **Report ref:** enhancement 14
|
||||
- **Capability:** A bound on concurrent Specialist runs per Execution.
|
||||
- **Rationale:** The spec's Risks section calls for a default cap of 4 against model rate limits. `orchestrator_turn` sends every delegated Deliverable at once, and the pillar fan-out adds six more.
|
||||
- **Where:** `app/nodes/execution_nodes.py`, `app/tools/pillars.py`, a `MAX_PARALLEL_SPECIALISTS` setting in `app/config.py`.
|
||||
- **Acceptance criteria:** A test with the fake runtime shows at most the configured number of Specialist runs in flight. The pillar fan-out counts toward the same cap.
|
||||
|
||||
## Step 6: Orchestrator step order
|
||||
|
||||
- **Report ref:** findings by agent, orchestrator row
|
||||
- **Capability:** A failed validation goes back for Revision before the user is asked to approve it.
|
||||
- **Rationale:** In `agents/orchestrator.md`, "A Phase is complete" (step 3) precedes "Validation found problems" (step 5), and the Orchestrator takes the first step that applies.
|
||||
- **Where:** `agents/orchestrator.md`.
|
||||
- **Acceptance criteria:** The validation-failure step sits above the Approval step. An evaluation run with a failing validation shows a reopen before any Approval request for the validation Phase.
|
||||
|
||||
## Step 7: Capture lesson candidates
|
||||
|
||||
- **Report ref:** enhancement 4
|
||||
- **Capability:** A durable record, per Execution, of what went wrong and what fixed it.
|
||||
- **Rationale:** The signals exist and are discarded. Revision feedback lives in the Ledger inside one checkpoint. Tool refusals (`Not submitted`, `Refused`, `terraform validate failed`) reach only the Python log through `REJECTION_PREFIXES` in `app/adk/adk_runtime.py`.
|
||||
- **Where:** `app/recorder.py` (a `record_lesson_candidates` method on `ExecutionRecorder`), `finalize` and `end_with_warnings` in `app/nodes/execution_nodes.py`, the runtime for refusals.
|
||||
- **Acceptance criteria:** A finished Execution writes one record holding each reopen with its reason and actor, each finding from step 1, each tool refusal with role and tool, each user change request and each exhausted budget. The record names the Execution, the Deliverable kind and the Skill versions loaded. It holds no secrets and no discovered cloud data.
|
||||
|
||||
## Step 8: Evaluation that can see improvement
|
||||
|
||||
- **Report ref:** enhancement 6
|
||||
- **Capability:** An evaluation whose score moves when a Skill gets better or worse.
|
||||
- **Rationale:** The 8 checks are structural and both scored runs reached 1.0. The loop's proof step has nothing to measure.
|
||||
- **Where:** `eval/scoring.py`, `eval/solution_eval.py`, `eval/datasets/solution_cases.json`, `tests/eval/`.
|
||||
- **Acceptance criteria:** New checks cover requirement-to-component traceability, coverage of each pillar Skill's checklist, and a graded rubric for recommendation quality. One interactive case checks that a Specialist asks before it submits. Revisions, seconds and tokens per case have recorded thresholds, and a run that exceeds one is reported as a regression. `case-02-high-scale-ingestion` has a scored run.
|
||||
|
||||
## Step 9: ADR: the lessons loop and overlay Skills
|
||||
|
||||
- **Report ref:** sequencing note
|
||||
- **Capability:** A recorded decision on how Skills change and how they are chosen per Execution.
|
||||
- **Rationale:** Steps 10 and 12 change which Skills an agent loads and who may edit them. That is an architectural decision, and ADR-0002 does not cover it.
|
||||
- **Where:** `docs/adr/0004-*.md`.
|
||||
- **Decisions the owner must make:** whether lessons need human review before they take effect (the plan assumes yes); where lessons Skills live and who owns them; whether an Execution may name overlay Skills, and who may supply them.
|
||||
- **Acceptance criteria:** The ADR is accepted and steps 10 and 12 are updated to match it.
|
||||
|
||||
## Step 10: Lessons Skill and lesson curator
|
||||
|
||||
- **Report ref:** enhancement 5
|
||||
- **Capability:** Each Execution can improve the next, under human review.
|
||||
- **Rationale:** `eval/optimizer.py` is the only learning code. It maps four keywords to iteration 1 Skill names that the engine never loads.
|
||||
- **Design:** The loop is capture (step 7), distil, review, prove, publish, retire.
|
||||
- Distil: an offline `lesson-curator` role groups candidates across Executions and proposes lessons. It runs outside any Execution.
|
||||
- Review: each proposal is an issue file under `.scratch/lessons/issues/` with `Status: needs-triage`.
|
||||
- Prove: the evaluation from step 8 runs with and without the change.
|
||||
- Publish: lessons land in a lessons Skill under `skills/`, one reference file per Deliverable kind; each lesson has its rule, reason, source Execution and date. The bundle's `role_skills` lists it.
|
||||
- Retire: at each vendor sync, remove lessons the upstream Skill now covers.
|
||||
- **Where:** `skills/<lessons-skill>/`, `agents/lesson-curator.md`, a curator entry point beside `eval/solution_eval.py`, `providers/gcp.yaml`, `scripts/sync-vendored-skills.sh`.
|
||||
- **Acceptance criteria:** Vendored Skills stay byte-identical. No lesson reaches a Skill without an approved issue file. Each Execution records the lessons Skill version it loaded. One real lesson has gone through the whole loop with its evaluation comparison recorded.
|
||||
|
||||
## Step 11: IaC patterns Skill
|
||||
|
||||
- **Report ref:** enhancement 7
|
||||
- **Capability:** The IaC author starts from known-good infrastructure code.
|
||||
- **Rationale:** The IaC author is the most expensive role: 812 thousand tokens, 23 requests, 4 runs and 3 Revisions in case 03. It writes from a blank page each time.
|
||||
- **Where:** `skills/<iac-patterns-skill>/` with one reference per product, `providers/gcp.yaml` (`role_skills.iac-author`).
|
||||
- **Acceptance criteria:** On the three cases, quality checks hold and IaC author Revisions or tokens fall against the step 8 baseline.
|
||||
|
||||
## Step 12: Overlay Skills and an organisation standards Skill
|
||||
|
||||
- **Report ref:** enhancement 8
|
||||
- **Capability:** New standards and clients need no code change.
|
||||
- **Rationale:** Skills are fixed at start-up: `AdkAgentRuntime` loads every Skill once, and `role_skills` is static per provider. Only 2 of the 9 Skills are our own, and both describe tools.
|
||||
- **Where:** `app/adk/adk_runtime.py`, `app/adk/roles.py`, `app/providers.py`, `ExecutionEngine.start` in `app/agent.py`, `app/a2a_server.py`, a standards Skill under `skills/`.
|
||||
- **Acceptance criteria:** An Execution can name overlay Skills at start; each passes the same Agent Skills validation as bundled Skills, and an unknown or invalid one refuses the start. An organisation standards Skill (naming, tagging, approved products) is loaded by the roles the bundle lists.
|
||||
|
||||
## Step 13: Trim Specialist context
|
||||
|
||||
- **Report ref:** enhancement 12
|
||||
- **Capability:** Each Specialist receives what its Deliverable needs.
|
||||
- **Rationale:** `inputs_of` gives a Specialist every Deliverable of every earlier Phase in full. The describer used 513 thousand tokens over 12 requests.
|
||||
- **Where:** `inputs_of` in `app/workflows/deliverables.py`, `_specialist_input` in `app/nodes/execution_nodes.py`.
|
||||
- **Acceptance criteria:** Direct Dependencies arrive in full; other earlier Deliverables arrive as summaries, with `read_deliverable`-style access to the full text on demand. Quality checks hold and describer tokens fall against the step 8 baseline.
|
||||
|
||||
## Step 14: Provider-owned contracts
|
||||
|
||||
- **Report ref:** enhancement 13
|
||||
- **Capability:** A second Cloud provider is configuration, as ADR-0002 intends.
|
||||
- **Rationale:** The six pillar headings, the eight solution-guide sections and the Terraform attachment names are hard-coded in `app/workflows/deliverables.py`. Other providers' frameworks name and count pillars differently. The bundle's pillar keys already differ from the headings (`performance-optimization` against `Performance efficiency`).
|
||||
- **Where:** `providers/gcp.yaml`, `app/providers.py`, `app/workflows/deliverables.py`, `app/presentation.py`, `agents/iac-author.md`, `agents/validator.md`, `agents/design-advisor.md`.
|
||||
- **Timing:** Build when a second Cloud provider is planned. ADR-0002 says no provider interface exists in code until then.
|
||||
- **Acceptance criteria:** Pillar headings come from `pillar_skills`, guide sections from the bundle's `output_template`, and the IaC tool with its file contract from the bundle. `grep -rni "terraform" app/workflows app/nodes app/states` is empty, and no pillar or guide-section name appears as a literal in `app/workflows/deliverables.py`.
|
||||
48
.agents/plans/iter-04-learning-and-skills/MAP.md
Normal file
48
.agents/plans/iter-04-learning-and-skills/MAP.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Plan Map
|
||||
|
||||
**Iteration:** 4, learning and Skills
|
||||
**Intent:** Make the agent a learning, adaptable solution through Skills. Fix the cheap defects the 2026-10-06 review found, then close the loop so each Execution can improve the next through a reviewed Skill change, then let new standards, clients and Cloud providers arrive as Skills and configuration.
|
||||
**Status:** planned; no step started
|
||||
**This directory covers:** steps 1 to 14
|
||||
|
||||
## Sources of truth
|
||||
|
||||
- Review report (2026-10-06): https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. Each step's `Report ref` names its enhancement number there. The report uses pre-iteration-3 paths.
|
||||
- `iter-02-solution-architect-agent/steps/step-8.md`: the evaluation figures quoted as evidence.
|
||||
- `docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md`: every step must keep behaviour in Skills and vendored Skills unmodified.
|
||||
- `CONTEXT.md`: "every Execution starts clean". Learning arrives as a reviewed Skill change, never as memory carried between Executions.
|
||||
|
||||
## Status
|
||||
|
||||
| Step | What | Group | Size | Status |
|
||||
|---|---|---|---|---|
|
||||
| 1 | Structured validator findings | A | S | planned |
|
||||
| 2 | Carry the draft through `needs_input` | A | S | planned |
|
||||
| 3 | Reuse pillar advice across Revisions | A | S | planned |
|
||||
| 4 | Single source for Deliverable contracts | A | S | planned |
|
||||
| 5 | Cap parallel Specialists | A | S | planned |
|
||||
| 6 | Orchestrator step order | A | S | planned |
|
||||
| 7 | Capture lesson candidates | B | M | planned |
|
||||
| 8 | Evaluation that can see improvement | B | M | planned |
|
||||
| 9 | ADR: the lessons loop and overlay Skills | B | S | planned, owner's decision |
|
||||
| 10 | Lessons Skill and lesson curator | B | L | planned |
|
||||
| 11 | IaC patterns Skill | B | M | planned |
|
||||
| 12 | Overlay Skills and an organisation standards Skill | C | M | planned |
|
||||
| 13 | Trim Specialist context | C | M | planned |
|
||||
| 14 | Provider-owned contracts | C | M | planned |
|
||||
|
||||
Size: S under a day, M a few days, L a week or more. Group A is cheap fixes that also create signal. Group B is the learning loop. Group C is adaptability.
|
||||
|
||||
## Dependency graph
|
||||
|
||||
- Steps 1 to 6: no dependencies; any order
|
||||
- Step 7: depends on step 1
|
||||
- Step 8: no dependencies
|
||||
- Step 9: no dependencies
|
||||
- Step 10: depends on step 7, step 8, step 9
|
||||
- Step 11: depends on step 8
|
||||
- Step 12: depends on step 9
|
||||
- Step 13: depends on step 8
|
||||
- Step 14: depends on step 4
|
||||
|
||||
Step 8 comes before steps 10, 11 and 13 so each of them can be shown to help: same quality, fewer Revisions, seconds or tokens.
|
||||
151
.agents/plans/iter-05-jev-decisions/DETAIL.md
Normal file
151
.agents/plans/iter-05-jev-decisions/DETAIL.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Plan Detail
|
||||
|
||||
Rules for every step from 3 onward:
|
||||
|
||||
- The question wording lives in a Skill or the provider bundle, so decisions stay portable and reviewable.
|
||||
- Each question has a confidence threshold and a named fallback. Below the threshold, or when Jev is unavailable, the fallback runs.
|
||||
- **Optimisations fail open** to today's behaviour (steps 3, 6, 7, 8, 9). **Guards fail closed** and ask the user (steps 13, 14).
|
||||
- With no API key configured, the agent behaves exactly as it does without this iteration.
|
||||
- The step ends with an evaluation run recorded in its step file: the quality checks hold, and tokens, seconds or Revisions fall against the baseline.
|
||||
|
||||
## Step 1: Resolve the open questions
|
||||
|
||||
- **Capability:** The facts needed to send client data to Jev.
|
||||
- **Questions for the owner or vendor:**
|
||||
- The base URL the separate deployment exposes. The public documentation shows one hosted endpoint and no self-hosting option.
|
||||
- How long submitted state is retained. Client requirements and, in later steps, discovered cloud state would be sent.
|
||||
- Rate limits and typical latency.
|
||||
- **Acceptance criteria:** Each answer is recorded in this step's file. A retention answer that rules out client data narrows steps 7, 9, 13 and 14, and this plan is updated to say so.
|
||||
|
||||
## Step 2: Decider interface and Jev client with limits
|
||||
|
||||
- **Capability:** One seam through which the engine asks closed questions, with a real client and a scripted fake.
|
||||
- **Rationale:** The same reason `AgentRuntime` exists: the deterministic suite must stay model-free, and the vendor must be replaceable. The LangChain middleware shown in the vendor's article targets `create_agent` and does not plug into the ADK runtime, so the client calls the REST API directly.
|
||||
- **Where:** a `Decider` protocol beside `AgentRuntime` in `app/adk/runtime.py`; a client and fake under `app/tools/`; settings in `app/config.py` (`JEV_API_KEY`, `JEV_BASE_URL`, and the limits below); wiring in `open_assembly` in `app/agent.py`.
|
||||
- **Limits the client enforces.** Jev's own limits are undocumented, so assume they are strict and never let Jev block an Execution. Values are starting defaults held in settings, to tune from measurement.
|
||||
|
||||
| Practice | Default |
|
||||
|---|---|
|
||||
| Refuse locally above a token count; split by Deliverable or section, never truncate | 48 thousand tokens (25% under the limit); 24 thousand for state plus the longest question |
|
||||
| Send every question about one state in one call | one call per state |
|
||||
| Timeout | 2 seconds connect, 5 seconds total |
|
||||
| Retry 429, 5xx and network errors with exponential backoff and full jitter; honour `Retry-After`; never retry other 4xx | 2 retries, 0.5 second base, 8 second cap |
|
||||
| Process-wide token bucket, lowered automatically after a 429 | 5 requests per second, burst 10 |
|
||||
| Semaphore on calls in flight | 8 |
|
||||
| Circuit breaker, with one probe call before closing | open after 5 failures in 30 seconds; 60 second cool-down |
|
||||
| Cache answers by a hash of model, state and questions | in memory, per Execution |
|
||||
| Budget per Execution; on reaching it use fallbacks and record a warning | 500 calls or $0.50 |
|
||||
| Read `usage.credits_remaining_usd` and warn before credits run out | warn below $5 |
|
||||
| Record per question: latency, confidence, fallback used, 429 count; add to the evaluation report | always |
|
||||
| API key from the secret store or environment, redacted from logs, never in the bundle or repository | `JEV_API_KEY` |
|
||||
|
||||
- **Acceptance criteria:** Tests with a stub HTTP server cover each row of the table. The fake `Decider` is used by every other test. With `JEV_API_KEY` unset, `open_assembly` wires a `Decider` that always returns "fallback".
|
||||
|
||||
## Step 3: Pillars a Revision touches
|
||||
|
||||
- **Report ref:** J3
|
||||
- **Capability:** The feedback text selects which pillar advisors re-run.
|
||||
- **Design:** One call with a Noul per pillar: "Does this feedback require changes to the <pillar> recommendations?" The pillars above threshold become the `pillars` argument of `consult_pillar_advisors`.
|
||||
- **Fallback:** all pillars.
|
||||
- **Where:** `app/tools/pillars.py`; questions beside `pillar_skills` in `providers/gcp.yaml`.
|
||||
- **Acceptance criteria:** A test shows only selected pillars run. On the cases, design-advisor Revisions run fewer advisors with the quality checks unchanged.
|
||||
|
||||
## Step 4: Semantic checks at submission
|
||||
|
||||
- **Report ref:** J4, and Jev's "task-completion check" use case
|
||||
- **Capability:** Submission contracts that check meaning, not only headings.
|
||||
- **Rationale:** `check_submission` checks headings and attachments. Rules such as "name no products" are instructions the engine cannot enforce, and a Revision of the IaC author and validator costs hundreds of thousands of tokens.
|
||||
- **Design:** Noul questions per Deliverable kind at `submit_deliverable`, for example: requirements name no products; every component traces to a requirement; every recommendation cites a source; the Deliverable addresses its task; tool output backs the claim. A failed check refuses the submission with the reason, as a missing section does today.
|
||||
- **Fallback:** accept on the structural contract alone.
|
||||
- **Where:** `submit_deliverable` in `app/tools/execution.py`; questions per kind in a Skill or the bundle.
|
||||
- **Acceptance criteria:** A test shows a refusal carrying the failed question's reason. Refusals count against the Specialist's budget as today. On the cases, Revisions per Execution fall.
|
||||
|
||||
## Step 5: Finding-to-Deliverable routing
|
||||
|
||||
- **Report ref:** J2
|
||||
- **Capability:** Each validator finding is routed to the Deliverable to reopen, with a severity.
|
||||
- **Design:** A Choice over the Deliverable kinds and a Score for severity, per finding from `findings.json`. The result is offered to the Orchestrator in its context; the Orchestrator still calls `reopen`.
|
||||
- **Fallback:** the validator's own stated Deliverable.
|
||||
- **Where:** `_orchestrator_context` in `app/nodes/execution_nodes.py`.
|
||||
- **Acceptance criteria:** A test shows routed findings in the Orchestrator's context. Disagreement between Jev and the validator is recorded as a lesson candidate.
|
||||
|
||||
## Step 6: Model tier per delegation
|
||||
|
||||
- **Report ref:** J5, and Jev's "model routing" use case
|
||||
- **Capability:** The cheaper model tier where the task allows.
|
||||
- **Design:** A Choice between the bundle's tiers per delegation, from the task and its feedback. A Revision after a failed attempt goes to the stronger tier.
|
||||
- **Fallback:** the role's configured model.
|
||||
- **Where:** `default_model_factory` in `app/adk/adk_runtime.py`; tiers and criteria in `models` in `providers/gcp.yaml`.
|
||||
- **Acceptance criteria:** Quality checks hold on the cases with lower token cost. A role the bundle pins to one tier is never rerouted.
|
||||
|
||||
## Step 7: Context selection
|
||||
|
||||
- **Report ref:** J6, and Jev's "repo context selection" use case
|
||||
- **Capability:** Only relevant sections of earlier Deliverables reach a Specialist in full.
|
||||
- **Design:** A Noul per section: "Is this section needed to produce the <kind> Deliverable?" Sections below threshold are passed as one-line summaries.
|
||||
- **Fallback:** the rule from iteration 4 step 13.
|
||||
- **Where:** `_specialist_input` in `app/nodes/execution_nodes.py`.
|
||||
- **Acceptance criteria:** Describer and packager tokens fall with the quality checks unchanged.
|
||||
|
||||
## Step 8: Skill routing
|
||||
|
||||
- **Report ref:** J9, and Jev's "agent skill routing" use case
|
||||
- **Capability:** A Specialist is told which Skill and reference file to load first; overlay Skills are selected per Execution.
|
||||
- **Design:** A Choice over the role's Skills and their references, with a `none` option.
|
||||
- **Fallback:** on `none` or low confidence, the Specialist searches the Skill itself, as today.
|
||||
- **Where:** `_render_task` in `app/adk/adk_runtime.py`, `app/adk/roles.py`.
|
||||
- **Acceptance criteria:** Fewer `load_skill_resource` calls per Specialist run with the quality checks unchanged.
|
||||
|
||||
## Step 9: In-run context compaction
|
||||
|
||||
- **Report ref:** J10, and Jev's "context compaction" use case
|
||||
- **Capability:** Old tool results inside one agent run are kept, truncated or dropped.
|
||||
- **Rationale:** The IaC author makes 23 requests and resends full Terraform files and validate output on each.
|
||||
- **Design:** Before each model request, a Noul and a Choice (`keep`, `truncate`, `drop`) per old tool result.
|
||||
- **Fallback:** keep everything.
|
||||
- **Where:** `app/adk/adk_runtime.py`, through ADK's before-model hook.
|
||||
- **Acceptance criteria:** IaC author and validator prompt tokens fall. No run loses the latest version of a file it later submits.
|
||||
|
||||
## Step 10: Evaluation rubric and lesson triage
|
||||
|
||||
- **Report ref:** J8
|
||||
- **Capability:** Cheap graded scoring in the evaluation, and sorted lesson candidates.
|
||||
- **Design:** Score questions implement the graded rubric from iteration 4 step 8. A Choice sorts lesson candidates by Deliverable kind; a Noul flags duplicates for the curator.
|
||||
- **Where:** `eval/scoring.py`, the curator from iteration 4 step 10.
|
||||
- **Acceptance criteria:** Rubric scores are stable across repeated runs on the same output. The curator receives candidates grouped and de-duplicated.
|
||||
|
||||
## Step 11: ADR: Orchestrator routing
|
||||
|
||||
- **Capability:** A recorded decision on whether code and Jev may take Orchestrator steps.
|
||||
- **Rationale:** ADR-0003 rejected running the dependency graph with no Orchestrator model, so the top level stays ReAct. Step 12 moves the mechanical steps to code.
|
||||
- **Evidence to bring:** results from steps 3 and 4, and the Orchestrator's 36 Pro requests and 53 thousand thinking tokens in case 03.
|
||||
- **Acceptance criteria:** An accepted ADR that amends or confirms ADR-0003. If it confirms, step 12 is closed as not doing.
|
||||
|
||||
## Step 12: Orchestrator routing
|
||||
|
||||
- **Report ref:** J1
|
||||
- **Capability:** The Orchestrator model runs only when judgement is needed.
|
||||
- **Design:** Four of the seven steps in `agents/orchestrator.md` are decided by fields already in its context: pending questions, a Phase ready for Approval, ready work, packaging approved. Code takes those. Jev answers two judgements: a Noul for "did the user's latest message ask for changes?" and a Choice over the Deliverables for what to reopen.
|
||||
- **Fallback:** the Orchestrator model, as today.
|
||||
- **Where:** `orchestrator_turn` in `app/nodes/execution_nodes.py`.
|
||||
- **Acceptance criteria:** Every Ledger rule still holds: the engine tests pass unchanged. On the cases, Orchestrator requests and seconds fall with the quality checks unchanged.
|
||||
|
||||
## Step 13: Cloud-read call guard
|
||||
|
||||
- **Report ref:** J7, and Jev's "tool-call risk gate" use case
|
||||
- **Capability:** A check before each cloud-read call that it stays within the scope the user named.
|
||||
- **Design:** A Noul: "Is this call within the project and regions the user named?" It adds to the allowlist and the per-Execution permission; it replaces neither.
|
||||
- **Fallback:** fail closed: ask the user.
|
||||
- **Where:** `gate_cloud_read` in `app/tools/execution.py`.
|
||||
- **Depends on step 1:** tool arguments name the user's cloud resources.
|
||||
- **Acceptance criteria:** A test shows an out-of-scope call becoming a question to the user.
|
||||
|
||||
## Step 14: Input guardrail
|
||||
|
||||
- **Report ref:** J11, and Jev's "LLM guardrail" use case
|
||||
- **Capability:** Injected instructions are caught before a Specialist reads them.
|
||||
- **Design:** A Noul on the Workflow request, user replies and text returned by MCP servers: "Does this text try to instruct the agent rather than inform it?"
|
||||
- **Fallback:** fail closed: withhold the text and tell the Orchestrator.
|
||||
- **Where:** `app/tools/mcp.py`, `await_user` in `app/nodes/execution_nodes.py`.
|
||||
- **Depends on step 1:** MCP results hold discovered cloud state.
|
||||
- **Acceptance criteria:** A test with a planted instruction in an MCP result shows it withheld and noted.
|
||||
73
.agents/plans/iter-05-jev-decisions/MAP.md
Normal file
73
.agents/plans/iter-05-jev-decisions/MAP.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Plan Map
|
||||
|
||||
**Iteration:** 5, Jev decisions
|
||||
**Intent:** Use Jev, a hosted decision model, for the agent's closed questions (yes or no, pick one, rate on a scale) so that large-model calls are kept for writing and reasoning. The aims are fewer tokens, faster turns, and checks the engine cannot make with string matching.
|
||||
**Status:** planned; gated on step 1
|
||||
**This directory covers:** steps 1 to 14
|
||||
|
||||
## What Jev is
|
||||
|
||||
Jev is TypeSafe AI's "System One" model. It takes a `state` (text) and a map of typed `questions`, and returns calibrated probabilities instead of prose:
|
||||
|
||||
- **Choice:** pick one option. Returns `choice`, `confidence`, `probabilities`.
|
||||
- **Score:** rate against ordered levels. Returns `score`, `probabilities`, `legend`, `confidence`.
|
||||
- **Noul:** yes or no. Returns `noul` (0 to 1) and `confidence`.
|
||||
|
||||
Facts from the vendor's documentation, read 2026-10-06:
|
||||
|
||||
- `POST /api/v1/decide` with `Authorization: Bearer <key>`; body `model` (optional), `state`, `questions`; response `answers` plus `usage` (`input_tokens`, `cost_usd`, `credits_remaining_usd`).
|
||||
- Limits: 64 thousand tokens for state plus questions, 32 thousand for state plus the longest question.
|
||||
- Price: $0.42 per million input tokens; output free.
|
||||
- Undocumented: rate limits, latency, batching, data retention, self-hosting.
|
||||
- The vendor claims up to 200x faster and 400x cheaper than comparable LLMs on classification. That is unmeasured here.
|
||||
|
||||
Jev will be deployed separately and reached with an API key (owner, 2026-10-06).
|
||||
|
||||
## Sources of truth
|
||||
|
||||
- https://www.langchain.com/blog/building-a-harness-with-jev and https://www.jevtypesafeai.com/use-cases (and `/docs` on that site).
|
||||
- Review report, section "Where Jev fits": https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. `Report ref` names its J-number.
|
||||
- `iter-04-learning-and-skills/`: several steps here build on steps there.
|
||||
- `docs/adr/0003-langgraph-holds-state-adk-agents-reason.md`: step 12 changes what it says about the Orchestrator.
|
||||
|
||||
## Boundaries
|
||||
|
||||
Jev answers closed questions. Writing Deliverables, grounding product choices and producing infrastructure code stay with the large models. Dependencies and Approvals stay rules in the Ledger.
|
||||
|
||||
## Status
|
||||
|
||||
| Step | What | Report ref | Status |
|
||||
|---|---|---|---|
|
||||
| 1 | Resolve the open questions | none | open, owner's input |
|
||||
| 2 | Decider interface and Jev client with limits | none | planned |
|
||||
| 3 | Pillars a Revision touches | J3 | planned |
|
||||
| 4 | Semantic checks at submission | J4 | planned |
|
||||
| 5 | Finding-to-Deliverable routing | J2 | planned |
|
||||
| 6 | Model tier per delegation | J5 | planned |
|
||||
| 7 | Context selection | J6 | planned |
|
||||
| 8 | Skill routing | J9 | planned |
|
||||
| 9 | In-run context compaction | J10 | planned |
|
||||
| 10 | Evaluation rubric and lesson triage | J8 | planned |
|
||||
| 11 | ADR: Orchestrator routing | none | planned, owner's decision |
|
||||
| 12 | Orchestrator routing | J1 | planned |
|
||||
| 13 | Cloud-read call guard | J7 | planned |
|
||||
| 14 | Input guardrail | J11 | planned |
|
||||
|
||||
## Dependency graph
|
||||
|
||||
- Step 1: no dependencies
|
||||
- Step 2: depends on step 1
|
||||
- Step 3: depends on step 2, iteration 4 step 3
|
||||
- Step 4: depends on step 2
|
||||
- Step 5: depends on step 2, iteration 4 step 1
|
||||
- Step 6: depends on step 2
|
||||
- Step 7: depends on step 2, iteration 4 step 13
|
||||
- Step 8: depends on step 2, iteration 4 step 12
|
||||
- Step 9: depends on step 2
|
||||
- Step 10: depends on step 2, iteration 4 step 7, iteration 4 step 8
|
||||
- Step 11: depends on step 4 having evaluation results
|
||||
- Step 12: depends on step 2, step 11
|
||||
- Step 13: depends on step 2
|
||||
- Step 14: depends on step 2
|
||||
|
||||
Steps 3 and 4 come first: they are small, need no ADR change, and target Revisions, where the tokens are. The Orchestrator is 11% of case 03's tokens; the IaC author and validator together used 1.43 million over 7 runs.
|
||||
11
.dockerignore
Normal file
11
.dockerignore
Normal file
@@ -0,0 +1,11 @@
|
||||
# Keep the build context to what the image copies.
|
||||
.git
|
||||
.venv
|
||||
.cache
|
||||
.agents
|
||||
.scratch
|
||||
**/__pycache__
|
||||
*.db
|
||||
.env
|
||||
eval/results
|
||||
deliverables/executions
|
||||
54
.env.example
54
.env.example
@@ -1,37 +1,23 @@
|
||||
# GCP Solution Architecture Agent Environment Variables
|
||||
|
||||
# Agent Identity & Server Config
|
||||
# Agent Identity
|
||||
AGENT_NAME=gcp_solution_architecture_agent
|
||||
AGENT_VERSION=1.0.0
|
||||
LOG_LEVEL=INFO
|
||||
HOST=0.0.0.0
|
||||
PORT=8080
|
||||
|
||||
# LLM Provider Configuration (OpenAI or Azure OpenAI)
|
||||
OPENAI_API_KEY=your-openai-api-key-here
|
||||
AZURE_OPENAI_API_KEY=your-azure-openai-api-key-here
|
||||
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
|
||||
AZURE_OPENAI_DEPLOYMENT=gpt-4o
|
||||
AZURE_OPENAI_API_VERSION=2024-02-15-preview
|
||||
LLM_TEMPERATURE=0.2
|
||||
# A2A server (python -m app.main). The container image sets A2A_PORT=8080.
|
||||
A2A_HOST=0.0.0.0
|
||||
A2A_PORT=8000
|
||||
# A2A_PUBLIC_URL=https://agents.example.com/solution-architect
|
||||
|
||||
# Google Cloud Project & Scanner Credentials
|
||||
GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/gcp-key.json
|
||||
GCP_PROJECT_ID=your-gcp-project-id
|
||||
GCP_REGION=us-central1
|
||||
|
||||
# PostgreSQL Database & State Storage
|
||||
POSTGRES_HOST=localhost
|
||||
POSTGRES_PORT=5432
|
||||
POSTGRES_DB=gcp_agent_db
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=postgres
|
||||
# DATABASE_URL=postgresql://postgres:postgres@localhost:5432/gcp_agent_db
|
||||
SQLITE_FALLBACK_DB=gcp_agent_local.db
|
||||
|
||||
# Google ADK Multi-Agent Settings
|
||||
ADK_MAX_LOOP_ITERATIONS=5
|
||||
ADK_ENABLE_ARTIFACT_STORE=true
|
||||
# Models (providers/<name>.yaml): gemini/<model> uses ADK's native Gemini client.
|
||||
# Vertex AI with Application Default Credentials:
|
||||
# GOOGLE_GENAI_USE_VERTEXAI=true
|
||||
# GOOGLE_CLOUD_PROJECT=your-project-id
|
||||
# GOOGLE_CLOUD_LOCATION=global
|
||||
# GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/adc.json
|
||||
# Or the Gemini API: GOOGLE_API_KEY=...
|
||||
MODEL_TIMEOUT_SECONDS=180
|
||||
|
||||
# Cloud Provider Bundle (providers/<name>.yaml) & Remote MCP Servers
|
||||
CLOUD_PROVIDER=gcp
|
||||
@@ -46,9 +32,6 @@ TERRAFORM_TIMEOUT_SECONDS=300
|
||||
# TF_PLUGIN_CACHE_DIR=/var/cache/terraform-plugins
|
||||
|
||||
# Hierarchical Execution Engine
|
||||
A2A_HOST=0.0.0.0
|
||||
A2A_PORT=8000
|
||||
# A2A_PUBLIC_URL=https://agents.example.com/solution-architect
|
||||
ORCHESTRATOR_BUDGET=40
|
||||
SPECIALIST_BUDGET=15
|
||||
MAX_REVISIONS=3
|
||||
@@ -56,6 +39,11 @@ MAX_ORCHESTRATOR_TURNS=100
|
||||
CHECKPOINTER=sqlite
|
||||
CHECKPOINT_DB_PATH=checkpoints.db
|
||||
|
||||
# Google Developer Knowledge MCP Integration
|
||||
DEVELOPER_KNOWLEDGE_MCP_URL=https://developerknowledge.googleapis.com/mcp
|
||||
DEVELOPER_KNOWLEDGE_MCP_ENABLED=true
|
||||
# PostgreSQL (Execution records, and checkpoints when CHECKPOINTER=postgres)
|
||||
POSTGRES_HOST=localhost
|
||||
POSTGRES_PORT=5432
|
||||
POSTGRES_DB=gcp_agent_db
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=postgres
|
||||
# DATABASE_URL=postgresql://postgres:postgres@localhost:5432/gcp_agent_db
|
||||
SQLITE_FALLBACK_DB=gcp_agent_local.db
|
||||
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -26,3 +26,6 @@ tfplan
|
||||
# Container logs & temp files
|
||||
*.log
|
||||
.DS_Store
|
||||
|
||||
# Evaluation run outputs
|
||||
eval/results/
|
||||
|
||||
@@ -67,7 +67,7 @@ These are carried from the decision log. The numbers refer to it.
|
||||
|
||||
### Module layout
|
||||
|
||||
New code only. The current modules stay untouched until [increment 9](#build-increments).
|
||||
The engine sits in the KAB workflow-agent layout since 2026-10-06 (`.agents/plans/iter-03-kab-layout/`). The original pipeline was deleted the same day.
|
||||
|
||||
```
|
||||
agents/ # framework-neutral role files (one per role)
|
||||
@@ -75,6 +75,7 @@ agents/ # framework-neutral role files (one per rol
|
||||
requirements-analyst.md decomposer.md product-selector.md diagrammer.md
|
||||
describer.md design-advisor.md pillar-advisor.md iac-author.md
|
||||
validator.md packager.md
|
||||
partials/specialist-protocol.md
|
||||
providers/
|
||||
gcp.yaml # Cloud provider bundle
|
||||
skills/
|
||||
@@ -84,32 +85,41 @@ skills/
|
||||
cloud-discovery/SKILL.md # our tooling Skills (Agent Skills spec compliant)
|
||||
iac-validation/SKILL.md
|
||||
app/
|
||||
execution/
|
||||
deliverables.py # DeliverableKind, PhaseId, DEPENDENCY_GRAPH, contracts
|
||||
main.py # A2A server on A2A_PORT: /a2a, agent card, /health, /generate
|
||||
a2a_server.py # maps A2A tasks <-> Executions and interrupts
|
||||
agent.py # ExecutionEngine (start, resume, interaction) and open_assembly
|
||||
card.py # A2A agent card, derived from the dependency graph
|
||||
states/
|
||||
execution_state.py # ExecutionState TypedDict
|
||||
ledger.py # Ledger: Deliverable/Phase states, acceptance, Revision (pure, no I/O)
|
||||
graph.py # LangGraph StateGraph: orchestrator turn, Specialist fan-out, interrupts
|
||||
tools.py # Orchestrator and Specialist execution tools bound to the Ledger
|
||||
state.py # ExecutionState TypedDict
|
||||
recorder.py # records submissions, Approvals, Revisions, final files
|
||||
agents/
|
||||
nodes/
|
||||
execution_nodes.py # orchestrator turn, Specialist task, await user, finalize
|
||||
workflows/
|
||||
deliverables.py # DeliverableKind, PhaseId, DEPENDENCY_GRAPH, contracts
|
||||
execution_graph.py # LangGraph StateGraph wiring of the nodes
|
||||
adk/
|
||||
runtime.py # AgentRuntime protocol, RoleSpec, AgentTask, Outcome types
|
||||
roles.py # loads agents/*.md into RoleSpec
|
||||
adk_runtime.py # ADK LlmAgent + SkillToolset + LiteLlm implementation
|
||||
fake_runtime.py # scripted implementation for tests (under app/ so eval can reuse it)
|
||||
skills/
|
||||
agent_skills.py # Agent Skills discovery and frontmatter validation
|
||||
tools/
|
||||
general/ # mermaid check, terraform fmt/validate (local)
|
||||
cloud_read/ # terraform plan (local CLI, reads live state)
|
||||
execution.py # Orchestrator and Specialist execution tools bound to the Ledger
|
||||
toolkit.py # which tools each role gets
|
||||
pillars.py # the design advisor's fan-out to pillar advisors
|
||||
local.py # mermaid check, terraform fmt/validate/plan (local, tiered)
|
||||
mcp.py # McpServerSpec, read-only allowlist, tier + permission metadata
|
||||
providers.py # loads providers/<name>.yaml
|
||||
a2a/
|
||||
server.py # aiohttp A2A server on :8000, agent_card.json
|
||||
executor.py # maps A2A tasks <-> Executions and interrupts
|
||||
recorder.py # records submissions, Approvals, Revisions, status
|
||||
checkpointing.py # memory | sqlite | postgres checkpointer
|
||||
presentation.py # prompts and the /generate response
|
||||
scripts/sync-vendored-skills.sh
|
||||
```
|
||||
|
||||
### Deliverables and the dependency graph <a id="dependency-graph"></a>
|
||||
|
||||
`app/execution/deliverables.py` declares the graph as data. It is the only place Phases, Deliverables and Dependencies are listed.
|
||||
`app/workflows/deliverables.py` declares the graph as data. It is the only place Phases, Deliverables and Dependencies are listed.
|
||||
|
||||
| Phase | Deliverable kind | Specialist role | Depends on | Tools (tier) |
|
||||
|---|---|---|---|---|
|
||||
@@ -131,7 +141,7 @@ scripts/sync-vendored-skills.sh
|
||||
|
||||
### The Ledger: acceptance and Revision rules
|
||||
|
||||
`app/execution/ledger.py` is a pure module. It holds every rule code owns, and the tests exercise it heavily.
|
||||
`app/states/ledger.py` is a pure module. It holds every rule code owns, and the tests exercise it heavily.
|
||||
|
||||
- **Deliverable states:** `pending → in_progress → submitted`. Only a Revision goes backwards.
|
||||
- **Phase states:** `open → awaiting_approval → approved`. A Phase becomes `awaiting_approval` when the Orchestrator calls `request_approval` with all its Deliverables `submitted`.
|
||||
@@ -149,7 +159,7 @@ scripts/sync-vendored-skills.sh
|
||||
|
||||
### Execution graph (LangGraph)
|
||||
|
||||
`app/execution/graph.py` compiles once at startup with the configured checkpointer: `SqliteSaver` locally and in tests, `PostgresSaver` in production. The Execution id is the LangGraph `thread_id`.
|
||||
`app/workflows/execution_graph.py` wires the nodes in `app/nodes/execution_nodes.py`, and the graph compiles once at startup with the configured checkpointer: `SqliteSaver` locally and in tests, `PostgresSaver` in production. The Execution id is the LangGraph `thread_id`.
|
||||
|
||||
**State (`ExecutionState`):**
|
||||
- `execution_id`, `session_id`, `workflow_request`, `mode: "interactive" | "pre_approved"`, `allow_cloud_read: bool | None`
|
||||
@@ -182,7 +192,7 @@ await_user ───────────────────────
|
||||
- **Escalations:** end the Execution as `completed_with_warnings`.
|
||||
- **`finalize`:** runs after Phase 4 is `approved`. It writes files to `deliverables/executions/{execution_id}/`: the Solution guide, every Deliverable as markdown, and attachments under `terraform/`. It then records completion.
|
||||
|
||||
**Orchestrator tools** (`app/execution/tools.py`):
|
||||
**Orchestrator tools** (`app/tools/execution.py`):
|
||||
|
||||
| Tool | Effect |
|
||||
|---|---|
|
||||
@@ -200,7 +210,7 @@ await_user ───────────────────────
|
||||
|
||||
### Agent runtime interface
|
||||
|
||||
`app/agents/runtime.py`:
|
||||
`app/adk/runtime.py`:
|
||||
|
||||
```python
|
||||
class AgentRuntime(Protocol):
|
||||
@@ -299,10 +309,10 @@ role_mcp_servers:
|
||||
|
||||
### Tools
|
||||
|
||||
- **General, local** (`app/tools/general/`):
|
||||
- **General, local** (`app/tools/local.py`):
|
||||
- `mermaid_check`: moved from `validation_tools.validate_mermaid_diagram`.
|
||||
- `terraform_fmt` and `terraform_validate`: run the `terraform` CLI in a temp dir with `-backend=false`. There are no credentials, and the tool fails clearly if `terraform` is missing.
|
||||
- **Cloud read, local** (`app/tools/cloud_read/`): `terraform_plan` needs ADC and `-lock=false`, and never runs `apply`.
|
||||
- **Cloud read, local** (`app/tools/local.py`): `terraform_plan` needs ADC and `-lock=false`, and never runs `apply`.
|
||||
- **Remote MCP servers** (`app/tools/mcp.py`): the Cloud provider's own hosted servers, declared in the provider bundle and used for both discovery and grounding. They replace `GCPEnvironmentScanner` (hand-written REST calls) and the hand-rolled JSON-RPC `DeveloperKnowledgeMCPClient`, including its offline fallback knowledge base and the `data`/`documents` bug from review candidate 02. Both are deleted in increment 9.
|
||||
- `McpServerSpec(id, url, tier, allow, preview)` is framework-neutral. The ADK runtime maps each one to `McpToolset(StreamableHTTPConnectionParams(url, headers=auth), tool_filter=allow)`.
|
||||
- **Read-only in depth.** Tools outside `allow` are never offered to any agent. At startup, a server whose `tools/list` reports an allowed tool with `readOnlyHint` not true fails to load, loudly. Operators should also run the agent's identity with only viewer roles, plus Google Cloud's control that blocks read-write MCP tool use and an IAM deny policy (`docs/operations/mcp-access.md`, written in increment 6).
|
||||
@@ -314,7 +324,7 @@ role_mcp_servers:
|
||||
|
||||
### Interaction
|
||||
|
||||
**A2A** (`app/a2a/`), using `a2a-sdk>=0.3` on `aiohttp` :8000, with `agent_card.json` and `/.well-known/agent.json`:
|
||||
**A2A** (`app/main.py`, `app/a2a_server.py`, `app/card.py`), using `a2a-sdk>=0.3` on `aiohttp` :8000, with `agent_card.json` and `/.well-known/agent.json`:
|
||||
- **Task ↔ Execution:** an A2A task is an Execution (`task.id` = `execution_id`), and `context_id` is the Session.
|
||||
- **Status updates:** `working` while nodes run.
|
||||
- **`input-required`** at every `interrupt()`. The interrupt payload goes out as a `DataPart`, with a text rendering for plain clients.
|
||||
@@ -332,7 +342,7 @@ The route switches to the new graph in increment 7, behind `AGENT_ENGINE=hierarc
|
||||
|
||||
### Recording
|
||||
|
||||
`app/execution/recorder.py` reuses `DatabaseManager` tables until candidate 03:
|
||||
`app/recorder.py` reuses `DatabaseManager` tables until candidate 03:
|
||||
- `workflow_executions`: one row per Execution, updated at each interrupt and at completion.
|
||||
- `adk_artifacts`: one row per Deliverable submission. Rows for earlier Revisions stay as history.
|
||||
- `orchestrator_review_logs`: one row per Ledger event. `review_status` holds the event type, `reviewer_agent` holds the role or `user`, and `feedback` holds the reason or feedback.
|
||||
@@ -430,7 +440,7 @@ These are ordered, and each leaves the suite green. They are called *increments*
|
||||
- The A2A server and executor, `agent_card.json`, `test_executor.py`.
|
||||
- `/generate` behind `AGENT_ENGINE`, and the Dockerfile exposing :8000 and :8080.
|
||||
8. **Evaluation:** scripted answers, template-heading scoring, and a real-model eval run with results recorded under `## Comments`. Proceed to 9 only when the pass rate meets the current harness threshold (≥ 80% per case).
|
||||
9. **Cleanup** (carried over from the superseded spec):
|
||||
9. **Cleanup** (carried over from the superseded spec). **Done differently on 2026-10-06:** the engine moved into the KAB layout and the original pipeline was then deleted. See `.agents/plans/iter-03-kab-layout/`. The list below is the plan as first written:
|
||||
- **Delete:** `app/tools/gcp_scanner.py`, `app/tools/mcp_developer_knowledge.py` (replaced by remote MCP servers), `app/adk/`, `app/nodes/`, `app/states/`, `app/workflows/gcp_architecture_graph.py`, `app/skills/` (the loader and the 5 non-compliant Skills), `tests/test_graph_workflow.py`, the ADK and orchestrator tests in `tests/test_adk_architecture.py` (keeping the DB test as `tests/test_database.py`), and `test_adk_mcp_function_tools`.
|
||||
- **Fix:** remove the unreachable duplicate `return` in `routes.py`. Rename `ADK_MAX_LOOP_ITERATIONS` → removed, and delete `ADK_ENABLE_ARTIFACT_STORE`.
|
||||
- **Update:**
|
||||
@@ -444,8 +454,8 @@ These are ordered, and each leaves the suite green. They are called *increments*
|
||||
## Acceptance
|
||||
|
||||
- [ ] **Upstream parity:** a pre-approved evaluation run produces a Solution guide containing every heading of the upstream `output-template.md`, and all benchmark cases pass the threshold.
|
||||
- [ ] **Single source for the workflow:** Phases, Deliverables and Dependencies are listed only in `app/execution/deliverables.py`, and card and manifest data is derived from it.
|
||||
- [ ] **No framework types outside `app/agents/adk_runtime.py`:** `rg -n "google\.adk" app | grep -v adk_runtime.py` is empty. After increment 9, `rg -n "app\.adk|OrchestratorLoopAgent|compat" app eval tests` is empty.
|
||||
- [ ] **Single source for the workflow:** Phases, Deliverables and Dependencies are listed only in `app/workflows/deliverables.py`, and card and manifest data is derived from it (`tests/test_workflow_manifest.py` checks `workflow.yaml`).
|
||||
- [ ] **No framework types outside `app/adk/adk_runtime.py`:** `rg -n "google\.adk" app | grep -v adk_runtime.py` is empty. Nothing of the original pipeline remains: `rg -n "OrchestratorLoopAgent|compat|app\.legacy" app eval tests` is empty.
|
||||
- [ ] **Vendored Skills unchanged:** they match the `81a31a6` checksums, and every Skill under `skills/` passes the frontmatter rules.
|
||||
- [ ] **Approvals only by structured decision:** no path grants an Approval from free text (covered by tests).
|
||||
- [ ] **Nothing changes cloud resources:** no tool can run `terraform apply`, a deploy or a script (`run_skill_script` absent from the tool lists). Cloud-read tools, MCP included, are unreachable without permission.
|
||||
@@ -457,7 +467,7 @@ These are ordered, and each leaves the suite green. They are called *increments*
|
||||
|
||||
- **Specialists submitting without asking (D32).** A Specialist might submit without getting the user's conversational OK. Mitigation: role files require confirmation through `needs_input` before `submit_deliverable`, and the evaluation checks that submissions follow a question. If it's still observed, fall back to Approval per Deliverable (the Q31 alternative).
|
||||
- **ADK Skills are Experimental.** The `SkillToolset` API may change. It's confined to `adk_runtime.py`, and Deep Agents is the planned second implementation.
|
||||
- **Parallel Specialists and rate limits.** Fan-out multiplies concurrent model calls. Cap concurrency per Execution (default 4) in `graph.py`.
|
||||
- **Parallel Specialists and rate limits.** Fan-out multiplies concurrent model calls. Cap concurrency per Execution (default 4) in `app/nodes/execution_nodes.py` (not built; `.agents/plans/iter-04-learning-and-skills/` step 5).
|
||||
- **Upstream Skill drift.** Upstream edits may assume a different workflow. Re-sync only through the script, and review the diff against `agents/*.md` and `deliverables.py`.
|
||||
- **Two frameworks in one process.** LangGraph and ADK both manage async state. Keep ADK sessions in memory per run, and never put ADK objects in graph state.
|
||||
- **Remote MCP server availability and drift.** Several servers are Preview (Asset Inventory, Network Intelligence, Billing), each needs its product API enabled in the user's project, and tool sets can change. Mitigations:
|
||||
@@ -469,6 +479,8 @@ These are ordered, and each leaves the suite green. They are called *increments*
|
||||
|
||||
## Comments
|
||||
|
||||
The dated entries below are history and keep the module paths of their day. Translate them with the table in `.agents/plans/README.md`.
|
||||
|
||||
### 2026-09-15 · Increment 1: environment and baseline
|
||||
|
||||
**Environment**
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
## Direction
|
||||
|
||||
Before designing or restructuring agent code, read `docs/adr/`. ADR-0002 is the anchor: behaviour lives in Agent Skills (agentskills.io format) loaded by the framework's own skill runtime, and this repo builds the tooling around them. Google Cloud is the first Cloud provider, not the only one, so cloud-specific knowledge belongs in Skills and tooling, never in the agent's core. Reach cloud environments and cloud documentation through the provider's hosted remote MCP servers, not hand-written API clients. The current build plan is `.scratch/solution-architect-agent/spec.md`.
|
||||
Before designing or restructuring agent code, read `docs/adr/`. ADR-0002 is the anchor: behaviour lives in Agent Skills (agentskills.io format) loaded by the framework's own skill runtime, and this repo builds the tooling around them. Google Cloud is the first Cloud provider, not the only one, so cloud-specific knowledge belongs in Skills and tooling, never in the agent's core. Reach cloud environments and cloud documentation through the provider's hosted remote MCP servers, not hand-written API clients.
|
||||
|
||||
Plans and history live in `.agents/plans/`, one directory per iteration. Before changing anything under `app/`, `agents/`, `skills/` or `eval/`, read `.agents/plans/README.md`: it lists each iteration's status and maps the module paths that specs and ADRs written before the KAB layout move still use. When you finish a step, record it there as that README describes. The design spec for iteration 2 is `.scratch/solution-architect-agent/spec.md`.
|
||||
|
||||
## Agent skills
|
||||
|
||||
|
||||
26
Dockerfile
26
Dockerfile
@@ -28,7 +28,8 @@ FROM python:3.12-slim
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONPATH=/app \
|
||||
PORT=8080
|
||||
A2A_HOST=0.0.0.0 \
|
||||
A2A_PORT=8080
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
@@ -40,28 +41,25 @@ RUN groupadd -g 10001 appgroup && \
|
||||
COPY --from=builder /install /usr/local
|
||||
COPY --from=builder /usr/local/bin/terraform /usr/local/bin/terraform
|
||||
|
||||
# Copy application source, skills, and configuration
|
||||
# Copy the engine, role files, provider bundles and Skills.
|
||||
COPY app/ ./app/
|
||||
COPY agents/ ./agents/
|
||||
COPY providers/ ./providers/
|
||||
COPY skills/ ./skills/
|
||||
COPY eval/ ./eval/
|
||||
COPY docs/ ./docs/
|
||||
COPY deliverables/ ./deliverables/
|
||||
COPY terraform/ ./terraform/
|
||||
COPY workflow.yaml ./
|
||||
COPY requirements.yaml ./
|
||||
|
||||
RUN chown -R appuser:appgroup /app
|
||||
# Executions write their outputs under deliverables/executions/<execution_id>/.
|
||||
RUN mkdir -p deliverables && chown -R appuser:appgroup /app
|
||||
|
||||
USER appuser
|
||||
|
||||
# 8080: legacy REST app (removed in increment 9). 8000: A2A server with pre-approved /generate:
|
||||
# python -m app.a2a.server
|
||||
EXPOSE 8080 8000
|
||||
# The A2A server: JSON-RPC at /a2a, the agent card, /health and pre-approved /generate.
|
||||
# Set A2A_PUBLIC_URL to the address callers reach, so the agent card advertises it.
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
|
||||
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/health')" || exit 1
|
||||
# /health answers once the provider bundle and remote MCP catalogs have loaded.
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
|
||||
CMD python -c "import os, urllib.request; urllib.request.urlopen(f'http://localhost:{os.environ[\"A2A_PORT\"]}/health')" || exit 1
|
||||
|
||||
CMD ["python", "-m", "app.agent", "--host", "0.0.0.0", "--port", "8080"]
|
||||
CMD ["python", "-m", "app.main"]
|
||||
|
||||
|
||||
145
README.md
145
README.md
@@ -1,61 +1,46 @@
|
||||
# GCP Solution Architecture Agent (`gcp_solution_architecture_agent`)
|
||||
|
||||
A production-ready Google Cloud Solution Architecture Agent built using **Google ADK (Agent Development Kit)**, **LangChain**, and **LangGraph**, featuring a multi-agent orchestration graph with iterative review loops, dynamic local skill loading, PostgreSQL database persistence, a Starlette REST API server, and an offline evaluation harness.
|
||||
A hierarchical ReAct agent that turns a request for a cloud workload into a reviewed solution architecture: requirements, a target design with diagram and infrastructure-as-code, validation results, and a packaged guide. It replaces the [`google-cloud-solution-architecture`](https://github.com/google/skills/blob/main/skills/cloud/google-cloud-solution-architecture/SKILL.md) skill with an agent that runs anywhere. Google Cloud is the first Cloud provider.
|
||||
|
||||
> **Direction and current state (2026-09-15).** This agent is being built to replace the [`google-cloud-solution-architecture`](https://github.com/google/skills/blob/main/skills/cloud/google-cloud-solution-architecture/SKILL.md) skill with something that runs on any agent framework and, later, targets any Cloud provider. Google Cloud is the first provider.
|
||||
> - **Target:** behaviour lives in [Agent Skills](https://agentskills.io/specification), loaded by a framework's own skill runtime (Google ADK `SkillToolset`, or LangChain Deep Agents skills). This repo supplies the tooling those Skills call. See [ADR-0002](docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md).
|
||||
> - **Today:** the `google.adk` classes below are local compatibility stubs. The real `google-adk` package isn't used, and Phase outputs are fixed templates, not model-generated.
|
||||
>
|
||||
> Vocabulary is defined in [CONTEXT.md](CONTEXT.md); decisions are recorded in [docs/adr/](docs/adr/).
|
||||
Vocabulary is defined in [CONTEXT.md](CONTEXT.md), decisions in [docs/adr/](docs/adr/), and the plan and history of each iteration in [.agents/plans/](.agents/plans/README.md).
|
||||
|
||||
---
|
||||
|
||||
## 🌟 Architecture & Features
|
||||
|
||||
### 1. Multi-Agent Orchestration & `google.adk.agents.LoopAgent`
|
||||
The agent uses an end-to-end multi-agent graph managed by an `OrchestratorLoopAgent` (`google.adk.agents.LoopAgent`):
|
||||
- **`SourceDiscoveryAgent`** (Phase 0a): Pre-emptively audits and documents the existing pre-migration source environment (As-Is Architecture) and pain points.
|
||||
- **`DiscoveryAgent`** (Phase 0): Discovers functional & non-functional requirements while explicitly deferring product selection.
|
||||
- **`ArchitectureDesignAgent`** (Phase 1): Resolves product selections (Cloud Run, Pub/Sub, Cloud Storage, Firestore, IAM, KMS), grounds guidance with **Google Developer Knowledge MCP**, generates Mermaid diagrams (`architecture.mmd`), and builds Terraform IaC (`terraform/main.tf`).
|
||||
- **`ValidationReviewAgent`** (Phase 2): Evaluates Mermaid syntax, Terraform structural integrity, and artifact completeness using `google.adk.tools.FunctionTool`.
|
||||
- **`PackagingAgent`** (Phase 3): Compiles the final comprehensive `solution-architecture-guide.md` with **Before (Source) vs After (Target)** architecture comparison.
|
||||
- **`OrchestratorLoopAgent`**: Repeatedly loops through the multi-agent pipeline, reviewing and validating content in a feedback loop until production quality criteria pass or `max_iterations` is reached.
|
||||
### 1. Orchestrator and Specialists (`agents/`)
|
||||
One Orchestrator coordinates an Execution and is the only agent that talks to the user. It delegates each Deliverable to one Specialist, and independent Deliverables run in parallel:
|
||||
- **Requirements discovery:** `requirements-analyst`, `decomposer`.
|
||||
- **Solution architecture:** `product-selector`, `diagrammer`, `describer`, `design-advisor` (which consults one `pillar-advisor` per architecture-framework pillar), `iac-author`.
|
||||
- **Solution validation:** `validator`.
|
||||
- **Packaging:** `packager`.
|
||||
|
||||
### 2. Google ADK Integration (`google.adk`)
|
||||
Built directly on Google ADK framework primitives (with offline compatibility stubs):
|
||||
- **`google.adk.agents`**: `LlmAgent`, `SequentialAgent`, `ParallelAgent`, `LoopAgent`.
|
||||
- **`google.adk.tools`**: `FunctionTool` wrappers for static artifact and diagram validators, plus Google Developer Knowledge MCP tools (`developerknowledge:*`).
|
||||
- **`google.adk.artifacts`**: `ArtifactRepository` synchronized with disk and PostgreSQL.
|
||||
- **`google.adk.sessions`**: `PostgresSessionService` for durable session memory.
|
||||
- **`google.adk.runners`**: `Runner` for managing execution workflows.
|
||||
- **`google.adk.workflows`**: `Workflow` composition primitives.
|
||||
- **`google.adk.evaluation`**: `Evaluator` for scoring benchmark suites.
|
||||
Each Phase needs the user's explicit Approval before work that depends on it starts. A pre-approved mode serves batch and evaluation runs.
|
||||
|
||||
### 3. PostgreSQL Database Persistence (`app/database.py`)
|
||||
State, sessions, workflow executions, review iteration logs, and evaluation metrics are written to an external PostgreSQL database (with an automatic SQLite fallback for offline development):
|
||||
- `adk_sessions`: Durable session memory and state data.
|
||||
- `workflow_executions`: Workflow execution status, phase, and iteration metrics.
|
||||
- `adk_artifacts`: Generated solution architecture artifacts (including `docs/source-architecture.md` and `source-architecture.mmd`).
|
||||
- `orchestrator_review_logs`: Detailed iteration review logs from the `OrchestratorLoopAgent`.
|
||||
- `adk_evaluations`: Benchmark evaluation scores and pass rates.
|
||||
### 2. LangGraph state, ADK agents (`app/`)
|
||||
Per [ADR-0003](docs/adr/0003-langgraph-holds-state-adk-agents-reason.md):
|
||||
- A LangGraph `StateGraph` holds the Execution: the dependency graph of Deliverables (`app/workflows/deliverables.py`), the Ledger of their states (`app/states/ledger.py`), checkpoints, and `interrupt()` whenever the user must answer or approve.
|
||||
- Every agent is a Google ADK `LlmAgent` running its own ReAct loop behind one runtime interface (`app/adk/runtime.py`). `app/adk/adk_runtime.py` is the only module that imports `google.adk`.
|
||||
- Tools are tiered: local checks and public documentation run freely, reading the user's cloud needs permission once per Execution, and nothing can change cloud resources.
|
||||
|
||||
### 4. Dynamic Local Skill Loading (`app/skills/`)
|
||||
Domain knowledge is stored as self-contained markdown instruction templates under `app/skills/`:
|
||||
- `source_discovery/SKILL.md` (Phase 0a - Pre-emptive Source Environment Discovery)
|
||||
- `requirements_discovery/SKILL.md` (Phase 0 - Requirements Discovery & Deferral)
|
||||
- `architecture_design/SKILL.md` (Phase 1 - Product Selection & Terraform IaC)
|
||||
- `validation_rules/SKILL.md` (Phase 2 - Pre-deployment Validation Rules)
|
||||
- `packaging_guide/SKILL.md` (Phase 3 - Solution Guide Packaging)
|
||||
### 3. Persistence (`app/database.py`, `app/checkpointing.py`)
|
||||
Execution records (status, Ledger events, submitted Deliverables) go to PostgreSQL, with an automatic SQLite fallback for local development. Checkpoints use `CHECKPOINTER=memory | sqlite | postgres`.
|
||||
|
||||
The `SkillLoader` (`app/skills/loader.py`) dynamically parses frontmatter and injects instructions into phase system prompts.
|
||||
### 4. Agent Skills (`skills/`, loaded by `app/skills/`)
|
||||
Behaviour lives in Agent Skills (agentskills.io format) at the repository root, per ADR-0002:
|
||||
- `skills/vendor/google/`: the upstream solution-architecture Skill and six Well-Architected Framework pillar Skills, vendored unmodified.
|
||||
- `skills/cloud-discovery/` and `skills/iac-validation/`: our own Skills for the tooling.
|
||||
|
||||
### 5. Starlette REST API Server (`app/agent.py`)
|
||||
Production HTTP REST endpoints exposed via Starlette and Uvicorn:
|
||||
- `GET /health`: Operational health signal & database connection status.
|
||||
- `GET /card`: Agent capability card metadata.
|
||||
- `POST /generate`: Trigger the ADK multi-agent workflow execution.
|
||||
- `POST /validate`: Execute pre-deployment artifact validation.
|
||||
- `GET /sessions/{session_id}`: Query session state from the PostgreSQL database.
|
||||
`app/skills/agent_skills.py` discovers and validates them; the ADK runtime (`app/adk/adk_runtime.py`) loads them on demand. `providers/gcp.yaml` assigns Skills, tools, MCP servers and models to each role in `agents/`.
|
||||
|
||||
### 5. A2A Server (`app/main.py`)
|
||||
The container starts the hierarchical Execution engine (ADR-0003) behind an A2A server:
|
||||
- `POST /a2a`: A2A JSON-RPC for interactive, multi-turn Executions with Approvals.
|
||||
- `GET /.well-known/agent-card.json`: A2A agent card.
|
||||
- `GET /health`: Health signal, the Cloud provider and any MCP discovery gaps.
|
||||
- `POST /generate`: One-shot, pre-approved Execution for batch and evaluation callers.
|
||||
|
||||
The original linear pipeline and its REST routes (`/card`, `/validate`, `/sessions/{session_id}`) were removed on 2026-10-06; see `.agents/plans/iter-03-kab-layout/`.
|
||||
|
||||
---
|
||||
|
||||
@@ -69,19 +54,19 @@ pip install -r requirements-dev.txt
|
||||
|
||||
### Run Tests & Validation
|
||||
```bash
|
||||
# Run pytest test suite (100% pass rate)
|
||||
pytest -v
|
||||
# Run the deterministic test suite (no model or network calls)
|
||||
pytest -q
|
||||
|
||||
# Run offline benchmark evaluation harness
|
||||
python3 -m eval.eval_harness
|
||||
# Real-model evaluation of the engine (costs model usage; see eval/solution_eval.py)
|
||||
python3 -m eval.solution_eval
|
||||
|
||||
# Run static artifact validation
|
||||
python3 scripts/validate_artifacts.py
|
||||
```
|
||||
|
||||
### Start Starlette REST Server
|
||||
### Start the A2A Server
|
||||
```bash
|
||||
python3 app/agent.py --host 0.0.0.0 --port 8080
|
||||
python3 -m app.main --host 0.0.0.0 --port 8080
|
||||
```
|
||||
|
||||
### 🐳 Run with Podman
|
||||
@@ -100,7 +85,10 @@ podman run -d --name gcp-agent \
|
||||
-v $(pwd)/deliverables:/app/deliverables:Z \
|
||||
-v ~/.config/gcloud/application_default_credentials.json:/app/credentials/adc.json:ro \
|
||||
-e GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/adc.json \
|
||||
-e GCP_PROJECT_ID=your-project-id \
|
||||
-e GOOGLE_GENAI_USE_VERTEXAI=true \
|
||||
-e GOOGLE_CLOUD_PROJECT=your-project-id \
|
||||
-e GOOGLE_CLOUD_LOCATION=global \
|
||||
-e A2A_PUBLIC_URL=http://localhost:8080 \
|
||||
localhost/gcp-solution-architecture-agent:latest
|
||||
```
|
||||
|
||||
@@ -121,10 +109,10 @@ podman run -d --name gcp-agent \
|
||||
# Health Signal
|
||||
curl http://localhost:8080/health
|
||||
|
||||
# Agent Capability Card
|
||||
curl http://localhost:8080/card
|
||||
# A2A Agent Card
|
||||
curl http://localhost:8080/.well-known/agent-card.json
|
||||
|
||||
# Run Multi-Agent ADK Workflow
|
||||
# Run a pre-approved Execution
|
||||
curl -X POST http://localhost:8080/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"request": "Event-driven regional HTTP application in Podman"}'
|
||||
@@ -135,31 +123,30 @@ curl -X POST http://localhost:8080/generate \
|
||||
## 📁 Repository Structure
|
||||
```
|
||||
gcp_solution_architecture_agent/
|
||||
├── app/
|
||||
│ ├── adk/ # Google ADK framework primitives & fallbacks
|
||||
│ │ ├── agents.py # Multi-agents & OrchestratorLoopAgent
|
||||
│ │ ├── artifacts.py # PostgresArtifactRepository
|
||||
│ │ ├── compat.py # ADK compatibility layer
|
||||
│ │ ├── evaluation.py # ADKEvaluator
|
||||
│ │ ├── runners.py # ADKAgentRunner
|
||||
│ │ ├── sessions.py # PostgresSessionService
|
||||
│ │ ├── tools.py # ADK FunctionTools
|
||||
│ │ └── workflows.py # ADK Workflow composition
|
||||
│ ├── agent.py # Starlette REST server entrypoint
|
||||
│ ├── card.py # AgentCard & capability metadata
|
||||
├── agents/ # Framework-neutral role files, one per agent
|
||||
├── providers/ # Cloud provider bundles (gcp.yaml)
|
||||
├── skills/ # Agent Skills: vendored upstream + our tooling Skills
|
||||
├── app/ # KAB workflow-agent layout
|
||||
│ ├── main.py # A2A server entrypoint (python -m app.main)
|
||||
│ ├── a2a_server.py # Maps A2A tasks to Executions and their prompts
|
||||
│ ├── agent.py # ExecutionEngine and its assembly (composition root)
|
||||
│ ├── card.py # A2A agent card
|
||||
│ ├── states/ # ExecutionState and the Ledger
|
||||
│ ├── nodes/ # Execution graph nodes (orchestrator, specialist, user)
|
||||
│ ├── workflows/ # Execution graph wiring and the Deliverable dependency graph
|
||||
│ ├── adk/ # Agent-runtime interface, ADK runtime, role loader
|
||||
│ ├── skills/ # Agent Skills discovery and validation
|
||||
│ ├── tools/ # Execution tools, local tools, remote MCP, pillar fan-out
|
||||
│ ├── providers.py # Provider bundle loader
|
||||
│ ├── recorder.py # Execution recording
|
||||
│ ├── checkpointing.py # LangGraph checkpointer selection
|
||||
│ ├── presentation.py # Prompts and /generate responses
|
||||
│ ├── config.py # Typed settings (Pydantic BaseSettings)
|
||||
│ ├── database.py # PostgreSQL database persistence manager
|
||||
│ ├── nodes/ # Phase execution node handlers
|
||||
│ ├── skills/ # Local SKILL.md instruction templates & SkillLoader
|
||||
│ ├── states/ # GCPArchitectureState schema definition
|
||||
│ ├── tools/ # LangChain & ADK validation tools
|
||||
│ └── workflows/ # LangGraph StateGraph & Starlette REST routes
|
||||
├── eval/ # ADK Evaluation & Optimization Harness
|
||||
│ ├── datasets/ # Benchmark test cases (JSON)
|
||||
│ ├── eval_harness.py # Benchmark runner CLI
|
||||
│ ├── metrics.py # Scoring rubrics & evaluation logic
|
||||
│ ├── optimizer.py # Prompt tuning optimizer
|
||||
│ └── test_eval_harness.py # Evaluation unit tests
|
||||
│ └── database.py # Database persistence manager
|
||||
├── eval/ # Evaluation
|
||||
│ ├── solution_eval.py # Real-model evaluation of the Execution engine
|
||||
│ ├── scoring.py # Checks applied to a finished Execution
|
||||
│ └── datasets/ # Benchmark cases (JSON)
|
||||
├── deliverables/ # Scalable deliverable repository (categorized & versioned)
|
||||
│ ├── as-is/ # Pre-emptive Source Environment Deliverables (Before State)
|
||||
│ │ ├── source-architecture.md
|
||||
|
||||
@@ -9,8 +9,8 @@ python3 scripts/validate_artifacts.py
|
||||
# 2. Pytest unit & integration test suite
|
||||
pytest -v
|
||||
|
||||
# 3. Offline ADK evaluation benchmark suite
|
||||
python3 -m eval.eval_harness
|
||||
# 3. Real-model evaluation of the engine (needs model credentials; costs usage)
|
||||
python3 -m eval.solution_eval
|
||||
|
||||
# 4. Optional Terraform formatting & validation
|
||||
terraform -chdir=terraform fmt -check -recursive
|
||||
|
||||
@@ -2,13 +2,15 @@
|
||||
role: iac-author
|
||||
deliverable: deployment_guidance
|
||||
tools: [terraform_fmt, terraform_validate]
|
||||
budget: 30
|
||||
includes: [specialist-protocol]
|
||||
---
|
||||
You produce the **deployment guidance** Deliverable: the Skill's task for generating infrastructure-as-code and deployment instructions.
|
||||
|
||||
- Write Terraform that provisions every selected product with the configuration the design recommendations call for.
|
||||
- Take every environment-specific value (project, region, names, image references) from variables; embed no credentials or secrets.
|
||||
- Format and validate the Terraform with your tools when they are available, and fix everything they report.
|
||||
- Put `terraform/main.tf` and `terraform/variables.tf` in the attachments, adding `terraform/outputs.tf` when outputs help.
|
||||
- Format and validate the Terraform with your tools when they are available, and fix everything they report. Pass the files with the same `terraform/` paths you will attach.
|
||||
- Submit as soon as validation passes. If a submission is refused, fix exactly what the refusal lists and submit again; don't rewrite the Terraform.
|
||||
- Put `terraform/main.tf` and `terraform/variables.tf` in the attachments (a list of `{path, content}`), adding `terraform/outputs.tf` when outputs help.
|
||||
|
||||
Required headings: `Deployment prerequisites` (APIs to enable, tools, permissions) and `Step-by-step deployment instructions`, whose commands you give for the user to run, never run yourself.
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
"""A2A interface for interactive Executions (A2A 1.0, with v0.3 compatibility)."""
|
||||
@@ -1,66 +0,0 @@
|
||||
"""The A2A agent card, derived from the dependency graph and settings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from a2a.types import (
|
||||
AgentCapabilities,
|
||||
AgentCard,
|
||||
AgentInterface,
|
||||
AgentProvider,
|
||||
AgentSkill,
|
||||
)
|
||||
|
||||
from app.config import Settings
|
||||
from app.execution.deliverables import PHASE_ORDER, deliverables_in
|
||||
|
||||
A2A_PATH = "/a2a"
|
||||
|
||||
|
||||
def build_agent_card(settings: Settings) -> AgentCard:
|
||||
base_url = (
|
||||
settings.A2A_PUBLIC_URL or f"http://{settings.A2A_HOST}:{settings.A2A_PORT}"
|
||||
).rstrip("/")
|
||||
phases = "; ".join(
|
||||
f"{phase.value}: {', '.join(kind.value for kind in deliverables_in(phase))}"
|
||||
for phase in PHASE_ORDER
|
||||
)
|
||||
return AgentCard(
|
||||
name=settings.AGENT_NAME,
|
||||
description=(
|
||||
"Designs a cloud solution architecture with you: requirements discovery, "
|
||||
"grounded product selection, diagram, design recommendations, "
|
||||
"infrastructure-as-code and dry-run validation, packaged as a solution "
|
||||
f"guide. Cloud provider: {settings.CLOUD_PROVIDER}."
|
||||
),
|
||||
version=settings.AGENT_VERSION,
|
||||
provider=AgentProvider(
|
||||
organization="Kyndryl Agent Builder", url="https://www.kyndryl.com"
|
||||
),
|
||||
supported_interfaces=[
|
||||
AgentInterface(
|
||||
url=f"{base_url}{A2A_PATH}",
|
||||
protocol_binding="JSONRPC",
|
||||
protocol_version="1.0",
|
||||
)
|
||||
],
|
||||
capabilities=AgentCapabilities(streaming=True, push_notifications=False),
|
||||
default_input_modes=["text/plain", "application/json"],
|
||||
default_output_modes=["text/plain", "text/markdown", "application/json"],
|
||||
skills=[
|
||||
AgentSkill(
|
||||
id="solution-architecture",
|
||||
name="Cloud solution architecture",
|
||||
description=(
|
||||
"Interactive, multi-turn engagement. Phases and their Deliverables: "
|
||||
f"{phases}. Each Phase needs your approval, sent as data."
|
||||
),
|
||||
tags=["architecture", "cloud", settings.CLOUD_PROVIDER, "terraform"],
|
||||
examples=[
|
||||
"Design a regional event ingestion platform on Google Cloud.",
|
||||
"Migrate our on-premises order service to managed services.",
|
||||
],
|
||||
input_modes=["text/plain", "application/json"],
|
||||
output_modes=["text/plain", "text/markdown", "application/json"],
|
||||
)
|
||||
],
|
||||
)
|
||||
@@ -23,10 +23,10 @@ from a2a.server.tasks import TaskUpdater
|
||||
from a2a.types import TaskState
|
||||
from google.protobuf import json_format
|
||||
|
||||
from app.execution.deliverables import DeliverableKind
|
||||
from app.execution.engine import ExecutionEngine, Interaction
|
||||
from app.execution.graph import GUIDE_FILENAME
|
||||
from app.execution.presentation import describe_prompt
|
||||
from app.agent import ExecutionEngine, Interaction
|
||||
from app.nodes.execution_nodes import GUIDE_FILENAME
|
||||
from app.presentation import describe_prompt
|
||||
from app.workflows.deliverables import DeliverableKind
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -1,31 +1 @@
|
||||
"""ADK package for GCP Solution Architecture Agent."""
|
||||
|
||||
from .agents import (
|
||||
ArchitectureDesignAgent,
|
||||
DiscoveryAgent,
|
||||
OrchestratorLoopAgent,
|
||||
PackagingAgent,
|
||||
ValidationReviewAgent,
|
||||
build_adk_multi_agent_system,
|
||||
)
|
||||
from .artifacts import PostgresArtifactRepository
|
||||
from .evaluation import ADKEvaluator
|
||||
from .runners import ADKAgentRunner
|
||||
from .sessions import PostgresSessionService
|
||||
from .tools import ADK_TOOLS
|
||||
from .workflows import create_gcp_adk_workflow
|
||||
|
||||
__all__ = [
|
||||
"DiscoveryAgent",
|
||||
"ArchitectureDesignAgent",
|
||||
"ValidationReviewAgent",
|
||||
"PackagingAgent",
|
||||
"OrchestratorLoopAgent",
|
||||
"build_adk_multi_agent_system",
|
||||
"PostgresArtifactRepository",
|
||||
"PostgresSessionService",
|
||||
"ADKAgentRunner",
|
||||
"ADKEvaluator",
|
||||
"ADK_TOOLS",
|
||||
"create_gcp_adk_workflow",
|
||||
]
|
||||
"""Agent runtimes: the framework-neutral interface agents run behind, and its implementations."""
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
@@ -17,6 +18,7 @@ from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from google.adk.agents import LlmAgent
|
||||
from google.adk.models.base_llm import BaseLlm
|
||||
from google.adk.runners import Runner
|
||||
@@ -28,7 +30,7 @@ from google.adk.tools.skill_toolset import SkillToolset
|
||||
from google.adk.tools.tool_context import ToolContext
|
||||
from google.genai import types
|
||||
|
||||
from app.agents.runtime import (
|
||||
from app.adk.runtime import (
|
||||
AgentTask,
|
||||
BudgetExhausted,
|
||||
Outcome,
|
||||
@@ -41,19 +43,90 @@ APP_NAME = "solution-architect"
|
||||
# run_skill_script is deliberately absent: generated or bundled scripts never run.
|
||||
SKILL_TOOLS = ["list_skills", "load_skill", "load_skill_resource"]
|
||||
TURN_ENDED_REPLY = "Your turn has already ended; this call was not run."
|
||||
NUDGE = (
|
||||
"You stopped without calling a tool that ends your turn, so nothing was recorded. "
|
||||
"Continue the task, and finish by calling the tool your instructions name for "
|
||||
"ending your turn. If you need the user, use the tool for asking questions."
|
||||
)
|
||||
TIMED_OUT = (
|
||||
"Your last model request timed out, so its reply was lost. Continue the task, "
|
||||
"and finish by calling the tool your instructions name for ending your turn."
|
||||
)
|
||||
# Tool replies that mean the call did nothing, logged so stuck agents can be diagnosed.
|
||||
REJECTION_PREFIXES = (
|
||||
"Not submitted",
|
||||
"Refused",
|
||||
"Discovery gap",
|
||||
"terraform validate failed",
|
||||
)
|
||||
# Client-side timeouts from either HTTP stack google-genai may use (httpx or aiohttp,
|
||||
# whose timeout errors subclass TimeoutError).
|
||||
MODEL_TIMEOUTS = (TimeoutError, httpx.TimeoutException)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ModelFactory = Callable[[RoleSpec], BaseLlm | str]
|
||||
|
||||
|
||||
def default_model_factory(role: RoleSpec) -> BaseLlm:
|
||||
"""Wrap the role's LiteLLM model id, so any LiteLLM provider can serve the agent."""
|
||||
from google.adk.models.lite_llm import LiteLlm
|
||||
GEMINI_PREFIX = "gemini/"
|
||||
|
||||
|
||||
def default_model_factory(role: RoleSpec) -> BaseLlm:
|
||||
"""Model for a role's LiteLLM-style id.
|
||||
|
||||
`gemini/<model>` uses ADK's native Gemini client, which targets Vertex AI when
|
||||
GOOGLE_GENAI_USE_VERTEXAI=true (with GOOGLE_CLOUD_PROJECT and
|
||||
GOOGLE_CLOUD_LOCATION) and the Gemini API otherwise. Any other id goes through
|
||||
LiteLLM, so other model providers need no code changes.
|
||||
"""
|
||||
if not role.model:
|
||||
raise ValueError(f"Role {role.name!r} has no model configured")
|
||||
if role.model.startswith(GEMINI_PREFIX):
|
||||
from google.adk.models.google_llm import Gemini
|
||||
|
||||
return Gemini(
|
||||
model=role.model.removeprefix(GEMINI_PREFIX),
|
||||
retry_options=types.HttpRetryOptions(attempts=5, initial_delay=2.0),
|
||||
)
|
||||
from google.adk.models.lite_llm import LiteLlm
|
||||
|
||||
return LiteLlm(model=role.model)
|
||||
|
||||
|
||||
USAGE_FIELDS = {
|
||||
"requests": None,
|
||||
"prompt": "prompt_token_count",
|
||||
"cached": "cached_content_token_count",
|
||||
"tool_prompt": "tool_use_prompt_token_count",
|
||||
"output": "candidates_token_count",
|
||||
"thinking": "thoughts_token_count",
|
||||
"total": "total_token_count",
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class TokenUsage:
|
||||
"""Model token counts, summed over every model response of one role and model."""
|
||||
|
||||
requests: int = 0
|
||||
prompt: int = 0
|
||||
cached: int = 0
|
||||
tool_prompt: int = 0
|
||||
output: int = 0
|
||||
thinking: int = 0
|
||||
total: int = 0
|
||||
|
||||
def add(self, metadata: Any) -> None:
|
||||
self.requests += 1
|
||||
for name, source in USAGE_FIELDS.items():
|
||||
if source:
|
||||
setattr(
|
||||
self,
|
||||
name,
|
||||
getattr(self, name) + (getattr(metadata, source, 0) or 0),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _RunControl:
|
||||
budget: int
|
||||
@@ -68,6 +141,9 @@ class _RunControl:
|
||||
if self.outcome is not None:
|
||||
return {"error": TURN_ENDED_REPLY}
|
||||
self.calls += 1
|
||||
logger.info(
|
||||
"%s tool call %d/%d: %s", self.role, self.calls, self.budget, tool.name
|
||||
)
|
||||
if self.calls > self.budget:
|
||||
self.outcome = BudgetExhausted(
|
||||
f"{self.role} used all {self.budget} tool calls before finishing."
|
||||
@@ -102,9 +178,17 @@ class _SpecTool(BaseTool):
|
||||
try:
|
||||
result = await self._spec.handler(**args)
|
||||
except TypeError as exc:
|
||||
logger.info("%s: invalid arguments: %s", self._spec.name, exc)
|
||||
return {"error": f"Invalid arguments for {self._spec.name}: {exc}"}
|
||||
if result.outcome is not None:
|
||||
self._control.outcome = result.outcome
|
||||
elif result.message.startswith(REJECTION_PREFIXES):
|
||||
logger.info(
|
||||
"%s rejected by %s: %s",
|
||||
self._control.role,
|
||||
self._spec.name,
|
||||
result.message[:500],
|
||||
)
|
||||
return {"result": result.message}
|
||||
|
||||
|
||||
@@ -114,7 +198,11 @@ class AdkAgentRuntime:
|
||||
*,
|
||||
skill_dirs: Mapping[str, Path],
|
||||
model_factory: ModelFactory = default_model_factory,
|
||||
max_nudges: int = 2,
|
||||
model_timeout_seconds: float | None = None,
|
||||
) -> None:
|
||||
self._max_nudges = max_nudges
|
||||
self._model_timeout_seconds = model_timeout_seconds
|
||||
self._skills: dict[str, Skill] = {}
|
||||
for name, path in skill_dirs.items():
|
||||
try:
|
||||
@@ -129,6 +217,8 @@ class AdkAgentRuntime:
|
||||
)
|
||||
self._skills[name] = skill
|
||||
self._model_factory = model_factory
|
||||
# (role, model) -> usage across every run of this runtime
|
||||
self.usage: dict[tuple[str, str], TokenUsage] = {}
|
||||
|
||||
async def run(
|
||||
self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]
|
||||
@@ -154,6 +244,7 @@ class AdkAgentRuntime:
|
||||
instruction=lambda _context: instructions,
|
||||
tools=agent_tools,
|
||||
before_tool_callback=control.before_tool,
|
||||
generate_content_config=self._content_config(),
|
||||
)
|
||||
runner = Runner(
|
||||
app_name=APP_NAME,
|
||||
@@ -161,36 +252,72 @@ class AdkAgentRuntime:
|
||||
session_service=InMemorySessionService(),
|
||||
auto_create_session=True,
|
||||
)
|
||||
message = types.Content(
|
||||
role="user", parts=[types.Part(text=_render_task(task))]
|
||||
)
|
||||
|
||||
session_id = str(uuid.uuid4())
|
||||
usage = self.usage.setdefault((role.name, role.model or "?"), TokenUsage())
|
||||
prompt = _render_task(task, role.skills)
|
||||
final_text = ""
|
||||
async with aclosing(
|
||||
runner.run_async(
|
||||
user_id=APP_NAME, session_id=str(uuid.uuid4()), new_message=message
|
||||
)
|
||||
) as events:
|
||||
async for event in events:
|
||||
for attempt in range(self._max_nudges + 1):
|
||||
if attempt:
|
||||
logger.info(
|
||||
"Nudging %s to end its turn (attempt %d)", role.name, attempt
|
||||
)
|
||||
message = types.Content(role="user", parts=[types.Part(text=prompt)])
|
||||
prompt = NUDGE
|
||||
try:
|
||||
async with aclosing(
|
||||
runner.run_async(
|
||||
user_id=APP_NAME, session_id=session_id, new_message=message
|
||||
)
|
||||
) as events:
|
||||
async for event in events:
|
||||
if event.usage_metadata is not None:
|
||||
usage.add(event.usage_metadata)
|
||||
if control.outcome is not None:
|
||||
break
|
||||
if (
|
||||
event.is_final_response()
|
||||
and event.content
|
||||
and event.content.parts
|
||||
):
|
||||
text = "".join(
|
||||
part.text for part in event.content.parts if part.text
|
||||
).strip()
|
||||
final_text = text or final_text # keep the last real reply
|
||||
except MODEL_TIMEOUTS as exc:
|
||||
if control.outcome is not None:
|
||||
break
|
||||
if event.is_final_response() and event.content and event.content.parts:
|
||||
final_text = "".join(
|
||||
part.text for part in event.content.parts if part.text
|
||||
).strip()
|
||||
return control.outcome
|
||||
logger.warning(
|
||||
"%s model request timed out (%s)", role.name, type(exc).__name__
|
||||
)
|
||||
final_text = final_text or f"{role.name}'s model requests timed out."
|
||||
prompt = TIMED_OUT
|
||||
if control.outcome is not None:
|
||||
return control.outcome
|
||||
|
||||
if control.outcome is not None:
|
||||
return control.outcome
|
||||
return Stopped(
|
||||
final_text
|
||||
or f"{role.name} ended without calling a tool that ends its turn."
|
||||
)
|
||||
|
||||
def _content_config(self) -> types.GenerateContentConfig | None:
|
||||
if self._model_timeout_seconds is None:
|
||||
return None
|
||||
return types.GenerateContentConfig(
|
||||
http_options=types.HttpOptions(
|
||||
timeout=int(self._model_timeout_seconds * 1000)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _agent_name(role: str) -> str:
|
||||
return re.sub(r"\W", "_", role)
|
||||
|
||||
|
||||
def _render_task(task: AgentTask) -> str:
|
||||
def _render_task(task: AgentTask, skills: Sequence[str] = ()) -> str:
|
||||
context = json.dumps(task.context, indent=2, default=str, ensure_ascii=False)
|
||||
return f"{task.prompt}\n\nContext:\n```json\n{context}\n```"
|
||||
skill_line = (
|
||||
f"\n\nSkills available to you (load with load_skill): {', '.join(skills)}."
|
||||
if skills
|
||||
else ""
|
||||
)
|
||||
return f"{task.prompt}{skill_line}\n\nContext:\n```json\n{context}\n```"
|
||||
@@ -1,196 +0,0 @@
|
||||
"""ADK Multi-Agent Architecture for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.agents primitives:
|
||||
- SourceDiscoveryAgent (LlmAgent)
|
||||
- DiscoveryAgent (LlmAgent)
|
||||
- ArchitectureDesignAgent (LlmAgent)
|
||||
- ValidationReviewAgent (LlmAgent)
|
||||
- PackagingAgent (LlmAgent)
|
||||
- OrchestratorLoopAgent (LoopAgent)
|
||||
"""
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from app.adk.compat import BaseAgent, LlmAgent, LoopAgent, SequentialAgent
|
||||
from app.adk.tools import ADK_TOOLS
|
||||
from app.config import get_settings
|
||||
from app.database import get_db_manager
|
||||
from app.nodes import design_node, discover_node, package_node, source_discover_node, validate_node
|
||||
from app.skills.loader import SkillLoader
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SourceDiscoveryAgent(LlmAgent):
|
||||
"""ADK Agent responsible for Phase 0a Pre-emptive Source Environment Discovery."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader) -> None:
|
||||
super().__init__(
|
||||
name="SourceDiscoveryAgent",
|
||||
description="Audits and documents the pre-existing source environment (As-Is Architecture) before target migration.",
|
||||
instruction=skill_loader.format_skills_for_prompt("source_discover"),
|
||||
tools=[],
|
||||
)
|
||||
self.skill_loader = skill_loader
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
logger.info("Executing SourceDiscoveryAgent")
|
||||
res = source_discover_node(state, self.skill_loader)
|
||||
state_copy = dict(state)
|
||||
state_copy.update(res)
|
||||
return state_copy
|
||||
|
||||
|
||||
class DiscoveryAgent(LlmAgent):
|
||||
"""ADK Agent responsible for Phase 0 Requirements Discovery."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader) -> None:
|
||||
super().__init__(
|
||||
name="DiscoveryAgent",
|
||||
description="Extracts functional and non-functional requirements with product selection deferred.",
|
||||
instruction=skill_loader.format_skills_for_prompt("discover"),
|
||||
tools=[],
|
||||
)
|
||||
self.skill_loader = skill_loader
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
logger.info("Executing DiscoveryAgent")
|
||||
res = discover_node(state, self.skill_loader)
|
||||
state_copy = dict(state)
|
||||
state_copy.update(res)
|
||||
return state_copy
|
||||
|
||||
|
||||
class ArchitectureDesignAgent(LlmAgent):
|
||||
"""ADK Agent responsible for Phase 1 Product Selection, Diagrams, & Terraform IaC."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader) -> None:
|
||||
super().__init__(
|
||||
name="ArchitectureDesignAgent",
|
||||
description="Selects GCP products, generates Mermaid diagram, and produces Terraform IaC grounded by Developer Knowledge MCP.",
|
||||
instruction=skill_loader.format_skills_for_prompt("design"),
|
||||
tools=ADK_TOOLS,
|
||||
)
|
||||
self.skill_loader = skill_loader
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
logger.info("Executing ArchitectureDesignAgent")
|
||||
res = design_node(state, self.skill_loader)
|
||||
state_copy = dict(state)
|
||||
state_copy.update(res)
|
||||
return state_copy
|
||||
|
||||
|
||||
class ValidationReviewAgent(LlmAgent):
|
||||
"""ADK Agent responsible for Phase 2 Artifact Review & Quality Validation."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader) -> None:
|
||||
super().__init__(
|
||||
name="ValidationReviewAgent",
|
||||
description="Validates Mermaid syntax, Terraform configuration, and required sections.",
|
||||
instruction=skill_loader.format_skills_for_prompt("validate"),
|
||||
tools=ADK_TOOLS,
|
||||
)
|
||||
self.skill_loader = skill_loader
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
logger.info("Executing ValidationReviewAgent")
|
||||
res = validate_node(state, self.skill_loader)
|
||||
state_copy = dict(state)
|
||||
state_copy.update(res)
|
||||
return state_copy
|
||||
|
||||
|
||||
class PackagingAgent(LlmAgent):
|
||||
"""ADK Agent responsible for Phase 3 Solution Guide Packaging."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader) -> None:
|
||||
super().__init__(
|
||||
name="PackagingAgent",
|
||||
description="Packages final solution-architecture-guide.md.",
|
||||
instruction=skill_loader.format_skills_for_prompt("package"),
|
||||
tools=[],
|
||||
)
|
||||
self.skill_loader = skill_loader
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
logger.info("Executing PackagingAgent")
|
||||
res = package_node(state, self.skill_loader)
|
||||
state_copy = dict(state)
|
||||
state_copy.update(res)
|
||||
return state_copy
|
||||
|
||||
|
||||
class OrchestratorLoopAgent(LoopAgent):
|
||||
"""ADK Orchestrator LoopAgent that coordinates multi-agent execution & iterative quality review.
|
||||
|
||||
Uses google.adk.agents.LoopAgent to execute discovery -> design -> validation review -> packaging
|
||||
in a loop until validation passes 100% or max_iterations is reached. Writes every review iteration
|
||||
event to PostgreSQL database (`orchestrator_review_logs`).
|
||||
"""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader, max_iterations: Optional[int] = None) -> None:
|
||||
settings = get_settings()
|
||||
max_iters = max_iterations or settings.ADK_MAX_LOOP_ITERATIONS
|
||||
self.db_manager = get_db_manager()
|
||||
|
||||
self.source_discovery_agent = SourceDiscoveryAgent(skill_loader)
|
||||
self.discovery_agent = DiscoveryAgent(skill_loader)
|
||||
self.design_agent = ArchitectureDesignAgent(skill_loader)
|
||||
self.validation_agent = ValidationReviewAgent(skill_loader)
|
||||
self.packaging_agent = PackagingAgent(skill_loader)
|
||||
|
||||
sub_pipeline = SequentialAgent(
|
||||
name="MultiAgentGCPPipeline",
|
||||
sub_agents=[
|
||||
self.source_discovery_agent,
|
||||
self.discovery_agent,
|
||||
self.design_agent,
|
||||
self.validation_agent,
|
||||
self.packaging_agent,
|
||||
],
|
||||
description="Sequential pipeline of GCP architecture multi-agents.",
|
||||
)
|
||||
|
||||
def review_validator(state: Dict[str, Any]) -> bool:
|
||||
"""Check if solution meets production quality validation standards."""
|
||||
is_valid = state.get("validation_passed", False)
|
||||
errors = state.get("errors", [])
|
||||
iteration = state.get("loop_count", 1)
|
||||
execution_id = state.get("execution_id", str(uuid.uuid4()))
|
||||
|
||||
status_str = "APPROVED" if is_valid else "NEEDS_REVISION"
|
||||
feedback_str = "All architecture validation rules passed." if is_valid else f"Validation errors: {', '.join(errors)}"
|
||||
|
||||
# Record review iteration in PostgreSQL database
|
||||
self.db_manager.record_orchestrator_log(
|
||||
log_id=str(uuid.uuid4()),
|
||||
execution_id=execution_id,
|
||||
iteration=iteration,
|
||||
review_status=status_str,
|
||||
reviewer_agent="ValidationReviewAgent",
|
||||
feedback=feedback_str,
|
||||
)
|
||||
logger.info(
|
||||
"Orchestrator Review Loop #%d: status=%s, valid=%s",
|
||||
iteration,
|
||||
status_str,
|
||||
is_valid,
|
||||
)
|
||||
|
||||
return is_valid
|
||||
|
||||
super().__init__(
|
||||
name="OrchestratorLoopAgent",
|
||||
sub_agent=sub_pipeline,
|
||||
max_iterations=max_iters,
|
||||
description="Production-ready multi-agent orchestrator loop agent.",
|
||||
validator_fn=review_validator,
|
||||
)
|
||||
|
||||
|
||||
def build_adk_multi_agent_system(skill_loader: SkillLoader, max_iterations: Optional[int] = None) -> OrchestratorLoopAgent:
|
||||
"""Factory function for building the complete ADK Orchestrator LoopAgent system."""
|
||||
return OrchestratorLoopAgent(skill_loader, max_iterations=max_iterations)
|
||||
@@ -1,77 +0,0 @@
|
||||
"""ADK Artifact Management for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.artifacts.ArtifactRepository backed by local filesystem and PostgreSQL persistence.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional, List
|
||||
|
||||
from app.adk.compat import Artifact, ArtifactRepository
|
||||
from app.database import get_db_manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PostgresArtifactRepository(ArtifactRepository):
|
||||
"""ADK Artifact Repository with PostgreSQL database & filesystem synchronization."""
|
||||
|
||||
def __init__(self, base_dir: Optional[Path] = None) -> None:
|
||||
super().__init__()
|
||||
self.base_dir = base_dir or Path(".").resolve()
|
||||
self.db_manager = get_db_manager()
|
||||
|
||||
def save_solution_artifacts(self, execution_id: str, state: Dict[str, Any]) -> List[Artifact]:
|
||||
"""Write workflow execution output artifacts to disk and PostgreSQL database."""
|
||||
saved_artifacts: List[Artifact] = []
|
||||
|
||||
artifact_mapping = [
|
||||
# Structured scalable active deliverables
|
||||
("source_discovery_doc", "deliverables/as-is/source-architecture.md", "markdown"),
|
||||
("source_mermaid_diagram", "deliverables/as-is/source-architecture.mmd", "mermaid"),
|
||||
("requirements_doc", "deliverables/target/requirements.md", "markdown"),
|
||||
("architecture_doc", "deliverables/target/architecture.md", "markdown"),
|
||||
("mermaid_diagram", "deliverables/target/architecture.mmd", "mermaid"),
|
||||
("terraform_code", "terraform/main.tf", "hcl"),
|
||||
("validation_results", "deliverables/validation/validation-results.md", "markdown"),
|
||||
("solution_guide", "deliverables/guides/solution-architecture-guide.md", "markdown"),
|
||||
|
||||
# Per-execution isolated deliverables for scalable history tracking
|
||||
("source_discovery_doc", f"deliverables/executions/{execution_id}/as-is/source-architecture.md", "markdown"),
|
||||
("source_mermaid_diagram", f"deliverables/executions/{execution_id}/as-is/source-architecture.mmd", "mermaid"),
|
||||
("requirements_doc", f"deliverables/executions/{execution_id}/target/requirements.md", "markdown"),
|
||||
("architecture_doc", f"deliverables/executions/{execution_id}/target/architecture.md", "markdown"),
|
||||
("mermaid_diagram", f"deliverables/executions/{execution_id}/target/architecture.mmd", "mermaid"),
|
||||
("validation_results", f"deliverables/executions/{execution_id}/validation/validation-results.md", "markdown"),
|
||||
("solution_guide", f"deliverables/executions/{execution_id}/guides/solution-architecture-guide.md", "markdown"),
|
||||
|
||||
# Mirror docs for specification compatibility
|
||||
("source_discovery_doc", "docs/source-architecture.md", "markdown"),
|
||||
("requirements_doc", "docs/requirements.md", "markdown"),
|
||||
("architecture_doc", "docs/architecture.md", "markdown"),
|
||||
]
|
||||
|
||||
for state_key, rel_path, art_type in artifact_mapping:
|
||||
content = state.get(state_key)
|
||||
if not content:
|
||||
continue
|
||||
|
||||
full_path = self.base_dir / rel_path
|
||||
full_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
full_path.write_text(content, encoding="utf-8")
|
||||
|
||||
# Save in ADK memory store
|
||||
art = self.save_artifact(art_type, str(rel_path), content)
|
||||
saved_artifacts.append(art)
|
||||
|
||||
# Persist to PostgreSQL database
|
||||
self.db_manager.save_artifact(
|
||||
artifact_id=art.artifact_id,
|
||||
execution_id=execution_id,
|
||||
artifact_type=art_type,
|
||||
file_path=str(rel_path),
|
||||
content=content,
|
||||
)
|
||||
logger.info("Persisted ADK artifact '%s' (%s) to DB and %s", art_type, art.artifact_id, rel_path)
|
||||
|
||||
return saved_artifacts
|
||||
@@ -1,288 +0,0 @@
|
||||
"""Google ADK (Agent Development Kit) Compatibility & Abstraction Layer.
|
||||
|
||||
Re-exports native `google.adk` framework components when available, or provides
|
||||
fully functional compatibility stubs matching ADK interfaces for:
|
||||
- google.adk.agents (Agent, LlmAgent, SequentialAgent, ParallelAgent, LoopAgent)
|
||||
- google.adk.workflows (Workflow, WorkflowStep)
|
||||
- google.adk.runners (Runner)
|
||||
- google.adk.sessions (Session, SessionService, InMemorySessionService, PostgresSessionService)
|
||||
- google.adk.artifacts (Artifact, ArtifactRepository)
|
||||
- google.adk.tools (Tool, FunctionTool)
|
||||
- google.adk.evaluation (Evaluator, BenchmarkRunner)
|
||||
"""
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Attempt importing native google.adk modules
|
||||
HAS_NATIVE_ADK = False
|
||||
|
||||
try:
|
||||
import google.adk.agents as _native_agents
|
||||
import google.adk.workflows as _native_workflows
|
||||
import google.adk.runners as _native_runners
|
||||
import google.adk.sessions as _native_sessions
|
||||
import google.adk.artifacts as _native_artifacts
|
||||
import google.adk.tools as _native_tools
|
||||
import google.adk.evaluation as _native_evaluation
|
||||
|
||||
HAS_NATIVE_ADK = True
|
||||
logger.info("Successfully imported native google.adk framework modules.")
|
||||
except ImportError:
|
||||
HAS_NATIVE_ADK = False
|
||||
logger.info("Native google.adk not installed; using ADK framework compatibility layer.")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. google.adk.agents
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class BaseAgent:
|
||||
"""Base class for ADK Agents."""
|
||||
|
||||
def __init__(self, name: str, description: str = "", instruction: str = "", tools: Optional[List[Any]] = None) -> None:
|
||||
self.name = name
|
||||
self.description = description
|
||||
self.instruction = instruction
|
||||
self.tools = tools or []
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Execute agent logic against state."""
|
||||
return state
|
||||
|
||||
|
||||
class LlmAgent(BaseAgent):
|
||||
"""ADK LLM Agent primitive."""
|
||||
|
||||
def __init__(self, name: str, description: str = "", instruction: str = "", model: Any = None, tools: Optional[List[Any]] = None) -> None:
|
||||
super().__init__(name, description, instruction, tools)
|
||||
self.model = model
|
||||
|
||||
|
||||
class SequentialAgent(BaseAgent):
|
||||
"""ADK Sequential Workflow Agent."""
|
||||
|
||||
def __init__(self, name: str, sub_agents: List[BaseAgent], description: str = "") -> None:
|
||||
super().__init__(name, description)
|
||||
self.sub_agents = sub_agents
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
curr_state = dict(state)
|
||||
for agent in self.sub_agents:
|
||||
curr_state = agent.execute(curr_state)
|
||||
return curr_state
|
||||
|
||||
|
||||
class ParallelAgent(BaseAgent):
|
||||
"""ADK Parallel Workflow Agent."""
|
||||
|
||||
def __init__(self, name: str, sub_agents: List[BaseAgent], description: str = "") -> None:
|
||||
super().__init__(name, description)
|
||||
self.sub_agents = sub_agents
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
curr_state = dict(state)
|
||||
for agent in self.sub_agents:
|
||||
res = agent.execute(curr_state)
|
||||
curr_state.update(res)
|
||||
return curr_state
|
||||
|
||||
|
||||
class LoopAgent(BaseAgent):
|
||||
"""ADK Loop Agent for iterative review and validation feedback cycles."""
|
||||
|
||||
def __init__(self, name: str, sub_agent: BaseAgent, max_iterations: int = 5, description: str = "", validator_fn: Optional[Callable[[Dict[str, Any]], bool]] = None) -> None:
|
||||
super().__init__(name, description)
|
||||
self.sub_agent = sub_agent
|
||||
self.max_iterations = max_iterations
|
||||
self.validator_fn = validator_fn
|
||||
|
||||
def execute(self, state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
curr_state = dict(state)
|
||||
loop_count = 0
|
||||
while loop_count < self.max_iterations:
|
||||
loop_count += 1
|
||||
curr_state["loop_count"] = loop_count
|
||||
curr_state = self.sub_agent.execute(curr_state)
|
||||
if self.validator_fn and self.validator_fn(curr_state):
|
||||
curr_state["loop_completed_successfully"] = True
|
||||
break
|
||||
curr_state["total_loop_iterations"] = loop_count
|
||||
return curr_state
|
||||
|
||||
|
||||
# Alias Agent to LlmAgent / BaseAgent
|
||||
Agent = LlmAgent
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. google.adk.tools
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Tool:
|
||||
"""ADK Tool Interface."""
|
||||
|
||||
def __init__(self, name: str, description: str, func: Callable) -> None:
|
||||
self.name = name
|
||||
self.description = description
|
||||
self.func = func
|
||||
|
||||
def run(self, *args, **kwargs) -> Any:
|
||||
return self.func(*args, **kwargs)
|
||||
|
||||
|
||||
class FunctionTool(Tool):
|
||||
"""ADK Function Tool primitive."""
|
||||
|
||||
@classmethod
|
||||
def from_defaults(cls, fn: Callable, name: Optional[str] = None, description: Optional[str] = None) -> "FunctionTool":
|
||||
tool_name = name or fn.__name__
|
||||
tool_desc = description or (fn.__doc__ or "")
|
||||
return cls(name=tool_name, description=tool_desc, func=fn)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. google.adk.artifacts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Artifact:
|
||||
"""ADK Artifact container."""
|
||||
|
||||
def __init__(self, artifact_id: str, artifact_type: str, file_path: str, content: str) -> None:
|
||||
self.artifact_id = artifact_id
|
||||
self.artifact_type = artifact_type
|
||||
self.file_path = file_path
|
||||
self.content = content
|
||||
|
||||
|
||||
class ArtifactRepository:
|
||||
"""ADK Artifact Repository interface."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._store: Dict[str, Artifact] = {}
|
||||
|
||||
def save_artifact(self, artifact_type: str, file_path: str, content: str) -> Artifact:
|
||||
artifact_id = str(uuid.uuid4())
|
||||
art = Artifact(artifact_id, artifact_type, file_path, content)
|
||||
self._store[artifact_id] = art
|
||||
return art
|
||||
|
||||
def get_artifact(self, artifact_id: str) -> Optional[Artifact]:
|
||||
return self._store.get(artifact_id)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. google.adk.sessions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Session:
|
||||
"""ADK Session object."""
|
||||
|
||||
def __init__(self, session_id: str, agent_name: str, state: Optional[Dict[str, Any]] = None) -> None:
|
||||
self.session_id = session_id
|
||||
self.agent_name = agent_name
|
||||
self.state = state or {}
|
||||
|
||||
|
||||
class SessionService:
|
||||
"""ADK Session Service interface."""
|
||||
|
||||
def create_session(self, agent_name: str, session_id: Optional[str] = None) -> Session:
|
||||
sid = session_id or str(uuid.uuid4())
|
||||
return Session(sid, agent_name)
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Session]:
|
||||
raise NotImplementedError
|
||||
|
||||
def save_session(self, session: Session) -> bool:
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class InMemorySessionService(SessionService):
|
||||
"""In-memory ADK Session Service."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._sessions: Dict[str, Session] = {}
|
||||
|
||||
def create_session(self, agent_name: str, session_id: Optional[str] = None) -> Session:
|
||||
sid = session_id or str(uuid.uuid4())
|
||||
sess = Session(sid, agent_name)
|
||||
self._sessions[sid] = sess
|
||||
return sess
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Session]:
|
||||
return self._sessions.get(session_id)
|
||||
|
||||
def save_session(self, session: Session) -> bool:
|
||||
self._sessions[session.session_id] = session
|
||||
return True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. google.adk.runners
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Runner:
|
||||
"""ADK Runner for executing agents and workflows."""
|
||||
|
||||
def __init__(self, agent: BaseAgent, session_service: Optional[SessionService] = None) -> None:
|
||||
self.agent = agent
|
||||
self.session_service = session_service or InMemorySessionService()
|
||||
|
||||
def run(self, session_id: str, input_state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
session = self.session_service.get_session(session_id)
|
||||
if not session:
|
||||
session = self.session_service.create_session(self.agent.name, session_id)
|
||||
|
||||
merged_state = {**session.state, **input_state}
|
||||
result_state = self.agent.execute(merged_state)
|
||||
session.state = result_state
|
||||
self.session_service.save_session(session)
|
||||
return result_state
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. google.adk.workflows
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class WorkflowStep:
|
||||
"""ADK Workflow Step."""
|
||||
|
||||
def __init__(self, step_name: str, agent: BaseAgent) -> None:
|
||||
self.step_name = step_name
|
||||
self.agent = agent
|
||||
|
||||
|
||||
class Workflow:
|
||||
"""ADK Workflow composition container."""
|
||||
|
||||
def __init__(self, name: str, steps: List[WorkflowStep]) -> None:
|
||||
self.name = name
|
||||
self.steps = steps
|
||||
|
||||
def execute(self, initial_state: Dict[str, Any]) -> Dict[str, Any]:
|
||||
state = dict(initial_state)
|
||||
for step in self.steps:
|
||||
state = step.agent.execute(state)
|
||||
return state
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. google.adk.evaluation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class Evaluator:
|
||||
"""ADK Benchmark Evaluator."""
|
||||
|
||||
def evaluate(self, agent: BaseAgent, test_case: Dict[str, Any]) -> Dict[str, Any]:
|
||||
initial_state = test_case.get("input_state", {})
|
||||
result = agent.execute(initial_state)
|
||||
return {
|
||||
"case_id": test_case.get("case_id", "default"),
|
||||
"result_state": result,
|
||||
"passed": True,
|
||||
"score": 1.0,
|
||||
}
|
||||
@@ -1,57 +0,0 @@
|
||||
"""ADK Evaluation module for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.evaluation.Evaluator with PostgreSQL database persistence.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from app.adk.compat import Evaluator
|
||||
from app.adk.runners import ADKAgentRunner
|
||||
from app.database import get_db_manager
|
||||
from eval.metrics import evaluate_case_run
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ADKEvaluator(Evaluator):
|
||||
"""ADK Evaluator persisting evaluation results to PostgreSQL."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.runner = ADKAgentRunner()
|
||||
self.db_manager = get_db_manager()
|
||||
|
||||
def evaluate_benchmark_case(self, case: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Execute and score benchmark case via ADK runner and record to PostgreSQL."""
|
||||
eval_id = str(uuid.uuid4())
|
||||
case_id = case.get("id", "case-unknown")
|
||||
req_summary = case.get("workflow_request", "")
|
||||
|
||||
run_res = self.runner.run_execution(request_summary=req_summary)
|
||||
artifacts = run_res.get("artifacts", {})
|
||||
|
||||
state_for_metrics = {
|
||||
"requirements_doc": artifacts.get("requirements_doc", ""),
|
||||
"architecture_doc": artifacts.get("architecture_doc", ""),
|
||||
"mermaid_diagram": artifacts.get("mermaid_diagram", ""),
|
||||
"terraform_code": artifacts.get("terraform_code", ""),
|
||||
"solution_guide": artifacts.get("solution_guide", ""),
|
||||
}
|
||||
|
||||
eval_result = evaluate_case_run(state_for_metrics, case)
|
||||
|
||||
# Save evaluation to PostgreSQL database
|
||||
self.db_manager.save_evaluation(
|
||||
eval_id=eval_id,
|
||||
case_id=case_id,
|
||||
total_score=eval_result["total_score"],
|
||||
max_score=eval_result["max_score"],
|
||||
pass_rate=eval_result["percentage"],
|
||||
passed=eval_result["passed"],
|
||||
details=eval_result,
|
||||
)
|
||||
|
||||
logger.info("Recorded ADK evaluation %s for case %s (Score: %.1f%%)", eval_id, case_id, eval_result["percentage"])
|
||||
return eval_result
|
||||
@@ -12,7 +12,7 @@ from collections.abc import Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import (
|
||||
from app.adk.runtime import (
|
||||
AgentTask,
|
||||
BudgetExhausted,
|
||||
Outcome,
|
||||
@@ -9,10 +9,10 @@ from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from app.agent_skills import SkillFormatError, split_frontmatter
|
||||
from app.agents.runtime import RoleSpec
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||
from app.adk.runtime import RoleSpec
|
||||
from app.providers import ProviderBundle
|
||||
from app.skills.agent_skills import SkillFormatError, split_frontmatter
|
||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||
|
||||
ORCHESTRATOR = "orchestrator"
|
||||
PILLAR_ADVISOR = "pillar-advisor"
|
||||
@@ -1,93 +0,0 @@
|
||||
"""ADK Execution Runner for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.runners.Runner integrated with PostgresSessionService,
|
||||
PostgresArtifactRepository, and PostgreSQL database state updates.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from app.adk.agents import build_adk_multi_agent_system
|
||||
from app.adk.artifacts import PostgresArtifactRepository
|
||||
from app.adk.compat import Runner
|
||||
from app.adk.sessions import PostgresSessionService
|
||||
from app.config import get_settings
|
||||
from app.database import get_db_manager
|
||||
from app.skills.loader import SkillLoader
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ADKAgentRunner:
|
||||
"""Production-ready ADK Execution Runner."""
|
||||
|
||||
def __init__(self, skill_loader: SkillLoader | None = None) -> None:
|
||||
settings = get_settings()
|
||||
self.skill_loader = skill_loader or SkillLoader(settings.SKILLS_DIR)
|
||||
self.skill_loader.load_skills()
|
||||
|
||||
self.session_service = PostgresSessionService()
|
||||
self.artifact_repo = PostgresArtifactRepository(settings.BASE_DIR)
|
||||
self.db_manager = get_db_manager()
|
||||
|
||||
self.orchestrator = build_adk_multi_agent_system(
|
||||
self.skill_loader,
|
||||
max_iterations=settings.ADK_MAX_LOOP_ITERATIONS,
|
||||
)
|
||||
self.runner = Runner(agent=self.orchestrator, session_service=self.session_service)
|
||||
|
||||
def run_execution(self, session_id: Optional[str] = None, request_summary: str = "", target_dir: str = ".") -> Dict[str, Any]:
|
||||
"""Execute multi-agent workflow using ADK runner with PostgreSQL state persistence."""
|
||||
sid = session_id or str(uuid.uuid4())
|
||||
execution_id = str(uuid.uuid4())
|
||||
|
||||
initial_input = {
|
||||
"execution_id": execution_id,
|
||||
"workflow_request": request_summary or "Event-driven regional HTTP application reference architecture",
|
||||
"target_dir": target_dir,
|
||||
"active_skills": [],
|
||||
}
|
||||
|
||||
logger.info("Starting ADK runner execution %s for session %s", execution_id, sid)
|
||||
|
||||
# Run through ADK Runner
|
||||
result_state = self.runner.run(session_id=sid, input_state=initial_input)
|
||||
|
||||
# Save artifacts to disk & PostgreSQL database
|
||||
saved_artifacts = self.artifact_repo.save_solution_artifacts(execution_id, result_state)
|
||||
|
||||
# Record execution in PostgreSQL database
|
||||
self.db_manager.save_workflow_execution(
|
||||
execution_id=execution_id,
|
||||
session_id=sid,
|
||||
status="completed" if result_state.get("validation_passed") else "failed",
|
||||
current_phase=result_state.get("current_phase", "package"),
|
||||
loop_count=result_state.get("total_loop_iterations", 1),
|
||||
request_summary=request_summary,
|
||||
results={
|
||||
"validation_passed": result_state.get("validation_passed", False),
|
||||
"active_skills": result_state.get("active_skills", []),
|
||||
"artifacts_saved": len(saved_artifacts),
|
||||
},
|
||||
)
|
||||
|
||||
return {
|
||||
"execution_id": execution_id,
|
||||
"session_id": sid,
|
||||
"status": "success" if result_state.get("validation_passed") else "completed_with_warnings",
|
||||
"validation_passed": result_state.get("validation_passed", False),
|
||||
"total_loop_iterations": result_state.get("total_loop_iterations", 1),
|
||||
"current_phase": result_state.get("current_phase", "package"),
|
||||
"artifacts": {
|
||||
"source_discovery_doc": result_state.get("source_discovery_doc"),
|
||||
"source_mermaid_diagram": result_state.get("source_mermaid_diagram"),
|
||||
"requirements_doc": result_state.get("requirements_doc"),
|
||||
"architecture_doc": result_state.get("architecture_doc"),
|
||||
"mermaid_diagram": result_state.get("mermaid_diagram"),
|
||||
"terraform_code": result_state.get("terraform_code"),
|
||||
"validation_results": result_state.get("validation_results"),
|
||||
"solution_guide": result_state.get("solution_guide"),
|
||||
},
|
||||
"active_skills": result_state.get("active_skills", []),
|
||||
}
|
||||
@@ -86,6 +86,38 @@ class ToolResult:
|
||||
outcome: Outcome | None = None
|
||||
|
||||
|
||||
# Files cross tool calls as a list of {path, content}, never a map keyed by path:
|
||||
# models rewrite object keys (Gemini turns "main.tf" into "main_tf").
|
||||
FILES_SCHEMA: Mapping[str, Any] = {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "Relative path, e.g. terraform/main.tf",
|
||||
},
|
||||
"content": {"type": "string"},
|
||||
},
|
||||
"required": ["path", "content"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def files_from_argument(value: Any) -> dict[str, str]:
|
||||
"""A FILES_SCHEMA argument as {path: content}. A {path: content} map is accepted too."""
|
||||
if not value:
|
||||
return {}
|
||||
if isinstance(value, Mapping):
|
||||
return {str(path): str(content) for path, content in value.items()}
|
||||
files: dict[str, str] = {}
|
||||
for item in value:
|
||||
if not isinstance(item, Mapping) or "path" not in item:
|
||||
raise TypeError("each file needs a 'path' and 'content'")
|
||||
files[str(item["path"])] = str(item.get("content", ""))
|
||||
return files
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ToolSpec:
|
||||
name: str
|
||||
@@ -95,6 +127,10 @@ class ToolSpec:
|
||||
tier: str = "general" # general | cloud_read | execution
|
||||
|
||||
|
||||
# The tools a role gets beyond the Execution tools, given the Deliverable it is producing.
|
||||
ToolFactory = Callable[[RoleSpec, str | None], Sequence[ToolSpec]]
|
||||
|
||||
|
||||
class AgentRuntime(Protocol):
|
||||
async def run(
|
||||
self, role: RoleSpec, task: AgentTask, tools: Sequence[ToolSpec]
|
||||
@@ -1,56 +0,0 @@
|
||||
"""ADK Postgres Session Service for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.sessions.SessionService backed by external PostgreSQL database.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from app.adk.compat import Session, SessionService
|
||||
from app.database import get_db_manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PostgresSessionService(SessionService):
|
||||
"""ADK Session Service persisting state to PostgreSQL."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.db_manager = get_db_manager()
|
||||
|
||||
def create_session(self, agent_name: str, session_id: Optional[str] = None) -> Session:
|
||||
"""Create a new session record in PostgreSQL database."""
|
||||
sid = session_id or str(uuid.uuid4())
|
||||
session = Session(session_id=sid, agent_name=agent_name, state={})
|
||||
self.db_manager.save_session(
|
||||
session_id=sid,
|
||||
agent_name=agent_name,
|
||||
state_data=session.state,
|
||||
metadata={"created_via": "PostgresSessionService"},
|
||||
)
|
||||
logger.info("Created ADK session %s in PostgreSQL database.", sid)
|
||||
return session
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Session]:
|
||||
"""Fetch session state from PostgreSQL database."""
|
||||
sess_data = self.db_manager.get_session(session_id)
|
||||
if not sess_data:
|
||||
return None
|
||||
|
||||
return Session(
|
||||
session_id=sess_data["session_id"],
|
||||
agent_name=sess_data["agent_name"],
|
||||
state=sess_data.get("state_data", {}),
|
||||
)
|
||||
|
||||
def save_session(self, session: Session) -> bool:
|
||||
"""Update session state in PostgreSQL database."""
|
||||
success = self.db_manager.save_session(
|
||||
session_id=session.session_id,
|
||||
agent_name=session.agent_name,
|
||||
state_data=session.state,
|
||||
)
|
||||
if success:
|
||||
logger.info("Updated ADK session %s in PostgreSQL database.", session.session_id)
|
||||
return success
|
||||
@@ -1,64 +0,0 @@
|
||||
"""ADK FunctionTool Wrappers for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.tools.FunctionTool primitives.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
from app.adk.compat import FunctionTool
|
||||
from app.tools.mcp_developer_knowledge import (
|
||||
developerknowledge_answer_query as _mcp_answer,
|
||||
developerknowledge_get_documents as _mcp_get,
|
||||
developerknowledge_search_documents as _mcp_search,
|
||||
)
|
||||
from app.tools.validation_tools import (
|
||||
validate_mermaid_diagram as _validate_mermaid,
|
||||
validate_repository_artifacts as _validate_repo,
|
||||
validate_terraform_syntax as _validate_terraform,
|
||||
)
|
||||
|
||||
# Convert validation functions to ADK FunctionTools
|
||||
mermaid_tool = FunctionTool.from_defaults(
|
||||
fn=_validate_mermaid.func if hasattr(_validate_mermaid, "func") else _validate_mermaid,
|
||||
name="validate_mermaid_diagram",
|
||||
description="Validates syntax and graph directives of a Mermaid diagram.",
|
||||
)
|
||||
|
||||
terraform_tool = FunctionTool.from_defaults(
|
||||
fn=_validate_terraform.func if hasattr(_validate_terraform, "func") else _validate_terraform,
|
||||
name="validate_terraform_syntax",
|
||||
description="Validates Terraform HCL basic structure and required Google Cloud resources.",
|
||||
)
|
||||
|
||||
repo_artifacts_tool = FunctionTool.from_defaults(
|
||||
fn=_validate_repo.func if hasattr(_validate_repo, "func") else _validate_repo,
|
||||
name="validate_repository_artifacts",
|
||||
description="Validates offline file existence and required section headings across repository artifacts.",
|
||||
)
|
||||
|
||||
# Convert Developer Knowledge MCP tools to ADK FunctionTools
|
||||
mcp_search_tool = FunctionTool.from_defaults(
|
||||
fn=_mcp_search.func if hasattr(_mcp_search, "func") else _mcp_search,
|
||||
name="developerknowledge_search_documents",
|
||||
description="Searches Google Cloud reference architecture, decision-making, and best-practice documents.",
|
||||
)
|
||||
|
||||
mcp_get_tool = FunctionTool.from_defaults(
|
||||
fn=_mcp_get.func if hasattr(_mcp_get, "func") else _mcp_get,
|
||||
name="developerknowledge_get_documents",
|
||||
description="Retrieves official Google Cloud document content and citations by URI.",
|
||||
)
|
||||
|
||||
mcp_answer_tool = FunctionTool.from_defaults(
|
||||
fn=_mcp_answer.func if hasattr(_mcp_answer, "func") else _mcp_answer,
|
||||
name="developerknowledge_answer_query",
|
||||
description="Answers architectural questions and checks GCP product release statuses and best practices.",
|
||||
)
|
||||
|
||||
ADK_TOOLS = [
|
||||
mermaid_tool,
|
||||
terraform_tool,
|
||||
repo_artifacts_tool,
|
||||
mcp_search_tool,
|
||||
mcp_get_tool,
|
||||
mcp_answer_tool,
|
||||
]
|
||||
@@ -1,20 +0,0 @@
|
||||
"""ADK Workflows module for GCP Solution Architecture Agent.
|
||||
|
||||
Uses google.adk.workflows.Workflow and google.adk.workflows.WorkflowStep primitives.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List
|
||||
from app.adk.agents import OrchestratorLoopAgent
|
||||
from app.adk.compat import Workflow, WorkflowStep
|
||||
from app.skills.loader import SkillLoader
|
||||
|
||||
|
||||
def create_gcp_adk_workflow(skill_loader: SkillLoader, max_iterations: int = 5) -> Workflow:
|
||||
"""Compose the multi-agent ADK workflow."""
|
||||
orchestrator = OrchestratorLoopAgent(skill_loader, max_iterations=max_iterations)
|
||||
step = WorkflowStep(step_name="OrchestratorReviewLoopStep", agent=orchestrator)
|
||||
|
||||
return Workflow(
|
||||
name="GCP_Solution_Architecture_Workflow",
|
||||
steps=[step],
|
||||
)
|
||||
280
app/agent.py
280
app/agent.py
@@ -1,61 +1,243 @@
|
||||
"""gcp_solution_architecture_agent: Starlette application wiring & CLI entrypoint."""
|
||||
"""The agent: the ExecutionEngine that starts and resumes Executions, and its assembly.
|
||||
|
||||
ExecutionEngine is the one interface the A2A server, /generate and the evaluation
|
||||
harness use. Each call runs the Execution until it needs the user or finishes, and
|
||||
returns an Interaction describing which.
|
||||
|
||||
open_assembly is the composition root: the only place concrete runtimes, tools,
|
||||
checkpointers and recorders are chosen, from settings and the provider bundle.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import AsyncIterator, Mapping, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
import logging
|
||||
import click
|
||||
import uvicorn
|
||||
from starlette.applications import Starlette
|
||||
from starlette.routing import Route
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from app.config import get_settings
|
||||
from app.database import get_db_manager
|
||||
from app.workflows.routes import (
|
||||
get_card,
|
||||
get_session_route,
|
||||
health_check,
|
||||
run_workflow,
|
||||
validate_artifacts_route,
|
||||
)
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||
from langgraph.types import Command
|
||||
|
||||
settings = get_settings()
|
||||
from app.adk.roles import load_roles
|
||||
from app.adk.runtime import AgentRuntime, RoleSpec, ToolFactory
|
||||
from app.checkpointing import open_checkpointer
|
||||
from app.config import Settings
|
||||
from app.providers import ProviderBundle, load_provider
|
||||
from app.recorder import DatabaseRecorder, ExecutionRecorder
|
||||
from app.states.ledger import Ledger
|
||||
from app.tools.execution import ORCHESTRATOR
|
||||
from app.tools.local import TerraformRunner, local_tools
|
||||
from app.tools.mcp import McpCatalog, auth_from_config
|
||||
from app.tools.toolkit import make_tool_factory
|
||||
from app.workflows.deliverables import DEPENDENCY_GRAPH
|
||||
from app.workflows.execution_graph import build_execution_graph
|
||||
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, settings.LOG_LEVEL.upper(), logging.INFO),
|
||||
format="%(asctime)s %(name)s %(levelname)s %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
_RECURSION_LIMIT = 10_000 # loops are bounded by max_orchestrator_turns instead
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Interaction:
|
||||
execution_id: str
|
||||
status: str # awaiting_user | completed | completed_with_warnings
|
||||
prompt: Mapping[str, Any] | None # questions | approval | permission
|
||||
state: Mapping[str, Any]
|
||||
|
||||
|
||||
class ExecutionNotAwaitingUser(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class ExecutionEngine:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
recorder: ExecutionRecorder,
|
||||
checkpointer: BaseCheckpointSaver,
|
||||
output_root: Path,
|
||||
max_revisions: int = 3,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> None:
|
||||
required = {
|
||||
ORCHESTRATOR,
|
||||
*(spec.specialist for spec in DEPENDENCY_GRAPH.values()),
|
||||
}
|
||||
missing = sorted(required - set(roles))
|
||||
if missing:
|
||||
raise ValueError(f"Missing roles: {', '.join(missing)}")
|
||||
self._max_revisions = max_revisions
|
||||
self._graph = build_execution_graph(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder,
|
||||
output_root=output_root,
|
||||
max_orchestrator_turns=max_orchestrator_turns,
|
||||
tool_factory=tool_factory,
|
||||
).compile(checkpointer=checkpointer)
|
||||
|
||||
async def start(
|
||||
self,
|
||||
workflow_request: str,
|
||||
*,
|
||||
session_id: str | None = None,
|
||||
pre_approved: bool = False,
|
||||
allow_cloud_read: bool | None = None,
|
||||
execution_id: str | None = None,
|
||||
scripted_answers: Sequence[tuple[str, str]] = (),
|
||||
) -> Interaction:
|
||||
"""Start an Execution. `scripted_answers` are (regex, answer) pairs a
|
||||
pre-approved Execution uses to answer questions, for evaluation runs."""
|
||||
execution_id = execution_id or str(uuid.uuid4())
|
||||
if await self._exists(execution_id):
|
||||
raise ValueError(f"Execution {execution_id} already exists")
|
||||
if pre_approved and allow_cloud_read is None:
|
||||
allow_cloud_read = False
|
||||
initial: dict[str, Any] = {
|
||||
"execution_id": execution_id,
|
||||
"session_id": session_id or str(uuid.uuid4()),
|
||||
"workflow_request": workflow_request,
|
||||
"pre_approved": pre_approved,
|
||||
"allow_cloud_read": allow_cloud_read,
|
||||
"ledger": Ledger(
|
||||
pre_approved=pre_approved, max_revisions=self._max_revisions
|
||||
).to_dict(),
|
||||
"deliverables": {},
|
||||
"conversation": [{"from": "user", "text": workflow_request}],
|
||||
"notes": [],
|
||||
"pending_questions": [],
|
||||
"awaiting_questions": [],
|
||||
"answers": {},
|
||||
"pending_permission": None,
|
||||
"prompt": None,
|
||||
"outcomes": [],
|
||||
"outcomes_applied": 0,
|
||||
"events_recorded": 0,
|
||||
"turns": 0,
|
||||
"status": "running",
|
||||
"warnings": [],
|
||||
"scripted_answers": [
|
||||
[pattern, answer] for pattern, answer in scripted_answers
|
||||
],
|
||||
}
|
||||
await self._graph.ainvoke(initial, self._config(execution_id))
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
|
||||
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
|
||||
current = await self.interaction(execution_id)
|
||||
if current.status != "awaiting_user":
|
||||
raise ExecutionNotAwaitingUser(
|
||||
f"Execution {execution_id} is {current.status}, not awaiting the user"
|
||||
)
|
||||
await self._graph.ainvoke(
|
||||
Command(resume=dict(reply)), self._config(execution_id)
|
||||
)
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def interaction(self, execution_id: str) -> Interaction:
|
||||
snapshot = await self._graph.aget_state(self._config(execution_id))
|
||||
if not snapshot.values:
|
||||
raise KeyError(f"Unknown Execution {execution_id}")
|
||||
if snapshot.interrupts:
|
||||
return Interaction(
|
||||
execution_id,
|
||||
"awaiting_user",
|
||||
snapshot.interrupts[0].value,
|
||||
snapshot.values,
|
||||
)
|
||||
return Interaction(
|
||||
execution_id,
|
||||
snapshot.values.get("status", "running"),
|
||||
None,
|
||||
snapshot.values,
|
||||
)
|
||||
|
||||
async def _exists(self, execution_id: str) -> bool:
|
||||
snapshot = await self._graph.aget_state(self._config(execution_id))
|
||||
return bool(snapshot.values)
|
||||
|
||||
@staticmethod
|
||||
def _config(execution_id: str) -> RunnableConfig:
|
||||
return {
|
||||
"configurable": {"thread_id": execution_id},
|
||||
"recursion_limit": _RECURSION_LIMIT,
|
||||
}
|
||||
|
||||
|
||||
# --- assembly ----------------------------------------------------------------------
|
||||
|
||||
AGENTS_DIR = Path(__file__).resolve().parent.parent / "agents"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Assembly:
|
||||
engine: ExecutionEngine
|
||||
bundle: ProviderBundle
|
||||
roles: dict[str, RoleSpec]
|
||||
mcp_gaps: dict[str, str]
|
||||
|
||||
routes = [
|
||||
Route("/health", health_check, methods=["GET"]),
|
||||
Route("/card", get_card, methods=["GET"]),
|
||||
Route("/generate", run_workflow, methods=["POST"]),
|
||||
Route("/validate", validate_artifacts_route, methods=["POST"]),
|
||||
Route("/sessions/{session_id}", get_session_route, methods=["GET"]),
|
||||
]
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: Starlette):
|
||||
"""Initialize PostgreSQL database schema on server startup."""
|
||||
logger.info("Initializing PostgreSQL database persistence...")
|
||||
get_db_manager()
|
||||
yield
|
||||
async def open_assembly(
|
||||
settings: Settings,
|
||||
*,
|
||||
runtime: AgentRuntime | None = None,
|
||||
recorder: ExecutionRecorder | None = None,
|
||||
output_root: Path | None = None,
|
||||
) -> AsyncIterator[Assembly]:
|
||||
bundle = load_provider(settings.CLOUD_PROVIDER)
|
||||
roles = load_roles(
|
||||
AGENTS_DIR,
|
||||
bundle,
|
||||
orchestrator_budget=settings.ORCHESTRATOR_BUDGET,
|
||||
specialist_budget=settings.SPECIALIST_BUDGET,
|
||||
include_preview_mcp=settings.MCP_ENABLE_PREVIEW,
|
||||
)
|
||||
if runtime is None:
|
||||
from app.adk.adk_runtime import AdkAgentRuntime
|
||||
|
||||
app = Starlette(
|
||||
debug=True,
|
||||
routes=routes,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
runtime = AdkAgentRuntime(
|
||||
skill_dirs={name: meta.path for name, meta in bundle.skills.items()},
|
||||
model_timeout_seconds=settings.MODEL_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
catalog = McpCatalog()
|
||||
if settings.MCP_ENABLED:
|
||||
used = {server_id for role in roles.values() for server_id in role.mcp_servers}
|
||||
catalog = await McpCatalog.load(
|
||||
[bundle.mcp_servers[server_id] for server_id in sorted(used)],
|
||||
auth_from_config(bundle.mcp_auth, settings.MCP_QUOTA_PROJECT),
|
||||
timeout=settings.MCP_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
@click.command()
|
||||
@click.option("--host", default=settings.HOST, help="Host address to bind")
|
||||
@click.option("--port", default=settings.PORT, type=int, help="Port to listen on")
|
||||
def main(host: str, port: int) -> None:
|
||||
"""Start the GCP Solution Architecture Agent Starlette REST server."""
|
||||
logger.info("Starting GCP Solution Architecture Agent on %s:%d", host, port)
|
||||
uvicorn.run(app, host=host, port=port, log_level=settings.LOG_LEVEL.lower())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
runner = TerraformRunner(
|
||||
binary=settings.TERRAFORM_BINARY,
|
||||
timeout_seconds=settings.TERRAFORM_TIMEOUT_SECONDS,
|
||||
plugin_cache_dir=settings.TF_PLUGIN_CACHE_DIR,
|
||||
)
|
||||
offered = [name for names in bundle.tools.values() for name in names]
|
||||
tool_factory = make_tool_factory(
|
||||
runtime,
|
||||
roles,
|
||||
bundle.pillar_skills,
|
||||
local_tools=local_tools(offered, runner),
|
||||
catalog=catalog,
|
||||
)
|
||||
|
||||
async with open_checkpointer(settings) as checkpointer:
|
||||
engine = ExecutionEngine(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder or DatabaseRecorder(),
|
||||
checkpointer=checkpointer,
|
||||
output_root=output_root or settings.BASE_DIR,
|
||||
max_revisions=settings.MAX_REVISIONS,
|
||||
max_orchestrator_turns=settings.MAX_ORCHESTRATOR_TURNS,
|
||||
tool_factory=tool_factory,
|
||||
)
|
||||
yield Assembly(engine, bundle, roles, dict(catalog.gaps))
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
"""Agent runtimes: the framework-neutral interface agents run behind, and its implementations."""
|
||||
@@ -1,92 +0,0 @@
|
||||
"""Composition root: build the Execution engine from settings and the provider bundle.
|
||||
|
||||
The only place concrete runtimes, tools, checkpointers and recorders are chosen.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from app.agents.roles import load_roles
|
||||
from app.agents.runtime import AgentRuntime, RoleSpec
|
||||
from app.config import Settings
|
||||
from app.execution.checkpointing import open_checkpointer
|
||||
from app.execution.engine import ExecutionEngine
|
||||
from app.execution.recorder import DatabaseRecorder, ExecutionRecorder
|
||||
from app.execution.toolkit import make_tool_factory
|
||||
from app.providers import ProviderBundle, load_provider
|
||||
from app.tools.local import TerraformRunner, local_tools
|
||||
from app.tools.mcp import McpCatalog, auth_from_config
|
||||
|
||||
AGENTS_DIR = Path(__file__).resolve().parent.parent / "agents"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Assembly:
|
||||
engine: ExecutionEngine
|
||||
bundle: ProviderBundle
|
||||
roles: dict[str, RoleSpec]
|
||||
mcp_gaps: dict[str, str]
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def open_assembly(
|
||||
settings: Settings,
|
||||
*,
|
||||
runtime: AgentRuntime | None = None,
|
||||
recorder: ExecutionRecorder | None = None,
|
||||
output_root: Path | None = None,
|
||||
) -> AsyncIterator[Assembly]:
|
||||
bundle = load_provider(settings.CLOUD_PROVIDER)
|
||||
roles = load_roles(
|
||||
AGENTS_DIR,
|
||||
bundle,
|
||||
orchestrator_budget=settings.ORCHESTRATOR_BUDGET,
|
||||
specialist_budget=settings.SPECIALIST_BUDGET,
|
||||
include_preview_mcp=settings.MCP_ENABLE_PREVIEW,
|
||||
)
|
||||
if runtime is None:
|
||||
from app.agents.adk_runtime import AdkAgentRuntime
|
||||
|
||||
runtime = AdkAgentRuntime(
|
||||
skill_dirs={name: meta.path for name, meta in bundle.skills.items()}
|
||||
)
|
||||
|
||||
catalog = McpCatalog()
|
||||
if settings.MCP_ENABLED:
|
||||
used = {server_id for role in roles.values() for server_id in role.mcp_servers}
|
||||
catalog = await McpCatalog.load(
|
||||
[bundle.mcp_servers[server_id] for server_id in sorted(used)],
|
||||
auth_from_config(bundle.mcp_auth, settings.MCP_QUOTA_PROJECT),
|
||||
timeout=settings.MCP_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
runner = TerraformRunner(
|
||||
binary=settings.TERRAFORM_BINARY,
|
||||
timeout_seconds=settings.TERRAFORM_TIMEOUT_SECONDS,
|
||||
plugin_cache_dir=settings.TF_PLUGIN_CACHE_DIR,
|
||||
)
|
||||
offered = [name for names in bundle.tools.values() for name in names]
|
||||
tool_factory = make_tool_factory(
|
||||
runtime,
|
||||
roles,
|
||||
bundle.pillar_skills,
|
||||
local_tools=local_tools(offered, runner),
|
||||
catalog=catalog,
|
||||
)
|
||||
|
||||
async with open_checkpointer(settings) as checkpointer:
|
||||
engine = ExecutionEngine(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder or DatabaseRecorder(),
|
||||
checkpointer=checkpointer,
|
||||
output_root=output_root or settings.BASE_DIR,
|
||||
max_revisions=settings.MAX_REVISIONS,
|
||||
max_orchestrator_turns=settings.MAX_ORCHESTRATOR_TURNS,
|
||||
tool_factory=tool_factory,
|
||||
)
|
||||
yield Assembly(engine, bundle, roles, dict(catalog.gaps))
|
||||
98
app/card.py
98
app/card.py
@@ -1,40 +1,66 @@
|
||||
"""AgentCard + skill declarations for GCP Solution Architecture Agent."""
|
||||
"""The A2A agent card, derived from the dependency graph and settings."""
|
||||
|
||||
from typing import Any, Dict
|
||||
from __future__ import annotations
|
||||
|
||||
AGENT_CARD: Dict[str, Any] = {
|
||||
"name": "gcp_solution_architecture_agent",
|
||||
"version": "1.0.0",
|
||||
"description": (
|
||||
"Automated Google Cloud Solution Architecture Agent executing an end-to-end "
|
||||
"migration workflow: Pre-emptive Source Environment Discovery -> Requirements Discovery -> "
|
||||
"Product Selection & Architecture Design -> Pre-deployment Validation -> Guide Packaging."
|
||||
),
|
||||
"capabilities": {
|
||||
"phases": ["source_discover", "discover", "design", "validate", "package"],
|
||||
"streaming": False,
|
||||
"async": True,
|
||||
"local_skills": [
|
||||
"source_discovery",
|
||||
"requirements_discovery",
|
||||
"architecture_design",
|
||||
"validation_rules",
|
||||
"packaging_guide",
|
||||
from a2a.types import (
|
||||
AgentCapabilities,
|
||||
AgentCard,
|
||||
AgentInterface,
|
||||
AgentProvider,
|
||||
AgentSkill,
|
||||
)
|
||||
|
||||
from app.config import Settings
|
||||
from app.workflows.deliverables import PHASE_ORDER, deliverables_in
|
||||
|
||||
A2A_PATH = "/a2a"
|
||||
|
||||
|
||||
def build_agent_card(settings: Settings) -> AgentCard:
|
||||
base_url = (
|
||||
settings.A2A_PUBLIC_URL or f"http://{settings.A2A_HOST}:{settings.A2A_PORT}"
|
||||
).rstrip("/")
|
||||
phases = "; ".join(
|
||||
f"{phase.value}: {', '.join(kind.value for kind in deliverables_in(phase))}"
|
||||
for phase in PHASE_ORDER
|
||||
)
|
||||
return AgentCard(
|
||||
name=settings.AGENT_NAME,
|
||||
description=(
|
||||
"Designs a cloud solution architecture with you: requirements discovery, "
|
||||
"grounded product selection, diagram, design recommendations, "
|
||||
"infrastructure-as-code and dry-run validation, packaged as a solution "
|
||||
f"guide. Cloud provider: {settings.CLOUD_PROVIDER}."
|
||||
),
|
||||
version=settings.AGENT_VERSION,
|
||||
provider=AgentProvider(
|
||||
organization="Kyndryl Agent Builder", url="https://www.kyndryl.com"
|
||||
),
|
||||
supported_interfaces=[
|
||||
AgentInterface(
|
||||
url=f"{base_url}{A2A_PATH}",
|
||||
protocol_binding="JSONRPC",
|
||||
protocol_version="1.0",
|
||||
)
|
||||
],
|
||||
},
|
||||
"metadata": {
|
||||
"cloud_provider": "gcp",
|
||||
"supported_outputs": [
|
||||
"deliverables/as-is/source-architecture.md",
|
||||
"deliverables/as-is/source-architecture.mmd",
|
||||
"deliverables/target/requirements.md",
|
||||
"deliverables/target/architecture.md",
|
||||
"deliverables/target/architecture.mmd",
|
||||
"deliverables/validation/validation-results.md",
|
||||
"deliverables/guides/solution-architecture-guide.md",
|
||||
"deliverables/executions/{execution_id}/*",
|
||||
"terraform/main.tf",
|
||||
capabilities=AgentCapabilities(streaming=True, push_notifications=False),
|
||||
default_input_modes=["text/plain", "application/json"],
|
||||
default_output_modes=["text/plain", "text/markdown", "application/json"],
|
||||
skills=[
|
||||
AgentSkill(
|
||||
id="solution-architecture",
|
||||
name="Cloud solution architecture",
|
||||
description=(
|
||||
"Interactive, multi-turn engagement. Phases and their Deliverables: "
|
||||
f"{phases}. Each Phase needs your approval, sent as data."
|
||||
),
|
||||
tags=["architecture", "cloud", settings.CLOUD_PROVIDER, "terraform"],
|
||||
examples=[
|
||||
"Design a regional event ingestion platform on Google Cloud.",
|
||||
"Migrate our on-premises order service to managed services.",
|
||||
],
|
||||
input_modes=["text/plain", "application/json"],
|
||||
output_modes=["text/plain", "text/markdown", "application/json"],
|
||||
)
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Typed settings for GCP Solution Architecture Agent."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
@@ -19,24 +19,7 @@ class Settings(BaseSettings):
|
||||
AGENT_VERSION: str = "1.0.0"
|
||||
LOG_LEVEL: str = "INFO"
|
||||
|
||||
# Server Configuration
|
||||
HOST: str = "0.0.0.0"
|
||||
PORT: int = 8080
|
||||
|
||||
# LLM Settings
|
||||
OPENAI_API_KEY: str | None = None
|
||||
AZURE_OPENAI_API_KEY: str | None = None
|
||||
AZURE_OPENAI_ENDPOINT: str | None = None
|
||||
AZURE_OPENAI_DEPLOYMENT: str = "gpt-4o"
|
||||
AZURE_OPENAI_API_VERSION: str = "2024-02-15-preview"
|
||||
LLM_TEMPERATURE: float = 0.2
|
||||
|
||||
# Google Cloud Environment & Scanner Credentials
|
||||
GOOGLE_APPLICATION_CREDENTIALS: str | None = None
|
||||
GCP_PROJECT_ID: str | None = None
|
||||
GCP_REGION: str = "us-central1"
|
||||
|
||||
# Database & Storage Settings (External PostgreSQL)
|
||||
# Database & Storage Settings (External PostgreSQL, SQLite fallback)
|
||||
POSTGRES_HOST: str = "localhost"
|
||||
POSTGRES_PORT: int = 5432
|
||||
POSTGRES_DB: str = "gcp_agent_db"
|
||||
@@ -45,10 +28,6 @@ class Settings(BaseSettings):
|
||||
DATABASE_URL: str | None = None
|
||||
SQLITE_FALLBACK_DB: str = "gcp_agent_local.db"
|
||||
|
||||
# ADK Multi-Agent Orchestrator Settings
|
||||
ADK_MAX_LOOP_ITERATIONS: int = 5
|
||||
ADK_ENABLE_ARTIFACT_STORE: bool = True
|
||||
|
||||
# Cloud Provider Bundle (providers/<name>.yaml) & Remote MCP Servers
|
||||
CLOUD_PROVIDER: str = "gcp"
|
||||
MCP_ENABLED: bool = True
|
||||
@@ -61,6 +40,9 @@ class Settings(BaseSettings):
|
||||
TERRAFORM_TIMEOUT_SECONDS: float = 300
|
||||
TF_PLUGIN_CACHE_DIR: str | None = None
|
||||
|
||||
# Models: a request with no response in this time is abandoned and retried
|
||||
MODEL_TIMEOUT_SECONDS: float = 180
|
||||
|
||||
# Hierarchical Execution Engine (LangGraph state, ReAct agents)
|
||||
A2A_HOST: str = "0.0.0.0"
|
||||
A2A_PORT: int = 8000
|
||||
@@ -72,14 +54,8 @@ class Settings(BaseSettings):
|
||||
CHECKPOINTER: str = "sqlite" # memory | sqlite | postgres
|
||||
CHECKPOINT_DB_PATH: str = "checkpoints.db"
|
||||
|
||||
# Google Developer Knowledge MCP Server Settings
|
||||
DEVELOPER_KNOWLEDGE_MCP_URL: str = "https://developerknowledge.googleapis.com/mcp"
|
||||
DEVELOPER_KNOWLEDGE_MCP_ENABLED: bool = True
|
||||
|
||||
# Paths
|
||||
BASE_DIR: Path = Path(__file__).resolve().parent.parent
|
||||
SKILLS_DIR: Path = Path(__file__).resolve().parent / "skills"
|
||||
EVAL_DATASET_PATH: Path = Path(__file__).resolve().parent.parent / "eval" / "datasets" / "benchmark_cases.json"
|
||||
|
||||
def get_database_url(self) -> str:
|
||||
"""Construct database connection URL (PostgreSQL or SQLite fallback)."""
|
||||
@@ -97,4 +73,3 @@ def get_settings() -> Settings:
|
||||
if _settings is None:
|
||||
_settings = Settings()
|
||||
return _settings
|
||||
|
||||
|
||||
194
app/database.py
194
app/database.py
@@ -9,11 +9,11 @@ import json
|
||||
import logging
|
||||
import sqlite3
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any
|
||||
|
||||
try:
|
||||
import psycopg2
|
||||
from psycopg2.extras import RealDictCursor
|
||||
|
||||
HAS_PSYCOPG2 = True
|
||||
except ImportError:
|
||||
HAS_PSYCOPG2 = False
|
||||
@@ -26,10 +26,12 @@ logger = logging.getLogger(__name__)
|
||||
class DatabaseManager:
|
||||
"""Handles persistence of sessions, execution runs, artifacts, and review logs."""
|
||||
|
||||
def __init__(self, db_url: Optional[str] = None) -> None:
|
||||
def __init__(self, db_url: str | None = None) -> None:
|
||||
self.settings = get_settings()
|
||||
self.db_url = db_url or self.settings.get_database_url()
|
||||
self.is_postgres = self.db_url.startswith("postgresql://") or self.db_url.startswith("postgres://")
|
||||
self.is_postgres = self.db_url.startswith(
|
||||
"postgresql://"
|
||||
) or self.db_url.startswith("postgres://")
|
||||
self._init_db()
|
||||
|
||||
def _get_connection(self):
|
||||
@@ -38,7 +40,7 @@ class DatabaseManager:
|
||||
try:
|
||||
conn = psycopg2.connect(self.db_url, connect_timeout=3)
|
||||
return conn
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - any connection failure falls back to SQLite
|
||||
logger.warning(
|
||||
"PostgreSQL connection to %s failed (%s); falling back to local SQLite database %s",
|
||||
self.db_url,
|
||||
@@ -56,7 +58,7 @@ class DatabaseManager:
|
||||
conn = self._get_connection()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
|
||||
|
||||
# 1. Sessions Table
|
||||
cursor.execute("""
|
||||
CREATE TABLE IF NOT EXISTS adk_sessions (
|
||||
@@ -125,12 +127,18 @@ class DatabaseManager:
|
||||
|
||||
conn.commit()
|
||||
logger.info("Database schema initialized successfully.")
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Failed to initialize database schema: %s", exc)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def save_session(self, session_id: str, agent_name: str, state_data: Dict[str, Any], metadata: Optional[Dict[str, Any]] = None) -> bool:
|
||||
def save_session(
|
||||
self,
|
||||
session_id: str,
|
||||
agent_name: str,
|
||||
state_data: dict[str, Any],
|
||||
metadata: dict[str, Any] | None = None,
|
||||
) -> bool:
|
||||
"""Persist or update ADK session state in PostgreSQL."""
|
||||
conn = self._get_connection()
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
@@ -140,41 +148,51 @@ class DatabaseManager:
|
||||
meta_json = json.dumps(metadata or {})
|
||||
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_sessions (session_id, agent_name, created_at, updated_at, state_data, metadata)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(session_id) DO UPDATE SET
|
||||
updated_at = excluded.updated_at,
|
||||
state_data = excluded.state_data,
|
||||
metadata = excluded.metadata
|
||||
""", (session_id, agent_name, now, now, state_json, meta_json))
|
||||
""",
|
||||
(session_id, agent_name, now, now, state_json, meta_json),
|
||||
)
|
||||
else:
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_sessions (session_id, agent_name, created_at, updated_at, state_data, metadata)
|
||||
VALUES (%s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT(session_id) DO UPDATE SET
|
||||
updated_at = EXCLUDED.updated_at,
|
||||
state_data = EXCLUDED.state_data,
|
||||
metadata = EXCLUDED.metadata
|
||||
""", (session_id, agent_name, now, now, state_json, meta_json))
|
||||
""",
|
||||
(session_id, agent_name, now, now, state_json, meta_json),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error saving session %s: %s", session_id, exc)
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Dict[str, Any]]:
|
||||
def get_session(self, session_id: str) -> dict[str, Any] | None:
|
||||
"""Retrieve ADK session state by session_id."""
|
||||
conn = self._get_connection()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("SELECT * FROM adk_sessions WHERE session_id = ?", (session_id,))
|
||||
cursor.execute(
|
||||
"SELECT * FROM adk_sessions WHERE session_id = ?", (session_id,)
|
||||
)
|
||||
else:
|
||||
cursor.execute("SELECT * FROM adk_sessions WHERE session_id = %s", (session_id,))
|
||||
cursor.execute(
|
||||
"SELECT * FROM adk_sessions WHERE session_id = %s", (session_id,)
|
||||
)
|
||||
|
||||
row = cursor.fetchone()
|
||||
if not row:
|
||||
@@ -197,7 +215,7 @@ class DatabaseManager:
|
||||
"state_data": json.loads(row[4]),
|
||||
"metadata": json.loads(row[5] or "{}"),
|
||||
}
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error reading session %s: %s", session_id, exc)
|
||||
return None
|
||||
finally:
|
||||
@@ -211,7 +229,7 @@ class DatabaseManager:
|
||||
current_phase: str,
|
||||
loop_count: int,
|
||||
request_summary: str,
|
||||
results: Dict[str, Any],
|
||||
results: dict[str, Any],
|
||||
) -> bool:
|
||||
"""Save workflow execution record to database."""
|
||||
conn = self._get_connection()
|
||||
@@ -221,7 +239,8 @@ class DatabaseManager:
|
||||
results_json = json.dumps(results)
|
||||
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO workflow_executions
|
||||
(execution_id, session_id, status, current_phase, loop_count, started_at, completed_at, request_summary, results_json)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
@@ -231,9 +250,22 @@ class DatabaseManager:
|
||||
loop_count = excluded.loop_count,
|
||||
completed_at = excluded.completed_at,
|
||||
results_json = excluded.results_json
|
||||
""", (execution_id, session_id, status, current_phase, loop_count, now, now, request_summary, results_json))
|
||||
""",
|
||||
(
|
||||
execution_id,
|
||||
session_id,
|
||||
status,
|
||||
current_phase,
|
||||
loop_count,
|
||||
now,
|
||||
now,
|
||||
request_summary,
|
||||
results_json,
|
||||
),
|
||||
)
|
||||
else:
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO workflow_executions
|
||||
(execution_id, session_id, status, current_phase, loop_count, started_at, completed_at, request_summary, results_json)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
@@ -243,42 +275,67 @@ class DatabaseManager:
|
||||
loop_count = EXCLUDED.loop_count,
|
||||
completed_at = EXCLUDED.completed_at,
|
||||
results_json = EXCLUDED.results_json
|
||||
""", (execution_id, session_id, status, current_phase, loop_count, now, now, request_summary, results_json))
|
||||
""",
|
||||
(
|
||||
execution_id,
|
||||
session_id,
|
||||
status,
|
||||
current_phase,
|
||||
loop_count,
|
||||
now,
|
||||
now,
|
||||
request_summary,
|
||||
results_json,
|
||||
),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error saving workflow execution %s: %s", execution_id, exc)
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def save_artifact(self, artifact_id: str, execution_id: str, artifact_type: str, file_path: str, content: str) -> bool:
|
||||
def save_artifact(
|
||||
self,
|
||||
artifact_id: str,
|
||||
execution_id: str,
|
||||
artifact_type: str,
|
||||
file_path: str,
|
||||
content: str,
|
||||
) -> bool:
|
||||
"""Write ADK artifact metadata & content to database."""
|
||||
conn = self._get_connection()
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_artifacts (artifact_id, execution_id, artifact_type, file_path, content, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(artifact_id) DO UPDATE SET
|
||||
content = excluded.content,
|
||||
file_path = excluded.file_path
|
||||
""", (artifact_id, execution_id, artifact_type, file_path, content, now))
|
||||
""",
|
||||
(artifact_id, execution_id, artifact_type, file_path, content, now),
|
||||
)
|
||||
else:
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_artifacts (artifact_id, execution_id, artifact_type, file_path, content, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT(artifact_id) DO UPDATE SET
|
||||
content = EXCLUDED.content,
|
||||
file_path = EXCLUDED.file_path
|
||||
""", (artifact_id, execution_id, artifact_type, file_path, content, now))
|
||||
""",
|
||||
(artifact_id, execution_id, artifact_type, file_path, content, now),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error saving artifact %s: %s", artifact_id, exc)
|
||||
return False
|
||||
finally:
|
||||
@@ -299,25 +356,56 @@ class DatabaseManager:
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO orchestrator_review_logs (log_id, execution_id, iteration, review_status, reviewer_agent, feedback, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""", (log_id, execution_id, iteration, review_status, reviewer_agent, feedback, now))
|
||||
""",
|
||||
(
|
||||
log_id,
|
||||
execution_id,
|
||||
iteration,
|
||||
review_status,
|
||||
reviewer_agent,
|
||||
feedback,
|
||||
now,
|
||||
),
|
||||
)
|
||||
else:
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO orchestrator_review_logs (log_id, execution_id, iteration, review_status, reviewer_agent, feedback, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
""", (log_id, execution_id, iteration, review_status, reviewer_agent, feedback, now))
|
||||
""",
|
||||
(
|
||||
log_id,
|
||||
execution_id,
|
||||
iteration,
|
||||
review_status,
|
||||
reviewer_agent,
|
||||
feedback,
|
||||
now,
|
||||
),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error recording orchestrator review log %s: %s", log_id, exc)
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def save_evaluation(self, eval_id: str, case_id: str, total_score: float, max_score: float, pass_rate: float, passed: bool, details: Dict[str, Any]) -> bool:
|
||||
def save_evaluation(
|
||||
self,
|
||||
eval_id: str,
|
||||
case_id: str,
|
||||
total_score: float,
|
||||
max_score: float,
|
||||
pass_rate: float,
|
||||
passed: bool,
|
||||
details: dict[str, Any],
|
||||
) -> bool:
|
||||
"""Save ADK evaluation benchmark result to database."""
|
||||
conn = self._get_connection()
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
@@ -325,26 +413,50 @@ class DatabaseManager:
|
||||
cursor = conn.cursor()
|
||||
details_json = json.dumps(details)
|
||||
if isinstance(conn, sqlite3.Connection):
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_evaluations (eval_id, case_id, total_score, max_score, pass_rate, passed, details_json, evaluated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (eval_id, case_id, total_score, max_score, pass_rate, passed, details_json, now))
|
||||
""",
|
||||
(
|
||||
eval_id,
|
||||
case_id,
|
||||
total_score,
|
||||
max_score,
|
||||
pass_rate,
|
||||
passed,
|
||||
details_json,
|
||||
now,
|
||||
),
|
||||
)
|
||||
else:
|
||||
cursor.execute("""
|
||||
cursor.execute(
|
||||
"""
|
||||
INSERT INTO adk_evaluations (eval_id, case_id, total_score, max_score, pass_rate, passed, details_json, evaluated_at)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
|
||||
""", (eval_id, case_id, total_score, max_score, pass_rate, passed, details_json, now))
|
||||
""",
|
||||
(
|
||||
eval_id,
|
||||
case_id,
|
||||
total_score,
|
||||
max_score,
|
||||
pass_rate,
|
||||
passed,
|
||||
details_json,
|
||||
now,
|
||||
),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as exc:
|
||||
except Exception as exc: # noqa: BLE001 - persistence is best-effort; the failure is logged
|
||||
logger.error("Error saving evaluation %s: %s", eval_id, exc)
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
_db_manager: Optional[DatabaseManager] = None
|
||||
_db_manager: DatabaseManager | None = None
|
||||
|
||||
|
||||
def get_db_manager() -> DatabaseManager:
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
"""Execution of the solution-architecture workflow: Deliverables, the Ledger and the graph."""
|
||||
@@ -1,152 +0,0 @@
|
||||
"""ExecutionEngine: start and resume Executions.
|
||||
|
||||
The one interface the A2A server, /generate and the evaluation harness use. Each
|
||||
call runs the Execution until it needs the user or finishes, and returns an
|
||||
Interaction describing which.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||
from langgraph.types import Command
|
||||
|
||||
from app.agents.runtime import AgentRuntime, RoleSpec
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH
|
||||
from app.execution.graph import ToolFactory, build_execution_graph
|
||||
from app.execution.ledger import Ledger
|
||||
from app.execution.recorder import ExecutionRecorder
|
||||
from app.execution.tools import ORCHESTRATOR
|
||||
|
||||
_RECURSION_LIMIT = 10_000 # loops are bounded by max_orchestrator_turns instead
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Interaction:
|
||||
execution_id: str
|
||||
status: str # awaiting_user | completed | completed_with_warnings
|
||||
prompt: Mapping[str, Any] | None # questions | approval | permission
|
||||
state: Mapping[str, Any]
|
||||
|
||||
|
||||
class ExecutionNotAwaitingUser(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class ExecutionEngine:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
recorder: ExecutionRecorder,
|
||||
checkpointer: BaseCheckpointSaver,
|
||||
output_root: Path,
|
||||
max_revisions: int = 3,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> None:
|
||||
required = {
|
||||
ORCHESTRATOR,
|
||||
*(spec.specialist for spec in DEPENDENCY_GRAPH.values()),
|
||||
}
|
||||
missing = sorted(required - set(roles))
|
||||
if missing:
|
||||
raise ValueError(f"Missing roles: {', '.join(missing)}")
|
||||
self._max_revisions = max_revisions
|
||||
self._graph = build_execution_graph(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder,
|
||||
output_root=output_root,
|
||||
max_orchestrator_turns=max_orchestrator_turns,
|
||||
tool_factory=tool_factory,
|
||||
).compile(checkpointer=checkpointer)
|
||||
|
||||
async def start(
|
||||
self,
|
||||
workflow_request: str,
|
||||
*,
|
||||
session_id: str | None = None,
|
||||
pre_approved: bool = False,
|
||||
allow_cloud_read: bool | None = None,
|
||||
execution_id: str | None = None,
|
||||
) -> Interaction:
|
||||
execution_id = execution_id or str(uuid.uuid4())
|
||||
if await self._exists(execution_id):
|
||||
raise ValueError(f"Execution {execution_id} already exists")
|
||||
if pre_approved and allow_cloud_read is None:
|
||||
allow_cloud_read = False
|
||||
initial: dict[str, Any] = {
|
||||
"execution_id": execution_id,
|
||||
"session_id": session_id or str(uuid.uuid4()),
|
||||
"workflow_request": workflow_request,
|
||||
"pre_approved": pre_approved,
|
||||
"allow_cloud_read": allow_cloud_read,
|
||||
"ledger": Ledger(
|
||||
pre_approved=pre_approved, max_revisions=self._max_revisions
|
||||
).to_dict(),
|
||||
"deliverables": {},
|
||||
"conversation": [{"from": "user", "text": workflow_request}],
|
||||
"notes": [],
|
||||
"pending_questions": [],
|
||||
"awaiting_questions": [],
|
||||
"answers": {},
|
||||
"pending_permission": None,
|
||||
"prompt": None,
|
||||
"outcomes": [],
|
||||
"outcomes_applied": 0,
|
||||
"events_recorded": 0,
|
||||
"turns": 0,
|
||||
"status": "running",
|
||||
"warnings": [],
|
||||
}
|
||||
await self._graph.ainvoke(initial, self._config(execution_id))
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
|
||||
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
|
||||
current = await self.interaction(execution_id)
|
||||
if current.status != "awaiting_user":
|
||||
raise ExecutionNotAwaitingUser(
|
||||
f"Execution {execution_id} is {current.status}, not awaiting the user"
|
||||
)
|
||||
await self._graph.ainvoke(
|
||||
Command(resume=dict(reply)), self._config(execution_id)
|
||||
)
|
||||
return await self.interaction(execution_id)
|
||||
|
||||
async def interaction(self, execution_id: str) -> Interaction:
|
||||
snapshot = await self._graph.aget_state(self._config(execution_id))
|
||||
if not snapshot.values:
|
||||
raise KeyError(f"Unknown Execution {execution_id}")
|
||||
if snapshot.interrupts:
|
||||
return Interaction(
|
||||
execution_id,
|
||||
"awaiting_user",
|
||||
snapshot.interrupts[0].value,
|
||||
snapshot.values,
|
||||
)
|
||||
return Interaction(
|
||||
execution_id,
|
||||
snapshot.values.get("status", "running"),
|
||||
None,
|
||||
snapshot.values,
|
||||
)
|
||||
|
||||
async def _exists(self, execution_id: str) -> bool:
|
||||
snapshot = await self._graph.aget_state(self._config(execution_id))
|
||||
return bool(snapshot.values)
|
||||
|
||||
@staticmethod
|
||||
def _config(execution_id: str) -> RunnableConfig:
|
||||
return {
|
||||
"configurable": {"thread_id": execution_id},
|
||||
"recursion_limit": _RECURSION_LIMIT,
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
"""The A2A server: JSON-RPC at /a2a, the agent card, health, and pre-approved /generate.
|
||||
|
||||
Run with: python -m app.a2a.server
|
||||
Run with: python -m app.main
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -23,11 +23,11 @@ from starlette.requests import Request
|
||||
from starlette.responses import JSONResponse
|
||||
from starlette.routing import Route
|
||||
|
||||
from app.a2a.card import A2A_PATH, build_agent_card
|
||||
from app.a2a.executor import SolutionArchitectExecutor
|
||||
from app.assembly import Assembly, open_assembly
|
||||
from app.a2a_server import SolutionArchitectExecutor
|
||||
from app.agent import Assembly, open_assembly
|
||||
from app.card import A2A_PATH, build_agent_card
|
||||
from app.config import Settings, get_settings
|
||||
from app.execution.presentation import to_generate_response
|
||||
from app.presentation import to_generate_response
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -1,9 +1 @@
|
||||
"""Nodes package."""
|
||||
|
||||
from .design_node import design_node
|
||||
from .discover_node import discover_node
|
||||
from .package_node import package_node
|
||||
from .source_discover_node import source_discover_node
|
||||
from .validate_node import validate_node
|
||||
|
||||
__all__ = ["source_discover_node", "discover_node", "design_node", "validate_node", "package_node"]
|
||||
"""The nodes of the Execution graph."""
|
||||
|
||||
@@ -1,139 +0,0 @@
|
||||
"""Phase 1: Architecture Design Node."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict
|
||||
from app.states.state import GCPArchitectureState
|
||||
from app.skills.loader import SkillLoader
|
||||
from app.tools.mcp_developer_knowledge import get_mcp_client
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def design_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dict[str, Any]:
|
||||
"""Processes Phase 1 Design: select GCP managed products, generate Mermaid graph, and Terraform IaC."""
|
||||
logger.info("Executing design_node")
|
||||
skill_prompt = skill_loader.format_skills_for_prompt("design")
|
||||
|
||||
# Developer Knowledge MCP Grounding Search
|
||||
mcp_client = get_mcp_client()
|
||||
mcp_knowledge = mcp_client.search_documents(query="Cloud Run PubSub event-driven architecture", category="compute")
|
||||
docs = mcp_knowledge.get("documents", [])
|
||||
citations_str = "\n".join([f"- [{doc['title']}]({doc['uri']})" for doc in docs[:3]]) if docs else "- [Google Cloud Architecture Framework](https://cloud.google.com/architecture/framework)"
|
||||
|
||||
architecture_doc = f"""# Phase 1 — Architecture & Product Selection
|
||||
|
||||
## Selected Products
|
||||
- **Compute / Serving**: Google Cloud Run (Fully Managed Container Ingress & Stateless Execution)
|
||||
- **Messaging & Eventing**: Google Cloud Pub/Sub (Regional Event Bus for Asynchronous Decoupling)
|
||||
- **State & Storage**: Google Cloud Storage & Firestore (Database & Bucket Storage for Durable Audit Event Replay)
|
||||
- **Security & Identity**: Cloud IAM (Least Privilege Service Accounts) & KMS (Customer-Managed Encryption Keys)
|
||||
- **Artifact Registry**: Google Artifact Registry (OCI Container Image Hosting)
|
||||
|
||||
## Component Responsibilities
|
||||
1. **Cloud Run Service**: Accepts HTTPS requests, validates client signatures, enqueues events to Pub/Sub, returns 202 Accepted.
|
||||
2. **Pub/Sub Topic & Subscription**: Buffer incoming payloads, deliver events asynchronously with exponential backoff retries to consumer handlers.
|
||||
3. **Audit Bucket (GCS)**: Raw event retention for replay, payload audit, and operational troubleshooting.
|
||||
|
||||
## Security & Compliance
|
||||
- HTTPS ingress with TLS 1.3 encryption in transit.
|
||||
- Default Google-managed encryption at rest for Cloud Storage and Pub/Sub.
|
||||
- Cloud Run service account bound strictly to `roles/pubsub.publisher` and `roles/storage.objectCreator`.
|
||||
|
||||
## Grounded Documentation Citations (Google Developer Knowledge MCP)
|
||||
{citations_str}
|
||||
"""
|
||||
|
||||
mermaid_diagram = """flowchart TD
|
||||
Client[External HTTPS Client] -->|HTTPS POST /events| CloudRun[Google Cloud Run Service]
|
||||
CloudRun -->|Publish Event| PubSubTopic[Cloud Pub/Sub Topic]
|
||||
CloudRun -->|Write Raw Payload| GCSAudit[Cloud Storage Audit Bucket]
|
||||
PubSubTopic -->|Push Delivery| EventConsumer[Cloud Run Consumer Service]
|
||||
EventConsumer -->|Acknowledge| PubSubTopic
|
||||
"""
|
||||
|
||||
terraform_code = """# Google Cloud Solution Architecture Baseline
|
||||
terraform {
|
||||
required_version = ">= 1.5.0"
|
||||
required_providers {
|
||||
google = {
|
||||
source = "hashicorp/google"
|
||||
version = "~> 5.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "google" {
|
||||
project = var.project_id
|
||||
region = var.region
|
||||
}
|
||||
|
||||
# Cloud Run v2 Service
|
||||
resource "google_cloud_run_v2_service" "app_service" {
|
||||
name = "${var.environment}-app-service"
|
||||
location = var.region
|
||||
|
||||
template {
|
||||
containers {
|
||||
image = var.container_image
|
||||
ports {
|
||||
container_port = 8080
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Pub/Sub Topic for Event Ingestion
|
||||
resource "google_pubsub_topic" "event_ingestion" {
|
||||
name = "${var.environment}-event-ingestion-topic"
|
||||
labels = {
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
}
|
||||
}
|
||||
|
||||
# Cloud Storage Bucket for Event Replay Audit
|
||||
resource "google_storage_bucket" "audit_bucket" {
|
||||
name = "${var.project_id}-${var.environment}-audit-bucket"
|
||||
location = var.region
|
||||
force_destroy = false
|
||||
uniform_bucket_level_access = true
|
||||
|
||||
versioning {
|
||||
enabled = true
|
||||
}
|
||||
|
||||
lifecycle_rule {
|
||||
condition {
|
||||
age = 30
|
||||
}
|
||||
action {
|
||||
type = "Delete"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Least-Privilege IAM Service Account
|
||||
resource "google_service_account" "ingress_sa" {
|
||||
account_id = "${var.environment}-ingress-sa"
|
||||
display_name = "Cloud Run Ingress Identity"
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic_iam_member" "publisher_binding" {
|
||||
topic = google_pubsub_topic.event_ingestion.name
|
||||
role = "roles/pubsub.publisher"
|
||||
member = "serviceAccount:${google_service_account.ingress_sa.email}"
|
||||
}
|
||||
"""
|
||||
|
||||
active_skills = state.get("active_skills", [])
|
||||
if "architecture_design" not in active_skills:
|
||||
active_skills.append("architecture_design")
|
||||
|
||||
return {
|
||||
"architecture_doc": architecture_doc,
|
||||
"mermaid_diagram": mermaid_diagram,
|
||||
"terraform_code": terraform_code,
|
||||
"product_selection_deferred": False,
|
||||
"current_phase": "design",
|
||||
"active_skills": active_skills,
|
||||
}
|
||||
@@ -1,10 +1,4 @@
|
||||
"""The LangGraph that holds an Execution (ADR-0003).
|
||||
|
||||
START -> orchestrator_turn
|
||||
orchestrator_turn -> specialist_task x N (Send, parallel) -> orchestrator_turn
|
||||
orchestrator_turn -> await_user (interrupt) -> orchestrator_turn
|
||||
orchestrator_turn -> finalize -> END
|
||||
orchestrator_turn -> end_with_warnings -> END
|
||||
"""The nodes of the Execution graph (ADR-0003); app/workflows/execution_graph.py wires them.
|
||||
|
||||
Only orchestrator_turn and await_user change the Ledger; parallel Specialist tasks
|
||||
append outcomes that the next Orchestrator turn applies in order.
|
||||
@@ -13,14 +7,16 @@ append outcomes that the next Orchestrator turn applies in order.
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
from typing import Any
|
||||
|
||||
from langgraph.graph import END, START, StateGraph
|
||||
from langgraph.types import Command, Send, interrupt
|
||||
|
||||
from app.agents.runtime import (
|
||||
from app.adk.runtime import (
|
||||
AgentRuntime,
|
||||
AgentTask,
|
||||
BudgetExhausted,
|
||||
@@ -29,24 +25,23 @@ from app.agents.runtime import (
|
||||
RoleSpec,
|
||||
Stopped,
|
||||
Submitted,
|
||||
ToolFactory,
|
||||
ToolSpec,
|
||||
TurnEnded,
|
||||
)
|
||||
from app.execution.deliverables import (
|
||||
from app.recorder import ExecutionRecorder
|
||||
from app.states.execution_state import ExecutionState
|
||||
from app.states.ledger import Ledger, LedgerRefusal
|
||||
from app.tools.execution import ORCHESTRATOR, OrchestratorToolbox, specialist_tools
|
||||
from app.workflows.deliverables import (
|
||||
DEPENDENCY_GRAPH,
|
||||
PHASE_ORDER,
|
||||
DeliverableKind,
|
||||
inputs_of,
|
||||
)
|
||||
from app.execution.ledger import Ledger, LedgerRefusal
|
||||
from app.execution.recorder import ExecutionRecorder
|
||||
from app.execution.state import ExecutionState
|
||||
from app.execution.tools import ORCHESTRATOR, OrchestratorToolbox, specialist_tools
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ToolFactory = Callable[[RoleSpec, str | None], Sequence[ToolSpec]]
|
||||
|
||||
NO_USER_REPLY = (
|
||||
"No user is available. State your assumption, record it in the Deliverable under "
|
||||
"'Assumptions', and proceed."
|
||||
@@ -58,7 +53,19 @@ def _no_tools(role: RoleSpec, deliverable: str | None) -> Sequence[ToolSpec]:
|
||||
return ()
|
||||
|
||||
|
||||
def build_execution_graph(
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionNodes:
|
||||
"""The node functions of one Execution graph, bound to its runtime and recorder."""
|
||||
|
||||
orchestrator_turn: Callable[[ExecutionState], Awaitable[Command]]
|
||||
# Receives its Send payload, not the Execution state.
|
||||
specialist_task: Callable[[dict[str, Any]], Awaitable[dict[str, Any]]]
|
||||
await_user: Callable[[ExecutionState], Awaitable[Command]]
|
||||
finalize: Callable[[ExecutionState], Awaitable[dict[str, Any]]]
|
||||
end_with_warnings: Callable[[ExecutionState], Awaitable[dict[str, Any]]]
|
||||
|
||||
|
||||
def make_execution_nodes(
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
@@ -66,13 +73,22 @@ def build_execution_graph(
|
||||
output_root: Path,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> StateGraph:
|
||||
) -> ExecutionNodes:
|
||||
tool_factory = tool_factory or _no_tools
|
||||
|
||||
async def record_progress(
|
||||
state: ExecutionState, ledger: Ledger, status: str | None = None
|
||||
) -> int:
|
||||
recorded = state.get("events_recorded", 0)
|
||||
for event in ledger.events[recorded:]:
|
||||
logger.info(
|
||||
"[%s] ledger %s %s by %s%s",
|
||||
state["execution_id"],
|
||||
event.type,
|
||||
event.subject,
|
||||
event.actor,
|
||||
f" ({event.detail})" if event.detail else "",
|
||||
)
|
||||
await recorder.record_events(state["execution_id"], ledger.events[recorded:])
|
||||
if status:
|
||||
await recorder.record_status(
|
||||
@@ -169,7 +185,16 @@ def build_execution_graph(
|
||||
prompt="Decide the next step for this Execution using your tools.",
|
||||
context=_orchestrator_context(state, ledger, pending_questions, notes),
|
||||
)
|
||||
started = time.monotonic()
|
||||
outcome = await runtime.run(roles[ORCHESTRATOR], task, toolbox.tools())
|
||||
logger.info(
|
||||
"[%s] orchestrator turn %d (%s phase) ended in %.0fs: %s",
|
||||
state["execution_id"],
|
||||
turns,
|
||||
_current_phase(ledger),
|
||||
time.monotonic() - started,
|
||||
_summarise_outcome(outcome),
|
||||
)
|
||||
notes = []
|
||||
pre_approved = bool(state.get("pre_approved"))
|
||||
conversation = list(state.get("conversation", []))
|
||||
@@ -243,11 +268,27 @@ def build_execution_graph(
|
||||
context=task_input["context"],
|
||||
deliverable=kind.value,
|
||||
)
|
||||
logger.info(
|
||||
"[%s] %s started %s (attempt %d)",
|
||||
task_input["execution_id"],
|
||||
role.name,
|
||||
kind,
|
||||
task_input["attempt"],
|
||||
)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
outcome: Outcome = await runtime.run(role, task, tools)
|
||||
except Exception as exc:
|
||||
logger.exception("Specialist %s failed on %s", role.name, kind)
|
||||
outcome = Stopped(f"it failed with {type(exc).__name__}: {exc}")
|
||||
logger.info(
|
||||
"[%s] %s finished %s in %.0fs: %s",
|
||||
task_input["execution_id"],
|
||||
role.name,
|
||||
kind,
|
||||
time.monotonic() - started,
|
||||
_summarise_outcome(outcome),
|
||||
)
|
||||
return {
|
||||
"outcomes": [
|
||||
_outcome_record(role.name, kind, task_input["attempt"], outcome)
|
||||
@@ -257,7 +298,7 @@ def build_execution_graph(
|
||||
async def await_user(state: ExecutionState) -> Command:
|
||||
prompt = state["prompt"] or {}
|
||||
if state.get("pre_approved"):
|
||||
reply = _automatic_reply(prompt)
|
||||
reply = _automatic_reply(prompt, state.get("scripted_answers", []))
|
||||
else:
|
||||
reply = interrupt(prompt)
|
||||
|
||||
@@ -330,23 +371,30 @@ def build_execution_graph(
|
||||
await record_progress(state, ledger, "completed_with_warnings")
|
||||
return {"status": "completed_with_warnings"}
|
||||
|
||||
graph = StateGraph(ExecutionState)
|
||||
graph.add_node("orchestrator_turn", orchestrator_turn)
|
||||
# Specialist tasks receive their Send payload, not the Execution state.
|
||||
graph.add_node("specialist_task", cast(Any, specialist_task))
|
||||
graph.add_node("await_user", await_user)
|
||||
graph.add_node("finalize", finalize)
|
||||
graph.add_node("end_with_warnings", end_with_warnings)
|
||||
graph.add_edge(START, "orchestrator_turn")
|
||||
graph.add_edge("specialist_task", "orchestrator_turn")
|
||||
graph.add_edge("finalize", END)
|
||||
graph.add_edge("end_with_warnings", END)
|
||||
return graph
|
||||
return ExecutionNodes(
|
||||
orchestrator_turn=orchestrator_turn,
|
||||
specialist_task=specialist_task,
|
||||
await_user=await_user,
|
||||
finalize=finalize,
|
||||
end_with_warnings=end_with_warnings,
|
||||
)
|
||||
|
||||
|
||||
# --- helpers ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _summarise_outcome(outcome: Outcome) -> str:
|
||||
if isinstance(outcome, TurnEnded):
|
||||
return f"called {outcome.tool}"
|
||||
if isinstance(outcome, Submitted):
|
||||
return f"submitted ({len(outcome.content)} chars)"
|
||||
if isinstance(outcome, NeedsInput):
|
||||
return f"needs input ({len(outcome.questions)} questions)"
|
||||
if isinstance(outcome, BudgetExhausted):
|
||||
return f"budget exhausted: {outcome.summary[:200]}"
|
||||
return f"stopped: {outcome.message[:200]}"
|
||||
|
||||
|
||||
def _current_phase(ledger: Ledger) -> str:
|
||||
for phase in PHASE_ORDER:
|
||||
if ledger.phase_status(phase).value != "approved":
|
||||
@@ -395,6 +443,7 @@ def _specialist_input(
|
||||
) -> dict[str, Any]:
|
||||
spec = DEPENDENCY_GRAPH[kind]
|
||||
return {
|
||||
"execution_id": state["execution_id"],
|
||||
"role": spec.specialist,
|
||||
"kind": kind.value,
|
||||
"attempt": attempt,
|
||||
@@ -446,14 +495,35 @@ def _describe_unfinished(outcome: Outcome) -> str:
|
||||
return f"Ended with an unexpected outcome: {type(outcome).__name__}."
|
||||
|
||||
|
||||
def _automatic_reply(prompt: Mapping[str, Any]) -> dict[str, Any]:
|
||||
def _automatic_reply(
|
||||
prompt: Mapping[str, Any], scripted_answers: Sequence[Sequence[str]] = ()
|
||||
) -> dict[str, Any]:
|
||||
"""The reply in a pre-approved Execution: approve, deny cloud reads, and answer
|
||||
questions from scripted (pattern, answer) pairs where one matches."""
|
||||
if "approval" in prompt:
|
||||
return {
|
||||
"approval": {"phase": prompt["approval"]["phase"], "decision": "approve"}
|
||||
}
|
||||
if "permission" in prompt:
|
||||
return {"permission": {prompt["permission"]["tier"]: False}}
|
||||
return {"reply": NO_USER_REPLY}
|
||||
lines = []
|
||||
unanswered = False
|
||||
for item in prompt.get("questions", []):
|
||||
answer = next(
|
||||
(
|
||||
scripted
|
||||
for pattern, scripted in scripted_answers
|
||||
if re.search(pattern, item["question"], re.IGNORECASE)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if answer is None:
|
||||
unanswered = True
|
||||
else:
|
||||
lines.append(f"Q: {item['question']}\nA: {answer}")
|
||||
if unanswered or not lines:
|
||||
lines.append(NO_USER_REPLY)
|
||||
return {"reply": "\n\n".join(lines)}
|
||||
|
||||
|
||||
def _reply_text(reply: object) -> str:
|
||||
@@ -1,43 +0,0 @@
|
||||
"""Phase 0a: Existing Source Environment Discovery Node."""
|
||||
|
||||
import logging
|
||||
import re
|
||||
from typing import Any, Dict
|
||||
from app.states.state import GCPArchitectureState
|
||||
from app.skills.loader import SkillLoader
|
||||
from app.tools.gcp_scanner import GCPEnvironmentScanner
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def source_discover_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dict[str, Any]:
|
||||
"""Processes Phase 0a Source Environment Discovery: documents existing as-is architecture before target migration."""
|
||||
logger.info("Executing source_discover_node")
|
||||
skill_prompt = skill_loader.format_skills_for_prompt("source_discover")
|
||||
|
||||
# Extract target GCP project ID from request if specified
|
||||
request_summary = state.get("workflow_request", "")
|
||||
project_id = state.get("project_id")
|
||||
if not project_id and request_summary:
|
||||
match = re.search(r"project\s+([a-z0-9-]+)", request_summary, re.IGNORECASE)
|
||||
if match:
|
||||
project_id = match.group(1)
|
||||
|
||||
# Run live GCP environment scanner
|
||||
scanner = GCPEnvironmentScanner(project_id=project_id)
|
||||
scan_result = scanner.scan_environment()
|
||||
report = scanner.format_scan_report(scan_result)
|
||||
|
||||
source_discovery_doc = report["doc"]
|
||||
source_mermaid_diagram = report["mermaid"]
|
||||
|
||||
active_skills = state.get("active_skills", [])
|
||||
if "source_discovery" not in active_skills:
|
||||
active_skills.append("source_discovery")
|
||||
|
||||
return {
|
||||
"source_discovery_doc": source_discovery_doc,
|
||||
"source_mermaid_diagram": source_mermaid_diagram,
|
||||
"current_phase": "source_discover",
|
||||
"active_skills": active_skills,
|
||||
}
|
||||
@@ -1,60 +0,0 @@
|
||||
"""Phase 2: Pre-deployment Validation Node."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict
|
||||
from app.states.state import GCPArchitectureState
|
||||
from app.skills.loader import SkillLoader
|
||||
from app.tools.validation_tools import (
|
||||
validate_mermaid_diagram,
|
||||
validate_terraform_syntax,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def validate_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dict[str, Any]:
|
||||
"""Processes Phase 2 Validation: verify diagram, Terraform IaC, and artifact integrity."""
|
||||
logger.info("Executing validate_node")
|
||||
skill_prompt = skill_loader.format_skills_for_prompt("validate")
|
||||
|
||||
mermaid_content = state.get("mermaid_diagram", "")
|
||||
terraform_content = state.get("terraform_code", "")
|
||||
|
||||
mermaid_check = validate_mermaid_diagram.invoke({"diagram_content": mermaid_content})
|
||||
terraform_check = validate_terraform_syntax.invoke({"terraform_content": terraform_content})
|
||||
|
||||
errors = []
|
||||
if not mermaid_check.get("valid"):
|
||||
errors.append(f"Mermaid Check Failed: {mermaid_check.get('error')}")
|
||||
if not terraform_check.get("valid"):
|
||||
errors.append(f"Terraform Check Failed: {terraform_check.get('error')}")
|
||||
|
||||
validation_passed = len(errors) == 0
|
||||
|
||||
validation_results = f"""# Validation Results
|
||||
|
||||
## Summary
|
||||
- **Overall Validation Status**: {"PASS" if validation_passed else "FAIL"}
|
||||
- **Mermaid Diagram Syntax**: {"PASS" if mermaid_check.get("valid") else "FAIL"}
|
||||
- **Terraform Structural Check**: {"PASS" if terraform_check.get("valid") else "FAIL"}
|
||||
- **Resource Provisioning Triggered**: False (Static non-deployment check enforced)
|
||||
|
||||
## Verification Rules Checklist
|
||||
- [x] Functional & Non-functional requirements specified
|
||||
- [x] Product selection deferred during discovery and resolved in design phase
|
||||
- [x] Regional High Availability and Security IAM boundaries configured
|
||||
- [x] Mermaid diagram follows valid graph syntax
|
||||
- [x] Terraform HCL declares provider, resources, and least-privilege IAM bindings
|
||||
"""
|
||||
|
||||
active_skills = state.get("active_skills", [])
|
||||
if "validation_rules" not in active_skills:
|
||||
active_skills.append("validation_rules")
|
||||
|
||||
return {
|
||||
"validation_results": validation_results,
|
||||
"validation_passed": validation_passed,
|
||||
"errors": errors,
|
||||
"current_phase": "validate",
|
||||
"active_skills": active_skills,
|
||||
}
|
||||
@@ -7,9 +7,9 @@ import re
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import RoleSpec
|
||||
from app.execution.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||
from app.execution.engine import Interaction
|
||||
from app.adk.runtime import RoleSpec
|
||||
from app.agent import Interaction
|
||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||
|
||||
|
||||
def describe_prompt(prompt: Mapping[str, Any]) -> str:
|
||||
@@ -14,7 +14,7 @@ from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from app.agent_skills import SkillFormatError, SkillMeta, discover_skills
|
||||
from app.skills.agent_skills import SkillFormatError, SkillMeta, discover_skills
|
||||
|
||||
TIERS = ("general", "cloud_read")
|
||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
@@ -13,7 +13,7 @@ from collections import defaultdict
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any, Protocol
|
||||
|
||||
from app.execution.ledger import LedgerEvent
|
||||
from app.states.ledger import LedgerEvent
|
||||
|
||||
SESSION_AGENT_NAME = "solution-architect"
|
||||
|
||||
@@ -1,5 +1 @@
|
||||
"""Skills package."""
|
||||
|
||||
from .loader import LocalSkill, SkillLoader
|
||||
|
||||
__all__ = ["LocalSkill", "SkillLoader"]
|
||||
"""Agent Skills discovery and validation. The Skills themselves live in skills/ at the repository root."""
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
name: architecture_design
|
||||
phase: design
|
||||
description: Guidance for GCP product selection, architecture design markdown, Mermaid diagrams, and Terraform IaC generation.
|
||||
---
|
||||
|
||||
# Phase 1: Architecture Design & Product Selection Skill
|
||||
|
||||
When designing a Google Cloud Solution Architecture based on `docs/requirements.md`:
|
||||
|
||||
1. **Resolve Deferred Product Choices**: Select Google Cloud managed services suited for regional high availability and scale:
|
||||
- Compute: Cloud Run / Cloud Functions / GKE.
|
||||
- Messaging & Ingestion: Pub/Sub, Eventarc, or Cloud Tasks.
|
||||
- State & Storage: Cloud Storage, Firestore, Cloud SQL, or Spanner.
|
||||
- Identity & Security: Cloud IAM, Secret Manager, KMS, Artifact Registry.
|
||||
2. **Architecture Documentation (`docs/architecture.md`)**:
|
||||
- Provide executive summary, component responsibilities, data flow, security model, and cost model.
|
||||
3. **Mermaid Diagram (`architecture.mmd`)**:
|
||||
- Render clean, valid Mermaid syntax (`graph TD` or `flowchart TD`) mapping client ingress, compute, messaging, and storage components.
|
||||
4. **Terraform Infrastructure as Code (`terraform/`)**:
|
||||
- Produce valid Terraform HCL files (`main.tf`, `variables.tf`, `outputs.tf`, `versions.tf`).
|
||||
- Ensure planability without credentials or resource provisioning during validation.
|
||||
@@ -1,105 +0,0 @@
|
||||
"""Dynamic Local Skill Loader."""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional
|
||||
import yaml
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LocalSkill:
|
||||
"""Represents a local skill loaded from SKILL.md."""
|
||||
|
||||
name: str
|
||||
phase: str
|
||||
description: str
|
||||
content: str
|
||||
filepath: Path
|
||||
|
||||
|
||||
class SkillLoader:
|
||||
"""Discovers and parses local skills from SKILL.md files."""
|
||||
|
||||
def __init__(self, skills_dir: Path) -> None:
|
||||
self.skills_dir = skills_dir
|
||||
self._skills: Dict[str, LocalSkill] = {}
|
||||
|
||||
def load_skills(self) -> Dict[str, LocalSkill]:
|
||||
"""Scan skills_dir and load all valid SKILL.md files."""
|
||||
self._skills.clear()
|
||||
if not self.skills_dir.exists():
|
||||
logger.warning("Skills directory does not exist: %s", self.skills_dir)
|
||||
return self._skills
|
||||
|
||||
for skill_file in self.skills_dir.glob("**/SKILL.md"):
|
||||
try:
|
||||
skill = self._parse_skill_file(skill_file)
|
||||
if skill:
|
||||
self._skills[skill.name] = skill
|
||||
logger.info("Loaded skill '%s' (phase: %s)", skill.name, skill.phase)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to parse skill file %s: %s", skill_file, exc)
|
||||
|
||||
return self._skills
|
||||
|
||||
def get_skill(self, name: str) -> Optional[LocalSkill]:
|
||||
"""Get skill by name."""
|
||||
if not self._skills:
|
||||
self.load_skills()
|
||||
return self._skills.get(name)
|
||||
|
||||
def get_skills_by_phase(self, phase: str) -> List[LocalSkill]:
|
||||
"""Get all skills matching a specific workflow phase."""
|
||||
if not self._skills:
|
||||
self.load_skills()
|
||||
return [skill for skill in self._skills.values() if skill.phase == phase]
|
||||
|
||||
def format_skills_for_prompt(self, phase: Optional[str] = None) -> str:
|
||||
"""Format skill instruction content into a single string for system prompt injection."""
|
||||
skills = self.get_skills_by_phase(phase) if phase else list(self._skills.values())
|
||||
if not skills:
|
||||
return ""
|
||||
|
||||
prompt_parts = ["## Injected Skill Instructions\n"]
|
||||
for skill in skills:
|
||||
prompt_parts.append(f"### Skill: {skill.name} (Phase: {skill.phase})\n{skill.content}\n")
|
||||
return "\n".join(prompt_parts)
|
||||
|
||||
def _parse_skill_file(self, filepath: Path) -> Optional[LocalSkill]:
|
||||
"""Parse frontmatter and markdown content from SKILL.md."""
|
||||
raw_text = filepath.read_text(encoding="utf-8")
|
||||
if not raw_text.startswith("---"):
|
||||
return LocalSkill(
|
||||
name=filepath.parent.name,
|
||||
phase="general",
|
||||
description="",
|
||||
content=raw_text,
|
||||
filepath=filepath,
|
||||
)
|
||||
|
||||
parts = raw_text.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
return None
|
||||
|
||||
frontmatter_raw = parts[1]
|
||||
content = parts[2].strip()
|
||||
|
||||
try:
|
||||
metadata = yaml.safe_load(frontmatter_raw) or {}
|
||||
except Exception:
|
||||
metadata = {}
|
||||
|
||||
name = metadata.get("name", filepath.parent.name)
|
||||
phase = metadata.get("phase", "general")
|
||||
description = metadata.get("description", "")
|
||||
|
||||
return LocalSkill(
|
||||
name=name,
|
||||
phase=phase,
|
||||
description=description,
|
||||
content=content,
|
||||
filepath=filepath,
|
||||
)
|
||||
@@ -1,37 +0,0 @@
|
||||
---
|
||||
name: packaging_guide
|
||||
phase: package
|
||||
description: Guidance for assembling the comprehensive solution-architecture-guide.md following Google's official 8-section output template.
|
||||
---
|
||||
|
||||
# Phase 3: Solution Packaging Skill
|
||||
|
||||
When producing `solution-architecture-guide.md`, follow Google's official 8-section solution architecture output template:
|
||||
|
||||
1. **Executive summary and workload overview**
|
||||
- High-level business goals, workload description, and proposed solution architecture summary.
|
||||
2. **Requirements and current state**
|
||||
- **2.1. Functional requirements**: Business processes, key activities, and use cases.
|
||||
- **2.2. Non-functional requirements**: Security, reliability, cost, operations, performance, and sustainability requirements.
|
||||
- **2.3. Current state**: Existing As-Is architecture, pain points, and migration drivers.
|
||||
- **2.4. Dependencies**: Internal and external system dependencies.
|
||||
3. **Technical decomposition of the workload**
|
||||
- Layered breakdown of application tiers and logical components.
|
||||
4. **Proposed solution architecture**
|
||||
- **4.1. Google Cloud products and features mapping**: Selected products mapping table with justifications, doc citations, and alternatives considered.
|
||||
- **4.2. Architecture diagram**: Embedded Mermaid diagrams for As-Is (Before) and Target Google Cloud (After) architecture.
|
||||
- **4.3. Architecture description**: Data flow and task/control flow between components.
|
||||
5. **Design and configuration recommendations** (Architecture Framework Pillars)
|
||||
- 5.1 Security, privacy, and compliance (IAM, encryption, network security)
|
||||
- 5.2 Reliability (Redundancy, disaster recovery, RTO/RPO)
|
||||
- 5.3 Operational excellence (Monitoring, logging, IaC)
|
||||
- 5.4 Cost optimization (Sizing, autoscaling, pricing models)
|
||||
- 5.5 Performance efficiency (Caching, database & query optimization)
|
||||
- 5.6 Sustainability (Resource efficiency, serverless adoption)
|
||||
6. **Deployment guidance**
|
||||
- **6.1. Deployment prerequisites**: Required GCP APIs, SDK tools, and permissions.
|
||||
- **6.2. Step-by-step deployment instructions**: Complete Terraform IaC blueprint (`terraform/main.tf`) and CLI execution runbook.
|
||||
7. **Validation plan**
|
||||
- Pre-deployment validation results, dry-run checks, and Verification Checklist.
|
||||
8. **References**
|
||||
- Citations and documentation links grounded by Google Developer Knowledge MCP.
|
||||
@@ -1,19 +0,0 @@
|
||||
---
|
||||
name: requirements_discovery
|
||||
phase: discover
|
||||
description: Guidance for discovering, analyzing, and documenting GCP solution requirements.
|
||||
---
|
||||
|
||||
# Phase 0: Requirements Discovery Skill
|
||||
|
||||
When performing requirements discovery for a Google Cloud Solution Architecture:
|
||||
|
||||
1. **Analyze Workflow Request**: Extract functional and non-functional requirements from the user request or baseline specification.
|
||||
2. **Defer Product Selection**: During discovery, product selection must explicitly be marked as `deferred: true`. Do not commit to specific GCP products (e.g. Cloud Run vs GKE) until the design phase.
|
||||
3. **Capture Core Requirements**:
|
||||
- Authenticated HTTPS ingress, stateless processing, asynchronous domain events.
|
||||
- High availability within a selected GCP region.
|
||||
- At-least-once delivery with idempotent processing.
|
||||
- Least-privilege IAM service accounts and private networking where practical.
|
||||
4. **Document Assumptions and Open Questions**: Highlight unknowns regarding scale, traffic spikes, compliance, and specific region requirements.
|
||||
5. **Output Standard**: Produce a valid `docs/requirements.md` file matching the phase spec.
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
name: source_discovery
|
||||
phase: source_discover
|
||||
description: Guidance for discovering, auditing, and documenting the existing pre-emptive source environment before target migration.
|
||||
---
|
||||
|
||||
# Phase 0a: Existing Source Environment Discovery Skill
|
||||
|
||||
When auditing and discovering an existing pre-emptive source environment:
|
||||
|
||||
1. **Assess As-Is Architecture**: Analyze existing workloads, on-premises applications, legacy infrastructure, or existing cloud provider setups.
|
||||
2. **Identify Existing Components**:
|
||||
- Monolithic / legacy application services & ingress endpoints.
|
||||
- Self-hosted database engines (e.g. MySQL, PostgreSQL, Oracle) or legacy file shares.
|
||||
- Traditional messaging queues (e.g. RabbitMQ, ActiveMQ).
|
||||
- Network topology, firewall rules, and authentication mechanisms.
|
||||
3. **Capture Source Metrics & Pain Points**:
|
||||
- Current scale, capacity constraints, downtime risks, and operational bottlenecks.
|
||||
- Security vulnerabilities, compliance gaps, and maintenance costs.
|
||||
4. **Document Source Architecture**:
|
||||
- Produce a clear `docs/source-architecture.md` baseline documenting the "Before" state.
|
||||
- Render a Mermaid diagram representing the existing source topology (`source-architecture.mmd`).
|
||||
@@ -1,19 +0,0 @@
|
||||
---
|
||||
name: validation_rules
|
||||
phase: validate
|
||||
description: Guidance for verifying Terraform syntax, Mermaid diagrams, and repository artifact completeness.
|
||||
---
|
||||
|
||||
# Phase 2: Pre-Deployment Validation Skill
|
||||
|
||||
When validating solution architecture artifacts:
|
||||
|
||||
1. **Static File Validation**:
|
||||
- Verify existence and non-emptiness of `workflow.yaml`, `requirements.yaml`, `docs/requirements.md`, `docs/architecture.md`, `architecture.mmd`, `terraform/main.tf`, `solution-architecture-guide.md`.
|
||||
2. **Mermaid Syntax Validation**:
|
||||
- Check diagram for matching subgraphs, valid node definitions, arrow syntaxes, and clean structure.
|
||||
3. **Terraform Integrity**:
|
||||
- Ensure Terraform modules, variables, and resource blocks adhere to Google Cloud provider standards.
|
||||
- Ensure static non-provisioning check passes (`deploy_resources: false`).
|
||||
4. **Output Report**:
|
||||
- Generate `validation-results.md` summarizing pass/fail status across all verification rules.
|
||||
@@ -1,5 +1 @@
|
||||
"""States package."""
|
||||
|
||||
from .state import GCPArchitectureState
|
||||
|
||||
__all__ = ["GCPArchitectureState"]
|
||||
"""Checkpointed Execution state and the Ledger that guards it."""
|
||||
|
||||
@@ -33,3 +33,4 @@ class ExecutionState(TypedDict, total=False):
|
||||
turns: int
|
||||
status: str # running | completed | completed_with_warnings
|
||||
warnings: list[str]
|
||||
scripted_answers: list[list[str]] # [regex, answer], pre-approved runs only
|
||||
@@ -12,7 +12,7 @@ from dataclasses import asdict, dataclass, field
|
||||
from enum import StrEnum
|
||||
from typing import Any
|
||||
|
||||
from app.execution.deliverables import (
|
||||
from app.workflows.deliverables import (
|
||||
DEPENDENCY_GRAPH,
|
||||
PHASE_ORDER,
|
||||
DeliverableKind,
|
||||
@@ -1,31 +0,0 @@
|
||||
"""State definition for GCP Solution Architecture Agent workflow."""
|
||||
|
||||
from typing import Any, Dict, List, Optional, TypedDict
|
||||
|
||||
|
||||
class GCPArchitectureState(TypedDict, total=False):
|
||||
"""LangGraph State tracking state across all workflow phases."""
|
||||
|
||||
# Workflow Request / Input Intent
|
||||
workflow_request: str
|
||||
target_dir: str
|
||||
|
||||
# Pre-emptive Source Environment Discovery Outputs (Before State)
|
||||
source_discovery_doc: Optional[str]
|
||||
source_mermaid_diagram: Optional[str]
|
||||
|
||||
# Target Phase Outputs (After State)
|
||||
requirements_doc: Optional[str]
|
||||
architecture_doc: Optional[str]
|
||||
mermaid_diagram: Optional[str]
|
||||
terraform_code: Optional[str]
|
||||
validation_results: Optional[str]
|
||||
solution_guide: Optional[str]
|
||||
|
||||
# Workflow Metadata & Dynamic Skill Context
|
||||
product_selection_deferred: bool
|
||||
current_phase: str
|
||||
active_skills: List[str]
|
||||
errors: List[str]
|
||||
validation_passed: bool
|
||||
status_summary: Dict[str, Any]
|
||||
@@ -1,27 +1 @@
|
||||
"""Tools package."""
|
||||
|
||||
from .gcp_scanner import GCPEnvironmentScanner
|
||||
from .mcp_developer_knowledge import (
|
||||
DeveloperKnowledgeMCPClient,
|
||||
developerknowledge_answer_query,
|
||||
developerknowledge_get_documents,
|
||||
developerknowledge_search_documents,
|
||||
get_mcp_client,
|
||||
)
|
||||
from .validation_tools import (
|
||||
validate_mermaid_diagram,
|
||||
validate_repository_artifacts,
|
||||
validate_terraform_syntax,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"validate_mermaid_diagram",
|
||||
"validate_repository_artifacts",
|
||||
"validate_terraform_syntax",
|
||||
"developerknowledge_search_documents",
|
||||
"developerknowledge_get_documents",
|
||||
"developerknowledge_answer_query",
|
||||
"DeveloperKnowledgeMCPClient",
|
||||
"get_mcp_client",
|
||||
"GCPEnvironmentScanner",
|
||||
]
|
||||
|
||||
@@ -11,21 +11,23 @@ from collections.abc import Mapping, Sequence
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import (
|
||||
from app.adk.runtime import (
|
||||
FILES_SCHEMA,
|
||||
NeedsInput,
|
||||
Submitted,
|
||||
ToolResult,
|
||||
ToolSpec,
|
||||
TurnEnded,
|
||||
files_from_argument,
|
||||
)
|
||||
from app.execution.deliverables import (
|
||||
from app.states.ledger import Ledger, LedgerRefusal, PhaseStatus
|
||||
from app.workflows.deliverables import (
|
||||
DEPENDENCY_GRAPH,
|
||||
DeliverableKind,
|
||||
PhaseId,
|
||||
check_submission,
|
||||
deliverables_in,
|
||||
)
|
||||
from app.execution.ledger import Ledger, LedgerRefusal, PhaseStatus
|
||||
|
||||
ORCHESTRATOR = "orchestrator"
|
||||
CLOUD_READ = "cloud_read"
|
||||
@@ -232,10 +234,11 @@ def specialist_tools(
|
||||
) -> list[ToolSpec]:
|
||||
"""A Specialist's tools: submit, ask, and its role's tools with the cloud-read gate."""
|
||||
|
||||
async def submit_deliverable(
|
||||
content: str, attachments: Mapping[str, str] | None = None
|
||||
) -> ToolResult:
|
||||
attachments = dict(attachments or {})
|
||||
async def submit_deliverable(content: str, attachments: Any = None) -> ToolResult:
|
||||
try:
|
||||
attachments = files_from_argument(attachments)
|
||||
except TypeError as exc:
|
||||
return _refused(f"attachments: {exc}")
|
||||
unsafe = [name for name in attachments if not _safe_relative(name)]
|
||||
if unsafe:
|
||||
return _refused(
|
||||
@@ -262,10 +265,7 @@ def specialist_tools(
|
||||
_schema(
|
||||
{
|
||||
"content": {"type": "string"},
|
||||
"attachments": {
|
||||
"type": "object",
|
||||
"additionalProperties": {"type": "string"},
|
||||
},
|
||||
"attachments": FILES_SCHEMA,
|
||||
},
|
||||
["content"],
|
||||
),
|
||||
@@ -1,346 +0,0 @@
|
||||
"""Live Google Cloud Environment & Resource Scanner.
|
||||
|
||||
Discovers and audits existing resources in a target Google Cloud project (GCS buckets,
|
||||
Compute Engine instances, Cloud SQL, Pub/Sub, Cloud Run) using Google Cloud REST APIs.
|
||||
Authenticated via Application Default Credentials (ADC), Service Account keys, or gcloud CLI tokens.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Any, Dict, List, Optional
|
||||
import httpx
|
||||
|
||||
try:
|
||||
import google.auth
|
||||
import google.auth.transport.requests
|
||||
HAS_GOOGLE_AUTH = True
|
||||
except ImportError:
|
||||
HAS_GOOGLE_AUTH = False
|
||||
|
||||
from app.config import get_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GCPEnvironmentScanner:
|
||||
"""Scans and audits live resources in a target Google Cloud project."""
|
||||
|
||||
def __init__(self, project_id: Optional[str] = None) -> None:
|
||||
settings = get_settings()
|
||||
self.project_id = project_id or settings.GCP_PROJECT_ID or os.getenv("GCP_PROJECT_ID")
|
||||
self.timeout = httpx.Timeout(10.0)
|
||||
if not self.project_id:
|
||||
self._auto_detect_project()
|
||||
|
||||
def _auto_detect_project(self) -> None:
|
||||
"""Attempt auto-detecting project ID via gcloud CLI."""
|
||||
try:
|
||||
cmd = ["gcloud", "config", "get-value", "project", "--quiet"]
|
||||
proj = subprocess.check_output(cmd, text=True, stderr=subprocess.DEVNULL).strip()
|
||||
if proj and proj != "(unset)":
|
||||
self.project_id = proj
|
||||
logger.info("Auto-detected GCP project_id: '%s'", proj)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _get_access_token(self) -> Optional[str]:
|
||||
"""Obtain a valid OAuth2 bearer token from GCP_ACCESS_TOKEN env, service account key, google.auth, or gcloud CLI."""
|
||||
# Method 0: Direct access token passed via environment
|
||||
env_token = os.getenv("GCP_ACCESS_TOKEN")
|
||||
if env_token and env_token.strip():
|
||||
logger.info("Obtained GCP OAuth2 bearer token from GCP_ACCESS_TOKEN environment variable.")
|
||||
return env_token.strip()
|
||||
|
||||
settings = get_settings()
|
||||
key_path = settings.GOOGLE_APPLICATION_CREDENTIALS or os.getenv("GOOGLE_APPLICATION_CREDENTIALS")
|
||||
|
||||
# Method 1: Try Service Account Key file directly if specified
|
||||
if key_path and os.path.isfile(key_path) and HAS_GOOGLE_AUTH:
|
||||
try:
|
||||
from google.oauth2 import service_account
|
||||
scopes = ["https://www.googleapis.com/auth/cloud-platform"]
|
||||
creds = service_account.Credentials.from_service_account_file(key_path, scopes=scopes)
|
||||
auth_req = google.auth.transport.requests.Request()
|
||||
creds.refresh(auth_req)
|
||||
if creds.project_id and not self.project_id:
|
||||
self.project_id = creds.project_id
|
||||
if creds.token:
|
||||
logger.info("Obtained GCP token via Service Account Key file: %s", key_path)
|
||||
return creds.token
|
||||
except Exception as exc:
|
||||
logger.debug("Service account key auth failed for %s: %s", key_path, exc)
|
||||
|
||||
# Method 2: Try google.auth.default()
|
||||
if HAS_GOOGLE_AUTH:
|
||||
try:
|
||||
creds, proj = google.auth.default(
|
||||
scopes=["https://www.googleapis.com/auth/cloud-platform"]
|
||||
)
|
||||
auth_req = google.auth.transport.requests.Request()
|
||||
creds.refresh(auth_req)
|
||||
if not self.project_id and proj:
|
||||
self.project_id = proj
|
||||
if creds.token:
|
||||
logger.info("Obtained GCP token via Application Default Credentials.")
|
||||
return creds.token
|
||||
except Exception as exc:
|
||||
logger.debug("google.auth.default token acquisition failed: %s", exc)
|
||||
|
||||
# Method 3: Fallback to gcloud CLI token (with --quiet)
|
||||
try:
|
||||
cmd = ["gcloud", "auth", "print-access-token", "--quiet"]
|
||||
token = subprocess.check_output(cmd, text=True, stderr=subprocess.DEVNULL).strip()
|
||||
if token:
|
||||
logger.info("Obtained GCP token via gcloud auth print-access-token --quiet.")
|
||||
return token
|
||||
except Exception as exc:
|
||||
logger.debug("gcloud auth print-access-token fallback failed: %s", exc)
|
||||
|
||||
return None
|
||||
|
||||
def scan_environment(self) -> Dict[str, Any]:
|
||||
"""Execute a full live environment scan against the target GCP project."""
|
||||
token = self._get_access_token()
|
||||
if not token or not self.project_id:
|
||||
logger.warning(
|
||||
"GCP Scanner unable to authenticate or project_id missing (project_id=%s, token_present=%s). "
|
||||
"Using fallback baseline template.",
|
||||
self.project_id,
|
||||
bool(token),
|
||||
)
|
||||
return {"authenticated": False, "project_id": self.project_id, "resources": {}}
|
||||
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
resources: Dict[str, Any] = {
|
||||
"project_id": self.project_id,
|
||||
"gcs_buckets": [],
|
||||
"compute_instances": [],
|
||||
"cloud_sql_instances": [],
|
||||
"pubsub_topics": [],
|
||||
"cloud_run_services": [],
|
||||
"disabled_apis": [],
|
||||
}
|
||||
|
||||
with httpx.Client(timeout=self.timeout) as client:
|
||||
# 1. Scan GCS Buckets
|
||||
try:
|
||||
r = client.get(
|
||||
f"https://storage.googleapis.com/storage/v1/b?project={self.project_id}",
|
||||
headers=headers,
|
||||
)
|
||||
if r.status_code == 200:
|
||||
items = r.json().get("items", [])
|
||||
resources["gcs_buckets"] = [
|
||||
{"name": item.get("name"), "location": item.get("location"), "storage_class": item.get("storageClass")}
|
||||
for item in items
|
||||
]
|
||||
elif r.status_code == 403 and "SERVICE_DISABLED" in r.text:
|
||||
resources["disabled_apis"].append("storage.googleapis.com")
|
||||
except Exception as exc:
|
||||
logger.debug("GCS scan error: %s", exc)
|
||||
|
||||
# 2. Scan Compute Engine Instances
|
||||
try:
|
||||
r = client.get(
|
||||
f"https://compute.googleapis.com/compute/v1/projects/{self.project_id}/aggregated/instances",
|
||||
headers=headers,
|
||||
)
|
||||
if r.status_code == 200:
|
||||
items_dict = r.json().get("items", {})
|
||||
instances = []
|
||||
for zone, zone_data in items_dict.items():
|
||||
for inst in zone_data.get("instances", []):
|
||||
instances.append({
|
||||
"name": inst.get("name"),
|
||||
"zone": zone.replace("zones/", ""),
|
||||
"status": inst.get("status"),
|
||||
"machine_type": inst.get("machineType", "").split("/")[-1],
|
||||
})
|
||||
resources["compute_instances"] = instances
|
||||
elif r.status_code == 403 and "SERVICE_DISABLED" in r.text:
|
||||
resources["disabled_apis"].append("compute.googleapis.com")
|
||||
except Exception as exc:
|
||||
logger.debug("Compute Engine scan error: %s", exc)
|
||||
|
||||
# 3. Scan Cloud SQL Instances
|
||||
try:
|
||||
r = client.get(
|
||||
f"https://sqladmin.googleapis.com/v1/projects/{self.project_id}/instances",
|
||||
headers=headers,
|
||||
)
|
||||
if r.status_code == 200:
|
||||
items = r.json().get("items", [])
|
||||
resources["cloud_sql_instances"] = [
|
||||
{
|
||||
"name": item.get("name"),
|
||||
"database_version": item.get("databaseVersion"),
|
||||
"region": item.get("region"),
|
||||
"state": item.get("state"),
|
||||
}
|
||||
for item in items
|
||||
]
|
||||
elif r.status_code == 403 and "SERVICE_DISABLED" in r.text:
|
||||
resources["disabled_apis"].append("sqladmin.googleapis.com")
|
||||
except Exception as exc:
|
||||
logger.debug("Cloud SQL scan error: %s", exc)
|
||||
|
||||
# 4. Scan Pub/Sub Topics
|
||||
try:
|
||||
r = client.get(
|
||||
f"https://pubsub.googleapis.com/v1/projects/{self.project_id}/topics",
|
||||
headers=headers,
|
||||
)
|
||||
if r.status_code == 200:
|
||||
topics = r.json().get("topics", [])
|
||||
resources["pubsub_topics"] = [
|
||||
{"name": t.get("name", "").split("/")[-1]} for t in topics
|
||||
]
|
||||
elif r.status_code == 403 and "SERVICE_DISABLED" in r.text:
|
||||
resources["disabled_apis"].append("pubsub.googleapis.com")
|
||||
except Exception as exc:
|
||||
logger.debug("Pub/Sub scan error: %s", exc)
|
||||
|
||||
# 5. Scan Cloud Run Services
|
||||
try:
|
||||
r = client.get(
|
||||
f"https://run.googleapis.com/v2/projects/{self.project_id}/locations/-/services",
|
||||
headers=headers,
|
||||
)
|
||||
if r.status_code == 200:
|
||||
services = r.json().get("services", [])
|
||||
resources["cloud_run_services"] = [
|
||||
{"name": s.get("name", "").split("/")[-1], "uri": s.get("uri")}
|
||||
for s in services
|
||||
]
|
||||
elif r.status_code == 403 and "SERVICE_DISABLED" in r.text:
|
||||
resources["disabled_apis"].append("run.googleapis.com")
|
||||
except Exception as exc:
|
||||
logger.debug("Cloud Run scan error: %s", exc)
|
||||
|
||||
return {"authenticated": True, "project_id": self.project_id, "resources": resources}
|
||||
|
||||
def format_scan_report(self, scan_result: Dict[str, Any]) -> Dict[str, str]:
|
||||
"""Format live environment scan findings into markdown report and Mermaid diagram."""
|
||||
if not scan_result.get("authenticated") or not scan_result.get("resources"):
|
||||
# Fallback
|
||||
doc = """# Pre-emptive Source Environment Discovery (As-Is Architecture)
|
||||
|
||||
## Existing Workload Audit
|
||||
- **Workload Summary**: Event-Driven Regional Application (Legacy / Pre-existing Environment)
|
||||
- **Current Hosting**: On-Premises Data Center / Legacy VM Infrastructure
|
||||
- **Ingress Layer**: Self-managed NGINX Reverse Proxy listening on HTTP/HTTPS
|
||||
- **Application Runtime**: Monolithic Application Instance (Single Point of Failure)
|
||||
- **Database Layer**: Self-hosted PostgreSQL Instance (Unreplicated, Local Disk)
|
||||
- **Queue / Messaging**: Local RabbitMQ Queue Instance
|
||||
|
||||
## Current Operational Pain Points & Bottlenecks
|
||||
- Single-instance compute leading to downtime during maintenance windows.
|
||||
- Manual scaling capabilities unable to handle unexpected traffic spikes.
|
||||
- Unencrypted local storage and unmanaged backups creating data loss risks.
|
||||
- Elevated operational overhead and hardware lifecycle costs.
|
||||
|
||||
## Source Component Topology
|
||||
- `Client` -> `NGINX Proxy` -> `Monolith Application` -> `Local PostgreSQL / RabbitMQ`
|
||||
"""
|
||||
mmd = """flowchart TD
|
||||
Client[External Client] -->|HTTP/HTTPS| NginxProxy[Legacy NGINX Proxy]
|
||||
NginxProxy --> MonolithApp[Monolithic Application VM]
|
||||
MonolithApp --> LocalDB[(Self-Hosted PostgreSQL)]
|
||||
MonolithApp --> LocalQueue[Local RabbitMQ Queue]
|
||||
"""
|
||||
return {"doc": doc, "mermaid": mmd}
|
||||
|
||||
res = scan_result["resources"]
|
||||
proj_id = res.get("project_id", "Unknown")
|
||||
buckets = res.get("gcs_buckets", [])
|
||||
instances = res.get("compute_instances", [])
|
||||
sql_instances = res.get("cloud_sql_instances", [])
|
||||
topics = res.get("pubsub_topics", [])
|
||||
run_services = res.get("cloud_run_services", [])
|
||||
disabled_apis = res.get("disabled_apis", [])
|
||||
|
||||
# Build Markdown Document
|
||||
doc_lines = [
|
||||
f"# Pre-emptive Live GCP Environment Discovery (Project: `{proj_id}`)",
|
||||
"",
|
||||
"## Live Resource Audit",
|
||||
f"- **Target Google Cloud Project**: `{proj_id}`",
|
||||
f"- **Discovered Storage Buckets**: {len(buckets)} GCS Bucket(s)" if buckets else "- **Discovered Storage Buckets**: None / Default Bucket",
|
||||
f"- **Discovered Compute Instances**: {len(instances)} Compute VM(s)" if instances else "- **Discovered Compute Instances**: None active",
|
||||
f"- **Discovered Database Instances**: {len(sql_instances)} Cloud SQL DB(s)" if sql_instances else "- **Discovered Database Instances**: None active",
|
||||
f"- **Discovered Pub/Sub Topics**: {len(topics)} Topic(s)" if topics else "- **Discovered Pub/Sub Topics**: None active",
|
||||
f"- **Discovered Cloud Run Services**: {len(run_services)} Service(s)" if run_services else "- **Discovered Cloud Run Services**: None active",
|
||||
"",
|
||||
"## Resource Inventory Breakdown",
|
||||
]
|
||||
|
||||
if buckets:
|
||||
doc_lines.append("### Cloud Storage Buckets")
|
||||
for b in buckets:
|
||||
doc_lines.append(f"- `b/{b['name']}` (Location: `{b['location']}`, Storage Class: `{b['storage_class']}`)")
|
||||
doc_lines.append("")
|
||||
|
||||
if instances:
|
||||
doc_lines.append("### Compute Engine Instances")
|
||||
for inst in instances:
|
||||
doc_lines.append(f"- VM `instance/{inst['name']}` (Zone: `{inst['zone']}`, Type: `{inst['machine_type']}`, Status: `{inst['status']}`)")
|
||||
doc_lines.append("")
|
||||
|
||||
if sql_instances:
|
||||
doc_lines.append("### Cloud SQL Databases")
|
||||
for sql in sql_instances:
|
||||
doc_lines.append(f"- DB `sql/{sql['name']}` (Engine: `{sql['database_version']}`, Region: `{sql['region']}`, State: `{sql['state']}`)")
|
||||
doc_lines.append("")
|
||||
|
||||
if topics:
|
||||
doc_lines.append("### Cloud Pub/Sub Topics")
|
||||
for t in topics:
|
||||
doc_lines.append(f"- Topic `pubsub/{t['name']}`")
|
||||
doc_lines.append("")
|
||||
|
||||
if run_services:
|
||||
doc_lines.append("### Cloud Run Services")
|
||||
for s in run_services:
|
||||
doc_lines.append(f"- Service `run/{s['name']}` (URI: `{s['uri']}`)")
|
||||
doc_lines.append("")
|
||||
|
||||
if disabled_apis:
|
||||
doc_lines.append("### API Status Diagnostics")
|
||||
for api in disabled_apis:
|
||||
doc_lines.append(f"- `SERVICE_DISABLED`: `{api}` is not enabled on project `{proj_id}`.")
|
||||
doc_lines.append("")
|
||||
|
||||
doc_lines.extend([
|
||||
"## Current Operational Bottlenecks & Migration Drivers",
|
||||
"- As-is infrastructure requires serverless auto-scaling and managed high availability.",
|
||||
"- Need for declarative IaC management via Terraform.",
|
||||
"- Transition to least-privilege IAM service identities and automated CI validation.",
|
||||
])
|
||||
|
||||
# Build Mermaid Diagram
|
||||
mmd_lines = ["flowchart TD", f" subgraph GCPProject[\"Google Cloud Project: {proj_id}\"]"]
|
||||
if buckets:
|
||||
for i, b in enumerate(buckets[:3]):
|
||||
mmd_lines.append(f" GCS{i}[(\"GCS: {b['name']}\")]")
|
||||
if instances:
|
||||
for i, inst in enumerate(instances[:3]):
|
||||
mmd_lines.append(f" VM{i}[\"GCE: {inst['name']}\"]")
|
||||
if sql_instances:
|
||||
for i, sql in enumerate(sql_instances[:3]):
|
||||
mmd_lines.append(f" SQL{i}[(\"Cloud SQL: {sql['name']}\")]")
|
||||
if topics:
|
||||
for i, t in enumerate(topics[:3]):
|
||||
mmd_lines.append(f" PubSub{i}[\"PubSub: {t['name']}\"]")
|
||||
if run_services:
|
||||
for i, s in enumerate(run_services[:3]):
|
||||
mmd_lines.append(f" Run{i}[\"Cloud Run: {s['name']}\"]")
|
||||
|
||||
if not (buckets or instances or sql_instances or topics or run_services):
|
||||
mmd_lines.append(f" EmptyProject[\"Project {proj_id} (No Active Resources Detected)\"]")
|
||||
|
||||
mmd_lines.append(" end")
|
||||
mmd_lines.append(" Client[External Traffic] --> GCPProject")
|
||||
|
||||
return {"doc": "\n".join(doc_lines), "mermaid": "\n".join(mmd_lines)}
|
||||
@@ -10,12 +10,12 @@ import asyncio
|
||||
import os
|
||||
import re
|
||||
import tempfile
|
||||
from collections.abc import Iterable, Mapping
|
||||
from collections.abc import Awaitable, Callable, Iterable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path, PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
from app.agents.runtime import ToolResult, ToolSpec
|
||||
from app.adk.runtime import FILES_SCHEMA, ToolResult, ToolSpec, files_from_argument
|
||||
|
||||
_MAX_OUTPUT = 8_000
|
||||
_TERRAFORM_SUFFIXES = (".tf", ".tfvars", ".tf.json")
|
||||
@@ -108,7 +108,7 @@ class TerraformRunner:
|
||||
if refusal:
|
||||
return refusal
|
||||
with tempfile.TemporaryDirectory(prefix="tf-validate-") as tmp:
|
||||
root = _write(Path(tmp), files)
|
||||
root = _write(Path(tmp), files) / _module_dir(files)
|
||||
failure = await self._init(root)
|
||||
if failure:
|
||||
return failure
|
||||
@@ -127,7 +127,7 @@ class TerraformRunner:
|
||||
if bad:
|
||||
return ToolResult(f"Refused: invalid variable names {', '.join(bad)}")
|
||||
with tempfile.TemporaryDirectory(prefix="tf-plan-") as tmp:
|
||||
root = _write(Path(tmp), files)
|
||||
root = _write(Path(tmp), files) / _module_dir(files)
|
||||
failure = await self._init(root)
|
||||
if failure:
|
||||
return failure
|
||||
@@ -198,9 +198,20 @@ def _check_files(files: Mapping[str, str]) -> ToolResult | None:
|
||||
"Refused: file names must be relative .tf, .tf.json or .tfvars paths "
|
||||
f"without '..': {', '.join(bad)}"
|
||||
)
|
||||
directories = {str(PurePosixPath(name).parent) for name in files}
|
||||
if len(directories) > 1:
|
||||
return ToolResult(
|
||||
"Refused: put all files in one directory (one root module), for example "
|
||||
f"terraform/main.tf and terraform/variables.tf; got {', '.join(sorted(directories))}"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _module_dir(files: Mapping[str, str]) -> str:
|
||||
"""The one directory holding the files; Terraform runs there."""
|
||||
return str(PurePosixPath(next(iter(files))).parent)
|
||||
|
||||
|
||||
def _write(root: Path, files: Mapping[str, str]) -> Path:
|
||||
for name, content in files.items():
|
||||
target = root / name
|
||||
@@ -214,11 +225,26 @@ def _clip(text: str) -> str:
|
||||
|
||||
|
||||
_FILES_SCHEMA: dict[str, Any] = {
|
||||
"type": "object",
|
||||
"description": "Terraform files by relative path, e.g. {'main.tf': '...'}",
|
||||
"additionalProperties": {"type": "string"},
|
||||
**FILES_SCHEMA,
|
||||
"description": "Terraform files in one directory, e.g. terraform/main.tf",
|
||||
}
|
||||
|
||||
|
||||
def _with_files(
|
||||
method: Callable[..., Awaitable[ToolResult]],
|
||||
) -> Callable[..., Awaitable[ToolResult]]:
|
||||
"""Adapt a runner method to take the FILES_SCHEMA argument."""
|
||||
|
||||
async def handler(files: Any, **kwargs: Any) -> ToolResult:
|
||||
try:
|
||||
converted = files_from_argument(files)
|
||||
except TypeError as exc:
|
||||
return ToolResult(f"Refused: files: {exc}")
|
||||
return await method(converted, **kwargs)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
LOCAL_TOOLS: dict[str, tuple[str, str]] = {
|
||||
"mermaid_check": ("general", "Check a Mermaid diagram's basic structure."),
|
||||
"terraform_fmt": ("general", "Format Terraform files and return any that changed."),
|
||||
@@ -254,7 +280,7 @@ def local_tools(names: Iterable[str], runner: TerraformRunner) -> list[ToolSpec]
|
||||
"properties": {"files": _FILES_SCHEMA},
|
||||
"required": ["files"],
|
||||
},
|
||||
runner.fmt,
|
||||
_with_files(runner.fmt),
|
||||
tier="general",
|
||||
),
|
||||
"terraform_validate": lambda: ToolSpec(
|
||||
@@ -265,7 +291,7 @@ def local_tools(names: Iterable[str], runner: TerraformRunner) -> list[ToolSpec]
|
||||
"properties": {"files": _FILES_SCHEMA},
|
||||
"required": ["files"],
|
||||
},
|
||||
runner.validate,
|
||||
_with_files(runner.validate),
|
||||
tier="general",
|
||||
),
|
||||
"terraform_plan": lambda: ToolSpec(
|
||||
@@ -282,7 +308,7 @@ def local_tools(names: Iterable[str], runner: TerraformRunner) -> list[ToolSpec]
|
||||
},
|
||||
"required": ["files"],
|
||||
},
|
||||
runner.plan,
|
||||
_with_files(runner.plan),
|
||||
tier="cloud_read",
|
||||
),
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ from mcp import ClientSession
|
||||
from mcp.client.streamable_http import streamable_http_client
|
||||
from mcp.shared._httpx_utils import create_mcp_http_client
|
||||
|
||||
from app.agents.runtime import ToolResult, ToolSpec
|
||||
from app.adk.runtime import ToolResult, ToolSpec
|
||||
from app.providers import McpServerSpec
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -1,258 +0,0 @@
|
||||
"""Google Developer Knowledge MCP (Model Context Protocol) Client & Tools.
|
||||
|
||||
Provides integration with the Google Developer Knowledge MCP server (https://developerknowledge.googleapis.com/mcp).
|
||||
Implements official MCP tools:
|
||||
- developerknowledge:search_documents
|
||||
- developerknowledge:get_documents
|
||||
- developerknowledge:answer_query
|
||||
|
||||
Supports live HTTP/JSON-RPC MCP requests with built-in offline GCP knowledge fallback
|
||||
for offline testing and high availability.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional
|
||||
import httpx
|
||||
from langchain_core.tools import tool
|
||||
|
||||
from app.config import get_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Fallback Offline GCP Developer Knowledge Base
|
||||
OFFLINE_GCP_KNOWLEDGE_BASE: Dict[str, Dict[str, Any]] = {
|
||||
"cloud_run": {
|
||||
"title": "Google Cloud Run Architecture Guide",
|
||||
"uri": "https://cloud.google.com/run/docs/overview/what-is-cloud-run",
|
||||
"release_status": "GA (General Availability)",
|
||||
"category": "compute",
|
||||
"summary": "Stateless container execution platform with automatic scaling from zero to thousands of instances, built on Knative.",
|
||||
"citations": ["https://cloud.google.com/run/docs/securing/service-identity"],
|
||||
},
|
||||
"pubsub": {
|
||||
"title": "Google Cloud Pub/Sub Messaging Best Practices",
|
||||
"uri": "https://cloud.google.com/pubsub/docs/overview",
|
||||
"release_status": "GA (General Availability)",
|
||||
"category": "messaging",
|
||||
"summary": "Globally distributed, asynchronous message bus providing at-least-once delivery with dead-letter topics and exponential backoff.",
|
||||
"citations": ["https://cloud.google.com/pubsub/docs/dead-letter-topics"],
|
||||
},
|
||||
"storage": {
|
||||
"title": "Google Cloud Storage Object Lifecycle & Security",
|
||||
"uri": "https://cloud.google.com/storage/docs/overview",
|
||||
"release_status": "GA (General Availability)",
|
||||
"category": "storage",
|
||||
"summary": "Unified object storage with uniform bucket-level access, retention lifecycle rules, customer-managed encryption (CMEK), and audit logging.",
|
||||
"citations": ["https://cloud.google.com/storage/docs/uniform-bucket-level-access"],
|
||||
},
|
||||
"firestore": {
|
||||
"title": "Google Cloud Firestore Document Database",
|
||||
"uri": "https://cloud.google.com/firestore/docs/overview",
|
||||
"release_status": "GA (General Availability)",
|
||||
"category": "database",
|
||||
"summary": "Serverless, flexible NoSQL document database built for automatic scaling, rich queries, and ACID multi-document transactions.",
|
||||
"citations": ["https://cloud.google.com/firestore/docs/best-practices"],
|
||||
},
|
||||
"iam": {
|
||||
"title": "Google Cloud IAM Least-Privilege Identity Guide",
|
||||
"uri": "https://cloud.google.com/iam/docs/overview",
|
||||
"release_status": "GA (General Availability)",
|
||||
"category": "security",
|
||||
"summary": "Fine-grained access control and least-privilege service identity management for Google Cloud resources.",
|
||||
"citations": ["https://cloud.google.com/iam/docs/using-iam-securely"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class DeveloperKnowledgeMCPClient:
|
||||
"""Client for Google Developer Knowledge MCP server."""
|
||||
|
||||
def __init__(self, mcp_url: Optional[str] = None) -> None:
|
||||
settings = get_settings()
|
||||
self.mcp_url = mcp_url or settings.DEVELOPER_KNOWLEDGE_MCP_URL
|
||||
self.enabled = settings.DEVELOPER_KNOWLEDGE_MCP_ENABLED
|
||||
self.timeout = httpx.Timeout(5.0)
|
||||
|
||||
def search_documents(self, query: str, category: str = "all") -> Dict[str, Any]:
|
||||
"""Search Google Cloud reference architecture, decision-making, and best-practice documents."""
|
||||
if self.enabled and self.mcp_url:
|
||||
try:
|
||||
payload = {
|
||||
"jsonrpc": "2.0",
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "developerknowledge:search_documents",
|
||||
"arguments": {"query": query, "category": category},
|
||||
},
|
||||
"id": 1,
|
||||
}
|
||||
with httpx.Client(timeout=self.timeout) as client:
|
||||
resp = client.post(self.mcp_url, json=payload)
|
||||
if resp.status_code == 200:
|
||||
data = resp.json()
|
||||
if "result" in data:
|
||||
return {"source": "live_mcp", "data": data["result"]}
|
||||
except Exception as exc:
|
||||
logger.debug("Live MCP search_documents query failed (%s); using offline knowledge base.", exc)
|
||||
|
||||
# Offline fallback search
|
||||
results = []
|
||||
q_lower = query.lower()
|
||||
for key, doc in OFFLINE_GCP_KNOWLEDGE_BASE.items():
|
||||
if (
|
||||
q_lower in doc["title"].lower()
|
||||
or q_lower in doc["summary"].lower()
|
||||
or q_lower in doc["category"].lower()
|
||||
or category == "all"
|
||||
or category == doc["category"]
|
||||
):
|
||||
results.append(doc)
|
||||
|
||||
return {
|
||||
"source": "offline_fallback",
|
||||
"query": query,
|
||||
"category": category,
|
||||
"total_matches": len(results),
|
||||
"documents": results,
|
||||
}
|
||||
|
||||
def get_documents(self, document_uri: str) -> Dict[str, Any]:
|
||||
"""Fetch official Google Cloud document content and citations by URI."""
|
||||
if self.enabled and self.mcp_url:
|
||||
try:
|
||||
payload = {
|
||||
"jsonrpc": "2.0",
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "developerknowledge:get_documents",
|
||||
"arguments": {"document_uri": document_uri},
|
||||
},
|
||||
"id": 2,
|
||||
}
|
||||
with httpx.Client(timeout=self.timeout) as client:
|
||||
resp = client.post(self.mcp_url, json=payload)
|
||||
if resp.status_code == 200:
|
||||
data = resp.json()
|
||||
if "result" in data:
|
||||
return {"source": "live_mcp", "data": data["result"]}
|
||||
except Exception as exc:
|
||||
logger.debug("Live MCP get_documents query failed (%s); using offline knowledge base.", exc)
|
||||
|
||||
# Search matching offline doc
|
||||
for doc in OFFLINE_GCP_KNOWLEDGE_BASE.values():
|
||||
if document_uri in doc["uri"] or doc["uri"] in document_uri:
|
||||
return {"source": "offline_fallback", "document": doc}
|
||||
|
||||
return {
|
||||
"source": "offline_fallback",
|
||||
"document_uri": document_uri,
|
||||
"document": {
|
||||
"title": f"Google Cloud Documentation ({document_uri})",
|
||||
"uri": document_uri,
|
||||
"release_status": "GA",
|
||||
"summary": "Official Google Cloud architecture reference documentation.",
|
||||
"citations": [document_uri],
|
||||
},
|
||||
}
|
||||
|
||||
def answer_query(self, query: str) -> Dict[str, Any]:
|
||||
"""Answer architectural questions and check GCP product release statuses and best practices."""
|
||||
if self.enabled and self.mcp_url:
|
||||
try:
|
||||
payload = {
|
||||
"jsonrpc": "2.0",
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "developerknowledge:answer_query",
|
||||
"arguments": {"query": query},
|
||||
},
|
||||
"id": 3,
|
||||
}
|
||||
with httpx.Client(timeout=self.timeout) as client:
|
||||
resp = client.post(self.mcp_url, json=payload)
|
||||
if resp.status_code == 200:
|
||||
data = resp.json()
|
||||
if "result" in data:
|
||||
return {"source": "live_mcp", "answer": data["result"]}
|
||||
except Exception as exc:
|
||||
logger.debug("Live MCP answer_query failed (%s); using offline knowledge base.", exc)
|
||||
|
||||
# Offline grounding logic
|
||||
q_lower = query.lower()
|
||||
if "release status" in q_lower or "deprecated" in q_lower:
|
||||
return {
|
||||
"source": "offline_fallback",
|
||||
"query": query,
|
||||
"status_check": "All recommended products (Cloud Run, Pub/Sub, Cloud Storage, Firestore, Cloud IAM) are Active GA (General Availability). None are deprecated.",
|
||||
"supported": True,
|
||||
}
|
||||
|
||||
return {
|
||||
"source": "offline_fallback",
|
||||
"query": query,
|
||||
"answer": (
|
||||
"Google Cloud Architecture Best Practice: Design regional event-driven workloads "
|
||||
"using Cloud Run for stateless container execution, Pub/Sub for asynchronous message buffering, "
|
||||
"and Cloud Storage / Firestore for durable state retention under least-privilege IAM."
|
||||
),
|
||||
"citations": ["https://cloud.google.com/architecture/framework"],
|
||||
}
|
||||
|
||||
|
||||
# Singleton client instance
|
||||
_mcp_client: Optional[DeveloperKnowledgeMCPClient] = None
|
||||
|
||||
|
||||
def get_mcp_client() -> DeveloperKnowledgeMCPClient:
|
||||
"""Get singleton DeveloperKnowledgeMCPClient."""
|
||||
global _mcp_client
|
||||
if _mcp_client is None:
|
||||
_mcp_client = DeveloperKnowledgeMCPClient()
|
||||
return _mcp_client
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# LangChain @tool wrappers matching official Google spec
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@tool
|
||||
def developerknowledge_search_documents(query: str, category: str = "all") -> Dict[str, Any]:
|
||||
"""Searches Google Cloud reference architecture, decision-making, and best-practice documents.
|
||||
|
||||
Args:
|
||||
query: Search query string (e.g. 'Cloud Run PubSub event architecture').
|
||||
category: Optional category filter ('compute', 'messaging', 'storage', 'security', or 'all').
|
||||
|
||||
Returns:
|
||||
Dict containing matching Google Cloud documents and citations.
|
||||
"""
|
||||
client = get_mcp_client()
|
||||
return client.search_documents(query=query, category=category)
|
||||
|
||||
|
||||
@tool
|
||||
def developerknowledge_get_documents(document_uri: str) -> Dict[str, Any]:
|
||||
"""Retrieves official Google Cloud document content and citations by URI.
|
||||
|
||||
Args:
|
||||
document_uri: Official Google Cloud documentation URL or document identifier.
|
||||
|
||||
Returns:
|
||||
Dict containing document metadata, summary, and citations.
|
||||
"""
|
||||
client = get_mcp_client()
|
||||
return client.get_documents(document_uri=document_uri)
|
||||
|
||||
|
||||
@tool
|
||||
def developerknowledge_answer_query(query: str) -> Dict[str, Any]:
|
||||
"""Answers architectural questions and checks GCP product release statuses and best practices.
|
||||
|
||||
Args:
|
||||
query: Architectural question or product status query string.
|
||||
|
||||
Returns:
|
||||
Dict containing grounded answer, release status, and documentation citations.
|
||||
"""
|
||||
client = get_mcp_client()
|
||||
return client.answer_query(query=query)
|
||||
@@ -10,7 +10,7 @@ import asyncio
|
||||
import dataclasses
|
||||
from collections.abc import Mapping, Sequence
|
||||
|
||||
from app.agents.runtime import (
|
||||
from app.adk.runtime import (
|
||||
AgentRuntime,
|
||||
AgentTask,
|
||||
RoleSpec,
|
||||
@@ -9,12 +9,11 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping, Sequence
|
||||
|
||||
from app.agents.roles import PILLAR_ADVISOR
|
||||
from app.agents.runtime import AgentRuntime, RoleSpec, ToolSpec
|
||||
from app.execution.deliverables import DeliverableKind
|
||||
from app.execution.graph import ToolFactory
|
||||
from app.execution.pillars import pillar_consultation_tool
|
||||
from app.adk.roles import PILLAR_ADVISOR
|
||||
from app.adk.runtime import AgentRuntime, RoleSpec, ToolFactory, ToolSpec
|
||||
from app.tools.mcp import McpCatalog
|
||||
from app.tools.pillars import pillar_consultation_tool
|
||||
from app.workflows.deliverables import DeliverableKind
|
||||
|
||||
|
||||
def make_tool_factory(
|
||||
@@ -1,124 +0,0 @@
|
||||
"""LangChain decorated tools for artifact and schema validation."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
from typing import Dict, Any
|
||||
from langchain_core.tools import tool
|
||||
|
||||
|
||||
@tool
|
||||
def validate_mermaid_diagram(diagram_content: str) -> Dict[str, Any]:
|
||||
"""Validates the syntax and graph structure of a Mermaid diagram.
|
||||
|
||||
Args:
|
||||
diagram_content: Raw text content of the Mermaid diagram file.
|
||||
|
||||
Returns:
|
||||
Dict with status, valid boolean, and error details if any.
|
||||
"""
|
||||
if not diagram_content or not diagram_content.strip():
|
||||
return {"valid": False, "error": "Mermaid diagram is empty."}
|
||||
|
||||
if not re.search(r"\b(flowchart|graph)\b", diagram_content):
|
||||
return {
|
||||
"valid": False,
|
||||
"error": "Mermaid diagram must start with 'flowchart' or 'graph'.",
|
||||
}
|
||||
|
||||
# Basic bracket matching check
|
||||
open_brackets = diagram_content.count("[") + diagram_content.count("(") + diagram_content.count("{")
|
||||
close_brackets = diagram_content.count("]") + diagram_content.count(")") + diagram_content.count("}")
|
||||
if open_brackets != close_brackets:
|
||||
return {
|
||||
"valid": False,
|
||||
"error": "Mismatched brackets in Mermaid diagram syntax.",
|
||||
}
|
||||
|
||||
return {"valid": True, "message": "Mermaid diagram syntax is valid."}
|
||||
|
||||
|
||||
@tool
|
||||
def validate_terraform_syntax(terraform_content: str) -> Dict[str, Any]:
|
||||
"""Validates Terraform HCL basic structure and required Google Cloud blocks.
|
||||
|
||||
Args:
|
||||
terraform_content: Raw HCL text content of main.tf.
|
||||
|
||||
Returns:
|
||||
Dict with valid boolean and diagnostic information.
|
||||
"""
|
||||
if not terraform_content or not terraform_content.strip():
|
||||
return {"valid": False, "error": "Terraform configuration is empty."}
|
||||
|
||||
if "resource" not in terraform_content and "module" not in terraform_content:
|
||||
return {
|
||||
"valid": False,
|
||||
"error": "Terraform configuration must contain at least one resource or module block.",
|
||||
}
|
||||
|
||||
return {"valid": True, "message": "Terraform HCL structural check passed."}
|
||||
|
||||
|
||||
@tool
|
||||
def validate_repository_artifacts(target_dir: str) -> Dict[str, Any]:
|
||||
"""Executes full offline validation against solution architecture artifacts in a directory.
|
||||
|
||||
Args:
|
||||
target_dir: Directory path containing the solution architecture repository.
|
||||
|
||||
Returns:
|
||||
Dict containing validation pass status and missing items list.
|
||||
"""
|
||||
root = Path(target_dir)
|
||||
|
||||
def find_artifact(filename):
|
||||
search_dirs = [
|
||||
root / "deliverables" / "as-is",
|
||||
root / "deliverables" / "target",
|
||||
root / "deliverables" / "validation",
|
||||
root / "deliverables" / "guides",
|
||||
root / "deliverables",
|
||||
root / "docs",
|
||||
root,
|
||||
]
|
||||
for d in search_dirs:
|
||||
candidate = d / filename
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
required_names = ["requirements.md", "architecture.md", "architecture.mmd", "solution-architecture-guide.md"]
|
||||
missing = []
|
||||
|
||||
for fname in required_names:
|
||||
if not find_artifact(fname):
|
||||
missing.append(fname)
|
||||
|
||||
if not (root / "terraform/main.tf").is_file():
|
||||
missing.append("terraform/main.tf")
|
||||
if not (root / "terraform/variables.tf").is_file():
|
||||
missing.append("terraform/variables.tf")
|
||||
|
||||
guide_path = find_artifact("solution-architecture-guide.md")
|
||||
if guide_path:
|
||||
guide_text = guide_path.read_text(encoding="utf-8")
|
||||
required_sections = ["Functional requirements", "Selected products", "Validation results", "Terraform", "Mermaid"]
|
||||
for section in required_sections:
|
||||
if section not in guide_text:
|
||||
missing.append(f"Guide section missing: {section}")
|
||||
else:
|
||||
missing.append("solution-architecture-guide.md missing")
|
||||
|
||||
mmd_path = find_artifact("architecture.mmd")
|
||||
if mmd_path:
|
||||
mmd_text = mmd_path.read_text(encoding="utf-8")
|
||||
if not re.search(r"flowchart|graph", mmd_text):
|
||||
missing.append("Invalid Mermaid graph directive in architecture.mmd")
|
||||
|
||||
passed = len(missing) == 0
|
||||
return {
|
||||
"valid": passed,
|
||||
"missing_items": missing,
|
||||
"message": "All required architecture artifacts valid." if passed else f"Validation failed for: {', '.join(missing)}",
|
||||
}
|
||||
@@ -1,5 +1 @@
|
||||
"""Workflows package."""
|
||||
|
||||
from .gcp_architecture_graph import create_agent, create_gcp_architecture_graph
|
||||
|
||||
__all__ = ["create_agent", "create_gcp_architecture_graph"]
|
||||
"""The Execution graph and the dependency graph of Deliverables it runs."""
|
||||
|
||||
56
app/workflows/execution_graph.py
Normal file
56
app/workflows/execution_graph.py
Normal file
@@ -0,0 +1,56 @@
|
||||
"""The LangGraph that holds an Execution (ADR-0003).
|
||||
|
||||
START -> orchestrator_turn
|
||||
orchestrator_turn -> specialist_task x N (Send, parallel) -> orchestrator_turn
|
||||
orchestrator_turn -> await_user (interrupt) -> orchestrator_turn
|
||||
orchestrator_turn -> finalize -> END
|
||||
orchestrator_turn -> end_with_warnings -> END
|
||||
|
||||
The nodes live in app/nodes/execution_nodes.py; they route with Command and Send, so
|
||||
only the fixed edges are declared here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from langgraph.graph import END, START, StateGraph
|
||||
|
||||
from app.adk.runtime import AgentRuntime, RoleSpec, ToolFactory
|
||||
from app.nodes.execution_nodes import make_execution_nodes
|
||||
from app.recorder import ExecutionRecorder
|
||||
from app.states.execution_state import ExecutionState
|
||||
|
||||
|
||||
def build_execution_graph(
|
||||
*,
|
||||
runtime: AgentRuntime,
|
||||
roles: Mapping[str, RoleSpec],
|
||||
recorder: ExecutionRecorder,
|
||||
output_root: Path,
|
||||
max_orchestrator_turns: int = 100,
|
||||
tool_factory: ToolFactory | None = None,
|
||||
) -> StateGraph:
|
||||
nodes = make_execution_nodes(
|
||||
runtime=runtime,
|
||||
roles=roles,
|
||||
recorder=recorder,
|
||||
output_root=output_root,
|
||||
max_orchestrator_turns=max_orchestrator_turns,
|
||||
tool_factory=tool_factory,
|
||||
)
|
||||
|
||||
graph = StateGraph(ExecutionState)
|
||||
graph.add_node("orchestrator_turn", cast(Any, nodes.orchestrator_turn))
|
||||
# Specialist tasks receive their Send payload, not the Execution state.
|
||||
graph.add_node("specialist_task", cast(Any, nodes.specialist_task))
|
||||
graph.add_node("await_user", cast(Any, nodes.await_user))
|
||||
graph.add_node("finalize", cast(Any, nodes.finalize))
|
||||
graph.add_node("end_with_warnings", cast(Any, nodes.end_with_warnings))
|
||||
graph.add_edge(START, "orchestrator_turn")
|
||||
graph.add_edge("specialist_task", "orchestrator_turn")
|
||||
graph.add_edge("finalize", END)
|
||||
graph.add_edge("end_with_warnings", END)
|
||||
return graph
|
||||
@@ -1,61 +0,0 @@
|
||||
"""Dynamic LangGraph StateGraph Factory with Skill Injection."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Callable
|
||||
from langgraph.graph import END, START, StateGraph
|
||||
from langgraph.graph.state import CompiledStateGraph
|
||||
|
||||
from app.config import get_settings
|
||||
from app.nodes import design_node, discover_node, package_node, source_discover_node, validate_node
|
||||
from app.skills.loader import SkillLoader
|
||||
from app.states.state import GCPArchitectureState
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def create_gcp_architecture_graph(skill_loader: SkillLoader | None = None) -> CompiledStateGraph:
|
||||
"""Creates and compiles the LangGraph StateGraph with dynamic local skill injection."""
|
||||
if skill_loader is None:
|
||||
settings = get_settings()
|
||||
skill_loader = SkillLoader(settings.SKILLS_DIR)
|
||||
skill_loader.load_skills()
|
||||
|
||||
builder = StateGraph(GCPArchitectureState)
|
||||
|
||||
# Wrap nodes with skill loader injection
|
||||
def run_source_discover(state: GCPArchitectureState) -> dict[str, Any]:
|
||||
return source_discover_node(state, skill_loader)
|
||||
|
||||
def run_discover(state: GCPArchitectureState) -> dict[str, Any]:
|
||||
return discover_node(state, skill_loader)
|
||||
|
||||
def run_design(state: GCPArchitectureState) -> dict[str, Any]:
|
||||
return design_node(state, skill_loader)
|
||||
|
||||
def run_validate(state: GCPArchitectureState) -> dict[str, Any]:
|
||||
return validate_node(state, skill_loader)
|
||||
|
||||
def run_package(state: GCPArchitectureState) -> dict[str, Any]:
|
||||
return package_node(state, skill_loader)
|
||||
|
||||
# Add graph nodes
|
||||
builder.add_node("source_discover", run_source_discover)
|
||||
builder.add_node("discover", run_discover)
|
||||
builder.add_node("design", run_design)
|
||||
builder.add_node("validate", run_validate)
|
||||
builder.add_node("package", run_package)
|
||||
|
||||
# Add sequential workflow edges
|
||||
builder.add_edge(START, "source_discover")
|
||||
builder.add_edge("source_discover", "discover")
|
||||
builder.add_edge("discover", "design")
|
||||
builder.add_edge("design", "validate")
|
||||
builder.add_edge("validate", "package")
|
||||
builder.add_edge("package", END)
|
||||
|
||||
return builder.compile()
|
||||
|
||||
|
||||
def create_agent(skill_loader: SkillLoader | None = None) -> CompiledStateGraph:
|
||||
"""Alias for create_gcp_architecture_graph to support standardized agent creation."""
|
||||
return create_gcp_architecture_graph(skill_loader)
|
||||
@@ -1,89 +0,0 @@
|
||||
"""Starlette REST Route Handlers for GCP Solution Architecture Agent.
|
||||
|
||||
Powered by Google ADK runner, PostgreSQL database persistence, and multi-agent orchestration.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
from app.adk.evaluation import ADKEvaluator
|
||||
from app.adk.runners import ADKAgentRunner
|
||||
from app.adk.sessions import PostgresSessionService
|
||||
from app.card import AGENT_CARD
|
||||
from app.config import get_settings
|
||||
from app.tools.validation_tools import validate_repository_artifacts
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def health_check(request: Request) -> JSONResponse:
|
||||
"""GET /health - Operational health endpoint."""
|
||||
settings = get_settings()
|
||||
return JSONResponse({
|
||||
"status": "healthy",
|
||||
"agent": settings.AGENT_NAME,
|
||||
"version": settings.AGENT_VERSION,
|
||||
"adk_framework": "enabled",
|
||||
"database": "postgresql",
|
||||
})
|
||||
|
||||
|
||||
async def get_card(request: Request) -> JSONResponse:
|
||||
"""GET /card - Agent metadata and capability card."""
|
||||
return JSONResponse(AGENT_CARD)
|
||||
|
||||
|
||||
async def run_workflow(request: Request) -> JSONResponse:
|
||||
"""POST /generate - Execute multi-agent ADK workflow with PostgreSQL persistence."""
|
||||
try:
|
||||
body = await request.json() if request.headers.get("content-type") == "application/json" else {}
|
||||
except Exception:
|
||||
body = {}
|
||||
|
||||
workflow_request = body.get("request", "Default event-driven HTTP architecture")
|
||||
target_dir = body.get("target_dir", ".")
|
||||
session_id = body.get("session_id")
|
||||
|
||||
runner = ADKAgentRunner()
|
||||
result = runner.run_execution(
|
||||
session_id=session_id,
|
||||
request_summary=workflow_request,
|
||||
target_dir=target_dir,
|
||||
)
|
||||
|
||||
return JSONResponse(result)
|
||||
|
||||
|
||||
async def get_session_route(request: Request) -> JSONResponse:
|
||||
"""GET /sessions/{session_id} - Query session state from PostgreSQL database."""
|
||||
session_id = request.path_params.get("session_id")
|
||||
session_service = PostgresSessionService()
|
||||
session = session_service.get_session(session_id)
|
||||
|
||||
if not session:
|
||||
return JSONResponse({"error": f"Session {session_id} not found"}, status_code=404)
|
||||
|
||||
return JSONResponse({
|
||||
"session_id": session.session_id,
|
||||
"agent_name": session.agent_name,
|
||||
"state": session.state,
|
||||
})
|
||||
|
||||
|
||||
async def validate_artifacts_route(request: Request) -> JSONResponse:
|
||||
"""POST /validate - Run pre-deployment validation checks against target directory."""
|
||||
try:
|
||||
body = await request.json() if request.headers.get("content-type") == "application/json" else {}
|
||||
except Exception:
|
||||
body = {}
|
||||
|
||||
target_dir = body.get("target_dir", ".")
|
||||
result = validate_repository_artifacts.invoke({"target_dir": target_dir})
|
||||
|
||||
status_code = 200 if result.get("valid") else 400
|
||||
return JSONResponse(result, status_code=status_code)
|
||||
|
||||
status_code = 200 if result.get("valid") else 400
|
||||
return JSONResponse(result, status_code=status_code)
|
||||
@@ -0,0 +1,15 @@
|
||||
# Pre-emptive Live GCP Environment Discovery (Project: `gcp-solution-architecture-demo`)
|
||||
|
||||
## Live Resource Audit
|
||||
- **Target Google Cloud Project**: `gcp-solution-architecture-demo`
|
||||
- **Discovered Storage Buckets**: None / Default Bucket
|
||||
- **Discovered Compute Instances**: None active
|
||||
- **Discovered Database Instances**: None active
|
||||
- **Discovered Pub/Sub Topics**: None active
|
||||
- **Discovered Cloud Run Services**: None active
|
||||
|
||||
## Resource Inventory Breakdown
|
||||
## Current Operational Bottlenecks & Migration Drivers
|
||||
- As-is infrastructure requires serverless auto-scaling and managed high availability.
|
||||
- Need for declarative IaC management via Terraform.
|
||||
- Transition to least-privilege IAM service identities and automated CI validation.
|
||||
@@ -0,0 +1,5 @@
|
||||
flowchart TD
|
||||
subgraph GCPProject["Google Cloud Project: gcp-solution-architecture-demo"]
|
||||
EmptyProject["Project gcp-solution-architecture-demo (No Active Resources Detected)"]
|
||||
end
|
||||
Client[External Traffic] --> GCPProject
|
||||
@@ -1,27 +1,4 @@
|
||||
"""Phase 3: Solution Guide Packaging Node."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict
|
||||
from app.states.state import GCPArchitectureState
|
||||
from app.skills.loader import SkillLoader
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def package_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dict[str, Any]:
|
||||
"""Processes Phase 3 Packaging: assemble final solution-architecture-guide.md document."""
|
||||
logger.info("Executing package_node")
|
||||
skill_prompt = skill_loader.format_skills_for_prompt("package")
|
||||
|
||||
src_doc = state.get("source_discovery_doc", "")
|
||||
src_mmd = state.get("source_mermaid_diagram", "")
|
||||
req_doc = state.get("requirements_doc", "")
|
||||
arch_doc = state.get("architecture_doc", "")
|
||||
mmd_doc = state.get("mermaid_diagram", "")
|
||||
tf_doc = state.get("terraform_code", "")
|
||||
val_doc = state.get("validation_results", "")
|
||||
|
||||
solution_guide = f"""# Google Cloud solution architecture: Event-Driven Regional Workload
|
||||
# Google Cloud solution architecture: Event-Driven Regional Workload
|
||||
|
||||
## 1. Executive summary and workload overview
|
||||
This document serves as the comprehensive reference architecture guide for migrating an event-driven application from a legacy pre-existing environment to a highly available, serverless Google Cloud architecture.
|
||||
@@ -44,10 +21,28 @@ See [`docs/requirements.md`](docs/requirements.md). Requirements include Functio
|
||||
- **Sustainability**: Efficient resource utilization via auto-scaling serverless runtimes.
|
||||
|
||||
### 2.3. Current state (As-Is Architecture)
|
||||
{src_doc.strip()}
|
||||
# Pre-emptive Live GCP Environment Discovery (Project: `gcp-solution-architecture-demo`)
|
||||
|
||||
## Live Resource Audit
|
||||
- **Target Google Cloud Project**: `gcp-solution-architecture-demo`
|
||||
- **Discovered Storage Buckets**: None / Default Bucket
|
||||
- **Discovered Compute Instances**: None active
|
||||
- **Discovered Database Instances**: None active
|
||||
- **Discovered Pub/Sub Topics**: None active
|
||||
- **Discovered Cloud Run Services**: None active
|
||||
|
||||
## Resource Inventory Breakdown
|
||||
## Current Operational Bottlenecks & Migration Drivers
|
||||
- As-is infrastructure requires serverless auto-scaling and managed high availability.
|
||||
- Need for declarative IaC management via Terraform.
|
||||
- Transition to least-privilege IAM service identities and automated CI validation.
|
||||
|
||||
```mermaid
|
||||
{(src_mmd or "flowchart TD\n Client --> LegacyApp").strip()}
|
||||
flowchart TD
|
||||
subgraph GCPProject["Google Cloud Project: gcp-solution-architecture-demo"]
|
||||
EmptyProject["Project gcp-solution-architecture-demo (No Active Resources Detected)"]
|
||||
end
|
||||
Client[External Traffic] --> GCPProject
|
||||
```
|
||||
|
||||
### 2.4. Dependencies
|
||||
@@ -70,7 +65,12 @@ See [`docs/requirements.md`](docs/requirements.md). Requirements include Functio
|
||||
|
||||
### 4.2. Architecture diagram (Mermaid)
|
||||
```mermaid
|
||||
{mmd_doc.strip()}
|
||||
flowchart TD
|
||||
Client[External HTTPS Client] -->|HTTPS POST /events| CloudRun[Google Cloud Run Service]
|
||||
CloudRun -->|Publish Event| PubSubTopic[Cloud Pub/Sub Topic]
|
||||
CloudRun -->|Write Raw Payload| GCSAudit[Cloud Storage Audit Bucket]
|
||||
PubSubTopic -->|Push Delivery| EventConsumer[Cloud Run Consumer Service]
|
||||
EventConsumer -->|Acknowledge| PubSubTopic
|
||||
```
|
||||
|
||||
### 4.3. Architecture description
|
||||
@@ -109,7 +109,78 @@ See [`docs/requirements.md`](docs/requirements.md). Requirements include Functio
|
||||
|
||||
### 6.2. Step-by-step deployment instructions (Terraform)
|
||||
```hcl
|
||||
{tf_doc.strip()}
|
||||
# Google Cloud Solution Architecture Baseline
|
||||
terraform {
|
||||
required_version = ">= 1.5.0"
|
||||
required_providers {
|
||||
google = {
|
||||
source = "hashicorp/google"
|
||||
version = "~> 5.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "google" {
|
||||
project = var.project_id
|
||||
region = var.region
|
||||
}
|
||||
|
||||
# Cloud Run v2 Service
|
||||
resource "google_cloud_run_v2_service" "app_service" {
|
||||
name = "${var.environment}-app-service"
|
||||
location = var.region
|
||||
|
||||
template {
|
||||
containers {
|
||||
image = var.container_image
|
||||
ports {
|
||||
container_port = 8080
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Pub/Sub Topic for Event Ingestion
|
||||
resource "google_pubsub_topic" "event_ingestion" {
|
||||
name = "${var.environment}-event-ingestion-topic"
|
||||
labels = {
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
}
|
||||
}
|
||||
|
||||
# Cloud Storage Bucket for Event Replay Audit
|
||||
resource "google_storage_bucket" "audit_bucket" {
|
||||
name = "${var.project_id}-${var.environment}-audit-bucket"
|
||||
location = var.region
|
||||
force_destroy = false
|
||||
uniform_bucket_level_access = true
|
||||
|
||||
versioning {
|
||||
enabled = true
|
||||
}
|
||||
|
||||
lifecycle_rule {
|
||||
condition {
|
||||
age = 30
|
||||
}
|
||||
action {
|
||||
type = "Delete"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Least-Privilege IAM Service Account
|
||||
resource "google_service_account" "ingress_sa" {
|
||||
account_id = "${var.environment}-ingress-sa"
|
||||
display_name = "Cloud Run Ingress Identity"
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic_iam_member" "publisher_binding" {
|
||||
topic = google_pubsub_topic.event_ingestion.name
|
||||
role = "roles/pubsub.publisher"
|
||||
member = "serviceAccount:${google_service_account.ingress_sa.email}"
|
||||
}
|
||||
```
|
||||
|
||||
Apply blueprint instructions:
|
||||
@@ -120,7 +191,20 @@ terraform -chdir=terraform apply
|
||||
```
|
||||
|
||||
## 7. Validation plan (Validation results)
|
||||
{val_doc.strip()}
|
||||
# Validation Results
|
||||
|
||||
## Summary
|
||||
- **Overall Validation Status**: PASS
|
||||
- **Mermaid Diagram Syntax**: PASS
|
||||
- **Terraform Structural Check**: PASS
|
||||
- **Resource Provisioning Triggered**: False (Static non-deployment check enforced)
|
||||
|
||||
## Verification Rules Checklist
|
||||
- [x] Functional & Non-functional requirements specified
|
||||
- [x] Product selection deferred during discovery and resolved in design phase
|
||||
- [x] Regional High Availability and Security IAM boundaries configured
|
||||
- [x] Mermaid diagram follows valid graph syntax
|
||||
- [x] Terraform HCL declares provider, resources, and least-privilege IAM bindings
|
||||
|
||||
### Verification Checklist
|
||||
- Step 4 guide persistence: non-empty solution-architecture-guide.md.
|
||||
@@ -131,21 +215,3 @@ terraform -chdir=terraform apply
|
||||
- [Google Cloud Architecture Framework](https://cloud.google.com/architecture/framework)
|
||||
- [Cloud Run Overview](https://cloud.google.com/run/docs/overview/what-is-cloud-run)
|
||||
- [Cloud Pub/Sub Overview](https://cloud.google.com/pubsub/docs/overview)
|
||||
"""
|
||||
|
||||
active_skills = state.get("active_skills", [])
|
||||
if "packaging_guide" not in active_skills:
|
||||
active_skills.append("packaging_guide")
|
||||
|
||||
status_summary = {
|
||||
"phases_completed": ["discover", "design", "validate", "package"],
|
||||
"validation_passed": state.get("validation_passed", True),
|
||||
"total_active_skills": len(active_skills),
|
||||
}
|
||||
|
||||
return {
|
||||
"solution_guide": solution_guide,
|
||||
"current_phase": "package",
|
||||
"active_skills": active_skills,
|
||||
"status_summary": status_summary,
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
# Phase 1 — Architecture & Product Selection
|
||||
|
||||
## Selected Products
|
||||
- **Compute / Serving**: Google Cloud Run (Fully Managed Container Ingress & Stateless Execution)
|
||||
- **Messaging & Eventing**: Google Cloud Pub/Sub (Regional Event Bus for Asynchronous Decoupling)
|
||||
- **State & Storage**: Google Cloud Storage & Firestore (Database & Bucket Storage for Durable Audit Event Replay)
|
||||
- **Security & Identity**: Cloud IAM (Least Privilege Service Accounts) & KMS (Customer-Managed Encryption Keys)
|
||||
- **Artifact Registry**: Google Artifact Registry (OCI Container Image Hosting)
|
||||
|
||||
## Component Responsibilities
|
||||
1. **Cloud Run Service**: Accepts HTTPS requests, validates client signatures, enqueues events to Pub/Sub, returns 202 Accepted.
|
||||
2. **Pub/Sub Topic & Subscription**: Buffer incoming payloads, deliver events asynchronously with exponential backoff retries to consumer handlers.
|
||||
3. **Audit Bucket (GCS)**: Raw event retention for replay, payload audit, and operational troubleshooting.
|
||||
|
||||
## Security & Compliance
|
||||
- HTTPS ingress with TLS 1.3 encryption in transit.
|
||||
- Default Google-managed encryption at rest for Cloud Storage and Pub/Sub.
|
||||
- Cloud Run service account bound strictly to `roles/pubsub.publisher` and `roles/storage.objectCreator`.
|
||||
|
||||
## Grounded Documentation Citations (Google Developer Knowledge MCP)
|
||||
- [Google Cloud Run Architecture Guide](https://cloud.google.com/run/docs/overview/what-is-cloud-run)
|
||||
@@ -0,0 +1,6 @@
|
||||
flowchart TD
|
||||
Client[External HTTPS Client] -->|HTTPS POST /events| CloudRun[Google Cloud Run Service]
|
||||
CloudRun -->|Publish Event| PubSubTopic[Cloud Pub/Sub Topic]
|
||||
CloudRun -->|Write Raw Payload| GCSAudit[Cloud Storage Audit Bucket]
|
||||
PubSubTopic -->|Push Delivery| EventConsumer[Cloud Run Consumer Service]
|
||||
EventConsumer -->|Acknowledge| PubSubTopic
|
||||
@@ -1,25 +1,7 @@
|
||||
"""Phase 0: Requirements Discovery Node."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict
|
||||
from app.states.state import GCPArchitectureState
|
||||
from app.skills.loader import SkillLoader
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def discover_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dict[str, Any]:
|
||||
"""Processes Phase 0 Discovery: document functional/non-functional requirements with product selection deferred."""
|
||||
logger.info("Executing discover_node")
|
||||
skill_prompt = skill_loader.format_skills_for_prompt("discover")
|
||||
|
||||
# Generate or format requirements document baseline
|
||||
request_summary = state.get("workflow_request", "Event-driven HTTP application reference architecture")
|
||||
|
||||
requirements_doc = f"""# Step 0 — Requirements discovery
|
||||
# Step 0 — Requirements discovery
|
||||
|
||||
## Workflow request
|
||||
{request_summary}
|
||||
Event-driven regional HTTP application
|
||||
|
||||
## Functional requirements
|
||||
- Accept authenticated HTTPS requests from external clients.
|
||||
@@ -60,15 +42,3 @@ def discover_node(state: GCPArchitectureState, skill_loader: SkillLoader) -> Dic
|
||||
- Which data is relational, document, object, or analytical?
|
||||
|
||||
**Product selection deferred:** `true` for this phase.
|
||||
"""
|
||||
|
||||
active_skills = state.get("active_skills", [])
|
||||
if "requirements_discovery" not in active_skills:
|
||||
active_skills.append("requirements_discovery")
|
||||
|
||||
return {
|
||||
"requirements_doc": requirements_doc,
|
||||
"product_selection_deferred": True,
|
||||
"current_phase": "discover",
|
||||
"active_skills": active_skills,
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
# Validation Results
|
||||
|
||||
## Summary
|
||||
- **Overall Validation Status**: PASS
|
||||
- **Mermaid Diagram Syntax**: PASS
|
||||
- **Terraform Structural Check**: PASS
|
||||
- **Resource Provisioning Triggered**: False (Static non-deployment check enforced)
|
||||
|
||||
## Verification Rules Checklist
|
||||
- [x] Functional & Non-functional requirements specified
|
||||
- [x] Product selection deferred during discovery and resolved in design phase
|
||||
- [x] Regional High Availability and Security IAM boundaries configured
|
||||
- [x] Mermaid diagram follows valid graph syntax
|
||||
- [x] Terraform HCL declares provider, resources, and least-privilege IAM bindings
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user