latest refactoring
Some checks failed
validation / verify (push) Failing after 10s

This commit is contained in:
2026-10-06 20:22:37 +01:00
parent e94b2493e7
commit 7ee49ab352
414 changed files with 15768 additions and 3896 deletions

View File

@@ -0,0 +1,55 @@
# Plan Detail
## Step 1: Container starts the Execution engine
- **Capability:** The image runs the reviewed and evaluated engine.
- **Rationale:** The `Dockerfile` started `python -m app.agent`, the iteration 1 pipeline, while every evaluation ran the new engine.
- **Where:** `Dockerfile`, `README.md`.
- **Acceptance criteria:** The container's main process is the A2A server. `/health` and `/.well-known/agent-card.json` answer on the container port. Existing `-p 8080:8080` run commands still work.
## Step 2: Legacy pipeline moved to `app/legacy/`
- **Capability:** The KAB folder names are free for the new engine, with the old pipeline still importable and tested.
- **Rationale:** The owner chose to keep the old code rather than delete it (2026-10-06). The old and new code cannot share `app/nodes/`, `app/states/`, `app/workflows/`.
- **Where:** `app/legacy/`, the five legacy tests in `tests/`, `eval/eval_harness.py`, `SKILLS_DIR` in `app/config.py`.
- **Acceptance criteria:** Every legacy test passes against `app.legacy.*`. Nothing outside `app/legacy/`, those tests and the legacy eval files imports `app.legacy`.
## Step 3: Engine moved into the KAB folders
- **Capability:** The engine's modules sit where the KAB workflow standard expects them.
- **Rationale:** Platform consistency: decomposers, reviewers and other agents on the platform look for state in `app/states/`, nodes in `app/nodes/`, graphs in `app/workflows/`, the manager in `app/agent.py`.
- **Where:** The path table in `../README.md`.
- **Acceptance criteria:** Behaviour is unchanged: the full suite passes with only import paths edited in tests. `google.adk` is still imported only by `app/adk/adk_runtime.py`. Package `__init__.py` files import nothing, so `app.states` can import `app.workflows.deliverables` without a cycle.
## Step 4: Graph file split into nodes and wiring
- **Capability:** Node logic in `app/nodes/`, graph assembly in `app/workflows/`.
- **Rationale:** The KAB layout separates stage nodes from graph construction; the engine had both in one 560-line file.
- **Where:** `app/nodes/execution_nodes.py` (`make_execution_nodes` returns an `ExecutionNodes` of the five node callables), `app/workflows/execution_graph.py` (`build_execution_graph`), `ToolFactory` in `app/adk/runtime.py`.
- **Acceptance criteria:** `build_execution_graph` keeps its signature. The nodes still route with `Command` and `Send`, so the wiring declares only the fixed edges.
## Step 5: Refresh paths in the spec and ADR text
- **Capability:** Design documents that name modules where they are.
- **Rationale:** `.scratch/solution-architect-agent/spec.md` (module layout, acceptance list, comments) and the ADRs still use pre-move paths. An agent following them will look in `app/legacy/` or in folders that no longer exist.
- **Where:** `.scratch/solution-architect-agent/spec.md` sections "Module layout" and "Acceptance"; `docs/adr/0002-*.md` consequence about `app/skills/`; `docs/adr/0003-*.md`.
- **Acceptance criteria:** The spec's module layout matches the tree. Each acceptance command in the spec runs against current paths. Dated `## Comments` entries stay as written, since they are history; one note at the top of the comments points to `../README.md` for translation.
## Step 6: Finish iteration 2's cleanup list
- **Capability:** The parts of iteration 2 step 9 that still apply once the legacy code is kept.
- **Rationale:** The spec's cleanup list mixed deletions (now dropped) with changes the new engine still needs.
- **Where and what:**
- `workflow.yaml`: rewrite its phases to the four upstream Phases and their Deliverables.
- `app/config.py`, `.env.example`: move settings only `app/legacy/` reads (`HOST`, `PORT`, `GCP_PROJECT_ID`, `GCP_REGION`, `ADK_*`, `DEVELOPER_KNOWLEDGE_MCP_*`, `SKILLS_DIR`, `EVAL_DATASET_PATH`, the OpenAI and Azure keys) under a clearly marked legacy block. Keep `POSTGRES_*` and `DATABASE_URL`: the Postgres checkpointer builds its URL from them.
- `mypy.ini`, `ruff.toml`: exclude `app/legacy/` and the legacy eval files, so `ruff check .` and `mypy .` report on live code only (198 and 38 findings today, almost all legacy).
- `Dockerfile`: stop copying what only the legacy pipeline reads, once step 7 is decided.
- **Acceptance criteria:** `ruff check .` and `mypy .` are clean or list only known, recorded exceptions. `workflow.yaml` names the four Phases in `app/workflows/deliverables.py`. The suite passes.
## Step 7: Decide the future of `app/legacy/`
- **Capability:** One code path, or a deliberate, documented second one.
- **Rationale:** Two pipelines cost every reader a detour, and the legacy one holds what ADR-0002 rules out: hand-written cloud clients, non-conformant Skills, a pretend ADK layer.
- **Options for the owner:** delete `app/legacy/` with its tests and the legacy eval files; or keep it frozen and excluded from lint, types and the image.
- **Guardrail:** Deleting files with uncommitted changes was blocked once by the permission system. Commit first, then delete in a commit of its own.
- **Acceptance criteria:** The decision is recorded here. If deleted: `grep -rn "app\.legacy" app eval tests` is empty and the suite passes.

View File

@@ -0,0 +1,34 @@
# Plan Map
**Iteration:** 3, KAB layout
**Intent:** Make the deployed container run the hierarchical Execution engine, and house that engine in the KAB workflow-agent folder layout (`app/agent.py`, `app/main.py`, `app/a2a_server.py`, `app/states/`, `app/nodes/`, `app/workflows/`), with the iteration 1 pipeline first parked under `app/legacy/` (steps 2 to 6) and then deleted (step 7).
**Branch:** `feat/solution-architect-agent` (all of this iteration is uncommitted as of 2026-10-06)
**This directory covers:** steps 1 to 7
## Sources of truth
- `../README.md`: the old-to-new module path table.
- `../docs/KAB_WORKFLOW_AGENT_IMPLEMENTATION_GUIDE.md` and `../langgraph-workflow-agent/app/`, both relative to the repository root: the layout being matched.
- `docs/adr/0003-langgraph-holds-state-adk-agents-reason.md`: the engine the layout must keep unchanged.
## Status
| Step | What | Status |
|---|---|---|
| 1 | Container starts the Execution engine | complete |
| 2 | Legacy pipeline moved to `app/legacy/` | complete |
| 3 | Engine moved into the KAB folders | complete |
| 4 | Graph file split into nodes and wiring | complete |
| 5 | Refresh paths in the spec and ADR text | complete |
| 6 | Finish iteration 2's cleanup list | complete; two items carried forward in `steps/step-6.md` |
| 7 | Decide the future of `app/legacy/` | complete: deleted on the owner's instruction |
## Dependency graph
- Step 1: no dependencies
- Step 2: no dependencies
- Step 3: depends on step 2
- Step 4: depends on step 3
- Step 5: depends on step 4
- Step 6: depends on step 4
- Step 7: depends on step 6

View File

@@ -0,0 +1,16 @@
# Step 1 — Implementation Detail
1. `Dockerfile`: the start command became the A2A server; the image sets `A2A_HOST=0.0.0.0` and `A2A_PORT=8080`; only port 8080 is exposed.
2. The health check reads `A2A_PORT` and allows 60 seconds to start, because `/health` answers only after the provider bundle and MCP catalogs load.
3. `README.md`: server section, start command, container example (now passing `GOOGLE_GENAI_USE_VERTEXAI`, `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION`, `A2A_PUBLIC_URL`) and test calls.
Verified: built with podman and ran the image with `MCP_ENABLED=false`; `/health` and the agent card returned 200 and process 1 was the server.
Consequences for callers:
- `/card`, `/validate` and `/sessions/{session_id}` are no longer served. The card is at `/.well-known/agent-card.json`.
- `/generate` now runs a full pre-approved Execution with real models, about 45 minutes in the evaluation runs.
- Without `A2A_PUBLIC_URL` the card advertises `http://0.0.0.0:8080/a2a`.
- Podman's default image format ignores `HEALTHCHECK`.
Not verified: a full Execution with real models and MCP servers inside the container.

View File

@@ -0,0 +1,9 @@
# Step 2 — Implementation Detail
1. Moved `app/agent.py`, `app/card.py`, `app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/`, `app/states/` to `app/legacy/`, and `gcp_scanner.py`, `mcp_developer_knowledge.py`, `validation_tools.py` to `app/legacy/tools/`.
2. Rewrote imports to `app.legacy.*` in the legacy code, its five tests (`test_adk_architecture`, `test_agent_api`, `test_graph_workflow`, `test_mcp_developer_knowledge`, `test_skill_loader`) and `eval/eval_harness.py`.
3. Pointed `SKILLS_DIR` at `app/legacy/skills`, and the paths quoted in `eval/optimizer.py` likewise.
Verified: 226 passed before any engine file moved; `python -m eval.eval_harness` still runs.
Nothing was deleted. A first attempt to delete these files was blocked by the permission system because 25 of them held uncommitted changes.

View File

@@ -0,0 +1,10 @@
# Step 3 — Implementation Detail
1. Moved each engine module to the path in `../../README.md`. `app/execution/`, `app/agents/` and `app/a2a/` no longer exist.
2. Merged `engine.py` and `assembly.py` into `app/agent.py`: `ExecutionEngine`, `Interaction`, then `Assembly` and `open_assembly`.
3. The entry point is `python -m app.main`; `Dockerfile` and `README.md` use it.
4. Left the test folders (`tests/execution/`, `tests/agents/`, `tests/a2a/`) where they were; only their imports changed.
Verified: 226 passed; the rebuilt image starts `python -m app.main` and answers `/health` and the agent card.
Side effect: ruff's import sort and unused-import fix ran across `app`, `eval`, `tests` and `scripts`, so legacy and test files show import-only diffs.

View File

@@ -0,0 +1,9 @@
# Step 4 — Implementation Detail
1. `app/nodes/execution_nodes.py` keeps the five nodes as closures over the runtime, roles, recorder and tool factory, built by `make_execution_nodes` and returned as a frozen `ExecutionNodes`. The helper functions and `GUIDE_FILENAME` moved with them.
2. `app/workflows/execution_graph.py` holds `build_execution_graph`, which builds the nodes and declares the fixed edges.
3. `ToolFactory` moved to `app/adk/runtime.py`, beside `ToolSpec`, so `app/tools/toolkit.py` no longer imports the nodes module.
The node bodies were moved verbatim; no logic changed.
Verified: 226 passed; the moved files are ruff-clean.

View File

@@ -0,0 +1,11 @@
# Step 5 — Implementation Detail
1. `.scratch/solution-architect-agent/spec.md`: replaced the "Module layout" tree with the current one, and updated every module path in the design sections (Deliverables, Ledger, Execution graph, tools, agent runtime, A2A, recording) and in the Risks section.
2. Rewrote two acceptance commands to match the tree:
- `rg -n "google\.adk" app -g '!app/legacy/**' | grep -v adk_runtime.py` is empty.
- `rg -n "app\.legacy" app eval tests -g '!**/legacy/**'` is empty.
3. Marked build increment 9 as replaced, with a pointer here; its original list stays below the note.
4. Left the dated `## Comments` entries as written and added one note above them pointing to `../../README.md` for path translation.
5. `docs/adr/0002-*.md`: added a dated note to the `app/skills/` consequence. ADR-0003 names no repository paths and needed no change.
Verified: both acceptance commands return nothing (run with `grep`).

View File

@@ -0,0 +1,19 @@
# Step 6 — Implementation Detail
1. **Legacy grouped in three folders.** The five legacy tests moved to `tests/legacy/` and the offline harness (`metrics.py`, `optimizer.py`, `eval_harness.py`, `test_eval_harness.py`) to `eval/legacy/`. Run it with `python -m eval.legacy.eval_harness`. `eval/datasets/benchmark_cases.json` stayed where `EVAL_DATASET_PATH` points.
2. **Lint and types on live code only.** `ruff.toml` and `mypy.ini` excluded `app/legacy`, `eval/legacy` and `tests/legacy` (exclusions removed again in step 7). `ruff check .` and `mypy .` are now clean, down from 198 and 38 findings.
3. **Live findings fixed, none suppressed by config:**
- `app/database.py`: modern annotations, an unused import removed, and each broad `except` marked with its reason (connection failure falls back to SQLite; persistence is best-effort and logged).
- `app/workflows/execution_graph.py`: every node is cast for `add_node`, since the nodes are now typed callables.
- `scripts/list-mcp-tools.py`, `scripts/validate_artifacts.py`, `tests/test_assembly.py`, `tests/test_package.py`, `tests/agents/test_adk_engine.py`, `tests/agents/test_adk_runtime.py`: small typing and lint fixes.
- `requirements-dev.txt`: added `types-protobuf` and `types-psycopg2`.
4. **Settings.** `app/config.py` and `.env.example` list live settings first and the legacy-only ones under a marked block. No setting was removed or renamed.
5. **`workflow.yaml`** names the four Phases and their Deliverables. `tests/test_workflow_manifest.py` fails if it drifts from `app/workflows/deliverables.py`.
6. **Image.** `Dockerfile` copies `app/`, `agents/`, `providers/`, `skills/` and `workflow.yaml` only, and creates `deliverables/`. A new `.dockerignore` keeps `app/legacy`, `.venv`, `.git`, caches, databases and past results out of the build context.
Verified: 227 passed; ruff and mypy clean; the rebuilt image contains no `app/legacy`, `eval/`, `docs/` or `terraform/`, and `/health` answers.
Still open from iteration 2's cleanup list:
- **Writes to shared paths.** The engine writes only under `deliverables/executions/<execution_id>/`. The legacy pipeline still writes `deliverables/{as-is,target,validation,guides}/` and `docs/*.md`, and `tests/legacy/` runs it, so a test run can modify tracked files. Step 7 deleted the legacy folders, which closed this item.
- **CI.** `.github/workflows/ci.yml` pins `ruff==0.6.9` and `mypy==1.11.2` and installs no project dependencies; the local tools are newer (ruff 0.16.7). CI was not run for this step.

View File

@@ -0,0 +1,15 @@
# Step 7 — Implementation Detail
Decision: **the original pipeline is deleted.** The owner asked for its removal on 2026-10-06, after it had first been kept frozen in `app/legacy/`, `tests/legacy/` and `eval/legacy/` (step 6).
1. Deleted `app/legacy/`, `tests/legacy/`, `eval/legacy/` and `eval/datasets/benchmark_cases.json`.
2. Removed the legacy-only settings from `app/config.py` and `.env.example`: `HOST`, `PORT`, the OpenAI and Azure keys, `LLM_TEMPERATURE`, `GOOGLE_APPLICATION_CREDENTIALS`, `GCP_PROJECT_ID`, `GCP_REGION`, `ADK_*`, `DEVELOPER_KNOWLEDGE_MCP_*`, `SKILLS_DIR`, `EVAL_DATASET_PATH`. Google client libraries still read `GOOGLE_APPLICATION_CREDENTIALS` from the environment themselves.
3. Removed the legacy exclusions from `ruff.toml`, `mypy.ini` and `.dockerignore`.
4. Removed four dependencies no live module imports: `langchain`, `langchain-openai`, `sqlalchemy`, `asyncpg`.
5. Rewrote the top of `README.md` around the Orchestrator, Specialists, Skills and A2A, and pointed `VALIDATION.md` at `eval.solution_eval`.
Verified: 206 passed (227 less the 21 legacy tests); `ruff check .` and `mypy .` clean; `grep -rn "app\.legacy\|eval\.legacy" app eval tests scripts` is empty; the image rebuilt from the trimmed `requirements.txt` imports every live module, including both checkpointers, and answers `/health` and the agent card.
Recovering the old code: the committed versions are in git history at their pre-move paths (`app/adk/`, `app/nodes/`, `app/workflows/`, `app/skills/`, `app/states/`, `app/agent.py`, `app/card.py`) up to commit `e94b249`. Uncommitted formatting changes to those files were not kept.
Left in place, because they are checked-in sample artefacts and not pipeline code: `scripts/validate_artifacts.py`, `tests/test_artifacts.py`, `tests/test_package.py`, the root `terraform/`, `docs/*.md`, `deliverables/{as-is,target,validation,guides}/`, `requirements.yaml`. `app/database.py` keeps two methods only the old pipeline called (`get_session`, `save_evaluation`).