Add hyperscaler discovery MCP servers and Well-Architected guidance
Some checks failed
check agents / check (push) Failing after 12s
Some checks failed
check agents / check (push) Failing after 12s
azure-discovery, aws-discovery and gcp-discovery collect read-only metadata on servers, workloads, Kubernetes clusters, networks, data stores, policies and provider recommendations. Credentials arrive per call from the vault. Replies are paged, scrubbed of secrets and anonymised, and throttled calls back off. cloud-knowledge searches each provider's Well-Architected guidance from fixed lists only, so no customer detail can leave the platform. Adds shared skills, the render and test tooling, a public Python Dockerfile until the hardened base image is reachable, the customer access guide, the implementation plan and the Vault Service platform issue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
339
outcome-pattern/solution_architecture/.agents/PLAN.md
Normal file
339
outcome-pattern/solution_architecture/.agents/PLAN.md
Normal file
@@ -0,0 +1,339 @@
|
||||
# Plan: Solution Architecture agent on the modernization platform
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Date | 2026-10-30, revision 3 |
|
||||
| Status | Draft for review. Nothing is built. |
|
||||
| Source | `kyndryl-agent-builder/gcp_solution_architecture_agent` (read only) |
|
||||
| Platform reviewed | `kyndryl-agent-builder/modernization-platform` @ `c89aedcb9` (2026-10-29), runtime 0.37.0. The other checkout, `platform-engineering/modernization-platform`, is older. |
|
||||
| Target runtime | `agent-runtime-adk:0.35.7`, the version `cp1-agents` pins. |
|
||||
| Boundary | Every new file sits under `outcome-pattern/solution_architecture/`. Changes needed elsewhere are listed in section 8, not made. |
|
||||
|
||||
**Changed in revision 2, at the owner's request:** every model is `claude-sonnet-5-5`; Azure, AWS and Google Cloud are all in the first version; live conversation stays as an option; Jev stays as an option.
|
||||
|
||||
**Changed in revision 3, at the owner's request:** the agent replicates the source with full feature parity. Everything the source does is in v1 (section 5). The engine is still rebuilt on KAB so that it integrates. The only intended differences are the ones the owner chose, such as one model for every role.
|
||||
|
||||
## 1. Summary
|
||||
|
||||
Build one KAB agent that delivers the same outcome as the source: a request goes in, and a reviewed solution architecture comes out, with Deliverables, Target architecture, Terraform, validation, work breakdown and Solution guide. An Execution targets one Cloud provider, chosen from Azure, AWS and Google Cloud.
|
||||
|
||||
Don't lift the source's container onto the platform. Its engine conflicts with the runtime: A2A server, LangGraph checkpointing, `interrupt()` approvals, database and direct model and MCP clients. Keep what makes the source valuable and rebuild the execution layer on platform features.
|
||||
|
||||
| Source | Size | Plan |
|
||||
|---|---|---|
|
||||
| Deterministic core: `scope`, `deliverables`, `target_architecture`, `chain`, `work_breakdown`, `changes`, `ledger` | about 3,000 lines | **Port**, made provider-neutral, with tests |
|
||||
| Router and its playbook (`app/nodes/router.py`) | about 900 lines | **Port.** It's now needed for conversation and for Jev |
|
||||
| Jev and its reviews: `jev`, `meaning`, `skill_routing`, `guardrail`, `assessment`, `design_review`, `schema_review`, `coverage` | about 1,200 lines | **Port** behind the source's `Decider` interface |
|
||||
| Change requests: `baseline`, `customer_case` and the Change route | about 250 lines, plus `changes` above | **Port** |
|
||||
| Lessons loop: `app/learning/*`, `states/lessons`, `eval/curate_lessons.py` | about 1,000 lines | **Port** |
|
||||
| Evaluation harness: `eval/` scoring, judge, calibration | | **Port** as scripts that run against the deployed agent |
|
||||
| 16 role files in `agents/` | | **Port** as sub-agent prompts, made provider-neutral |
|
||||
| Google Skills, including vendored ones | 49 files | **Port** with the licence and checksums |
|
||||
| Azure and AWS Skills | none exist | **New** (section 4.8) |
|
||||
| 8 evaluation cases | | **Port**, then add Azure and AWS cases |
|
||||
| `execution_nodes`, `app/adk/*`, `a2a_server`, `database`, `checkpointing` | about 3,400 lines | **Replace** with the platform |
|
||||
|
||||
## 2. Platform constraints that shape the design
|
||||
|
||||
| Constraint | Where | Consequence |
|
||||
|---|---|---|
|
||||
| The runtime fails any run that pauses with LangGraph `interrupt()`, and keeps no checkpointer | `a2a/server.py` `_refuse_interrupt` | The source's Approvals can't move as they are |
|
||||
| Agents keep nothing between requests | `continuation.py` | Execution state has to live outside the process |
|
||||
| A workflow step is reviewed after it runs: continue, stop, rerun with comments (`hitl_feedback`), or replace the output. Up to 5 reruns per step, and 50 steps per workflow | `generic_workflow_executor.py`, `workflow_template_validator.py` | In a workflow, each Phase is a step and each Approval a review point |
|
||||
| Run chat fills in a workflow's inputs and handles its reviews. Catalog chat answers about the catalog. I found no KAB screen for talking directly to a deployed agent | `services/run_chat/`, `api/routes/catalog_chat.py` | Live conversation needs an A2A client outside those screens (section 4.6) |
|
||||
| Agents reach only the model and MCP gateways, plus hosts in `AGENT_EGRESS_ALLOW` | `egress.py` | Jev needs an explicit allowance |
|
||||
| Sub-agents have a prompt and functions only, with no MCP tools or Skills | Discovery `.agents/CONTEXT.md` | Skills and guidance need a code path (section 4.4) |
|
||||
| An agent with its own `app/agent.py` gets no sub-agent registry or resume | `sub_agents.py`, `continuation.py` | Use a spec agent, not a wrapped graph |
|
||||
| `cloud-knowledge` takes fixed pillars and topics only, and `mcp-servers/README.md` forbids free-text fields | `cloud-knowledge.server.yaml` | Free-text documentation search needs its own server (section 4.4) |
|
||||
| A declared MCP tool the gateway doesn't offer stops the agent at startup | runtime README | Tool names must be confirmed first |
|
||||
| Requests have a wall-clock limit (`AGENT_MAX_DURATION_SECONDS`) | `configmap.yaml` | One full Execution took 22 to 35 minutes, so Phases need separate requests |
|
||||
|
||||
## 3. Approach and alternatives
|
||||
|
||||
**Chosen: an ADK spec agent with two ways in, a workflow and a live conversation, sharing one Phase runner and one workspace.**
|
||||
|
||||
| Alternative | Why not |
|
||||
|---|---|
|
||||
| Keep the source's container and register it by URL | No platform limits, tracing or cost logs, and it breaks egress |
|
||||
| `framework: langgraph` with the graph in `app/agent.py` | Keeps the graph but loses Approvals, which were its main reason to exist (source ADR-0003) |
|
||||
| One agent per Deliverable, chained in a workflow | That's the `individual-agents/` pattern, and it rules out live conversation |
|
||||
| Port `orchestration_engine` and `solution_architecture_agent` | A cleaner split with the same conflicts. Compare module by module when porting (step 2). |
|
||||
|
||||
## 4. Design
|
||||
|
||||
### 4.1 Folder layout
|
||||
|
||||
`solution_architecture/` has the same depth as a Feature folder, so the shared scripts work with it unchanged.
|
||||
|
||||
```
|
||||
outcome-pattern/solution_architecture/
|
||||
├── CLAUDE.md README.md .agents/PLAN.md
|
||||
├── cp1-solution-architecture-agent/
|
||||
│ ├── agent.yaml prompt.md Dockerfile RELEASE_NOTES.md README.md CLAUDE.md
|
||||
│ ├── schemas/{input,output}.json
|
||||
│ ├── prompts/<specialist>.md from agents/*.md
|
||||
│ ├── skills/{common,gcp,azure,aws}/
|
||||
│ ├── providers/{gcp,azure,aws}.yaml
|
||||
│ ├── app/
|
||||
│ │ ├── business_logic.py hooks: bind the run, route the message, build the reply
|
||||
│ │ ├── router.py ported router and playbook
|
||||
│ │ ├── phases.py the Phase runner (replaces execution_nodes)
|
||||
│ │ ├── workspace.py run store
|
||||
│ │ ├── tools.py Specialists' functions
|
||||
│ │ ├── skill_reader.py role-scoped Skill loading
|
||||
│ │ ├── terraform.py fmt, validate and schema lookup
|
||||
│ │ ├── decisions/ Decider, NoDecider, Jev and its reviews
|
||||
│ │ ├── learning/ Lesson candidates and curation
|
||||
│ │ └── core/ the ported deterministic modules
|
||||
│ ├── tests/ eval/ kubernetes/ docs/adr/ .agents/CONTEXT.md
|
||||
├── mcp-servers/
|
||||
│ ├── cloud-docs.server.yaml cloud-docs/ free-text documentation search
|
||||
│ └── terraform-plan.server.yaml terraform-plan/ plan with a project's credential
|
||||
├── workflows/cp1-solution-architecture.json
|
||||
└── scripts/{build_workflow.py, dry_run.py, curate_lessons.py}
|
||||
```
|
||||
|
||||
The two servers sit here because only this agent uses them, which the repository's rule allows: a server belongs at the root when more than one Feature or pattern needs it.
|
||||
|
||||
The agent's Skills are listed in `agent.yaml` by folder name, so nested folders need checking in step 1. If they're refused, use flat names such as `azure-waf-security`.
|
||||
|
||||
### 4.2 Source to platform mapping
|
||||
|
||||
| Source concept | On the platform |
|
||||
|---|---|
|
||||
| Execution | A workflow run, or a conversation's A2A `contextId` |
|
||||
| Approval | A review point (workflow), or an approving message (conversation) |
|
||||
| Request for changes (Revision) | `hitl_feedback` rerun, or a message. Both reach the same router route, and code reopens the Deliverable and its dependants. |
|
||||
| Pre-approved Execution | Workflow review points switched off, or `pre_approved` in a conversation's first message |
|
||||
| Orchestrator | Code: the ported router, `phases.py` and the Ledger |
|
||||
| Specialist | `sub_agents:` entry, `as_tool: false`, called with `ask_sub_agent` |
|
||||
| LangGraph state and checkpoints | Run workspace: write-once records and a hash-chained manifest on `execution-storage-pvc` |
|
||||
| Customer case | `engagement`: `/data/executions/solution-architecture/<engagement>/<run>/` |
|
||||
| Scope spec file | KAB upload or artifact step, or inline in a conversation message |
|
||||
| Model per role | One model, `claude-sonnet-5-5`, through LiteLLM |
|
||||
| Google's hosted MCP servers | `cloud-knowledge` plus the provider's discovery server, through the MCP gateway |
|
||||
| Cloud-read permission | `allow_cloud_read`, and the project's stored credential |
|
||||
|
||||
### 4.3 How a Phase runs
|
||||
|
||||
1. `pre_process` binds the run, verifies the manifest chain, takes the run's lock and imports files.
|
||||
2. The router reads the request or message and the Execution's state, and picks a route. With Jev on, Jev informs that choice; with it off, the playbook's rules decide.
|
||||
3. `phases.py` asks the Ledger what's ready, and runs ready Specialists in parallel with a concurrency limit.
|
||||
4. Each result goes through the source's submission checks. A refusal returns to the Specialist with the reason, up to the Revision limit.
|
||||
5. `post_process` builds the reply from the workspace. It owns `status`, `deliverables`, `open_questions` and `manifest_head`, so a model can't assert them.
|
||||
|
||||
The root agent runs no model to choose work, matching source ADR-0007. Step 0 must prove `ask_sub_agent` works inside `pre_process` with `Reply`. If it doesn't, the fallback is a root model with a single `run_turn` function.
|
||||
|
||||
Records stay write-once, so a Revision writes a new version (`design/target-architecture.v2`) and the manifest marks the current one. Finished Deliverables are skipped on resume.
|
||||
|
||||
### 4.4 Skills and guidance for Specialists
|
||||
|
||||
- `read_skill(name, reference?)` returns Skill text, limited to what the provider bundle grants that role.
|
||||
- `read_guidance(pillar, topic)` wraps `cloud_search_well_architected`, for pillar advisors. It relies on `call_tool` working from a sub-agent's function (spike b). If not, code fetches guidance per pillar before the advisor starts.
|
||||
- `search_cloud_docs(query)` replicates the source's free-text search, for the product selector, describer, target architect, design advisor and pillar advisors. It calls a new `cloud-docs` MCP server, with each provider's public documentation behind one tool.
|
||||
- **This is an exception to a repository rule.** `mcp-servers/README.md` limits knowledge queries to fixed lists, so client detail can't reach a vendor. The source treated this search as public documentation with no customer data, but a model writes the query. So `cloud-docs` scrubs every query, limits its length, and logs it, and the exception gets an ADR that the repository's owner needs to accept.
|
||||
- It reuses the endpoints `knowledge_support.py` already calls for each provider (section 4.12).
|
||||
|
||||
### 4.5 Terraform
|
||||
|
||||
The agent runs `fmt`, `validate`, schema lookup and `plan`, never `apply`, as the source does. The Dockerfile adds a checksum-verified Terraform binary and a mirror of the three pinned providers (`hashicorp/google`, `hashicorp/azurerm`, `hashicorp/aws`), so validation needs no network.
|
||||
|
||||
- **`plan`** needs a customer's cloud credential and network access to their cloud, and agents get neither. So a new `terraform-plan` MCP server runs it, using the read-only credential stored per project at the gateway. It takes the generated files, runs `plan` only, and has no apply path. As in the source, it's a `cloud_read` tool, available only when `allow_cloud_read` is granted.
|
||||
- **Published modules** are a separate problem: `terraform init` downloads them, and a provider mirror doesn't cover modules. The modules each provider's Skill approves are baked into the image at their exact versions, and anything else is refused, as the source already refuses unlisted namespaces.
|
||||
|
||||
Three provider mirrors will make the image much larger, so measure it in spike (d). If it's a problem, build one image per provider. Kyndryl's CI may also hold the Dockerfile to a stricter shape, so confirm this before raising a pull request there. The fallback is a Terraform MCP server, which would sit outside this folder.
|
||||
|
||||
### 4.6 Live conversation
|
||||
|
||||
The agent answers two kinds of caller, and the request says which one it is: a workflow sends `phase`, and a conversation sends a message without one.
|
||||
|
||||
```
|
||||
Workflow: step(phase) ──────────► Phase runner ──► structured reply ──► review point
|
||||
Conversation: message + contextId ──► router ──┬─► question or approval request (reply writer)
|
||||
└─► Phase runner ──► progress notes, then summary
|
||||
```
|
||||
|
||||
- **One Execution per `contextId`.** The workspace maps it to the run, so any replica can take the next turn and a restart loses nothing.
|
||||
- **What a turn does.** The router routes the message as in the source: an answer to a waiting question, a request to change submitted work, an approval, a question about delivered work, a new request, or something out of scope. It asks one set of questions at a time and waits.
|
||||
- **A turn running a Phase.** The caller sees `progress()` notes throughout. If the turn reaches its time limit, finished Deliverables are kept, the reply says so, and the next message continues.
|
||||
- **Concurrency.** A lock on the run stops a second message starting work at the same time. It gets a plain "still working" reply.
|
||||
- **Approvals are recorded** in the manifest with the message that gave them, and the caller's identity when the request carries one.
|
||||
|
||||
**The gap is the client.** Direct conversation works from any A2A client, such as the platform's A2A Inspector or the template's `test_chatbot.py`, but those are developer tools, not a customer screen. Two options to close it, neither assumed in v1:
|
||||
|
||||
1. Spike (j): raise a question as an action-review pause, so it appears in KAB's run chat during a workflow. It's a stretch of a feature built for tool approvals, a pause inside a sub-agent reruns that sub-agent, and a step allows 10 pauses, so only the root code could ask.
|
||||
2. A KAB change to add direct chat with an agent. It belongs to the platform team and sits outside this folder.
|
||||
|
||||
Without either, a workflow run in the KAB UI still falls back to answering questions in review comments, within the 5 reruns.
|
||||
|
||||
### 4.7 Workflow
|
||||
|
||||
`scripts/build_workflow.py` generates this, using `kab_front_steps.py`:
|
||||
|
||||
```
|
||||
upload / artifacts → scoping ⏸ → requirements ⏸ → design ⏸ → implementation ⏸ → validation ⏸ → packaging ⏸
|
||||
```
|
||||
|
||||
Discovery's output can come in through `select_artifacts`, which links the two CP1 agents. Phase timeouts start at 30 minutes, with `AGENT_MAX_DURATION_SECONDS` at 1740. Measure these in step 9.
|
||||
|
||||
### 4.8 Three hyperscalers
|
||||
|
||||
The engine mostly separates providers already, but the source built only for Google. `deliverables.py` hard-codes the six pillar headings, the eight guide sections and the Terraform attachment names. The source's own step 14 ("provider-owned contracts") deferred that fix until a second provider existed, so it now comes first.
|
||||
|
||||
| Per provider | Google Cloud | Azure | AWS |
|
||||
|---|---|---|---|
|
||||
| Bundle: scope values, models, roles, Terraform pins | port `gcp.yaml` | new | new |
|
||||
| Well-Architected pillars | 6, vendored Skills | 5 pillars, new Skills | 6 pillars, new Skills |
|
||||
| Solution guide template | vendored | new | new |
|
||||
| IaC patterns Skill | port | new | new |
|
||||
| Terraform modules Skill | port | new | new |
|
||||
| Lessons Skill | port | starts empty | starts empty |
|
||||
| Discovery server | `gcp-discovery` | `azure-discovery` | `aws-discovery` |
|
||||
| Guidance | `cloud-knowledge`, provider `gcp` | provider `azure` | provider `aws` |
|
||||
| Evaluation | 8 ported cases | at least 3 new | at least 3 new |
|
||||
|
||||
- **The rules are a checklist:** `grep -rni "terraform\|google\|gcp" app/core app/router.py app/phases.py` should come back empty, and no pillar or section name may be a literal in `deliverables.py`.
|
||||
- **Azure and AWS Skills are the biggest new content.** Google's come from its published `google/skills` repository. Use an official equivalent where one exists under a usable licence; otherwise write them from each provider's published Well-Architected Framework. Google's IaC patterns (100 KB) and modules (32 KB) are the guide to size.
|
||||
- **Scope catalogue values** such as `shared_vpc_hub_and_spoke` become provider-specific in the bundles: Azure hub-and-spoke or Virtual WAN, AWS Transit Gateway.
|
||||
- **Discovery tools are declared with their provider prefix.** `agent.yaml` is static, so it declares all three servers, and code offers only the Execution's provider. Each server must be registered in the tenant, and a customer credential is stored per project.
|
||||
- **Multi-cloud designs are out of scope:** one Execution, one provider.
|
||||
|
||||
### 4.9 Jev, optional
|
||||
|
||||
Jev stays off unless `TYPESAFE_API_KEY` is set. The source's `Decider` interface is ported with it: `NoDecider` makes every decision take its fallback, and `JevDecider` calls the service. Each review keeps its own `JEV_*` switch and threshold.
|
||||
|
||||
- **Without a key:** nothing calls Jev, so the agent behaves the same on the platform with no Jev hosting or allowance.
|
||||
- **To turn it on:** add `typesafe-sdk==0.7.2` to the image, put the key in the agent's Secret, set `TYPESAFE_BASE_URL`, add that host to `AGENT_EGRESS_ALLOW`, and allow it in any NetworkPolicy.
|
||||
- **Data leaves the platform's gateways.** Jev receives requirements and user messages, which may be client information, and bypasses the gateway's logging and cost tracking. Each customer deployment should approve this before it's turned on. Record that in the ADR.
|
||||
- **Evaluation runs both ways,** so the fallback isn't left untested.
|
||||
|
||||
### 4.10 Models
|
||||
|
||||
Every role, including the reply writer, uses `claude-sonnet-5-5`, set once as `LITELLM_MODEL` with no per-role overrides. `MODEL_TIERS` is removed.
|
||||
|
||||
- Discovery's critic deliberately runs on a different model from the agents it judges. Here the validator and coverage reviewer share the authors' model, so the checks written in code matter more.
|
||||
- Whether the cluster's LiteLLM serves this name isn't confirmed (spike f).
|
||||
- Prompts were tuned on Gemini, so quality has to be measured, not assumed (step 9).
|
||||
|
||||
### 4.11 Change requests, lessons and overlays
|
||||
|
||||
- **Change requests** start a new Execution from a completed Baseline in the same engagement. The Baseline's run folder is read and never modified. The Execution delivers the full updated Deliverables with `changes.json`, as in the source.
|
||||
- **Lesson candidates** are recorded in the workspace as they happen: Revisions and their reasons, tool refusals, and a user's requests for changes.
|
||||
- **Curation stays offline and human-approved.** `scripts/curate_lessons.py` proposes Lessons from the volume, a person approves them, evaluation has to show they help, and they're published into each provider's lessons Skill. Publishing means a new image version. Azure and AWS lessons start empty and fill the same way.
|
||||
- **Overlay Skills** are named when an Execution starts, and come only from configured folders: `skills/overlays/` in the image, plus an optional folder on the volume set by `OVERLAY_SKILLS_DIRS`. `read_skill` refuses anything else.
|
||||
|
||||
### 4.12 Discovery server review (2026-10-30)
|
||||
|
||||
**Result: not at parity.** The three servers are read from their definitions and code, not run against a cloud. They cover inventory, topology and policy well, but miss live metrics and much of the identity picture. All three also depend on a credential path that doesn't work yet.
|
||||
|
||||
The source allows 39 cloud-read tools across 11 Google servers, plus 3 documentation tools. The servers each define 13 tools, one a connection check, built for estate discovery, so depth differs even where the names match.
|
||||
|
||||
| Source capability | Google Cloud | Azure | AWS |
|
||||
|---|---|---|---|
|
||||
| Projects, accounts, structure | Yes | Yes | Yes |
|
||||
| Asset inventory | Yes | Yes | Yes, needs AWS Config recording |
|
||||
| Compute, disks, instance groups | Yes | Yes | Yes |
|
||||
| Kubernetes clusters, node pools | Yes | Yes | Yes |
|
||||
| Serverless | Yes | Yes | Yes |
|
||||
| Databases, storage | Yes | Yes | Yes |
|
||||
| Network topology | Yes, beyond the source | Yes | Yes |
|
||||
| Recommendations | Yes, beyond the source | Yes | Yes |
|
||||
| Org policies | Yes, beyond the source | Yes | Yes |
|
||||
| Detailed configuration of one resource | **Unproven:** Cloud Asset search replaces each product's `get_*` | Likely: Resource Graph `properties` | Likely: Config record |
|
||||
| Instance templates, Pub/Sub, alerts, dashboards | **Unproven:** only through a generic asset type | **Unproven:** through generic resource type | **Unproven:** through Config |
|
||||
| Reservations, commitments | **Unproven**, same route | Not checked | Not checked |
|
||||
| IAM on resources | Partial: lists bindings | **Missing:** no role assignments; `authorizationresources` isn't an allowed table | Partial: trust and resource policies, not role permissions |
|
||||
| Who has access (`analyze_iam_policy`) | **Missing** | **Missing** | **Missing** |
|
||||
| Roles and deny policies | **Missing.** The server plan says deny policies are covered, but the code doesn't read them | **Missing** | **Missing** |
|
||||
| Live metrics (`list_timeseries`, metric descriptors) | **Missing** | **Missing**, planned "later" | **Missing** |
|
||||
| Connectivity tests (validator role) | **Missing** | **Missing** | **Missing** |
|
||||
| Available Kubernetes versions | **Missing** | **Missing** | **Missing** |
|
||||
|
||||
**Impact.** The missing metrics weaken sizing from real usage. The missing identity tools weaken the current-state step "identities and their roles", worst on Azure. The source's prompts don't name these tools, so nothing breaks; Specialists would record discovery gaps instead, as `cloud-discovery` tells them to. Connectivity tests matter least: the validator's prompt relies on Terraform tools.
|
||||
|
||||
**Fix.** Add these read-only tools to the three discovery servers: `get_metrics` with fixed metric lists, `list_iam` (role assignments, roles, deny policies), and `analyze_access`. They belong in the shared servers, not in this folder, because Discovery wants them too, so this is a change outside the boundary (section 8). Allow `authorizationresources` for Azure. Any read beyond each server's current customer role also changes `docs/customer-cloud-access.md`.
|
||||
|
||||
**Status facts, including corrections:**
|
||||
|
||||
| Fact | State |
|
||||
|---|---|
|
||||
| Credentials | **Blocker.** The Vault Service isn't deployed on the dev cluster, so no stored credential can reach a server through the gateway. `docs/platform-issues/vault-service-not-deployed/`. This also blocks `terraform-plan`, and `cloud-docs` if it needs keys |
|
||||
| Real-cloud testing | Azure once, on one dev subscription, bypassing the gateway. AWS and Google Cloud have only been tested against simulations, with no sandbox |
|
||||
| Deployed and registered | Yes, in tenant `CP1 Agentic Landing Zone`. Not connected to any agent |
|
||||
| Stale docs | `.agents/CONTEXT.md` says none is tested on a real cloud. The server plan says Azure doesn't back off on 429, but `azure_support.py` now does, unverified live |
|
||||
| Rate limits | AWS calls don't use the shared back-off helper. Whether boto3's own retries are enough is unchecked |
|
||||
| Documentation search | Endpoints exist for all three, in `knowledge_support.py`: `microsoft_docs_search`, `aws___search_documentation`, `search_documents`. `cloud-docs` can reuse them |
|
||||
|
||||
## 5. Scope
|
||||
|
||||
Parity is the rule: anything the source does is in v1 unless this table says otherwise. Step 0 produces a line-by-line parity checklist from the source's README, `providers/gcp.yaml` and `.env.example`, and step 9 closes it.
|
||||
|
||||
| In v1 | Replaced by an equivalent | Intentionally different |
|
||||
|---|---|---|
|
||||
| All 6 Phases and 11 Deliverables | Own A2A server and `/generate`: the runtime, and a pre-approved run | One model for every role, so no `MODEL_TIERS` |
|
||||
| Target architecture graph, Checks, Requirements chain | Postgres and SQLite records: the workspace and KAB's run records | Azure and AWS added |
|
||||
| Work breakdown and Solution guide | LangGraph checkpoints: the workspace | KAB workflow route added |
|
||||
| Live conversation, Approvals and Revisions | Google's hosted discovery servers: the three discovery servers, once the gaps in 4.12 are closed | |
|
||||
| Change requests and Baselines | | |
|
||||
| Lessons loop, curation and proof | | |
|
||||
| Overlay Skills | | |
|
||||
| Terraform `fmt`, `validate`, `plan`, schema lookup, pinned modules | | |
|
||||
| Free-text documentation search | | |
|
||||
| Jev and every review, off unless configured | | |
|
||||
| Evaluation harness and judge | | |
|
||||
|
||||
Still not included, because the source doesn't do them either: multi-cloud in one Execution, and a customer-ready chat screen in KAB.
|
||||
|
||||
## 6. Build steps
|
||||
|
||||
| # | Step | Done when |
|
||||
|---|---|---|
|
||||
| 0 | **Spikes on `agent-runtime-adk:0.35.7`:** (a) `ask_sub_agent` in `pre_process` with `Reply`, and parallel calls; (b) `call_tool` from a sub-agent's function; (c) rerun delivers `hitl_feedback` in an imported workflow; (d) Terraform with three provider mirrors, as the runtime user, and the image size; (e) exact tool names on the gateway for all four servers (it reports 14 tools per discovery server and 3 on `cloud-knowledge`, one more than each definition), and the "Unproven" rows in 4.12 tested on a real estate per provider; (f) `claude-sonnet-5-5` on LiteLLM; (g) writes to the volume; (h) a plain-text message passes the input contract, and a free-text reply passes the output contract; (i) `contextId` reaches the hooks across turns; (j) a question through action review shows in run chat; (k) Jev reachable from the cluster; (l) `render_mcp_server.py` renders a server inside this folder; (m) a server can run `plan` with a per-project credential | Each has a recorded result, the design is updated, and the parity checklist exists. Features were read from 0.37.0 source, so 0.35.7 may differ. |
|
||||
| 1 | Skeleton agent from `subagent-pattern`, with one stub Specialist | `check_agents.py` passes |
|
||||
| 2 | Port the core and tests, then make it provider-owned (source step 14) | Ported tests pass; the grep in 4.8 is empty |
|
||||
| 3 | Workspace, Ledger, router, Phase runner, hooks, `NoDecider` | `dry_run.py` completes all 6 Phases with fake Specialists, as a workflow and as a conversation, including a Revision and a resume from another process |
|
||||
| 4 | Specialists and Google Cloud: prompts, `read_skill`, guidance, bundle | Roles load only their granted Skills; vendored checksums match |
|
||||
| 5 | Terraform and validation | A known-bad case is refused and a known-good one passes offline, in the built image, for all three providers |
|
||||
| 6 | Azure and AWS: bundles, Skills, templates, evaluation cases | A dry run completes for each provider with no code change |
|
||||
| 7 | Parity features: Change requests, lessons loop, overlay Skills, `cloud-docs` and `terraform-plan` servers | Ported tests pass; both servers pass their tests and refuse what they must; a Change request dry run leaves its Baseline unchanged |
|
||||
| 8 | Jev: port the reviews, and write the workflow export, ADRs, README and user guide | Tests pass with and without a key; `kab_import_workflow.py` validates without `--apply` |
|
||||
| 9 | Deploy to dev, run evaluation with real models, tune timeouts. **Needs your approval for cluster changes.** | All 8 Google cases compare well with the source; 3 cases pass on each of Azure and AWS; one full conversation runs to a Solution guide; the parity checklist is closed |
|
||||
|
||||
ADRs to write: pattern and two ways in; Approvals as review points and messages; live conversation; Jev as an option; Skills for sub-agents; Terraform in the image; `terraform plan` through a server; free-text search exception; provider-owned contracts; one model for every role.
|
||||
|
||||
## 7. Risks
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|---|---|---|
|
||||
| Spike (a), (b) or (h) fails | The runner, guidance or conversation changes shape | Fallbacks in 4.3 and 4.4; a separate agent for conversation if (h) fails; spikes run before any porting |
|
||||
| Azure and AWS have no proven designs behind them | Weaker first results than Google, which went through ten iterations | New evaluation cases; describe them as first-version until they pass |
|
||||
| Free-text search could carry client detail to a vendor | Confidentiality; the repository's owner may refuse the exception | Scrubbed, limited and logged queries, and an ADR. If refused, this one parity item falls back to `cloud-knowledge` |
|
||||
| `terraform-plan` handles a customer credential | A credential or plan output could leak, or a plan could show secrets | Read-only credential per project, no apply path, every reply scrubbed, `allow_cloud_read` required |
|
||||
| Parity makes v1 much larger | Longer before anything can be used | Run one Google Cloud case with real models after step 5, with your approval for the cluster change, so problems show early |
|
||||
| No customer-ready conversation screen | Live conversation stays a developer route | Spike (j); raise with the platform team |
|
||||
| The image becomes very large | Slow builds and pulls | Measure; one image per provider if needed |
|
||||
| One model authors and checks | Shared blind spots | Code checks stay authoritative; the Requirements chain is enforced in code |
|
||||
| A Phase overruns a turn or step | Lost time | Write-once records make the rerun resume |
|
||||
| Jev sends client data outside the gateways | Compliance | Off by default; approval per deployment |
|
||||
| Client data on a shared volume | Retention not agreed | Same open question as Discovery ADR 0010 |
|
||||
| Vault Service isn't deployed | No customer credential reaches any server, so cloud reads and `terraform plan` can't run | Needs the platform deployment's owner. Everything else works without it, with cloud reads recorded as discovery gaps |
|
||||
| Discovery servers miss metrics and identity detail (4.12) | Designs are based on less of the current state than the source | Extend the shared servers; until then Specialists record gaps |
|
||||
| AWS and Google Cloud servers have never met a real cloud, and there are no sandboxes | Discovery may fail at first use | A sandbox per provider (D7 in the server plan); spike (e) |
|
||||
| Vendored Google Skills are third-party | Licence obligations | Keep `LICENSE`, `SHA256SUMS`, `VENDOR.md`; don't edit them |
|
||||
|
||||
## 8. Needed outside this folder (not done here)
|
||||
|
||||
Until these change, the agent can be checked, and its workflow validated, by passing explicit paths.
|
||||
|
||||
| File | Change | Effect until then |
|
||||
|---|---|---|
|
||||
| `scripts/render_k8s.py` | Add `outcome-pattern` to `PATTERNS` | **Blocks deployment through the normal route** |
|
||||
| `deploy/crucible-kab-dev.yaml` | Add the agent's entry | Blocks deployment |
|
||||
| `scripts/render_k8s.py` and `deploy/crucible-kab-dev.yaml` | Find and list the two MCP servers in this folder; the script looks only in the root `mcp-servers/` | **Blocks deploying `cloud-docs` and `terraform-plan`** |
|
||||
| `mcp-servers/README.md` | Record the free-text exception, if accepted | Rule and server disagree |
|
||||
| `mcp-servers/{gcp,azure,aws}-discovery.server.yaml`, `_common/` | Add metrics, IAM and access tools; allow `authorizationresources` (4.12) | **Discovery isn't at parity** |
|
||||
| `docs/customer-cloud-access.md` | Any extra roles those tools need | Customers grant too little |
|
||||
| `.agents/CONTEXT.md`, `docs/hyperscaler-mcp-servers-plan-2026-10-30.md` | Correct the stale facts in 4.12 | Docs mislead |
|
||||
| KAB dev cluster | Deploy the Vault Service. Platform owner, not this repository | **Blocks every stored credential** |
|
||||
| `.gitea/workflows/check.yml` | Add `outcome-pattern` to the three `find` commands, a dry-run step and the two servers' tests | CI doesn't check this agent |
|
||||
| `CLAUDE.md`, `README.md`, `.agents/CONTEXT.md` | Describe the third pattern | Docs say there are two |
|
||||
Reference in New Issue
Block a user