Files
cp1-agents/outcome-pattern/solution_architecture/.agents/PLAN.md
Jonathan Boniface bc84ee4df6
Some checks failed
check agents / check (push) Failing after 13s
Solution architecture outcome pattern: agent and its MCP servers
The cp1-solution-architecture-agent and the sa-aws/azure/gcp-cloud MCP servers, a separate
stream from the individual and sub-agent patterns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

44 KiB

Plan: Solution Architecture agent on the modernization platform

Date 2026-10-30, revision 4
Status mcp-servers/ is built and tested locally (section 4.13). Step 0 is done as far as it can go without the cluster (section 4.14). Steps 1 to 5 are done, except what needs a built image. Step 6 is partly done (section 6).
Source kyndryl-agent-builder/gcp_solution_architecture_agent (read only)
Platform reviewed kyndryl-agent-builder/modernization-platform @ c89aedcb9 (2026-10-29), runtime 0.37.0. The other checkout, platform-engineering/modernization-platform, is older.
Target runtime agent-runtime-adk:0.35.7, the version cp1-agents pins.
Boundary Every new file sits under outcome-pattern/solution_architecture/. Changes needed elsewhere are listed in section 8, not made.

Changed in revision 2, at the owner's request: every model is claude-sonnet-5-5; Azure, AWS and Google Cloud are all in the first version; live conversation stays as an option; Jev stays as an option.

Changed in revision 3, at the owner's request: the agent replicates the source with full feature parity. Everything the source does is in v1 (section 5). The engine is still rebuilt on KAB so that it integrates. The only intended differences are the ones the owner chose, such as one model for every role.

Changed in revision 4, at the owner's request: the MCP layer is built like for like inside this folder, one server per hyperscaler. This replaces the plan to use and extend the shared discovery servers and cloud-knowledge. Sections 4.4, 4.12 and 8 are updated to match.

Changed in revision 5, at the owner's request: the Execution runs on real LangGraph, as in the source, not on a hand-written driver. See section 4.15.

1. Summary

Build one KAB agent that delivers the same outcome as the source: a request goes in, and a reviewed solution architecture comes out, with Deliverables, Target architecture, Terraform, validation, work breakdown and Solution guide. An Execution targets one Cloud provider, chosen from Azure, AWS and Google Cloud.

Don't lift the source's container onto the platform. Its engine conflicts with the runtime: A2A server, LangGraph checkpointing, interrupt() approvals, database and direct model and MCP clients. Keep what makes the source valuable and rebuild the execution layer on platform features.

Source Size Plan
Deterministic core: scope, deliverables, target_architecture, chain, work_breakdown, changes, ledger about 3,000 lines Port, made provider-neutral, with tests
Router and its playbook (app/nodes/router.py) about 900 lines Port. It's now needed for conversation and for Jev
Jev and its reviews: jev, meaning, skill_routing, guardrail, assessment, design_review, schema_review, coverage about 1,200 lines Port behind the source's Decider interface
Change requests: baseline, customer_case and the Change route about 250 lines, plus changes above Port
Lessons loop: app/learning/*, states/lessons, eval/curate_lessons.py about 1,000 lines Port
Evaluation harness: eval/ scoring, judge, calibration Port as scripts that run against the deployed agent
16 role files in agents/ Port as sub-agent prompts, made provider-neutral
Google Skills, including vendored ones 49 files Port with the licence and checksums
Azure and AWS Skills none exist New (section 4.8)
8 evaluation cases Port, then add Azure and AWS cases
execution_nodes, app/adk/*, a2a_server, database, checkpointing about 3,400 lines Replace with the platform

2. Platform constraints that shape the design

Constraint Where Consequence
The runtime fails any run that pauses with LangGraph interrupt(), and keeps no checkpointer a2a/server.py _refuse_interrupt The source's Approvals can't move as they are
Agents keep nothing between requests continuation.py Execution state has to live outside the process
A workflow step is reviewed after it runs: continue, stop, rerun with comments (hitl_feedback), or replace the output. Up to 5 reruns per step, and 50 steps per workflow generic_workflow_executor.py, workflow_template_validator.py In a workflow, each Phase is a step and each Approval a review point
Run chat fills in a workflow's inputs and handles its reviews. Catalog chat answers about the catalog. I found no KAB screen for talking directly to a deployed agent services/run_chat/, api/routes/catalog_chat.py Live conversation needs an A2A client outside those screens (section 4.6)
Agents reach only the model and MCP gateways, plus hosts in AGENT_EGRESS_ALLOW egress.py Jev needs an explicit allowance
Sub-agents have a prompt and functions only, with no MCP tools or Skills Discovery .agents/CONTEXT.md Skills and guidance need a code path (section 4.4)
An agent with its own app/agent.py gets no sub-agent registry or resume sub_agents.py, continuation.py Use a spec agent, not a wrapped graph
cloud-knowledge takes fixed pillars and topics only, and mcp-servers/README.md forbids free-text fields cloud-knowledge.server.yaml The agent uses its own servers for free-text search (section 4.13)
A declared MCP tool the gateway doesn't offer stops the agent at startup runtime README Tool names must be confirmed first
Requests have a wall-clock limit (AGENT_MAX_DURATION_SECONDS) configmap.yaml One full Execution took 22 to 35 minutes, so Phases need separate requests

3. Approach and alternatives

Chosen: an ADK spec agent with two ways in, a workflow and a live conversation, sharing one Phase runner and one workspace.

Alternative Why not
Keep the source's container and register it by URL No platform limits, tracing or cost logs, and it breaks egress
framework: langgraph with the graph in app/agent.py Loses ask_sub_agent and the runtime's handling of Specialists. Real LangGraph runs inside the ADK agent's hooks instead (section 4.15)
One agent per Deliverable, chained in a workflow That's the individual-agents/ pattern, and it rules out live conversation
Port orchestration_engine and solution_architecture_agent A cleaner split with the same conflicts. Compare module by module when porting (step 2).

4. Design

4.1 Folder layout

solution_architecture/ has the same depth as a Feature folder, so the shared scripts work with it unchanged.

outcome-pattern/solution_architecture/
├── CLAUDE.md  README.md  .agents/PLAN.md
├── cp1-solution-architecture-agent/
│   ├── agent.yaml  prompt.md  Dockerfile  RELEASE_NOTES.md  README.md  CLAUDE.md
│   ├── schemas/{input,output}.json
│   ├── prompts/<specialist>.md        from agents/*.md
│   ├── skills/{common,gcp,azure,aws}/
│   ├── providers/{gcp,azure,aws}.yaml
│   ├── app/
│   │   ├── business_logic.py          hooks: bind the run, route the message, build the reply
│   │   ├── kab_runtime.py             runs a role through ask_sub_agent
│   │   ├── engine/                    the source's engine on LangGraph
│   │   ├── skill_reader.py            role-scoped Skill loading
│   │   ├── terraform.py               fmt, validate and schema lookup
│   │   ├── decisions/                 Decider, NoDecider, Jev and its reviews
│   │   ├── learning/                  Lesson candidates and curation
│   │   └── core/                      the ported deterministic modules
│   ├── tests/  eval/  kubernetes/  docs/adr/  .agents/CONTEXT.md
├── mcp-servers/                       built: see its README.md
│   ├── sa-gcp-cloud/  sa-azure-cloud/  sa-aws-cloud/
│   └── terraform-plan/                not built: plan with a project's credential
├── workflows/cp1-solution-architecture.json
└── scripts/{build_workflow.py, dry_run.py, curate_lessons.py}

terraform-plan sits here because only this agent uses it, which the repository's rule allows: a server belongs at the root when more than one Feature or pattern needs it.

The agent's Skills are listed in agent.yaml by folder name, so nested folders need checking in step 1. If they're refused, use flat names such as azure-waf-security.

4.2 Source to platform mapping

Source concept On the platform
Execution A workflow run, or a conversation's A2A contextId
Approval A review point (workflow), or an approving message (conversation)
Request for changes (Revision) hitl_feedback rerun, or a message. Both reach the same router route, and code reopens the Deliverable and its dependants.
Pre-approved Execution Workflow review points switched off, or pre_approved in a conversation's first message
Orchestrator The ported router and Ledger, in the source's LangGraph graph
Specialist sub_agents: entry, as_tool: false, called with ask_sub_agent
LangGraph state and checkpoints Real LangGraph, with VolumeSaver on execution-storage-pvc (section 4.15)
Customer case engagement: /data/executions/solution-architecture/<engagement>/<run>/
Scope spec file KAB upload or artifact step, or inline in a conversation message
Model per role One model, claude-sonnet-5-5, through LiteLLM
Google's hosted MCP servers The same servers through sa-gcp-cloud, plus sa-azure-cloud and sa-aws-cloud, through the MCP gateway
Cloud-read permission allow_cloud_read, and the project's stored credential

4.3 How a Phase runs

  1. pre_process binds the run, verifies the manifest chain, takes the run's lock and imports files.
  2. The router reads the request or message and the Execution's state, and picks a route. With Jev on, Jev informs that choice; with it off, the playbook's rules decide.
  3. The graph asks the Ledger what's ready, and runs ready Specialists in parallel with a concurrency limit.
  4. Each result goes through the source's submission checks. A refusal returns to the Specialist with the reason, up to the Revision limit.
  5. post_process builds the reply from the workspace. It owns status, deliverables, open_questions and manifest_head, so a model can't assert them.

The root agent runs no model to choose work on a JSON request, matching source ADR-0007. A plain-text message needs one model call to reach this code (section 4.14).

Every finished step is checkpointed, so a stopped request loses nothing it finished.

4.4 Skills and guidance for Specialists

  • read_skill(name, reference?) returns Skill text, limited to what the provider bundle grants that role.
  • search_cloud_docs(query) replicates the source's free-text search, for the product selector, describer, target architect, design advisor and pillar advisors. It calls the documentation tools on the Execution's provider server, and relies on call_tool working from a sub-agent's function (spike b). If not, code fetches documents per Deliverable before the Specialist starts.
    • This is an accepted exception to a repository rule (owner decision, 2026-10-30). The root mcp-servers/README.md limits knowledge queries to fixed lists, so client detail can't reach a vendor. Here a model writes the query. The rule still holds for the shared servers.
    • Queries aren't scrubbed (owner decision, 2026-10-30). They go to the vendor as the model wrote them. Specialists' prompts tell them to keep client detail out of searches, but that's guidance, not enforcement.
    • Safeguards that remain: every query is logged, and replies pass through _scrub. A per-deployment setting on the agent turns search_cloud_docs off, and it starts off until a deployment enables it. The ADR records the decision.

4.5 Terraform

The agent runs fmt, validate, schema lookup and plan, never apply, as the source does. The Dockerfile adds a checksum-verified Terraform binary and a mirror of the three pinned providers (hashicorp/google, hashicorp/azurerm, hashicorp/aws), so validation needs no network.

  • plan needs a customer's cloud credential and network access to their cloud, and agents get neither. So a new terraform-plan MCP server runs it, using the read-only credential stored per project at the gateway. It takes the generated files, runs plan only, and has no apply path. As in the source, it's a cloud_read tool, available only when allow_cloud_read is granted.
  • Published modules need the network. Terraform has no offline mirror for registry modules, so the earlier plan to bake them in doesn't work. terraform init and the module lookup both reach registry.terraform.io, so a deployment that sets TERRAFORM_MODULES must allow that host in AGENT_EGRESS_ALLOW and its network policy. It's off by default, as in the source. Unlisted namespaces are still refused.

The three provider mirrors add 260 MB, so one image serves all three clouds. Kyndryl's CI may also hold the Dockerfile to a stricter shape, so confirm this before raising a pull request there. The fallback is a Terraform MCP server, which would sit outside this folder.

4.6 Live conversation

The agent answers two kinds of caller, and the request says which one it is: a workflow sends phase, and a conversation sends a message without one.

Workflow:      step(phase) ──────────► Phase runner ──► structured reply ──► review point
Conversation:  message + contextId ──► router ──┬─► question or approval request (reply writer)
                                                └─► Phase runner ──► progress notes, then summary
  • One Execution per contextId. The workspace maps it to the run, so any replica can take the next turn and a restart loses nothing.
  • What a turn does. The router routes the message as in the source: an answer to a waiting question, a request to change submitted work, an approval, a question about delivered work, a new request, or something out of scope. It asks one set of questions at a time and waits.
  • A turn running a Phase. The caller sees progress() notes throughout. If the turn reaches its time limit, finished Deliverables are kept, the reply says so, and the next message continues.
  • Concurrency. A lock on the run stops a second message starting work at the same time. It gets a plain "still working" reply.
  • Approvals are recorded in the manifest with the message that gave them, and the caller's identity when the request carries one.

The gap is the client. Direct conversation works from any A2A client, such as the platform's A2A Inspector or the template's test_chatbot.py, but those are developer tools, not a customer screen. Two options to close it, neither assumed in v1:

  1. Spike (j): raise a question as an action-review pause, so it appears in KAB's run chat during a workflow. It's a stretch of a feature built for tool approvals, a pause inside a sub-agent reruns that sub-agent, and a step allows 10 pauses, so only the root code could ask.
  2. A KAB change to add direct chat with an agent. It belongs to the platform team and sits outside this folder.

Without either, a workflow run in the KAB UI still falls back to answering questions in review comments, within the 5 reruns.

4.7 Workflow

scripts/build_workflow.py generates this, using kab_front_steps.py:

upload / artifacts → scoping ⏸ → requirements ⏸ → design ⏸ → implementation ⏸ → validation ⏸ → packaging ⏸

Discovery's output can come in through select_artifacts, which links the two CP1 agents. Phase timeouts start at 30 minutes, with AGENT_MAX_DURATION_SECONDS at 1740. Measure these in step 9.

4.8 Three hyperscalers

The engine mostly separates providers already, but the source built only for Google. deliverables.py hard-codes the six pillar headings, the eight guide sections and the Terraform attachment names. The source's own step 14 ("provider-owned contracts") deferred that fix until a second provider existed, so it now comes first.

Per provider Google Cloud Azure AWS
Bundle: scope values, models, roles, Terraform pins port gcp.yaml new new
Well-Architected pillars 6, vendored Skills 5 pillars, new Skills 6 pillars, new Skills
Solution guide template vendored new new
IaC patterns Skill port new new
Terraform modules Skill port new new
Lessons Skill port starts empty starts empty
Discovery server gcp-discovery azure-discovery aws-discovery
Guidance cloud-knowledge, provider gcp provider azure provider aws
Evaluation 8 ported cases at least 3 new at least 3 new
  • The rules are a checklist: grep -rni "terraform\|google\|gcp" app/core app/engine should come back empty, and no pillar or section name may be a literal in deliverables.py.
  • Azure and AWS Skills are the biggest new content. Google's come from its published google/skills repository. Use an official equivalent where one exists under a usable licence; otherwise write them from each provider's published Well-Architected Framework. Google's IaC patterns (100 KB) and modules (32 KB) are the guide to size.
  • Scope catalogue values such as shared_vpc_hub_and_spoke become provider-specific in the bundles: Azure hub-and-spoke or Virtual WAN, AWS Transit Gateway.
  • Discovery tools are declared with their provider prefix. agent.yaml is static, so it declares all three servers, and code offers only the Execution's provider. Each server must be registered in the tenant, and a customer credential is stored per project.
  • Multi-cloud designs are out of scope: one Execution, one provider.

4.9 Jev, optional

Jev stays off unless TYPESAFE_API_KEY is set. The source's Decider interface is ported with it: NoDecider makes every decision take its fallback, and JevDecider calls the service. Each review keeps its own JEV_* switch and threshold.

  • Without a key: nothing calls Jev, so the agent behaves the same on the platform with no Jev hosting or allowance.
  • To turn it on: add typesafe-sdk==0.7.2 to the image, put the key in the agent's Secret, set TYPESAFE_BASE_URL, add that host to AGENT_EGRESS_ALLOW, and allow it in any NetworkPolicy.
  • Data leaves the platform's gateways. Jev receives requirements and user messages, which may be client information, and bypasses the gateway's logging and cost tracking. Each customer deployment should approve this before it's turned on. Record that in the ADR.
  • Evaluation runs both ways, so the fallback isn't left untested.

4.10 Models

Every role, including the reply writer, uses claude-sonnet-5-5, set once as LITELLM_MODEL with no per-role overrides. MODEL_TIERS is removed.

  • Discovery's critic deliberately runs on a different model from the agents it judges. Here the validator and coverage reviewer share the authors' model, so the checks written in code matter more.
  • Spike (f) failed on 2026-11-01: the cluster's LiteLLM doesn't serve claude-sonnet-5-5. The agent can't run on it until the model is added there.
  • Prompts were tuned on Gemini, so quality has to be measured, not assumed (step 9).

4.11 Change requests, lessons and overlays

  • Change requests start a new Execution from a completed Baseline in the same engagement. The Baseline's run folder is read and never modified. The Execution delivers the full updated Deliverables with changes.json, as in the source.
  • Lesson candidates are recorded in the workspace as they happen: Revisions and their reasons, tool refusals, and a user's requests for changes.
  • Curation stays offline and human-approved. scripts/curate_lessons.py proposes Lessons from the volume, a person approves them, evaluation has to show they help, and they're published into each provider's lessons Skill. Publishing means a new image version. Azure and AWS lessons start empty and fill the same way.
  • Overlay Skills are named when an Execution starts, and come only from configured folders: skills/overlays/ in the image, plus an optional folder on the volume set by OVERLAY_SKILLS_DIRS. read_skill refuses anything else.

4.12 Discovery server review (2026-10-30)

Superseded by 4.13 for this agent: the gaps below are what the agent's own servers now fill. The table stays as the record of why.

Result: the shared servers are not at parity. The three servers are read from their definitions and code, not run against a cloud. They cover inventory, topology and policy well, but miss live metrics and much of the identity picture. All three also depend on a credential path that doesn't work yet.

The source allows 39 cloud-read tools across 11 Google servers, plus 3 documentation tools. The servers each define 13 tools, one a connection check, built for estate discovery, so depth differs even where the names match.

Source capability Google Cloud Azure AWS
Projects, accounts, structure Yes Yes Yes
Asset inventory Yes Yes Yes, needs AWS Config recording
Compute, disks, instance groups Yes Yes Yes
Kubernetes clusters, node pools Yes Yes Yes
Serverless Yes Yes Yes
Databases, storage Yes Yes Yes
Network topology Yes, beyond the source Yes Yes
Recommendations Yes, beyond the source Yes Yes
Org policies Yes, beyond the source Yes Yes
Detailed configuration of one resource Unproven: Cloud Asset search replaces each product's get_* Likely: Resource Graph properties Likely: Config record
Instance templates, Pub/Sub, alerts, dashboards Unproven: only through a generic asset type Unproven: through generic resource type Unproven: through Config
Reservations, commitments Unproven, same route Not checked Not checked
IAM on resources Partial: lists bindings Missing: no role assignments; authorizationresources isn't an allowed table Partial: trust and resource policies, not role permissions
Who has access (analyze_iam_policy) Missing Missing Missing
Roles and deny policies Missing. The server plan says deny policies are covered, but the code doesn't read them Missing Missing
Live metrics (list_timeseries, metric descriptors) Missing Missing, planned "later" Missing
Connectivity tests (validator role) Missing Missing Missing
Available Kubernetes versions Missing Missing Missing

Impact. The missing metrics weaken sizing from real usage. The missing identity tools weaken the current-state step "identities and their roles", worst on Azure. The source's prompts don't name these tools, so nothing breaks; Specialists would record discovery gaps instead, as cloud-discovery tells them to. Connectivity tests matter least: the validator's prompt relies on Terraform tools.

Status facts, including corrections:

Fact State
Credentials Blocker. The Vault Service isn't deployed on the dev cluster, so no stored credential can reach a server through the gateway. docs/platform-issues/vault-service-not-deployed/. This also blocks the agent's own servers and terraform-plan. Microsoft Learn itself needs no credential, but the gateway may still require a stored connection
Real-cloud testing Azure once, on one dev subscription, bypassing the gateway. AWS and Google Cloud have only been tested against simulations, with no sandbox
Deployed and registered Yes, in tenant CP1 Agentic Landing Zone. Not connected to any agent
Stale docs .agents/CONTEXT.md says none is tested on a real cloud. The server plan says Azure doesn't back off on 429, but azure_support.py now does, unverified live
Rate limits AWS calls don't use the shared back-off helper. Whether boto3's own retries are enough is unchecked
Documentation search Endpoints exist for all three, in knowledge_support.py: microsoft_docs_search, aws___search_documentation, search_documents. The agent's servers call these

4.13 The agent's own MCP servers (built 2026-10-30)

mcp-servers/ holds three servers, rendered with the platform's builder, reusing the shared _common/ code read-only. Nothing outside this folder changed. See its README.md.

Check Result
sa-gcp-cloud 44 tools: all 42 of the source's, plus a connection check and gcp_describe_tool
sa-azure-cloud 40 tools, sa-aws-cloud 44 tools, matched by capability. No equivalent of answer_query
Local tests All pass on the three servers, with upstreams simulated
Google tools, live All 42 are offered and declare readOnlyHint, from anonymous tools/list on 2026-10-30. No tool has been called
AWS operations Every operation, result key and paging token exists in boto3's service models. No call to AWS
Azure documentation, live azure_search_documents and azure_get_documents both worked through the server on 2026-10-30 with a generic query, and each query was logged

Not proven:

  • No server has been called with a customer credential, against any real estate.
  • Whether Google's hosted servers accept the cloud-platform.read-only token gcp_support.py issues. The source used its own Application Default Credentials.
  • The AWS documentation tool aws___read_documentation and the sign-in to it. That name is from memory, and the server refuses cleanly if it isn't offered.
  • Azure's Resource Graph queries and ARM API versions have never run, including AuthorizationResources.
  • Images aren't built, nothing is deployed or registered, and no agent uses these servers.

One credential per project. These servers hold the customer's credential, so documentation search needs a stored connection too, unlike the source, where it ran on the agent's own identity. Google and AWS have optional documentation key fields for Kyndryl's keys. search_cloud_docs therefore can't run for a project until its connection is stored, which depends on the Vault Service.

4.14 Step 0 results (2026-10-30)

Run on the real 0.35.7 runtime source (commit c0b4a4bf8, ADK 2.12.0) on a local machine, with stub model and MCP gateway servers. This wasn't the image or the cluster. "Open" spikes need the cluster, which needs the owner's approval each time.

Spike Result Consequence
(a) ask_sub_agent in pre_process, with Reply Pass. No root model call. Two sub-agents ran in parallel: 2.2 s against 3 s in sequence The design in 4.3 stands
(b) call_tool from a sub-agent's function Pass. Tenant and project headers reach the gateway search_cloud_docs as designed. No fallback needed
(c) hitl_feedback Partial. It reaches pre_process if it's in the payload and declared in schemas/input.json. Delivery by the workflow engine is untested Declare it in the schema
(d) Terraform mirror Pass for size: 260 MB for all three providers, so one image. Running as the runtime user is untested Pins chosen for now: google 8.6.0 (the source's), azurerm 4.40.0, aws 6.10.0
(h) Plain text Changes the design. A message with no JSON skips pre_process and the input contract, and goes straight to the root model See below
(h) Free-text reply Works, with a logged warning if an output contract is declared. Replies must carry JSON to pass it Conversation replies are JSON with a reply field
(i) contextId Pass. Hooks read it from structlog.contextvars, as context_id Works, but it's not a documented interface. Fall back to conversation_id in the payload
Streaming Pass. progress() notes arrive during pre_process The design in 4.6 stands
(e), (f), (g), (k), (m), and (c) in the engine Open Need the cluster
(j) Questions in run chat Open, not worth doing yet. It needs an action review, and the source shows those exist to approve tool calls Revisit after v1

Plain text in conversations. Any A2A client can send prose, and prose never reaches pre_process. So the root agent gets one function, run_turn(message), and its prompt says to call it with the user's message and return the result unchanged. This was tested: the function ran and could call ask_sub_agent. That adds one model call per plain-text turn, and relies on the model calling it. A client that sends {"message": ...} skips that call. The tests must check that a turn which doesn't call run_turn is refused in post_process.

4.15 LangGraph inside the agent (decided by the owner, 2026-10-30)

  • How it runs. The graph is the source's, compiled in app/engine/execution_graph.py and run from pre_process. interrupt() pauses it there, and the hook returns the prompt as a Reply, so the runtime's refusal of interrupt() is never reached.
  • Checkpoints. app/engine/checkpoints.py is a BaseCheckpointSaver on the shared volume, with write-once files and a file lock. There's no Postgres, and SQLite isn't safe on a network volume with several replicas. This replaces the hash-chained workspace for state, so checkpoints have no tamper check. Delivered files are written as in the source.
  • Time limits. The engine reads astream one step at a time and stops between steps when time runs short. carry_on continues from the last checkpoint.
  • Versions. langgraph 1.2.11, langgraph-checkpoint 4.2.0 and langchain-core 1.6.3, the source's, are installed into /agent/vendor. They install beside ADK 2.12.0 in a local environment. The image isn't built, so the Dockerfile's pip stage is untested.

Verified on the 0.35.7 runtime source with a stub model: a JSON request and a plain-text message both ran a sub-agent through use_tool, inside LangGraph, inside pre_process. The Execution paused, and a later request resumed it from the volume. With the stub, the scope analyst only ever asks a question, so this run stopped at scoping. All six Phases have run through the hooks with fake Specialists, but not on the runtime.

4.16 Official Skills for Azure and AWS (reviewed 2026-10-30)

Nothing is vendored yet. Licences are as each repository states them; Kyndryl's legal or open-source review hasn't seen them. Commits are the ones reviewed.

Repository Commit Licence What it offers Verdict
microsoft/skills: cloud-solution-architect 3f5b461b MIT About 130 KB of written guidance: principles, styles, 44 patterns, technology choices, mission-critical Vendor. Closest to Google's solution architecture Skill. No output template
MicrosoftDocs/agent-skills: azure-well-architected 95544200 CC BY 4.0 for content, MIT for code One 30 KB index of Microsoft Learn URLs, no guidance of its own Vendor with attribution, only useful with documentation access
microsoft/azure-skills: azure-reliability, cost, compliance, azure-enterprise-infra-planner fbda857c MIT Candidates, not read in depth. The repository holds 220 script files Review each; any Skill with scripts/ is refused
aws/agent-toolkit-for-aws: service Skills 8eee01d9 Apache-2.0 Official and maintained. aws-security (288 KB), aws-iam, aws-compute, aws-database, aws-storage, aws-serverless, aws-networking have no scripts Vendor those that fit, with NOTICE. aws-observability has 26 scripts, so it needs trimming, which breaks "unmodified"
aws/agent-toolkit-for-aws: aws-well-architected-review 8eee01d9 Apache-2.0 Reviews existing code and IaC. Reads the framework live and writes corpus files Don't use as a pillar Skill. Wrong job, and heavy. Its references are useful background
aws-samples/sample-well-architected-skills-and-steering e81835b1 MIT-0 Six-pillar samples Don't use: deprecated, and marked not for production
awslabs/agent-plugins fbefb376 Apache-2.0 Superseded by the Agent Toolkit Don't use
hashicorp/agent-skills: terraform-style-guide, azure-verified-modules bc6b14b7 MPL-2.0 Terraform style and Azure modules. Most of the rest is for provider authors Vendor unmodified, once legal has looked at MPL-2.0

Still to write, because nothing official exists: one Skill per pillar for Azure (5) and AWS (6), to match pillar_skills; a Solution guide template for each; IaC patterns for each; a modules Skill for AWS.

Consequences:

  • The pillar Skills must carry their checklists themselves. CLOUD_DOCS_SEARCH is off by default, and Microsoft's index and AWS's review both depend on live documentation.
  • Write them in our own words from each provider's published framework, and cite it.
  • Vendored Skills are instructions a model follows. Pin each commit, keep checksums, and read every file before it's added, as with Google's.
  • Bug found in sa-aws-cloud: AWS's documentation tool is aws___read_documentation, as assumed, but it takes {"requests": [{"url": ..., "max_length": ...}]}. aws_get_documents sends {"url": ...}, so it needs fixing in step 6.

5. Scope

Parity is the rule: anything the source does is in v1 unless this table says otherwise. Step 0 produces a line-by-line parity checklist from the source's README, providers/gcp.yaml and .env.example, and step 9 closes it.

In v1 Replaced by an equivalent Intentionally different
All 6 Phases and 11 Deliverables Own A2A server and /generate: the runtime, and a pre-approved run One model for every role, so no MODEL_TIERS
Target architecture graph, Checks, Requirements chain Postgres and SQLite records: the workspace and KAB's run records Azure and AWS added
Work breakdown and Solution guide Postgres or SQLite checkpoints: VolumeSaver on the volume KAB workflow route added
Live conversation, Approvals and Revisions Google's hosted servers: the same servers through sa-gcp-cloud
Change requests and Baselines
Lessons loop, curation and proof
Overlay Skills
Terraform fmt, validate, plan, schema lookup, pinned modules
Free-text documentation search
Jev and every review, off unless configured
Evaluation harness and judge

Still not included, because the source doesn't do them either: multi-cloud in one Execution, and a customer-ready chat screen in KAB.

6. Build steps

# Step Done when
0 Spikes on agent-runtime-adk:0.35.7: (a) ask_sub_agent in pre_process with Reply, and parallel calls; (b) call_tool from a sub-agent's function; (c) rerun delivers hitl_feedback in an imported workflow; (d) Terraform with three provider mirrors, as the runtime user, and the image size; (e) the agent's three servers registered at the gateway, with their exact tool names, and each server in 4.13 called on a real estate per provider; (f) claude-sonnet-5-5 on LiteLLM; (g) writes to the volume; (h) a plain-text message passes the input contract, and a free-text reply passes the output contract; (i) contextId reaches the hooks across turns; (j) a question through action review shows in run chat; (k) Jev reachable from the cluster; (m) a server can run plan with a per-project credential Each has a recorded result, the design is updated, and the parity checklist exists. Features were read from 0.37.0 source, so 0.35.7 may differ.
1 Skeleton agent from subagent-pattern, with one stub Specialist check_agents.py passes
2 Port the core and tests, then make it provider-owned (source step 14) Ported tests pass; the grep in 4.8 is empty
3 Workspace, Ledger, router, Phase runner, hooks, NoDecider dry_run.py completes all 6 Phases with fake Specialists, as a workflow and as a conversation, including a Revision and a resume from another process
4 Specialists and Google Cloud: prompts, read_skill, guidance, bundle Roles load only their granted Skills; vendored checksums match
5 Terraform and validation A known-bad case is refused and a known-good one passes offline, in the built image, for all three providers
6 Azure and AWS: bundles, Skills, templates, evaluation cases A dry run completes for each provider with no code change
7 Parity features: Change requests, lessons loop, overlay Skills, terraform-plan server, search_cloud_docs wired to the built servers Ported tests pass; terraform-plan passes its tests and refuses what it must; a Change request dry run leaves its Baseline unchanged
8 Jev: port the reviews, and write the workflow export, ADRs, README and user guide Tests pass with and without a key; kab_import_workflow.py validates without --apply
9 Deploy to dev, run evaluation with real models, tune timeouts. Needs your approval for cluster changes. All 8 Google cases compare well with the source; 3 cases pass on each of Azure and AWS; one full conversation runs to a Solution guide; the parity checklist is closed

Progress, 2026-10-30:

  • Step 1 done. The skeleton passes check_agents.py. It holds one stub Specialist, and every request returns failed until step 3.

  • Step 2 done for the code. The core and provider loader are ported, with 80 tests passing and the provider grep empty on the core modules. Pillars and code files come from ProviderContract, which a bundle supplies. tests/core/test_scope_spec.py and two tests in test_work_breakdown.py drive the source's engine, so they return in step 3. sa-server mapping in providers/gcp.yaml is step 4: it still lists Google's URLs. Azure and AWS bundles are step 6.

  • Step 3 done. The router, nodes, Ledger, intake, change requests, lessons recording and Jev's reviews are ported, and turn in app/business_logic.py serves both routes. 257 tests pass, 251 of them the source's own or ported from it. tests/test_turns.py runs a whole Execution through the hooks with a fresh engine for every request, including a Revision. A last complete workflow step ends the Execution. Seven source test files wait for toolkit and the Terraform tools (tests/conftest.py). assemble uses NoDecider and no tools yet, and Intake isn't wired into a conversation's first message: step 4 and step 8.

  • Step 4 done for Google Cloud. All 16 roles' prompts are in, with 14 sub-agents declared. load_skill and load_skill_resource give each role its own Skills only. app/mcp_tools.py gives each role its bundle's tools on sa-gcp-cloud, and a failed call is a discovery gap, as in the source. before_model strips the root model's tools down to run_turn, confirmed on the runtime. mcp: declares sa-gcp-cloud, so the agent won't start until that server is registered at the gateway.

  • Step 5 done, except the image. The Terraform tools and the schema review are ported. fmt and validate ran offline against a real mirror on all three providers, and bad code was refused. terraform_plan isn't offered until the terraform-plan server exists (step 7). The Dockerfile hasn't been built.

  • Step 6 partly done. Azure and AWS bundles load with all roles. Written in Google's shape: 5 Azure and 6 AWS pillar Skills (3 to 6 KB each, against Google's 7 to 17 KB), a solution architecture Skill and guide template for each (derived from Google's, Apache-2.0, marked in their metadata), and IaC patterns (7 Azure, 8 AWS, against Google's 18), each passing terraform validate offline. A conversation keeps its cloud after the first message. The aws_get_documents bug is fixed. Still to do: vendoring (Microsoft's references need reading; AWS's service Skills turned out to be mostly routers to Skills we don't hold, so they may not be worth it), Azure and AWS evaluation cases, module Skills (terraform_module is refused on both for now), and a review of the Skills by someone who knows each cloud. Whole Executions on Azure and AWS have run through a fake Specialist only as far as the first Approval.

ADRs to write: LangGraph inside the agent and VolumeSaver; pattern and two ways in; Approvals as review points and messages; live conversation; Jev as an option; Skills for sub-agents; Terraform in the image; terraform plan through a server; free-text search exception; provider-owned contracts; one model for every role.

7. Risks

Risk Impact Mitigation
Spike (a), (b) or (h) fails The runner, guidance or conversation changes shape Fallbacks in 4.3 and 4.4; a separate agent for conversation if (h) fails; spikes run before any porting
Azure and AWS have no proven designs behind them Weaker first results than Google, which went through ten iterations New evaluation cases; describe them as first-version until they pass
Free-text search could carry client detail to a vendor, accepted by the owner Queries aren't scrubbed, so a customer's name, project or anything else a model writes can reach the vendor Prompt guidance, the query log and the per-deployment switch in 4.4. Check customer contracts before enabling it for a deployment
terraform-plan handles a customer credential A credential or plan output could leak, or a plan could show secrets Read-only credential per project, no apply path, every reply scrubbed, allow_cloud_read required
Parity makes v1 much larger Longer before anything can be used Run one Google Cloud case with real models after step 5, with your approval for the cluster change, so problems show early
No customer-ready conversation screen Live conversation stays a developer route Spike (j); raise with the platform team
One model authors and checks Shared blind spots Code checks stay authoritative; the Requirements chain is enforced in code
A Phase overruns a turn or step Lost time Write-once records make the rerun resume
Jev sends client data outside the gateways Compliance Off by default; approval per deployment
Client data on a shared volume Retention not agreed Same open question as Discovery ADR 0010
Vault Service isn't deployed No customer credential reaches any server, so cloud reads and terraform plan can't run Needs the platform deployment's owner. Everything else works without it, with cloud reads recorded as discovery gaps
The agent's servers have never met a real cloud with a credential, and AWS and Google Cloud have no sandbox Tools may fail at first use A sandbox per provider (D7 in the server plan); spike (e)
Two sets of hyperscaler servers now exist Fixes must be made twice, and customers may need to grant access to both Shared _common/ limits this; revisit if the Discovery agents adopt the same tools
Vendored Google Skills are third-party Licence obligations Keep LICENSE, SHA256SUMS, VENDOR.md; don't edit them

8. Needed outside this folder (not done here)

Until these change, the agent can be checked, and its workflow validated, by passing explicit paths.

File Change Effect until then
scripts/render_k8s.py Add outcome-pattern to PATTERNS Blocks deployment through the normal route
deploy/crucible-kab-dev.yaml Add the agent's entry Blocks deployment
scripts/render_k8s.py and deploy/crucible-kab-dev.yaml Find and list the servers in this folder; the script looks only in the root mcp-servers/ Blocks deploying this folder's servers
mcp-servers/README.md Note that this folder's servers are an accepted exception to the fixed-list rule Rule and servers disagree
docs/customer-cloud-access.md The roles these servers need, such as Google's roles/mcp.toolUser, monitoring reads and AWS cloudwatch Customers grant too little
.agents/CONTEXT.md, docs/hyperscaler-mcp-servers-plan-2026-10-30.md Correct the stale facts in 4.12 Docs mislead
KAB dev cluster Deploy the Vault Service. Platform owner, not this repository Blocks every stored credential
.gitea/workflows/check.yml Add outcome-pattern to the three find commands, a dry-run step and these servers' tests CI doesn't check this agent
CLAUDE.md, README.md, .agents/CONTEXT.md Describe the third pattern Docs say there are two