All checks were successful
check agents / check (push) Successful in 10s
ALZ-F01 Discovery as a sub-agent orchestrator (sub-agents/) and as eight individual declarative agents (individual-agents/), with their KAB workflow exports. Shared check, render and import scripts, the alz-discovery-sample MCP server, the crucible-kab-dev deploy profile, deployment docs, platform issue reports and the source material. Agent steps time out at 20 minutes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
122 lines
7.2 KiB
Markdown
122 lines
7.2 KiB
Markdown
# Deploying cp1-agents with kustomize
|
|
|
|
For the platform operator who deploys the CP1 agents and their MCP server to the KAB dev cluster `crucible-kab-dev-aks`. You deploy by hand with kustomize. Kyndryl's agent CI can deploy the same folders unchanged, because each `kubernetes/` folder keeps the CI's placeholders.
|
|
|
|
Run every command from the repository root. The Feature folder names contain spaces and a colon, so quote them.
|
|
|
|
## What gets deployed
|
|
|
|
| Component | Pattern | Folder | Namespace | Port |
|
|
|---|---|---|---|---|
|
|
| `cp1-alz-discovery-orchestrator-agent` | Sub-agents | `sub-agents/ALZ-F01: Discovery Workflow Agent/` | `agents` | 8080 |
|
|
| `cp1-discovery-planner-agent` … `cp1-discovery-critic-agent` (8 agents) | Individual agents | `individual-agents/ALZ-F01: Discovery Workflow Agent/` | `agents` | 8080 |
|
|
| `alz-discovery-sample` (sample MCP server, Meridian estate; both patterns use it) | — | `mcp-servers/alz-discovery-sample/` | `mcp-servers` | 8110 |
|
|
|
|
Images are `cruciblecoredevacr.azurecr.io/kab/cp1/<component>:<version>`. An agent's version comes from its `agent.yaml`; the MCP server's comes from `mcp-servers/alz-discovery-sample.server.yaml`.
|
|
|
|
The target, registry and gateway addresses are in [`deploy/crucible-kab-dev.yaml`](../deploy/crucible-kab-dev.yaml). `scripts/render_k8s.py` finds each agent by name under `sub-agents/` or `individual-agents/`.
|
|
|
|
The eight individual agents keep the label `repo: cp1-agents-individual`. The label is part of each Deployment's selector, which can't be changed in place, and they were first deployed from that repository.
|
|
|
|
Keys are never in a file. Each component reads its keys from a Kubernetes Secret that you create by hand:
|
|
|
|
| Secret | Namespace | Key | Who creates it |
|
|
|---|---|---|---|
|
|
| `agent-builder-secrets` | `agents` | `litellm-master-key` | The platform; it already exists |
|
|
| `<agent>-secrets` (one per agent) | `agents` | `MCP_AGENT_API_KEY` | You, after KAB issues the agent's key |
|
|
| `alz-discovery-sample-secrets` | `mcp-servers` | `MCP_REGISTRATION_API_KEY` | You, with a new random key |
|
|
|
|
## Before you start
|
|
|
|
- **kubectl context:** `crucible-kab-dev-aks`. On this workstation it's in the Windows kubeconfig: `export KUBECONFIG=/mnt/c/Users/<you>/.kube/config`.
|
|
- **Registry access:** you can push to `cruciblecoredevacr` (`az acr login -n cruciblecoredevacr`).
|
|
- **Models:** the cluster's LiteLLM must serve `gpt-5.6-sol` and `claude-sonnet-5-5` (used by the critic).
|
|
- **MCP server registration works.** As of 2026-10-29 it fails with a 500 in the MCP gateway; see [platform-issues/mcp-gateway-default-tenant](platform-issues/mcp-gateway-default-tenant/report-2026-10-29.md).
|
|
- **The checks pass:**
|
|
|
|
```sh
|
|
find sub-agents individual-agents -mindepth 3 -maxdepth 3 -name agent.yaml -print0 | xargs -0 -n1 dirname -z \
|
|
| xargs -0 python3 scripts/check_agents.py --runtime-image cruciblecoredevacr.azurecr.io/kab/agent-runtime-adk:0.35.7
|
|
(cd "sub-agents/ALZ-F01: Discovery Workflow Agent" && python3 scripts/dry_run.py cp1-alz-discovery-orchestrator-agent "$(mktemp -d)")
|
|
```
|
|
|
|
## 1. Build and push the images
|
|
|
|
```sh
|
|
az acr login --name cruciblecoredevacr
|
|
R=cruciblecoredevacr.azurecr.io/kab/cp1
|
|
find sub-agents individual-agents -mindepth 3 -maxdepth 3 -name agent.yaml -print0 | while IFS= read -r -d '' y; do
|
|
d=$(dirname "$y"); a=$(basename "$d"); v=$(sed -n 's/^version: "\{0,1\}\([^"#]*\)"\{0,1\}.*/\1/p' "$y" | tr -d ' ')
|
|
podman build -t "$R/$a:$v" "$d" && podman push "$R/$a:$v"
|
|
done
|
|
podman build -t $R/alz-discovery-sample:0.1.0 mcp-servers/alz-discovery-sample && podman push $R/alz-discovery-sample:0.1.0
|
|
```
|
|
|
|
Every agent's Dockerfile builds on `cruciblecoredevacr.azurecr.io/kab/agent-runtime-adk:0.35.7`. A released version is never pushed twice. For a new build, bump `version` in `agent.yaml` and add an entry to `RELEASE_NOTES.md`.
|
|
|
|
## 2. Render the manifests
|
|
|
|
```sh
|
|
python3 scripts/render_k8s.py deploy/crucible-kab-dev.yaml # every component
|
|
python3 scripts/render_k8s.py deploy/crucible-kab-dev.yaml <component> # or just one
|
|
```
|
|
|
|
This writes `build/crucible-kab-dev/<component>/` (git-ignored) with every placeholder filled. It leaves out two files:
|
|
- `secretnames.yaml`, so a deploy can't empty your Secrets;
|
|
- `namespace.yaml`, so the shared namespaces are never relabelled or deleted.
|
|
|
|
## 3. Deploy the MCP server, then register it
|
|
|
|
```sh
|
|
read -rs REG_KEY < <(openssl rand -hex 32; echo)
|
|
kubectl --context crucible-kab-dev-aks -n mcp-servers create secret generic alz-discovery-sample-secrets \
|
|
--from-literal=MCP_REGISTRATION_API_KEY="$REG_KEY"
|
|
kubectl --context crucible-kab-dev-aks diff -k build/crucible-kab-dev/alz-discovery-sample
|
|
kubectl --context crucible-kab-dev-aks apply -k build/crucible-kab-dev/alz-discovery-sample
|
|
kubectl --context crucible-kab-dev-aks -n mcp-servers rollout status deploy/alz-discovery-sample
|
|
```
|
|
|
|
Then register the server in KAB:
|
|
1. Use the address `http://alz-discovery-sample.mcp-servers.svc.cluster.local:8110` and the key `$REG_KEY`.
|
|
2. Register it under each of the nine logical names the agents use: `alz-scope`, `alz-estate`, `alz-cloud`, `alz-cmdb`, `alz-inventory`, `alz-schema`, `alz-dependency`, `alz-network` and `alz-evidence`.
|
|
3. Run `unset REG_KEY`.
|
|
|
|
## 4. Register each agent, then deploy it
|
|
|
|
In KAB, create the agent at `http://<agent>.agents.svc.cluster.local:8080`. KAB issues the agent's MCP key, which you store without writing it anywhere:
|
|
|
|
```sh
|
|
read -rs MCP_KEY && kubectl --context crucible-kab-dev-aks -n agents create secret generic <agent>-secrets \
|
|
--from-literal=MCP_AGENT_API_KEY="$MCP_KEY"; unset MCP_KEY
|
|
kubectl --context crucible-kab-dev-aks diff -k build/crucible-kab-dev/<agent>
|
|
kubectl --context crucible-kab-dev-aks apply -k build/crucible-kab-dev/<agent>
|
|
kubectl --context crucible-kab-dev-aks -n agents rollout status deploy/<agent>
|
|
```
|
|
|
|
An agent without its key still starts and serves its agent card, but every request fails with `MCP_AGENT_API_KEY is not set`.
|
|
|
|
## 5. Test, then import the workflows
|
|
|
|
```sh
|
|
kubectl --context crucible-kab-dev-aks -n agents port-forward svc/<agent> 18080:8080 &
|
|
curl -s localhost:18080/.well-known/agent-card.json
|
|
python -m kyndryl_agent_runtime.eval --url http://localhost:18080 --cases "<Feature folder>/<agent>/eval/cases.yaml"
|
|
```
|
|
|
|
Each Feature has its own workflow export in `<Feature folder>/workflows/`. Import it with your KAB session token in `~/.kab_token`. Importing needs the Editor role.
|
|
|
|
```sh
|
|
python3 scripts/kab_import_workflow.py "sub-agents/ALZ-F01: Discovery Workflow Agent" --project-id <id> # validate
|
|
python3 scripts/kab_import_workflow.py "sub-agents/ALZ-F01: Discovery Workflow Agent" --project-id <id> --apply # import as a draft
|
|
python3 scripts/kab_import_workflow.py "individual-agents/ALZ-F01: Discovery Workflow Agent" --project-id <id> --apply
|
|
```
|
|
|
|
## Rolling back and removing
|
|
|
|
```sh
|
|
kubectl --context crucible-kab-dev-aks -n agents rollout undo deploy/<agent> # the previous version
|
|
kubectl --context crucible-kab-dev-aks delete -k build/crucible-kab-dev/<agent> # remove it entirely
|
|
```
|
|
|
|
A rendered build holds only the component's Deployment, Service and ConfigMap. **Never `kubectl delete -k` an agent's own `kubernetes/` folder.** That folder lists the shared namespace, and the delete would remove the namespace together with every other agent in it.
|