# Shakedown Runs, rates and experiments on KAB workflows. The plan is [docs/implementation-plan.md](docs/implementation-plan.md). What is built so far is the **runner**, which runs a voyage remotely with no person's token: F01 Discovery → F02 Assessment → F03 Strategy for one engagement. ## The runner ```sh export KAB_CLIENT_SECRET=... # the kab-automation client's secret; never in a file python3 -m shakedown_lib token-check python3 -m shakedown_lib run capabilities/cp1-agentic-landing-zone/voyages/ENG-CYMBAL-001.yaml --dry-run python3 -m shakedown_lib run capabilities/cp1-agentic-landing-zone/voyages/ENG-CYMBAL-001.yaml python3 -m shakedown_lib run --resume ledger/ENG-CYMBAL-001/ # pick up where it stopped python3 -m unittest discover -s tests -t . # tests, against a fake KAB ``` **Identity.** The runner signs in as the confidential client `kab-automation` of realm `kab`, using client credentials. Its tokens carry the automation user's email, which KAB makes an Editor. The Stellwerk orchestrator provisions the client and its secret: crucible-platform-deploy, `kab-cluster/dev` component 85. The secret is held in Key Vault `crucible-kab-dev-kv` as `kab-automation-client-secret`. A token is renewed a minute before it expires, so a run that lasts hours needs nobody. **What one stage does:** 1. **Check the workflow.** The stage's workflow id must still name its registered workflow (`workflows/*.yaml`, sub-agent pattern). Ids change on every re-import. 2. **Build the inputs.** They come from the case's context file and the voyage. Upstream execution ids, approval references and package artifact ids (`_step_`) come from earlier stages. Each context adapter delivers the file differently: - `upload` (F01): uploaded as the run's file; - `document` (F03): sent as text in one run input; - `none` (F02): inputs come from the voyage. 3. **Check and start.** Before starting, it checks the required inputs, the engagement id (one per chain) and the customer block, using cp1-agents' own helpers. 4. **Follow the run.** At a review point: - in a project listed in `config/runner.yaml` `gate_projects`, the runner continues **only** when the critic's verdict is `pass` and no step has failed. Its note says the system decided, not a person. Otherwise the pause is left as it is and the voyage stops for a fix; - in any other project, it waits for a person to decide in KAB and records who did. 5. **Finish.** It pulls the outputs, finds the package and judges the run against its case with cp1-agents `scripts/evaluate_case.py`. A failed run, a missing package or a failed critical check stops the voyage. **Ledger.** Each voyage run gets its own folder, `ledger///` (or `$SHAKEDOWN_LEDGER`). It holds: - `state.json`; - for each stage: its inputs (uploads by name only), the context as sent, the status, the steps and the evaluation. No token or secret is written there. **Exit status:** | Code | Meaning | |---|---| | 0 | Done | | 2 | Configuration error, or KAB refused | | 3 | Stopped for a fix | | 4 | Nobody decided a review point in time | | 5 | A stage ran too long | **cp1-agents** is read from a clone beside this repository, or from `$CP1_AGENTS_DIR`, and its commit is recorded with every run. Nothing in cp1-agents is changed. **On the cluster:** `deploy/Dockerfile` and `deploy/runner-job.yaml`, a Job template with placeholders. Nothing is applied without approval. ## Not built yet These come from the plan: - Bosun (the deep agent); - fixes on `agentic/*` branches; - deploying from a branch; - experiments and variations; - the portal views; - the nightly cycle.