Files
cp1-shakedown/README.md
Jonathan Boniface 099f9a87e8 Runner: run a voyage remotely as the kab-automation identity
- shakedown_lib: runs F01 -> F02 -> F03 for one engagement on KAB with no
  personal token. Signs in as the kab-automation client (client credentials,
  provisioned by Stellwerk from crucible-platform-deploy) and renews the token
  itself.
- Each stage checks its workflow id against the registered sub-agent name,
  builds inputs from the case and upstream stages, starts the run, follows it
  and judges it with cp1-agents evaluate_case.py.
- Gates (plan 5b): in config/runner.yaml gate_projects only, continue on a
  critic pass with no failed step and a note saying the system decided.
  Never on a fail. Elsewhere, wait for a person and record who decided.
  gate_projects is empty for now.
- Ledger per voyage run, with --resume; nothing secret is written.
- Cymbal voyage ENG-CYMBAL-001; workflow registry; Dockerfile and an
  unapplied Job template; 15 tests against a fake KAB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 05:08:31 +00:00

3.7 KiB

Shakedown

Runs, rates and experiments on KAB workflows. The plan is docs/implementation-plan.md. What is built so far is the runner, which runs a voyage remotely with no person's token: F01 Discovery → F02 Assessment → F03 Strategy for one engagement.

The runner

export KAB_CLIENT_SECRET=...      # the kab-automation client's secret; never in a file
python3 -m shakedown_lib token-check
python3 -m shakedown_lib run capabilities/cp1-agentic-landing-zone/voyages/ENG-CYMBAL-001.yaml --dry-run
python3 -m shakedown_lib run capabilities/cp1-agentic-landing-zone/voyages/ENG-CYMBAL-001.yaml
python3 -m shakedown_lib run <voyage> --resume ledger/ENG-CYMBAL-001/<started>   # pick up where it stopped
python3 -m unittest discover -s tests -t .                                      # tests, against a fake KAB

Identity. The runner signs in as the confidential client kab-automation of realm kab, using client credentials. Its tokens carry the automation user's email, which KAB makes an Editor. The Stellwerk orchestrator provisions the client and its secret: crucible-platform-deploy, kab-cluster/dev component 85. The secret is held in Key Vault crucible-kab-dev-kv as kab-automation-client-secret. A token is renewed a minute before it expires, so a run that lasts hours needs nobody.

What one stage does:

  1. Check the workflow. The stage's workflow id must still name its registered workflow (workflows/*.yaml, sub-agent pattern). Ids change on every re-import.
  2. Build the inputs. They come from the case's context file and the voyage. Upstream execution ids, approval references and package artifact ids (<execution id>_step_<n>) come from earlier stages. Each context adapter delivers the file differently:
    • upload (F01): uploaded as the run's file;
    • document (F03): sent as text in one run input;
    • none (F02): inputs come from the voyage.
  3. Check and start. Before starting, it checks the required inputs, the engagement id (one per chain) and the customer block, using cp1-agents' own helpers.
  4. Follow the run. At a review point:
    • in a project listed in config/runner.yaml gate_projects, the runner continues only when the critic's verdict is pass and no step has failed. Its note says the system decided, not a person. Otherwise the pause is left as it is and the voyage stops for a fix;
    • in any other project, it waits for a person to decide in KAB and records who did.
  5. Finish. It pulls the outputs, finds the package and judges the run against its case with cp1-agents scripts/evaluate_case.py. A failed run, a missing package or a failed critical check stops the voyage.

Ledger. Each voyage run gets its own folder, ledger/<voyage>/<started>/ (or $SHAKEDOWN_LEDGER). It holds:

  • state.json;
  • for each stage: its inputs (uploads by name only), the context as sent, the status, the steps and the evaluation.

No token or secret is written there.

Exit status:

Code Meaning
0 Done
2 Configuration error, or KAB refused
3 Stopped for a fix
4 Nobody decided a review point in time
5 A stage ran too long

cp1-agents is read from a clone beside this repository, or from $CP1_AGENTS_DIR, and its commit is recorded with every run. Nothing in cp1-agents is changed.

On the cluster: deploy/Dockerfile and deploy/runner-job.yaml, a Job template with placeholders. Nothing is applied without approval.

Not built yet

These come from the plan:

  • Bosun (the deep agent);
  • fixes on agentic/* branches;
  • deploying from a branch;
  • experiments and variations;
  • the portal views;
  • the nightly cycle.