feat: working TOM agent

This commit is contained in:
2026-10-21 19:47:12 +01:00
commit efe543e6e2
83 changed files with 11779 additions and 0 deletions

10
.agents/plans/README.md Normal file
View File

@@ -0,0 +1,10 @@
# Plans
One directory per iteration. `../../IMPLEMENTATION_PLAN.md` is the plan across this repository and the engine's; its steps T1 onward are built here.
| # | Directory | Status | What it is |
| --- | --- | --- | --- |
| 1 | `iter-01-first-slice/` | the Workflow, its checks and views built on 2026-10-13 from a written operating spec; the digest and evaluation remain | Six Deliverables, from an operating spec to a priced, validated operating model |
| 2 | `iter-02-platform-deployment/` | steps 1 to 3 built 2026-10-20; step 4 in progress | The agent as a container on the local KAB platform, then behind the Orchestration engine |
Each iteration has a `MAP.md` (intent, each step's status, sources of truth) and `steps/step-N.md`, written when a step is finished. A step with no file is not done.

View File

@@ -0,0 +1,31 @@
# Plan Map
**Iteration:** 1, the first slice
**Intent:** From an operating spec to a priced, validated operating model, in six Deliverables.
**Status:** T1 to T14 built; both evaluation cases passed with real models on 2026-10-14
## Sources of truth
- `.scratch/target-operating-model/spec.md`: the formats, the rules and the arithmetic.
- `docs/adr/`: 0001 (the graph, and findings), 0002 (the operating spec), 0003 (cost).
- `IMPLEMENTATION_PLAN.md`, section 10: the steps.
- `../orchestration_engine/execution_engine/workflows/workflow.py`: what a Workflow supplies.
## Status
| Step | What | Status |
| --- | --- | --- |
| T1 | The repository on the engine | built (`steps/step-1.md`) |
| T2 | The operating catalogue and scoping Skill | built (`steps/step-1.md`) |
| T3 | The architecture adapter and the digest command | built (`steps/step-2.md`) |
| T4 | The model-drafted digest for other documents | built as instructions to the scope analyst (`steps/step-2.md`); not yet run by a model |
| T5 | The capability catalogue Skill and its sources register | built (`steps/step-1.md`); references unverified |
| T6 | Operational requirements and capabilities, and their checks | built (`steps/step-1.md`) |
| T7 | The graph: refusals and rendered views | built (`steps/step-1.md`) |
| T8 | Findings | built (`steps/step-1.md`) |
| T9 | Role files, the provider bundle, the Workflow | built (`steps/step-1.md`) |
| T10 | Validation in code: verdict rules and scenario walk-throughs | built (`steps/step-1.md`) |
| T11 | The guide | built (`steps/step-2.md`) |
| T12 | The rate card Skill's format and checks | built (`steps/step-1.md`) |
| T13 | Staffing and cost in code | built (`steps/step-1.md`) |
| T14 | Evaluation with real models | built; two cases passed (`steps/step-2.md`) |

View File

@@ -0,0 +1,44 @@
# Steps T1, T2, T5 to T13: the Workflow, from a written operating spec
Built 2026-10-13, on the Execution engine as it stood that day (not yet committed there).
## What was built
- **The Workflow** (`app/operating_model/workflow.py`): five Phases, six Deliverables, each with its Specialist, sections and data file; the validation policy; the views; the account of a Change request; the card.
- **The operating spec's catalogue** (`skills/operating-scoping/`): 32 fields, read by the engine's spec reader. The provider bundle supplies the Cloud provider and the currencies.
- **The capability catalogue** (`skills/operating-capabilities/`): 17 entries in five domains, each citing frameworks from a register of nine. `catalogue.py` refuses an entry that cites nothing or cites an unregistered framework, and proposes the entries that apply to a spec.
- **The chain** (`chain.py`): the checks on operational requirements and on capabilities.
- **The graph** (`graph.py`): its shape for the engine's graph tools, its refusals, findings, and the rendered responsibility matrix, organisation chart, and staffing and cost.
- **Staffing and cost** (`staffing.py`): the rate card's loader and checks, and the arithmetic.
- **Scenarios** (`scenarios.py`): five kinds, generated and traced by code, their verdicts written into `verdicts.json`.
- **The agent's folder**: eight role files and a shared protocol, `providers/gcp.yaml`, the `operating-chain` Skill, `app/main.py`.
- **106 tests**, among them seven whole Executions on the scripted runtime.
## Differed from the plan and the spec
- **The spec's files keep the engine's names,** `scope.yaml` and `scope.json`. The plan said `operating-spec.yaml`. The engine's spec support does not yet let a Workflow name them.
- **The inventory travels with the rest of the spec** to every Specialist, not as a separate `solution.json`. The engine's digest step (E6) is where that changes.
- **The graph holds no list of Capabilities.** They are the capability model's, named by id, so a failed Check can reopen who does the work without reopening what the work is.
- **The currencies a spec may state are listed in the provider bundle,** so the engine refuses another at the start. A test keeps the list the same as the rate card's conversion rates.
- **A Role on business hours is as small as its work.** The spec's formula applied the minimum roster to every Role; the plan, which the owner approved, applies it only to hours wider than a working day. The plan was followed.
- **A Check that depends on a finding is recorded as `waits_on_finding`.** The validator gives it `not_verifiable`; code rewrites the verdict when the validation is submitted. Left as `not_verifiable`, the engine's router reopened the capability model for it, again and again: a finding is not a gap in the model.
- **A scenario's verdicts are written into `verdicts.json` by code,** as Checks whose ids begin `SC-`, and the walk-through as a whole is the file's `dry_run`. A failed scenario is therefore a failed Check to the router, which reopens the graph.
- **The matrix shows both parts** when one Role is accountable and responsible for a Capability (`A/R`).
## Not built
- **The digest** (T3, T4). An Execution starts from an operating spec that is already written. Nothing reads the solution architecture agent's files yet.
- **A guide template** (T11). The packager has its sections and its instructions.
- **Evaluation** (T14). Nothing has run with a real model: no role file, Skill or catalogue entry has been read by one.
- An interactive Execution and a Change request are not tested here; the engine's own tests cover both on other Workflows.
- The capability catalogue's references are drafted from general knowledge and marked `unverified`. Nobody has checked them against the frameworks.
- The rate card's figures are placeholders.
## What it found in the engine
- The engine was not typed as a package (`py.typed` added there), so this repository could not be type-checked against it.
- An editable install of the engine is invisible to the type checker unless made with `editable_mode=compat`.
## Verified
`pytest -q`: 106 passed. `ruff check .`, `ruff format --check .`, `mypy .` (20 files): clean. The engine's suite still passes (521). `git status` in the solution architecture agent shows the same nine modified files as before any of this work began.

View File

@@ -0,0 +1,38 @@
# Steps T3, T4, T11 and T14: the digest, the guide, and the first runs with real models
Built 2026-10-13 and 14, with the engine's digest step (its `steps/step-6.md`).
## What was built
- **The architecture adapter** (`app/operating_model/digest.py`, `python -m app.operating_model.digest <folder>`). An Item is a component of the architecture, with the Elements that realise it; a Condition is what its requirements and recommendations say can only be shown in operation (`outside_code`); a Check that did not pass is an inherited risk. Only what the customer stated in the architecture's scope is carried across. The provider bundle's `operating.item_kinds` says what kind of thing each product is. Tested against a real completed landing-zone Execution, copied into `tests/fixtures/architecture/`.
- **Documents code cannot read** go to the scope analyst, who records what it takes from them with the passage quoted (T4, as instructions; no model has run it).
- **The guide** (`app/operating_model/guide.py`, `skills/operating-guide/`). The packager writes the summary; code writes every other section and the key figures from the data.
- **The evaluation harness** (`eval/`): two cases, structural scoring, tokens per role.
## Differed from the plan
- An Item is a component, not an Element. The real architecture had 60 Elements in 9 components; an organisation operates the components.
- The architecture's 104 Checks on infrastructure settings are not Conditions. Keeping them true is configuration management. Its 12 statements of what can only be shown in operation are.
## The runs with real models, 2026-10-14
| Case | Result | Minutes | Revisions | Capabilities | Roles | Checks | Scenarios | People | Cost a month |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 01, written spec (4 Items) | pass | 25.8 | operating model ×2, capability model ×1 | 17 | 4 | 8 | 13 of 13 | 5.2 | 25,355 GBP |
| 02, digested architecture (9 Items) | pass | 16.8 | operating model ×1 | 17 | 5 | 36 | 31 of 31 | 11.6 | 67,298 GBP |
Costs are from the placeholder rate card. An earlier run of case 01 passed every structural check in 30.6 minutes, over its limit then of 30.
What they showed:
- **Most time goes on the architect placing a Check on the wrong Capability**, such as a restore-time Check on service-level reporting. Case 01 revised the same way in both runs. Not yet acted on.
- **The router's reasons still speak of code.** A Check that could not be verified reopens the capability model with "could not be verified from the code", and a Check that failed twice is said to be possibly "not implementable". These are the engine's words from the architecture's subject.
- **The capability modeller takes all 17 catalogue entries** whatever the size of the solution, as its instructions say.
- **The architect is most of the cost:** about 380,000 to 490,000 of each case's 780,000 to 940,000 prompt tokens.
- One request was retried after Vertex AI's tokens-per-minute quota was reached; it did not affect the result.
Nobody has yet read the outputs for quality: these checks are structural only.
## Verified
The engine's 529 tests and this repository's 140 pass, with `ruff` and `mypy` clean in both.

View File

@@ -0,0 +1,67 @@
# Socio-technical and organisational frameworks
Added 2026-10-14, at the owner's request.
## What was added
- **Six frameworks in the register** (`skills/operating-capabilities/assets/sources.yaml`): Team Topologies; platform as a product, cited from the CNCF Platforms White Paper; the SPACE framework; and the DevEx framework. The DORA delivery metrics were already there. Like every other entry, their references are unverified.
- **Four Capabilities in the catalogue:** platform product management, self-service and golden paths, and enablement, proposed for a platform foundation; and delivery and developer experience measurement, proposed for a platform foundation or a workload. The catalogue can now propose an entry by the kind of engagement.
- **Measurement at three layers.** The measurement Capability cites DORA, SPACE and DevEx together, each for a different layer: delivery speed and stability as the outcome, sustainable productivity as the guardrail, friction as the leading signal. The Skill says no one layer stands in for another.
- **Teams and interactions in the graph,** both optional. A Team is of one of Team Topologies' four kinds, in one Party; a Role may be in a Team of its own Party. An Interaction between two Teams is in one of the three modes, and a collaboration must say when it ends. The organisation chart draws teams inside their Parties and interactions as labelled edges; the guide lists both.
- **Two new terms** in `CONTEXT.md`: Team and Interaction.
- **The architect's instructions** say how to use teams and interaction modes, and that a Check goes on the Capability that does the work, not the one that reports it. The second was the cause of most Revisions in the runs of 2026-10-14.
## Not done
- No run with real models since. The four new Capabilities add to what the capability modeller takes, which was already all 17 entries for a platform foundation; it will now be 21.
- Teams are not used by staffing, cost or the scenario walk-throughs.
## Verified
159 tests pass; `ruff` and `mypy` clean.
## The run with real models after this change, 2026-10-14
| Case | Minutes | Revisions | Capabilities | Teams | Interactions | Checks | Scenarios | People | Cost a month | Model cost |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 01, written spec | 20.9 (was 25.8) | operating model ×1 (was 3 in all) | 21 | 2 | 2 | 13 | 13 of 13 | 6.6 | 32,234 GBP | $1.87 (was $2.27) |
| 02, digested architecture | 16.6 (was 16.8) | operating model ×1 (was 1) | 21 | 3 | 2 | 31 | 35 of 35 | 12.5 | 77,550 GBP | $1.61 (was $1.55) |
Both passed every check. The one Revision in each was still a Check whose hours or Capability did not fit, but the capability model was not reopened in either. Both models used teams: platform and stream-aligned teams, an X-as-a-Service interaction in each, and in case 01 a collaboration that ends three months after go-live, in case 02 a facilitating interaction that ends with the build. Case 02 named six Parties.
# Business outcomes
Added 2026-10-14, at the owner's request: every measure is tied back to an outcome the business stated.
- **`intent.business_outcomes`** in the operating spec: each with an id, the customer's words, and a target where given. The digest carries the architecture's stated success criteria across as business outcomes.
- **Measures** in the graph: what is measured, its layer (outcome, guardrail, leading, service level or cost), its framework, its target, the outcome it shows and the Capability that takes it. Refused: a stated outcome shown by no Measure, a Measure for an outcome that was not stated, a Measure taken by a Capability nobody is responsible for, a framework not in the register, and, where the model measures how work flows, any of the three developer layers left unmeasured.
- **An Operational requirement may come from a business outcome.** It need not: the Measures are what show each outcome is served.
- **The guide's second section is Business outcomes,** written by code: each outcome with the Measures that show it and the Role answerable for each.
- **Two new terms** in `CONTEXT.md`: Business outcome and Measure. The benchmark spec now states three outcomes.
Not yet run with real models. 173 tests pass; `ruff` and `mypy` clean.
## The run with business outcomes, 2026-10-14
- **Case 01 passed** in 29.2 minutes, for $2.73. It tied four Measures to the spec's three business outcomes: deployment lead time (DORA, outcome layer) to faster releases, time to restore service (DORA) to short outages, and satisfaction and wellbeing (SPACE, guardrail) and feedback loops and cognitive load (DevEx, leading) both to keeping engineers. Two Revisions of the operating model and one of the capability model.
- **Case 02 looped and was stopped** after about two and a half hours. The architecture states no business outcome, so every Measure was refused as serving an outcome "none were stated", while the measurement Capability's rule asked for Measures at three layers. The two rules could not both be met. The architect asked a question five times in a pre-approved run, each answered automatically, each run taking 10 to 28 minutes, and the last model request hung. About 173 Pro requests; an estimated $7 to $10.
Fixed after:
- `intent.business_outcomes` is essential. With someone present it is asked for; a pre-approved start must state them or leave them to the agent with `decide`. Case 02 now does the latter.
- The three-layer rule applies only when there is an outcome to tie Measures to.
- The evaluation harness stops a case at one and a half times its time limit and records it as stopped.
Not fixed: the engine lets a pre-approved Specialist end without submitting again and again, each time answered automatically, until the Orchestrator's turn limit. A guard there would have stopped case 02 within minutes.
## Case 02 rerun, and findings kept to the solution, 2026-10-14
Case 02 passed with its business outcomes left to the agent: 31.6 minutes, $2.71, two Revisions. The agent assumed three outcomes from the request (the landing zone ready within 10 weeks, continuous DORA and PCI DSS compliance, high availability with low-latency hybrid connectivity) and tied six Measures to them, all three developer layers on the first.
It also filed a gap in its own model as a finding: service continuity staffed only in business hours against a 4-hour recovery target. That drafted a change request to the architecture agent for a fault that was the operating model's. Fixed:
- **A finding names the Items of the solution it is about.** One with none is refused.
- **The validator judges each finding.** A Check waiting on a finding the operating model could meet itself (staffing, hours, a Role, a process, a tool) is given `fail`, which sends the model back to the architect. Only a Check left `not_verifiable` is recorded as waiting on its finding.
- **The architect is told** what a finding is and is not.
177 tests pass. Not yet run with real models.

View File

@@ -0,0 +1,31 @@
# Plan Map
**Iteration:** 2, on the platform
**Intent:** The operating model agent runs as a container beside the local KAB platform, is registered in its catalogue, and completes an Execution started by a platform workflow. The Orchestration engine (F1, F2, in `../orchestration_engine`) is then put in front of it.
**Status:** steps 1 to 3 built 2026-10-20; step 4 in progress
## Decided by the owner, 2026-10-20
- **E7, locally:** the image is built with a named build context, `--build-context engine=../orchestration_engine`, and installs the Execution engine from it. No file is copied by hand. CI will use a dependency on the engine's repository at a pinned commit.
- **Order:** this agent first (steps 1 to 4), then the Orchestration engine (step 5).
- **Models:** Gemini on Vertex AI, as in the evaluation, with the developer's application default credentials mounted read-only. The platform's LiteLLM is not used.
## Sources of truth
- `IMPLEMENTATION_PLAN.md`, sections 2 and 10 (E7, F1, F2).
- `../modernization-platform/packages/agent-builder-local-dev-kit/README.md`: the local platform, `agents-compose.yml` and `scripts/compose.sh agents`.
- `../modernization-platform/packages/agent-builder-temporal-workers/src/temporal/a2a_client.py`: how a platform workflow calls an agent.
## Known mismatch
A platform workflow step treats an A2A `input_required` as the agent runtime's action review (a list of tool calls to approve, at most `MAX_ACTION_REVIEWS` rounds). The engine's questions and Approvals are not that, so a platform workflow calls this agent pre-approved. Interactive Executions go through the Orchestration engine, which relays every prompt (F2). Step 4 confirms this rather than assuming it.
## Status
| Step | What | Done when | Status |
| --- | --- | --- | --- |
| 1 | The image (E7): `Dockerfile`, `.dockerignore`, the engine from the named build context | The image builds from a clean checkout of both repositories; `/health` and the agent card answer | built (`steps/step-1.md`) |
| 2 | A real-model run through the container | One pre-approved Execution of case 01 completes through the container's A2A endpoint | built (`steps/step-2.md`) |
| 3 | On the platform's network: an entry in the dev kit's `agents-compose.personal.yml`, registered in the catalogue | It is listed in the catalogue; the API container reaches `http://tom-analysis:8080/health` | built (`steps/step-3.md`) |
| 4 | A platform workflow with one step calling this agent | The workflow completes in Temporal with the Deliverables in its output | in progress: the engine's card declares KAB's input contract (`request`, `customer_case`, `scope`, `pre_approved`), chosen by the owner; waiting on the catalogue refresh and the workflow |
| 5 | The Orchestration engine in front (F1, F2 in `../orchestration_engine`) | An Execution of either agent runs through it, Approvals relayed | F2 built (`../../../orchestration_engine/.agents/plans/iter-02-orchestration-engine/steps/step-2.md`): this agent's questions relayed through it on the platform; F1 awaits the owner |

View File

@@ -0,0 +1,29 @@
# Step 1: the image (E7)
Built 2026-10-20.
## What was built
- **`Dockerfile`.** Two stages. The builder installs the Execution engine from a named build context, `engine`, copying only its `pyproject.toml` and `execution_engine/`. The runtime stage holds `app/`, `agents/`, `providers/` and `skills/` in `/app` and runs `python -m app.main` as a non-root user (10001) on port 8080. The agent is not installed: `ROOT` in `workflow.py` is found from `__file__`, so it must run from its own folder, as the reference agent does.
- **Defaults for a container:** `CHECKPOINTER=sqlite` with its files in the `/data` volume, and `ENV_FILE_EXPORT=false`.
- **`.dockerignore`:** keeps `.venv`, `.cache`, `.env`, `eval/results`, `deliverables`, tests and plans out of the image.
```bash
podman build --build-context engine=../orchestration_engine -t localhost/agents/tom-analysis:local .
```
## What differed from the plan
- **Credentials and the non-root user.** The application default credentials file is mode 600, so the image's user cannot read it under rootless podman. The container is run with `--userns=keep-id:uid=10001,gid=10001`, which maps the developer's user onto the image's. The file's mode is not changed.
- **Podman ignores `HEALTHCHECK`** in an OCI-format build. The compose entry declares its own (step 3).
## Verified
- Neither repository has a commit yet, so "a clean checkout" was taken as the files git would commit (`git ls-files --cached --others --exclude-standard`) of both, copied to an empty folder. The image built from that, and the Workflow loads in it with its five Phases.
- The container answers `/health` and both agent card paths; the card advertises `A2A_PUBLIC_URL`. From inside it, the credentials are readable and a Gemini request on Vertex AI answers.
- An A2A `message/send` with `pre_approved`, `customer_case` and `scope` in the message metadata reached the engine, which checked the spec. A Customer case that differs from the spec's is refused by name.
- 177 tests pass; `ruff` and `mypy` are clean. No Python changed.
## Not done
- CI's means, a dependency on the engine's repository at a pinned commit, needs the engine's repository to have a commit to pin.

View File

@@ -0,0 +1,20 @@
# Step 2: a real-model run through the container
Run 2026-10-20.
## What was run
Evaluation case 01's written spec (`eval/datasets/specs/landing-zone-shared.yaml`), sent to the container from step 1 as one A2A `message/send` with `configuration.blocking: false`, then polled with `tasks/get`. The message metadata carried `pre_approved: true`, `customer_case: bench-landing-zone-shared` and the spec as `scope`. Models were Gemini on Vertex AI with the developer's application default credentials, mounted read-only.
## Result
| Minutes | State | Capabilities | People | Scenarios |
| --- | --- | --- | --- | --- |
| 31.6 | completed | 20 | 5.1 | 13 of 13 pass |
The task's artifacts were all eleven files: the guide, `scope.yaml` and `scope.json`, operational requirements, capabilities, the Target operating model and its organisation chart, staffing, findings, verdicts and scenarios.
## What differed from the plan
- **The Customer case must be the spec's.** The first attempt named another slug and was refused at once: "The scope spec is for bench-landing-zone-shared, and this was started for smoke-01." A caller that sends a spec sends its Customer case with it.
- **Postgres.** The engine tries `localhost:5432`, logs a warning and falls back to SQLite in `/data`. Harmless here; a shared deployment sets `DATABASE_URL`.

View File

@@ -0,0 +1,28 @@
# Step 3: on the platform's network
Built 2026-10-20.
## What was built
In the dev kit (`../modernization-platform/packages/agent-builder-local-dev-kit`), two personal files, excluded from git in that clone's `.git/info/exclude`:
- **`agents-compose.build.personal.yml`:** `tom-analysis` built from `../../../tom_analysis` with `additional_contexts: {engine: ../../../orchestration_engine}`.
- **`agents-compose.personal.yml`:** the container `tom-analysis` on `agent-builder-network`, port `${TOM_ANALYSIS_PORT:-10020}:8080`, `A2A_PUBLIC_URL=http://tom-analysis:8080`, the credentials mounted read-only, named volumes for `/data` and `/app/deliverables`, and its own healthcheck.
```bash
./scripts/compose.sh agents build tom-analysis
./scripts/compose.sh agents up tom-analysis
```
The owner registered it in the catalogue from the admin page (Agents, Discover agent, `http://tom-analysis:8080`). It is listed as `target_operating_model_agent`.
## What differed from the plan
- **No pod.** podman-compose puts every container in one pod, and podman refuses a per-container user namespace (`keep-id`, step 1) inside a pod. The personal file sets `x-podman: in_pod: false`. Agents reach each other by name on the network, not through the pod.
- **The platform calls the address an agent was registered with,** not the A2A interface its card names (`agent_catalog_service.py` sets `endpoint.url` to the discovery URL). The engine served JSON-RPC only at `/a2a`, so every platform call would have been a 404. Chosen by the owner: the engine serves the same handler at `/` as well (`execution_engine/main.py`, with a test in `tests/toy/test_server.py`). The card still names `/a2a`. The platform's choice is worth raising with its team.
## Verified
- From the API and workers containers: `/health`, both card paths, and JSON-RPC at `/` and at `/a2a`.
- The catalogue record names the agent, its skill and its examples.
- The engine: 532 tests pass; `ruff` and `mypy` clean. This agent: 177 tests pass.

13
.dockerignore Normal file
View File

@@ -0,0 +1,13 @@
.git
.venv
.cache
.env
*.env
*.db
**/__pycache__
deliverables
lessons/candidates
eval/results
tests
.scratch
.agents

29
.gitignore vendored Normal file
View File

@@ -0,0 +1,29 @@
# Environment variables
.env
*.env
# Local SQLite databases
*.db
# Python bytecode and tool caches (pytest, mypy and ruff write to .cache/)
__pycache__/
*.pyc
*.pyo
.cache/
# Virtual environments
.venv/
venv/
# Everything generated for a Customer case: operating specs and operating models
# name people, teams and suppliers
deliverables/
# Evaluation run outputs
eval/results/
# Lesson candidates captured from Executions (may quote user feedback)
lessons/candidates/
*.log
.DS_Store

View File

@@ -0,0 +1,144 @@
# The target operating model agent
Status: needs-triage
The design for the first slice. Vocabulary is in `CONTEXT.md`. The decisions the owner has taken, the Phases and the order of work are in `IMPLEMENTATION_PLAN.md`; this spec says what the plan leaves loose: the formats, the rules and the arithmetic. The engine it runs on is specified in `../orchestration_engine/.scratch/engine/spec.md`.
## The first slice
Six Deliverables, enough to go from Sources to a priced, validated operating model.
| Phase | Deliverable | Specialist | Data file |
| --- | --- | --- | --- |
| scoping | `operating_scope` | `scope-analyst` | `operating-spec.yaml`, `scope.json`, `solution.json` |
| operational_requirements | `operational_requirements` | `operations-analyst` | `operational-requirements.json` |
| operating_model_design | `capability_model` | `capability-modeller` | `capabilities.json` |
| operating_model_design | `target_operating_model` | `operating-model-architect` | `target-operating-model.json`, `findings.json` |
| validation | `validation_results` | `validator` | `verdicts.json`, `scenarios.json` |
| packaging | `operating_model_guide` | `packager` | |
Each depends on the one above it. `capability_model` and `target_operating_model` share a Phase, so the customer approves the design once, from the graph.
## The Operating spec
Read against `skills/operating-scoping/assets/catalogue.yaml` by the engine's spec reader. Every field holds a value, `ask`, `decide` or `not_applicable`.
| Field | Type | Tier | Notes |
| --- | --- | --- | --- |
| `metadata.customer_case` | string | essential | |
| `engagement.kind` | enum | essential | `workload`, `platform_foundation`, `migration` |
| `intent.statement` | text | essential | The customer's words |
| `intent.requirements` | records `id`, `statement` | optional | Binds: each must reach an Operational requirement |
| `intent.exclusions` | list | optional | |
| `solution.items` | records `id`, `name`, `kind`, `product`, `environments`, `source_ref` | essential | At least one |
| `solution.conditions` | records `id`, `on`, `condition`, `source_ref` | important | `on` names an Item |
| `solution.build.duration_days` | integer | important | The transition plan is checked against it |
| `solution.build.customer_owned` | list | optional | |
| `solution.inherited_risks` | records `on`, `risk`, `source_ref` | optional | |
| `solution.cloud_provider` | enum | essential | Values from the provider bundle |
| `operations.support_hours` | enum | essential | `business`, `extended`, `24x7`. Binds |
| `operations.availability_hours` | enum | important | The hours the solution must be available |
| `operations.change_process` | text | important | |
| `operations.monitoring` | text | optional | |
| `sourcing.model` | enum | essential | `customer_operated`, `provider_operated`, `shared` |
| `sourcing.delivery_countries` | list | important | ISO country codes. Binds where Roles may sit |
| `organisation.existing_teams` | records `name`, `party`, `skills`, `hours` | important for `migration`, else optional | |
| `organisation.headcount_limit` | integer | optional | |
| `service_management.tooling` | text | important | |
| `service_levels.priority_1_response_minutes` | integer | important | |
| `service_levels.recovery_time_hours` | number | important | |
| `compliance.frameworks` | list | important | Selects Capabilities from the catalogue |
| `commercial.currency` | enum | essential | Values are the currencies the rate card can convert to |
| `commercial.monthly_run_budget` | number | optional | In `commercial.currency` |
| `transition.go_live_date` | date | optional | |
| `run.pre_approved` | boolean | optional | |
`solution.items` and `solution.conditions` are written to `solution.json` and reach only the Specialists that list it. Everything else is in `scope.json`, which every Specialist is given.
### Digesting an architecture Execution
A code adapter, `app/operating_model/digest/architecture.py`. No model.
| From | To |
| --- | --- |
| `target-architecture.json` Elements | `solution.items`: `id` is `S-n` in order, `source_ref` the Element's id, `name`, `product` and the component as `kind` |
| `target-architecture.json` Checks | `solution.conditions`: `id` is `K-n`, `on` the Item of the Check's Element, `source_ref` the Check's id |
| A Check on a Relationship | A condition `on` the Item of the Relationship's source |
| `scope.json` | Every field both catalogues share, as stated; `solution.cloud_provider` from `estate.cloud_provider` |
| `requirements.json` | Nothing directly. Kept as a Source so a Specialist can read what a condition's requirement said |
| `verdicts.json`, verdict `fail` or `not_verifiable` | `solution.inherited_risks` |
| `work-breakdown.json` | `solution.build.duration_days` from its computed schedule; activities owned by the customer to `customer_owned` |
Rules: every Element yields exactly one Item and every Check exactly one Condition, or the digest fails and names what it could not place. `target-architecture.json` is required; each other file that is absent leaves its fields as `ask`. A value digested has provenance `digested`, with the file and its hash.
## Data files
`operational-requirements.json`: a list.
```json
{"id": "OR-1", "statement": "A fault on the hub network is responded to at any hour.",
"kind": "service_level", "source": ["K-4", "operations.support_hours"]}
```
`source` names a stated requirement's id, a Condition's id or a spec field. Refused: an entry with no source; a source that does not exist; a stated requirement, or a spec field marked as binding with a stated value, that no entry names.
`capabilities.json`: a list.
```json
{"id": "C-1", "catalogue": "incident-management", "serves": ["OR-1"], "items": ["S-1", "S-4"],
"hours": "24x7", "effort_hours_per_month": 30, "basis": "Two priority 1 incidents a month at eight hours, plus triage."}
```
Refused: a `catalogue` id that is not in the catalogue, unless `catalogue` is `custom` with a `reason`; a Capability that serves nothing; an Item no Capability covers; an Operational requirement no Capability serves; `effort_hours_per_month` missing or not positive.
`target-operating-model.json`: Parties, Roles, Capabilities as above by reference, Responsibilities and Checks, as in the plan's section 7. A Role carries `country` and `level`.
`findings.json`: a list, as in the plan's section 7. May be empty.
`verdicts.json`: one entry per Check, `pass`, `fail` or `not_verifiable`, with evidence. `scenarios.json`: each Scenario and the path code traced for it.
## Staffing and cost
Computed when the Target operating model is submitted, from the graph and `skills/rate-card/assets/rates.yaml`.
For a Role *r* in country *c* at level *l*, covering hours *h*:
- **effort** = the sum of `effort_hours_per_month` of every Capability *r* is responsible for. Where several Roles are responsible for one Capability, its effort is divided equally among them unless the Responsibility states a `share`.
- **productive hours a month** = `productive_days[c]` × `hours_per_day[c]` ÷ 12.
- **people** = effort ÷ productive hours a month; and, where *h* is wider than business hours, never fewer than `cover[h].minimum_roster`. Shown to one decimal place.
- **cost a month**, only where the Role's Party is of a kind the card prices and is not the customer = people × `day_rate[c][l]` × `productive_days[c]` ÷ 12 × `cover[h].uplift` × `conversion.rates[commercial.currency]`.
A Role's hours *h* are the widest hours of any Capability it is responsible for.
Refused at submission: a Role with a country or level the card lacks. Refused at the start: a `commercial.currency` the card has no conversion rate for.
Written into the Deliverable by code: people and cost per Role; totals by Party, by country and by Capability; the total against `commercial.monthly_run_budget`; the card's version, and the conversion rate and its date.
### Digesting anything else
A model drafts `solution.items` and `solution.conditions` from a document. Each record carries `evidence`: the file and the passage it was read from, quoted. A record without evidence is dropped by code and reported. Its provenance is `drafted`, and the scoping Approval lists drafted records apart from digested ones.
## Validation
Structural verdicts, by code: one per refusal rule of the graph, restated as a Check that passes, so the guide shows what was verified and not only what was refused.
Scenarios, generated by code and traced by code:
| Scenario | Generated for | Passes when |
| --- | --- | --- |
| A fault out of hours | Each Item whose Capabilities must cover hours wider than `business` | A Capability covers the Item, a Role responsible for it works those hours, and the Capability has an accountable Role |
| A change | Each Item | A Capability from the catalogue's change domain covers it, with a responsible Role |
| A Condition falls due | Each Condition | A Check carries it, on a Capability with a responsible Role |
| Loss of a Party | Each Party that is not the customer | Every Capability it is accountable for names a Role in another Party as consulted or responsible |
| Over budget | When a budget is stated | The total is within it, or the Deliverable names the overrun |
The validator, a model, judges only what code cannot: whether each Check's condition is verifiable as written, and whether a Capability's stated effort is plausible for the Items it covers. A Check that depends on a Finding is `not_verifiable`.
## Not in the first slice
The current operating model, the service model, dimension advisors, process definitions, organisation design, tooling, the transition plan, Change requests, and the lessons loop.
## Open
- The first entries of the capability catalogue, and which frameworks are in the register.
- Whether effort shared between Roles defaults to an equal division, as written above, or must always be stated.

27
CLAUDE.md Normal file
View File

@@ -0,0 +1,27 @@
# CLAUDE.md
## Direction
This agent turns a description of a solution into a reviewed target operating model. It is the Execution engine (`../orchestration_engine`) and one Workflow, `app/operating_model/workflow.py`. Read `CONTEXT.md` and `docs/adr/` before designing or restructuring anything.
Three rules are the point of the design:
- **The solution is never changed here.** What operations needs and the solution lacks is a finding, returned to whoever owns the design (ADR-0001).
- **No agent writes a number of people or a cost.** They are computed from the rate card Skill, whose prices only the service provider sets (ADR-0003).
- **Behaviour lives in Skills.** The core names no Cloud provider and no framework. The capability catalogue is in our own words and cites the frameworks it draws on.
Engine code is not edited here. A fault in it is fixed in `../orchestration_engine`.
Plans and history live in `.agents/plans/`; `IMPLEMENTATION_PLAN.md` is the plan across both repositories. The design spec is `.scratch/target-operating-model/spec.md`.
## Checks
A step is done when all three pass:
```bash
.venv/bin/python -m pytest -q
.venv/bin/ruff check . && .venv/bin/ruff format --check .
.venv/bin/mypy .
```
Set up with `pip install -e ../orchestration_engine --config-settings editable_mode=compat` and then `pip install -e '.[dev]'`.

99
CONTEXT.md Normal file
View File

@@ -0,0 +1,99 @@
# Target Operating Model
Turns a description of a solution into a reviewed target operating model: who runs it, through which processes, with which tools, to which service levels, at what cost, and how the organisation gets there.
Accepted by the owner on 2026-10-13.
Terms of the engine this agent runs on (Execution, Phase, Approval, Deliverable, Workflow, Skill and the rest) are defined in `../orchestration_engine/CONTEXT.md` and mean the same here.
## Language
### What it starts from
**Operating spec**:
The structured statement an Execution starts from: what the customer wants from operations, the facts an operating model depends on, and the solution to be operated, as one file. It may arrive complete or partly filled, and a value it states is the customer's decision.
_Avoid_: intent file, brief, scope spec (which is the architecture agent's)
**Item**:
One thing to be operated, listed in the Operating spec: a network, a database, a pipeline. When the spec was digested from a solution architecture it is that architecture's Element, and keeps its id as a reference.
_Avoid_: Element (which is the architecture's), asset, component, resource
**Condition**:
Something that must be kept true of an Item while it is run, listed in the Operating spec: logs retained for a period, recovery within a time.
_Avoid_: Check (which is this agent's own, on its operating model), control, SLA
**Source**:
Anything a caller supplies for an Operating spec to be digested from: a solution architecture agent's files, a design document, a sentence.
_Avoid_: input, upload, baseline
### The operating model
**Target operating model**:
The operating organisation to be established, as one explicit graph of Parties, Roles, Capabilities, the Responsibilities between them, and the Checks each must meet. It is established before any process, organisation design or tooling is written in detail.
_Avoid_: TOM (in prose), blueprint, to-be state
**Business outcome**:
What the customer's business wants from the solution, in their words, such as getting new products to customers sooner. Every Measure shows one.
_Avoid_: goal, KPI (which is a Measure), objective, benefit
**Measure**:
Something measured to show a business outcome is being served, at one layer: delivery outcome, productivity guardrail, leading signal of friction, service level or cost. Taken by a Capability someone is responsible for.
_Avoid_: metric (alone), KPI, Check (which is a condition that holds or not)
**Operational requirement**:
A numbered statement of what running the solution must achieve, traced to a requirement the customer stated, a Condition, or a field of the Operating spec.
_Avoid_: NFR, service requirement
**Capability**:
Something the operating organisation must be able to do, such as manage incidents or recover from loss. It comes from the capability catalogue, names the Operational requirements it serves and the Items it covers.
_Avoid_: process (which is how a Capability is carried out), function, practice, service
**Capability catalogue**:
Our own list of Capabilities, in our own words, each citing the industry frameworks it draws on.
_Avoid_: framework, library, ITIL (which is one source)
**Party**:
An organisation with a part in operating the solution: the customer, the service provider, a supplier, the Cloud provider.
_Avoid_: team (which is inside a Party), vendor, stakeholder
**Role**:
A position in a Party, with its skills, the hours it works, and, for pricing, a country and a level.
_Avoid_: person, resource, job title
**Team**:
A group of Roles within one Party, of one of the four kinds Team Topologies names: stream-aligned, platform, enabling or complicated-subsystem. Optional: an operating model need not have teams.
_Avoid_: squad, unit, department, Party (which is the organisation the team is in)
**Interaction**:
How two Teams work together, in one of Team Topologies' three modes: collaboration, X-as-a-Service or facilitating. A collaboration is a stage with a stated end.
_Avoid_: dependency, handoff, relationship (which is the architecture's)
**Responsibility**:
A Role's part in a Capability: accountable, responsible, consulted or informed.
_Avoid_: RACI (which is a view of all of them), ownership, assignment
**Check**:
A numbered, verifiable condition that a Capability, Role or Responsibility must meet, traced to the Operational requirement it comes from. Validation gives each a verdict.
_Avoid_: Condition (which is in the spec, about an Item), control, test
**Finding**:
Something operating the solution needs that the solution, as the Operating spec describes it, does not provide. It is recorded and returned to whoever owns the design. This agent never repairs the design.
_Avoid_: gap, issue, defect, recommendation
**Scenario**:
An event the operating model must be able to handle, generated from the graph and the spec and traced through it by code: an incident on a named Item out of hours, the loss of a region.
_Avoid_: test case, use case, dry run
### Cost
**Rate card**:
The service provider's prices: a day rate by country and level, with the facts needed to turn effort into people and people into cost. It is a Skill, and only the service provider sets what is in it.
_Avoid_: price list, overlay, the customer's rates
**Staffing**:
The number of people each Role needs, computed from the effort of its Capabilities and the hours it must cover. Never written by an agent.
_Avoid_: headcount (which is a limit the customer may state), FTE count, resourcing
**Transition plan**:
The Deliverable that divides establishing the Target operating model into work packages and activities, with a computed timeline that fits the solution's build.
_Avoid_: roadmap, project plan

52
Dockerfile Normal file
View File

@@ -0,0 +1,52 @@
# The target operating model agent: the Execution engine and this Workflow.
#
# The engine comes from a named build context, so no file is copied by hand:
#
# podman build --build-context engine=../orchestration_engine -t tom-analysis .
#
# The agent itself is not installed: the Workflow finds its Skills, role files
# and provider bundles beside its own code (`ROOT` in workflow.py), so it runs
# from /app as the reference agent does.
FROM python:3.12-slim AS builder
WORKDIR /engine
COPY --from=engine pyproject.toml ./
COPY --from=engine execution_engine/ ./execution_engine/
RUN pip install --no-cache-dir --prefix=/install .
FROM python:3.12-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=/app \
A2A_HOST=0.0.0.0 \
A2A_PORT=8080 \
CHECKPOINTER=sqlite \
CHECKPOINT_DB_PATH=/data/checkpoints.db \
SQLITE_FALLBACK_DB=/data/agent_local.db \
ENV_FILE_EXPORT=false
WORKDIR /app
RUN groupadd -g 10001 appgroup && \
useradd -u 10001 -g appgroup -s /bin/sh -m appuser
COPY --from=builder /install /usr/local
COPY app/ ./app/
COPY agents/ ./agents/
COPY providers/ ./providers/
COPY skills/ ./skills/
RUN mkdir -p deliverables lessons /data && chown -R appuser:appgroup /app /data
USER appuser
VOLUME ["/data"]
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
CMD python -c "import os, urllib.request; urllib.request.urlopen(f'http://localhost:{os.environ[\"A2A_PORT\"]}/health')" || exit 1
CMD ["python", "-m", "app.main"]

393
IMPLEMENTATION_PLAN.md Normal file
View File

@@ -0,0 +1,393 @@
# Orchestration engine and target operating model agent: implementation plan
Status: seventh draft, 2026-10-13. Building has started: E1 to E5 are built, and the first slice of the operating model agent but for its digest and its evaluation. Each repository's `.agents/plans/` records what is done.
Two things are built, in two folders, and a third is left exactly as it is.
| Folder | What it is | Changed by this plan |
| --- | --- | --- |
| `gcp_solution_architecture_agent/` | The solution architecture agent, running on its own built-in engine. The reference. | No file is modified |
| `orchestration_engine/` | Two things. The **Execution engine**: the installable package an agent runs on. The **Orchestration engine**: the service every request comes through | New |
| `tom_analysis/` | The target operating model agent: its workflow, roles, Skills, provider bundle and evaluation, running on the Execution engine | New |
Terms in capitals without a definition here are the architecture agent's, from its `CONTEXT.md`. "The reference" means `gcp_solution_architecture_agent/` at commit `8b397cd`.
## 1. Decided
By the owner on 2026-10-13:
1. **The operating model agent is a separate agent, in `tom_analysis/`.**
2. **The Execution engine is built in `orchestration_engine/`.** The reference is read, never modified.
3. **The Execution engine is shown to be able to run the architecture workflow,** as an example in its own tests.
4. **The operating model agent starts from an operating spec.** Whatever it is given, the architecture agent's files among other things, is digested into that file first. The architecture's files are one source, not the only one.
5. **The operating spec holds the inventory inline.** It does not point at source files.
6. **Every request comes through the Orchestration engine,** which decides, with Jev, whether the work is solution architecture, an operating model, or both. The KAB platform does not route between agents, so nothing existing is duplicated.
7. **A gap in the architecture is never repaired by the operating model agent.** It is recorded as a finding and goes back as a Change request.
8. **Our own capability catalogue,** in our words, each entry citing the industry frameworks it draws on.
9. **The run is priced.** Staffing is costed from a rate card by country, in the currency the operating spec states. The rate card is a Skill, and its prices are set by the service provider who runs this agent: the owner. The customer's own staff are counted and never priced (section 9).
10. **How an agent's container build obtains the Execution engine is in scope** (step E7). Which means is used is still to be chosen.
Still open: section 11.
## 2. How the pieces run
```text
caller
│ A2A
┌────────▼────────┐
│ orchestration │ orchestration_engine: reads the request, asks Jev
│ engine │ which work it is, dispatches, relays every prompt
└───┬─────────┬───┘
A2A │ │ A2A
┌──────────▼───┐ ┌─▼──────────────────┐
│ architecture │ │ operating model │
│ agent │ │ agent │
│ (as it is) │ │ (tom_analysis, on │
│ │ │ the Execution engine) │
└──────────────┘ └────────────────────┘
```
Three containers. Each agent still has its own Orchestrator for its own Execution; the Orchestration engine sits above Executions and produces nothing itself. The architecture agent is called through the `/a2a` and `/generate` routes it already has.
Two names are kept apart throughout. The **Orchestration engine** is the service above Executions. The **Execution engine** is the package that holds one Execution; the reference already calls its own `ExecutionEngine`. The **Orchestrator** remains the top-level agent inside one Execution.
## 3. The Execution engine
### What is brought across
The reference's `app/` holds the Execution engine and the solution-architecture workflow together.
| | Modules in the reference |
| --- | --- |
| Engine, brought as it is (done, E2) | `config.py`, `checkpointing.py`, `customer_case.py`, `database.py`, `scope.py`, `baseline.py`, `providers.py`, `adk/runtime.py`, `adk/adk_runtime.py`, `skills/`, `states/execution_state.py`, `tools/jev.py`, `tools/guardrail.py`, `tools/mcp.py`, `tools/workspace.py`, `tools/skill_routing.py`, `learning/proposals.py`, `learning/proof.py`, `learning/skill.py` |
| Engine, rewritten as it is brought (E3): names architecture Deliverables, or imports a module that does | `nodes/execution_nodes.py`, `nodes/router.py`, `nodes/intake.py`, `tools/execution.py`, `tools/toolkit.py`, `tools/scoping.py`, `tools/meaning.py`, `states/ledger.py`, `states/lessons.py`, `recorder.py`, `adk/roles.py`, `adk/fake_runtime.py`, `workflows/execution_graph.py`, `learning/curator.py`, `learning/evidence.py`, `agent.py`, `a2a_server.py`, `card.py`, `main.py` |
| Split | `workflows/work_breakdown.py`: the scheduler comes, the rule about Elements stays behind |
| Not brought: the solution-architecture workflow | `workflows/deliverables.py` (its graph), `workflows/chain.py`, `workflows/target_architecture.py`, `workflows/changes.py`, `tools/pillars.py`, `tools/local.py`, `tools/modules.py`, the reviews, `tools/assessment.py`, `presentation.py` |
Also brought, because they are not about architecture: the Specialist protocol partial, the role files for the scope analyst, the reply writer and the lesson curator, and the evaluation harness.
### The seam
The second row refers to named Deliverables about 45 times. Each becomes something a **Workflow** supplies: one value, given to the Execution engine when it is assembled.
| In the reference | What is hard-coded | What a Workflow supplies |
| --- | --- | --- |
| `workflows/deliverables.py` | Phases, the Deliverable graph, data checks, sections for readers | The same, as data |
| `router.py` 161, 313–344, 631–652 | Which Deliverable is validation, how its verdicts are read, what a failure reopens | A validation policy |
| `router.py` 747–826 | Facts for an Approval summary | A function per Phase |
| `tools/execution.py` 320–413 | What code adds on submission: tables, diagram, rendered plan | A submission hook per Deliverable |
| `execution_nodes.py` 1016, `toolkit.py` 81 | Pillar advisors under design recommendations | A fan-out declared on a Deliverable |
| `execution_nodes.py` 1145, 1376–1449, `a2a_server.py` 178 | The final document; manifest entries | The final Deliverable and its manifest entries |
| `adk/roles.py`, `agent.py` | Required roles; the Terraform runner and the reviews | Roles, local tools and reviews, listed |
| `presentation.py`, `card.py` | Response fields and the card's wording | Supplied |
| `scope.py`, `tools/scoping.py` | One catalogue; a spec drafted from a sentence | The catalogue, and the digest adapters of section 5 |
### Two proofs, in the Execution engine's own tests
- **A toy workflow** of three Deliverables in two Phases, run on the fake runtime through questions, Approvals, a Revision and a Change request. It is what the brought-across tests for the Ledger, the router, intake and the A2A server are driven by.
- **The architecture workflow as an example.** Its Deliverable graph, data checks and hooks are written as a Workflow under the Execution engine's tests, from the reference, and the reference's tests for them are brought with it. All of it runs on the fake runtime.
The second shows the seam is wide enough for the real case. It does not show the same results with real models: that would need the architecture agent's Skills, roles and bundle on the new engine, which is moving that agent, and is not in this plan.
### What two Execution engines means
The reference keeps its built-in engine and has open steps in four iterations. Fixes do not flow between the two unless someone carries them. The Execution engine records the reference commit it was built from, so a later comparison has a starting point.
## 4. The Orchestration engine
### Deciding what a request is
In order, following ADR-0007: readings, disposition, playbook, record.
1. **Read by code.** What the caller named settles it: `work` set outright; an operating spec or `sources` supplied (operating model); a scope spec supplied (solution architecture); a `baseline` named (the agent that Execution belongs to).
2. **Read by Jev,** when code cannot tell. One typed choice about the message: solution architecture, operating model, both, a question about delivered work, or none of these. Jev is sent the message and the names of what was supplied, never file contents.
3. **Disposition.** A choice below the confidence threshold is not relied on. With someone present, the Orchestration engine asks them which they want, in plain terms. With nobody present, it starts nothing and says how to name the work. It does not guess: a wrong guess here starts an Execution of the wrong kind.
4. **Jev unreachable.** Step 1's rules, then step 3. Never a stop.
5. **Record.** The decision, the confidence and what was dispatched, per Customer case.
### Dispatching
| Decision | What happens |
| --- | --- |
| Solution architecture | The message and its metadata go to the architecture agent unchanged |
| Operating model | They go to the operating model agent |
| Both | Architecture first. When it completes, the files it returned are passed to the operating model agent as `sources`, with the Orchestration engine's own record that the Execution completed. An architecture Execution that ends with warnings stops here and the user is asked |
| A question | To the agent that delivered the work it is about |
| None | A reply saying what can be done. Nothing is started |
- **Relaying.** Each agent pauses for questions and Approvals. The Orchestration engine passes every prompt to the caller and every reply back, unchanged, and holds which downstream task is waiting.
- **"Both" is in sequence, not in parallel.** The operating model depends on the architecture. The user sees the digest of the architecture at the operating model's scoping Approval.
- **One Customer case across both agents.** The same slug is sent to each.
- **Two readings of one message.** The architecture agent's own intake still reads a message as new, a change or a question. The Orchestration engine does not replace that; it only chooses the agent.
## 5. The operating spec
The operating model agent's only input. The counterpart of the scope spec (ADR-0008), with the same rules: a structured file read against a catalogue in code; complete or partly filled; every field a value, `ask`, `decide` or `not_applicable`; the file supplied never altered; a completed copy written beside it with where each value came from; approved in scoping before anything is derived from it.
It differs in one way: it carries the system to be operated.
```yaml
metadata: {customer_case: acme-landing-zone}
engagement: {kind: platform_foundation}
intent:
statement: |
We need the landing zone run as a managed service from go-live ...
requirements:
- {id: REQ-01, statement: Platform foundation ready within 10 weeks.}
solution:
items: # what is to be operated
- {id: S-1, name: Hub network, kind: network, product: Shared VPC, environments: [prod], source_ref: E-1}
conditions: # what must be kept true
- {id: K-1, 'on': S-1, condition: Flow logs are retained for 400 days., source_ref: CHK-12}
build: {duration_days: 46, customer_owned: [Order the carrier circuit]}
inherited_risks:
- {'on': S-4, risk: Recovery was not verified in validation., source_ref: CHK-31}
operations: {support_hours: 24x7, change_process: ask}
sourcing: {model: decide, delivery_countries: [GB, PL, IN]}
organisation: {existing_teams: ask, headcount_limit: ask}
commercial: {currency: GBP, monthly_run_budget: ask}
service_management: {tooling: ask}
service_levels: {}
transition: {go_live_date: ask}
run: {pre_approved: false}
```
### Digesting sources into it
| Source | Read by | What is guaranteed |
| --- | --- | --- |
| An architecture Execution: a folder, files in the request, or forwarded A2A artefacts | A code adapter in `tom_analysis/`. It reads plain JSON for the fields it needs; nothing is copied from the reference | Every Element becomes an item and every Check a condition, with its id as `source_ref`. Counts in equal counts out, tested |
| Anything else: a design document, a spreadsheet, a diagram's description | A model, drafting records | Each record quotes the passage it came from. Nothing is guaranteed complete, which is why the customer approves it |
| A sentence, or nothing | The scope analyst, in conversation | As the architecture agent does today |
| An operating spec already written | Read as it is | |
- **Provenance per value:** supplied, digested (with the source file and its hash), answered, assumed, unknown.
- **The digest is a command and a step.** `python -m app.digest <sources> > operating-spec.yaml` runs nothing else. Started with `sources`, the agent runs the same code, and the Execution starts from the spec it wrote.
- **The mechanism is the Execution engine's; the adapters are the agent's.** The Execution engine already drafts a spec from a sentence; a Workflow lists the adapters for the formats it knows.
- **A format change in the architecture agent** is caught by a contract test against one captured Execution, since no version can be added to its files.
- **The inventory is not sent to every Specialist.** The reference puts the scope in every prompt. `solution.items` and `solution.conditions` travel as a data file to the Specialists that use them.
## 6. The capability catalogue
A Skill, `operating-capabilities`, holding `assets/catalogue.yaml`.
```yaml
- id: incident-management
name: Incident management
domain: service_operations
purpose: Restore a service that is degraded or down, and say what happened.
applies_when: {always: true}
typical_checks:
- A priority 1 incident is acknowledged within an agreed time, at any hour the service is offered.
sources:
- {framework: ITIL 4, reference: "Incident management practice"}
- {framework: Google SRE, reference: "Site Reliability Engineering, ch. 14, Managing Incidents"}
- {framework: ISO/IEC 20000-1:2018, reference: "8.6.1"}
- {framework: "Regulation (EU) 2022/2554 (DORA)", reference: "Art. 17"}
```
- **Sources are cited, never copied.** `references/sources.md` is the register: each framework, its version, publisher and licence, and how it is cited. Candidates: ITIL 4, ISO/IEC 20000-1, COBIT 2019, Google's SRE books, the FinOps Framework, NIST CSF 2.0, ISO/IEC 27001, SFIA for skills, the EU's DORA regulation, and each Cloud provider's architecture framework. The references in the example are illustrative; each is verified against its source when the entry is written.
- **Checked in code.** Every entry has a source that is in the register. Every Capability in a Target operating model names a catalogue entry, or is marked `custom` with its reason.
- **`applies_when` is read by code** against the operating spec: an item's kind, the engagement kind, a compliance framework. It proposes; the Specialist decides, and says why an applicable capability was left out.
- **No Cloud provider and no organisation.** The provider bundle maps capabilities to that provider's tooling. An organisation's role names, standards and service tiers are an overlay Skill.
- **Two things are called DORA:** the EU regulation and the DevOps research programme's metrics. The register names each in full.
## 7. The Target operating model graph, and findings
One graph, held as data, approved before detail is written. Vocabulary for `tom_analysis/CONTEXT.md`:
- **Operating spec**, **Item**, **Condition**: section 5.
- **Operational requirement**: a numbered statement of what running the solution must achieve, traced to a stated requirement, a Condition or a spec field.
- **Capability**: something the operating organisation must be able to do, from the catalogue, naming the Operational requirements it serves and the Items it covers.
- **Party**, **Role**, **Responsibility**: an organisation with a part in operating the solution; a position in it, with skills and hours; a Role's part in a Capability (accountable, responsible, consulted, informed).
- **Check**: a numbered verifiable condition on a Capability, Role or Responsibility, traced to its source.
```json
{
"parties": [{"id": "P-1", "name": "Customer platform team", "kind": "customer"}],
"roles": [{"id": "R-1", "party": "P-1", "name": "Platform engineer", "hours": "business", "skills": ["networking"]}],
"capabilities": [{"id": "C-1", "catalogue": "incident-management", "serves": ["OR-3"],
"items": ["S-1", "S-4"], "hours": "24x7", "effort_hours_per_month": 30}],
"responsibilities": [{"id": "RS-1", "capability": "C-1", "role": "R-1", "kind": "accountable"}],
"checks": [{"id": "OC-1", "on": "C-1", "condition": "A priority 1 incident is acknowledged within 15 minutes", "source": "OR-3"}]
}
```
Refused on submission, in code, against the approved operating spec:
- an Item that no Capability covers, or a Condition that no Check carries;
- an Operational requirement that no Capability or Check serves;
- a Capability without exactly one accountable Role, or with none responsible;
- a Role without a Party, or a reference to an id that is not in the graph or the spec;
- a Capability whose hours are wider than the hours of every Role responsible for it;
- support hours narrower than the availability the spec states;
- a stated `operations.*` choice that no Operational requirement names.
Written by code, never by a model: the responsibility matrix, the organisation chart, the capability map, and staffing and cost per Role (section 9).
### Findings
- A **finding** is something operating the solution needs that the solution, as the spec describes it, does not provide: nothing raises alerts, no Condition requires backups to be restored, the responding team has no means of access.
- A Specialist records it. It never adds an Item, alters a Condition, or designs around the gap in silence.
```json
{"id": "AF-1", "about": ["S-3"], "source_ref": ["E-3"],
"lacks": "No Condition requires the database's backups to be restored on a schedule.",
"needed_by": ["OR-6", "C-4"], "proposed": "Require a restore to be tested quarterly."}
```
- What depends on a finding is `not_verifiable` in validation, with the finding as the reason. The Execution still completes, with its findings listed at the Approval and in the guide.
- **When the spec was digested from an architecture Execution,** code writes `architecture-change-request.md` from the findings, in the form the architecture agent accepts as a Change request against that Execution. In a "both" request the Orchestration engine can offer to send it.
- **Otherwise** the findings are questions for whoever owns the design.
- When changed architecture files arrive, the agent runs a Change request of its own from a new digest. Findings that are now met close.
## 8. Phases and Deliverables
| Phase | Deliverable | Specialist | Depends on | Data file |
| --- | --- | --- | --- | --- |
| Scoping | operating scope | `scope-analyst` (Execution engine) | the operating spec | `operating-spec.yaml`, `scope.json`, `solution.json` |
| Operational requirements | operational requirements | `operations-analyst` | operating scope | `operational-requirements.json` |
| | current operating model | `current-state-assessor` | operating scope | `current-state.json` |
| Operating model design | service model | `service-designer` | operational requirements | `services.json` |
| | capability model | `capability-modeller` | operational requirements | `capabilities.json` |
| | design recommendations | `design-advisor`, with one `dimension-advisor` per dimension | service model, capability model | `recommendations.json` |
| | **Target operating model** | `operating-model-architect` | the three above | `target-operating-model.json`, `findings.json` |
| Operating model definition | process definitions | `process-author` | Target operating model | `processes.json` |
| | organisation design | `organisation-designer` | Target operating model | `organisation.json` |
| | tooling and observability | `tooling-author` | Target operating model | `tooling.json` |
| Validation | validation results | `validator` | the three above | `verdicts.json`, `scenarios.json` |
| Packaging | transition plan | `transition-planner` | validation results | `transition-plan.json` |
| | operating model guide | `packager` | validation results | |
- **Dimensions,** the counterpart of the pillars: process, people and organisation, tooling and automation, governance and risk, sourcing, security operations, financial operations. The bundle maps each to a Skill.
- **Scenario walk-throughs,** the counterpart of the dry run. Scenarios are generated from the graph and the spec: a priority 1 incident on a named Item outside business hours, a change to the network, loss of the primary region, a key rotation falling due, a month over budget. Code traces each: is it detected by a tool, is a responsible Role on hours, does a process exist, does it reach an accountable Role. A broken path is a failed Check.
- **Transition plan.** The Execution engine's scheduler, with Capabilities where the work breakdown has Elements. It is checked against the build duration the spec gives, so nothing goes live before the Capability that runs it is ready.
## 9. Staffing, rates and cost
The Target operating model already gives each Capability an effort in hours a month and each Role its hours. Pricing adds two facts to a Role and one file.
- **A Role has a country and a level.** `{"id": "R-2", "party": "P-2", "name": "Platform SRE", "country": "PL", "level": "senior", "hours": "24x7"}`.
- **The rate card is a Skill, `skills/rate-card/`, and it is the actual card.** Its prices are the service provider's, set by the owner and changed only by the owner, in review like any other Skill. It is not an overlay and not the Cloud provider's: every Execution prices from it, whichever Cloud provider the solution is on. A caller cannot name another card, and an operating spec cannot state a rate: a customer does not set the provider's prices.
- **The figures in it today are placeholders** drafted with this plan. They are the owner's to replace before any priced result is shown to a customer.
What the card holds, per country: a day rate for each level, the productive days in a year, and the working hours in a day. For the whole card: the currency its rates are written in, fixed conversion rates to other currencies with the date they were set, the smallest team that can cover each pattern of hours, and an uplift for work outside business hours.
Computed by code, never by a model:
1. **People per Role.** The effort of the Capabilities the Role is responsible for, divided by the productive hours of its country. Where the Role covers hours wider than a working day, never fewer than the card's minimum roster for that cover: a rota is sized by the hours it must fill, not by the work done in them.
2. **Cost per Role a month.** People, times the day rate for its country and level, times productive days over twelve, times the uplift where it applies.
3. **Conversion** into `commercial.currency`, at the card's rate for it.
4. **Totals** by Party, by country and by Capability, and against `commercial.monthly_run_budget` when the spec states one.
Rules:
- A Role whose country or level the card does not have is refused at submission, by name.
- **The operating spec states the currency.** `commercial.currency` is an essential field: asked for when someone is present, and a start with nobody to ask is refused without it. A currency the card has no conversion rate for is refused at the start, by name.
- **The customer's own staff are never priced.** A Role in a Party of kind `customer` is counted as people and carries no cost, whatever the card says. Of the other Parties, only the kinds the card lists are costed.
- The Specialist chooses where Roles sit, within `sourcing.delivery_countries` when the spec states them. It never writes a number of people or a cost.
- A plan over a stated budget is not refused. It is shown as over, with the Capabilities that cost most, as a plan over its delivery window is today.
- The card's version and its hash are recorded with the Execution and named in the guide, so a result can be traced to the prices in force when it was written.
- No exchange rate is looked up. Conversion uses the fixed rates in the card, and the guide states the rate and its date beside every converted total.
Later: the same graph priced under a different mix of countries, side by side.
## 10. Iterations
Each repository keeps its own `.agents/plans/iter-NN-<slug>/` with `MAP.md`, `DETAIL.md` and `steps/`. Every step ends with `pytest`, `ruff` and `mypy` passing.
### E: the Execution engine (`orchestration_engine/`)
| Step | What | Done when |
| --- | --- | --- |
| E1 | The repository: package layout, `CONTEXT.md` with the Execution engine's vocabulary, ADR-0001 (the Execution engine takes a Workflow), ADR-0002 (built from a reference that is never modified, at a recorded commit) | The package installs |
| E2 | The modules brought as they are, with their tests | Those tests pass |
| E3 | The Workflow value and the hooks of section 3; the ten modules rewritten to use them | No engine module names a Deliverable |
| E4 | The toy workflow | It completes on the fake runtime through a question, an Approval, a Revision and a Change request |
| E5 | The architecture workflow as an example in the tests | The reference's tests for its graph, checks and hooks pass against it |
| E6 | The digest step: sources to a spec, adapters listed by a Workflow, provenance per value | The toy workflow starts from a digested spec |
| E7 | How an agent's container build obtains the Execution engine: a dependency on its repository at a pinned commit, a package registry, or a submodule | One agent image builds from a clean checkout, with no file copied by hand |
### F: the Orchestration engine (`orchestration_engine/`)
| Step | What | Done when |
| --- | --- | --- |
| F1 | ADR-0003: one Orchestration engine above the agents; what it decides and how | Accepted by the owner |
| F2 | The service: an A2A server, a configured list of agents, a request passed through to one agent with every prompt relayed | An architecture Execution runs through it, Approvals included, with the architecture agent untouched |
| F3 | The decision: readings in code, the Jev question, thresholds, the fallbacks, the record | A labelled set of messages is routed correctly; low confidence asks; no Jev key still routes what is named |
| F4 | "Both": architecture, then its files as `sources` to the operating model agent | One request yields both sets of Deliverables under one Customer case |
| F5 | Later messages: a question or a Change request goes to the agent that owns the work; a finding's Change request offered back | A change to the architecture is followed by a change to the operating model |
### T: the operating model agent (`tom_analysis/`)
First slice, six Deliverables: operating scope, operational requirements, capability model, Target operating model, validation results, operating model guide.
| Step | What | Done when |
| --- | --- | --- |
| T1 | The repository on the Execution engine: `CONTEXT.md`, ADR-0001 (the Target operating model is an explicit graph), ADR-0002 (it starts from an operating spec), `CLAUDE.md`, `.agents/plans/` | The container starts and `/health` answers |
| T2 | The operating catalogue, the scoping Skill, `python -m app.scope check` | A spec with an unknown field is refused by name |
| T3 | The architecture adapter and `python -m app.digest`; one captured Execution as the fixture | The fixture digests to the same spec from a folder, from files in the request and from artefacts; every Element and Check is accounted for |
| T4 | The model-drafted digest for other documents | A design document yields records that each quote their source |
| T5 | The capability catalogue Skill: first entries, the sources register, the checks on both | An entry without a registered source fails a test |
| T6 | `operational-requirements.json` and `capabilities.json`, their checks, and an `operating-chain` Skill giving each file's format | A requirement with no source, and a Capability serving nothing, are refused |
| T7 | The graph: parsing, the refusals of section 7, the rendered matrix and chart | One test per refusal; the matrix is byte-identical for the same graph |
| T8 | Findings: the tool, the file, `not_verifiable` verdicts, the drafted Change request | A graph that relies on something the spec lacks is refused unless a finding names it |
| T9 | Role files, the provider bundle, the Workflow value | An Execution completes on the fake runtime |
| T10 | Validation in code: structural verdicts and scenario walk-throughs | A graph with a Role off hours fails the matching scenario |
| T11 | The guide template and packager | The guide names every Capability and finding |
| T12 | The rate card Skill: its format, the checks on it, Roles with a country and a level, and `commercial.currency` in the catalogue | A Role in a country the card lacks, and a currency it cannot convert to, are each refused by name |
| T13 | Staffing and cost in code: people per Role with the roster minimum, cost per Role, conversion, totals against the budget | The figures for a fixed graph and card are reproduced exactly; a 24x7 Role is never below the minimum roster; a customer Role has no cost |
| T14 | Evaluation: one case digested from an architecture Execution, one from a hand-written spec | Both run with real models; results recorded |
Later slices, as before: the full design Phase (service model, current operating model, dimension advisors); definition and transition (processes, organisation, tooling, the transition plan); change and learning.
### Order
```text
E1 → E2 → E3 → E4 ─┬─ E5, E6, E7
└─ T1 → T2 … T14
F1 → F2 → F3 ───────────────────── F4 → F5
```
- T starts when the toy workflow runs (E4), not when the Execution engine is finished, so the Workflow's shape is settled by a real user of it.
- F1 to F3 need only the architecture agent as it is, and can start at once.
- F4 needs the first operating model slice.
## 11. Risks and what is still open
- **Two Execution engines.** Section 3. Accepted with decision 2.
- **A wrong routing decision** starts work of the wrong kind. The Orchestration engine asks or declines when it is not confident; it never guesses.
- **A lossy digest.** A model reading a document can drop or invent records. Each record quotes its source, the code adapter is used wherever a format is known, and the customer approves the spec.
- **Nothing objective to validate against.** An operating model has no `terraform validate`. The structural refusals and scenario walk-throughs are the substitute, which is why they are in the first slice.
- **Generic output.** The defence is the chain: every Capability names Items and requirements of this spec, and evaluation fails a model that could have been written without reading it.
- **Client data.** Operating specs and operating models name people, teams and suppliers. They stay with the Customer case, never in a repository.
- **Placeholder prices.** Until the owner sets them, a priced operating model is an illustration. The card's `version` says `draft`, and the guide prints the version beside every total.
- **Prices in a repository.** The actual card is now in `tom_analysis/`, so whoever can read the repository can read the prices.
- **Open: the means for E7.** Recommended: each agent depends on the engine's repository at a pinned commit, with access given to the build as a secret. It needs no registry and pins exactly what was tested. The first agent image cannot be built until this is chosen.
- **Stale conversion rates.** The owner sets them with the prices; the guide prints their date.
- **Open: framework licences.** Citing is safe. Confirm each framework's terms before any wording is taken from it.
## 12. Where it stands
Built on 2026-10-13:
- `orchestration_engine/`: the Execution engine takes a Workflow (E1 to E3). Two Workflows run on it in its tests: the architecture agent's own (E5), and a toy with nothing of that subject in it (E4). 521 tests pass.
- `tom_analysis/`: the operating model agent's Workflow and its folder (T1, T2, T5 to T13). From a written operating spec it produces operational requirements, a capability model from the catalogue, the Target operating model with its matrix, staffing and cost, and a validation with scenario walk-throughs. 106 tests pass, among them whole Executions on the scripted runtime. The record is `.agents/plans/iter-01-first-slice/steps/step-1.md`.
Not yet done:
- **The digest** (E6, then T3 and T4). An Execution starts from an operating spec that is already written; nothing reads the architecture agent's files yet.
- **Any run with a real model** (T14). No role file, Skill or catalogue entry has been read by one.
- **The Orchestration engine** (F1 to F5).
- **The container build** (E7).
Next:
1. E6 and T3: the digest step in the engine, and the adapter that turns an architecture Execution into an operating spec. This needs one completed architecture Execution to capture as a fixture.
2. T14: a first run with real models.
3. F1 to F3 beside them.
Accepted by the owner on 2026-10-13: the engine's ADR-0001 and ADR-0002, and the vocabulary in both `CONTEXT.md` files.
Waiting on the owner: this repository's three ADRs (`docs/adr/`), which record decisions already taken in conversation; the means for E7; prices in the rate card; a completed architecture Execution to use as a fixture.

36
README.md Normal file
View File

@@ -0,0 +1,36 @@
# Target operating model agent
Turns a description of a solution into a reviewed target operating model: who runs it, through which processes, to which service levels, with how many people and at what cost.
It is the Execution engine (`../orchestration_engine`) and one Workflow. Vocabulary is in [CONTEXT.md](CONTEXT.md), decisions in [docs/adr/](docs/adr/), the design in [.scratch/target-operating-model/spec.md](.scratch/target-operating-model/spec.md), the plan in [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md), and what has been built in [.agents/plans/](.agents/plans/README.md).
## What it produces
An Execution starts from an **operating spec** and goes through five Phases, each approved before the next begins.
| Phase | Deliverable | What code checks or writes |
| --- | --- | --- |
| Scoping | Operating scope | The spec is read against a catalogue; nothing essential may be left unknown |
| Operational requirements | Operational requirements | Each is traced to what the customer stated; everything that binds is carried |
| Operating model design | Capability model | Each Capability is from the catalogue; every Item is covered and every requirement served |
| | Target operating model | One accountable Role for each Capability, someone responsible at its hours, every Condition carried by a Check. The responsibility matrix, organisation chart, staffing and cost are written from the graph |
| Validation | Validation results | A verdict on every Check; scenario walk-throughs generated and traced by code |
| Packaging | Operating model guide | |
## Layout
```
app/operating_model/ the Workflow: names, the spec reader, the chain's checks, the
graph, staffing and cost, scenarios, and the Workflow value
app/main.py python -m app.main serves the agent over A2A
agents/ role files, one per agent
providers/gcp.yaml the first Cloud provider's bundle
skills/ operating-scoping (the spec's catalogue), operating-capabilities
(the capability catalogue and its sources), operating-chain
(the data files), rate-card (the service provider's prices)
tests/ the deterministic suite; no model and no network
```
## Prices
`skills/rate-card/assets/rates.yaml` is the service provider's rate card. Its figures are placeholders while its `version` starts with `draft`, and every cost says so.

View File

@@ -0,0 +1,27 @@
---
role: capability-modeller
deliverable: capability_model
tools: []
budget: 20
includes: [specialist-protocol]
---
You produce the **capability model** Deliverable: what the operating organisation must be able to do to meet the operational requirements, drawn from the capability catalogue. You say what must be done and how much work it is. You do not say who does it.
Load the operating-capabilities Skill. `capabilities_proposed` in your context lists the catalogue's entries that code found to apply.
Attach `capabilities.json`, a list:
```json
{"id": "C-1", "catalogue": "incident-management", "serves": ["OR-1"], "items": ["S-1", "S-4"],
"hours": "24x7", "effort_hours_per_month": 30,
"basis": "Two serious incidents a month at eight hours each, and triage of alerts."}
```
- `catalogue` is an entry's id. Use `custom` only when no entry fits, with a `name` and a `reason`.
- Take every proposed entry, or say under **Capabilities** why it does not apply here.
- `serves` lists the Operational requirements the Capability exists for. `items` lists the Items it covers. Every Item must be covered by at least one Capability, and every Operational requirement served by at least one.
- `hours` is when the Capability must be staffed: `business`, `extended` or `24x7`. A Capability that responds to faults or watches for them is staffed for at least the hours `scope` says faults are responded to.
- `effort_hours_per_month` is the work the Capability takes in a steady month, across everyone who does it, for this solution. Estimate from the Items it covers and say how in `basis`. Do not pad it and do not shrink it to reach a cost: staffing and cost are computed from it.
- One Capability per catalogue entry, unless the hours or the Items differ enough that different people would do it.
Your content has two sections. Under **Capabilities**, a table of each Capability, what it covers and why it is needed. Under **Sources**, the frameworks the chosen entries draw on, cited as the catalogue cites them: by framework and by the name of the part. Say there that the references are unverified when the Skill says they are.

View File

@@ -0,0 +1,83 @@
---
role: operating-model-architect
deliverable: target_operating_model
tools: []
budget: 30
includes: [specialist-protocol]
---
You produce the **Target operating model** Deliverable: who does what. It is one graph of Parties, Roles, Responsibilities and Checks, over the Capabilities of the capability model. The customer approves it before anything else is written from it.
Build the graph with `graph_put`, ten to twenty items a call, and read it back with `graph_read`. It is attached as `target-operating-model.json` when you submit.
```json
{
"parties": [{"id": "P-1", "name": "Customer platform team", "kind": "customer"}],
"roles": [{"id": "R-1", "party": "P-2", "name": "Platform SRE", "country": "PL",
"level": "senior", "hours": "24x7"}],
"responsibilities": [{"id": "RS-1", "capability": "C-1", "role": "R-1", "kind": "responsible"}],
"checks": [{"id": "OC-1", "on": "C-1", "source": "OR-1",
"condition": "A fault on the hub network is acknowledged within 15 minutes at any hour."}]
}
```
Parties and Roles:
- A Party's `kind` is `customer`, `provider`, `supplier` or `cloud_provider`. `scope.stated.sourcing.model` decides which Parties there are.
- A Role is a position, not a person. Give each a `country` and a `level` that `rate_card` lists, and the `hours` it works. Load the rate-card Skill for how to choose them. When `scope` states `sourcing.delivery_countries`, a Role may sit only in one of them.
- Covering every hour from one country needs a rota. Covering it from several countries in turn needs a Role in each.
Teams, when the model is for a platform or names teams that exist:
- Group Roles into `teams` in the graph, each in one Party and of one of Team Topologies' kinds: `stream_aligned`, `platform`, `enabling` or `complicated_subsystem`. Give a Role its `team`.
- Say how teams work together in `interactions`: `x_as_a_service` for what one team consumes from another through a defined interface or self-service, `facilitating` for one team helping another adopt something, and `collaboration` only for a bounded period of discovering something together. A collaboration must give `until`: when it ends, and what the teams work in after.
- A platform run as a product has a Role accountable for its roadmap. Its teams' satisfaction and adoption are measured, not assumed.
```json
{"teams": [{"id": "T-1", "party": "P-2", "name": "Platform team", "type": "platform"}],
"interactions": [{"id": "I-1", "from": "T-2", "to": "T-1", "mode": "x_as_a_service",
"purpose": "Teams request environments through the self-service portal."}]}
```
Responsibilities:
- Every Capability has exactly one `accountable` Role, and at least one `responsible` Role.
- At least one Role responsible for a Capability must work the hours the Capability is needed.
- Where several Roles are responsible for a Capability they divide its effort equally, unless every one of them states a `share` and the shares add up to 1.
- For a Capability a Party other than the customer is accountable for, name a Role in another Party as `consulted` or `responsible`: there must be someone to turn to when that Party cannot be reached.
Measures:
- Tie what is measured to what the business wants. For each business outcome in `scope.stated.intent.business_outcomes`, add one or more `measures` that would show it is being served, each taken by a Capability with someone responsible for it.
- A Measure's `layer` says what it tells: `outcome` (delivery speed and stability, as DORA measures it), `guardrail` (sustainable productivity across several dimensions, as SPACE does), `leading` (friction, feedback loops and cognitive load, as DevEx does), `service_level` or `cost`. Name the `framework` from the register when one applies, and a `target` when the customer gave one.
- When the model has the delivery and developer experience measurement Capability, measure all three of its layers: none stands in for another.
- Measure nothing that serves no stated outcome. When no outcome was stated, add no Measures and say in your rationale that none could be tied to the business.
```json
{"measures": [{"id": "M-1", "name": "Deployment frequency", "layer": "outcome",
"outcome": "BO-1", "measured_by": "C-18", "framework": "dora-metrics",
"target": "Daily"}]}
```
Checks:
- A Check is a condition that can be shown to hold or not, on a Capability, a Role or a Responsibility. Its `source` is the Operational requirement or the Condition it comes from.
- Every Condition in `scope.stated.solution.conditions` must be carried by at least one Check whose `source` is that Condition's id.
- Start from the catalogue's `typical_checks` and rewrite each for this solution, with the customer's own numbers.
- Put a Check on the Capability that does the work it describes. A Check that service is restored within four hours is on the Capability that restores it, not on the one that reports service levels.
Findings:
- A finding is something the **solution** does not provide: a part of the design that is missing, such as an alert, a backup, a second region or a way in for the responders. When operating the solution needs one, attach `findings.json` with an entry for it, name the Items it is about under `about`, and do not design around it.
- Anything the operating model can provide itself is never a finding: staffing, hours of cover, a Role, a team, a process, a tool, a rota. When a Check needs one of those, design it. A finding about your own model is refused in validation and sends the model back to you.
```json
[{"id": "AF-1", "about": ["S-3"], "needed_by": ["C-4"],
"lacks": "Nothing requires the database's backups to be restored on a schedule.",
"proposed": "Require a restore of the database to be tested quarterly."}]
```
- A Check that cannot hold until a finding is met names it under `depends_on_finding`.
Never write a number of people, a rate or a cost: the responsibility matrix, the organisation chart, and the staffing and cost are computed from your graph and added to your Deliverable when you submit.
Your content is one section, **Design rationale**: how the Parties divide the work and why, why each Role sits where it does and at its level, and what the model depends on that the customer has not confirmed.

View File

@@ -0,0 +1,25 @@
---
role: operations-analyst
deliverable: operational_requirements
tools: []
budget: 20
includes: [specialist-protocol]
---
You produce the **operational requirements** Deliverable: what running the solution must achieve, each requirement traced to what the customer stated.
Attach `operational-requirements.json`, a list:
```json
{"id": "OR-1", "kind": "service_level",
"statement": "A fault on any Item is acknowledged within 15 minutes, at any hour.",
"source": ["service_levels.priority_1_response_minutes", "operations.support_hours"]}
```
- `kind` is one of `service_level`, `process`, `organisation`, `tooling`, `compliance`.
- `source` names where the requirement comes from: the id of a business outcome under `scope.stated.intent.business_outcomes`, the id of a requirement under `scope.stated.intent.requirements`, the id of a Condition under `scope.stated.solution.conditions`, or the path of a settled field of the operating spec, such as `operations.support_hours`. A requirement with no source is refused.
- Every requirement the customer stated, and every field listed under `scope.binding`, must be the source of at least one Operational requirement. A submission is refused while one is not.
- Write what must be achieved, never how or by whom: no Capability, no Role, no tool. Those come next.
- Use the customer's own numbers. Where a number was assumed, say so in the statement's requirement by naming the field; do not invent a tighter one.
- One requirement says one thing that can be shown to hold or not. Group several Conditions under one requirement only when they ask the same thing of different Items.
Your content has two sections. Under **Operational requirements**, list them by kind with a sentence on each. Under **What was not stated**, say what the spec leaves assumed or unknown that would change these requirements if answered differently.

9
agents/orchestrator.md Normal file
View File

@@ -0,0 +1,9 @@
---
role: orchestrator
tools: []
---
You coordinate an Execution that designs a target operating model. You are the only agent that talks to the user, and you produce no Deliverable yourself.
Each turn, use your tools to move the Execution on: `ready_work` shows which Deliverables can be delegated and which Phases are ready for Approval; `delegate` starts Specialists on everything that is ready; `ask_user` relays their questions; `request_approval` asks the user to approve a Phase whose Deliverables are all submitted; `reopen` sends a Deliverable back with a reason; `finish` completes the Execution once the last Phase is approved.
Delegate all ready work at once. Never write or change a Deliverable's content, and never approve a Phase on the user's behalf.

15
agents/packager.md Normal file
View File

@@ -0,0 +1,15 @@
---
role: packager
deliverable: operating_model_guide
tools: []
budget: 15
includes: [specialist-protocol]
---
You produce the **operating model guide**: the one document that assembles every other Deliverable for the people who will run the solution. You add no decision of your own.
Load the operating-guide Skill and read its template.
- Write the **Summary**: one or two short paragraphs saying what is operated, who operates it and how the work is divided between the Parties, during which hours, and what the reader should look at first. Read it from the Deliverables in `dependencies`.
- State no number of people, no cost and no count, in the summary or anywhere else. The key figures are set under your summary from the data.
- Submit every other heading of your `contract` with one line under it: `Written from the data.` Every table, figure, verdict and citation in those sections is written when you submit, from the data files of the other Deliverables. Whatever you put under them is replaced.
- Give the guide a title that names the solution.

View File

@@ -0,0 +1,37 @@
## How you work
You are a Specialist in an Execution that designs a target operating model. You produce exactly one Deliverable, named in your task. An Orchestrator coordinates the Execution and is the only agent that talks to the user.
Your task message carries JSON context:
- `workflow_request`: what the user asked for.
- `scope`: the agreed operating spec. `stated` is the customer's and binds your Deliverable; `assumed` was chosen for them and may change when they answer; `to_ask` is unknown. `stated.solution.items` is what is to be operated and `stated.solution.conditions` is what must be kept true of it. Load the operating-scoping Skill for the rules.
- `dependencies`: every earlier Deliverable your work builds on, by kind. The ones you work from directly come whole, with `content` and `attachments`. The rest come as their data file alone: the ids to keep traceable. Call `read_deliverable` for the full text of one only when the ids are not enough.
- `feedback`: why earlier versions of your Deliverable were revised. Address every item.
- `answers`: the user's replies to questions you asked in earlier runs, oldest first.
- `contract`: what `submit_deliverable` checks. `sections` are headings your content must have and `attachments` are files you must attach. A data file must keep every id traceable; the operating-chain Skill gives each file's format.
- `draft`: the draft you sent the user with your last questions, or `null`.
- `working_copy`: what your own tools already hold from earlier attempts, or `null`. Start from it; do not start again.
- `baseline`: set only in a Change request. `deliverable` is the delivered version of your own Deliverable, and `changes` is what has changed so far. `null` otherwise.
- `rate_card`: which rate card the cost is worked from. While `draft` is `true` its figures are placeholders: say so wherever you mention a cost.
- `overlay_skills`: Skills that carry the organisation's standards or this engagement's constraints. Load each one and apply it.
- `pre_approved`: `true` when no user is available.
Steps:
1. Load the Skills available to you and read the part that covers your Deliverable.
2. Ground your work in `scope`, `dependencies`, `feedback` and `answers`. When you need something only the user has, call `needs_input` with all your questions at once, each saying why it matters. Your run ends there, and you are started again with the answers.
3. Draft the Deliverable to its `contract`. When `pre_approved` is `false`, confirm the draft with the user before submitting: call `needs_input` with a short summary, asking them to approve it or request changes, and pass the complete draft as `draft` with its `attachments`. When `draft` is set and the latest entry in `answers` approves it, submit that draft as it is. When `pre_approved` is `true`, ask nothing: record each assumption under an `Assumptions` heading and continue.
4. Call `submit_deliverable` with the complete markdown content and every attachment in `contract`. When it reports what is wrong, fix each item and submit again.
Three rules hold for every Deliverable:
- **Never change the solution.** Do not add an Item, alter a Condition, or design around something the solution lacks. What operations needs and the solution does not provide is a finding.
- **Never write a number of people or a cost.** They are computed from the rate card.
- **Never claim compliance with a framework.** The capability catalogue draws on frameworks and cites them; it certifies nothing.
## When `baseline` is set
You are revising a delivered Deliverable, not writing a new one. Start from `baseline.deliverable`. Change only what the change reaches, and leave everything else as it was: the same words, the same ids. A new entry takes an id that was never used; a removed one is left out; never renumber.
You are done when `submit_deliverable` accepts the Deliverable.

30
agents/reply-writer.md Normal file
View File

@@ -0,0 +1,30 @@
---
role: reply-writer
tools: []
---
You write what the user reads on behalf of the Orchestrator of an Execution that designs a target operating model. You put into words what has already been decided or delivered, and you decide nothing. Your task message says which of two things you are writing, and carries JSON context.
## A summary to approve a Phase from
The context has `phase`, the Phase the user is being asked to approve, and `facts`, the data about it: its Deliverables, counts, staffing and cost, findings, validation verdicts, and Revisions.
Write a summary a service delivery lead can approve from:
- Say what the Phase produced and what approving it commits them to. For the design Phase, say that validation and the guide are written from this model once they approve it.
- For the scoping Phase, `facts.scope` says which fields were assumed for them and which are still unknown. Say how many of each, name the ones that most shape an operating model, and say that operational requirements are derived from this scope once they approve it.
- State every number in `facts` that bears on the decision, exactly as given. Add no fact that is not there.
- When `facts.staffing.rate_card.draft` is `true`, say that the cost is worked from placeholder prices and is an illustration.
- Name anything they should look at before approving: findings, failed or unverifiable Checks, Deliverables that were revised.
- Use plain sentences and no headings. Keep it under 120 words.
## An answer about delivered work
The context has `question`, what the user asked, and `delivered`: the `request` the operating model was designed for and its `final_document`, the operating model guide.
- Answer from the guide alone. Give the choice or the fact, and the reason the guide gives for it.
- When the guide does not say, say that it does not. Add nothing from general knowledge, and do not guess.
- Name the section of the guide the answer comes from.
- Propose no change. If the user seems to want one, say they can ask for it as a change to this operating model.
- Use plain sentences and no headings. Keep it under 150 words.
Call `write_reply` with the text. You are done when it accepts it.

24
agents/scope-analyst.md Normal file
View File

@@ -0,0 +1,24 @@
---
role: scope-analyst
deliverable: operating_scope
tools: []
budget: 30
includes: [specialist-protocol]
---
You produce the **operating scope** Deliverable: the agreed statement of what the customer wants from operations and the facts an operating model depends on, as a completed operating spec. You design nothing. Load the operating-scoping Skill: it says what each field means.
The operating spec is kept for you. `scope_read` shows what is settled, what is not, and the question to ask for each field. `scope_set` records fields, several per call.
1. Call `scope_read`. When a spec was supplied or digested, most of it is already there, and what was stated stays as stated. In particular, `solution.items` and `solution.conditions` that were supplied or digested are the solution as it was described: never add to them, remove from them or reword them.
- When your context has `source_documents`, the caller supplied documents that describe the solution and that code could not read. Read them. Where the spec does not yet hold the solution, record `solution.items` and `solution.conditions` from them with `source` `request`: one Item for each thing an organisation would operate as a unit, with a short id (`S-1`, `S-2`) and a `kind`; one Condition for each thing the documents say must be kept true in operation. Give every entry an `evidence`: the document's name and the passage, quoted, that it was read from. Record only what a document says. An entry you cannot quote evidence for is not one.
2. Read `workflow_request`. Record what it states and the spec does not yet hold, with `source` `request`: the statement itself, word for word, under `intent.statement`; the kind of engagement; and any fact it gives. Record only what the words say.
3. Settle what is not settled, the essential fields first.
- **When you may ask the user** (`pre_approved` is `false`): call `needs_input` with the questions for the essential fields, and the important ones that most change an operating model, at most eight in one call, each in the catalogue's words. Tell them they may answer, say "you decide", or skip one. Record each answer with `source` `answered`. For "you decide", choose a value and record it with `source` `assumed` and a `note`.
- **When you may not ask** (`pre_approved` is `true`): for each essential or important field not settled, choose a value from the catalogue's default guidance and what the request implies, and record it with `source` `assumed` and a `note` saying why. Leave optional fields alone.
- A field marked `decide` is yours: choose, and record it as assumed with a note.
4. Call `scope_read` again. Every essential field must be settled before you can submit.
5. Submit. Your content is the scope statement under its heading: three to six sentences saying what is to be operated, who is to operate it, during which hours, in which currency its cost will be stated, and what is out of scope, using only what the spec holds. The completed spec and the information register are written for you. Pass no attachments.
`intent.business_outcomes` is essential: everything the operating model measures is tied to one. When you may not ask, record two or three from what the request says the solution is for, as `assumed`, each with an id (`BO-1`), the outcome in plain words, and a note saying what you read it from.
Never record a number, a country, a regulation or a sourcing choice the customer did not state as though they had: that is `assumed`, with its note. Never record a rate or a price at all: prices are the service provider's.

20
agents/validator.md Normal file
View File

@@ -0,0 +1,20 @@
---
role: validator
deliverable: validation_results
tools: []
budget: 30
includes: [specialist-protocol]
---
You produce the **validation results** Deliverable: a verdict on every Check of the Target operating model.
- Give each Check a verdict from the graph and the capability model: `pass`, `fail` or `not_verifiable`, each with the evidence you read. The Checks are the whole bar: judge the model against them and nothing else. Judge each Check by its own words.
- A Check passes when the model, as designed, can meet it: the Capability it is on exists, a Role is responsible for it at the hours the Check needs, and nothing in the model contradicts it. It fails when the model cannot meet it as designed. It is `not_verifiable` when its condition cannot be shown to hold or not as it is worded.
- A Check that names a finding under `depends_on_finding` waits on that finding. Judge the finding first: is it something the solution lacks, or something the operating model could provide itself, such as staffing, hours of cover, a Role, a process or a tool?
- When the solution lacks it, the finding stands: give the Check `not_verifiable`, with the finding as the evidence.
- When the operating model could provide it, the finding does not stand: give the Check `fail`, and say in the evidence what the operating model should provide instead. The model is then revised.
- Judge whether each Capability's stated effort is plausible for the Items it covers. One that is not is a proposed addition, naming the Capability: never a failed Check.
- What you think should also be true is a proposed addition, never a failure.
- Record your verdicts with `verdict_put`, a few Checks per call. `verdict_read` shows which Checks still have none. Submit with your content and no attachments when every Check has a verdict: what you recorded is attached as `verdicts.json`.
- Give no verdict on anything whose id begins `SC-`. Those are scenario walk-throughs: they are generated and traced by code when you submit, and added to your Deliverable for you.
Your content is one section, **Check verdicts**: what passed, what failed and why, and what could not be verified, ending with a pass or fail verdict for the model as a whole.

0
app/__init__.py Normal file
View File

8
app/main.py Normal file
View File

@@ -0,0 +1,8 @@
"""The target operating model agent's entrypoint: `python -m app.main`."""
from execution_engine.main import serve
from app.operating_model.workflow import ROOT, build
if __name__ == "__main__":
serve(build(ROOT), ROOT)

View File

View File

@@ -0,0 +1,153 @@
"""The capability catalogue and the register of frameworks it cites.
Both are files of the `operating-capabilities` Skill. Code reads them to check that
every entry cites a registered framework, to propose the entries that apply to an
operating spec, and to check that a Capability names an entry that exists.
"""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import yaml
from app.operating_model import spec
SKILL = "operating-capabilities"
class CatalogueError(ValueError):
"""A catalogue or register that cannot be used; the message lists every fault."""
@dataclass(frozen=True)
class Entry:
id: str
name: str
domain: str
purpose: str
applies_when: Mapping[str, Any]
follows_support_hours: bool
typical_checks: tuple[str, ...]
sources: tuple[Mapping[str, str], ...]
@dataclass(frozen=True)
class Catalogue:
entries: Mapping[str, Entry]
frameworks: Mapping[str, Mapping[str, Any]]
verified: bool # whether a person has checked the references
def applicable(self, scope: Mapping[str, Any]) -> list[str]:
"""The entries that apply to an operating spec, in the catalogue's order.
A proposal: the Specialist decides, and says why one it leaves out does
not apply.
"""
kinds = {str(item.get("kind")) for item in spec.items(scope)}
frameworks = {
str(name) for name in spec.value(scope, "compliance.frameworks") or []
}
sourcing = spec.value(scope, "sourcing.model")
engagement = scope.get("kind") or spec.value(scope, "engagement.kind")
found = []
for entry in self.entries.values():
when = entry.applies_when
if (
when.get("always")
or kinds & set(when.get("item_kinds") or ())
or frameworks & set(when.get("compliance") or ())
or sourcing in (when.get("sourcing") or ())
or engagement in (when.get("engagement_kinds") or ())
):
found.append(entry.id)
return found
def _mapping(path: Path) -> dict[str, Any]:
data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
if not isinstance(data, dict):
raise CatalogueError(f"{path.name}: must be a mapping")
return data
def load(skill_dir: Path) -> Catalogue:
"""Read the catalogue and its register from the Skill's folder, and check them."""
register = _mapping(skill_dir / "assets" / "sources.yaml")
frameworks = register.get("frameworks") or {}
problems = [
f"sources.yaml: {name} needs {field}"
for name, framework in frameworks.items()
for field in ("name", "publisher", "licence")
if not (framework or {}).get(field)
]
raw = _mapping(skill_dir / "assets" / "catalogue.yaml").get("capabilities") or []
entries: dict[str, Entry] = {}
for index, item in enumerate(raw):
where = f"catalogue.yaml[{index}]"
if not isinstance(item, Mapping) or not item.get("id"):
problems.append(f"{where}: needs an id")
continue
identifier = str(item["id"])
where = f"catalogue.yaml: {identifier}"
if identifier in entries:
problems.append(f"{where} is listed more than once")
problems += [
f"{where} needs {field}"
for field in ("name", "domain", "purpose", "applies_when")
if not item.get(field)
]
sources = [s for s in item.get("sources") or [] if isinstance(s, Mapping)]
if not sources:
problems.append(f"{where} cites no source")
for source in sources:
if source.get("framework") not in frameworks:
problems.append(
f"{where} cites {source.get('framework')!r}, which is not in "
"the register"
)
if not source.get("reference"):
problems.append(
f"{where} cites {source.get('framework')} with no reference"
)
entries[identifier] = Entry(
id=identifier,
name=str(item.get("name", "")),
domain=str(item.get("domain", "")),
purpose=str(item.get("purpose", "")),
applies_when=dict(item.get("applies_when") or {}),
follows_support_hours=bool(item.get("follows_support_hours")),
typical_checks=tuple(str(c) for c in item.get("typical_checks") or ()),
sources=tuple(
{
"framework": str(s.get("framework")),
"reference": str(s.get("reference")),
}
for s in sources
),
)
if problems:
raise CatalogueError("; ".join(problems))
return Catalogue(
entries=entries,
frameworks=dict(frameworks),
verified=register.get("status") == "verified",
)
def cited(catalogue: Catalogue, entry_ids: Sequence[str]) -> list[str]:
"""How the entries' sources are cited, one line a framework, for a reader."""
by_framework: dict[str, list[str]] = {}
for identifier in entry_ids:
entry = catalogue.entries.get(identifier)
for source in entry.sources if entry else ():
references = by_framework.setdefault(source["framework"], [])
if source["reference"] not in references:
references.append(source["reference"])
return [
f"{catalogue.frameworks[name]['name']}: {'; '.join(references)}"
for name, references in by_framework.items()
]

View File

@@ -0,0 +1,240 @@
"""The data files along the chain, and what each must hold.
operational-requirements.json what running the solution must achieve, each
traced to what the customer stated
capabilities.json what the operating organisation must be able to
do, each serving requirements and covering Items
target-operating-model.json who does it (see graph.py)
verdicts.json the verdict on each Check
Each checker returns what is wrong, one entry per fault, so a Specialist can fix it
and submit again. `upstream` is the Deliverables a submission builds on, by kind.
"""
from __future__ import annotations
import json
from collections.abc import Callable, Mapping, Sequence
from typing import Any
from app.operating_model import spec
from app.operating_model.catalogue import Catalogue
from app.operating_model.names import (
CAPABILITIES_FILE,
CAPABILITY_MODEL,
CUSTOM,
HOURS,
OPERATIONAL_REQUIREMENTS,
REQUIREMENTS_FILE,
wider,
)
from app.operating_model.spec import Upstream
REQUIREMENT_KINDS = (
"service_level",
"process",
"organisation",
"tooling",
"compliance",
)
DataCheck = Callable[[Mapping[str, str], Upstream | None], list[str]]
def objects(text: str, name: str) -> tuple[list[dict[str, Any]], list[str]]:
"""The file's entries, and what is wrong with it as a list of objects with ids."""
try:
data = json.loads(text)
except ValueError as exc:
return [], [f"{name}: not valid JSON ({exc})"]
if not isinstance(data, list):
return [], [f"{name}: must be a JSON list"]
problems = [
f"{name}[{index}]: must be an object with an id"
for index, item in enumerate(data)
if not isinstance(item, dict) or not str(item.get("id") or "").strip()
]
accepted = [
item
for item in data
if isinstance(item, dict) and str(item.get("id") or "").strip()
]
seen = [str(item["id"]) for item in accepted]
problems += [
f"{name}: id {identifier} is used more than once"
for identifier in sorted({i for i in seen if seen.count(i) > 1})
]
return accepted, problems
def read(upstream: Upstream | None, kind: str, name: str) -> list[dict[str, Any]]:
"""An upstream data file's entries; none when it is absent or unreadable."""
text = ((upstream or {}).get(kind) or {}).get("attachments", {}).get(name, "")
return objects(text, name)[0] if text else []
def requirements(upstream: Upstream | None) -> list[dict[str, Any]]:
return read(upstream, OPERATIONAL_REQUIREMENTS, REQUIREMENTS_FILE)
def capabilities(upstream: Upstream | None) -> list[dict[str, Any]]:
return read(upstream, CAPABILITY_MODEL, CAPABILITIES_FILE)
def _strings(item: Mapping[str, Any], field: str) -> list[str]:
found = item.get(field)
return [str(entry) for entry in found] if isinstance(found, list) else []
def check_requirements(
attachments: Mapping[str, str], upstream: Upstream | None
) -> list[str]:
"""Every Operational requirement is traced to what the customer stated, and
everything they stated that binds is carried by one."""
name = REQUIREMENTS_FILE
entries, problems = objects(attachments[name], name)
if not entries and not problems:
return [f"{name}: lists no Operational requirements"]
scope = spec.agreed(upstream)
stated = set(spec.ids(spec.stated_requirements(scope)))
# A business outcome may be what a requirement comes from. It need not be: what
# shows each outcome is served is a Measure, checked on the operating model.
outcomes = set(spec.ids(spec.business_outcomes(scope)))
kept_true = set(spec.ids(spec.conditions(scope)))
named: set[str] = set()
for entry in entries:
where = f"{name}: {entry['id']}"
if not str(entry.get("statement") or "").strip():
problems.append(f"{where} needs a statement")
if entry.get("kind") not in REQUIREMENT_KINDS:
problems.append(
f"{where}: kind must be one of {', '.join(REQUIREMENT_KINDS)}"
)
sources = _strings(entry, "source")
if not sources:
problems.append(
f"{where} has no source: name the stated requirement, the Condition "
"or the field of the operating spec it comes from"
)
for source in sources:
named.add(source)
known = (
source in stated
or source in outcomes
or source in kept_true
or spec.settled(scope, source)
)
if scope and not known:
problems.append(
f"{where}: source {source} is not a stated requirement, a "
"Condition or a settled field of the operating spec"
)
if scope:
problems += [
f"{name}: the customer's requirement {identifier} is carried by no "
"Operational requirement"
for identifier in sorted(stated - named)
]
problems += [
f"{name}: the customer stated {path}, and no Operational requirement "
"names it as a source"
for path in spec.binding(scope)
if path not in named
]
return problems
def check_capabilities(catalogue: Catalogue) -> DataCheck:
"""Every Capability comes from the catalogue, serves a requirement and covers
Items; every Item is covered and every requirement served."""
def check(attachments: Mapping[str, str], upstream: Upstream | None) -> list[str]:
name = CAPABILITIES_FILE
entries, problems = objects(attachments[name], name)
if not entries and not problems:
return [f"{name}: lists no Capabilities"]
scope = spec.agreed(upstream)
known_items = set(spec.ids(spec.items(scope)))
known_requirements = {str(r["id"]) for r in requirements(upstream)}
support = spec.value(scope, "operations.support_hours")
covered: set[str] = set()
served: set[str] = set()
for entry in entries:
where = f"{name}: {entry['id']}"
entry_id = entry.get("catalogue")
listed = catalogue.entries.get(str(entry_id))
if entry_id == CUSTOM:
if not str(entry.get("reason") or "").strip():
problems.append(f"{where} is custom and gives no reason")
if not str(entry.get("name") or "").strip():
problems.append(f"{where} is custom and needs a name")
elif listed is None:
problems.append(
f"{where}: {entry_id!r} is not in the capability catalogue; "
f"use an entry's id, or {CUSTOM} with a reason"
)
serves, on = _strings(entry, "serves"), _strings(entry, "items")
if not serves:
problems.append(f"{where} serves no Operational requirement")
served.update(serves)
covered.update(on)
if upstream is not None:
problems += [
f"{where} serves {identifier}, which is not an Operational requirement"
for identifier in serves
if identifier not in known_requirements
]
problems += [
f"{where} covers {identifier}, which is not an Item of the operating spec"
for identifier in on
if identifier not in known_items
]
hours = entry.get("hours")
if hours not in HOURS:
problems.append(f"{where}: hours must be one of {', '.join(HOURS)}")
elif (
listed is not None
and listed.follows_support_hours
and support in HOURS
and wider(str(support), str(hours))
):
problems.append(
f"{where}: faults are responded to {support}, and {listed.name} "
f"is staffed only {hours}"
)
effort = entry.get("effort_hours_per_month")
if (
isinstance(effort, bool)
or not isinstance(effort, (int, float))
or effort <= 0
):
problems.append(
f"{where}: effort_hours_per_month must be a number above zero"
)
if not str(entry.get("basis") or "").strip():
problems.append(f"{where} needs a basis for its effort")
if upstream is not None:
problems += [
f"{name}: Item {identifier} is covered by no Capability"
for identifier in sorted(known_items - covered)
]
problems += [
f"{name}: Operational requirement {identifier} is served by no Capability"
for identifier in sorted(known_requirements - served)
]
return problems
return check
def names_of(
capabilities_list: Sequence[Mapping[str, Any]], catalogue: Catalogue
) -> dict[str, str]:
"""Each Capability's name: its catalogue entry's, or its own when it is custom."""
return {
str(entry["id"]): (
catalogue.entries[str(entry.get("catalogue"))].name
if str(entry.get("catalogue")) in catalogue.entries
else str(entry.get("name") or entry["id"])
)
for entry in capabilities_list
}

View File

@@ -0,0 +1,413 @@
"""Digesting a solution architecture into an operating spec, in code.
The solution architecture agent delivers its design as data files. This reads the
ones an operating model depends on and writes the operating spec's fields from
them: nothing is summarised and nothing is guessed, so what is digested can be
counted against what was delivered.
components.json, products.json each component becomes an Item to be operated,
target-architecture.json with the Elements that realise it
requirements.json, what each says can only be shown in operation
recommendations.json (`outside_code`) becomes a Condition
verdicts.json a Check that failed or could not be verified
becomes an inherited risk
scope.json what the customer stated of operations, and of
the solution, is carried across as stated
work-breakdown.json, how long the build takes, and what the customer
execution.json owns of it
An Item is a component, not a resource: an organisation operates a hybrid
connectivity gateway, not each of its six routers and attachments. A Condition is
what the architecture itself says infrastructure code cannot verify: an
availability measured in operation, a recovery rehearsed. The settings its Checks
verified at build are kept true by configuration management, and are not Conditions.
Only what the customer stated in the architecture's scope is carried across as
stated. What that agent assumed for them is left to be asked again.
"""
from __future__ import annotations
import json
import math
import re
import sys
from collections.abc import Mapping, Sequence
from pathlib import Path
from typing import Any
import yaml
from execution_engine.workflows.workflow import Digested
GRAPH = "target-architecture.json"
COMPONENTS = "components.json"
PRODUCTS = "products.json"
REQUIREMENTS = "requirements.json"
RECOMMENDATIONS = "recommendations.json"
VERDICTS = "verdicts.json"
SCOPE = "scope.json"
PLAN = "work-breakdown.json"
MANIFEST = "execution.json"
# Every file the digest reads. A folder is read for these and nothing else.
FILES = (
MANIFEST,
SCOPE,
REQUIREMENTS,
RECOMMENDATIONS,
COMPONENTS,
PRODUCTS,
GRAPH,
VERDICTS,
PLAN,
)
DEFAULT_KIND = "service"
SOURCE_PREFIX = "architecture:"
_HOURS = {"business_hours": "business", "extended": "extended", "always": "24x7"}
_NUMBER = re.compile(r"\d+(?:\.\d+)?")
def _load(sources: Mapping[str, str], name: str) -> Any:
if name not in sources:
return None
try:
return json.loads(sources[name])
except ValueError as exc:
raise ValueError(f"{name}: not valid JSON ({exc})") from exc
def _records(data: Any) -> list[dict[str, Any]]:
return (
[item for item in data if isinstance(item, dict)]
if isinstance(data, list)
else []
)
def _within(document: Any, path: str) -> Any:
for part in path.split("."):
if not isinstance(document, Mapping) or part not in document:
return None
document = document[part]
return document
def kind_of(product: str, item_kinds: Mapping[str, Sequence[str]]) -> str:
"""What kind of thing a product is, by the words the provider bundle lists."""
lowered = product.lower()
for kind, words in item_kinds.items():
if any(str(word).lower() in lowered for word in words):
return kind
return DEFAULT_KIND
def item_kinds(root: Path, provider: str) -> dict[str, list[str]]:
"""The provider bundle's table of what kind of thing each product is."""
path = root / "providers" / f"{provider}.yaml"
data = yaml.safe_load(path.read_text(encoding="utf-8")) if path.is_file() else {}
kinds = ((data or {}).get("operating") or {}).get("item_kinds") or {}
return {
str(kind): [str(word) for word in words or []] for kind, words in kinds.items()
}
def architecture(
sources: Mapping[str, str], kinds: Mapping[str, Sequence[str]] | None = None
) -> Digested:
"""The operating spec's fields, read from a solution architecture's data files.
Returns nothing digested when the sources are not a solution architecture.
Raises ValueError, saying what is wrong, when they are one that cannot be used.
"""
if GRAPH not in sources:
return Digested({})
graph = _load(sources, GRAPH)
if not isinstance(graph, dict) or not isinstance(graph.get("elements"), list):
# A ValueError is what the engine turns into a refused start.
raise ValueError( # noqa: TRY004
f"{GRAPH}: must be an object with a list of elements"
)
manifest = _load(sources, MANIFEST) or {}
if manifest and manifest.get("status", "completed") != "completed":
raise ValueError(
f"{MANIFEST}: the Execution did not complete ({manifest.get('status')}); "
"an operating model is designed for an architecture that was approved"
)
elements = _records(graph["elements"])
components = _records(_load(sources, COMPONENTS))
product_of = {
str(item.get("component")): str(item.get("product"))
for item in _records(_load(sources, PRODUCTS))
}
kinds = kinds or {}
notes: dict[str, str] = {}
# --- Items: one a component, with the Elements that realise it -------------------
realised: dict[str, list[dict[str, Any]]] = {}
for element in elements:
realised.setdefault(str(element.get("component") or ""), []).append(element)
if not components:
components = [
{"id": identifier, "name": identifier, "requirements": []}
for identifier in realised
if identifier
]
items: list[dict[str, Any]] = []
item_of: dict[str, str] = {}
for component in components:
component_id = str(component.get("id"))
own = realised.get(component_id, [])
products = [str(e.get("product")) for e in own if e.get("product")]
product = product_of.get(component_id) or (
max(set(products), key=products.count) if products else ""
)
item_id = f"S-{len(items) + 1}"
item_of[component_id] = item_id
items.append(
{
"id": item_id,
"name": str(component.get("name") or component_id),
"kind": kind_of(product, kinds),
**({"product": product} if product else {}),
"source_ref": component_id,
"elements": [str(e.get("id")) for e in own],
}
)
loose = realised.get("", [])
if loose:
# Elements the architecture filed under no component are still operated.
item_id = f"S-{len(items) + 1}"
item_of[""] = item_id
items.append(
{
"id": item_id,
"name": "Resources outside any component",
"kind": DEFAULT_KIND,
"elements": [str(e.get("id")) for e in loose],
}
)
if not items:
raise ValueError(f"{GRAPH}: lists no Elements to operate")
placed = sum(len(item["elements"]) for item in items)
if placed != len(elements):
unplaced = sorted(
str(e.get("id"))
for e in elements
if str(e.get("component") or "") not in item_of
)
raise ValueError(
f"{GRAPH}: these Elements name a component that {COMPONENTS} does not "
f"have: {', '.join(unplaced)}"
)
notes["solution.items"] = (
f"{len(items)} components of the solution architecture, standing for "
f"{len(elements)} Elements"
)
# --- Conditions: what can only be shown to hold in operation --------------------
serving: dict[str, list[str]] = {}
for component in components:
for requirement in component.get("requirements") or []:
serving.setdefault(str(requirement), []).append(str(component.get("id")))
conditions: list[dict[str, Any]] = []
stated = 0
for name in (REQUIREMENTS, RECOMMENDATIONS):
for entry in _records(_load(sources, name)):
if entry.get("change") == "removed":
continue
entry_id = str(entry.get("id"))
behind = [str(c) for c in entry.get("components") or []] or serving.get(
entry_id, []
)
on = [item_of[c] for c in behind if c in item_of] or [
str(i["id"]) for i in items
]
for statement in entry.get("outside_code") or []:
stated += 1
conditions.append(
{
"id": f"K-{len(conditions) + 1}",
"on": on,
"condition": " ".join(str(statement).split()),
"source_ref": entry_id,
}
)
if len(conditions) != stated:
raise ValueError(
"a condition the architecture leaves to operation was not placed"
)
if conditions:
notes["solution.conditions"] = (
f"{len(conditions)} conditions the solution architecture says cannot be "
"verified from infrastructure code"
)
# --- Inherited risks: Checks that did not pass ------------------------------------
element_item = {
str(e.get("id")): item_of.get(str(e.get("component") or ""), items[0]["id"])
for e in elements
}
check_on = {
str(check.get("id")): element_item.get(
str(check.get("applies_to")), items[0]["id"]
)
for check in _records(graph.get("checks"))
}
statement_of = {
str(check.get("id")): str(check.get("statement") or "")
for check in _records(graph.get("checks"))
}
verdicts = _load(sources, VERDICTS) or {}
risks = [
{
"on": check_on.get(str(item.get("check")), items[0]["id"]),
"risk": (
f"Check {item.get('check')} "
+ (
"failed"
if item.get("verdict") == "fail"
else "could not be verified"
)
+ f" in validation: {statement_of.get(str(item.get('check')), '')} "
f"({' '.join(str(item.get('evidence') or '').split())})"
).strip(),
"source_ref": str(item.get("check")),
}
for item in _records(
verdicts.get("checks") if isinstance(verdicts, dict) else None
)
if item.get("verdict") in ("fail", "not_verifiable")
]
# --- What the customer stated, carried across as stated --------------------------
scope = _load(sources, SCOPE) or {}
said = scope.get("stated") if isinstance(scope, dict) else {}
spec: dict[str, Any] = {
"solution": {
"items": items,
**({"conditions": conditions} if conditions else {}),
}
}
def put(path: str, value: Any, note: str = "") -> None:
if value in (None, "", [], {}):
return
node = spec
*parents, last = path.split(".")
for part in parents:
node = node.setdefault(part, {})
node[last] = value
if note:
notes[path] = note
if risks:
put("solution.inherited_risks", risks, f"{VERDICTS}: Checks that did not pass")
execution_id = manifest.get("execution_id")
put("metadata.customer_case", manifest.get("customer_case"))
put(
"metadata.source",
f"{SOURCE_PREFIX}{execution_id}"
if execution_id
else f"{SOURCE_PREFIX}unverified",
"the Execution these files are of"
if execution_id
else f"no {MANIFEST} came with the files, so they are not known to be one "
"completed Execution",
)
put("engagement.kind", scope.get("kind") if isinstance(scope, dict) else None)
put(
"intent.statement",
manifest.get("workflow_request") or _within(said, "intent.statement"),
"the request the solution architecture was designed for",
)
criteria = _within(said, "intent.success_criteria")
put(
"intent.business_outcomes",
[
{"id": f"BO-{n}", "outcome": " ".join(str(text).split())}
for n, text in enumerate(criteria or [], start=1)
if str(text).strip()
]
if isinstance(criteria, list)
else None,
f"{SCOPE}: the success criteria the customer stated",
)
put("solution.cloud_provider", _within(said, "estate.cloud_provider"))
hours = _within(said, "operations.support_hours")
put("operations.support_hours", _HOURS.get(str(hours)), f"{SCOPE}: {hours}")
available = _within(said, "workload.availability.hours")
put(
"operations.availability_hours",
_HOURS.get(str(available)),
f"{SCOPE}: {available}",
)
put("operations.change_process", _within(said, "operations.change_process"))
put("operations.monitoring", _within(said, "operations.monitoring"))
put("compliance.frameworks", _within(said, "compliance.frameworks"))
recovery = _NUMBER.search(
str(_within(said, "workload.availability.recovery_time") or "")
)
put(
"service_levels.recovery_time_hours",
float(recovery.group()) if recovery else None,
)
currency = _within(said, "cost.currency")
put("commercial.currency", currency if isinstance(currency, str) else None)
plan = manifest.get("delivery_plan") if isinstance(manifest, dict) else None
days = (plan or {}).get("working_days")
if isinstance(days, (int, float)) and not isinstance(days, bool):
put(
"solution.build.duration_days",
math.ceil(days),
"the build's computed length, in working days",
)
activities = _records((_load(sources, PLAN) or {}).get("activities"))
put(
"solution.build.customer_owned",
[str(a.get("name")) for a in activities if a.get("owner") == "customer"],
f"{PLAN}: the activities the customer owns",
)
return Digested(
spec=spec, used=tuple(n for n in FILES if n in sources), notes=notes
)
def read_folder(folder: Path) -> dict[str, str]:
"""The data files of an architecture Execution's folder that the digest reads."""
return {
name: (folder / name).read_text(encoding="utf-8")
for name in FILES
if (folder / name).is_file()
}
def main(arguments: Sequence[str]) -> int:
"""`python -m app.operating_model.digest <folder> [provider]`: print the operating
spec digested from an architecture Execution's folder, and what is left to settle."""
from app.operating_model.workflow import ROOT
if not arguments:
print(main.__doc__, file=sys.stderr)
return 2
folder = Path(arguments[0])
provider = arguments[1] if len(arguments) > 1 else "gcp"
sources = read_folder(folder)
try:
digested = architecture(sources, item_kinds(ROOT, provider))
except ValueError as exc:
print(f"Cannot digest {folder}: {exc}", file=sys.stderr)
return 1
if not digested.spec:
print(
f"{folder} holds no {GRAPH}: it is not a solution architecture.",
file=sys.stderr,
)
return 1
print(yaml.safe_dump(dict(digested.spec), sort_keys=False, allow_unicode=True))
for path, note in digested.notes.items():
print(f"# {path}: {note}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

View File

@@ -0,0 +1,719 @@
"""The Target operating model: one explicit graph, checked before anything is derived.
parties the organisations with a part in operating the solution
teams optional: the teams within a Party, each of one of Team
Topologies' four kinds
roles positions in a Party, each in a country and at a level, and
in a team when there are teams
interactions optional: how two teams work together, in one of Team
Topologies' three modes; a collaboration says when it ends
measures what is measured to show each business outcome the customer
stated is being served, at which layer, and by which
Capability
responsibilities a Role's part in a Capability: accountable, responsible,
consulted or informed
checks numbered, verifiable conditions on a Capability, a Role or a
Responsibility, each traced to what it comes from
The Capabilities are the capability model's, by id. A submission is refused while
the graph does not hold together, so what the customer approves is complete: every
Capability has one Role answerable for it and someone to do it at the hours it is
needed, and everything the customer said must be kept true is carried by a Check.
The responsibility matrix, the organisation chart and the staffing and cost are
written from the graph by code, as views of it.
"""
from __future__ import annotations
import json
from collections.abc import Callable, Collection, Mapping, Sequence
from typing import Any
from execution_engine.workflows.deliverables import GraphShape, ListShape
from app.operating_model import chain, spec, staffing
from app.operating_model.catalogue import Catalogue
from app.operating_model.names import (
ACCOUNTABLE,
COLLABORATION,
DEVELOPER_LAYERS,
FINDINGS_FILE,
GRAPH_FILE,
HOURS,
INTERACTION_MODES,
MEASURE_LAYERS,
MEASUREMENT_ENTRY,
PARTY_KINDS,
RESPONSIBILITY_KINDS,
RESPONSIBLE,
TARGET_OPERATING_MODEL,
TEAM_TYPES,
wider,
)
from app.operating_model.spec import Upstream
from app.operating_model.staffing import RateCard
LISTS = (
"parties",
"teams",
"roles",
"responsibilities",
"interactions",
"measures",
"checks",
)
OUTCOMES_HEADING = "Business outcomes"
MATRIX_HEADING = "Responsibility matrix"
STAFFING_HEADING = "Staffing and cost"
FINDINGS_HEADING = "Findings"
_LETTER = {"accountable": "A", "responsible": "R", "consulted": "C", "informed": "I"}
# The graph as the operating-model architect builds it, a few items at a time.
SHAPE = GraphShape(
attachment=GRAPH_FILE,
lists={
"parties": ListShape(("id", "name", "kind"), label="Parties"),
"teams": ListShape(
("id", "party", "name", "type"), optional=("purpose",), label="Teams"
),
"roles": ListShape(
("id", "party", "name", "country", "level", "hours"),
optional=("skills", "team"),
label="Roles",
),
"interactions": ListShape(
("id", "from", "to", "mode"),
optional=("purpose", "until"),
label="Interactions",
),
"measures": ListShape(
("id", "name", "layer", "outcome", "measured_by"),
optional=("framework", "target"),
label="Measures",
),
"responsibilities": ListShape(
("id", "capability", "role", "kind"),
optional=("share",),
label="Responsibilities",
),
"checks": ListShape(
("id", "on", "condition", "source"),
optional=("depends_on_finding",),
label="Checks",
),
},
)
def parse(text: str) -> tuple[dict[str, list[dict[str, Any]]], list[str]]:
"""The graph's four lists, and what is wrong with the file's shape."""
try:
data = json.loads(text)
except ValueError as exc:
return {name: [] for name in LISTS}, [f"{GRAPH_FILE}: not valid JSON ({exc})"]
if not isinstance(data, dict):
return {name: [] for name in LISTS}, [
f"{GRAPH_FILE}: must be an object with {', '.join(LISTS)}"
]
graph: dict[str, list[dict[str, Any]]] = {}
problems = []
for name in LISTS:
entries = data.get(name) or []
if not isinstance(entries, list):
problems.append(f"{GRAPH_FILE}: {name} must be a list")
entries = []
graph[name] = [dict(item) for item in entries if isinstance(item, dict)]
for index, item in enumerate(entries):
where = f"{GRAPH_FILE}: {name}[{index}]"
if not isinstance(item, dict):
problems.append(f"{where} must be an object")
continue
missing = [
field
for field in SHAPE.lists[name].required
if not str(item.get(field) or "").strip()
]
if missing:
problems.append(
f"{where} ({item.get('id', 'no id')}) needs {', '.join(missing)}"
)
seen = [str(item.get("id")) for item in graph[name] if item.get("id")]
problems += [
f"{GRAPH_FILE}: id {identifier} is used more than once in {name}"
for identifier in sorted({i for i in seen if seen.count(i) > 1})
]
return graph, problems
def read(upstream: Upstream | None) -> dict[str, list[dict[str, Any]]] | None:
"""The submitted graph from the Deliverables a submission builds on."""
text = (
((upstream or {}).get(TARGET_OPERATING_MODEL) or {})
.get("attachments", {})
.get(GRAPH_FILE, "")
)
return parse(text)[0] if text else None
def findings(attachments: Mapping[str, str]) -> list[dict[str, Any]]:
text = attachments.get(FINDINGS_FILE, "")
return chain.objects(text, FINDINGS_FILE)[0] if text.strip() else []
def check(card: RateCard, frameworks: Collection[str] = ()) -> chain.DataCheck:
"""What is wrong with a Target operating model, against the capability model,
the operating spec and the rate card. `frameworks` are the register's, which a
Measure may name."""
def checked(attachments: Mapping[str, str], upstream: Upstream | None) -> list[str]:
graph, problems = parse(attachments[GRAPH_FILE])
if problems:
return problems
name = GRAPH_FILE
scope = spec.agreed(upstream)
capabilities = chain.capabilities(upstream)
capability_ids = {str(c["id"]) for c in capabilities}
requirement_ids = {str(r["id"]) for r in chain.requirements(upstream)}
conditions = spec.ids(spec.conditions(scope))
parties = {str(p["id"]): p for p in graph["parties"]}
roles = {str(r["id"]): r for r in graph["roles"]}
for party in graph["parties"]:
if party.get("kind") not in PARTY_KINDS:
problems.append(
f"{name}: Party {party['id']}: kind must be one of "
f"{', '.join(PARTY_KINDS)}"
)
allowed = [
str(c) for c in spec.value(scope, "sourcing.delivery_countries") or []
]
for role in graph["roles"]:
where = f"{name}: Role {role['id']}"
if role.get("party") not in parties:
problems.append(
f"{where} is in {role.get('party')}, which is not a Party"
)
if role.get("hours") not in HOURS:
problems.append(f"{where}: hours must be one of {', '.join(HOURS)}")
country = str(role.get("country"))
if country not in card.countries:
problems.append(
f"{where} sits in {country}, which the rate card does not have: "
f"{', '.join(sorted(card.countries))}"
)
elif allowed and country not in allowed:
problems.append(
f"{where} sits in {country}; the customer allows only "
f"{', '.join(allowed)}"
)
if str(role.get("level")) not in card.levels:
problems.append(
f"{where} is at level {role.get('level')}, which the rate card "
f"does not have: {', '.join(card.levels)}"
)
problems += check_teams(graph, parties, name)
by_capability: dict[str, list[dict[str, Any]]] = {}
for item in graph["responsibilities"]:
where = f"{name}: Responsibility {item['id']}"
if item.get("kind") not in RESPONSIBILITY_KINDS:
problems.append(
f"{where}: kind must be one of {', '.join(RESPONSIBILITY_KINDS)}"
)
if item.get("role") not in roles:
problems.append(
f"{where} names {item.get('role')}, which is not a Role"
)
if upstream is not None and item.get("capability") not in capability_ids:
problems.append(
f"{where} names {item.get('capability')}, which is not a "
"Capability of the capability model"
)
by_capability.setdefault(str(item.get("capability")), []).append(item)
for capability in capabilities:
identifier = str(capability["id"])
held = by_capability.get(identifier, [])
accountable = [r for r in held if r.get("kind") == ACCOUNTABLE]
responsible = [r for r in held if r.get("kind") == RESPONSIBLE]
if len(accountable) != 1:
problems.append(
f"{name}: Capability {identifier} has {len(accountable)} "
"accountable Roles; it needs exactly one"
)
if not responsible:
problems.append(
f"{name}: Capability {identifier} has no responsible Role"
)
continue
needed = str(capability.get("hours"))
worked = [
str(roles[str(r["role"])].get("hours"))
for r in responsible
if r.get("role") in roles
]
if (
needed in HOURS
and worked
and all(hours in HOURS and wider(needed, hours) for hours in worked)
):
problems.append(
f"{name}: Capability {identifier} is needed {needed}, and no "
"Role responsible for it works those hours"
)
problems += [
f"{name}: {fault}"
for fault in staffing.shares(graph["responsibilities"])[1]
]
found = findings(attachments)
finding_ids = {str(item["id"]) for item in found}
on = (
capability_ids
| set(roles)
| {str(r["id"]) for r in graph["responsibilities"]}
| {str(team["id"]) for team in graph["teams"]}
| {str(i["id"]) for i in graph["interactions"]}
| {str(m["id"]) for m in graph["measures"]}
)
carried: set[str] = set()
for item in graph["checks"]:
where = f"{name}: Check {item['id']}"
source = str(item.get("source"))
carried.add(source)
if upstream is not None and item.get("on") not in on:
problems.append(
f"{where} is on {item.get('on')}, which is not a Capability, a "
"Role or a Responsibility"
)
if upstream is not None and source not in requirement_ids | set(conditions):
problems.append(
f"{where}: source {source} is not an Operational requirement "
"or a Condition"
)
depends = item.get("depends_on_finding")
if depends and depends not in finding_ids:
problems.append(f"{where} depends on {depends}, which is not a finding")
problems += [
f"{name}: Condition {identifier} is carried by no Check"
for identifier in conditions
if identifier not in carried
]
problems += check_measures(
graph, scope, capabilities, by_capability, roles, frameworks, name
)
problems += check_findings(
attachments,
scope,
on | {str(c["id"]) for c in graph["checks"]} | requirement_ids,
)
return problems
return checked
def check_teams(
graph: Mapping[str, Sequence[Mapping[str, Any]]],
parties: Mapping[str, Any],
name: str,
) -> list[str]:
"""What is wrong with the teams and how they work together.
Teams are optional. When there are any, each is of one of Team Topologies' four
kinds, a Role in a team belongs to that team's Party, and two teams work
together in one of its three modes. A collaboration is a stage, not a standing
arrangement: it says when it ends.
"""
problems = []
teams = {str(team["id"]): team for team in graph["teams"]}
for team in graph["teams"]:
where = f"{name}: Team {team['id']}"
if team.get("type") not in TEAM_TYPES:
problems.append(f"{where}: type must be one of {', '.join(TEAM_TYPES)}")
if team.get("party") not in parties:
problems.append(f"{where} is in {team.get('party')}, which is not a Party")
for role in graph["roles"]:
team_id = role.get("team")
if not team_id:
continue
found = teams.get(str(team_id))
if found is None:
problems.append(
f"{name}: Role {role['id']} is in {team_id}, which is not a Team"
)
elif found.get("party") != role.get("party"):
problems.append(
f"{name}: Role {role['id']} is in {role.get('party')}, and its team "
f"{team_id} is in {found.get('party')}"
)
for item in graph["interactions"]:
where = f"{name}: Interaction {item['id']}"
if item.get("mode") not in INTERACTION_MODES:
problems.append(
f"{where}: mode must be one of {', '.join(INTERACTION_MODES)}"
)
for end in ("from", "to"):
if item.get(end) not in teams:
problems.append(f"{where}: {end} {item.get(end)} is not a Team")
if item.get("from") == item.get("to"):
problems.append(f"{where} is between a team and itself")
if (
item.get("mode") == COLLABORATION
and not str(item.get("until") or "").strip()
):
problems.append(
f"{where} is a collaboration and does not say when it ends: give "
"`until`, and what the teams work in after"
)
return problems
def check_measures(
graph: Mapping[str, Sequence[Mapping[str, Any]]],
scope: Mapping[str, Any],
capabilities: Sequence[Mapping[str, Any]],
by_capability: Mapping[str, Sequence[Mapping[str, Any]]],
roles: Mapping[str, Any],
frameworks: Collection[str],
name: str,
) -> list[str]:
"""Every business outcome the customer stated is shown by something measured,
and everything measured shows one: nothing is measured for its own sake.
A Measure is taken by a Capability someone is responsible for. Where the model
measures how work flows, it measures all three layers: delivery outcomes,
sustainable productivity and friction. No one of them stands in for the others.
"""
problems = []
outcomes = {str(o["id"]): o for o in spec.business_outcomes(scope)}
capability_ids = {str(c["id"]) for c in capabilities}
measured: set[str] = set()
for item in graph["measures"]:
where = f"{name}: Measure {item['id']}"
if item.get("layer") not in MEASURE_LAYERS:
problems.append(
f"{where}: layer must be one of {', '.join(MEASURE_LAYERS)}"
)
outcome = str(item.get("outcome"))
measured.add(outcome)
if outcome not in outcomes:
problems.append(
f"{where} shows {outcome}, which is not a business outcome the "
"customer stated"
+ (f": {', '.join(outcomes)}" if outcomes else "; none were stated")
)
taker = str(item.get("measured_by"))
if taker not in capability_ids:
problems.append(
f"{where} is measured by {taker}, which is not a Capability"
)
elif not any(
r.get("kind") == "responsible" and r.get("role") in roles
for r in by_capability.get(taker, ())
):
problems.append(
f"{where} is measured by {taker}, and nobody is responsible for it"
)
if item.get("framework") and frameworks and item["framework"] not in frameworks:
problems.append(
f"{where} names framework {item['framework']}, which is not in the register"
)
problems += [
f"{name}: business outcome {identifier} ({outcomes[identifier].get('outcome')}) "
"is shown by no Measure"
for identifier in outcomes
if identifier not in measured
]
measurers = {
str(c["id"]) for c in capabilities if c.get("catalogue") == MEASUREMENT_ENTRY
}
# Only once there is an outcome to tie Measures to: with none, no Measure can
# be accepted, and asking for three of them could never be met.
if measurers and outcomes:
layers = {
str(m.get("layer"))
for m in graph["measures"]
if m.get("measured_by") in measurers
}
missing = [layer for layer in DEVELOPER_LAYERS if layer not in layers]
if missing:
problems.append(
f"{name}: {', '.join(sorted(measurers))} measures how work flows and has "
f"no Measure at the {', '.join(missing)} layer; each layer answers a "
"different question and none stands in for another"
)
return problems
def render_outcomes(
graph: Mapping[str, Sequence[Mapping[str, Any]]],
scope: Mapping[str, Any],
names: Mapping[str, str],
) -> str:
"""Each business outcome, with what shows it is served and who answers for it."""
outcomes = spec.business_outcomes(scope)
if not outcomes:
return (
"The customer stated no business outcomes, so nothing in this operating "
"model is tied to one. Ask what the business wants from the solution, "
"and how it would know."
)
roles = {str(r["id"]): r for r in graph["roles"]}
answerable: dict[str, str] = {}
for item in graph["responsibilities"]:
if item.get("kind") == "accountable" and item.get("role") in roles:
answerable[str(item.get("capability"))] = str(
roles[str(item["role"])].get("name")
)
lines = []
for outcome in outcomes:
target = f" Target: {outcome['target']}." if outcome.get("target") else ""
lines += [f"**{outcome['id']}: {outcome.get('outcome')}**{target}", ""]
shown = [m for m in graph["measures"] if m.get("outcome") == outcome["id"]]
if not shown:
lines += ["Nothing measured shows this outcome.", ""]
continue
lines += [
"| Measure | Layer | Framework | Target | Measured by | Answerable |",
"| --- | --- | --- | --- | --- | --- |",
]
lines += [
f"| {m.get('name')} | {str(m.get('layer', '')).replace('_', ' ')} | "
f"{m.get('framework') or ''} | {m.get('target') or ''} | "
f"{names.get(str(m.get('measured_by')), m.get('measured_by'))} | "
f"{answerable.get(str(m.get('measured_by')), '')} |"
for m in shown
]
lines.append("")
return "\n".join(lines).rstrip()
def check_findings(
attachments: Mapping[str, str], scope: Mapping[str, Any], known: set[str]
) -> list[str]:
"""What is wrong with findings.json. It may be an empty list."""
text = attachments.get(FINDINGS_FILE, "")
if not text.strip():
return []
entries, problems = chain.objects(text, FINDINGS_FILE)
item_ids = set(spec.ids(spec.items(scope)))
for entry in entries:
where = f"{FINDINGS_FILE}: {entry['id']}"
for field in ("lacks", "proposed"):
if not str(entry.get(field) or "").strip():
problems.append(f"{where} needs {field}")
about = [str(i) for i in entry.get("about") or []]
needed_by = [str(i) for i in entry.get("needed_by") or []]
if not about:
# A finding is about the solution: it names what in it is lacking.
problems.append(
f"{where} names no Item of the solution under about. What the "
"operating model can provide itself, such as staffing, hours, a Role, "
"a process or a tool, is not a finding: design it"
)
if not needed_by:
problems.append(f"{where} must say what needs it, under needed_by")
problems += [
f"{where} is about {identifier}, which is not an Item of the operating spec"
for identifier in about
if scope and identifier not in item_ids
]
problems += [
f"{where} is needed by {identifier}, which is not in the operating model"
for identifier in needed_by
if identifier not in known
]
return problems
# --- the views code writes -----------------------------------------------------------
def render_matrix(
graph: Mapping[str, Sequence[Mapping[str, Any]]], names: Mapping[str, str]
) -> str:
"""Who is accountable, responsible, consulted and informed for each Capability."""
roles = list(graph["roles"])
# A Role may hold more than one part in a Capability: answerable for it and
# doing it. Both are shown.
held: dict[tuple[str, str], str] = {}
for item in graph["responsibilities"]:
key = (str(item.get("capability")), str(item.get("role")))
letter = _LETTER.get(str(item.get("kind")), "?")
held[key] = f"{held[key]}/{letter}" if key in held else letter
header = "| Capability | " + " | ".join(str(r.get("name")) for r in roles) + " |"
rule = "| --- | " + " | ".join("---" for _ in roles) + " |"
rows = [
f"| {name} ({identifier}) | "
+ " | ".join(held.get((identifier, str(r["id"])), "") for r in roles)
+ " |"
for identifier, name in names.items()
]
legend = "A accountable, R responsible, C consulted, I informed."
return "\n".join([f"## {MATRIX_HEADING}", "", header, rule, *rows, "", legend, ""])
def _node(identifier: Any) -> str:
return str(identifier).replace("-", "_")
def _role_node(role: Mapping[str, Any], indent: str) -> str:
return (
f'{indent}{_node(role["id"])}["{role.get("name")}<br/>{role.get("country")}, '
f'{role.get("level")}, {role.get("hours")}"]'
)
def render_organisation(graph: Mapping[str, Sequence[Mapping[str, Any]]]) -> str:
"""The Parties, their teams and the Roles in each, and how the teams work
together, as a Mermaid chart."""
teams = graph.get("teams") or []
lines = ["flowchart TB"]
for party in graph["parties"]:
lines.append(
f' subgraph {_node(party["id"])}["{party.get("name")} ({party.get("kind")})"]'
)
for team in teams:
if team.get("party") != party["id"]:
continue
kind = str(team.get("type", "")).replace("_", "-")
lines.append(
f' subgraph {_node(team["id"])}["{team.get("name")} ({kind} team)"]'
)
lines += [
_role_node(role, " ")
for role in graph["roles"]
if role.get("team") == team["id"]
]
lines.append(" end")
lines += [
_role_node(role, " ")
for role in graph["roles"]
if role.get("party") == party["id"] and not role.get("team")
]
lines.append(" end")
for item in graph.get("interactions") or []:
mode = str(item.get("mode", "")).replace("_", "-")
label = mode + (f" until {item['until']}" if item.get("until") else "")
lines.append(f' {_node(item["from"])} -- "{label}" --> {_node(item["to"])}')
return "\n".join(lines) + "\n"
def _money(amount: Any, currency: str) -> str:
return "not priced" if amount is None else f"{amount:,} {currency}"
def render_staffing(result: Mapping[str, Any], budget: Any = None) -> str:
"""People and cost per Role and in total, with the prices they were worked from."""
currency = str(result["currency"])
card = result["rate_card"]
conversion = result["conversion"]
lines = [
f"## {STAFFING_HEADING}",
"",
"| Role | Party | Country | Level | Hours | Effort a month | People | Cost a month |",
"| --- | --- | --- | --- | --- | --- | --- | --- |",
]
lines += [
f"| {row['name']} | {row.get('party_name', row['party'])} | "
f"{row['country']} | {row['level']} | "
f"{row['hours']} | {row['effort_hours_per_month']} h | {row['people']} | "
f"{_money(row['cost_per_month'], currency)} |"
for row in result["roles"]
]
lines += [
"",
(
f"{result['people']} people in all. Cost a month: "
f"{_money(result['cost_per_month'], currency)}."
),
]
if isinstance(budget, (int, float)) and not isinstance(budget, bool):
over = result["cost_per_month"] - budget
lines.append(
f"The monthly budget is {_money(round(budget), currency)}: "
+ (
f"this is over it by {_money(round(over), currency)}."
if over > 0
else "this is within it."
)
)
lines += [
"",
f"Worked from rate card `{card['name']}`, version `{card['version']}`"
+ (
". **Its figures are placeholders: no price has been set, and this cost "
"is an illustration.**"
if card["draft"]
else "."
),
]
if conversion["from"] != currency:
lines.append(
f"Converted from {conversion['from']} at {conversion['rate']}, a rate "
f"set on {conversion['as_of']}."
)
lines += [
(
"The customer's own staff are counted and not priced. A Role that covers "
"hours wider than a working day is never smaller than the team that can "
"cover them."
),
"",
]
return "\n".join(lines)
def render_findings(found: Sequence[Mapping[str, Any]]) -> str:
if not found:
return ""
lines = [
f"## {FINDINGS_HEADING}",
"",
(
"What operating this solution needs that the solution, as described, does "
"not provide. Nothing here was designed around: each is for whoever owns "
"the design."
),
"",
]
lines += [
f"- **{item['id']}** ({', '.join(str(i) for i in item.get('about') or []) or 'the solution'}): "
f"{item.get('lacks')} Proposed: {item.get('proposed')}"
for item in found
]
return "\n".join(lines) + "\n"
Views = Callable[[str, dict[str, str], Upstream | None], tuple[str, dict[str, str]]]
def views(catalogue: Catalogue, card: RateCard) -> Views:
"""Add the matrix, the staffing and the findings to an accepted submission."""
from app.operating_model.names import ORGANISATION_FILE, STAFFING_FILE
def add(
content: str, attachments: dict[str, str], upstream: Upstream | None
) -> tuple[str, dict[str, str]]:
graph, _ = parse(attachments[GRAPH_FILE])
scope = spec.agreed(upstream)
capabilities = chain.capabilities(upstream)
currency = str(spec.value(scope, "commercial.currency") or card.currency)
result = staffing.compute(graph, capabilities, card, currency)
found = findings(attachments)
# A Revision replaces these sections, never stacks a second copy.
content = content.split(f"\n## {MATRIX_HEADING}")[0].rstrip()
parts = [
content,
render_matrix(graph, chain.names_of(capabilities, catalogue)),
render_staffing(result, spec.value(scope, "commercial.monthly_run_budget")),
render_findings(found),
]
attachments[ORGANISATION_FILE] = render_organisation(graph)
attachments[STAFFING_FILE] = json.dumps(result, indent=2, ensure_ascii=False)
attachments.setdefault(FINDINGS_FILE, "[]")
return "\n\n".join(part.rstrip() for part in parts if part) + "\n", attachments
return add

View File

@@ -0,0 +1,387 @@
"""The operating model guide: one document, assembled from every other Deliverable.
The packager writes the summary. Every other section is written here, from the data
the other Deliverables carry: the Items and Conditions, the requirements, the
Capabilities, who does what, staffing and cost, the verdicts and the findings. A
table or a figure copied by a model can be copied wrongly; one written from the data
cannot disagree with it.
"""
from __future__ import annotations
import json
import re
from collections.abc import Mapping, Sequence
from typing import Any
from app.operating_model import catalogue as capability_catalogue
from app.operating_model import chain, graph, scenarios, spec
from app.operating_model.catalogue import Catalogue
from app.operating_model.names import (
SCENARIOS_FILE,
STAFFING_FILE,
TARGET_OPERATING_MODEL,
VALIDATION_RESULTS,
VERDICTS_FILE,
)
from app.operating_model.spec import Upstream
SUMMARY = "Summary"
KEY_FIGURES = "Key figures"
# The sections of the guide, in order. The packager writes the first; code writes
# the rest.
SECTIONS = (
SUMMARY,
"Business outcomes",
"What is operated",
"Operational requirements",
"Capabilities",
"Who does what",
"Staffing and cost",
"Validation",
"Findings and open questions",
"Sources",
)
_HEADING = re.compile(r"^##\s+(?:\d+(?:\.\d+)*\.?\s+)?(.+?)\s*#*\s*$", re.MULTILINE)
def _json(deliverables: Upstream, kind: str, name: str) -> Any:
text = (deliverables.get(kind) or {}).get("attachments", {}).get(name, "")
try:
return json.loads(text) if text else None
except ValueError:
return None
def _cell(value: Any) -> str:
"""A value as one cell of a table."""
if isinstance(value, (list, tuple)):
value = ", ".join(str(item) for item in value)
return " ".join(str(value if value is not None else "").split()).replace("|", "\\|")
def _table(headers: Sequence[str], rows: Sequence[Sequence[Any]]) -> list[str]:
return [
"| " + " | ".join(headers) + " |",
"| " + " | ".join("---" for _ in headers) + " |",
*("| " + " | ".join(_cell(value) for value in row) + " |" for row in rows),
]
def _body(rendered: str) -> str:
"""A rendered view without its own heading."""
return (
rendered.split("\n", 1)[1].strip() if rendered.startswith("## ") else rendered
)
def business_outcomes(deliverables: Upstream, catalogue: Catalogue) -> str:
model = graph.read(deliverables) or {
"roles": [],
"responsibilities": [],
"measures": [],
}
names = chain.names_of(chain.capabilities(deliverables), catalogue)
return graph.render_outcomes(model, spec.agreed(deliverables), names)
def what_is_operated(scope: Mapping[str, Any]) -> str:
lines = _table(
("Item", "What it is", "Kind", "Product", "From"),
[
(
i["id"],
i.get("name"),
i.get("kind"),
i.get("product"),
i.get("source_ref"),
)
for i in spec.items(scope)
],
)
conditions = spec.conditions(scope)
if conditions:
lines += ["", "What must be kept true while it is run:", ""]
lines += _table(
("Condition", "On", "What must hold", "From"),
[
(c["id"], c.get("on"), c.get("condition"), c.get("source_ref"))
for c in conditions
],
)
risks = spec.records(scope, "solution.inherited_risks")
if risks:
lines += ["", "Known to be unverified or at risk when operations begin:", ""]
lines += [f"- {r.get('on')}: {r.get('risk')}" for r in risks]
return "\n".join(lines)
def requirements(deliverables: Upstream) -> str:
return "\n".join(
_table(
("Requirement", "Kind", "What must be achieved", "From"),
[
(r["id"], r.get("kind"), r.get("statement"), r.get("source"))
for r in chain.requirements(deliverables)
],
)
)
def capabilities(deliverables: Upstream, catalogue: Catalogue) -> str:
listed = chain.capabilities(deliverables)
names = chain.names_of(listed, catalogue)
return "\n".join(
_table(
("Capability", "What it is", "Serves", "Covers", "Hours", "Effort a month"),
[
(
c["id"],
names[str(c["id"])],
c.get("serves"),
c.get("items"),
c.get("hours"),
f"{c.get('effort_hours_per_month')} h",
)
for c in listed
],
)
)
def who_does_what(deliverables: Upstream, catalogue: Catalogue) -> str:
model = graph.read(deliverables)
if model is None:
return ""
party = {str(p["id"]): p for p in model["parties"]}
lines = _table(
("Role", "Party", "Country", "Level", "Hours"),
[
(
r.get("name"),
(
f"{party.get(str(r.get('party')), {}).get('name')} "
f"({party.get(str(r.get('party')), {}).get('kind')})"
),
r.get("country"),
r.get("level"),
r.get("hours"),
)
for r in model["roles"]
],
)
teams = model.get("teams") or []
if teams:
lines += ["", "The teams, as Team Topologies names their kinds:", ""]
lines += _table(
("Team", "Kind", "Party", "Roles", "Purpose"),
[
(
team.get("name"),
str(team.get("type", "")).replace("_", "-"),
party.get(str(team.get("party")), {}).get("name"),
[
r.get("name")
for r in model["roles"]
if r.get("team") == team["id"]
],
team.get("purpose"),
)
for team in teams
],
)
interactions = model.get("interactions") or []
if interactions:
named = {str(team["id"]): team.get("name") for team in teams}
lines += ["", "How the teams work together:", ""]
lines += _table(
("From", "To", "Mode", "Until", "Purpose"),
[
(
named.get(str(i.get("from")), i.get("from")),
named.get(str(i.get("to")), i.get("to")),
str(i.get("mode", "")).replace("_", "-"),
i.get("until") or "",
i.get("purpose"),
)
for i in interactions
],
)
names = chain.names_of(chain.capabilities(deliverables), catalogue)
return "\n".join([*lines, "", _body(graph.render_matrix(model, names))])
def staffing_and_cost(deliverables: Upstream) -> str:
result = _json(deliverables, TARGET_OPERATING_MODEL, STAFFING_FILE)
if not isinstance(result, dict):
return ""
budget = spec.value(spec.agreed(deliverables), "commercial.monthly_run_budget")
return _body(graph.render_staffing(result, budget))
def validation(deliverables: Upstream) -> str:
verdicts = _json(deliverables, VALIDATION_RESULTS, VERDICTS_FILE) or {}
model = graph.read(deliverables) or {"checks": []}
condition = {str(c["id"]): c.get("condition") for c in model["checks"]}
own = [
item
for item in verdicts.get("checks") or []
if isinstance(item, dict)
and not str(item.get("check", "")).startswith(scenarios.PREFIX)
]
lines = _table(
("Check", "What must hold", "Verdict", "Evidence"),
[
(
v.get("check"),
condition.get(str(v.get("check"))),
str(v.get("verdict")).replace("_", " ")
+ (f" ({v['finding']})" if v.get("finding") else ""),
v.get("evidence"),
)
for v in own
],
)
walked = _json(deliverables, VALIDATION_RESULTS, SCENARIOS_FILE)
if isinstance(walked, list) and walked:
lines += ["", _body(scenarios.render(walked))]
return "\n".join(lines)
def findings_and_questions(deliverables: Upstream) -> str:
scope = spec.agreed(deliverables)
found = graph.findings(
(deliverables.get(TARGET_OPERATING_MODEL) or {}).get("attachments", {})
)
lines = []
if found:
lines += [_body(graph.render_findings(found)), ""]
else:
lines += ["No findings: the solution provides what operating it needs.", ""]
assumed = [a for a in scope.get("assumed") or [] if isinstance(a, Mapping)]
if assumed:
lines += ["Assumed for the customer, and open to change when they answer:", ""]
lines += [
f"- `{a.get('field')}`: {_cell(a.get('value'))}. {a.get('why') or ''}".rstrip()
for a in assumed
]
lines.append("")
undecided = [d for d in scope.get("to_decide") or [] if isinstance(d, Mapping)]
if undecided:
lines += ["Left to the agent by the customer, and not yet settled:", ""]
lines += [
f"- `{d.get('field')}`: affects {d.get('affects') or 'the operating model'}."
for d in undecided
]
lines.append("")
# What matters to an operating model and is not known. The optional fields
# nobody was asked for are not open questions.
unknown = [
g
for g in scope.get("to_ask") or []
if isinstance(g, Mapping) and g.get("severity") in ("HIGH", "MEDIUM")
]
if unknown:
lines += ["Still unknown:", ""]
lines += [
f"- `{g.get('field')}`: {g.get('prompt') or ''}".rstrip() for g in unknown
]
return "\n".join(lines).rstrip()
def sources(deliverables: Upstream, catalogue: Catalogue) -> str:
entries = [str(c.get("catalogue")) for c in chain.capabilities(deliverables)]
lines = [
(
"The Capabilities are from our own capability catalogue, which draws on "
"these frameworks. It cites them and certifies nothing against them."
),
"",
*(f"- {line}" for line in capability_catalogue.cited(catalogue, entries)),
]
if not catalogue.verified:
lines += [
"",
(
"**These references have not been checked against the frameworks by a "
"person.** Treat each as a pointer to where to look."
),
]
return "\n".join(lines)
def key_figures(deliverables: Upstream) -> str:
"""The figures a reader takes from the summary, written from the data."""
scope = spec.agreed(deliverables)
cost = _json(deliverables, TARGET_OPERATING_MODEL, STAFFING_FILE)
walked = _json(deliverables, VALIDATION_RESULTS, SCENARIOS_FILE) or []
found = graph.findings(
(deliverables.get(TARGET_OPERATING_MODEL) or {}).get("attachments", {})
)
lines = [
(
f"- Operated: {len(spec.items(scope))} Items, with "
f"{len(spec.conditions(scope))} Conditions to keep true."
),
f"- Faults responded to: {spec.value(scope, 'operations.support_hours')} hours.",
f"- Capabilities: {len(chain.capabilities(deliverables))}.",
]
if isinstance(cost, dict):
card = cost["rate_card"]
lines.append(
f"- People: {cost['people']}. Cost a month: {cost['cost_per_month']:,} "
f"{cost['currency']}, from rate card version `{card['version']}`"
+ (" (placeholder prices: an illustration)." if card["draft"] else ".")
)
if walked:
passed = sum(1 for item in walked if item.get("passes"))
lines.append(f"- Scenario walk-throughs: {passed} of {len(walked)} pass.")
outcomes = spec.business_outcomes(scope)
model = graph.read(deliverables)
if outcomes:
shown = {str(m.get("outcome")) for m in (model or {}).get("measures", [])}
lines.append(
f"- Business outcomes: {len(outcomes)}, "
f"{sum(1 for o in outcomes if str(o['id']) in shown)} shown by a Measure."
)
else:
lines.append("- Business outcomes: none stated.")
lines.append(f"- Findings for whoever owns the design: {len(found)}.")
return "\n".join(lines)
def assemble(content: str, deliverables: Upstream, catalogue: Catalogue) -> str:
"""The guide, with every section but the summary written from the data.
The packager's summary is kept as written, and the key figures are set under it.
Whatever it wrote under another heading is replaced.
"""
written = {
"Business outcomes": business_outcomes(deliverables, catalogue),
"What is operated": what_is_operated(spec.agreed(deliverables)),
"Operational requirements": requirements(deliverables),
"Capabilities": capabilities(deliverables, catalogue),
"Who does what": who_does_what(deliverables, catalogue),
"Staffing and cost": staffing_and_cost(deliverables),
"Validation": validation(deliverables),
"Findings and open questions": findings_and_questions(deliverables),
"Sources": sources(deliverables, catalogue),
}
headings = list(_HEADING.finditer(content))
title = content[: headings[0].start()].rstrip() if headings else content.rstrip()
summary = ""
for index, match in enumerate(headings):
if match.group(1).strip().lower() == SUMMARY.lower():
end = (
headings[index + 1].start()
if index + 1 < len(headings)
else len(content)
)
summary = content[match.end() : end].strip()
summary = summary.split(f"\n### {KEY_FIGURES}")[0].rstrip()
parts = [
title,
f"## {SUMMARY}\n\n{summary}\n\n### {KEY_FIGURES}\n\n{key_figures(deliverables)}",
*(f"## {heading}\n\n{written[heading]}" for heading in SECTIONS[1:]),
]
return "\n\n".join(part.rstrip() for part in parts if part.strip()) + "\n"

View File

@@ -0,0 +1,66 @@
"""The names this Workflow is written in: its Phases, its Deliverables and its files."""
from __future__ import annotations
# Phases, in order.
SCOPING = "scoping"
REQUIREMENTS = "operational_requirements"
DESIGN = "operating_model_design"
VALIDATION = "validation"
PACKAGING = "packaging"
PHASES = (SCOPING, REQUIREMENTS, DESIGN, VALIDATION, PACKAGING)
# Deliverable kinds.
OPERATING_SCOPE = "operating_scope"
OPERATIONAL_REQUIREMENTS = "operational_requirements"
CAPABILITY_MODEL = "capability_model"
TARGET_OPERATING_MODEL = "target_operating_model"
VALIDATION_RESULTS = "validation_results"
GUIDE = "operating_model_guide"
# Data files.
SCOPE_DATA_FILE = "scope.json" # the engine's: what every Specialist reads of the spec
REQUIREMENTS_FILE = "operational-requirements.json"
CAPABILITIES_FILE = "capabilities.json"
GRAPH_FILE = "target-operating-model.json"
FINDINGS_FILE = "findings.json"
STAFFING_FILE = "staffing.json"
ORGANISATION_FILE = "organisation.mmd"
VERDICTS_FILE = "verdicts.json"
SCENARIOS_FILE = "scenarios.json"
CHANGES_FILE = "changes.json"
GUIDE_FILE = "operating-model-guide.md"
CHANGE_REQUEST_FILE = "architecture-change-request.md"
# Hours a Capability must be staffed for and a Role works, narrowest first.
HOURS = ("business", "extended", "24x7")
PARTY_KINDS = ("customer", "provider", "supplier", "cloud_provider")
CUSTOMER = "customer"
ACCOUNTABLE, RESPONSIBLE, CONSULTED, INFORMED = (
"accountable",
"responsible",
"consulted",
"informed",
)
RESPONSIBILITY_KINDS = (ACCOUNTABLE, RESPONSIBLE, CONSULTED, INFORMED)
CUSTOM = "custom" # a Capability the catalogue does not have
# Team Topologies: the four kinds of team, and the three ways two teams work together.
TEAM_TYPES = ("stream_aligned", "platform", "enabling", "complicated_subsystem")
COLLABORATION, X_AS_A_SERVICE, FACILITATING = (
"collaboration",
"x_as_a_service",
"facilitating",
)
INTERACTION_MODES = (COLLABORATION, X_AS_A_SERVICE, FACILITATING)
# What a Measure tells about a business outcome. The first three are the layers of
# measuring how work flows (DORA, SPACE, DevEx); none stands in for another.
OUTCOME_LAYER, GUARDRAIL_LAYER, LEADING_LAYER = "outcome", "guardrail", "leading"
DEVELOPER_LAYERS = (OUTCOME_LAYER, GUARDRAIL_LAYER, LEADING_LAYER)
MEASURE_LAYERS = (*DEVELOPER_LAYERS, "service_level", "cost")
# The catalogue entry whose measures must cover all three developer layers.
MEASUREMENT_ENTRY = "delivery-and-developer-experience-measurement"
def wider(first: str, second: str) -> bool:
"""Whether `first` hours are wider than `second`."""
return HOURS.index(first) > HOURS.index(second)

View File

@@ -0,0 +1,346 @@
"""Scenario walk-throughs: events the operating model must be able to handle.
Infrastructure code can be validated by running it. An operating model cannot, so
each scenario is generated from the graph and the operating spec and traced through
the graph by code: is there a Capability for it, is someone responsible at the hours
it happens, is one Role answerable. A broken path is a failed Check. No model writes
a scenario or judges one.
"""
from __future__ import annotations
import json
from collections.abc import Mapping, Sequence
from typing import Any
from app.operating_model import spec
from app.operating_model.catalogue import Catalogue
from app.operating_model.names import (
ACCOUNTABLE,
CONSULTED,
CUSTOMER,
HOURS,
RESPONSIBLE,
wider,
)
PREFIX = "SC-" # a scenario's id; the validator gives no verdict under it
# The verdict on a Check that cannot hold until a finding is met. It is neither a
# failure of the model nor a gap in it: nothing is revised for it, and it is
# reported with the finding it waits on.
WAITING = "waits_on_finding"
HEADING = "Scenario walk-throughs"
CHANGE_DOMAIN = "change"
def _scenario(
identifier: str, scenario: str, about: str, fault: str | None, path: str
) -> dict[str, Any]:
return {
"id": f"{PREFIX}{identifier}",
"scenario": scenario,
"about": about,
"passes": fault is None,
"path": fault or path,
}
def walk(
graph: Mapping[str, Sequence[Mapping[str, Any]]],
capabilities: Sequence[Mapping[str, Any]],
scope: Mapping[str, Any],
catalogue: Catalogue,
staffing: Mapping[str, Any] | None = None,
) -> list[dict[str, Any]]:
"""Every scenario for this operating model, traced."""
roles = {str(r["id"]): r for r in graph["roles"]}
parties = {str(p["id"]): p for p in graph["parties"]}
by_capability: dict[str, list[Mapping[str, Any]]] = {}
for item in graph["responsibilities"]:
by_capability.setdefault(str(item.get("capability")), []).append(item)
def holders(capability: str, kind: str) -> list[Mapping[str, Any]]:
return [
roles[str(r["role"])]
for r in by_capability.get(capability, [])
if r.get("kind") == kind and str(r.get("role")) in roles
]
def entry_of(capability: Mapping[str, Any]) -> Any:
return catalogue.entries.get(str(capability.get("catalogue")))
def covering(item: str) -> list[Mapping[str, Any]]:
return [c for c in capabilities if item in (c.get("items") or [])]
support = str(spec.value(scope, "operations.support_hours") or HOURS[0])
found = []
for item in spec.items(scope):
item_id, label = str(item["id"]), f"{item.get('name')} ({item['id']})"
responders = [
c
for c in covering(item_id)
if entry_of(c) and entry_of(c).follows_support_hours
]
fault = None
if not responders:
fault = "No Capability that responds to faults covers it."
else:
on_hours = [
(c, role)
for c in responders
for role in holders(str(c["id"]), RESPONSIBLE)
if str(role.get("hours")) in HOURS
and not wider(support, str(role.get("hours")))
]
answerable = [c for c, _ in on_hours if holders(str(c["id"]), ACCOUNTABLE)]
if not on_hours:
fault = f"No Role responsible for responding works {support} hours."
elif not answerable:
fault = "The Capability that responds has no accountable Role."
chosen = None if fault else on_hours[0]
found.append(
_scenario(
f"fault-{item_id}",
f"{label} fails at an hour faults are responded to ({support}).",
item_id,
fault,
""
if chosen is None
else f"{chosen[1].get('name')} responds under {chosen[0]['id']}, and "
f"{holders(str(chosen[0]['id']), ACCOUNTABLE)[0].get('name')} is "
"answerable.",
)
)
changers = [
c
for c in covering(item_id)
if entry_of(c)
and entry_of(c).domain == CHANGE_DOMAIN
and holders(str(c["id"]), RESPONSIBLE)
]
found.append(
_scenario(
f"change-{item_id}",
f"{label} must be changed.",
item_id,
None
if changers
else "No Capability for making changes covers it with a responsible Role.",
""
if not changers
else f"{holders(str(changers[0]['id']), RESPONSIBLE)[0].get('name')} "
f"makes the change under {changers[0]['id']}.",
)
)
capability_ids = {str(c["id"]) for c in capabilities}
for condition in spec.conditions(scope):
condition_id = str(condition["id"])
carrying = [k for k in graph["checks"] if k.get("source") == condition_id]
kept = [
k
for k in carrying
if str(k.get("on")) not in capability_ids
or holders(str(k.get("on")), RESPONSIBLE)
]
found.append(
_scenario(
f"condition-{condition_id}",
f"Condition {condition_id} must be shown to hold: "
f"{condition.get('condition')}",
condition_id,
None
if kept
else (
"No Check carries it."
if not carrying
else "The Check that carries it is on a Capability nobody is responsible for."
),
""
if not kept
else f"Check {kept[0]['id']} on {kept[0].get('on')} keeps it true.",
)
)
for party in graph["parties"]:
if party.get("kind") == CUSTOMER:
continue
party_id = str(party["id"])
stranded = [
str(c["id"])
for c in capabilities
if any(
a.get("party") == party_id for a in holders(str(c["id"]), ACCOUNTABLE)
)
and not any(
parties.get(str(role.get("party")), {}).get("id") != party_id
for kind in (RESPONSIBLE, CONSULTED)
for role in holders(str(c["id"]), kind)
)
]
found.append(
_scenario(
f"party-{party_id}",
f"{party.get('name')} cannot be reached.",
party_id,
None
if not stranded
else "No Role in another Party is responsible or consulted for "
+ ", ".join(stranded)
+ ".",
"Every Capability it is answerable for has a Role in another Party to turn to.",
)
)
budget = spec.value(scope, "commercial.monthly_run_budget")
if (
staffing is not None
and isinstance(budget, (int, float))
and not isinstance(budget, bool)
):
cost, currency = staffing["cost_per_month"], staffing["currency"]
found.append(
_scenario(
"budget",
f"A month's cost of operating is compared with the budget of {round(budget):,} {currency}.",
"commercial.monthly_run_budget",
None
if cost <= budget
else f"The cost is {cost:,} {currency}, over the budget.",
f"The cost is {cost:,} {currency}, within the budget.",
)
)
return found
def as_verdicts(scenarios: Sequence[Mapping[str, Any]]) -> list[dict[str, str]]:
return [
{
"check": str(item["id"]),
"verdict": "pass" if item["passes"] else "fail",
"evidence": f"{item['scenario']} {item['path']}",
}
for item in scenarios
]
def merge(
verdicts_text: str,
scenarios: Sequence[Mapping[str, Any]],
waiting: Mapping[str, str] | None = None,
) -> str:
"""verdicts.json with the scenarios' verdicts in it, written by code.
The validator's verdicts on the Checks are kept as given, but for one thing:
a Check in `waiting`, which names the finding it depends on, is recorded as
waiting on that finding. The overall run is the walk-through: it passes when
every scenario does.
"""
try:
data = json.loads(verdicts_text)
except ValueError:
data = {}
if not isinstance(data, dict):
data = {}
given = [
(
{**item, "verdict": WAITING, "finding": waiting[str(item.get("check"))]}
# Only where the validator found the finding stands. One it failed is
# something the operating model can provide itself, and is a fault in it.
if waiting
and str(item.get("check")) in waiting
and item.get("verdict") in ("not_verifiable", WAITING)
else item
)
for item in data.get("checks") or []
if isinstance(item, dict) and not str(item.get("check", "")).startswith(PREFIX)
]
failed = [item for item in scenarios if not item["passes"]]
return json.dumps(
{
"dry_run": {
"verdict": "fail" if failed else "pass",
"evidence": f"{len(scenarios) - len(failed)} of {len(scenarios)} "
"scenario walk-throughs pass.",
},
"checks": [*given, *as_verdicts(scenarios)],
"proposed_additions": data.get("proposed_additions") or [],
},
indent=2,
ensure_ascii=False,
)
def render(scenarios: Sequence[Mapping[str, Any]]) -> str:
lines = [
f"## {HEADING}",
"",
(
"Each scenario is generated from the operating model and traced through "
"it by code."
),
"",
"| Scenario | Result | Path |",
"| --- | --- | --- |",
]
lines += [
f"| {item['scenario']} | {'pass' if item['passes'] else '**fail**'} | {item['path']} |"
for item in scenarios
]
return "\n".join(lines) + "\n"
def check_verdicts(
attachments: Mapping[str, str],
graph: Mapping[str, Sequence[Mapping[str, Any]]] | None,
name: str,
) -> list[str]:
"""What is wrong with the validator's verdicts: one on every Check, none on a
scenario, and never a pass on a Check that depends on a finding."""
try:
data = json.loads(attachments[name])
except ValueError as exc:
return [f"{name}: not valid JSON ({exc})"]
if not isinstance(data, dict) or not isinstance(data.get("checks"), list):
return [f"{name}: must be an object with a list of checks"]
problems = []
given: dict[str, str] = {}
for index, item in enumerate(data["checks"]):
where = f"{name}: checks[{index}]"
if not isinstance(item, dict) or not item.get("check"):
problems.append(f"{where} needs a check")
continue
identifier = str(item["check"])
if identifier.startswith(PREFIX):
continue # the walk-throughs' own, written by code on an earlier attempt
if item.get("verdict") == WAITING:
# Recorded by code on an earlier attempt; the validator's own verdict
# on such a Check is not_verifiable.
item = {**item, "verdict": "not_verifiable"}
if item.get("verdict") not in ("pass", "fail", "not_verifiable"):
problems.append(
f"{where} ({identifier}): verdict must be pass, fail or not_verifiable"
)
if not str(item.get("evidence") or "").strip():
problems.append(f"{where} ({identifier}) needs evidence")
given[identifier] = str(item.get("verdict"))
for check in (graph or {}).get("checks", []):
identifier = str(check["id"])
if identifier not in given:
problems.append(f"{name}: Check {identifier} has no verdict")
elif check.get("depends_on_finding") and given[identifier] == "pass":
problems.append(
f"{name}: Check {identifier} depends on finding "
f"{check['depends_on_finding']}. It is not_verifiable while the finding "
"stands, or fail when the operating model could meet it itself"
)
known = {str(check["id"]) for check in (graph or {}).get("checks", [])}
if graph is not None:
problems += [
f"{name}: {identifier} is not a Check of the Target operating model"
for identifier in given
if identifier not in known
]
return problems

View File

@@ -0,0 +1,89 @@
"""Reading the agreed operating spec, as every Specialist is given it.
The engine writes the spec's data file from what the scope analyst recorded: what
the customer stated, nested by field; what was assumed for them, as a list; and
which stated fields bind. Code that checks a Deliverable against the spec reads it
here, and a value counts whichever of the two it came from.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from typing import Any
from app.operating_model.names import OPERATING_SCOPE, SCOPE_DATA_FILE
Upstream = Mapping[str, Mapping[str, Any]]
def agreed(upstream: Upstream | None) -> dict[str, Any]:
"""The agreed spec from the Deliverables a submission builds on; {} without one."""
text = (
((upstream or {}).get(OPERATING_SCOPE) or {})
.get("attachments", {})
.get(SCOPE_DATA_FILE, "")
)
try:
data = json.loads(text) if text else {}
except ValueError:
return {}
return data if isinstance(data, dict) else {}
def value(scope: Mapping[str, Any], path: str, default: Any = None) -> Any:
"""A field's value, stated or assumed; `default` when it is neither."""
node: Any = scope.get("stated") or {}
for part in path.split("."):
if not isinstance(node, Mapping) or part not in node:
node = None
break
node = node[part]
if node is not None:
return node
for item in scope.get("assumed") or []:
if isinstance(item, Mapping) and item.get("field") == path:
return item.get("value", default)
return default
def settled(scope: Mapping[str, Any], path: str) -> bool:
return value(scope, path) is not None
def records(scope: Mapping[str, Any], path: str) -> list[dict[str, Any]]:
found = value(scope, path)
return (
[dict(item) for item in found if isinstance(item, Mapping)]
if isinstance(found, list)
else []
)
def items(scope: Mapping[str, Any]) -> list[dict[str, Any]]:
"""What is to be operated."""
return records(scope, "solution.items")
def conditions(scope: Mapping[str, Any]) -> list[dict[str, Any]]:
"""What must be kept true of an Item while it is run."""
return records(scope, "solution.conditions")
def business_outcomes(scope: Mapping[str, Any]) -> list[dict[str, Any]]:
"""What the customer's business wants from the solution, in their words: what
every measure of operations is ultimately for."""
return records(scope, "intent.business_outcomes")
def stated_requirements(scope: Mapping[str, Any]) -> list[dict[str, Any]]:
return records(scope, "intent.requirements")
def binding(scope: Mapping[str, Any]) -> list[str]:
"""The stated fields an operating model must honour."""
return [str(path) for path in scope.get("binding") or []]
def ids(entries: list[dict[str, Any]]) -> list[str]:
return [str(entry["id"]) for entry in entries if entry.get("id")]

View File

@@ -0,0 +1,280 @@
"""Staffing and cost, computed from the Target operating model and the rate card.
No agent writes a number of people or a cost. A Specialist says what each Capability
takes in hours a month, which Roles are responsible for it, and where each Role sits
and at what level. Everything else is arithmetic on the service provider's rate card,
so the figures cannot disagree with the model they are about.
effort the hours a month of the Capabilities a Role is responsible for
people effort over the productive hours of its country; and, where the
Role covers hours wider than a working day, never fewer than the
smallest team that can cover them
cost a month people, at the day rate for its country and level, for the
productive days in a month, with the uplift for its hours, in
the currency the operating spec states
"""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import yaml
from app.operating_model.names import CUSTOMER, HOURS, RESPONSIBLE, wider
SKILL = "rate-card"
BUSINESS = HOURS[0]
class RateCardError(ValueError):
"""A rate card that cannot be used; the message lists every fault."""
@dataclass(frozen=True)
class Country:
name: str
productive_days: float
hours_per_day: float
day_rate: Mapping[str, float]
@property
def productive_hours_a_month(self) -> float:
return self.productive_days * self.hours_per_day / 12
@dataclass(frozen=True)
class RateCard:
name: str
version: str
currency: str # what its rates are written in
conversion: Mapping[str, float] # units of a currency for one of `currency`
conversion_as_of: str
prices: tuple[str, ...] # the kinds of Party it prices
levels: Mapping[str, str]
cover: Mapping[str, Mapping[str, float]] # hours -> minimum_roster, uplift
countries: Mapping[str, Country]
@property
def is_draft(self) -> bool:
return self.version.startswith("draft")
def currencies(self) -> list[str]:
return sorted(self.conversion)
def _number(value: Any) -> bool:
return isinstance(value, (int, float)) and not isinstance(value, bool) and value > 0
def load(skill_dir: Path) -> RateCard:
"""Read the rate card from the Skill's folder, and check it holds together."""
path = skill_dir / "assets" / "rates.yaml"
data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
if not isinstance(data, dict):
raise RateCardError("rates.yaml: must be a mapping")
problems = [
f"rates.yaml: needs {field}"
for field in ("name", "version", "currency", "levels", "cover", "countries")
if not data.get(field)
]
if problems:
raise RateCardError("; ".join(problems))
levels = {str(name): str(text) for name, text in data["levels"].items()}
conversion_data = data.get("conversion") or {}
rates = {str(k): v for k, v in (conversion_data.get("rates") or {}).items()}
rates.setdefault(str(data["currency"]), 1.0)
problems += [
f"rates.yaml: the conversion rate for {name} must be a number above zero"
for name, rate in rates.items()
if not _number(rate)
]
prices = tuple(str(kind) for kind in data.get("prices") or ())
if CUSTOMER in prices:
problems.append(
"rates.yaml: `prices` lists customer; the customer's own staff are "
"never priced"
)
cover = {str(name): dict(entry or {}) for name, entry in data["cover"].items()}
for hours in HOURS:
entry = cover.get(hours) or {}
if not _number(entry.get("minimum_roster")) or not _number(entry.get("uplift")):
problems.append(
f"rates.yaml: cover for {hours} needs a minimum_roster and an "
"uplift, both above zero"
)
countries = {}
for code, entry in data["countries"].items():
entry = entry or {}
where = f"rates.yaml: {code}"
if not _number(entry.get("productive_days")) or not _number(
entry.get("hours_per_day")
):
problems.append(f"{where} needs productive_days and hours_per_day")
continue
day_rate = entry.get("day_rate") or {}
problems += [
f"{where} has no day rate for level {level}"
for level in levels
if not _number(day_rate.get(level))
]
countries[str(code)] = Country(
name=str(entry.get("name") or code),
productive_days=float(entry["productive_days"]),
hours_per_day=float(entry["hours_per_day"]),
day_rate={str(k): float(v) for k, v in day_rate.items() if _number(v)},
)
if problems:
raise RateCardError("; ".join(problems))
return RateCard(
name=str(data["name"]),
version=str(data["version"]),
currency=str(data["currency"]),
conversion={name: float(rate) for name, rate in rates.items()},
conversion_as_of=str(conversion_data.get("as_of") or ""),
prices=prices,
levels=levels,
cover={
name: {
"minimum_roster": float(entry["minimum_roster"]),
"uplift": float(entry["uplift"]),
}
for name, entry in cover.items()
if name in HOURS
},
countries=countries,
)
def shares(
responsibilities: Sequence[Mapping[str, Any]],
) -> tuple[dict[tuple[str, str], float], list[str]]:
"""Each responsible Role's share of each Capability's effort, and what is wrong
with how shares were stated.
The Roles responsible for a Capability divide its effort equally, unless every
one of them states a `share`, and the shares add up to 1.
"""
by_capability: dict[str, list[Mapping[str, Any]]] = {}
for item in responsibilities:
if item.get("kind") == RESPONSIBLE:
by_capability.setdefault(str(item.get("capability")), []).append(item)
result: dict[tuple[str, str], float] = {}
problems = []
for capability, entries in by_capability.items():
stated = [e.get("share") for e in entries if e.get("share") is not None]
if not stated:
for entry in entries:
result[(capability, str(entry.get("role")))] = 1 / len(entries)
continue
valid = all(
isinstance(s, (int, float)) and not isinstance(s, bool) and 0 < s <= 1
for s in stated
)
if len(stated) != len(entries) or not valid:
problems.append(
f"the Roles responsible for {capability} must all state a share "
"between 0 and 1, or none of them"
)
continue
total = sum(float(entry["share"]) for entry in entries)
if abs(total - 1) > 0.001:
problems.append(f"the shares of {capability} do not add up to 1")
continue
for entry in entries:
result[(capability, str(entry.get("role")))] = float(entry["share"])
return result, problems
def compute(
graph: Mapping[str, Any],
capabilities: Sequence[Mapping[str, Any]],
card: RateCard,
currency: str,
) -> dict[str, Any]:
"""People and cost per Role, and the totals. The graph is taken to be sound:
`graph.check` refuses one whose Roles the card cannot price."""
parties = {str(p.get("id")): p for p in graph.get("parties") or []}
effort_of = {
str(c["id"]): float(c.get("effort_hours_per_month") or 0) for c in capabilities
}
hours_of = {str(c["id"]): str(c.get("hours") or BUSINESS) for c in capabilities}
share_of, _ = shares(graph.get("responsibilities") or [])
rate = card.conversion[currency]
rows: list[dict[str, Any]] = []
by_capability: dict[str, float] = {}
for role in graph.get("roles") or []:
role_id = str(role.get("id"))
country = card.countries[str(role.get("country"))]
own = {cap: share for (cap, who), share in share_of.items() if who == role_id}
effort = float(
sum(effort_of.get(cap, 0.0) * share for cap, share in own.items())
)
hours = BUSINESS
for cap in own:
if wider(hours_of.get(cap, BUSINESS), hours):
hours = hours_of[cap]
people = effort / country.productive_hours_a_month
if hours != BUSINESS:
# A rota is sized by the hours it must fill, not by the work done in them.
people = max(people, card.cover[hours]["minimum_roster"])
party = parties.get(str(role.get("party")), {})
priced = party.get("kind") != CUSTOMER and party.get("kind") in card.prices
cost = (
people
* country.day_rate[str(role.get("level"))]
* country.productive_days
/ 12
* card.cover[hours]["uplift"]
* rate
if priced
else None
)
if cost is not None and effort:
for cap, share in own.items():
portion = effort_of.get(cap, 0.0) * share / effort
by_capability[cap] = by_capability.get(cap, 0.0) + cost * portion
rows.append(
{
"role": role_id,
"name": str(role.get("name") or role_id),
"party": str(role.get("party")),
"party_name": str(party.get("name") or role.get("party")),
"country": str(role.get("country")),
"level": str(role.get("level")),
"hours": hours,
"effort_hours_per_month": round(effort, 1),
"people": round(people, 1),
"cost_per_month": None if cost is None else round(cost),
}
)
def total(key: str) -> dict[str, int]:
sums: dict[str, float] = {}
for row in rows:
if row["cost_per_month"] is not None:
sums[row[key]] = sums.get(row[key], 0.0) + row["cost_per_month"]
return {name: round(amount) for name, amount in sums.items()}
return {
"currency": currency,
"rate_card": {
"name": card.name,
"version": card.version,
"draft": card.is_draft,
},
"conversion": {
"from": card.currency,
"rate": rate,
"as_of": card.conversion_as_of,
},
"roles": rows,
"people": round(sum(row["people"] for row in rows), 1),
"cost_per_month": round(sum(row["cost_per_month"] or 0 for row in rows)),
"by_party": total("party"),
"by_country": total("country"),
"by_capability": {cap: round(amount) for cap, amount in by_capability.items()},
}

View File

@@ -0,0 +1,572 @@
"""The target operating model, as a Workflow for the Execution engine.
`build(root)` reads the capability catalogue and the rate card from the agent's
Skills and returns the one value that makes the engine about this subject: the
Phases and Deliverables, the checks along the chain, the graph and its views, the
scenario walk-throughs that stand in for a dry run, and what a Change request
changed.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from pathlib import Path
from typing import Any
from execution_engine.workflows.deliverables import DeliverableSpec
from execution_engine.workflows.workflow import (
Bench,
CardText,
ChangePolicy,
ValidationPolicy,
Workflow,
)
from app.operating_model import catalogue as capability_catalogue
from app.operating_model import chain, digest, graph, guide, scenarios, spec, staffing
from app.operating_model.catalogue import Catalogue
from app.operating_model.names import (
CAPABILITIES_FILE,
CAPABILITY_MODEL,
CHANGE_REQUEST_FILE,
CHANGES_FILE,
DESIGN,
FINDINGS_FILE,
GRAPH_FILE,
GUIDE,
GUIDE_FILE,
OPERATING_SCOPE,
OPERATIONAL_REQUIREMENTS,
PACKAGING,
PHASES,
REQUIREMENTS,
REQUIREMENTS_FILE,
SCENARIOS_FILE,
SCOPE_DATA_FILE,
SCOPING,
STAFFING_FILE,
TARGET_OPERATING_MODEL,
VALIDATION,
VALIDATION_RESULTS,
VERDICTS_FILE,
)
from app.operating_model.spec import Upstream
from app.operating_model.staffing import RateCard
ROOT = Path(__file__).resolve().parents[2] # the agent's own folder
SCOPE_FILE = "scope.yaml" # the engine's: the completed operating spec
ABOUT = (
"I design target operating models. Give me the solution to be operated, as an "
"operating spec or as a solution architecture to digest into one, and I will "
"work through operational requirements, the Capabilities needed, who is "
"accountable and responsible for each, staffing and cost, and a validation of "
"the model against the events it must handle, asking for your approval at each "
"Phase. Nothing was started for this message."
)
def _staffing(deliverables: Upstream) -> dict[str, Any] | None:
text = (
(deliverables.get(TARGET_OPERATING_MODEL) or {})
.get("attachments", {})
.get(STAFFING_FILE, "")
)
try:
data = json.loads(text) if text else None
except ValueError:
return None
return data if isinstance(data, dict) else None
def _findings(deliverables: Upstream) -> list[dict[str, Any]]:
return graph.findings(
(deliverables.get(TARGET_OPERATING_MODEL) or {}).get("attachments", {})
)
def _waiting(deliverables: Upstream) -> dict[str, str]:
"""The Checks that cannot hold until a finding is met, with the finding."""
text = (
(deliverables.get(VALIDATION_RESULTS) or {})
.get("attachments", {})
.get(VERDICTS_FILE, "")
)
try:
data = json.loads(text) if text else {}
except ValueError:
return {}
return {
str(item.get("check")): str(item.get("finding"))
for item in (data.get("checks") or [] if isinstance(data, dict) else [])
if isinstance(item, dict) and item.get("verdict") == scenarios.WAITING
}
def approval_facts(phase: str, deliverables: Upstream) -> dict[str, Any]:
"""This Workflow's own facts for an Approval summary: counts and cost, as data."""
scope = spec.agreed(deliverables)
model = graph.read(deliverables)
facts: dict[str, Any] = {
"counts": {
name: count
for name, count in (
("items", len(spec.items(scope))),
("conditions", len(spec.conditions(scope))),
("business_outcomes", len(spec.business_outcomes(scope))),
("operational_requirements", len(chain.requirements(deliverables))),
("capabilities", len(chain.capabilities(deliverables))),
("roles", len(model["roles"]) if model else 0),
("checks", len(model["checks"]) if model else 0),
("measures", len(model["measures"]) if model else 0),
)
if count
}
}
cost = _staffing(deliverables)
if cost:
facts["staffing"] = {
"people": cost["people"],
"cost_per_month": cost["cost_per_month"],
"currency": cost["currency"],
"rate_card": cost["rate_card"],
}
found = _findings(deliverables)
if found:
facts["findings"] = [f"{item['id']}: {item.get('lacks')}" for item in found]
waiting = _waiting(deliverables)
if waiting:
facts["checks_waiting_on_findings"] = waiting
return facts
def summary_lines(phase: str, facts: Mapping[str, Any]) -> list[str]:
lines = []
counts = facts.get("counts") or {}
if counts:
lines.append(
", ".join(
f"{count} {name.replace('_', ' ')}" for name, count in counts.items()
)
+ "."
)
cost = facts.get("staffing")
if cost:
lines.append(
f"{cost['people']} people, {cost['cost_per_month']:,} {cost['currency']} "
"a month"
+ (
", from placeholder prices."
if cost["rate_card"]["draft"]
else f", from rate card version {cost['rate_card']['version']}."
)
)
found = facts.get("findings")
if found:
lines.append(
f"{len(found)} findings for whoever owns the design: " + "; ".join(found)
)
waiting = facts.get("checks_waiting_on_findings")
if waiting:
lines.append(
"Waiting on a finding, and so not yet shown to hold: "
+ ", ".join(f"{check} ({finding})" for check, finding in waiting.items())
+ "."
)
return lines
def describe_approval(approval: Mapping[str, Any]) -> str:
changes = approval.get("changes")
if not changes:
return ""
lines = [
f"This changes the operating model delivered in {approval.get('baseline')}."
]
lines += [
f"- {word} {part.replace('_', ' ')}: {', '.join(ids)}"
for part, by_change in changes.items()
for word, ids in (
("Added", by_change.get("added")),
("Removed", by_change.get("removed")),
("Changed", by_change.get("changed")),
)
if ids
]
return "\n".join(lines) + "\n\n"
# --- what a Change request changed -----------------------------------------------------
_PARTS = {
"operational_requirements": (OPERATIONAL_REQUIREMENTS, REQUIREMENTS_FILE, None),
"capabilities": (CAPABILITY_MODEL, CAPABILITIES_FILE, None),
"roles": (TARGET_OPERATING_MODEL, GRAPH_FILE, "roles"),
"responsibilities": (TARGET_OPERATING_MODEL, GRAPH_FILE, "responsibilities"),
"checks": (TARGET_OPERATING_MODEL, GRAPH_FILE, "checks"),
}
def _entries(deliverables: Upstream | None, part: str) -> dict[str, dict[str, Any]]:
kind, name, within = _PARTS[part]
text = ((deliverables or {}).get(kind) or {}).get("attachments", {}).get(name, "")
if not text:
return {}
listed = graph.parse(text)[0][within] if within else chain.objects(text, name)[0]
return {str(item["id"]): item for item in listed if item.get("id")}
def change_summary(before: Upstream | None, after: Upstream) -> dict[str, Any]:
"""What was added, removed and changed against the Baseline, by id."""
summary = {}
for part in _PARTS:
was, now = _entries(before, part), _entries(after, part)
summary[part] = {
"added": sorted(set(now) - set(was)),
"removed": sorted(set(was) - set(now)),
"changed": sorted(k for k in set(was) & set(now) if was[k] != now[k]),
}
return summary
def changes_empty(summary: Mapping[str, Any]) -> bool:
return not any(ids for part in summary.values() for ids in part.values())
def changes_in_words(summary: Mapping[str, Any]) -> str:
part = summary.get("operational_requirements") or {}
return "\n".join(
f"Operational requirement {identifier} was {word}."
for word in ("added", "changed", "removed")
for identifier in part.get(word) or []
)
def changes_for(phase_kinds: list[str], summary: Mapping[str, Any]) -> dict[str, Any]:
"""The parts of the summary that belong to a Phase's Deliverables."""
return {
part: change
for part, change in summary.items()
if _PARTS[part][0] in phase_kinds and any(change.values())
}
# --- the Workflow ----------------------------------------------------------------------
def build(root: Path = ROOT) -> Workflow:
"""The Workflow, for the capability catalogue and the rate card under `root`."""
catalogue: Catalogue = capability_catalogue.load(
root / "skills" / capability_catalogue.SKILL
)
card: RateCard = staffing.load(root / "skills" / staffing.SKILL)
add_views = graph.views(catalogue, card)
# Every provider's table of what kind of thing each product is, as one: the
# digest names an Item's kind before the operating spec says which provider.
kinds: dict[str, list[str]] = {}
for bundle in sorted((root / "providers").glob("*.yaml")):
for kind, words in digest.item_kinds(root, bundle.stem).items():
kinds.setdefault(kind, []).extend(words)
def with_graph_views(
bench: Bench, content: str, attachments: dict[str, str]
) -> tuple[str, dict[str, str]]:
return add_views(content, attachments, bench.upstream)
def check_verdicts(
attachments: Mapping[str, str], upstream: Upstream | None
) -> list[str]:
return scenarios.check_verdicts(
attachments, graph.read(upstream), VERDICTS_FILE
)
def with_scenarios(
bench: Bench, content: str, attachments: dict[str, str]
) -> tuple[str, dict[str, str]]:
# The walk-throughs are generated and traced here, and their verdicts
# written into the validator's file: no model judges one.
upstream = bench.upstream or {}
model = graph.read(upstream)
if model is None:
return content, attachments
walked = scenarios.walk(
model,
chain.capabilities(upstream),
spec.agreed(upstream),
catalogue,
_staffing(upstream),
)
attachments[SCENARIOS_FILE] = json.dumps(walked, indent=2, ensure_ascii=False)
waiting = {
str(check["id"]): str(check["depends_on_finding"])
for check in model["checks"]
if check.get("depends_on_finding")
}
attachments[VERDICTS_FILE] = scenarios.merge(
attachments[VERDICTS_FILE], walked, waiting
)
content = content.split(f"\n## {scenarios.HEADING}")[0].rstrip()
return content + "\n\n" + scenarios.render(walked), attachments
def with_guide_sections(
bench: Bench, content: str, attachments: dict[str, str]
) -> tuple[str, dict[str, str]]:
return guide.assemble(content, bench.upstream or {}, catalogue), attachments
def check_ids(upstream: Upstream | None) -> list[str]:
model = graph.read(upstream)
return [str(item["id"]) for item in model["checks"]] if model else []
def known_ids(deliverables: Upstream) -> set[str]:
return {str(r["id"]) for r in chain.requirements(deliverables)} | set(
spec.ids(spec.conditions(spec.agreed(deliverables)))
)
def approval_details(
phase: str, deliverables: Upstream, baseline: Mapping[str, Any] | None
) -> dict[str, Any]:
if not baseline:
return {}
in_phase = [
kind for kind, each in deliverable_specs.items() if each.phase == phase
]
return {
"changes": changes_for(
in_phase, change_summary(baseline.get("deliverables"), deliverables)
)
}
def task_entries(
state: Mapping[str, Any], kind: str
) -> tuple[dict[str, Any], dict[str, Any]]:
context: dict[str, Any] = {
# Which rate card the cost is worked from; its figures are placeholders
# while its version is a draft.
"rate_card": {"version": card.version, "draft": card.is_draft},
}
if kind == CAPABILITY_MODEL:
# The catalogue's entries that apply to this operating spec, found by
# code. Where to start: take each, or say why it does not apply.
scope = spec.agreed(state.get("deliverables") or {})
context["capabilities_proposed"] = catalogue.applicable(scope)
if kind == TARGET_OPERATING_MODEL:
context["rate_card"].update(
countries=sorted(card.countries), levels=dict(card.levels)
)
return {}, context
def outputs(
state: Mapping[str, Any], deliverables: Upstream, files: dict[str, str]
) -> dict[str, Any]:
cost = _staffing(deliverables)
found = _findings(deliverables)
if found:
files[CHANGE_REQUEST_FILE] = change_request(state, found)
return {
"staffing": None
if cost is None
else {
"people": cost["people"],
"cost_per_month": cost["cost_per_month"],
"currency": cost["currency"],
"rate_card": cost["rate_card"],
},
"findings": [str(item["id"]) for item in found],
"change_request": CHANGE_REQUEST_FILE if found else None,
"capability_sources_verified": catalogue.verified,
}
deliverable_specs = {
each.kind: each
for each in (
DeliverableSpec(
OPERATING_SCOPE,
SCOPING,
"scope-analyst",
sections=("Scope statement",),
attachments=(SCOPE_FILE, SCOPE_DATA_FILE),
),
DeliverableSpec(
OPERATIONAL_REQUIREMENTS,
REQUIREMENTS,
"operations-analyst",
depends_on=(OPERATING_SCOPE,),
sections=("Operational requirements", "What was not stated"),
attachments=(REQUIREMENTS_FILE,),
),
DeliverableSpec(
CAPABILITY_MODEL,
DESIGN,
"capability-modeller",
depends_on=(OPERATIONAL_REQUIREMENTS,),
sections=("Capabilities", "Sources"),
attachments=(CAPABILITIES_FILE,),
),
DeliverableSpec(
TARGET_OPERATING_MODEL,
DESIGN,
"operating-model-architect",
depends_on=(CAPABILITY_MODEL,),
# The Specialist writes the reasoning and the graph. On submission
# code adds the responsibility matrix, the staffing and cost and
# the findings to the content, and the organisation chart and the
# staffing figures to the attachments.
sections=("Design rationale",),
attachments=(GRAPH_FILE,),
graph=graph.SHAPE,
),
DeliverableSpec(
VALIDATION_RESULTS,
VALIDATION,
"validator",
depends_on=(TARGET_OPERATING_MODEL,),
sections=("Check verdicts",),
attachments=(VERDICTS_FILE,),
),
DeliverableSpec(
GUIDE,
PACKAGING,
"packager",
depends_on=(VALIDATION_RESULTS,),
# The packager writes the summary; code writes every other
# section from the data when the guide is submitted.
sections=guide.SECTIONS,
reads_in_full=(
OPERATING_SCOPE,
OPERATIONAL_REQUIREMENTS,
CAPABILITY_MODEL,
TARGET_OPERATING_MODEL,
VALIDATION_RESULTS,
),
),
)
}
return Workflow(
name="target-operating-model",
about=ABOUT,
phases=PHASES,
deliverables=deliverable_specs,
final=GUIDE,
final_file=GUIDE_FILE,
data_checks={
OPERATIONAL_REQUIREMENTS: (REQUIREMENTS_FILE, chain.check_requirements),
CAPABILITY_MODEL: (CAPABILITIES_FILE, chain.check_capabilities(catalogue)),
TARGET_OPERATING_MODEL: (
GRAPH_FILE,
graph.check(card, catalogue.frameworks),
),
VALIDATION_RESULTS: (VERDICTS_FILE, check_verdicts),
},
for_readers={
TARGET_OPERATING_MODEL: graph.MATRIX_HEADING,
VALIDATION_RESULTS: scenarios.HEADING,
},
spec_deliverable=OPERATING_SCOPE,
validation=ValidationPolicy(
kind=VALIDATION_RESULTS,
data_file=VERDICTS_FILE,
# A failed Check is a fault in who does what: the graph is revised. One
# that fails twice running is a fault in what the organisation must be
# able to do: the capability model is.
implementation=TARGET_OPERATING_MODEL,
design=CAPABILITY_MODEL,
check_ids=check_ids,
known_ids=known_ids,
),
on_submit={
TARGET_OPERATING_MODEL: with_graph_views,
VALIDATION_RESULTS: with_scenarios,
GUIDE: with_guide_sections,
},
approval_details=approval_details,
approval_facts=approval_facts,
summary_lines=summary_lines,
describe_approval=describe_approval,
agreed=lambda deliverables: json.dumps(chain.requirements(deliverables)),
changes=ChangePolicy(
file=CHANGES_FILE,
summary=change_summary,
is_empty=changes_empty,
in_words=changes_in_words,
),
subject="a target operating model",
describe_design=lambda deliverables: "\n".join(
f"- {item.get('name')} ({item.get('kind')}, {item.get('product', '')})"
for item in spec.items(spec.agreed(deliverables))
),
task_entries=task_entries,
outputs=outputs,
# A solution architecture's data files are read in code. Anything else a
# caller supplies is a document for the scope analyst to read.
digest=lambda sources: digest.architecture(sources, kinds),
card=CardText(
description=(
"Designs a target operating model with you for a solution: "
"operational requirements, the Capabilities needed, who is "
"accountable and responsible, staffing and cost from a rate card, "
"and a validation against the events it must handle."
),
skill_id="target-operating-model",
skill_name="Target operating model",
tags=("operations", "operating-model"),
examples=(
"Design the operating model for this landing zone, run as a managed service.",
"Who runs this platform, at what cost, from which countries?",
),
),
done="The target operating model is complete and approved.",
)
def change_request(state: Mapping[str, Any], found: list[dict[str, Any]]) -> str:
"""The findings, as a request to whoever owns the design to change it.
Written by code from the findings. Where the operating spec was digested from a
solution architecture, it is in the form that agent accepts as a Change request
against the Execution it came from.
"""
values = ((state.get("scope") or {}).get("working") or {}).get("values") or {}
source = str(values.get("metadata.source") or "")
lines = ["# Changes operations needs to the solution", ""]
if source:
lines += [
(
f"The operating spec was digested from `{source}`. Send what follows to "
"the solution architecture agent as a Change request against that "
"Execution: the `baseline` is its id, in the same Customer case."
),
"",
]
else:
lines += [
(
"The operating spec did not come from a solution architecture this "
"agent can name. What follows is for whoever owns the design."
),
"",
]
lines += [
(
"Operating the solution as designed needs these changes to it. Each was "
"found while designing its operating model; none was designed around."
),
"",
]
for item in found:
about = ", ".join(
str(i) for i in item.get("source_ref") or item.get("about") or []
)
lines += [
f"## {item['id']}" + (f": {about}" if about else ""),
"",
f"**What is missing.** {item.get('lacks')}",
"",
f"**What is asked for.** {item.get('proposed')}",
"",
]
return "\n".join(lines)
__all__ = ["FINDINGS_FILE", "ROOT", "build", "change_request", "change_summary"]

View File

@@ -0,0 +1,27 @@
---
status: proposed
---
# The Target operating model is an explicit graph, and the solution is never changed here
An operating model written as prose cannot be checked: nobody can say whether every part of the solution has someone to run it, or whether anyone is answerable when it fails at night. We decide that the **Target operating model** is one explicit graph, held as data: Parties, Roles, the Responsibilities between Roles and Capabilities, and the Checks each must meet. It is established and approved before anything is written from it, and what is written from it is a view of it.
This follows the solution architecture agent's ADR-0005, applied to a different subject.
## What follows
- **Completeness is a submission rule.** The graph is refused while a Capability has no single accountable Role, or nobody responsible at the hours it is needed, or while something the customer said must be kept true is carried by no Check.
- **Views are written by code.** The responsibility matrix, the organisation chart, and staffing and cost are rendered or computed from the graph. No model draws or counts.
- **Validation is a verdict per Check,** with scenario walk-throughs in place of a dry run: events generated from the graph and traced through it by code.
- **The solution is never changed.** The Items to be operated and the Conditions on them are the customer's description, or the digest of their architecture. Something operations needs that the solution does not provide is a **finding**: it is recorded, a Check that depends on it waits on it, and it goes back to whoever owns the design. Where the spec was digested from a solution architecture, the findings are drafted as a Change request to that agent.
## Considered Options
- **Repair gaps in the solution while designing its operations:** rejected. Two agents would then hold different versions of one design, and the architecture's own Checks would no longer describe what is run.
- **Refuse to proceed while a gap exists:** rejected. Most operating models can be designed around a known, stated gap, and the customer is better served by a model with its findings listed than by none.
- **Hold Capabilities inside the graph:** rejected for now. The capability model is its own Deliverable, so that what must be done is agreed apart from who does it, and a failed Check can reopen one without the other.
## Consequences
- A finding is neither a failed Check nor a gap in the model. Nothing is revised for it, and the Execution completes with it listed.
- More is decided before detail is written. A fault in the graph costs a Revision of it and of validation.

View File

@@ -0,0 +1,26 @@
---
status: proposed
---
# An Execution starts from an operating spec
This agent was first conceived as reading the solution architecture agent's files. That would make it usable only after that agent, and tie it to the shape of files that are still changing. We decide that an Execution starts from an **operating spec**: a structured file read against a catalogue in code, complete or partly filled, completed in conversation and approved before anything is derived from it. It is the engine's spec mechanism, with this agent's catalogue.
The spec carries the solution to be operated inline: its Items, the Conditions on them, and what is known of its build and its risks. It does not point at other files.
## What follows
- **Anything supplied is digested into a spec first.** The solution architecture agent's files are one Source, read by a code adapter that keeps every Element and Check with its id. A document is another, drafted by a model with each record quoting where it came from. A sentence is a third.
- **Everything downstream reads only the approved spec.** No Specialist opens an architecture file. Completeness is checked against the spec's Items and Conditions.
- **A digest error is caught at the scoping Approval,** where the customer sees what was digested, and not later.
- **The agent can design an operating model for a solution the architecture agent never saw.**
## Considered Options
- **Read the architecture agent's files directly:** rejected by the owner. It is the only way in, and it copies that agent's readers.
- **Keep the inventory in separate files the spec points to:** rejected by the owner. It brings back the dependency the spec removes.
## Consequences
- The spec is larger than the architecture agent's scope spec, and the engine gives the agreed spec to every Specialist. The inventory should travel as a data file to the Specialists that use it; until the engine's digest step is built it travels with the rest.
- The digest step is not built. Today an Execution starts from an operating spec that is already written.

View File

@@ -0,0 +1,29 @@
---
status: proposed
---
# Cost is computed from the service provider's rate card
An operating model that names Roles invites the question of what they cost. A model asked for a cost will give one, and it will be a guess. We decide that staffing and cost are computed by code from the graph and one file, the **rate card**, and that no agent writes a number of people, a rate or a cost.
## Decisions
Taken by the owner on 2026-10-13:
1. **The rate card is a Skill, and it is the actual card.** Its prices are the service provider's: set by the owner, changed in review like any other Skill. A caller cannot name another card and an operating spec cannot state a rate.
2. **Rates are by country and level.** A Role has both. The Specialist chooses them and says why; it never chooses to reach a cost.
3. **The operating spec states the currency.** The card is written in one currency and carries fixed conversion rates with their date. No exchange rate is looked up. A currency the card cannot convert to refuses the start.
4. **The customer's own staff are counted and never priced,** whatever the card says.
## How it is computed
- **People** are the effort of the Capabilities a Role is responsible for, over the productive hours of its country. Where the Role covers hours wider than a working day, never fewer than the smallest team that can cover them: a rota is sized by the hours it fills.
- **Cost** is people at the day rate for the Role's country and level, with the uplift for its hours, converted into the spec's currency.
- A cost over a stated budget is shown as over. It is not refused.
## Consequences
- Every cost is traceable: the card's version is recorded with the Execution and printed beside every total.
- The card in the repository holds placeholder figures until the owner sets them. Its version says `draft`, and every cost worked from it says it is an illustration.
- The owner's real prices will be readable by whoever can read the repository.
- The effort a Specialist states for a Capability is the one estimate in the chain. The validator judges whether it is plausible; nothing computes it.

0
eval/__init__.py Normal file
View File

32
eval/datasets/cases.json Normal file
View File

@@ -0,0 +1,32 @@
[
{
"id": "case-01-landing-zone-shared",
"what": "A written operating spec, complete enough to run with nobody to ask.",
"spec": "specs/landing-zone-shared.yaml",
"max_revisions": 2,
"max_minutes": 40
},
{
"id": "case-02-from-architecture",
"what": "The solution architecture agent's landing-zone Execution, digested, with who operates it stated and the business outcomes left to the agent.",
"sources": "../../tests/fixtures/architecture/case-08-landing-zone",
"overlay": {
"sourcing": {
"model": "provider_operated",
"delivery_countries": [
"GB",
"PL",
"IN"
]
},
"run": {
"pre_approved": true
},
"intent": {
"business_outcomes": "decide"
}
},
"max_revisions": 3,
"max_minutes": 60
}
]

View File

@@ -0,0 +1,53 @@
# A fictional benchmark: a small landing zone run as a shared service. No customer's
# data is in it. Complete enough to run with nobody to ask.
metadata:
customer_case: bench-landing-zone-shared
engagement:
kind: platform_foundation
intent:
statement: |
We want our landing zone run as a shared service from go-live. A provider should
respond to faults at every hour, and our own platform team keeps the final say
over changes to the network.
business_outcomes:
- {id: BO-1, outcome: New payment products reach customers in weeks rather than months., target: "A change reaches production within a day of approval."}
- {id: BO-2, outcome: Payments are never unavailable for long., target: "Restored within 4 hours."}
- {id: BO-3, outcome: We keep the platform engineers we hire.}
requirements:
- {id: REQ-01, statement: Our platform team approves every change to the network.}
- {id: REQ-02, statement: We receive a monthly report of service levels and cost.}
solution:
cloud_provider: gcp
items:
- {id: S-1, name: Hub network and interconnect, kind: network, product: Cloud Interconnect}
- {id: S-2, name: Audit log store, kind: storage, product: Cloud Storage}
- {id: S-3, name: Deployment pipeline, kind: pipeline, product: Cloud Build}
- {id: S-4, name: Encryption keys, kind: keys, product: Cloud KMS}
conditions:
- {id: K-1, "on": [S-1], condition: The platform is available at least 99.9% of each month.}
- {id: K-2, "on": [S-2], condition: Audit logs cannot be deleted for seven years.}
- {id: K-3, "on": [S-4], condition: Encryption keys are rotated every 90 days.}
build:
duration_days: 36
operations:
support_hours: 24x7
availability_hours: 24x7
change_process: decide
sourcing:
model: shared
delivery_countries: [GB, PL, IN]
organisation:
existing_teams:
- {name: Platform team, party: customer, hours: business, country: GB}
service_management:
tooling: decide
service_levels:
priority_1_response_minutes: 15
recovery_time_hours: 4
compliance:
frameworks: [DORA]
commercial:
currency: GBP
monthly_run_budget: 60000
run:
pre_approved: true

View File

@@ -0,0 +1,208 @@
"""Evaluate the target operating model agent on benchmark cases with real models.
Each case runs one pre-approved Execution through the full assembly (the ADK runtime
and the agent's own Skills and role files), then scores it with eval/scoring.py.
It costs model usage: run it deliberately.
Usage:
GOOGLE_GENAI_USE_ENTERPRISE=true GOOGLE_CLOUD_PROJECT=<project> GOOGLE_CLOUD_LOCATION=global \\
.venv/bin/python -m eval.operating_model_eval [case-id ...]
The model settings are read from the environment or from `.env`. With no case named,
every case in eval/datasets/cases.json runs, one after another. Results are written
under eval/results/<time>/: each Execution's files, and results.json. The exit code
is non-zero when a case fails a check or runs over its time.
"""
from __future__ import annotations
import asyncio
import json
import logging
import sys
import time
from collections.abc import Mapping, Sequence
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
import yaml
from execution_engine.adk.runtime import AgentRuntime
from execution_engine.agent import open_assembly
from execution_engine.config import Settings, get_settings
from execution_engine.recorder import InMemoryRecorder
from app.operating_model import digest
from app.operating_model.workflow import ROOT, build
from eval.scoring import CaseResult, Check, score
HERE = Path(__file__).resolve().parent
CASES = HERE / "datasets" / "cases.json"
RESULTS = HERE / "results"
logger = logging.getLogger(__name__)
def load_cases(wanted: Sequence[str] = ()) -> list[dict[str, Any]]:
cases: list[dict[str, Any]] = json.loads(CASES.read_text(encoding="utf-8"))
unknown = sorted(set(wanted) - {case["id"] for case in cases})
if unknown:
raise SystemExit(f"Unknown cases: {', '.join(unknown)}")
return [case for case in cases if not wanted or case["id"] in wanted]
def inputs(case: Mapping[str, Any]) -> dict[str, Any]:
"""What an Execution of the case is started with."""
given: dict[str, Any] = {}
if case.get("spec"):
given["scope"] = (CASES.parent / case["spec"]).read_text(encoding="utf-8")
if case.get("sources"):
given["sources"] = digest.read_folder(
(CASES.parent / case["sources"]).resolve()
)
if case.get("overlay"):
given["scope"] = yaml.safe_dump(case["overlay"])
return given
def _tokens(runtime: Any) -> dict[str, dict[str, int]]:
"""Tokens by role, when the runtime counts them."""
usage = getattr(runtime, "usage", None)
if not isinstance(usage, Mapping):
return {}
by_role: dict[str, dict[str, int]] = {}
for (role, _model), used in usage.items():
totals = by_role.setdefault(role, {})
for name, amount in vars(used).items():
if isinstance(amount, int):
totals[name] = totals.get(name, 0) + amount
return by_role
async def run_case(
case: Mapping[str, Any],
output_root: Path,
*,
settings: Settings | None = None,
runtime: AgentRuntime | None = None,
) -> CaseResult:
"""Run one case to the end and score it. `runtime` replaces the real one, for a
scripted run of the harness itself."""
settings = settings or get_settings()
workflow = build(ROOT)
started = time.monotonic()
async with open_assembly(
settings,
workflow,
root=ROOT,
runtime=runtime,
recorder=InMemoryRecorder(),
output_root=output_root,
) as assembly:
used = assembly.runtime
# A case is stopped at half again its time limit: a run that loops keeps
# spending until something stops it.
limit = float(case.get("max_minutes") or 60) * 1.5
try:
interaction = await asyncio.wait_for(
assembly.engine.start(
"", pre_approved=True, execution_id=case["id"], **inputs(case)
),
timeout=limit * 60,
)
except TimeoutError:
logger.error("Case %s stopped after %.0f minutes", case["id"], limit)
return CaseResult(
case=case["id"],
status="stopped",
checks=[Check("finished", False, f"stopped after {limit:.0f} minutes")],
minutes=round((time.monotonic() - started) / 60, 1),
tokens=_tokens(used),
)
except Exception as exc:
logger.exception("Case %s could not run", case["id"])
return CaseResult(
case=case["id"],
status="error",
checks=[Check("ran", False, f"{type(exc).__name__}: {exc}")],
minutes=round((time.monotonic() - started) / 60, 1),
)
minutes = round((time.monotonic() - started) / 60, 1)
state = interaction.state
checks, revisions, figures = score(
workflow,
state,
status=interaction.status,
max_revisions=int(case.get("max_revisions", 2)),
)
allowed = case.get("max_minutes")
if allowed:
checks.append(
Check(
f"finished within {allowed} minutes", minutes <= allowed, f"{minutes}"
)
)
return CaseResult(
case=case["id"],
status=interaction.status,
checks=checks,
minutes=minutes,
revisions=revisions,
tokens=_tokens(used),
figures=figures,
warnings=list(state.get("warnings") or []),
folder=str(
output_root
/ "deliverables"
/ "cases"
/ str(state.get("customer_case"))
/ "executions"
/ case["id"]
),
)
def report(results: Sequence[CaseResult]) -> str:
lines = []
for result in results:
lines.append(
f"{'PASS' if result.passed else 'FAIL'} {result.case} "
f"{result.status}, {result.minutes} min"
)
lines += [
f" {'ok ' if check.passed else 'FAIL'} {check.name}"
+ (f": {check.detail}" if check.detail else "")
for check in result.checks
]
if result.figures:
lines.append(
" " + ", ".join(f"{k}={v}" for k, v in result.figures.items())
)
lines += [f" warning: {warning}" for warning in result.warnings]
return "\n".join(lines)
async def main(arguments: Sequence[str]) -> int:
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(message)s")
cases = load_cases(arguments)
stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
output_root = RESULTS / stamp
output_root.mkdir(parents=True, exist_ok=True)
# An evaluation keeps nothing between runs and reads nobody's cloud.
settings = get_settings().model_copy(
update={"CHECKPOINTER": "memory", "MCP_ENABLED": False, "CLOUD_PROVIDER": "gcp"}
)
results = []
for case in cases:
logger.info("Running %s", case["id"])
results.append(await run_case(case, output_root, settings=settings))
(output_root / "results.json").write_text(
json.dumps([result.as_dict() for result in results], indent=2),
encoding="utf-8",
)
print(report(results))
print(f"\nWritten to {output_root}")
return 0 if all(result.passed for result in results) else 1
if __name__ == "__main__":
sys.exit(asyncio.run(main(sys.argv[1:])))

198
eval/scoring.py Normal file
View File

@@ -0,0 +1,198 @@
"""Checks applied to a finished Execution of the target operating model Workflow.
Every check here is structural: it reads the Execution's own data and asks whether
the chain holds and the result is what the Workflow promises. None judges whether
the operating model is a good one; that needs a person, or a judge calibrated
against one, and is not built.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import asdict, dataclass, field
from typing import Any
from execution_engine.workflows.workflow import Workflow
from app.operating_model import guide, scenarios, spec
from app.operating_model.names import (
GUIDE,
SCENARIOS_FILE,
STAFFING_FILE,
TARGET_OPERATING_MODEL,
VALIDATION_RESULTS,
VERDICTS_FILE,
)
@dataclass(frozen=True)
class Check:
name: str
passed: bool
detail: str = ""
@dataclass
class CaseResult:
case: str
status: str
checks: list[Check] = field(default_factory=list)
minutes: float = 0.0
revisions: dict[str, int] = field(default_factory=dict)
tokens: dict[str, dict[str, int]] = field(default_factory=dict)
figures: dict[str, Any] = field(default_factory=dict)
warnings: list[str] = field(default_factory=list)
folder: str = ""
@property
def passed(self) -> bool:
return all(check.passed for check in self.checks)
def as_dict(self) -> dict[str, Any]:
return {**asdict(self), "passed": self.passed}
def _attachment(deliverables: Mapping[str, Any], kind: str, name: str) -> Any:
text = (deliverables.get(kind) or {}).get("attachments", {}).get(name, "")
try:
return json.loads(text) if text else None
except ValueError:
return None
def score(
workflow: Workflow,
state: Mapping[str, Any],
*,
status: str,
max_revisions: int = 2,
) -> tuple[list[Check], dict[str, int], dict[str, Any]]:
"""The checks on a finished Execution, its Revisions, and its key figures."""
deliverables: Mapping[str, Any] = state.get("deliverables") or {}
ledger = (state.get("ledger") or {}).get("deliverables") or {}
revisions = {
kind: int(entry.get("revisions", 0))
for kind, entry in ledger.items()
if entry.get("revisions")
}
checks = [
Check("completed", status == "completed", status),
Check(
"every Deliverable was submitted",
set(deliverables) == set(workflow.deliverables),
", ".join(sorted(set(workflow.deliverables) - set(deliverables))) or "all",
),
]
# The chain, checked again on what was delivered: each Deliverable's data
# against the ones it builds on, as the engine checked it when it was submitted.
broken = []
for kind in workflow.deliverables:
if kind not in deliverables or kind not in workflow.data_checks:
continue
name, check = workflow.data_checks[kind]
attachments = deliverables[kind].get("attachments", {})
upstream = {
k: deliverables[k] for k in workflow.inputs_of(kind) if k in deliverables
}
broken += (
[f"{kind}: {fault}" for fault in check(attachments, upstream)]
if name in attachments
else [f"{kind}: {name} is missing"]
)
checks.append(Check("the chain holds", not broken, "; ".join(broken[:5])))
verdicts = _attachment(deliverables, VALIDATION_RESULTS, VERDICTS_FILE) or {}
given = [v for v in verdicts.get("checks") or [] if isinstance(v, dict)]
own = [v for v in given if not str(v.get("check", "")).startswith(scenarios.PREFIX)]
failed = [str(v.get("check")) for v in given if v.get("verdict") == "fail"]
checks.append(Check("no Check failed", not failed, ", ".join(failed)))
walked = _attachment(deliverables, VALIDATION_RESULTS, SCENARIOS_FILE) or []
checks.append(
Check(
"every scenario has a path",
bool(walked) and all(item.get("passes") for item in walked),
f"{sum(1 for i in walked if i.get('passes'))} of {len(walked)}",
)
)
content = str((deliverables.get(GUIDE) or {}).get("content", ""))
headings = [
line[3:].strip() for line in content.splitlines() if line.startswith("## ")
]
checks.append(
Check(
"the guide has its sections",
headings == list(guide.SECTIONS),
", ".join(headings),
)
)
summary = content.split(f"## {guide.SUMMARY}")[-1].split(
f"### {guide.KEY_FIGURES}"
)[0]
checks.append(
Check(
"the packager wrote a summary",
len(summary.split()) >= 30,
f"{len(summary.split())} words",
)
)
most = max(revisions.values(), default=0)
checks.append(
Check(
f"no Deliverable was revised more than {max_revisions} times",
most <= max_revisions,
", ".join(f"{kind} x {count}" for kind, count in revisions.items())
or "none",
)
)
scope = spec.agreed(deliverables)
cost = _attachment(deliverables, TARGET_OPERATING_MODEL, STAFFING_FILE) or {}
model = (
_attachment(deliverables, TARGET_OPERATING_MODEL, "target-operating-model.json")
or {}
)
figures = {
"items": len(spec.items(scope)),
"conditions": len(spec.conditions(scope)),
"operational_requirements": len(
_attachment(
deliverables,
"operational_requirements",
"operational-requirements.json",
)
or []
),
"capabilities": len(
_attachment(deliverables, "capability_model", "capabilities.json") or []
),
"custom_capabilities": sum(
1
for c in _attachment(deliverables, "capability_model", "capabilities.json")
or []
if c.get("catalogue") == "custom"
),
"parties": len(model.get("parties") or []),
"roles": len(model.get("roles") or []),
"checks": len(own),
"checks_not_verifiable": sum(
1 for v in own if v.get("verdict") == "not_verifiable"
),
"checks_waiting_on_findings": sum(
1 for v in own if v.get("verdict") == scenarios.WAITING
),
"proposed_additions": len(verdicts.get("proposed_additions") or []),
"scenarios": len(walked),
"findings": len(
_attachment(deliverables, TARGET_OPERATING_MODEL, "findings.json") or []
),
"business_outcomes": len(spec.business_outcomes(scope)),
"measures": len(model.get("measures") or []),
"people": cost.get("people"),
"cost_per_month": cost.get("cost_per_month"),
"currency": cost.get("currency"),
}
return checks, revisions, figures

64
providers/gcp.yaml Normal file
View File

@@ -0,0 +1,64 @@
# Cloud provider bundle: Google Cloud. Loaded by the Execution engine; select with
# CLOUD_PROVIDER=gcp. The operating model's own knowledge names no Cloud provider;
# what is Google Cloud's is here.
name: gcp
skills_dirs:
- skills
# The template the operating model guide follows.
output_template: skills/operating-guide/assets/guide-template.md
# Model ids, as the solution architecture agent pins them: explicit versions, since
# -latest aliases are not served on Vertex AI. Judgement about the whole model takes
# Pro; drafting takes Flash.
models:
default: gemini/gemini-3.8-flash
roles:
orchestrator: gemini/gemini-3.1-pro-preview
capability-modeller: gemini/gemini-3.1-pro-preview
operating-model-architect: gemini/gemini-3.1-pro-preview
validator: gemini/gemini-3.1-pro-preview
timeout_seconds:
# The capability modeller timed out once at 180 s in the run of 2026-10-14.
capability-modeller: 300
operating-model-architect: 420
validator: 300
packager: 300
# Skills each role may load. Role files name no Skill; the bundle does.
role_skills:
scope-analyst: [operating-scoping]
operations-analyst: [operating-scoping, operating-chain]
capability-modeller: [operating-capabilities, operating-chain]
operating-model-architect: [operating-capabilities, operating-chain, rate-card]
validator: [operating-chain]
packager: [operating-guide]
# What kind of thing each of this provider's products is, for the digest of a
# solution architecture: the capability catalogue proposes Capabilities by an
# Item's kind. A product is matched by any of these words in its name; one that
# matches none is a `service`.
operating:
item_kinds:
storage: [Cloud Storage, Filestore]
database: [Cloud SQL, Spanner, Firestore, Bigtable, BigQuery, AlloyDB, Memorystore]
network: [VPC, Interconnect, Cloud VPN, Cloud NAT, Load Balancing, Cloud DNS, Cloud Armor, Cloud Router, Network Connectivity]
compute: [Cloud Run, Kubernetes Engine, GKE, Compute Engine, Cloud Functions, App Engine]
pipeline: [Cloud Build, Cloud Deploy, Artifact Registry]
observability: [Cloud Monitoring, Cloud Logging, Cloud Trace]
identity: [IAM, Identity, Workload Identity]
keys: [Cloud KMS, Secret Manager]
governance: [Resource Manager, Organization Policy, Security Command Center]
billing: [Cloud Billing]
messaging: [Pub/Sub, Eventarc, Cloud Tasks]
# The operating spec. `skill` holds its catalogue, which names no Cloud provider and
# no currency. `values` are what is allowed for the fields the catalogue leaves open:
# this provider, and the currencies the rate card can convert to (a test keeps the
# two the same).
scope:
skill: operating-scoping
values:
solution.cloud_provider: [gcp]
commercial.currency: [AUD, CAD, EUR, GBP, INR, JPY, USD]

33
pyproject.toml Normal file
View File

@@ -0,0 +1,33 @@
[project]
name = "target-operating-model-agent"
version = "0.1.0"
description = "Turns a description of a solution into a reviewed target operating model."
requires-python = ">=3.12"
# The Execution engine is installed from its own repository. Locally:
# pip install -e ../orchestration_engine
# How a container build obtains it is step E7 of IMPLEMENTATION_PLAN.md.
dependencies = ["execution-engine"]
[project.optional-dependencies]
dev = [
"pytest>=8.0.0",
"pytest-asyncio>=0.23.0",
"ruff>=0.6.0",
"mypy>=1.11.0",
"types-pyyaml>=6.0.0",
]
[tool.setuptools.packages.find]
include = ["app*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
cache_dir = ".cache/pytest"
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
[tool.ruff]
cache-dir = ".cache/ruff"
[tool.mypy]
cache_dir = ".cache/mypy"

View File

@@ -0,0 +1,50 @@
---
name: operating-capabilities
description: >-
The capability catalogue: what an operating organisation may need to be able to
do, in our own words, with the industry frameworks each entry draws on. Use when
choosing the Capabilities of an operating model and writing the Checks on them.
---
# The capability catalogue
`assets/catalogue.yaml` lists the Capabilities an operating model is built from. `assets/sources.yaml` is the register of the frameworks they cite.
## How to use it
1. Your task's context lists the entries code found to apply to this operating spec, under `capabilities_proposed`. Start from those.
2. Take every proposed entry, or say in your Deliverable why one does not apply. Add any other entry the solution needs.
3. A Capability names its entry by `catalogue`. One the catalogue does not have is `custom`, with a `reason`: use it only when no entry fits, never to rename one.
4. Give each Capability the Items it covers and the Operational requirements it serves. Every Item must be covered by at least one Capability, and every Operational requirement served by at least one.
5. An entry's `typical_checks` are where to start when writing Checks. Rewrite each for this solution: name the Item, and put the customer's own numbers in.
## What the sources are for
A source says where an idea comes from. Cite it in a Deliverable as the catalogue does, by framework and by the name of the part. Never quote a framework's text and never say an operating model is certified against, or compliant with, a framework: the catalogue draws on them and promises nothing about them.
The register's `status` says whether a person has checked the references. While it is `unverified`, say so wherever a source is cited to a customer.
## People and teams
Two frameworks shape who does the work, not only what it is:
- **Team Topologies** (`team-topologies`). An operating model for a platform may group its Roles into teams, each of one of four kinds: `stream_aligned`, `platform`, `enabling` or `complicated_subsystem`. Two teams work together in one of three modes: `collaboration`, `x_as_a_service` or `facilitating`. A collaboration is a stage for discovering something together, not a standing arrangement, so it says when it ends.
- **Platform as a product** (`platform-as-a-product`). A platform is run for the teams that build on it, as a product they choose to use: someone owns its roadmap and orders the backlog by what slows those teams down, adoption is earned by a supported default path, and their satisfaction is measured.
For a platform foundation, the catalogue proposes `platform-product-management`, `self-service-and-golden-paths` and `enablement` from these.
## Measuring at three layers
`delivery-and-developer-experience-measurement` draws on three frameworks at once, because each measures a different layer and none stands in for another:
| Layer | Framework | What it tells you |
| --- | --- | --- |
| Outcome | `dora-metrics` | How fast and how safely changes reach production: lagging. |
| Guardrail | `space-framework` | Whether productivity is sustainable across several dimensions, including wellbeing. |
| Leading signal | `devex-framework` | The friction, feedback loops and cognitive load teams meet: what to improve next. |
Never write a Check that measures only one layer and calls it productivity.
Every one of these is a means. In the operating model each is a **Measure** tied to a business outcome the customer stated: deployment frequency to "new products reach customers sooner", developer satisfaction to "we keep the engineers we hire", time to restore to "payments are never down for long". A measure that serves no stated outcome is not measured.
Two things are called DORA. `eu-dora` is the European regulation on operational resilience. `dora-metrics` is the research programme's four delivery measures. Name each in full.

View File

@@ -0,0 +1,281 @@
# The capability catalogue: what an operating organisation may need to be able to do.
#
# Every entry is in our own words and cites the frameworks it draws on, each of
# which must be in sources.yaml. `domain` groups entries. `applies_when` says when
# code proposes the entry for an operating spec:
# always: true every operating model
# item_kinds: [database, storage] when the spec lists an Item of one of these kinds
# compliance: [DORA] when the spec names one of these frameworks
# sourcing: [shared] for these sourcing models
# engagement_kinds: [platform_foundation] for these kinds of engagement
# `follows_support_hours: true` marks a capability that must be staffed for at
# least the hours the spec says faults are responded to.
schema_version: 1
capabilities:
- id: incident-management
name: Incident management
domain: service_operations
purpose: Restore a service that is degraded or down, and say what happened.
applies_when: {always: true}
follows_support_hours: true
typical_checks:
- The most serious incident is acknowledged within the agreed time, at any hour faults are responded to.
- Every incident of the most serious kind has a written review.
sources:
- {framework: itil-4, reference: Incident management practice}
- {framework: google-sre, reference: Managing Incidents}
- {framework: iso-20000-1, reference: Incident management}
- {framework: eu-dora, reference: "Article 17, ICT-related incident management process"}
- id: monitoring-and-alerting
name: Monitoring and alerting
domain: service_operations
purpose: Know the state of what is run, and tell the right people when it needs them.
applies_when: {always: true}
follows_support_hours: true
typical_checks:
- Every Item has a signal that shows whether it is serving, and an alert when it is not.
- Every alert names who receives it and what they do.
sources:
- {framework: itil-4, reference: Monitoring and event management practice}
- {framework: google-sre, reference: Monitoring Distributed Systems}
- {framework: nist-csf-2, reference: "Detect: Continuous Monitoring"}
- {framework: eu-dora, reference: "Article 10, Detection"}
- id: problem-management
name: Problem management
domain: service_operations
purpose: Find why incidents happen and remove the cause.
applies_when: {always: true}
typical_checks:
- A recurring incident has a named cause and an owner for removing it.
sources:
- {framework: itil-4, reference: Problem management practice}
- {framework: google-sre, reference: "Postmortem Culture: Learning from Failure"}
- id: service-request-fulfilment
name: Service request fulfilment
domain: service_operations
purpose: Handle what users and teams ask for that is not a fault, such as access or a new environment.
applies_when: {always: true}
typical_checks:
- Each kind of request has a stated way to ask and a stated time to fulfil.
sources:
- {framework: itil-4, reference: Service request management practice}
- {framework: iso-20000-1, reference: Service request management}
- id: change-enablement
name: Change enablement
domain: change
purpose: Make changes to what is run with the risk of each understood and accepted.
applies_when: {always: true}
typical_checks:
- Every change to a production Item is reviewed by someone other than its author before it is applied.
- Every change can be traced to who approved it.
sources:
- {framework: itil-4, reference: Change enablement practice}
- {framework: iso-20000-1, reference: Change management}
- {framework: dora-metrics, reference: Change failure rate}
- id: release-and-deployment
name: Release and deployment
domain: change
purpose: Put approved changes into service repeatably, and take them out again when they fail.
applies_when: {always: true}
typical_checks:
- A failed deployment can be rolled back within the agreed recovery time.
sources:
- {framework: itil-4, reference: Release management and Deployment management practices}
- {framework: google-sre, reference: Release Engineering}
- {framework: dora-metrics, reference: Deployment frequency and lead time for changes}
- id: configuration-management
name: Configuration management
domain: change
purpose: Know what is run, how it is configured, and that it matches what was approved.
applies_when: {always: true}
typical_checks:
- The configuration of every Item is held in version control and compared with what is running.
sources:
- {framework: itil-4, reference: Service configuration management practice}
- {framework: nist-csf-2, reference: "Identify: Asset Management"}
- id: backup-and-recovery
name: Backup and recovery
domain: resilience
purpose: Keep copies of what cannot be rebuilt, and prove they can be restored.
applies_when: {item_kinds: [database, storage, data]}
typical_checks:
- A restore of every data store is tested on a schedule, and the result recorded.
sources:
- {framework: itil-4, reference: Service continuity management practice}
- {framework: nist-csf-2, reference: "Recover: Incident Recovery Plan Execution"}
- {framework: eu-dora, reference: "Article 12, Backup policies and procedures, restoration and recovery procedures and methods"}
- id: service-continuity
name: Service continuity
domain: resilience
purpose: Restore service after the loss of a site, a region or a supplier, within the agreed time.
applies_when: {always: true}
typical_checks:
- Recovery from the loss of the primary location is rehearsed, and takes no longer than the agreed recovery time.
sources:
- {framework: itil-4, reference: Service continuity management practice}
- {framework: iso-20000-1, reference: Service continuity management}
- {framework: eu-dora, reference: "Article 11, Response and recovery"}
- id: capacity-and-performance
name: Capacity and performance management
domain: resilience
purpose: Keep enough capacity for demand, and no more than is paid for knowingly.
applies_when: {always: true}
typical_checks:
- Each Item's use is reviewed against its limits on a schedule.
sources:
- {framework: itil-4, reference: Capacity and performance management practice}
- {framework: iso-20000-1, reference: Capacity management}
- id: access-management
name: Access management
domain: security_operations
purpose: Give people and systems the access they need, and take it away when they no longer do.
applies_when: {always: true}
typical_checks:
- Access to production is granted for a stated reason and reviewed on a schedule.
- The team that responds to incidents has a tested way in at any hour it is on call.
sources:
- {framework: nist-csf-2, reference: "Protect: Identity Management, Authentication, and Access Control"}
- {framework: iso-27001, reference: Access control}
- {framework: eu-dora, reference: "Article 9, Protection and prevention"}
- id: vulnerability-and-patch-management
name: Vulnerability and patch management
domain: security_operations
purpose: Find weaknesses in what is run and remove them within an agreed time.
applies_when: {always: true}
typical_checks:
- A weakness of the most serious kind is fixed or accepted in writing within the agreed time.
sources:
- {framework: nist-csf-2, reference: "Protect: Platform Security"}
- {framework: iso-27001, reference: Management of technical vulnerabilities}
- {framework: eu-dora, reference: "Article 9, Protection and prevention"}
- id: security-monitoring-and-response
name: Security monitoring and response
domain: security_operations
purpose: Detect misuse of what is run and respond to it.
applies_when: {always: true}
follows_support_hours: true
typical_checks:
- Security events from every Item reach the people who watch for them.
sources:
- {framework: nist-csf-2, reference: "Detect: Adverse Event Analysis; Respond: Incident Management"}
- {framework: iso-27001, reference: Information security incident management}
- {framework: eu-dora, reference: "Article 10, Detection"}
- id: cost-management
name: Cost management
domain: financial_operations
purpose: Know what running costs, who it is spent for, and when it departs from plan.
applies_when: {always: true}
typical_checks:
- Spend is reported against budget monthly, by Item.
- Spend that departs from forecast raises an alert to a named Role.
sources:
- {framework: finops-framework, reference: "Allocation; Budgeting; Forecasting; Anomaly Management"}
- id: service-level-management
name: Service level management
domain: governance
purpose: Agree what the service must achieve, measure it, and report it.
applies_when: {always: true}
typical_checks:
- Each service level the customer stated is measured and reported on a schedule.
sources:
- {framework: itil-4, reference: Service level management practice}
- {framework: google-sre, reference: Service Level Objectives}
- {framework: iso-20000-1, reference: Service level management}
- id: platform-product-management
name: Platform product management
domain: platform_product
purpose: >-
Run the platform as a product for the teams that build on it: learn what slows
them down, keep a roadmap and a backlog ordered by that friction, and measure
whether they choose to use it.
applies_when: {engagement_kinds: [platform_foundation]}
typical_checks:
- A named Role owns the platform's roadmap and orders its backlog by what slows the teams that use it.
- Adoption and the satisfaction of the teams that use the platform are measured on a schedule and reported.
sources:
- {framework: platform-as-a-product, reference: Platform as a product; Measuring platform success}
- {framework: team-topologies, reference: Platform team; Thinnest viable platform}
- id: self-service-and-golden-paths
name: Self-service and golden paths
domain: platform_product
purpose: >-
Offer what the teams building on the platform need as self-service, with a
supported default way to do the common things, so that using it is the
easier choice rather than a rule.
applies_when: {engagement_kinds: [platform_foundation]}
typical_checks:
- The common requests of the teams that use the platform are met through self-service, with no ticket.
- Every self-service path is documented and has a named owner.
sources:
- {framework: team-topologies, reference: X-as-a-Service interaction mode}
- {framework: platform-as-a-product, reference: Platform capabilities; Self-service}
- id: enablement
name: Enablement
domain: platform_product
purpose: >-
Help the teams that build on the platform adopt it, by pairing, pilots,
documentation and training, for a stated time, rather than by mandate.
applies_when: {engagement_kinds: [platform_foundation]}
typical_checks:
- Each engagement to help a team adopt the platform has a stated goal and end date.
sources:
- {framework: team-topologies, reference: Enabling team; Facilitating interaction mode}
- {framework: platform-as-a-product, reference: Platform as a product}
- id: delivery-and-developer-experience-measurement
name: Delivery and developer experience measurement
domain: governance
purpose: >-
Measure how well work flows through the platform at three layers that answer
different questions: delivery speed and stability, the outcome; productivity
and wellbeing across the team, the guardrail; and the friction teams meet,
the leading signal of what to improve next. No one layer stands in for another.
applies_when: {engagement_kinds: [platform_foundation, workload]}
typical_checks:
- Deployment frequency, lead time for changes, change failure rate and time to restore service are measured and reported monthly.
- Productivity is reviewed across more than one dimension, including satisfaction and wellbeing, and never by an activity count alone.
- The friction teams meet, their feedback loops and their cognitive load, are surveyed on a schedule and set the order of the platform's backlog.
sources:
- {framework: dora-metrics, reference: Deployment frequency; Lead time for changes; Change failure rate; Time to restore service}
- {framework: space-framework, reference: "Satisfaction and well-being; Performance; Activity; Communication and collaboration; Efficiency and flow"}
- {framework: devex-framework, reference: Feedback loops; Cognitive load; Flow state}
- id: supplier-management
name: Supplier management
domain: governance
purpose: Hold the suppliers the service depends on to what they agreed.
applies_when: {sourcing: [provider_operated, shared]}
typical_checks:
- Each supplier the service depends on has a named owner and a way to escalate.
sources:
- {framework: itil-4, reference: Supplier management practice}
- {framework: iso-20000-1, reference: Supplier management}
- {framework: eu-dora, reference: "Article 28, General principles on ICT third-party risk"}
- id: regulatory-evidence
name: Regulatory evidence and reporting
domain: governance
purpose: Keep the evidence a regulation asks for and report what it requires, when it requires it.
applies_when: {compliance: [DORA, PCI_DSS_v4, ISO27001, ISO_27001, SOC_2]}
typical_checks:
- Evidence each named framework asks for is kept for as long as it asks.
sources:
- {framework: eu-dora, reference: "Article 19, Reporting of major ICT-related incidents"}
- {framework: iso-27001, reference: Compliance with legal and contractual requirements}

View File

@@ -0,0 +1,82 @@
# The register of frameworks the capability catalogue cites.
#
# A capability's entry is written in our own words. A framework is named as a
# source, by the title of the part drawn on, and never quoted. `status` says
# whether a person has checked the entry's references against the framework:
# every reference here was drafted from general knowledge of the framework and is
# `unverified` until someone has.
schema_version: 1
status: unverified
frameworks:
itil-4:
name: ITIL 4
publisher: PeopleCert
version: "4"
licence: Proprietary; cited by practice name, not reproduced.
google-sre:
name: "Site Reliability Engineering: How Google Runs Production Systems"
publisher: Google / O'Reilly
version: "2016"
licence: Published under a no-derivatives licence; cited by chapter title, not reproduced.
finops-framework:
name: FinOps Framework
publisher: FinOps Foundation
version: "2024"
licence: Openly licensed with attribution; cited by capability name.
nist-csf-2:
name: NIST Cybersecurity Framework
publisher: National Institute of Standards and Technology
version: "2.0"
licence: Public domain; cited by function and category.
iso-20000-1:
name: ISO/IEC 20000-1
publisher: ISO/IEC
version: "2018"
licence: Proprietary; cited by requirement title, not reproduced.
iso-27001:
name: ISO/IEC 27001
publisher: ISO/IEC
version: "2022"
licence: Proprietary; cited by control theme, not reproduced.
eu-dora:
# The EU regulation. Not the DevOps Research and Assessment programme's metrics.
name: Regulation (EU) 2022/2554, the Digital Operational Resilience Act
publisher: European Union
version: "2022"
licence: Public law; cited by article.
dora-metrics:
# The research programme's delivery metrics. Not the EU regulation above.
name: DevOps Research and Assessment (DORA) software delivery metrics
publisher: Google Cloud
version: "2023"
licence: Publicly published research; cited by metric name.
# Socio-technical and organisational frameworks: how people, teams and
# responsibilities are structured around what is operated.
team-topologies:
name: "Team Topologies: Organizing Business and Technology Teams for Fast Flow"
publisher: Matthew Skelton and Manuel Pais, IT Revolution
version: "2019"
licence: Proprietary; cited by the names of its team types and interaction modes, not reproduced.
platform-as-a-product:
# The practice of running an internal platform as a product for the teams that
# use it. Cited from the CNCF's write-up of it, the most widely referenced one.
name: CNCF Platforms White Paper (platform as a product)
publisher: Cloud Native Computing Foundation, TAG App Delivery
version: "2023"
licence: Openly published; its licence is to be confirmed; cited by section, not reproduced.
# Measuring how well the work flows, at three layers. None replaces another.
space-framework:
name: "The SPACE of Developer Productivity"
publisher: Forsgren, Storey, Maddila, Zimmermann, Houck and Butler, ACM Queue
version: "2021"
licence: Published article; cited by its five dimensions, not reproduced.
devex-framework:
name: "DevEx: What Actually Drives Productivity"
publisher: Noda, Storey, Forsgren and Greiler, ACM Queue
version: "2023"
licence: Published article; cited by its three dimensions, not reproduced.
sfia:
name: Skills Framework for the Information Age
publisher: SFIA Foundation
version: "9"
licence: Proprietary; cited by skill name, not reproduced.

View File

@@ -0,0 +1,51 @@
---
name: operating-chain
description: >-
The data files of a target operating model and how each traces to the one before:
operational requirements, capabilities, the Target operating model graph,
findings and verdicts. Use when writing or checking any of these files, and
whenever a submission is refused for a missing link.
---
# The operating chain
Every Deliverable of a target operating model carries a data file. Each entry has an id, and each file names the ids of the file before it. A submission is refused while a link is missing, and the refusal says which.
```
operating spec solution.items (S-n), solution.conditions (K-n),
intent.requirements, and its settled fields
│
operational-requirements.json OR-n source: a stated requirement, a Condition, or a field
│
capabilities.json C-n serves: OR ids items: S ids
│
target-operating-model.json parties P-n, roles R-n,
responsibilities RS-n capability: C id, role: R id
measures M-n outcome: BO id measured_by: C id
checks OC-n on: C, R or RS id source: OR or K id
│
verdicts.json a verdict on every OC id
```
## What is refused
| File | Refused when |
| --- | --- |
| `operational-requirements.json` | A requirement has no `source`, or names one that is not a stated requirement, a Condition or a settled field. A stated requirement or a binding field is carried by no requirement. |
| `capabilities.json` | A Capability names no catalogue entry and is not `custom` with a reason. It serves nothing, or names an id that does not exist. An Item is covered by no Capability. A requirement is served by none. A Capability that responds to faults is staffed for narrower hours than faults are responded to. Its effort is missing, or has no basis. |
| `target-operating-model.json` | A Role is in no Party, or in a country or at a level the rate card lacks, or outside the countries the customer allows. A Capability has no accountable Role, or more than one, or no responsible Role, or none that works its hours. A Condition is carried by no Check. A Check is on, or sourced from, something that does not exist. |
| measures, in `target-operating-model.json` | A business outcome the customer stated is shown by no Measure. A Measure shows an outcome that was not stated, or is taken by a Capability nobody is responsible for. Where the model measures how work flows, the outcome, guardrail and leading layers are not all measured. |
| `findings.json` | A finding lacks `lacks`, `proposed` or `needed_by`, names no Item of the solution under `about`, or names an id that does not exist. |
| `verdicts.json` | A Check has no verdict. A Check that depends on a finding passes: it is `not_verifiable` while the finding stands, or `fail` when the validator judges the operating model could meet it itself, such as by staffing or hours. |
## What code writes
Never write these yourself. They are computed from your data when you submit, so they cannot disagree with it:
- the responsibility matrix, the organisation chart (`organisation.mmd`) and the staffing and cost (`staffing.json`), from the Target operating model;
- the scenario walk-throughs (`scenarios.json`) and their verdicts, from the Target operating model and the operating spec;
- the verdict `waits_on_finding` on a Check that depends on a finding. The validator gives such a Check `not_verifiable`; it is recorded as waiting on its finding, and nothing is revised for it.
## Ids
Keep an id for as long as the thing it names exists. In a Change request an unchanged entry keeps its id and its text; a new one takes an id never used before; a removed one is left out. Never renumber.

View File

@@ -0,0 +1,27 @@
---
name: operating-guide
description: >-
The template of the operating model guide and who writes each part of it: the
packager writes the summary, and code writes every table and figure from the
data. Use when packaging a target operating model into its guide.
---
# The operating model guide
`assets/guide-template.md` is the guide's template. It has nine sections, in a fixed order.
## What you write
Only the **Summary**: one or two short paragraphs for the person who will run the solution. Say what is operated, who operates it and how the work is divided between the Parties, during which hours, and what they should look at first: a finding, a Check that waits on one, a scenario that fails, a cost over its budget.
State no number of people, no cost and no count. The key figures are set under your paragraph from the data, and a figure you wrote could differ from them.
## What is written for you
Every other section. Submit each of the template's headings with nothing under it, or a line saying it is written from the data: whatever you put there is replaced when you submit. The tables, the figures, the verdicts and the citations all come from the data files of the other Deliverables, so they cannot disagree with them.
## What you never do
- Never add a decision, a recommendation or a caveat of your own. The guide assembles what was decided.
- Never restate a figure, even in the summary.
- Never say the operating model complies with, or is certified against, a framework.

View File

@@ -0,0 +1,58 @@
# Operating model guide: <the solution, in a few words>
## Summary
<Written by the packager. One or two short paragraphs for the person who will run
the solution: what is operated, who operates it and how the work is divided between
the Parties, during which hours, and what they should look at first. No number of
people, no cost and no count: the key figures are set under this paragraph from the
data.>
### Key figures
<Written by code: what is operated, the hours, the Capabilities, people and cost
with the rate card's version, the scenario walk-throughs, and the findings.>
## Business outcomes
<Written by code: each business outcome the customer stated, with the Measures
that show it is served, their layer and framework, the Capability that takes each,
and the Role answerable for it.>
## What is operated
<Written by code from the operating scope: the Items, the Conditions that must be
kept true, and anything known to be at risk when operations begin.>
## Operational requirements
<Written by code: each requirement, its kind, and what it comes from.>
## Capabilities
<Written by code: each Capability, the requirements it serves, the Items it covers,
its hours and its effort.>
## Who does what
<Written by code: the Roles and their Parties, and the responsibility matrix.>
## Staffing and cost
<Written by code from the rate card: people and cost for each Role and in total,
the budget when one was stated, the rate card's version, and the conversion rate.>
## Validation
<Written by code: the verdict on each Check with its evidence, and each scenario
walk-through with the path traced for it.>
## Findings and open questions
<Written by code: every finding, everything assumed for the customer, and
everything still unknown.>
## Sources
<Written by code: the frameworks the Capabilities draw on, cited by framework and
by the name of the part, and whether those references have been checked.>

View File

@@ -0,0 +1,47 @@
---
name: operating-scoping
description: >-
What an operating spec's fields mean and how each is settled: the solution to be
operated, how it must be operated, who operates it, and in which currency its
cost is stated. Use when completing the operating scope, and whenever a
Deliverable must honour what the customer stated.
---
# The operating spec
An operating spec is the structured statement an Execution starts from. It holds what the customer wants from operations, the facts an operating model depends on, and the solution to be operated. `assets/catalogue.yaml` lists every field.
## What a field can say
| Written | Meaning | What you do |
| --- | --- | --- |
| a value | The customer states this | Use it as given. It binds the operating model. |
| `ask`, or the field left out | Not known yet | Ask when someone is present. |
| `decide` | You choose | Pick a value and record it as an assumption, with what it would change. |
| `not_applicable` | Does not apply | Ask and assume nothing. |
## Business outcomes
`intent.business_outcomes` is what the business wants from the solution, in the customer's words, each with an id (`BO-1`) and, where they give one, a target. Every measure in the operating model must show one of them, and each must be shown by at least one. Ask for them when someone is present: an operating model whose measures serve no stated outcome cannot show it is worth what it costs. Never invent one; one the customer did not state is not an outcome of theirs.
## The solution
- `solution.items` is what is to be operated. Every Item must be covered by a Capability. Never add an Item, remove one or rename one: it is the customer's description of their solution, or the digest of their architecture.
- `solution.conditions` is what must be kept true of an Item while it is run. Every Condition must be carried by a Check.
- `source_ref` on an Item or a Condition is its id where it came from, such as an Element or a Check of a solution architecture. Keep it.
- Something operations needs that the solution does not provide is a **finding**. Record it; do not design around it.
## What binds
A stated value of a field marked `binds` in the catalogue, and every requirement under `intent.requirements`, must be named as the `source` of an Operational requirement. What was assumed does not bind: it is shown apart and may change when the customer answers.
## Who operates
- `sourcing.model` says whether the customer, a service provider, or both operate the solution. It decides which Parties the operating model has.
- `sourcing.delivery_countries`, when stated, are the only countries a Role may sit in.
## Cost
- `commercial.currency` is the currency every cost is stated in. It is always settled before work begins.
- The customer's own staff are counted and never priced.
- Nobody states a rate in a spec. Rates are the service provider's, in the rate card Skill.

View File

@@ -0,0 +1,214 @@
# The operating spec's catalogue: every field an operating spec may hold.
#
# A field is a dotted path. `type` is one of: string, text, integer, number, boolean,
# date, list, enum, records. `tier` says when it must be settled: essential (always,
# before any work), important (asked when someone is present) or optional. A tier
# may differ by the kind of engagement. `question` is what to ask the customer for
# it. `default` guides the choice when the customer says "decide". `affects` says
# what a wrong assumption would change. `binds: true` marks a choice an operating
# model must honour: stated by the customer, it has to reach an Operational
# requirement.
#
# This catalogue names no Cloud provider and no currency. A provider's bundle adds
# the allowed values for the fields marked `provider_values`.
schema_version: 1
kinds: [workload, platform_foundation, migration]
fields:
metadata.customer_case:
type: string
tier: essential
aliases: [metadata.engagement_id]
question: What short reference identifies this engagement?
metadata.client_name:
type: string
metadata.author:
type: string
metadata.revision:
type: string
# Where the spec was digested from, when it was: written by the digest.
metadata.source:
type: string
engagement.kind:
type: enum
values: [workload, platform_foundation, migration]
tier: essential
question: >-
Is what is to be operated an application workload, a platform foundation
such as a landing zone, or systems being migrated?
intent.statement:
type: text
tier: essential
question: In the customer's own words, what do they want from operations?
intent.requirements:
type: records
keys: [id, statement]
optional_keys: [category]
question: Which requirements for operations has the customer stated outright?
intent.business_outcomes:
# What the business wants from the solution, in the customer's words. Every
# measure in the operating model must show one of these.
type: records
keys: [id, outcome]
optional_keys: [target, category]
# Essential: everything measured is tied to one, so they are settled before
# anything is designed. With nobody to ask, they are assumed and shown apart.
tier: essential
question: >-
What does the business want from this solution, and how would it know it was
getting it? Give each outcome a short id, such as BO-1.
default: >-
Take them from what the request says the solution is for. Two or three, each
something the business would notice, such as changes reaching customers
sooner or outages being short.
affects: what is measured, and what each measure is for
intent.exclusions:
type: list
tier: important
question: What has the customer said is out of scope for operations?
default: Nothing is excluded beyond what the statement implies.
affects: which Capabilities are designed
# --- the solution to be operated -----------------------------------------------
solution.cloud_provider:
type: enum
provider_values: true
tier: essential
question: Which Cloud provider does the solution run on?
solution.items:
type: records
keys: [id, name, kind]
# `elements` are the ids of the parts of a design an Item stands for;
# `evidence` is the passage a drafted Item was read from.
optional_keys: [product, environments, source_ref, elements, evidence]
tier: essential
question: What is to be operated? List each thing, with a short id and its kind.
solution.conditions:
type: records
# `on` is the id of the Item it is a condition on, or a list of them.
keys: [id, "on", condition]
optional_keys: [source_ref, evidence]
tier: important
question: What must be kept true of each thing while it is run?
default: None beyond what the service levels state.
affects: the Checks the operating model must carry
solution.build.duration_days:
type: integer
tier: important
question: How many working days is the solution's build planned to take?
default: Not known; the transition is planned without a build to fit.
affects: when Capabilities must be ready
solution.build.customer_owned:
type: list
question: Which parts of the build does the customer or their supplier own?
solution.inherited_risks:
type: records
keys: ["on", risk]
optional_keys: [source_ref]
question: What is already known to be unverified or at risk in the solution?
# --- how it is to be operated ---------------------------------------------------
operations.support_hours:
binds: true
type: enum
values: [business, extended, 24x7]
tier: essential
question: >-
During which hours must faults be responded to: business hours, extended
hours, or every hour of every day?
operations.availability_hours:
type: enum
values: [business, extended, 24x7]
tier: important
question: During which hours must the solution be available to its users?
default: The same as the support hours.
affects: whether support hours are wide enough
operations.change_process:
type: text
tier: important
question: How are changes approved and released today, and must that continue?
default: Changes go through a pipeline with peer review and no separate board.
affects: change enablement and release
operations.monitoring:
type: text
question: Which monitoring and alerting is already in place or required?
sourcing.model:
binds: true
type: enum
values: [customer_operated, provider_operated, shared]
tier: essential
question: >-
Who operates the solution: the customer, a service provider, or the two
together?
sourcing.delivery_countries:
binds: true
type: list
tier: important
question: From which countries may the solution be operated? Use country codes.
default: No restriction beyond where the data may be handled.
affects: where Roles sit, and the cost
organisation.existing_teams:
type: records
keys: [name, party]
optional_keys: [skills, hours, country]
tier: {migration: essential, workload: important, platform_foundation: important}
question: Which teams exist today that will have a part in operating this?
default: None exist; every Role is new.
affects: which Roles are new and which are filled
organisation.headcount_limit:
type: integer
question: Is there a limit on the number of people operating this?
service_management.tooling:
type: text
tier: important
question: Which service management and on-call tooling must be used?
default: The Cloud provider's own operations tooling, with no separate system.
affects: tooling and the processes written for it
service_levels.priority_1_response_minutes:
binds: true
type: integer
tier: important
question: Within how many minutes must the most serious incident be acknowledged?
default: 30
affects: on-call arrangements and staffing
service_levels.recovery_time_hours:
binds: true
type: number
tier: important
question: Within how many hours must service be restored after a major failure?
default: 4
affects: continuity arrangements
compliance.frameworks:
type: list
tier: important
question: Which regulations or standards must operations comply with?
default: None beyond the organisation's own policies.
affects: which Capabilities apply and what they must evidence
commercial.currency:
type: enum
provider_values: true
tier: essential
question: In which currency should the cost of operating be stated?
commercial.monthly_run_budget:
type: number
question: Is there a monthly budget for operating this, in that currency?
transition.go_live_date:
type: date
question: When must the solution be in operation?
# --- how to run the agent -------------------------------------------------------
run.pre_approved:
type: boolean
question: Should the agent run without stopping for approvals?
run.overlay_skills:
type: list
question: Which overlay Skills should this Execution load?

36
skills/rate-card/SKILL.md Normal file
View File

@@ -0,0 +1,36 @@
---
name: rate-card
description: >-
The service provider's rate card: day rates by country and level, the
productive days in each country, conversion rates, and the smallest team that
can cover each pattern of hours. Use when placing Roles in countries and
choosing their levels, in the Target operating model and organisation design.
metadata:
scope: the Target operating model and organisation design
owner: the service provider
---
# Rate card
`assets/rates.yaml` is the service provider's rate card. The provider sets every figure in it. Nothing else does: not a caller, not an operating spec, and not you.
While its `version` starts with `draft`, the figures are placeholders and say so wherever a total is shown.
## What you do with it
- Give every Role a `country` and a `level` that the card has. A Role in a country or at a level the card lacks is refused when you submit.
- Place Roles only in the countries the operating spec lists under `sourcing.delivery_countries`, when it lists any.
- Choose the level from what the Role's Capabilities ask of it, using the description of each level in the card. Do not choose a level to reach a cost.
- Choose the country for a reason you state: the hours the Role must cover, the language it must work in, where the data may be handled, where the customer's own teams sit.
## What you never do
- Never write a number of people, a rate or a cost. People per Role, cost per Role and every total are computed from the card when you submit, and written into your Deliverable for you.
- Never use a rate from anywhere else. A rate a customer states is a question for the provider, recorded as a customer question, and changes nothing.
- Never convert a currency. The operating spec states the currency of the result, and the conversion is made from the card's fixed rates when you submit.
## What the card decides
- A Role that covers hours wider than a working day is never sized below the card's `minimum_roster` for that cover, however little effort its Capabilities need. Covering every hour from one country needs a rota; covering it from several countries in turn needs a Role in each, each on `business` or `extended` hours.
- Work outside business hours carries the card's `uplift`.
- The customer's own staff are never priced: a Role in a customer Party is shown as people, with no cost. Of the other Parties, only the kinds the card lists under `prices` are costed.

View File

@@ -0,0 +1,56 @@
# The service provider's rate card. Its prices are set by the provider who runs
# this agent, and by nobody else: not a caller, not an operating spec, not a model.
#
# PLACEHOLDERS: while `version` below starts with `draft`, every figure here was
# drafted to build and test the arithmetic and is not a price anyone has set.
# Set each one, then change `version`.
#
# `day_rate` is what one day of one person at that level is charged at, in
# `currency`. `productive_days` is the days in a year a person works on the
# service, after leave, public holidays and training. `hours_per_day` is the
# working day. Countries are ISO 3166-1 alpha-2 codes.
schema_version: 1
name: rate-card
version: draft-2026-10-13
currency: USD # what the rates below are written in
# The operating spec states the currency of the result (`commercial.currency`).
# These are the fixed rates used to convert into it: units of that currency for
# one unit of `currency` above. Set by the provider with the prices; no rate is
# looked up. A currency not listed cannot be asked for.
conversion:
as_of: 2026-10-13
rates: {USD: 1.00, GBP: 0.79, EUR: 0.92, CAD: 1.37, AUD: 1.52, JPY: 150, INR: 84}
# Which Parties this card prices, by kind. Roles of any other Party are counted
# as people and carry no cost. The customer's own staff are never priced, and
# `customer` may not be listed here.
prices: [provider]
levels:
engineer: Runs established procedures and resolves known faults.
senior: Diagnoses unfamiliar faults, automates, and leads a shift.
lead: Accountable for a Capability; designs the way of working.
# The fewest people who can staff a Role for a pattern of hours, and what is
# added to their rate for it. A Role is never sized below `minimum_roster`,
# however little effort its Capabilities need.
cover:
business: {minimum_roster: 1, uplift: 1.00} # one working day, five days a week
extended: {minimum_roster: 3, uplift: 1.10} # about sixteen hours, five days a week
24x7: {minimum_roster: 5, uplift: 1.25} # every hour of every day, from one country
countries:
US: {name: United States, productive_days: 225, hours_per_day: 8, day_rate: {engineer: 720, senior: 960, lead: 1250}}
CA: {name: Canada, productive_days: 222, hours_per_day: 7.5, day_rate: {engineer: 600, senior: 800, lead: 1020}}
GB: {name: United Kingdom, productive_days: 220, hours_per_day: 7.5, day_rate: {engineer: 600, senior: 800, lead: 1050}}
DE: {name: Germany, productive_days: 215, hours_per_day: 8, day_rate: {engineer: 640, senior: 850, lead: 1100}}
ES: {name: Spain, productive_days: 218, hours_per_day: 8, day_rate: {engineer: 420, senior: 560, lead: 740}}
PL: {name: Poland, productive_days: 220, hours_per_day: 8, day_rate: {engineer: 340, senior: 460, lead: 620}}
HU: {name: Hungary, productive_days: 220, hours_per_day: 8, day_rate: {engineer: 320, senior: 430, lead: 580}}
IN: {name: India, productive_days: 230, hours_per_day: 8, day_rate: {engineer: 180, senior: 260, lead: 380}}
PH: {name: Philippines, productive_days: 230, hours_per_day: 8, day_rate: {engineer: 190, senior: 270, lead: 390}}
BR: {name: Brazil, productive_days: 222, hours_per_day: 8, day_rate: {engineer: 260, senior: 360, lead: 500}}
MX: {name: Mexico, productive_days: 228, hours_per_day: 8, day_rate: {engineer: 280, senior: 380, lead: 520}}
JP: {name: Japan, productive_days: 225, hours_per_day: 8, day_rate: {engineer: 620, senior: 820, lead: 1080}}
AU: {name: Australia, productive_days: 222, hours_per_day: 7.6, day_rate: {engineer: 640, senior: 850, lead: 1100}}

0
tests/__init__.py Normal file
View File

5
tests/conftest.py Normal file
View File

@@ -0,0 +1,5 @@
# Env var setup that must run before the engine's settings are imported.
import os
# The suite must not pick up credentials from a developer's .env.
os.environ["ENV_FILE_EXPORT"] = "false"

7
tests/fixtures/architecture/README.md vendored Normal file
View File

@@ -0,0 +1,7 @@
# Architecture Executions, as fixtures
`case-08-landing-zone/` is the data files of one completed Execution of the solution architecture agent, copied unaltered from its evaluation results (run `20261011T185421Z`, at its commit `8b397cd`). The case is that agent's fictional benchmark landing zone; no customer's data is in it.
It is the contract between the two agents: the digest's tests read it, so a change to the shape of these files is caught here. Replace it by copying a newer Execution's files, never by editing these.
Not copied: the prose Deliverables, the Terraform, the solution guide and the working material. The digest reads none of them.

View File

@@ -0,0 +1,79 @@
[
{
"id": "C-1",
"name": "Resource Hierarchy and Environment Boundary Manager",
"requirements": [
"FR-1",
"NFR-1",
"NFR-8",
"NFR-9"
]
},
{
"id": "C-2",
"name": "Dedicated Private Hybrid Connectivity Gateway",
"requirements": [
"FR-2",
"NFR-3",
"NFR-14"
]
},
{
"id": "C-3",
"name": "Centralised Outbound Traffic Inspection and Egress Boundary",
"requirements": [
"FR-3",
"NFR-2",
"NFR-4"
]
},
{
"id": "C-4",
"name": "Centralised Immutable Audit Event and Compliance Log Repository",
"requirements": [
"FR-4",
"NFR-7",
"NFR-8"
]
},
{
"id": "C-5",
"name": "Automated Infrastructure Orchestration and Deployment Engine",
"requirements": [
"FR-5",
"NFR-10"
]
},
{
"id": "C-6",
"name": "Federated Identity and Role Governance Authority",
"requirements": [
"FR-6",
"NFR-5"
]
},
{
"id": "C-7",
"name": "Cryptographic Key Lifecycle and Encryption Controller",
"requirements": [
"NFR-6",
"NFR-8"
]
},
{
"id": "C-8",
"name": "Platform Telemetry and Operational Alerting Engine",
"requirements": [
"NFR-11",
"NFR-12",
"NFR-15"
]
},
{
"id": "C-9",
"name": "Financial Governance and Budget Enforcement Monitor",
"requirements": [
"NFR-13"
]
}
]

View File

@@ -0,0 +1,140 @@
{
"customer_case": "case-08-landing-zone",
"execution_id": "case-08-landing-zone-185426",
"session_id": "cdd6bfe7-2bd8-44af-8e15-9dcfa869eb84",
"intent_version": 1,
"workflow_request": "We are a UK-regulated payment platform expanding into Google Cloud. We need an\nenterprise-grade landing zone ready in under 10 weeks to support our core payments\nengine. All payment processing must comply with DORA and PCI-DSS v4.0. We have\nexisting on-premises systems that require private, low-latency connectivity over a\n10Gbps dedicated interconnect. We use Entra ID for corporate single sign-on and\nGitHub Actions for our delivery pipelines. Security requires that all egress\ntraffic flows through centralised inspection appliances before touching the public\ninternet, and customer data must be encrypted with keys we control.",
"baseline": null,
"overlay_skills": {},
"terraform_modules": false,
"scope": {
"status": "BASELINE_LOCKED",
"readiness": {
"supplied": 44,
"answered": 0,
"assumed": 12,
"unknown": 4,
"not_applicable": 3
}
},
"delivery_plan": {
"work_packages": 9,
"activities": 15,
"working_days": 36.0,
"weeks": 7.2,
"team_size": 3,
"window_weeks": 10,
"within_window": true,
"critical_path": [
"A-6",
"A-10",
"A-13",
"A-14",
"A-15"
]
},
"status": "completed",
"warnings": [],
"risks": [],
"assessment": {
"recommendations_quality": {
"specific": 0.91,
"actionable": 0.79,
"trade_offs": 0.95,
"traceable": 0.73
},
"mean": 0.84,
"meets_bar": true
},
"phases": {
"scoping": "approved",
"requirements_discovery": "approved",
"solution_architecture": "approved",
"solution_implementation": "approved",
"solution_validation": "approved",
"packaging": "approved"
},
"solution_guide": "solution-architecture-guide.md",
"customer_questions": "customer-questions.md",
"changes": null,
"deliverables": {
"scope": {
"document": "deliverables/scope.md",
"attachments": [
"scope.json",
"scope.yaml"
],
"status": "submitted"
},
"requirements": {
"document": "deliverables/requirements.md",
"attachments": [
"requirements.json"
],
"status": "submitted"
},
"technical_decomposition": {
"document": "deliverables/technical_decomposition.md",
"attachments": [
"components.json"
],
"status": "submitted"
},
"product_selection": {
"document": "deliverables/product_selection.md",
"attachments": [
"products.json"
],
"status": "submitted"
},
"design_recommendations": {
"document": "deliverables/design_recommendations.md",
"attachments": [
"recommendations.json"
],
"status": "submitted"
},
"target_architecture": {
"document": "deliverables/target_architecture.md",
"attachments": [
"architecture.mmd",
"target-architecture.json"
],
"status": "submitted"
},
"architecture_description": {
"document": "deliverables/architecture_description.md",
"attachments": [],
"status": "submitted"
},
"deployment_guidance": {
"document": "deliverables/deployment_guidance.md",
"attachments": [
"implementation.json",
"terraform/main.tf",
"terraform/variables.tf",
"terraform/versions.tf"
],
"status": "submitted"
},
"validation_results": {
"document": "deliverables/validation_results.md",
"attachments": [
"verdicts.json"
],
"status": "submitted"
},
"work_breakdown": {
"document": "deliverables/work_breakdown.md",
"attachments": [
"work-breakdown.json"
],
"status": "submitted"
},
"solution_guide": {
"document": "deliverables/solution_guide.md",
"attachments": [],
"status": "submitted"
}
}
}

View File

@@ -0,0 +1,38 @@
[
{
"component": "C-1",
"product": "Resource Manager"
},
{
"component": "C-2",
"product": "Cloud Interconnect"
},
{
"component": "C-3",
"product": "Palo Alto Networks VM-Series"
},
{
"component": "C-4",
"product": "Cloud Storage"
},
{
"component": "C-5",
"product": "GitHub Actions"
},
{
"component": "C-6",
"product": "IAM Workload Identity Federation"
},
{
"component": "C-7",
"product": "Cloud KMS"
},
{
"component": "C-8",
"product": "Cloud Monitoring"
},
{
"component": "C-9",
"product": "Cloud Billing"
}
]

View File

@@ -0,0 +1,248 @@
[
{
"id": "DR-SEC-1",
"pillar": "security",
"text": "Implement a VPC Service Controls perimeter around all Cardholder Data Environment (CDE) projects and the centralized audit log buckets.",
"components": [
"C-1",
"C-4"
],
"conditions": [
"A VPC Service Controls perimeter encloses the CDE and audit log projects.",
"The VPC Service Controls perimeter restricts ingress and egress to authorized traffic."
],
"outside_code": []
},
{
"id": "DR-SEC-2",
"pillar": "security",
"text": "Apply Organization Policy constraints to disable default network creation, deny external IPs, prohibit service account key creation, and restrict resource locations.",
"components": [
"C-1"
],
"conditions": [
"The compute.skipDefaultNetworkCreation constraint is enforced.",
"The compute.vmExternalIpAccess constraint is enforced to deny all.",
"The iam.disableServiceAccountKeyCreation constraint is enforced.",
"The gcp.resourceLocations constraint restricts resources to europe-west2 and europe-west1."
],
"outside_code": []
},
{
"id": "DR-SEC-3",
"pillar": "security",
"text": "Enable Confidential Computing on the payment engine Compute Engine instances to encrypt data in-use.",
"components": [],
"conditions": [
"Confidential Computing is enabled on the core payment engine Compute Engine instances."
],
"outside_code": []
},
{
"id": "DR-SEC-4",
"pillar": "security",
"text": "Deploy Sensitive Data Protection (Cloud DLP) inspection templates to detect and mask PAN data.",
"components": [
"C-4"
],
"conditions": [
"Sensitive Data Protection (Cloud DLP) inspection templates are configured to detect and mask PAN data."
],
"outside_code": []
},
{
"id": "DR-SEC-5",
"pillar": "security",
"text": "Enable Security Command Center at the organization level with Event and Container Threat Detection.",
"components": [
"C-1",
"C-8"
],
"conditions": [
"Security Command Center is activated at the organization level.",
"Event Threat Detection and Container Threat Detection are enabled in SCC."
],
"outside_code": []
},
{
"id": "DR-SEC-6",
"pillar": "security",
"text": "Implement event-driven automated responses using Eventarc and Cloud Run functions for SCC findings.",
"components": [
"C-8"
],
"conditions": [
"Eventarc is configured to route high-severity SCC findings.",
"Cloud Run functions are configured to execute automated remediation on Eventarc triggers."
],
"outside_code": []
},
{
"id": "DR-REL-1",
"pillar": "reliability",
"text": "Provision redundant Dedicated Cloud Interconnect circuits across distinct physical edge colocation facilities, terminating on redundant Cloud Routers with BFD.",
"components": [
"C-2"
],
"conditions": [
"Dedicated Cloud Interconnect circuits are provisioned across redundant edge colocation facilities.",
"Cloud Interconnects terminate on redundant Cloud Routers in each region.",
"Bidirectional Forwarding Detection (BFD) is enabled on the Cloud Router BGP sessions."
],
"outside_code": []
},
{
"id": "DR-REL-2",
"pillar": "reliability",
"text": "Deploy Palo Alto VM-Series firewalls in a regional Managed Instance Group across at least three zones, fronted by an Internal Passthrough Network Load Balancer.",
"components": [
"C-3"
],
"conditions": [
"Palo Alto VM-Series firewalls are deployed in a regional Managed Instance Group (MIG) across at least three zones.",
"An Internal Passthrough Network Load Balancer routes traffic to the VM-Series MIG.",
"Health checks are configured on the internal load balancer to probe firewall health."
],
"outside_code": []
},
{
"id": "DR-REL-3",
"pillar": "reliability",
"text": "Provision dedicated regional Cloud KMS key rings in both europe-west2 and europe-west1, granting workload identity service accounts Decrypter roles on both.",
"components": [
"C-7",
"C-6"
],
"conditions": [
"Regional Cloud KMS key rings are provisioned in both europe-west2 and europe-west1.",
"Workload Identity Federation service accounts are granted Decrypter roles on both regional key sets."
],
"outside_code": []
},
{
"id": "DR-REL-4",
"pillar": "reliability",
"text": "Configure the Cloud Storage buckets for audit logging as Dual-Region spanning europe-west2 and europe-west1.",
"components": [
"C-4"
],
"conditions": [
"Audit log Cloud Storage buckets are configured as Dual-Region spanning europe-west2 and europe-west1."
],
"outside_code": []
},
{
"id": "DR-REL-5",
"pillar": "reliability",
"text": "Isolate the Terraform state of the primary region from the standby region using multi-region Cloud Storage buckets with Object Versioning.",
"components": [
"C-5"
],
"conditions": [
"Terraform state for primary and standby regions is decoupled into separate failure domains.",
"Terraform state is stored in multi-region Cloud Storage buckets with Object Versioning enabled."
],
"outside_code": []
},
{
"id": "DR-OP-1",
"pillar": "operational-excellence",
"text": "Integrate Policy-as-Code validation into the GitHub Actions pull request workflows.",
"components": [
"C-5"
],
"conditions": [
"GitHub Actions workflows include automated Policy-as-Code validation steps prior to deployment execution."
],
"outside_code": []
},
{
"id": "DR-OP-2",
"pillar": "operational-excellence",
"text": "Establish explicit Service Level Indicators (SLIs) and configure multi-burn-rate alerting policies in Cloud Monitoring.",
"components": [
"C-8"
],
"conditions": [
"Service Level Objectives (SLOs) and SLIs are defined in Cloud Monitoring Service Monitoring.",
"Multi-burn-rate alerting policies are configured based on the SLO error budget."
],
"outside_code": []
},
{
"id": "DR-OP-3",
"pillar": "operational-excellence",
"text": "Activate the IAM Recommender to continuously evaluate role bindings.",
"components": [
"C-6"
],
"conditions": [
"IAM Recommender is activated on the foundation projects."
],
"outside_code": [
"Permissions are continuously rightsized based on IAM Recommender findings."
]
},
{
"id": "DR-COST-1",
"pillar": "cost-optimization",
"text": "Enforce a mandatory labeling taxonomy through Organization Policies and export granular billing data to BigQuery.",
"components": [
"C-9",
"C-1"
],
"conditions": [
"Cloud Billing export to BigQuery is enabled.",
"A mandatory labeling taxonomy is enforced via Organization Policy Service tags."
],
"outside_code": []
},
{
"id": "DR-COST-2",
"pillar": "cost-optimization",
"text": "Implement Cloud Storage Object Lifecycle Management to transition logs to Nearline storage after 30 days and to Coldline or Archive storage after 90 days.",
"components": [
"C-4"
],
"conditions": [
"Object Lifecycle Management rules transition Cloud Storage objects to Nearline storage after 30 days.",
"Object Lifecycle Management rules transition Cloud Storage objects to Coldline or Archive storage after 90 days."
],
"outside_code": []
},
{
"id": "DR-PERF-1",
"pillar": "performance-optimization",
"text": "Configure Cloud Routers with regional dynamic routing to localize egress traffic within the warm standby region.",
"components": [
"C-2"
],
"conditions": [
"Cloud Routers are configured with regional dynamic routing mode."
],
"outside_code": []
},
{
"id": "DR-PERF-2",
"pillar": "performance-optimization",
"text": "Instrument the deployment with OpenTelemetry exporting to Cloud Trace, and use Cloud Profiler on the compute instances.",
"components": [
"C-8"
],
"conditions": [
"OpenTelemetry tracing is configured to export to Cloud Trace.",
"Cloud Profiler agents are enabled on the core payments engine instances."
],
"outside_code": []
},
{
"id": "DR-SUST-1",
"pillar": "sustainability",
"text": "Implement a Pilot Light warm standby by maintaining active data replication while scaling the standby compute instances to a minimal baseline.",
"components": [],
"conditions": [
"Secondary region compute instances are scaled to a minimal baseline (Pilot Light) configuration during normal operations."
],
"outside_code": []
}
]

View File

@@ -0,0 +1,280 @@
[
{
"id": "FR-1",
"kind": "functional",
"text": "The platform foundation isolates development, staging, and production environments across distinct Google Cloud projects and resource folders.",
"from": [
"REQ-02"
]
},
{
"id": "FR-2",
"kind": "functional",
"text": "The platform foundation establishes private bidirectional network connectivity between Google Cloud and on-premises core payment systems over Cloud Interconnect.",
"from": [
"REQ-03"
]
},
{
"id": "FR-3",
"kind": "functional",
"text": "The platform foundation routes all outbound internet-bound traffic from workload spoke networks through centralised network security inspection appliances.",
"from": [
"REQ-03"
]
},
{
"id": "FR-4",
"kind": "functional",
"text": "The platform foundation captures and exports all administrative and data access audit logs across environments into a centralised, tamper-resistant log repository.",
"from": [
"REQ-02"
]
},
{
"id": "FR-5",
"kind": "functional",
"text": "The platform foundation provisions and manages all cloud infrastructure as code via automated CI/CD pipelines executing validated pull request workflows.",
"from": [
"REQ-01"
]
},
{
"id": "FR-6",
"kind": "functional",
"text": "The platform foundation federates corporate identity for single sign-on and enforces least-privilege role-based access control for platform administrators and application squads.",
"from": [
"REQ-04"
]
},
{
"id": "NFR-1",
"kind": "non_functional",
"text": "Environments are partitioned into development, staging, and production tiers, enforcing corp_workload controls on lower environments and pci_regulated_workload controls on production.",
"conditions": [
"Separate projects or folders exist for development, staging, and production environment tiers.",
"Production environments enforce pci_regulated_workload archetype security controls.",
"Development and staging environments enforce corp_workload archetype security controls."
],
"outside_code": [],
"from": [
"REQ-02",
"governance.environment_tiers"
]
},
{
"id": "NFR-2",
"kind": "non_functional",
"text": "Network connectivity is structured as a Shared VPC hub-and-spoke topology with zero direct internet ingress to workload subnets.",
"conditions": [
"A Shared VPC host project is connected to service projects across environment tiers.",
"Workload subnets have private Google access enabled and no external public IP addresses assigned.",
"Ingress firewall rules deny uninspected public internet traffic to workload subnets."
],
"outside_code": [],
"from": [
"REQ-02",
"networking.topology",
"networking.exposure"
]
},
{
"id": "NFR-3",
"kind": "non_functional",
"text": "Hybrid connectivity to on-premises core payment systems is enabled via a 10Gbps Dedicated Cloud Interconnect.",
"conditions": [
"Dedicated Cloud Interconnect connections are configured for hybrid connectivity.",
"Cloud Interconnect capacity is provisioned at 10Gbps bandwidth.",
"Interconnect VLAN attachments are associated with Cloud Routers in the Shared VPC hub."
],
"outside_code": [],
"from": [
"REQ-03",
"networking.hybrid_connectivity.enabled",
"networking.hybrid_connectivity.type",
"networking.hybrid_connectivity.bandwidth_gbps"
]
},
{
"id": "NFR-4",
"kind": "non_functional",
"text": "All outbound internet traffic from workload spokes is forced through centralised inspection appliances deployed in the hub network.",
"conditions": [
"Inspection appliance instances are deployed in a centralised hub network.",
"Custom VPC route tables direct 0.0.0.0/0 egress traffic from spokes to the centralised inspection appliance instances or internal load balancer."
],
"outside_code": [],
"from": [
"REQ-03",
"networking.egress_control.model"
]
},
{
"id": "NFR-5",
"kind": "non_functional",
"text": "Administrative access is authenticated via Entra ID single sign-on, and delivery pipelines authenticate via Workload Identity Federation without stored service account keys.",
"conditions": [
"Workload Identity Federation pools and providers are configured for GitHub Actions authentication.",
"IAM bindings grant roles to federated GitHub Actions identities without downloadable service account keys.",
"Corporate single sign-on is configured with Entra ID as the identity source of truth."
],
"outside_code": [],
"from": [
"REQ-04",
"governance.identity_source_of_truth",
"governance.pipeline_authentication"
]
},
{
"id": "NFR-6",
"kind": "non_functional",
"text": "Customer confidential data is encrypted at rest using customer-managed encryption keys with a 90-day rotation schedule.",
"conditions": [
"Cloud KMS customer-managed encryption keys are configured for infrastructure resources storing confidential data.",
"Cloud KMS keys have an automated rotation period set to 90 days (7776000s)."
],
"outside_code": [],
"from": [
"REQ-04",
"compliance.encryption.type",
"compliance.encryption.key_rotation_days"
]
},
{
"id": "NFR-7",
"kind": "non_functional",
"text": "Audit logs across all environments are retained for 365 days in a storage bucket protected by an immutable retention policy.",
"conditions": [
"An organization-level or project-level log sink exports audit logs to a centralised Cloud Storage bucket.",
"The centralised audit log storage bucket specifies a retention period of 365 days (31536000 seconds).",
"The centralised audit log storage bucket has retention policy locking enabled to prevent modification or early deletion."
],
"outside_code": [],
"from": [
"REQ-02",
"compliance.audit_logging.retention_days",
"compliance.audit_logging.immutable_retention_policy"
]
},
{
"id": "NFR-8",
"kind": "non_functional",
"text": "The platform foundation enforces DORA, ISO27001, and PCI-DSS v4.0 regulatory baselines and protects confidential data classification.",
"conditions": [
"Organization policy constraints enforce perimeter isolation and restrict unauthenticated resource sharing.",
"IAM policies restrict access to confidential data resources to authorized service identities and roles."
],
"outside_code": [
"The landing zone configuration aligns with DORA operational resiliency guidelines and PCI-DSS v4.0 perimeter isolation standards during external audit."
],
"from": [
"REQ-02",
"compliance.frameworks",
"compliance.data_classification"
]
},
{
"id": "NFR-9",
"kind": "non_functional",
"text": "Platform foundation resources are provisioned across europe-west2 (primary) and europe-west1 (secondary) in a warm standby model, restricting data residency to the UK and EU.",
"conditions": [
"Primary landing zone network and foundation resources are provisioned in europe-west2.",
"Secondary landing zone network and foundation standby resources are provisioned in europe-west1.",
"Resource location organization policies restrict resource and data storage locations to europe-west2 and europe-west1."
],
"outside_code": [
"The secondary region in europe-west1 operates as a warm standby ready to receive failover traffic.",
"Data residency is guaranteed within the United Kingdom and European Union."
],
"from": [
"REQ-02",
"estate.regions.primary",
"estate.regions.secondary",
"estate.regions.resiliency_tier",
"estate.data_residency"
]
},
{
"id": "NFR-10",
"kind": "non_functional",
"text": "The platform foundation landing zone is delivered and operational for workload ingestion within 10 weeks.",
"conditions": [
"All landing zone foundation resources are codified in modular Terraform configurations."
],
"outside_code": [
"Platform foundation is fully provisioned and validated for core payments engine workload ingestion within 10 weeks."
],
"from": [
"REQ-01"
]
},
{
"id": "NFR-11",
"kind": "non_functional",
"text": "The platform foundation maintains 99.9% availability with a 4-hour recovery time objective (RTO) and a 1-hour recovery point objective (RPO).",
"conditions": [
"Regional networking and core management services are deployed across at least three zones in the primary region."
],
"outside_code": [
"Platform foundation infrastructure maintains measured monthly availability of at least 99.9%.",
"Disaster recovery activation achieves a recovery time objective (RTO) of 4 hours.",
"Disaster recovery processes achieve a recovery point objective (RPO) of 1 hour."
],
"from": [
"REQ-02"
]
},
{
"id": "NFR-12",
"kind": "non_functional",
"text": "Platform operations monitoring and alerting are implemented using Google Cloud Operations Suite with 24/7 alerting capabilities.",
"conditions": [
"Cloud Monitoring workspaces and notification channels are configured for the platform foundation.",
"Alert policies are configured for Interconnect link degradation, appliance health, and egress routing failures."
],
"outside_code": [
"Alert notifications trigger operational incident workflows for the 24/7 central platform operations team."
],
"from": []
},
{
"id": "NFR-13",
"kind": "non_functional",
"text": "Infrastructure costs are tracked against a monthly budget of 25,000 GBP with automated alerting.",
"conditions": [
"A Cloud Billing budget is configured for the foundation projects with currency set to GBP.",
"Budget alert thresholds are configured at 50%, 80%, 90%, and 100% of the 25,000 GBP monthly budget."
],
"outside_code": [
"Monthly cloud foundation expenditure does not exceed the target allocation without operational sign-off."
],
"from": []
},
{
"id": "NFR-14",
"kind": "non_functional",
"text": "Hybrid networking provides dedicated 10Gbps unthrottled throughput with minimal latency between Google Cloud and on-premises core systems.",
"conditions": [
"Cloud Routers configured for Interconnect advertise non-overlapping RFC 1918 BGP routes.",
"Dedicated Interconnect VLAN attachments are configured with dedicated 10Gbps bandwidth allocation."
],
"outside_code": [
"Hybrid connection provides deterministic low latency suitable for real-time payment transaction processing."
],
"from": [
"REQ-03"
]
},
{
"id": "NFR-15",
"kind": "non_functional",
"text": "Landing zone resources are deployed in Google Cloud regions with strong low-carbon energy ratings.",
"conditions": [
"Foundation resources are constrained to europe-west2 (London) and europe-west1 (Belgium) regions."
],
"outside_code": [
"Deployments optimize carbon-free energy utilization based on Google Cloud regional CFE scores."
],
"from": []
}
]

View File

@@ -0,0 +1,233 @@
{
"kind": "platform_foundation",
"stated": {
"engagement": {
"kind": "platform_foundation"
},
"intent": {
"statement": "We are a UK-regulated payment platform expanding into Google Cloud. We need an\nenterprise-grade landing zone ready in under 10 weeks to support our core payments\nengine. All payment processing must comply with DORA and PCI-DSS v4.0. We have\nexisting on-premises systems that require private, low-latency connectivity over a\n10Gbps dedicated interconnect. We use Entra ID for corporate single sign-on and\nGitHub Actions for our delivery pipelines. Security requires that all egress\ntraffic flows through centralised inspection appliances before touching the public\ninternet, and customer data must be encrypted with keys we control.\n",
"requirements": [
{
"id": "REQ-01",
"category": "Timeline",
"statement": "Platform foundation must be ready for workload ingestion within 10 weeks."
},
{
"id": "REQ-02",
"category": "Compliance",
"statement": "Must achieve DORA resiliency alignment and PCI-DSS v4.0 perimeter isolation."
},
{
"id": "REQ-03",
"category": "Networking",
"statement": "10Gbps on-premises hybrid connectivity; centralised inspection appliances for egress."
},
{
"id": "REQ-04",
"category": "Identity and security",
"statement": "Workload identity federation with GitHub Actions; Entra ID single sign-on; customer-managed encryption keys."
}
],
"exclusions": [
"Application container deployment; workloads are managed by application squads.",
"An active-active secondary region in phase 1; warm standby is acceptable."
]
},
"estate": {
"cloud_provider": "gcp",
"mode": "greenfield",
"regions": {
"primary": "europe-west2",
"secondary": "europe-west1",
"resiliency_tier": "warm_standby"
},
"data_residency": "United Kingdom and European Union",
"existing_systems": "On-premises payment systems that stay in place and connect over the interconnect."
},
"workload": {
"availability": {
"hours": "always"
}
},
"compliance": {
"industry": "financial_services",
"frameworks": [
"DORA",
"ISO27001",
"PCI_DSS_v4"
],
"data_classification": "confidential",
"encryption": {
"type": "cmek",
"key_rotation_days": 90
},
"audit_logging": {
"retention_days": 365,
"immutable_retention_policy": true
}
},
"networking": {
"exposure": "internal_only",
"topology": "shared_vpc_hub_and_spoke",
"hybrid_connectivity": {
"enabled": true,
"type": "cloud_interconnect",
"bandwidth_gbps": 10
},
"egress_control": {
"model": "centralized_appliance"
}
},
"governance": {
"identity_source_of_truth": "Entra ID",
"pipeline_authentication": "workload identity federation",
"environment_tiers": [
{
"name": "development",
"archetype": "corp_workload"
},
{
"name": "staging",
"archetype": "corp_workload"
},
{
"name": "production",
"archetype": "pci_regulated_workload"
}
]
},
"operations": {
"operating_model": "central platform team brokering access for application squads",
"support_hours": "always"
},
"delivery": {
"window_weeks": 10,
"commercial_model": "fixed_price_milestone",
"iac_tooling": "terraform",
"ci_cd": "GitHub Actions",
"team_size": 3,
"working_days_per_week": 5
},
"cost": {
"currency": "GBP"
}
},
"binding": [
"estate.regions.primary",
"estate.regions.secondary",
"estate.regions.resiliency_tier",
"estate.data_residency",
"compliance.frameworks",
"compliance.data_classification",
"compliance.encryption.type",
"compliance.encryption.key_rotation_days",
"compliance.audit_logging.retention_days",
"compliance.audit_logging.immutable_retention_policy",
"networking.exposure",
"networking.topology",
"networking.hybrid_connectivity.enabled",
"networking.hybrid_connectivity.type",
"networking.hybrid_connectivity.bandwidth_gbps",
"networking.egress_control.model",
"governance.identity_source_of_truth",
"governance.pipeline_authentication",
"governance.environment_tiers"
],
"assumed": [
{
"field": "intent.success_criteria",
"value": [
"Platform foundation ready for workload ingestion within 10 weeks",
"Compliance achieved with DORA and PCI-DSS v4.0 requirements",
"Hybrid connectivity established via 10Gbps Dedicated Interconnect",
"Centralised egress inspection operational",
"Identity federation and CMEK security controls implemented"
],
"why": "Assumed standard success criteria based on delivery within 10 weeks and compliance with DORA and PCI-DSS v4.0."
},
{
"field": "workload.availability.target",
"value": "99.9%",
"why": "Assumed 99.9% availability target for platform foundation infrastructure aligned with warm standby secondary region."
},
{
"field": "workload.availability.recovery_time",
"value": "4 hours",
"why": "Assumed 4 hours recovery time objective (RTO) for platform foundation services in warm standby."
},
{
"field": "workload.availability.recovery_point",
"value": "1 hour",
"why": "Assumed 1 hour recovery point objective (RPO) for platform foundation configuration and state."
},
{
"field": "networking.egress_control.appliance_vendor",
"value": "Palo Alto Networks VM-Series",
"why": "Assumed Palo Alto Networks VM-Series as enterprise inspection appliance for centralized egress control."
},
{
"field": "networking.cidr_allocations",
"value": "RFC 1918 non-overlapping IP allocation (10.0.0.0/16 partitioned across environments) sized for growth",
"why": "Assumed RFC 1918 non-overlapping IP allocation (10.0.0.0/16 partitioned across environments) sized for growth."
},
{
"field": "operations.monitoring",
"value": "Google Cloud Operations Suite with availability and error alerting",
"why": "Assumed native Google Cloud Operations Suite (Cloud Monitoring and Cloud Logging) with availability and error alerting."
},
{
"field": "operations.change_process",
"value": "GitHub Actions pull request workflow with automated Terraform plan validation and peer review",
"why": "Assumed GitHub Actions pull request workflow with automated Terraform plan validation and peer approval."
},
{
"field": "integrations",
"value": [
{
"criticality": "critical",
"direction": "bidirectional",
"name": "On-premises core payment systems"
}
],
"why": "Assumed integration with existing on-premises core payment systems via Dedicated Interconnect."
},
{
"field": "cost.monthly_budget",
"value": 25000,
"why": "Assumed indicative monthly foundation infrastructure budget of 25000 GBP for 10Gbps interconnect, inspection appliances, and core services."
},
{
"field": "constraints.products_required",
"value": [],
"why": "No mandated third-party products beyond stated tooling (GCP, Terraform, GitHub Actions, Entra ID)."
},
{
"field": "constraints.products_forbidden",
"value": [],
"why": "No forbidden products specified."
}
],
"to_decide": [],
"to_ask": [
{
"field": "metadata.engagement_reference",
"prompt": "What is metadata.engagement_reference?",
"severity": "LOW"
},
{
"field": "delivery.target_start_date",
"prompt": "When should work start?",
"severity": "LOW"
},
{
"field": "constraints.other",
"prompt": "What is constraints.other?",
"severity": "LOW"
}
],
"not_applicable": [
"workload.users.count",
"workload.users.peak_concurrent",
"workload.users.who"
]
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,529 @@
{
"dry_run": {
"evidence": "Success! The configuration is valid.",
"verdict": "pass"
},
"checks": [
{
"check": "K-1",
"evidence": "display_name = \"Dev Folder\"",
"verdict": "pass"
},
{
"check": "K-2",
"evidence": "display_name = \"Staging Folder\"",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-3",
"evidence": "display_name = \"Production Folder\""
},
{
"verdict": "pass",
"evidence": "tag_value = google_tags_tag_value.corp_workload.id",
"check": "K-4"
},
{
"check": "K-5",
"verdict": "pass",
"evidence": "host_project = google_compute_shared_vpc_host_project.hub.project"
},
{
"verdict": "pass",
"evidence": "host_project = google_compute_shared_vpc_host_project.hub.project",
"check": "K-6"
},
{
"check": "K-7",
"evidence": "host_project = google_compute_shared_vpc_host_project.hub.project",
"verdict": "pass"
},
{
"verdict": "pass",
"evidence": "private_ip_google_access = true",
"check": "K-8"
},
{
"check": "K-9",
"evidence": "private_ip_google_access = true",
"verdict": "pass"
},
{
"check": "K-10",
"evidence": "direction = \"INGRESS\"",
"verdict": "pass"
},
{
"check": "K-11",
"verdict": "pass",
"evidence": "protocol = \"all\""
},
{
"verdict": "pass",
"evidence": "source_ranges = [\"0.0.0.0/0\"]",
"check": "K-12"
},
{
"check": "K-13",
"evidence": "type = \"DEDICATED\"",
"verdict": "pass"
},
{
"evidence": "type = \"DEDICATED\"",
"verdict": "pass",
"check": "K-14"
},
{
"check": "K-15",
"verdict": "pass",
"evidence": "bandwidth = \"BPS_10G\""
},
{
"evidence": "bandwidth = \"BPS_10G\"",
"verdict": "pass",
"check": "K-16"
},
{
"verdict": "pass",
"evidence": "router = google_compute_router.primary.id",
"check": "K-17"
},
{
"evidence": "network = google_compute_network.hub.id",
"verdict": "pass",
"check": "K-18"
},
{
"check": "K-19",
"evidence": "dest_range = \"0.0.0.0/0\"",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-20",
"evidence": "next_hop_ilb = google_compute_forwarding_rule.palo_alto_ilb.id"
},
{
"verdict": "pass",
"check": "K-21",
"evidence": "issuer_uri = \"https://token.actions.githubusercontent.com\""
},
{
"verdict": "pass",
"check": "K-22",
"evidence": "member = \"principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.wif_pool.name}/attribute.repository/my-org/my-repo\""
},
{
"evidence": "idp_metadata_xml = \"<EntityDescriptor></EntityDescriptor>\"",
"verdict": "pass",
"check": "K-23"
},
{
"evidence": "purpose = \"ENCRYPT_DECRYPT\"",
"verdict": "pass",
"check": "K-24"
},
{
"check": "K-25",
"evidence": "purpose = \"ENCRYPT_DECRYPT\"",
"verdict": "pass"
},
{
"check": "K-26",
"verdict": "pass",
"evidence": "rotation_period = \"7776000s\""
},
{
"check": "K-27",
"evidence": "rotation_period = \"7776000s\"",
"verdict": "pass"
},
{
"evidence": "destination = \"storage.googleapis.com/${google_storage_bucket.audit_logs.name}\"",
"verdict": "pass",
"check": "K-28"
},
{
"check": "K-29",
"verdict": "pass",
"evidence": "retention_period = 31536000"
},
{
"check": "K-30",
"verdict": "pass",
"evidence": "is_locked = true"
},
{
"evidence": "restricted_services = [\"compute.googleapis.com\"]",
"verdict": "pass",
"check": "K-31"
},
{
"check": "K-32",
"evidence": "role = \"roles/cloudkms.cryptoKeyEncrypterDecrypter\"",
"verdict": "pass"
},
{
"check": "K-33",
"evidence": "region = \"europe-west2\"",
"verdict": "pass"
},
{
"check": "K-34",
"verdict": "pass",
"evidence": "region = \"europe-west2\""
},
{
"verdict": "pass",
"check": "K-35",
"evidence": "region = \"europe-west1\""
},
{
"verdict": "pass",
"evidence": "constraint = \"gcp.resourceLocations\"",
"check": "K-36"
},
{
"check": "K-37",
"verdict": "pass",
"evidence": "values = [\"in:europe-west2\", \"in:europe-west1\"]"
},
{
"check": "K-38",
"evidence": "project = var.project_id",
"verdict": "pass"
},
{
"verdict": "pass",
"evidence": "distribution_policy_zones = [\"europe-west2-a\", \"europe-west2-b\", \"europe-west2-c\"]",
"check": "K-39"
},
{
"verdict": "pass",
"check": "K-40",
"evidence": "notification_channels = [google_monitoring_notification_channel.email.name]"
},
{
"verdict": "pass",
"evidence": "condition_threshold is defined",
"check": "K-41"
},
{
"evidence": "currency_code = \"GBP\"",
"check": "K-42",
"verdict": "pass"
},
{
"verdict": "pass",
"evidence": "threshold_percent includes 0.5, 0.8, 0.9, 1.0",
"check": "K-43"
},
{
"check": "K-44",
"verdict": "pass",
"evidence": "units = \"25000\""
},
{
"evidence": "advertise_mode = \"CUSTOM\"",
"verdict": "pass",
"check": "K-45"
},
{
"verdict": "pass",
"evidence": "advertise_mode = \"CUSTOM\"",
"check": "K-46"
},
{
"verdict": "pass",
"evidence": "bandwidth = \"BPS_10G\"",
"check": "K-47"
},
{
"evidence": "bandwidth = \"BPS_10G\"",
"verdict": "pass",
"check": "K-48"
},
{
"check": "K-49",
"verdict": "pass",
"evidence": "values = [\"in:europe-west2\", \"in:europe-west1\"]"
},
{
"evidence": "resources = [\"projects/${var.prod_project_id}\", \"projects/${var.project_id}\"]",
"check": "K-50",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-51",
"evidence": "ingress_policies and egress_policies are configured"
},
{
"verdict": "pass",
"evidence": "constraint = \"compute.skipDefaultNetworkCreation\"",
"check": "K-52"
},
{
"evidence": "enforced = true",
"verdict": "pass",
"check": "K-53"
},
{
"check": "K-54",
"evidence": "constraint = \"compute.vmExternalIpAccess\"",
"verdict": "pass"
},
{
"verdict": "pass",
"evidence": "all = true",
"check": "K-55"
},
{
"evidence": "constraint = \"iam.disableServiceAccountKeyCreation\"",
"check": "K-56",
"verdict": "pass"
},
{
"check": "K-57",
"evidence": "enforced = true",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-58",
"evidence": "constraint = \"gcp.resourceLocations\""
},
{
"evidence": "values = [\"in:europe-west2\", \"in:europe-west1\"]",
"verdict": "pass",
"check": "K-59"
},
{
"verdict": "pass",
"check": "K-60",
"evidence": "enable_confidential_compute = true"
},
{
"check": "K-61",
"evidence": "name = \"CREDIT_CARD_NUMBER\"",
"verdict": "pass"
},
{
"check": "K-62",
"evidence": "event_data_content_type = \"application/json\"",
"verdict": "pass"
},
{
"evidence": "attribute = \"type\", value = \"google.cloud.securitycenter.finding.v1.saved\"",
"check": "K-63",
"verdict": "pass"
},
{
"evidence": "event_type = \"google.cloud.securitycenter.finding.v1.saved\"",
"verdict": "pass",
"check": "K-64"
},
{
"verdict": "pass",
"evidence": "interconnect = var.interconnect_1_url",
"check": "K-65"
},
{
"evidence": "interconnect = var.interconnect_2_url",
"verdict": "pass",
"check": "K-66"
},
{
"check": "K-67",
"evidence": "router = google_compute_router.primary.id",
"verdict": "pass"
},
{
"check": "K-68",
"evidence": "session_initialization_mode = \"ACTIVE\"",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-69",
"evidence": "distribution_policy_zones = [\"europe-west2-a\", \"europe-west2-b\", \"europe-west2-c\"]"
},
{
"evidence": "load_balancing_scheme = \"INTERNAL\"",
"check": "K-70",
"verdict": "pass"
},
{
"evidence": "health_checks = [google_compute_health_check.palo_alto_ilb_hc.id]",
"verdict": "pass",
"check": "K-71"
},
{
"verdict": "pass",
"check": "K-72",
"evidence": "location = \"europe-west2\""
},
{
"evidence": "location = \"europe-west1\"",
"check": "K-73",
"verdict": "pass"
},
{
"evidence": "role = \"roles/cloudkms.cryptoKeyDecrypter\"",
"verdict": "pass",
"check": "K-74"
},
{
"verdict": "pass",
"evidence": "data_locations = [\"EUROPE-WEST1\", \"EUROPE-WEST2\"]",
"check": "K-75"
},
{
"check": "K-76",
"verdict": "pass",
"evidence": "name = \"${var.project_id}-tfstate-primary\""
},
{
"evidence": "name = \"${var.project_id}-tfstate-secondary\"",
"check": "K-77",
"verdict": "pass"
},
{
"evidence": "enabled = true",
"check": "K-78",
"verdict": "pass"
},
{
"check": "K-79",
"evidence": "enabled = true",
"verdict": "pass"
},
{
"evidence": "goal = 0.999",
"verdict": "pass",
"check": "K-80"
},
{
"evidence": "filter = \"select_slo_burn_rate(\\\"${google_monitoring_slo.platform_slo.name}\\\", 3600s)\"",
"verdict": "pass",
"check": "K-81"
},
{
"verdict": "pass",
"check": "K-82",
"evidence": "service = \"recommender.googleapis.com\""
},
{
"verdict": "pass",
"evidence": "age = 30",
"check": "K-83"
},
{
"verdict": "pass",
"check": "K-84",
"evidence": "type = \"SetStorageClass\", storage_class = \"NEARLINE\""
},
{
"check": "K-85",
"verdict": "pass",
"evidence": "age = 90"
},
{
"check": "K-86",
"evidence": "type = \"SetStorageClass\", storage_class = \"ARCHIVE\"",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-87",
"evidence": "routing_mode = \"REGIONAL\""
},
{
"evidence": "min_replicas = 1",
"check": "K-88",
"verdict": "pass"
},
{
"evidence": "short_name = \"environment\"",
"check": "K-89",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-90",
"evidence": "destination = \"bigquery.googleapis.com/...\""
},
{
"verdict": "pass",
"check": "K-91",
"evidence": "tag_value = google_tags_tag_value.pci_regulated.id"
},
{
"check": "K-92",
"evidence": "check_interval_sec = 5",
"verdict": "pass"
},
{
"verdict": "pass",
"check": "K-93",
"evidence": "disabled = false"
},
{
"verdict": "pass",
"check": "K-94",
"evidence": "disabled = false"
},
{
"evidence": "display_name = \"Platform Custom Service\"",
"check": "K-95",
"verdict": "pass"
},
{
"check": "K-96",
"verdict": "pass",
"evidence": "base_instance_name = \"palo-alto-secondary\""
},
{
"verdict": "pass",
"evidence": "session_initialization_mode = \"ACTIVE\"",
"check": "K-97"
},
{
"evidence": "min_receive_interval = 100",
"check": "K-99",
"verdict": "pass"
},
{
"check": "K-100",
"verdict": "pass",
"evidence": "min_receive_interval = 100"
},
{
"check": "K-101",
"evidence": "role = \"roles/cloudkms.cryptoKeyDecrypter\"",
"verdict": "pass"
},
{
"evidence": "tag_value = google_tags_tag_value.corp_workload.id",
"verdict": "pass",
"check": "K-102"
},
{
"verdict": "pass",
"evidence": "router = google_compute_router.primary.id",
"check": "K-103"
},
{
"verdict": "pass",
"evidence": "min_transmit_interval = 100",
"check": "K-104"
},
{
"check": "K-105",
"evidence": "min_transmit_interval = 100",
"verdict": "pass"
}
],
"proposed_additions": []
}

View File

@@ -0,0 +1,332 @@
{
"work_packages": [
{
"id": "WP-0",
"name": "Platform Foundation",
"parent": null,
"elements": [],
"scope": "The complete enterprise-grade landing zone.",
"produces": [
"an operational landing zone"
],
"skills": [
"cloud architecture"
]
},
{
"id": "WP-1",
"name": "Automated Infrastructure Orchestration",
"parent": "WP-0",
"elements": [
"E-42",
"E-43"
],
"scope": "CI/CD orchestration, state management, and API enablement.",
"produces": [
"Terraform state buckets for primary and secondary regions"
],
"skills": [
"cloud engineering"
]
},
{
"id": "WP-2",
"name": "Federated Identity and Role Governance",
"parent": "WP-0",
"elements": [
"E-23",
"E-24",
"E-25",
"E-26",
"E-27",
"E-45"
],
"scope": "Authentication for delivery pipelines and workforce SSO.",
"produces": [
"Workload Identity Federation for GitHub Actions and Entra ID SSO"
],
"skills": [
"identity and access management"
]
},
{
"id": "WP-3",
"name": "Resource Hierarchy and Environment Boundary",
"parent": "WP-0",
"elements": [
"E-1",
"E-2",
"E-3",
"E-4",
"E-5",
"E-6",
"E-7",
"E-8",
"E-9",
"E-10",
"E-34",
"E-35",
"E-36",
"E-37",
"E-38",
"E-50",
"E-51",
"E-55",
"E-60"
],
"scope": "Google Cloud resource hierarchy, shared VPC projects, networking, and organization policies.",
"produces": [
"Folders, projects, networking, and organization policies ensuring isolation"
],
"skills": [
"cloud engineering",
"network engineering"
]
},
{
"id": "WP-4",
"name": "Cryptographic Key Lifecycle",
"parent": "WP-0",
"elements": [
"E-28",
"E-29",
"E-30",
"E-31",
"E-49",
"E-58",
"E-59"
],
"scope": "Customer-managed encryption keys (CMEK) and cryptographic isolation.",
"produces": [
"Primary and secondary KMS key rings and crypto keys with IAM bindings"
],
"skills": [
"security engineering"
]
},
{
"id": "WP-5",
"name": "Audit, Compliance, and Financial Governance",
"parent": "WP-0",
"elements": [
"E-21",
"E-22",
"E-33",
"E-39",
"E-46",
"E-54"
],
"scope": "Immutable compliance logs, DLP templates, and budget enforcement.",
"produces": [
"Immutable audit log storage, billing budgets, and DLP inspection templates"
],
"skills": [
"cloud engineering",
"finops"
]
},
{
"id": "WP-6",
"name": "Centralised Outbound Traffic Inspection",
"parent": "WP-0",
"elements": [
"E-15",
"E-16",
"E-17",
"E-18",
"E-19",
"E-20",
"E-47",
"E-48",
"E-53"
],
"scope": "Egress inspection and threat prevention using Palo Alto VM-Series.",
"produces": [
"Palo Alto VM-Series firewalls with internal load balancing and egress routing"
],
"skills": [
"network security engineering"
]
},
{
"id": "WP-7",
"name": "Dedicated Private Hybrid Connectivity",
"parent": "WP-0",
"elements": [
"E-11",
"E-12",
"E-13",
"E-14",
"E-56",
"E-57"
],
"scope": "Dedicated Interconnect termination and dynamic BGP routing.",
"produces": [
"Primary and secondary Cloud Routers with Dedicated Interconnect VLAN attachments"
],
"skills": [
"network engineering"
]
},
{
"id": "WP-8",
"name": "Platform Telemetry and Operational Alerting",
"parent": "WP-0",
"elements": [
"E-32",
"E-40",
"E-41",
"E-44",
"E-52"
],
"scope": "SLO tracking, multi-burn-rate alerting, and SCC automated remediation.",
"produces": [
"SLOs, alerting policies, and SCC automated remediation functions"
],
"skills": [
"site reliability engineering"
]
}
],
"activities": [
{
"id": "A-1",
"work_package": "WP-1",
"name": "Enable Google Cloud APIs",
"days": 0.5,
"after": []
},
{
"id": "A-2",
"work_package": "WP-1",
"name": "Setup CI/CD Pipeline and State Buckets",
"days": 1,
"after": [
"A-1"
]
},
{
"id": "A-3",
"work_package": "WP-2",
"name": "Configure Workload Identity and Entra ID SSO",
"days": 1.5,
"after": [
"A-2"
]
},
{
"id": "A-4",
"work_package": "WP-3",
"name": "Provision Resource Hierarchy and Enforce Org Policies",
"days": 2.5,
"after": [
"A-3"
]
},
{
"id": "A-5",
"work_package": "WP-3",
"name": "Provision Hub and Spoke VPC Networks",
"days": 2,
"after": [
"A-4"
]
},
{
"id": "A-6",
"work_package": "WP-7",
"name": "Order and provision the Interconnect carrier circuits",
"days": 30,
"owner": "customer",
"assumption": "The customer manages the carrier's lead time and physical cross-connects.",
"after": []
},
{
"id": "A-7",
"work_package": "WP-4",
"name": "Configure Cloud KMS and Key Rings",
"days": 1.5,
"after": [
"A-4"
]
},
{
"id": "A-8",
"work_package": "WP-5",
"name": "Set up Audit Logging, DLP, and Billing Budgets",
"days": 2,
"after": [
"A-4"
]
},
{
"id": "A-9",
"work_package": "WP-6",
"name": "Deploy Palo Alto Egress Firewall Appliances",
"days": 5,
"after": [
"A-5"
]
},
{
"id": "A-10",
"work_package": "WP-7",
"name": "Configure Cloud Routers and Terminate BGP over Interconnect",
"days": 3,
"after": [
"A-5",
"A-6"
]
},
{
"id": "A-11",
"work_package": "WP-3",
"name": "Implement VPC Service Controls",
"days": 2,
"after": [
"A-5",
"A-7",
"A-8"
]
},
{
"id": "A-12",
"work_package": "WP-8",
"name": "Deploy Platform Telemetry, SLOs, and SCC Auto-remediation",
"days": 2.5,
"after": [
"A-8",
"A-11"
]
},
{
"id": "A-13",
"work_package": "WP-1",
"name": "Run validation in development environment",
"days": 1,
"after": [
"A-9",
"A-10",
"A-12"
]
},
{
"id": "A-14",
"work_package": "WP-1",
"name": "Run validation in staging environment",
"days": 1,
"after": [
"A-13"
]
},
{
"id": "A-15",
"work_package": "WP-1",
"name": "Run validation in production environment",
"days": 1,
"after": [
"A-14"
]
}
]
}

294
tests/support.py Normal file
View File

@@ -0,0 +1,294 @@
"""One small, coherent operating model, for tests: a fictional landing zone run as a
shared service. Every name in it is invented."""
from __future__ import annotations
import copy
import json
from typing import Any
from execution_engine.providers import load_provider
from execution_engine.scope import read_scope
from app.operating_model import catalogue as capability_catalogue
from app.operating_model import staffing
from app.operating_model.names import (
CAPABILITIES_FILE,
CAPABILITY_MODEL,
FINDINGS_FILE,
GRAPH_FILE,
OPERATING_SCOPE,
OPERATIONAL_REQUIREMENTS,
REQUIREMENTS_FILE,
SCOPE_DATA_FILE,
TARGET_OPERATING_MODEL,
VERDICTS_FILE,
)
from app.operating_model.workflow import ROOT, build
BUNDLE = load_provider("gcp", ROOT)
CATALOGUE = capability_catalogue.load(ROOT / "skills" / capability_catalogue.SKILL)
CARD = staffing.load(ROOT / "skills" / staffing.SKILL)
WORKFLOW = build(ROOT)
SPEC = """\
metadata:
customer_case: northwind-landing-zone
engagement:
kind: platform_foundation
intent:
statement: |
We want the landing zone run as a shared service from go-live, with our own
platform team keeping the final say.
business_outcomes:
- {id: BO-1, outcome: Outages of the platform are short., target: Restored within 4 hours.}
- {id: BO-2, outcome: Regulators always get the evidence they ask for.}
requirements:
- {id: REQ-01, statement: Our platform team approves every change to the network.}
solution:
cloud_provider: gcp
items:
- {id: S-1, name: Hub network, kind: network, product: Shared VPC, source_ref: E-1}
- {id: S-2, name: Audit log store, kind: storage, product: Cloud Storage, source_ref: E-2}
- {id: S-3, name: Deployment pipeline, kind: pipeline, product: Cloud Build, source_ref: E-3}
conditions:
- {id: K-1, "on": S-1, condition: Flow logs are retained for 400 days., source_ref: CHK-12}
- {id: K-2, "on": S-2, condition: Audit logs cannot be deleted for seven years., source_ref: CHK-20}
operations:
support_hours: 24x7
availability_hours: 24x7
change_process: decide
sourcing:
model: shared
delivery_countries: [GB, PL, IN]
service_levels:
priority_1_response_minutes: 15
recovery_time_hours: 4
compliance:
frameworks: [DORA]
commercial:
currency: GBP
monthly_run_budget: 120000
run:
pre_approved: true
"""
def scope_data(text: str = SPEC) -> dict[str, Any]:
"""What every Specialist is given of an operating spec."""
scope = read_scope(
text, BUNDLE.scope_catalogue(), provider_values=BUNDLE.scope_values
)
return scope.for_context()
REQUIREMENTS: list[dict[str, Any]] = [
{
"id": "OR-1",
"kind": "service_level",
"statement": "A fault on any Item is acknowledged within 15 minutes, at any hour.",
"source": [
"service_levels.priority_1_response_minutes",
"operations.support_hours",
],
},
{
"id": "OR-2",
"kind": "process",
"statement": "The customer's platform team approves every change to the network.",
"source": ["REQ-01", "sourcing.model"],
},
{
"id": "OR-3",
"kind": "compliance",
"statement": "Logs are kept for as long as each Condition says, and cannot be removed sooner.",
"source": ["K-1", "K-2"],
},
{
"id": "OR-4",
"kind": "service_level",
"statement": "Service is restored within 4 hours of a major failure.",
"source": ["service_levels.recovery_time_hours"],
},
{
"id": "OR-5",
"kind": "organisation",
"statement": "The solution is operated only from the countries the customer allows.",
"source": ["sourcing.delivery_countries"],
},
]
CAPABILITIES: list[dict[str, Any]] = [
{
"id": "C-1",
"catalogue": "incident-management",
"serves": ["OR-1", "OR-4"],
"items": ["S-1", "S-2", "S-3"],
"hours": "24x7",
"effort_hours_per_month": 60,
"basis": "Two serious incidents a month and daily triage.",
},
{
"id": "C-2",
"catalogue": "change-enablement",
"serves": ["OR-2"],
"items": ["S-1", "S-2", "S-3"],
"hours": "business",
"effort_hours_per_month": 40,
"basis": "About twenty changes a month at two hours each.",
},
{
"id": "C-3",
"catalogue": "regulatory-evidence",
"serves": ["OR-3", "OR-5"],
"items": ["S-1", "S-2"],
"hours": "business",
"effort_hours_per_month": 16,
"basis": "A monthly review of retention settings and evidence.",
},
]
GRAPH: dict[str, list[dict[str, Any]]] = {
"parties": [
{"id": "P-1", "name": "Northwind platform team", "kind": "customer"},
{"id": "P-2", "name": "Managed service provider", "kind": "provider"},
],
"roles": [
{
"id": "R-1",
"party": "P-1",
"name": "Platform owner",
"country": "GB",
"level": "lead",
"hours": "business",
},
{
"id": "R-2",
"party": "P-2",
"name": "Platform SRE",
"country": "PL",
"level": "senior",
"hours": "24x7",
},
{
"id": "R-3",
"party": "P-2",
"name": "Service manager",
"country": "GB",
"level": "lead",
"hours": "business",
},
],
"responsibilities": [
{"id": "RS-1", "capability": "C-1", "role": "R-3", "kind": "accountable"},
{"id": "RS-2", "capability": "C-1", "role": "R-2", "kind": "responsible"},
{"id": "RS-3", "capability": "C-1", "role": "R-1", "kind": "consulted"},
{"id": "RS-4", "capability": "C-2", "role": "R-1", "kind": "accountable"},
{"id": "RS-5", "capability": "C-2", "role": "R-2", "kind": "responsible"},
{"id": "RS-6", "capability": "C-3", "role": "R-3", "kind": "accountable"},
{"id": "RS-7", "capability": "C-3", "role": "R-3", "kind": "responsible"},
{"id": "RS-8", "capability": "C-3", "role": "R-1", "kind": "consulted"},
],
"measures": [
{
"id": "M-1",
"name": "Time to restore service",
"layer": "outcome",
"outcome": "BO-1",
"measured_by": "C-1",
"framework": "dora-metrics",
"target": "Under 4 hours",
},
{
"id": "M-2",
"name": "Evidence requests met on time",
"layer": "service_level",
"outcome": "BO-2",
"measured_by": "C-3",
},
],
"checks": [
{
"id": "OC-1",
"on": "C-1",
"source": "OR-1",
"condition": "A fault on the hub network is acknowledged within 15 minutes at any hour.",
},
{
"id": "OC-2",
"on": "C-3",
"source": "K-1",
"condition": "Flow log retention is reviewed monthly and is 400 days.",
},
{
"id": "OC-3",
"on": "C-3",
"source": "K-2",
"condition": "The audit log store's retention lock is reviewed monthly.",
},
{
"id": "OC-4",
"on": "RS-4",
"source": "OR-2",
"condition": "No change to the network is applied without the platform owner's approval.",
},
],
}
VERDICTS: dict[str, Any] = {
"checks": [
{
"check": check["id"],
"verdict": "pass",
"evidence": "The model provides for it.",
}
for check in GRAPH["checks"]
],
"proposed_additions": [],
}
def graph(**changes: Any) -> dict[str, list[dict[str, Any]]]:
"""A copy of the graph to change."""
copied = copy.deepcopy(GRAPH)
copied.update(changes)
return copied
def upstream(
*,
spec: str = SPEC,
requirements: list[dict[str, Any]] | None = None,
capabilities: list[dict[str, Any]] | None = None,
model: dict[str, Any] | None = None,
findings: list[dict[str, Any]] | None = None,
) -> dict[str, dict[str, Any]]:
"""The Deliverables a submission builds on, as the engine holds them."""
def deliverable(name: str, data: Any) -> dict[str, Any]:
return {"content": "# Prose\n", "attachments": {name: json.dumps(data)}}
built = {
OPERATING_SCOPE: deliverable(SCOPE_DATA_FILE, scope_data(spec)),
OPERATIONAL_REQUIREMENTS: deliverable(
REQUIREMENTS_FILE, REQUIREMENTS if requirements is None else requirements
),
CAPABILITY_MODEL: deliverable(
CAPABILITIES_FILE, CAPABILITIES if capabilities is None else capabilities
),
TARGET_OPERATING_MODEL: deliverable(
GRAPH_FILE, GRAPH if model is None else model
),
}
if findings is not None:
built[TARGET_OPERATING_MODEL]["attachments"][FINDINGS_FILE] = json.dumps(
findings
)
return built
def as_file(name: str, data: Any) -> dict[str, str]:
return {name: json.dumps(data)}
__all__ = ["VERDICTS_FILE"]

107
tests/test_catalogue.py Normal file
View File

@@ -0,0 +1,107 @@
"""The capability catalogue and its register of frameworks."""
import shutil
import pytest
import yaml
from app.operating_model import catalogue as capability_catalogue
from app.operating_model.catalogue import CatalogueError
from app.operating_model.workflow import ROOT
from tests.support import CATALOGUE, scope_data
SKILL = ROOT / "skills" / capability_catalogue.SKILL
def test_every_entry_cites_a_framework_that_is_in_the_register():
for entry in CATALOGUE.entries.values():
assert entry.sources, entry.id
assert all(s["framework"] in CATALOGUE.frameworks for s in entry.sources)
assert entry.purpose and entry.typical_checks, entry.id
def test_the_register_says_its_references_have_not_been_checked():
assert CATALOGUE.verified is False
def test_the_two_things_called_dora_are_kept_apart():
names = {key: value["name"] for key, value in CATALOGUE.frameworks.items()}
assert "Regulation (EU) 2022/2554" in names["eu-dora"]
assert "DevOps Research and Assessment" in names["dora-metrics"]
@pytest.fixture
def skill(tmp_path):
shutil.copytree(SKILL, tmp_path / "skill")
return tmp_path / "skill"
def edit(skill, name, change):
path = skill / "assets" / name
data = yaml.safe_load(path.read_text())
change(data)
path.write_text(yaml.safe_dump(data))
def test_an_entry_that_cites_nothing_is_refused(skill):
edit(skill, "catalogue.yaml", lambda d: d["capabilities"][0].update(sources=[]))
with pytest.raises(CatalogueError, match="incident-management cites no source"):
capability_catalogue.load(skill)
def test_an_entry_that_cites_an_unregistered_framework_is_refused(skill):
edit(
skill,
"catalogue.yaml",
lambda d: d["capabilities"][0]["sources"].append(
{"framework": "folklore", "reference": "What we always did"}
),
)
with pytest.raises(
CatalogueError, match="cites 'folklore', which is not in the register"
):
capability_catalogue.load(skill)
def test_a_framework_registered_without_its_licence_is_refused(skill):
edit(skill, "sources.yaml", lambda d: d["frameworks"]["itil-4"].pop("licence"))
with pytest.raises(CatalogueError, match="itil-4 needs licence"):
capability_catalogue.load(skill)
def test_entries_are_proposed_from_what_the_spec_says():
proposed = CATALOGUE.applicable(scope_data())
# Always; for an Item of a kind; for a framework named; for a sourcing model.
assert "incident-management" in proposed
assert "backup-and-recovery" in proposed # the audit log store is storage
assert "regulatory-evidence" in proposed # DORA
assert "supplier-management" in proposed # shared
def test_an_entry_is_not_proposed_when_nothing_calls_for_it():
bare = {
"stated": {
"solution": {"items": [{"id": "S-1", "name": "Hub", "kind": "network"}]}
}
}
proposed = CATALOGUE.applicable(bare)
assert "incident-management" in proposed
assert not {
"backup-and-recovery",
"regulatory-evidence",
"supplier-management",
} & set(proposed)
def test_sources_are_cited_by_framework_and_part_never_quoted():
lines = capability_catalogue.cited(CATALOGUE, ["incident-management"])
assert "ITIL 4: Incident management practice" in lines
assert any(line.startswith("Regulation (EU) 2022/2554") for line in lines)

142
tests/test_chain.py Normal file
View File

@@ -0,0 +1,142 @@
"""The chain's data files: what links each to the one before, and what is refused."""
import copy
import pytest
from app.operating_model import chain
from app.operating_model.names import CAPABILITIES_FILE, REQUIREMENTS_FILE
from tests.support import CAPABILITIES, CATALOGUE, REQUIREMENTS, as_file, upstream
check_capabilities = chain.check_capabilities(CATALOGUE)
def requirements(*changes) -> list[dict]:
entries = copy.deepcopy(REQUIREMENTS)
for change in changes:
change(entries)
return entries
def capabilities(*changes) -> list[dict]:
entries = copy.deepcopy(CAPABILITIES)
for change in changes:
change(entries)
return entries
def test_requirements_traced_to_what_the_customer_stated_are_accepted():
assert (
chain.check_requirements(as_file(REQUIREMENTS_FILE, REQUIREMENTS), upstream())
== []
)
@pytest.mark.parametrize(
("change", "problem"),
[
(lambda e: e[0].update(source=[]), "OR-1 has no source"),
(
lambda e: e[0].update(source=["OR-99"]),
"source OR-99 is not a stated requirement",
),
(
lambda e: e[0].update(source=["commercial.exchange_rate"]),
"source commercial.exchange_rate is not a stated requirement",
),
(lambda e: e[0].update(kind="wish"), "OR-1: kind must be one of"),
(lambda e: e[0].update(statement=" "), "OR-1 needs a statement"),
(lambda e: e.append(dict(e[0])), "id OR-1 is used more than once"),
(
lambda e: e[1].update(source=["sourcing.model"]),
"the customer's requirement REQ-01 is carried by no Operational requirement",
),
(
lambda e: e.pop(4),
"the customer stated sourcing.delivery_countries, and no Operational requirement",
),
],
)
def test_a_requirement_that_breaks_the_chain_is_refused(change, problem):
faults = chain.check_requirements(
as_file(REQUIREMENTS_FILE, requirements(change)), upstream()
)
assert any(problem in fault for fault in faults), faults
def test_what_was_assumed_can_be_a_source_but_does_not_bind():
# `change_process: decide` leaves it to the agent, so nothing has to name it.
scope = upstream()["operating_scope"]["attachments"]["scope.json"]
assert "operations.change_process" not in scope.split('"binding"')[1].split("]")[0]
def test_an_empty_file_is_refused_and_bad_json_is_named():
assert chain.check_requirements({REQUIREMENTS_FILE: "[]"}, upstream()) == [
"operational-requirements.json: lists no Operational requirements"
]
assert (
"not valid JSON" in chain.check_requirements({REQUIREMENTS_FILE: "{"}, None)[0]
)
def test_capabilities_from_the_catalogue_that_cover_everything_are_accepted():
assert (
check_capabilities(as_file(CAPABILITIES_FILE, CAPABILITIES), upstream()) == []
)
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda e: e[0].update(catalogue="firefighting"),
"'firefighting' is not in the capability catalogue",
),
(
lambda e: e[0].update(catalogue="custom"),
"C-1 is custom and gives no reason",
),
(lambda e: e[0].update(serves=[]), "C-1 serves no Operational requirement"),
(
lambda e: e[0].update(serves=["OR-1", "OR-9"]),
"C-1 serves OR-9, which is not an Operational requirement",
),
(
lambda e: e[0].update(items=["S-1", "S-9"]),
"C-1 covers S-9, which is not an Item",
),
(
lambda e: [c.update(items=["S-1", "S-2"]) for c in e],
"Item S-3 is covered by no Capability",
),
(lambda e: e.pop(1), "Operational requirement OR-2 is served by no Capability"),
(lambda e: e[0].update(hours="always"), "C-1: hours must be one of"),
(
lambda e: e[0].update(hours="business"),
"faults are responded to 24x7, and Incident management is staffed only business",
),
(
lambda e: e[0].update(effort_hours_per_month=0),
"effort_hours_per_month must be a number above zero",
),
(lambda e: e[0].update(basis=""), "C-1 needs a basis for its effort"),
],
)
def test_a_capability_that_breaks_the_chain_is_refused(change, problem):
faults = check_capabilities(
as_file(CAPABILITIES_FILE, capabilities(change)), upstream()
)
assert any(problem in fault for fault in faults), faults
def test_a_custom_capability_with_a_name_and_a_reason_is_accepted():
custom = capabilities(
lambda e: e[2].update(
catalogue="custom", name="Evidence keeping", reason="No entry fits."
)
)
assert check_capabilities(as_file(CAPABILITIES_FILE, custom), upstream()) == []

237
tests/test_digest.py Normal file
View File

@@ -0,0 +1,237 @@
"""Digesting a solution architecture's files into an operating spec, in code."""
import json
from pathlib import Path
import pytest
from execution_engine.scope import ESSENTIAL, merged, read_scope
from app.operating_model import digest, spec
from app.operating_model.workflow import ROOT
from tests.support import BUNDLE, CATALOGUE, WORKFLOW
FIXTURE = Path(__file__).parent / "fixtures" / "architecture" / "case-08-landing-zone"
SOURCES = digest.read_folder(FIXTURE)
KINDS = digest.item_kinds(ROOT, "gcp")
DIGESTED = digest.architecture(SOURCES, KINDS)
GRAPH = json.loads(SOURCES[digest.GRAPH])
def test_every_file_the_digest_reads_came_with_the_fixture():
assert set(SOURCES) == set(digest.FILES)
assert set(DIGESTED.used) == set(digest.FILES)
def test_every_component_becomes_one_item_and_every_element_is_accounted_for():
items = DIGESTED.spec["solution"]["items"]
components = json.loads(SOURCES[digest.COMPONENTS])
assert [item["source_ref"] for item in items] == [c["id"] for c in components]
assert [item["id"] for item in items] == [f"S-{n}" for n in range(1, 10)]
placed = [element for item in items for element in item["elements"]]
assert sorted(placed) == sorted(e["id"] for e in GRAPH["elements"])
assert len(placed) == 60
def test_an_item_takes_its_product_and_its_kind_from_the_providers_table():
by_ref = {item["source_ref"]: item for item in DIGESTED.spec["solution"]["items"]}
assert (by_ref["C-2"]["product"], by_ref["C-2"]["kind"]) == (
"Cloud Interconnect",
"network",
)
assert by_ref["C-4"]["kind"] == "storage"
assert by_ref["C-7"]["kind"] == "keys"
# A product the table does not know is a service; nothing is guessed.
assert by_ref["C-3"]["kind"] == "service"
def test_what_the_architecture_leaves_to_operation_becomes_the_conditions():
conditions = DIGESTED.spec["solution"]["conditions"]
left_to_operation = [
(entry["id"], statement)
for name in (digest.REQUIREMENTS, digest.RECOMMENDATIONS)
for entry in json.loads(SOURCES[name])
for statement in entry.get("outside_code") or []
]
assert [(c["source_ref"], c["condition"]) for c in conditions] == [
(identifier, " ".join(statement.split()))
for identifier, statement in left_to_operation
]
recovery = next(
c for c in conditions if "recovery time objective" in c["condition"]
)
assert recovery["source_ref"] == "NFR-11"
items = {item["id"] for item in DIGESTED.spec["solution"]["items"]}
assert all(c["on"] and set(c["on"]) <= items for c in conditions)
def test_the_settings_checked_at_build_are_not_conditions():
# 104 Checks on infrastructure settings were verified in validation. Keeping
# them true is configuration management, not a Condition each.
assert len(GRAPH["checks"]) == 104
assert len(DIGESTED.spec["solution"]["conditions"]) == 12
def test_only_what_the_customer_stated_is_carried_across_as_stated():
carried = DIGESTED.spec
assert carried["operations"]["support_hours"] == "24x7" # stated: always
assert carried["compliance"]["frameworks"] == ["DORA", "ISO27001", "PCI_DSS_v4"]
assert carried["commercial"]["currency"] == "GBP"
assert carried["solution"]["cloud_provider"] == "gcp"
assert carried["engagement"]["kind"] == "platform_foundation"
# The architecture agent assumed these for the customer; they are asked again.
assert "change_process" not in carried["operations"]
assert "service_levels" not in carried
def test_the_build_and_what_the_customer_owns_of_it_are_carried_across():
build = DIGESTED.spec["solution"]["build"]
assert build["duration_days"] == 36
assert build["customer_owned"] == [
"Order and provision the Interconnect carrier circuits"
]
def test_the_spec_says_which_execution_it_was_digested_from():
assert DIGESTED.spec["metadata"] == {
"customer_case": "case-08-landing-zone",
"source": "architecture:case-08-landing-zone-185426",
}
assert "payment platform" in DIGESTED.spec["intent"]["statement"]
def test_the_digested_spec_is_one_the_catalogue_accepts():
scope = read_scope(
DIGESTED.spec, BUNDLE.scope_catalogue(), provider_values=BUNDLE.scope_values
)
# What only the customer can say is still essential: what the business wants
# from the solution, and who operates it.
assert [gap["field"] for gap in scope.gaps(ESSENTIAL)] == [
"intent.business_outcomes",
"sourcing.model",
]
def test_every_digested_value_is_marked_and_what_the_caller_adds_is_theirs():
scope = merged(
DIGESTED.spec,
"sourcing:\n model: shared\ncommercial:\n currency: EUR\n"
"intent:\n business_outcomes: decide\n",
BUNDLE.scope_catalogue(),
provider_values=BUNDLE.scope_values,
notes=DIGESTED.notes,
)
assert scope.source("solution.items") == "digested"
assert scope.notes["solution.items"].startswith("9 components")
assert scope.source("sourcing.model") == "supplied"
assert scope.values["commercial.currency"] == "EUR"
assert scope.source("commercial.currency") == "supplied"
assert scope.gaps(ESSENTIAL) == []
def test_the_capabilities_proposed_follow_from_what_was_digested():
scope = read_scope(
DIGESTED.spec, BUNDLE.scope_catalogue(), provider_values=BUNDLE.scope_values
).for_context()
proposed = CATALOGUE.applicable(scope)
assert "backup-and-recovery" in proposed # the audit log store is storage
assert "regulatory-evidence" in proposed # DORA, ISO 27001, PCI DSS
assert len(spec.items(scope)) == 9
def test_a_check_that_did_not_pass_is_an_inherited_risk():
sources = dict(SOURCES)
verdicts = json.loads(sources[digest.VERDICTS])
verdicts["checks"][0].update(verdict="fail", evidence="display_name is empty")
sources[digest.VERDICTS] = json.dumps(verdicts)
risks = digest.architecture(sources, KINDS).spec["solution"]["inherited_risks"]
assert risks == [
{
"on": "S-1",
"risk": "Check K-1 failed in validation: `display_name` is set "
"(display_name is empty)",
"source_ref": "K-1",
}
]
assert "inherited_risks" not in DIGESTED.spec["solution"] # every Check passed
def test_files_passed_without_their_manifest_are_marked_unverified():
sources = {k: v for k, v in SOURCES.items() if k != digest.MANIFEST}
digested = digest.architecture(sources, KINDS)
assert digested.spec["metadata"] == {"source": "architecture:unverified"}
assert (
"not known to be one completed Execution" in digested.notes["metadata.source"]
)
assert len(digested.spec["solution"]["items"]) == 9
def test_the_graph_alone_is_enough_to_say_what_is_operated():
digested = digest.architecture({digest.GRAPH: SOURCES[digest.GRAPH]}, KINDS)
items = digested.spec["solution"]["items"]
# With no components file, they are taken from the Elements, in the order met.
assert sorted(item["source_ref"] for item in items) == [
f"C-{n}" for n in range(1, 10)
]
assert "conditions" not in digested.spec["solution"]
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda s: s.update({digest.GRAPH: "{"}),
"target-architecture.json: not valid JSON",
),
(
lambda s: s.update({digest.GRAPH: "[]"}),
"must be an object with a list of elements",
),
(
lambda s: s.update(
{digest.MANIFEST: json.dumps({"status": "completed_with_warnings"})}
),
"the Execution did not complete",
),
(
lambda s: s.update(
{digest.COMPONENTS: json.dumps([{"id": "C-1", "name": "Only one"}])}
),
"these Elements name a component that components.json does not have",
),
],
)
def test_an_architecture_that_cannot_be_used_is_refused_and_says_why(change, problem):
sources = dict(SOURCES)
change(sources)
with pytest.raises(ValueError, match=problem):
digest.architecture(sources, KINDS)
def test_sources_that_are_not_an_architecture_are_left_to_be_read():
assert digest.architecture({"design.md": "# Our platform\n"}, KINDS).spec == {}
assert WORKFLOW.digest is not None
assert WORKFLOW.digest({"design.md": "# Our platform\n"}).used == ()
def test_the_command_prints_the_spec_and_what_it_was_read_from(capsys):
assert digest.main([str(FIXTURE)]) == 0
printed = capsys.readouterr()
assert "source: architecture:case-08-landing-zone-185426" in printed.out
assert "# solution.items: 9 components" in printed.err
assert digest.main([str(FIXTURE.parent)]) == 1

69
tests/test_eval.py Normal file
View File

@@ -0,0 +1,69 @@
"""The evaluation harness itself, run on the scripted runtime: no model is called."""
import json
from execution_engine.adk.fake_runtime import FakeAgentRuntime
from execution_engine.config import Settings
from eval import operating_model_eval as harness
from tests.support import SPEC
from tests.test_execution import turns
def settings() -> Settings:
return Settings( # type: ignore[call-arg]
_env_file=None, CLOUD_PROVIDER="gcp", CHECKPOINTER="memory", MCP_ENABLED=False
)
CASE = {"id": "scripted", "max_revisions": 2, "max_minutes": 5}
async def run(tmp_path, scripted, monkeypatch):
monkeypatch.setattr(harness, "inputs", lambda case: {"scope": SPEC})
return await harness.run_case(
CASE, tmp_path, settings=settings(), runtime=FakeAgentRuntime(scripted)
)
async def test_a_sound_execution_passes_every_check(tmp_path, monkeypatch):
result = await run(tmp_path, turns(), monkeypatch)
assert result.status == "completed"
failed = [check for check in result.checks if not check.passed]
# The scripted packager writes one word; a real one must write a summary.
assert [check.name for check in failed] == ["the packager wrote a summary"]
assert result.figures["cost_per_month"] == 43410
assert result.figures["scenarios"] == 10
assert (tmp_path / "deliverables").is_dir()
assert json.dumps(result.as_dict()) # the whole result can be written down
async def test_a_run_that_cannot_finish_is_reported_not_raised(tmp_path, monkeypatch):
scripted = turns()
scripted["validator"] = []
result = await run(tmp_path, scripted, monkeypatch)
assert not result.passed
assert result.status != "completed"
def test_every_case_names_something_to_start_from():
for case in harness.load_cases():
given = harness.inputs(case)
assert "scope" in given or "sources" in given, case["id"]
def test_the_benchmark_spec_is_complete_enough_to_run_with_nobody_to_ask():
from execution_engine.scope import ESSENTIAL, read_scope
from tests.support import BUNDLE
text = (harness.CASES.parent / "specs" / "landing-zone-shared.yaml").read_text()
scope = read_scope(
text, BUNDLE.scope_catalogue(), provider_values=BUNDLE.scope_values
)
assert scope.gaps(ESSENTIAL) == []
assert scope.customer_case == "bench-landing-zone-shared"

411
tests/test_execution.py Normal file
View File

@@ -0,0 +1,411 @@
"""Whole Executions of the target operating model Workflow, on the scripted runtime."""
import json
from pathlib import Path
from typing import Any
from execution_engine.adk.fake_runtime import Call, FakeAgentRuntime
from execution_engine.adk.roles import load_roles
from execution_engine.adk.runtime import NoDecider
from execution_engine.agent import ExecutionEngine
from execution_engine.nodes.router import Router, Thresholds
from execution_engine.recorder import InMemoryRecorder
from langgraph.checkpoint.memory import InMemorySaver
from app.operating_model.names import (
CAPABILITIES_FILE,
CAPABILITY_MODEL,
FINDINGS_FILE,
GRAPH_FILE,
GUIDE,
OPERATING_SCOPE,
OPERATIONAL_REQUIREMENTS,
REQUIREMENTS_FILE,
TARGET_OPERATING_MODEL,
VALIDATION_RESULTS,
VERDICTS_FILE,
)
from app.operating_model.workflow import ROOT
from tests.support import (
BUNDLE,
CAPABILITIES,
GRAPH,
REQUIREMENTS,
SPEC,
VERDICTS,
WORKFLOW,
graph,
)
ROLES = load_roles(ROOT / "agents", BUNDLE, WORKFLOW)
CASE = "northwind-landing-zone"
FINDING = {
"id": "AF-1",
"about": ["S-2"],
"needed_by": ["C-3"],
"lacks": "Nothing alerts when the audit log store's retention lock is removed.",
"proposed": "Add an alert on a change to the retention lock.",
}
def submit(kind: str, **files: Any) -> list[Call]:
"""One turn that submits a Deliverable with every section of its contract."""
spec = WORKFLOW.deliverables[kind]
content = f"# {kind}\n" + "".join(
f"\n## {section}\n\nDetails.\n" for section in spec.sections
)
return [
Call(
"submit_deliverable",
{
"content": content,
"attachments": {name: json.dumps(data) for name, data in files.items()},
},
)
]
def turns(model=GRAPH, verdicts=VERDICTS, findings=None) -> dict[str, list[Any]]:
model_files = {GRAPH_FILE: model}
if findings is not None:
model_files[FINDINGS_FILE] = findings
return {
"scope-analyst": [submit(OPERATING_SCOPE)],
"operations-analyst": [
submit(OPERATIONAL_REQUIREMENTS, **{REQUIREMENTS_FILE: REQUIREMENTS})
],
"capability-modeller": [
submit(CAPABILITY_MODEL, **{CAPABILITIES_FILE: CAPABILITIES})
],
"operating-model-architect": [submit(TARGET_OPERATING_MODEL, **model_files)],
"validator": [submit(VALIDATION_RESULTS, **{VERDICTS_FILE: verdicts})],
"packager": [submit(GUIDE)],
}
def make_engine(
tmp_path: Path, scripted: dict[str, list[Any]]
) -> tuple[ExecutionEngine, FakeAgentRuntime]:
runtime = FakeAgentRuntime(scripted)
engine = ExecutionEngine(
workflow=WORKFLOW,
runtime=runtime,
roles=ROLES,
recorder=InMemoryRecorder(),
checkpointer=InMemorySaver(),
output_root=tmp_path,
orchestrator=Router(NoDecider(), None, Thresholds()),
scope_catalogue=BUNDLE.scope_catalogue(),
scope_values=BUNDLE.scope_values,
scope_skill=BUNDLE.scope_skill,
)
return engine, runtime
def folder(tmp_path: Path, execution_id: str) -> Path:
return tmp_path / "deliverables" / "cases" / CASE / "executions" / execution_id
async def test_an_operating_spec_becomes_a_priced_validated_operating_model(tmp_path):
engine, runtime = make_engine(tmp_path, turns())
result = await engine.start("", scope=SPEC, execution_id="e1")
assert result.status == "completed", result.state.get("warnings")
assert [run.role for run in runtime.runs] == [
"scope-analyst",
"operations-analyst",
"capability-modeller",
"operating-model-architect",
"validator",
"packager",
]
out = folder(tmp_path, "e1")
written = {p.relative_to(out).as_posix() for p in out.rglob("*") if p.is_file()}
assert {
"execution.json",
"operating-model-guide.md",
"scope.yaml",
"scope.json",
"operational-requirements.json",
"capabilities.json",
"target-operating-model.json",
"findings.json",
"staffing.json",
"organisation.mmd",
"verdicts.json",
"scenarios.json",
} <= written
record = json.loads((out / "execution.json").read_text())
assert record["customer_case"] == CASE
assert record["final_document"] == "operating-model-guide.md"
assert record["staffing"] == {
"people": 5.1,
"cost_per_month": 43410,
"currency": "GBP",
"rate_card": {
"name": "rate-card",
"version": "draft-2026-10-13",
"draft": True,
},
}
assert record["findings"] == []
assert record["capability_sources_verified"] is False
async def test_the_views_and_the_walk_throughs_are_written_by_code(tmp_path):
engine, _ = make_engine(tmp_path, turns())
result = await engine.start("", scope=SPEC)
model = result.state["deliverables"][TARGET_OPERATING_MODEL]["content"]
assert "| Incident management (C-1) | C | R | A |" in model
assert "Cost a month: 43,410 GBP." in model
validation = result.state["deliverables"][VALIDATION_RESULTS]
assert "## Scenario walk-throughs" in validation["content"]
verdicts = json.loads(validation["attachments"][VERDICTS_FILE])
assert verdicts["dry_run"]["evidence"] == "10 of 10 scenario walk-throughs pass."
assert len(json.loads(validation["attachments"]["scenarios.json"])) == 10
async def test_specialists_are_told_what_code_found_and_which_prices_apply(tmp_path):
engine, runtime = make_engine(tmp_path, turns())
await engine.start("", scope=SPEC)
by_role = {run.role: run.task.context for run in runtime.runs}
proposed = by_role["capability-modeller"]["capabilities_proposed"]
assert {"incident-management", "regulatory-evidence", "supplier-management"} <= set(
proposed
)
assert by_role["operating-model-architect"]["rate_card"]["draft"] is True
assert "PL" in by_role["operating-model-architect"]["rate_card"]["countries"]
# The architect builds the graph with the engine's tools, in this Workflow's shape.
architect = next(
run for run in runtime.runs if run.role == "operating-model-architect"
)
assert {"graph_put", "graph_read", "graph_remove"} <= set(architect.tools)
# Every Specialist is given the agreed operating spec, with the solution in it.
items = by_role["validator"]["scope"]["stated"]["solution"]["items"]
assert [item["id"] for item in items] == ["S-1", "S-2", "S-3"]
async def test_a_spec_with_a_currency_the_rate_card_cannot_convert_to_is_refused(
tmp_path,
):
import pytest
engine, runtime = make_engine(tmp_path, turns())
with pytest.raises(ValueError, match="commercial.currency"):
await engine.start("", scope=SPEC.replace("currency: GBP", "currency: CHF"))
assert runtime.runs == []
async def test_a_spec_with_no_currency_and_nobody_to_ask_is_refused(tmp_path):
import pytest
engine, _ = make_engine(tmp_path, turns())
with pytest.raises(ValueError, match="commercial.currency"):
await engine.start("", scope=SPEC.replace(" currency: GBP\n", ""))
async def test_a_model_nobody_can_run_at_night_is_refused_before_it_is_approved(
tmp_path,
):
off_hours = graph()
off_hours["roles"][1]["hours"] = "extended"
scripted = turns()
# Refused, and put right in the same run: the customer never sees it.
scripted["operating-model-architect"] = [
[
*submit(TARGET_OPERATING_MODEL, **{GRAPH_FILE: off_hours}),
*submit(TARGET_OPERATING_MODEL, **{GRAPH_FILE: GRAPH}),
]
]
engine, runtime = make_engine(tmp_path, scripted)
result = await engine.start("", scope=SPEC)
assert result.status == "completed", result.state.get("warnings")
refused, accepted = runtime.runs[3].results
assert (
"Capability C-1 is needed 24x7, and no Role responsible for it works "
"those hours" in refused[1]
)
assert accepted[1] == "Submitted target_operating_model."
assert (
result.state["ledger"]["deliverables"][TARGET_OPERATING_MODEL]["revisions"] == 0
)
async def test_a_failed_scenario_reopens_the_model_through_the_router(tmp_path):
stranded = graph()
stranded["responsibilities"] = [
r for r in stranded["responsibilities"] if r["id"] not in ("RS-3", "RS-8")
]
scripted = turns()
scripted["operating-model-architect"] = [
submit(TARGET_OPERATING_MODEL, **{GRAPH_FILE: stranded}),
lambda task: (
submit(TARGET_OPERATING_MODEL, **{GRAPH_FILE: GRAPH})
if any("SC-party-P-2 failed" in item for item in task.context["feedback"])
else []
),
]
scripted["validator"] = scripted["validator"] * 2
engine, runtime = make_engine(tmp_path, scripted)
result = await engine.start("", scope=SPEC)
assert result.status == "completed", result.state.get("warnings")
assert [d["action"] for d in result.state["decisions"]].count("revise_code") == 1
ledger = result.state["ledger"]["deliverables"]
assert ledger[TARGET_OPERATING_MODEL]["revisions"] == 1
assert ledger[CAPABILITY_MODEL]["revisions"] == 0
assert [run.role for run in runtime.runs].count("validator") == 2
async def test_a_finding_is_kept_out_of_the_design_and_drafted_as_a_change_request(
tmp_path,
):
waiting = graph()
waiting["checks"][2]["depends_on_finding"] = "AF-1"
verdicts = json.loads(json.dumps(VERDICTS))
verdicts["checks"][2].update(verdict="not_verifiable", evidence="Waits on AF-1.")
digested = SPEC.replace(
" customer_case: northwind-landing-zone\n",
" customer_case: northwind-landing-zone\n source: architecture:exec-42\n",
)
engine, _ = make_engine(tmp_path, turns(waiting, verdicts, [FINDING]))
result = await engine.start("", scope=digested, execution_id="e1")
assert result.status == "completed", result.state.get("warnings")
out = folder(tmp_path, "e1")
record = json.loads((out / "execution.json").read_text())
assert record["findings"] == ["AF-1"]
assert record["change_request"] == "architecture-change-request.md"
request = (out / "architecture-change-request.md").read_text()
assert "digested from `architecture:exec-42`" in request
assert (
"**What is asked for.** Add an alert on a change to the retention lock."
in request
)
assert (
"## Findings" in result.state["deliverables"][TARGET_OPERATING_MODEL]["content"]
)
# The Check that waits on the finding is reported with it, and nothing was
# revised for it: the model is not at fault, and neither are the Capabilities.
verdicts = json.loads((out / "verdicts.json").read_text())
waiting = [c for c in verdicts["checks"] if c["verdict"] == "waits_on_finding"]
assert [(c["check"], c["finding"]) for c in waiting] == [("OC-3", "AF-1")]
ledger = result.state["ledger"]["deliverables"]
assert all(entry["revisions"] == 0 for entry in ledger.values())
# The solution itself is as it was described: nothing was added to it.
scope = json.loads((out / "scope.json").read_text())
assert [item["id"] for item in scope["stated"]["solution"]["items"]] == [
"S-1",
"S-2",
"S-3",
]
# --- starting from sources ---------------------------------------------------------------
async def test_an_execution_starts_from_a_solution_architectures_files(tmp_path):
from pathlib import Path as _Path
from app.operating_model import digest
fixture = (
_Path(__file__).parent / "fixtures" / "architecture" / "case-08-landing-zone"
)
engine, runtime = make_engine(
tmp_path, {"scope-analyst": [submit(OPERATING_SCOPE)]}
)
result = await engine.start(
"",
sources=digest.read_folder(fixture),
# The one essential thing the architecture cannot say: who operates it.
scope="sourcing:\n model: shared\nintent:\n business_outcomes:\n"
" - {id: BO-1, outcome: Payments are never down for long.}\n",
execution_id="e1",
)
assert result.status == "awaiting_user"
assert result.prompt["approval"]["phase"] == "scoping"
assert result.state["customer_case"] == "case-08-landing-zone"
assert "payment platform" in result.state["workflow_request"]
scope = json.loads(
result.state["deliverables"][OPERATING_SCOPE]["attachments"]["scope.json"]
)
assert len(scope["stated"]["solution"]["items"]) == 9
assert len(scope["stated"]["solution"]["conditions"]) == 12
assert scope["stated"]["sourcing"] == {"model": "shared"}
# What was digested is shown to the customer as digested, to approve.
register = result.state["deliverables"][OPERATING_SCOPE]["content"]
assert "digested from the sources supplied" in register
# Every source is on record with its hash, and was read by code.
assert set(result.state["sources"]) == set(digest.FILES)
assert all(entry["digested"] for entry in result.state["sources"].values())
assert "source_documents" not in runtime.runs[0].task.context
async def test_a_document_code_cannot_read_is_given_to_the_scope_analyst(tmp_path):
engine, runtime = make_engine(tmp_path, {"scope-analyst": [[]]})
document = "# Our platform\n\nWe run a booking service on three clusters.\n"
await engine.start(
"Design the operating model for our booking platform.",
customer_case="bookings",
sources={"design.md": document},
)
context = runtime.runs[0].task.context
assert context["source_documents"] == {"design.md": document}
assert runtime.runs[0].role == "scope-analyst"
async def test_a_finding_the_model_could_meet_itself_sends_the_model_back(tmp_path):
"""A gap in the operating model's own staffing is not a finding: the validator
fails the Check that waits on it, and the architect designs it instead."""
excuse = {
"id": "AF-1",
"about": ["S-2"],
"needed_by": ["OC-3"],
"lacks": "The retention review is only staffed in business hours.",
"proposed": "Staff it at every hour.",
}
waiting = graph()
waiting["checks"][2]["depends_on_finding"] = "AF-1"
judged = json.loads(json.dumps(VERDICTS))
judged["checks"][2].update(
verdict="fail",
evidence="Staffing is the operating model's to provide; this is not a finding.",
)
scripted = turns(waiting, judged, [excuse])
scripted["operating-model-architect"].append(
submit(TARGET_OPERATING_MODEL, **{GRAPH_FILE: GRAPH, FINDINGS_FILE: []})
)
scripted["validator"].append(
submit(VALIDATION_RESULTS, **{VERDICTS_FILE: VERDICTS})
)
engine, runtime = make_engine(tmp_path, scripted)
result = await engine.start("", scope=SPEC)
assert result.status == "completed", result.state.get("warnings")
assert "revise_code" in [d["action"] for d in result.state["decisions"]]
second = [run for run in runtime.runs if run.role == "operating-model-architect"][1]
assert any("not a finding" in item for item in second.task.context["feedback"])
assert graph_findings(result) == []
def graph_findings(result) -> list:
attachments = result.state["deliverables"][TARGET_OPERATING_MODEL]["attachments"]
return json.loads(attachments.get(FINDINGS_FILE, "[]"))

241
tests/test_graph.py Normal file
View File

@@ -0,0 +1,241 @@
"""The Target operating model graph: what is refused, and the views code writes."""
import json
import pytest
from app.operating_model import graph as model
from app.operating_model.names import (
FINDINGS_FILE,
GRAPH_FILE,
ORGANISATION_FILE,
STAFFING_FILE,
)
from tests.support import CARD, CATALOGUE, GRAPH, as_file, graph, upstream
check = model.check(CARD)
add_views = model.views(CATALOGUE, CARD)
FINDING = {
"id": "AF-1",
"about": ["S-2"],
"needed_by": ["C-3"],
"lacks": "Nothing alerts when the audit log store's retention lock is removed.",
"proposed": "Add an alert on a change to the retention lock.",
}
def faults(changed, findings=None) -> list[str]:
attachments = as_file(GRAPH_FILE, changed)
if findings is not None:
attachments[FINDINGS_FILE] = json.dumps(findings)
return check(attachments, upstream())
def test_a_graph_that_holds_together_is_accepted():
assert faults(GRAPH) == []
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda g: g["parties"][0].update(kind="friend"),
"Party P-1: kind must be one of",
),
(
lambda g: g["roles"][0].update(party="P-9"),
"Role R-1 is in P-9, which is not a Party",
),
(
lambda g: g["roles"][1].update(country="FR"),
"Role R-2 sits in FR, which the rate card does not have",
),
(
lambda g: g["roles"][1].update(country="US"),
"Role R-2 sits in US; the customer allows only GB, PL, IN",
),
(
lambda g: g["roles"][1].update(level="wizard"),
"Role R-2 is at level wizard, which the rate card does not have",
),
(
lambda g: g["roles"][1].update(hours="nights"),
"Role R-2: hours must be one of",
),
(lambda g: g["roles"][1].pop("country"), "(R-2) needs country"),
(
lambda g: g["responsibilities"][0].update(kind="owner"),
"Responsibility RS-1: kind must be one of",
),
(
lambda g: g["responsibilities"][0].update(role="R-9"),
"RS-1 names R-9, which is not a Role",
),
(
lambda g: g["responsibilities"][0].update(capability="C-9"),
"RS-1 names C-9, which is not a Capability",
),
(
lambda g: g["responsibilities"].pop(0),
"Capability C-1 has 0 accountable Roles; it needs exactly one",
),
(
lambda g: g["responsibilities"][2].update(kind="accountable"),
"Capability C-1 has 2 accountable Roles; it needs exactly one",
),
(
lambda g: g["responsibilities"].pop(1),
"Capability C-1 has no responsible Role",
),
(
lambda g: g["roles"][1].update(hours="business"),
"Capability C-1 is needed 24x7, and no Role responsible for it works those hours",
),
(lambda g: g["checks"].pop(1), "Condition K-1 is carried by no Check"),
(
lambda g: g["checks"][0].update(on="C-9"),
"Check OC-1 is on C-9, which is not",
),
(
lambda g: g["checks"][0].update(source="OR-9"),
"Check OC-1: source OR-9 is not an Operational requirement",
),
(
lambda g: g["checks"][0].update(depends_on_finding="AF-1"),
"Check OC-1 depends on AF-1, which is not a finding",
),
(
lambda g: g["roles"].append(dict(g["roles"][0])),
"id R-1 is used more than once in roles",
),
],
)
def test_a_graph_that_does_not_hold_together_is_refused(change, problem):
changed = graph()
change(changed)
found = faults(changed)
assert any(problem in fault for fault in found), found
def test_a_file_that_is_not_a_graph_is_named():
assert "not valid JSON" in check({GRAPH_FILE: "{"}, upstream())[0]
assert "must be an object with" in check({GRAPH_FILE: "[]"}, upstream())[0]
def test_a_finding_is_recorded_and_a_check_may_wait_on_it():
changed = graph()
changed["checks"][2]["depends_on_finding"] = "AF-1"
assert faults(changed, [FINDING]) == []
@pytest.mark.parametrize(
("change", "problem"),
[
(lambda f: f.update(lacks=""), "AF-1 needs lacks"),
(lambda f: f.update(proposed=""), "AF-1 needs proposed"),
(lambda f: f.update(needed_by=[]), "AF-1 must say what needs it"),
(lambda f: f.update(about=["S-9"]), "AF-1 is about S-9, which is not an Item"),
(
lambda f: f.update(about=[]),
"AF-1 names no Item of the solution under about",
),
(
lambda f: f.update(needed_by=["C-9"]),
"AF-1 is needed by C-9, which is not in the operating model",
),
],
)
def test_a_finding_that_does_not_hold_together_is_refused(change, problem):
finding = dict(FINDING)
change(finding)
found = faults(GRAPH, [finding])
assert any(problem in fault for fault in found), found
def viewed(changed=GRAPH, findings=None, spec=None):
attachments = as_file(GRAPH_FILE, changed)
if findings is not None:
attachments[FINDINGS_FILE] = json.dumps(findings)
built = upstream() if spec is None else upstream(spec=spec)
return add_views(
"# Target operating model\n\n## Design rationale\n\nWhy.\n", attachments, built
)
def test_the_matrix_is_written_from_the_graph():
content, _ = viewed()
assert "## Responsibility matrix" in content
assert "| Capability | Platform owner | Platform SRE | Service manager |" in content
assert "| Incident management (C-1) | C | R | A |" in content
assert "| Change enablement (C-2) | A | R | |" in content
# One Role may be answerable for a Capability and do it: both parts are shown.
assert "| Regulatory evidence and reporting (C-3) | C | | A/R |" in content
def test_staffing_and_cost_are_written_with_the_prices_they_came_from():
content, attachments = viewed()
assert (
"| Platform SRE | Managed service provider | PL | senior | 24x7 | 100.0 h | 5.0 | 41,640 GBP |"
in content
)
assert (
"| Platform owner | Northwind platform team | GB | lead | business | 0.0 h | 0.0 | not priced |"
in content
)
assert "Cost a month: 43,410 GBP." in content
assert "The monthly budget is 120,000 GBP: this is within it." in content
assert "version `draft-2026-10-13`" in content
assert "Its figures are placeholders" in content
assert "Converted from USD at 0.79, a rate set on 2026-10-13." in content
assert json.loads(attachments[STAFFING_FILE])["cost_per_month"] == 43410
def test_a_cost_over_the_budget_is_shown_as_over_not_refused():
from tests.support import SPEC
tight = SPEC.replace("monthly_run_budget: 120000", "monthly_run_budget: 40000")
content, _ = viewed(spec=tight)
assert "this is over it by 3,410 GBP." in content
assert check(as_file(GRAPH_FILE, GRAPH), upstream(spec=tight)) == []
def test_the_organisation_chart_is_drawn_from_the_graph():
_, attachments = viewed()
chart = attachments[ORGANISATION_FILE]
assert chart.startswith("flowchart TB\n")
assert 'subgraph P_2["Managed service provider (provider)"]' in chart
assert 'R_2["Platform SRE<br/>PL, senior, 24x7"]' in chart
def test_findings_are_listed_for_whoever_owns_the_design():
content, attachments = viewed(findings=[FINDING])
assert "## Findings" in content
assert "**AF-1** (S-2): Nothing alerts when" in content
assert json.loads(attachments[FINDINGS_FILE]) == [FINDING]
def test_with_no_findings_the_file_is_empty_and_no_section_is_written():
content, attachments = viewed()
assert "## Findings" not in content
assert attachments[FINDINGS_FILE] == "[]"
def test_a_revision_replaces_the_views_and_never_stacks_them():
first, attachments = viewed()
second, _ = add_views(first, dict(attachments), upstream())
assert second == first
assert second.count("## Responsibility matrix") == 1

166
tests/test_guide.py Normal file
View File

@@ -0,0 +1,166 @@
"""The operating model guide: the packager writes the summary, code writes the rest."""
import json
from app.operating_model import guide, scenarios, spec, staffing
from app.operating_model.names import (
FINDINGS_FILE,
SCENARIOS_FILE,
STAFFING_FILE,
TARGET_OPERATING_MODEL,
VALIDATION_RESULTS,
VERDICTS_FILE,
)
from tests.support import (
CAPABILITIES,
CARD,
CATALOGUE,
GRAPH,
VERDICTS,
WORKFLOW,
upstream,
)
DRAFT = "# Northwind landing zone\n\n" + "".join(
f"## {section}\n\n"
+ (
"The provider runs the platform at every hour; Northwind approves changes.\n\n"
if section == guide.SUMMARY
else "It costs 9,999 GBP and needs 2 people.\n\n"
)
for section in guide.SECTIONS
)
def delivered(findings=None, model=GRAPH) -> dict:
"""Every Deliverable the guide is assembled from, with what code added to each."""
built = upstream(model=model, findings=findings)
scope = spec.agreed(built)
cost = staffing.compute(model, CAPABILITIES, CARD, "GBP")
built[TARGET_OPERATING_MODEL]["attachments"][STAFFING_FILE] = json.dumps(cost)
built[TARGET_OPERATING_MODEL]["attachments"].setdefault(FINDINGS_FILE, "[]")
walked = scenarios.walk(model, CAPABILITIES, scope, CATALOGUE, cost)
waiting = {
str(c["id"]): str(c["depends_on_finding"])
for c in model["checks"]
if c.get("depends_on_finding")
}
judged = json.loads(json.dumps(VERDICTS))
for verdict in judged["checks"]:
if verdict["check"] in waiting:
verdict.update(verdict="not_verifiable", evidence="Waits on its finding.")
built[VALIDATION_RESULTS] = {
"content": "# Validation\n",
"attachments": {
VERDICTS_FILE: scenarios.merge(json.dumps(judged), walked, waiting),
SCENARIOS_FILE: json.dumps(walked),
},
}
return built
def assembled(**options) -> str:
return guide.assemble(DRAFT, delivered(**options), CATALOGUE)
def section(text: str, heading: str) -> str:
return text.split(f"## {heading}\n")[1].split("\n## ")[0]
def test_the_guide_has_every_section_once_and_in_order():
text = assembled()
headings = [line[3:] for line in text.splitlines() if line.startswith("## ")]
assert headings == list(guide.SECTIONS)
assert text.startswith("# Northwind landing zone\n")
assert WORKFLOW.check_submission("operating_model_guide", text, {}) == ()
def test_the_packagers_summary_is_kept_and_the_key_figures_are_set_under_it():
summary = section(assembled(), guide.SUMMARY)
assert "The provider runs the platform at every hour" in summary
assert "### Key figures" in summary
assert "- Operated: 3 Items, with 2 Conditions to keep true." in summary
assert (
"- People: 5.1. Cost a month: 43,410 GBP, from rate card version "
"`draft-2026-10-13` (placeholder prices: an illustration)." in summary
)
assert "- Scenario walk-throughs: 10 of 10 pass." in summary
def test_a_figure_the_packager_wrote_under_another_heading_is_replaced():
text = assembled()
assert "9,999" not in text
assert "needs 2 people" not in text
assert "Cost a month: 43,410 GBP." in section(text, "Staffing and cost")
def test_each_section_is_written_from_the_data():
text = assembled()
assert "| S-2 | Audit log store | storage | Cloud Storage | E-2 |" in section(
text, "What is operated"
)
assert "| K-1 | S-1 | Flow logs are retained for 400 days. | CHK-12 |" in text
assert "| OR-2 | process |" in section(text, "Operational requirements")
assert (
"| C-1 | Incident management | OR-1, OR-4 | S-1, S-2, S-3 | 24x7 | 60 h |"
in section(text, "Capabilities")
)
who = section(text, "Who does what")
assert (
"| Platform SRE | Managed service provider (provider) | PL | senior | 24x7 |"
in who
)
assert "| Incident management (C-1) | C | R | A |" in who
checked = section(text, "Validation")
assert "| OC-1 | A fault on the hub network is acknowledged" in checked
assert "| Hub network (S-1) must be changed. | pass |" in checked
def test_sources_are_cited_and_said_to_be_unchecked():
cited = section(assembled(), "Sources")
assert "- ITIL 4: Incident management practice; Change enablement practice" in cited
assert "certifies nothing against them" in cited
assert "have not been checked against the frameworks by a person" in cited
def test_what_was_left_open_is_listed_and_what_nobody_asked_for_is_not():
left_open = section(assembled(), "Findings and open questions")
assert "No findings" in left_open
assert (
"`operations.change_process`: affects change enablement and release."
in left_open
)
assert "`service_management.tooling`: Which service management" in left_open
assert "metadata.author" not in left_open
def test_a_finding_and_the_check_that_waits_on_it_are_shown():
finding = {
"id": "AF-1",
"about": ["S-2"],
"needed_by": ["C-3"],
"lacks": "Nothing alerts when the retention lock is removed.",
"proposed": "Add an alert on a change to the retention lock.",
}
waiting = json.loads(json.dumps(GRAPH))
waiting["checks"][2]["depends_on_finding"] = "AF-1"
text = assembled(findings=[finding], model=waiting)
assert "**AF-1** (S-2): Nothing alerts when" in section(
text, "Findings and open questions"
)
assert "| waits on finding (AF-1) |" in section(text, "Validation")
assert "- Findings for whoever owns the design: 1." in text
def test_assembling_a_guide_again_changes_nothing():
first = assembled()
assert guide.assemble(first, delivered(), CATALOGUE) == first

218
tests/test_outcomes.py Normal file
View File

@@ -0,0 +1,218 @@
"""Measures tied to the customer's business outcomes."""
import json
import pytest
from app.operating_model import digest, guide
from app.operating_model import graph as model
from app.operating_model.names import GRAPH_FILE
from tests.support import CAPABILITIES, CARD, CATALOGUE, SPEC, as_file, graph, upstream
OUTCOMES = SPEC.replace(
" - {id: BO-1, outcome: Outages of the platform are short., target: Restored within 4 hours.}\n"
" - {id: BO-2, outcome: Regulators always get the evidence they ask for.}\n",
" - {id: BO-1, outcome: New products reach customers sooner., target: Daily releases.}\n"
" - {id: BO-2, outcome: Payments are never down for long.}\n",
)
# The same spec with no business outcome stated: only the customer can say none.
NO_OUTCOMES = (
OUTCOMES.split(" business_outcomes:\n")[0]
+ " business_outcomes: not_applicable\n"
+ " requirements:\n"
+ OUTCOMES.split(" requirements:\n")[1]
)
check = model.check(CARD, CATALOGUE.frameworks)
def measured() -> dict:
changed = graph()
changed["measures"] = [
{
"id": "M-1",
"name": "Time to restore service",
"layer": "outcome",
"outcome": "BO-2",
"measured_by": "C-1",
"framework": "dora-metrics",
"target": "Under 4 hours",
},
{
"id": "M-2",
"name": "Change lead time",
"layer": "outcome",
"outcome": "BO-1",
"measured_by": "C-2",
"framework": "dora-metrics",
},
]
return changed
def faults(changed, text=OUTCOMES, capabilities=None) -> list[str]:
return check(
as_file(GRAPH_FILE, changed), upstream(spec=text, capabilities=capabilities)
)
def test_every_outcome_shown_by_a_measure_is_accepted():
assert faults(measured()) == []
def test_with_no_outcomes_stated_no_measures_are_needed():
unmeasured = graph()
unmeasured["measures"] = []
assert faults(unmeasured, NO_OUTCOMES) == []
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda g: g["measures"].pop(1),
"business outcome BO-1 (New products reach customers sooner.) is shown by no Measure",
),
(
lambda g: g["measures"][0].update(outcome="BO-9"),
"Measure M-1 shows BO-9, which is not a business outcome the customer stated",
),
(
lambda g: g["measures"][0].update(layer="vanity"),
"Measure M-1: layer must be one of",
),
(
lambda g: g["measures"][0].update(measured_by="C-9"),
"Measure M-1 is measured by C-9, which is not a Capability",
),
(
lambda g: g["measures"][0].update(framework="okr"),
"Measure M-1 names framework okr, which is not in the register",
),
(
lambda g: g["responsibilities"].pop(1),
"Measure M-1 is measured by C-1, and nobody is responsible for it",
),
],
)
def test_a_measure_that_serves_no_stated_outcome_or_nobody_takes_is_refused(
change, problem
):
changed = measured()
change(changed)
found = faults(changed)
assert any(problem in fault for fault in found), found
def test_a_measure_is_refused_when_the_customer_stated_no_outcomes():
found = faults(measured(), NO_OUTCOMES)
assert any("none were stated" in fault for fault in found), found
def test_measuring_how_work_flows_measures_all_three_layers():
capabilities = [dict(c) for c in CAPABILITIES]
capabilities[2]["catalogue"] = "delivery-and-developer-experience-measurement"
changed = measured()
changed["measures"].append(
{
"id": "M-3",
"name": "Deployment frequency",
"layer": "outcome",
"outcome": "BO-1",
"measured_by": "C-3",
"framework": "dora-metrics",
}
)
found = faults(changed, capabilities=capabilities)
assert any(
"C-3 measures how work flows and has no Measure at the guardrail, leading layer"
in fault
for fault in found
), found
changed["measures"] += [
{
"id": "M-4",
"name": "Developer satisfaction",
"layer": "guardrail",
"outcome": "BO-1",
"measured_by": "C-3",
"framework": "space-framework",
},
{
"id": "M-5",
"name": "Perceived build wait",
"layer": "leading",
"outcome": "BO-1",
"measured_by": "C-3",
"framework": "devex-framework",
},
]
assert faults(changed, capabilities=capabilities) == []
def test_the_guide_shows_each_outcome_with_what_shows_it_and_who_answers():
built = upstream(spec=OUTCOMES, model=measured())
text = guide.business_outcomes(built, CATALOGUE)
assert (
"**BO-1: New products reach customers sooner.** Target: Daily releases." in text
)
assert (
"| Time to restore service | outcome | dora-metrics | Under 4 hours | "
"Incident management | Service manager |" in text
)
def test_the_guide_says_when_no_outcome_was_stated():
text = guide.business_outcomes(upstream(spec=NO_OUTCOMES), CATALOGUE)
assert text.startswith("The customer stated no business outcomes")
def test_an_operational_requirement_may_be_traced_to_a_business_outcome():
from app.operating_model import chain
from app.operating_model.names import REQUIREMENTS_FILE
from tests.support import REQUIREMENTS
requirements = json.loads(json.dumps(REQUIREMENTS))
requirements[3]["source"] = ["BO-2", "service_levels.recovery_time_hours"]
assert (
chain.check_requirements(
as_file(REQUIREMENTS_FILE, requirements), upstream(spec=OUTCOMES)
)
== []
)
def test_the_architectures_stated_success_criteria_become_business_outcomes():
from pathlib import Path
fixture = (
Path(__file__).parent / "fixtures" / "architecture" / "case-08-landing-zone"
)
sources = digest.read_folder(fixture)
scope = json.loads(sources[digest.SCOPE])
scope["stated"]["intent"]["success_criteria"] = [
"Payments engine onboarded within 10 weeks.",
"Audit findings closed before go-live.",
]
sources[digest.SCOPE] = json.dumps(scope)
digested = digest.architecture(
sources, digest.item_kinds(digest.Path(fixture).parents[3], "gcp")
)
assert digested.spec["intent"]["business_outcomes"] == [
{"id": "BO-1", "outcome": "Payments engine onboarded within 10 weeks."},
{"id": "BO-2", "outcome": "Audit findings closed before go-live."},
]
# In the real run they were assumed for the customer, so none are carried.
real = digest.architecture(digest.read_folder(fixture), {})
assert "business_outcomes" not in real.spec["intent"]

219
tests/test_scenarios.py Normal file
View File

@@ -0,0 +1,219 @@
"""Scenario walk-throughs: generated and traced by code, in place of a dry run."""
import json
from app.operating_model import scenarios, spec, staffing
from app.operating_model.names import VERDICTS_FILE
from tests.support import (
CAPABILITIES,
CARD,
CATALOGUE,
GRAPH,
SPEC,
VERDICTS,
graph,
scope_data,
)
def walked(model=GRAPH, capabilities=CAPABILITIES, text=SPEC) -> dict[str, dict]:
scope = scope_data(text)
currency = spec.value(scope, "commercial.currency")
cost = staffing.compute(model, capabilities, CARD, currency)
return {
item["id"]: item
for item in scenarios.walk(model, capabilities, scope, CATALOGUE, cost)
}
def test_every_scenario_of_a_sound_model_has_a_path():
found = walked()
assert set(found) == {
"SC-fault-S-1",
"SC-fault-S-2",
"SC-fault-S-3",
"SC-change-S-1",
"SC-change-S-2",
"SC-change-S-3",
"SC-condition-K-1",
"SC-condition-K-2",
"SC-party-P-2",
"SC-budget",
}
assert all(item["passes"] for item in found.values())
assert found["SC-fault-S-1"]["path"] == (
"Platform SRE responds under C-1, and Service manager is answerable."
)
assert found["SC-budget"]["path"] == "The cost is 43,410 GBP, within the budget."
def test_a_fault_out_of_hours_fails_when_nobody_responsible_is_on():
model = graph()
model["roles"][1]["hours"] = "business"
found = walked(model)
assert not found["SC-fault-S-1"]["passes"]
assert found["SC-fault-S-1"]["path"] == (
"No Role responsible for responding works 24x7 hours."
)
# Changes are made in business hours: that path still holds.
assert found["SC-change-S-1"]["passes"]
def test_an_item_no_responding_capability_covers_fails():
capabilities = [dict(c) for c in CAPABILITIES]
capabilities[0]["items"] = ["S-1", "S-2"]
found = walked(capabilities=capabilities)
assert (
found["SC-fault-S-3"]["path"]
== "No Capability that responds to faults covers it."
)
assert found["SC-fault-S-1"]["passes"]
def test_an_item_nobody_can_change_fails():
capabilities = [dict(c) for c in CAPABILITIES]
capabilities[1]["items"] = ["S-1", "S-2"]
assert not walked(capabilities=capabilities)["SC-change-S-3"]["passes"]
def test_a_condition_nothing_keeps_true_fails():
model = graph()
model["checks"].pop(1)
found = walked(model)
assert found["SC-condition-K-1"]["path"] == "No Check carries it."
assert found["SC-condition-K-2"]["passes"]
def test_a_party_with_nobody_else_to_turn_to_fails():
model = graph()
model["responsibilities"] = [
r for r in model["responsibilities"] if r["id"] not in ("RS-3", "RS-8")
]
found = walked(model)
assert not found["SC-party-P-2"]["passes"]
assert "C-1, C-3" in found["SC-party-P-2"]["path"]
def test_a_cost_over_the_budget_fails_the_budget_scenario():
tight = SPEC.replace("monthly_run_budget: 120000", "monthly_run_budget: 40000")
found = walked(text=tight)
assert found["SC-budget"]["path"] == "The cost is 43,410 GBP, over the budget."
def test_with_no_budget_stated_there_is_no_budget_scenario():
assert "SC-budget" not in walked(
text=SPEC.replace(" monthly_run_budget: 120000\n", "")
)
def test_the_scenarios_verdicts_are_written_into_the_validators_file_by_code():
found = list(walked().values())
merged = json.loads(scenarios.merge(json.dumps(VERDICTS), found))
assert merged["dry_run"] == {
"verdict": "pass",
"evidence": "10 of 10 scenario walk-throughs pass.",
}
assert [c["check"] for c in merged["checks"]][:4] == [
"OC-1",
"OC-2",
"OC-3",
"OC-4",
]
assert len(merged["checks"]) == 14
# Merging again replaces the scenarios' verdicts; it never adds them twice.
again = json.loads(scenarios.merge(json.dumps(merged), found))
assert again == merged
def test_a_failed_scenario_fails_the_run():
model = graph()
model["roles"][1]["hours"] = "business"
merged = json.loads(
scenarios.merge(json.dumps(VERDICTS), list(walked(model).values()))
)
assert merged["dry_run"]["verdict"] == "fail"
failed = [c["check"] for c in merged["checks"] if c["verdict"] == "fail"]
assert failed == ["SC-fault-S-1", "SC-fault-S-2", "SC-fault-S-3"]
def verdict_faults(verdicts, model=GRAPH) -> list[str]:
return scenarios.check_verdicts(
{VERDICTS_FILE: json.dumps(verdicts)}, model, VERDICTS_FILE
)
def test_a_verdict_on_every_check_is_accepted():
assert verdict_faults(VERDICTS) == []
def test_a_check_without_a_verdict_is_refused():
missing = {"checks": VERDICTS["checks"][:-1]}
assert verdict_faults(missing) == ["verdicts.json: Check OC-4 has no verdict"]
def test_a_check_that_waits_on_a_finding_cannot_pass():
model = graph()
model["checks"][2]["depends_on_finding"] = "AF-1"
assert (
"Check OC-3 depends on finding AF-1. It is not_verifiable while the finding stands"
in verdict_faults(VERDICTS, model)[0]
)
waiting = json.loads(json.dumps(VERDICTS))
waiting["checks"][2]["verdict"] = "not_verifiable"
assert verdict_faults(waiting, model) == []
def test_a_verdict_on_something_that_is_not_a_check_is_refused():
extra = {
"checks": [
*VERDICTS["checks"],
{"check": "OC-9", "verdict": "pass", "evidence": "x"},
]
}
assert verdict_faults(extra) == [
"verdicts.json: OC-9 is not a Check of the Target operating model"
]
def test_a_check_whose_finding_the_model_could_meet_itself_may_fail():
model = graph()
model["checks"][2]["depends_on_finding"] = "AF-1"
judged = json.loads(json.dumps(VERDICTS))
judged["checks"][2].update(
verdict="fail", evidence="Staff the review 24x7; this is not the solution's."
)
assert verdict_faults(judged, model) == []
merged = json.loads(scenarios.merge(json.dumps(judged), [], {"OC-3": "AF-1"}))
# The failure stands as a failure: the router sends the model back.
assert merged["checks"][2]["verdict"] == "fail"
def test_a_check_left_not_verifiable_on_its_finding_is_recorded_as_waiting():
judged = json.loads(json.dumps(VERDICTS))
judged["checks"][2].update(verdict="not_verifiable", evidence="Waits on AF-1.")
merged = json.loads(scenarios.merge(json.dumps(judged), [], {"OC-3": "AF-1"}))
assert merged["checks"][2]["verdict"] == "waits_on_finding"
assert merged["checks"][2]["finding"] == "AF-1"

161
tests/test_staffing.py Normal file
View File

@@ -0,0 +1,161 @@
"""Staffing and cost: arithmetic on the rate card, never an agent's figure."""
import shutil
import pytest
import yaml
from app.operating_model import staffing
from app.operating_model.staffing import RateCardError
from app.operating_model.workflow import ROOT
from tests.support import BUNDLE, CAPABILITIES, CARD, GRAPH, graph
SKILL = ROOT / "skills" / staffing.SKILL
def by_role(result) -> dict:
return {row["role"]: row for row in result["roles"]}
def test_the_figures_for_a_fixed_graph_and_card_are_reproduced_exactly():
result = staffing.compute(GRAPH, CAPABILITIES, CARD, "GBP")
rows = by_role(result)
# The SRE is responsible for incidents (60 h, 24x7) and changes (40 h): 100 h
# is two thirds of one person in Poland, but every hour of every day needs a
# rota of five. 5 x 460 a day x 220 days / 12 x 1.25 uplift x 0.79 to GBP.
assert rows["R-2"]["effort_hours_per_month"] == 100
assert rows["R-2"]["hours"] == "24x7"
assert rows["R-2"]["people"] == 5.0
assert rows["R-2"]["cost_per_month"] == 41640
# The service manager: 16 h of a 137.5 h month in the United Kingdom.
assert rows["R-3"]["people"] == 0.1
assert rows["R-3"]["cost_per_month"] == 1770
assert result["cost_per_month"] == 43410
assert result["by_party"] == {"P-2": 43410}
assert result["by_country"] == {"PL": 41640, "GB": 1770}
assert result["by_capability"] == {"C-1": 24984, "C-2": 16656, "C-3": 1770}
def test_the_customers_own_staff_are_counted_and_never_priced():
rows = by_role(staffing.compute(GRAPH, CAPABILITIES, CARD, "GBP"))
assert rows["R-1"]["cost_per_month"] is None
assert rows["R-1"]["people"] == 0.0
def test_a_customer_role_that_does_the_work_still_carries_no_cost():
model = graph()
model["responsibilities"][4]["role"] = "R-1" # the customer makes the changes
rows = by_role(staffing.compute(model, CAPABILITIES, CARD, "GBP"))
assert rows["R-1"]["effort_hours_per_month"] == 40
assert rows["R-1"]["people"] == 0.3
assert rows["R-1"]["cost_per_month"] is None
def test_a_role_on_business_hours_is_as_small_as_its_work():
model = graph()
model["roles"][1]["hours"] = "business"
capabilities = [dict(c, hours="business") for c in CAPABILITIES]
row = by_role(staffing.compute(model, capabilities, CARD, "GBP"))["R-2"]
assert row["hours"] == "business"
assert row["people"] == 0.7 # 100 h of a 146.7 h month, and no rota
def test_the_result_is_in_the_currency_the_spec_states():
pounds = staffing.compute(GRAPH, CAPABILITIES, CARD, "GBP")
dollars = staffing.compute(GRAPH, CAPABILITIES, CARD, "USD")
assert dollars["cost_per_month"] == 54948
assert pounds["conversion"] == {"from": "USD", "rate": 0.79, "as_of": "2026-10-13"}
assert dollars["conversion"]["rate"] == 1.0
def test_roles_that_share_a_capability_divide_its_effort():
model = graph()
model["responsibilities"].append(
{"id": "RS-9", "capability": "C-3", "role": "R-2", "kind": "responsible"}
)
equal = by_role(staffing.compute(model, CAPABILITIES, CARD, "GBP"))
model["responsibilities"][6]["share"] = 0.75
model["responsibilities"][-1]["share"] = 0.25
stated = by_role(staffing.compute(model, CAPABILITIES, CARD, "GBP"))
assert equal["R-2"]["effort_hours_per_month"] == 108 # 100 and half of 16
assert stated["R-2"]["effort_hours_per_month"] == 104 # 100 and a quarter of 16
@pytest.mark.parametrize(
("shares", "problem"),
[
((0.75, None), "must all state a share"),
((0.75, 0.5), "do not add up to 1"),
((1.5, -0.5), "must all state a share"),
],
)
def test_shares_that_do_not_hold_together_are_a_fault(shares, problem):
entries = [
{"capability": "C-3", "role": role, "kind": "responsible", "share": share}
for role, share in zip(("R-2", "R-3"), shares, strict=True)
]
assert problem in staffing.shares(entries)[1][0]
def test_the_card_in_the_repository_is_marked_as_placeholders():
assert CARD.is_draft
assert CARD.prices == ("provider",)
def test_the_currencies_a_spec_may_state_are_the_ones_the_card_converts_to():
assert list(BUNDLE.scope_values["commercial.currency"]) == CARD.currencies()
@pytest.fixture
def skill(tmp_path):
shutil.copytree(SKILL, tmp_path / "skill")
return tmp_path / "skill"
def edit(skill, change):
path = skill / "assets" / "rates.yaml"
data = yaml.safe_load(path.read_text())
change(data)
path.write_text(yaml.safe_dump(data))
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda d: d["prices"].append("customer"),
"the customer's own staff are never priced",
),
(
lambda d: d["countries"]["PL"]["day_rate"].pop("lead"),
"PL has no day rate for level lead",
),
(lambda d: d["cover"].pop("24x7"), "cover for 24x7 needs a minimum_roster"),
(
lambda d: d["conversion"]["rates"].update(GBP=0),
"conversion rate for GBP must be a number above zero",
),
(lambda d: d.pop("currency"), "needs currency"),
],
)
def test_a_rate_card_that_does_not_hold_together_is_refused(skill, change, problem):
edit(skill, change)
with pytest.raises(RateCardError, match=problem):
staffing.load(skill)
def test_a_card_whose_version_is_set_is_no_longer_a_draft(skill):
edit(skill, lambda d: d.update(version="2026-11"))
assert not staffing.load(skill).is_draft

207
tests/test_teams.py Normal file
View File

@@ -0,0 +1,207 @@
"""Teams and how they work together: Team Topologies in the operating model graph."""
import pytest
from app.operating_model import graph as model
from app.operating_model import guide
from app.operating_model.names import GRAPH_FILE
from tests.support import (
CARD,
CATALOGUE,
GRAPH,
SPEC,
as_file,
graph,
scope_data,
upstream,
)
check = model.check(CARD)
def with_teams() -> dict:
"""The test model, with the provider's Roles in a platform team that the
customer's platform owner consumes as a service."""
teamed = graph()
teamed["teams"] = [
{
"id": "T-1",
"party": "P-1",
"name": "Northwind product squad",
"type": "stream_aligned",
},
{
"id": "T-2",
"party": "P-2",
"name": "Platform team",
"type": "platform",
"purpose": "Runs the landing zone as a product.",
},
{"id": "T-3", "party": "P-2", "name": "Adoption", "type": "enabling"},
]
teamed["roles"][0]["team"] = "T-1"
teamed["roles"][1]["team"] = "T-2"
teamed["roles"][2]["team"] = "T-2"
teamed["interactions"] = [
{
"id": "I-1",
"from": "T-1",
"to": "T-2",
"mode": "x_as_a_service",
"purpose": "Environments through the self-service portal.",
},
{"id": "I-2", "from": "T-3", "to": "T-1", "mode": "facilitating"},
{
"id": "I-3",
"from": "T-1",
"to": "T-2",
"mode": "collaboration",
"until": "the first workload is live",
},
]
return teamed
def faults(changed) -> list[str]:
return check(as_file(GRAPH_FILE, changed), upstream())
def test_teams_are_optional():
assert faults(GRAPH) == []
def test_teams_of_the_four_kinds_working_in_the_three_modes_are_accepted():
assert faults(with_teams()) == []
@pytest.mark.parametrize(
("change", "problem"),
[
(
lambda g: g["teams"][1].update(type="squad"),
"Team T-2: type must be one of stream_aligned, platform",
),
(
lambda g: g["teams"][1].update(party="P-9"),
"Team T-2 is in P-9, which is not a Party",
),
(
lambda g: g["roles"][1].update(team="T-9"),
"Role R-2 is in T-9, which is not a Team",
),
(
lambda g: g["roles"][0].update(team="T-2"),
"Role R-1 is in P-1, and its team T-2 is in P-2",
),
(
lambda g: g["interactions"][0].update(mode="ticket"),
"Interaction I-1: mode must be one of",
),
(
lambda g: g["interactions"][0].update(to="T-9"),
"Interaction I-1: to T-9 is not a Team",
),
(
lambda g: g["interactions"][0].update(to="T-1"),
"Interaction I-1 is between a team and itself",
),
(
lambda g: g["interactions"][2].pop("until"),
"Interaction I-3 is a collaboration and does not say when it ends",
),
(lambda g: g["teams"][0].pop("type"), "(T-1) needs type"),
],
)
def test_teams_that_do_not_hold_together_are_refused(change, problem):
changed = with_teams()
change(changed)
found = faults(changed)
assert any(problem in fault for fault in found), found
def test_a_check_may_be_on_a_team_or_an_interaction():
changed = with_teams()
changed["checks"].append(
{
"id": "OC-9",
"on": "I-1",
"source": "OR-2",
"condition": "Every environment request is met through the portal, with no ticket.",
}
)
assert faults(changed) == []
def test_the_chart_draws_each_team_inside_its_party_and_how_teams_work_together():
chart = model.render_organisation(with_teams())
assert ' subgraph T_2["Platform team (platform team)"]' in chart
assert ' R_2["Platform SRE<br/>PL, senior, 24x7"]' in chart
assert ' T_1 -- "x-as-a-service" --> T_2' in chart
assert ' T_1 -- "collaboration until the first workload is live" --> T_2' in chart
def test_the_chart_without_teams_is_as_it_was():
chart = model.render_organisation(GRAPH)
assert " R_2[" in chart
assert " -- " not in chart
def test_the_guide_shows_the_teams_and_their_interactions():
built = upstream(model=with_teams())
text = guide.who_does_what(built, CATALOGUE)
assert (
"| Platform team | platform | Managed service provider | Platform SRE, Service manager | Runs the landing zone as a product. |"
in text
)
assert (
"| Northwind product squad | Platform team | x-as-a-service | | Environments through the self-service portal. |"
in text
)
# The matrix still follows.
assert "| Incident management (C-1) |" in text
def test_a_platform_foundation_is_proposed_the_platform_and_measurement_capabilities():
proposed = CATALOGUE.applicable(scope_data(SPEC))
assert {
"platform-product-management",
"self-service-and-golden-paths",
"enablement",
"delivery-and-developer-experience-measurement",
} <= set(proposed)
def test_a_workload_is_not_proposed_the_platform_capabilities():
workload = SPEC.replace("kind: platform_foundation", "kind: workload")
proposed = CATALOGUE.applicable(scope_data(workload))
assert "platform-product-management" not in proposed
assert "delivery-and-developer-experience-measurement" in proposed
def test_measurement_cites_three_layers_and_none_alone():
entry = CATALOGUE.entries["delivery-and-developer-experience-measurement"]
assert [s["framework"] for s in entry.sources] == [
"dora-metrics",
"space-framework",
"devex-framework",
]
assert len(entry.typical_checks) == 3
def test_the_socio_technical_frameworks_are_registered():
names = {key: value["name"] for key, value in CATALOGUE.frameworks.items()}
assert names["team-topologies"].startswith("Team Topologies")
assert "platform as a product" in names["platform-as-a-product"]
assert names["space-framework"] == "The SPACE of Developer Productivity"
assert names["devex-framework"].startswith("DevEx")