feat: start an Execution from a scope spec (ADR-0008, iteration 9 step 1)
Some checks failed
validation / verify (push) Failing after 10s

A scope spec is a YAML file of what the customer wants and the facts a design
depends on, complete or partly filled. It sits beside the Intent, which stays
in the customer's words.

- skills/scoping holds a provider-neutral catalogue of 63 fields; the Cloud
  provider's bundle supplies the values it allows for the fields left to it.
- app/scope.py reads a spec against the catalogue with no model: it refuses an
  unknown field or a value of the wrong type, and yields what is stated, what
  is left to the agent, what is unknown and how pressing each gap is. A field
  may say ask, decide or not_applicable.
- ExecutionEngine.start(scope=...), /generate, A2A metadata and evaluation
  cases accept one. Its statement is the request, its engagement reference
  names the Customer case, and its run options apply. A pre-approved start is
  refused when an essential field is not settled.
- The spec as supplied is kept unaltered beside a completed copy with
  assumptions, gaps, provenance, readiness and a status.
- Every Specialist is shown the spec and given the Skill: a stated choice
  binds, exclusions are out of scope.
- Covers workloads and platform foundations; case-08-landing-zone starts from
  a benchmark spec.

Plans for scoping (iteration 9) and the work breakdown structure (iteration
10, nothing built), and the design spec they come from. Not run with real
models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-11 17:53:00 +01:00
parent a48468d477
commit afe78640f8
27 changed files with 2122 additions and 8 deletions

View File

@@ -14,6 +14,8 @@ One directory per iteration of this agent. Each holds the plan for that iteratio
| 6 | `iter-06-target-architecture-graph/` | complete: all four cases pass with real models, each run twice on 2026-10-08; thresholds set | The Target architecture as one explicit graph of Elements, Relationships and Checks, established before infrastructure code, with validation as a verdict per Check. |
| 7 | `iter-07-change-requests/` | complete (2026-10-09): the change case ran with real models and passed | A user restates a request to say how requirements changed; a new Execution starts from a Baseline Execution of the same Customer case and delivers the updated architecture with what changed. |
| 8 | `iter-08-terraform-modules/` | steps 1–5 built and run once each way on 2026-10-09, both at 100%; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. |
| 9 | `iter-09-scoping/` | step 1 built on 2026-10-11, not run with real models; steps 2–6 planned; ADR-0008 accepted | An engagement is scoped from a scope spec: a structured file, complete or partly filled, read against a catalogue in code, completed in conversation and approved before requirements begin. Covers workloads and platform foundations. |
| 10 | `iter-10-work-breakdown/` | planned; nothing built | A work breakdown structure as a Deliverable: work packages traced to Elements and Checks, activities in sequence, and a timeline on a relative day axis computed and drawn by code. |
Numbers record when an iteration was planned, not the order to build in. Build iteration 6 before the remaining steps of iteration 4 (10 to 14) and before iteration 5: it changes the Deliverables, Phases and validation format they build on. Its `MAP.md` lists the effect on each of their steps.

View File

@@ -0,0 +1,34 @@
# Plan Map
**Iteration:** 9, Scoping
**Intent:** Scope an engagement from a scope spec: a structured file that may arrive complete, partly filled or not at all, completed in conversation and approved by the customer before requirements begin. So that a design's quality no longer rests on what the agents had to assume.
**Status:** step 1 built on 2026-10-11, not run with real models; steps 2 to 6 planned
**This directory covers:** steps 1 to 6
## Sources of truth
- `docs/adr/0008-scoping-starts-from-a-scope-spec.md`: the decisions.
- `.scratch/scoping-and-work-breakdown/spec.md`: the design, including the draft spec, the owner's own example and how the two were combined.
- `skills/scoping/assets/catalogue.yaml`: every field a scope spec may hold.
## Status
| Step | What | Status |
|---|---|---|
| 1 | The catalogue, reading and validating a spec, the gap list, the two files beside the Intent, run options from the spec, the spec shown to every Specialist | built 2026-10-11 (`steps/step-1.md`); not run with real models |
| 2 | The conversation that fills gaps and writes answers into the completed copy; scoping as a Phase with its Deliverable and Approval; a spec drafted from a sentence | planned |
| 3 | Requirements traced to the customer's requirement ids and to spec fields, in `requirements.json`; readiness in the solution guide | planned |
| 4 | An edited spec as a Change request, compared field by field | planned |
| 5 | Evaluation cases as specs, replacing answers matched by pattern | planned; `case-08-landing-zone` is the first |
| 6 | The status `BASELINE_LOCKED` set when the Execution completes | planned |
## Dependency graph
- Step 1: no dependencies
- Step 2: depends on step 1
- Step 3: depends on step 1
- Step 4: depends on step 1, iteration 7
- Step 5: depends on step 1
- Step 6: depends on step 2
The work breakdown structure is iteration 10 and uses the spec's `delivery` section from step 1.

View File

@@ -0,0 +1,20 @@
# Step 1 — Implementation Detail
Built on 2026-10-11. **Not run with real models.**
1. **The catalogue** is `skills/scoping/assets/catalogue.yaml`: 63 fields as dotted paths, each with a type, a tier (which may differ by kind of engagement), the question to ask and guidance for a default. It names no Cloud provider. `providers/gcp.yaml` adds, under `scope.values`, what Google Cloud allows for the four fields left to a provider: the provider itself, encryption type, network topology and the kind of hybrid connection.
2. **Reading a spec** is `read_scope` in `app/scope.py`, with no model. It refuses, naming every problem: YAML that does not parse, a field the catalogue does not have, a value of the wrong type, and an engagement reference and a customer case that disagree. A whole section may be marked `ask`, `decide` or `not_applicable` at once.
3. **What it yields:** the stated values; the gaps, most pressing first, each with its question and a severity (HIGH essential, MEDIUM important, LOW optional); the assumptions, one per `decide`, with the default to take and what a wrong one would change; a count of each; and a status.
4. **Starting from a spec:** `ExecutionEngine.start(scope=...)`, `scope` in the `/generate` body and in A2A metadata, and on an evaluation case. The spec's statement is the request when none is given, its engagement reference names the Customer case, and its `run` section applies where the caller gave nothing. A pre-approved start is refused when an essential field is not settled.
5. **The two files** are written beside the Intent: `NNN.scope.yaml`, the spec as supplied, byte for byte; and `NNN.scope.completed.yaml`, every field of the catalogue with the agent's sections (`inferred_assumptions`, `discovery_gaps`, `provenance`, `readiness`) and `metadata.status`. `history.json` names both. A completed copy can be supplied again as a spec; the agent's sections are replaced.
6. **Every Specialist is shown the spec** as `scope` in its context (what is stated, what is theirs to decide, what is unknown) and is given the scoping Skill. The protocol says a stated choice binds, an exclusion is out of scope, and the customer's requirement ids are to be named. The requirements analyst is told to derive from it and ask only for what it leaves unknown.
7. **For a person:** `python -m app.scope check <file>` says what a spec settles and what it lacks; `python -m app.scope template <kind>` prints one to fill in.
8. **A benchmark spec:** `eval/datasets/specs/landing-zone.yaml`, a fictional regulated payment platform, and `case-08-landing-zone` which starts from it.
Interim, until step 2:
- **Gaps are asked by the requirements analyst,** who is handed them, not by a scoping Phase. Answers are not written back into the completed copy.
- **Status** is set once, at the start: `DRAFT` with gaps to ask, `UNDER_REVIEW` when complete and someone is present, `APPROVED` when pre-approved. `BASELINE_LOCKED` is not set.
- **A caller's `pre_approved` wins over the spec's** when both are given. `/generate` is always pre-approved.
Tests: `tests/test_scope.py` (18), `tests/execution/test_scope_spec.py` (8).

View File

@@ -0,0 +1,23 @@
# Plan Map
**Iteration:** 10, Work breakdown structure
**Intent:** Deliver a plan for building the Target architecture: work packages traced to its Elements and Checks, activities in sequence, and a timeline on a relative day axis that code computes and draws.
**Status:** planned; nothing built
**This directory covers:** steps 1 to 4
## Sources of truth
- `.scratch/scoping-and-work-breakdown/spec.md`, section "The work breakdown structure".
- `docs/adr/0005-the-target-architecture-is-an-explicit-graph.md`: the ids it traces to.
- Iteration 9 step 1: the scope spec's `delivery` section (team size, window, commercial model).
## Status
| Step | What | Status |
|---|---|---|
| 1 | The Deliverable, its Specialist and its data file; every Element in a work package, checked in code | planned |
| 2 | The schedule computed in code: start and finish day, critical path, team size as a limit | planned |
| 3 | The three views written by code: WBS dictionary, activity sequence list, timeline on a relative axis | planned |
| 4 | The delivery window checked; assumptions the plan depends on shown on the timeline; milestones | planned |
Decisions the owner accepted on 2026-10-11: after validation, in the packaging Phase; grouped by component; the technical build only.

View File

@@ -0,0 +1,364 @@
# Spec: Scoping from a scope spec, and a work breakdown structure
Status: ready-for-agent
Accepted by the owner on 2026-10-11. Stage 1 is built; `.agents/plans/iter-09-scoping/` says what is done.
- Glossary: [`CONTEXT.md`](../../CONTEXT.md). New terms this would add are in **Language** below.
- Builds on: [ADR-0005](../../docs/adr/0005-the-target-architecture-is-an-explicit-graph.md) (ids along the Requirements chain), [ADR-0006](../../docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md) (Change requests), [ADR-0002](../../docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md) (behaviour in Skills).
- Would need: one new ADR, for a new first Phase and a new start to the Requirements chain.
## Problem
The Intent is the customer's request, saved word for word. For case 06 it is one sentence. Everything else a design needs (how many users, what availability, which region, what budget, which compliance regime, what exists today) is assumed by the requirements analyst or, in the evaluation, supplied by pattern-matched answers.
Three days of runs trace most downstream trouble to what nobody stated: Checks the validator asks for on topics the customer never raised, checklist items the request cannot settle, a "low-carbon region" no code can show. Nothing tells the customer at the start how much of their design rests on assumption, and a requirement does not record whether the customer stated it or the agent assumed it.
Separately, the agent delivers a design and its code but no plan for building it.
## Decisions taken
The owner accepted these defaults on 2026-10-11.
1. **Scoping is a Phase the customer approves,** before requirements discovery.
2. **Its centre is a scope spec:** a YAML file that may arrive complete, partly filled or not at all.
3. **Architects write the spec,** the owner and colleagues, not end customers. Field names may be technical.
4. **A small set of fields is essential** and always settled before work starts. Which they are depends on the kind of engagement.
5. **Two files.** The file supplied is never altered. A completed copy is written beside it.
6. **The run options live in the spec,** so one file starts an Execution.
7. **Where the spec and the prose request disagree, the spec wins** and the customer is shown the conflict.
8. **The work breakdown structure is a new Deliverable,** after validation, in the packaging Phase, grouped by component as the technical decomposition is, covering the technical build.
9. **Scoping is built before the work breakdown structure.**
## Language
Terms to add to `CONTEXT.md` when built:
- **Scope spec:** the structured statement of what the customer wants and the facts a design depends on, as a file. It sits beside the Intent, which stays in the customer's words.
- **Scoping:** the Phase that completes the scope spec and has the customer approve it.
- **Work breakdown structure:** the Deliverable that divides building the Target architecture into work packages and activities, with a timeline.
- **Work package:** a unit of work that delivers named Elements and is done when their Checks pass.
Already in the glossary and used here: Intent, Customer question, Change request, Baseline, Requirements chain.
## The scope spec
### What a field can say
Every field holds a value or one of three words:
| Written | Meaning | What the agent does |
|---|---|---|
| a value | The customer states this | Uses it. Recorded as `supplied`. |
| `ask`, or the field left out | Not known yet | Asks, when someone is present. Otherwise treats it as `decide` and lists it under `discovery_gaps`. |
| `decide` | You choose | Picks a default, recorded as `assumed` with the default and what it affects. |
| `not_applicable` | Does not apply | Asks nothing and assumes nothing. |
A field name the catalogue does not have is an error that names the field. Nothing is silently ignored.
### Tiers
| Tier | When it is settled |
|---|---|
| essential | Always, before any work: asked when someone is present, and refused with a list when nobody is and it is neither supplied nor `decide`. |
| important | Asked when someone is present; defaulted otherwise. |
| optional | Defaulted unless supplied. |
### The draft
Revised on 2026-10-11 to combine the first draft with the owner's own example, an engagement spec for a regulated landing zone. What each brought is listed under **How the two were combined**. Essential fields are marked; which are essential depends on `engagement.kind`. Values shown are from the owner's example.
```yaml
schema_version: 1 # the catalogue this file is written against
metadata:
customer_case: fintech-global-landing-zone # essential: groups this work's Executions
engagement_reference: ENG-2026-UK-084 # yours; carried through, never interpreted
client_name: FinTech Global UK
author: Pre-Sales Engineering
revision: "1.1.0" # yours: the version of this document
status: DRAFT # written by the agent; see Status below
engagement:
kind: platform_foundation # essential: workload | platform_foundation | migration
# 1. What the customer said. The source of truth; nothing here is rewritten.
intent:
statement: | # essential: verbatim, or consolidated from calls and documents
We are a UK-regulated payment platform expanding into Google Cloud. We need an
enterprise-grade landing zone ready in under 10 weeks ...
requirements: # what the customer stated outright, with their ids
- id: REQ-01
category: Timeline
statement: Platform foundation must be ready for workload ingestion within 10 weeks.
- id: REQ-03
category: Networking
statement: 10Gbps on-premises hybrid connectivity; centralized Palo Alto inspection for egress.
exclusions:
- Application container deployment (managed by app squads).
- Active-active secondary region in phase 1 (warm standby acceptable).
success_criteria: decide
# 2. Where it runs.
estate:
cloud_provider: gcp # essential; must be a provider this deployment has
mode: greenfield # essential: greenfield | brownfield_expansion | migration
regions:
primary: europe-west2 # essential
secondary: europe-west1
resiliency_tier: warm_standby
data_residency: United Kingdom # essential
existing_systems: ask
# 3. Who uses it and how hard. Essential for kind workload; not_applicable for a foundation.
workload:
users: {who: not_applicable, count: not_applicable, peak_concurrent: not_applicable}
availability: {target: ask, hours: always, recovery_time: ask, recovery_point: ask}
# 4. Regulation and governance.
compliance:
industry: financial_services
frameworks: [DORA, ISO27001, PCI_DSS_v4] # essential; an empty list means none apply
data_classification: confidential # essential
encryption: {type: cmek, key_rotation_days: 90}
audit_logging: {retention_days: 365, immutable_retention_policy: true}
# 5. Network.
networking:
exposure: internal_only # internal_only | internet_facing
topology: shared_vpc_hub_and_spoke # allowed values come from the Cloud provider's catalogue
hybrid_connectivity: {enabled: true, type: cloud_interconnect, bandwidth_gbps: 10}
egress_control: {model: centralized_appliance, appliance_vendor: palo_alto}
cidr_allocations: ask
# 6. Identity, environments and operations.
governance:
identity_source_of_truth: entra_id
pipeline_authentication: workload_identity_federation
environment_tiers:
- {name: development, archetype: corp_workload}
- {name: staging, archetype: corp_workload}
- {name: production, archetype: pci_regulated_workload}
operations:
operating_model: central_platform_broker
support_hours: decide
monitoring: decide
integrations: []
# 7. How it will be delivered. The work breakdown structure is built from this.
delivery:
target_start_date: 2026-11-01
window_weeks: 10 # essential when the customer has a deadline
commercial_model: fixed_price_milestone
iac_tooling: terraform
ci_cd: github_actions
team_size: ask
working_days_per_week: 5
cost:
monthly_budget: ask # essential for kind workload
currency: GBP
# 8. How to run the agent.
run:
pre_approved: false
allow_cloud_read: false
overlay_skills: []
terraform_modules: false
```
### Sections the agent writes
Only in the completed copy. The supplied file never has to carry them, and if it does they are read as the last run's and replaced.
```yaml
inferred_assumptions:
- id: ASSUMP-01
field: networking.hybrid_connectivity
category: Network Telecom
statement: Assumes the client manages the carrier circuit lead time for the 10Gbps interconnect.
impact_if_invalid: Blocks the hybrid routing test; delays every activity after it.
discovery_gaps:
- field: networking.cidr_allocations
prompt: Specify IP subnet sizes for spoke workloads, for example /22 per environment.
severity: HIGH # HIGH is an essential field; MEDIUM important; LOW optional
provenance: # where every settled value came from
estate.regions.primary: supplied
cost.monthly_budget: answered
operations.monitoring: assumed
readiness: {supplied: 34, answered: 3, assumed: 6, unknown: 2}
```
`provenance` is one of `supplied`, `answered`, `discovered` (read from the customer's cloud environment), `assumed`, `unknown`. Every `assumed` field has an entry in `inferred_assumptions`, and every `unknown` one in `discovery_gaps`.
### Status
The owner's example carries a status. The agent sets it in the completed copy from what has happened; a status written by the author is not trusted.
| Status | Means |
|---|---|
| `DRAFT` | Gaps remain, or the conversation is still filling them. |
| `UNDER_REVIEW` | Complete, and waiting for the scoping Approval. |
| `APPROVED` | The customer approved it; requirements and the later Phases run from it. |
| `BASELINE_LOCKED` | Its Execution completed. This copy is the record a Change request starts from, and is never altered. |
### How the two were combined
| From the owner's example | How it is kept |
|---|---|
| The request, verbatim, inside the spec | `intent.statement`. It is also still saved as the Intent, as today. |
| The customer's stated requirements, with ids | `intent.requirements`. The agent's requirements trace to these ids, so the Requirements chain starts at what the customer wrote. |
| Stated exclusions | `intent.exclusions`. The design is checked for nothing built that is excluded. |
| Engagement metadata and status | `metadata`, with the status set by the agent. |
| Estate, compliance, networking and governance sections | Kept as written. |
| Start date, delivery window, commercial model, tooling | `delivery`. The window becomes a limit the plan is checked against. |
| Assumptions with their impact, and gaps with a prompt and a severity | The two agent-written sections, in place of the first draft's flat provenance table. Severity is the tier. |
| From the first draft | How it is kept |
|---|---|
| `ask`, `decide` and `not_applicable` | Unchanged. The example had no way to say "not known" other than omission. |
| Tiers | Kept, as the severity of a gap. |
| Two files | Kept. The author's file is untouched; the agent's sections go in the completed copy. |
| Where each value came from | Kept, as `provenance`. |
| Users, scale, availability target, budget | Kept under `workload` and `cost`, and essential only for a workload. |
| Run options | Kept under `run`. |
### Where the two pull apart
Four tensions, each with the resolution proposed. They are the owner's to confirm.
1. **The example states solution choices; the agent keeps requirements free of them.** A hub-and-spoke topology, an interconnect, customer-managed keys and a named inspection appliance are design decisions. The rule that requirements name no products exists so the agents do not pick a design before the design Phase. It should not stop a customer who has already decided. Proposed: a choice the spec states is a constraint the design must honour and may not change, recorded as supplied. The agents still add none of their own.
2. **The example is a landing zone; the agent has only designed workloads.** Every benchmark case is an application. A platform foundation has no users or response times, and has an organisation hierarchy, shared networking and environment archetypes the cases have never exercised. `engagement.kind` lets one spec serve both, but a platform foundation is a new kind of work for the agent and needs its own evaluation case before it is relied on.
3. **The example uses one Cloud provider's vocabulary in general fields.** `shared_vpc_hub_and_spoke`, `cloud_interconnect` and `cmek` are Google Cloud's terms. Proposed: the general catalogue says a field such as `networking.topology` exists; the Cloud provider's bundle supplies its allowed values. A second provider then brings its own list and the spec's shape does not change.
4. **"Engagement" is a word the glossary avoids.** The agent's term is Customer case. Proposed: `customer_case` stays the key the agent groups by, and `engagement_reference` carries the owner's own number untouched. If the owner would rather the agent's term became Engagement, that is a glossary change to make before this is built.
### Where the catalogue lives
A provider-neutral Skill holds the catalogue: each field's type, tier by engagement kind, the question to ask, and guidance for the default. A Cloud provider's bundle adds its allowed values and any fields of its own. An overlay Skill may raise a field's tier, so an organisation can make `compliance.encryption` essential. Code validates a spec against the catalogue; no model decides what is missing.
## How scoping runs
| What arrives | What happens |
|---|---|
| A complete spec | Validated. No gaps, no questions. With `pre_approved` the Phase passes without stopping. |
| A partly filled spec, someone present | Gaps are asked for, essential first, a few related fields at a time, with a count of how many are settled. |
| A partly filled spec, nobody present | Essential gaps refuse the start, with the list. Others are defaulted and recorded. |
| Only a sentence | A model drafts a spec from it, marking everything it did not find as `ask`. It is then a partly filled spec. |
In conversation an answer is a value, "you decide", or "skip". A skipped field becomes `unknown` and a Customer question in the output. A free-text answer is turned into the field's value and read back before it is written. The customer may supply an edited file at any point in place of answering.
The Phase ends when the customer approves the completed spec.
### What it writes
Under the Customer case, beside the Intent:
```
intent/
001.md # the request, in the customer's words (as today)
001.scope.yaml # the spec as supplied, unaltered; absent when none was
001.scope.completed.yaml # the completed copy: status, assumptions, gaps, provenance, readiness
```
The scoping Deliverable is a short document written from the completed copy: the scope statement, the information register as a table, and the readiness summary. The solution guide opens with that summary.
### What it changes downstream
- **The requirements analyst derives; it no longer asks or assumes.** Each requirement names the stated requirement (`REQ-03`) or the spec field it comes from, so the Requirements chain starts at what the customer wrote. A requirement from an `assumed` field says so.
- **A stated choice binds the design.** A product, topology or setting the spec states is carried to the design as a constraint, and a design that departs from it is refused.
- **Exclusions are checked.** Nothing in `intent.exclusions` may appear in the Target architecture.
- **Customer questions** from the pillar checklists (built 2026-10-10) are joined by the `unknown` fields.
- **A Change request may be an edited spec.** Code compares the two specs field by field, and that list is what changed. A sentence still works, as now.
- **An evaluation case becomes a spec,** replacing the pattern-matched answers.
## The work breakdown structure
A new Deliverable in the packaging Phase, produced by a new Specialist after validation, summarised in the solution guide.
### Its data file
```json
{
"work_packages": [
{"id": "WP-1", "name": "Network foundation", "parent": null,
"components": ["C-2"], "elements": ["E-1", "E-2"],
"scope": "The VPC, its subnet and outbound access.",
"produces": ["a private network the services attach to"],
"skills": ["network engineering"]}
],
"activities": [
{"id": "A-1", "work_package": "WP-1", "name": "Create the VPC and subnet",
"days": 1, "after": []},
{"id": "A-2", "work_package": "WP-1", "name": "Configure outbound access",
"days": 1, "after": ["A-1"]}
]
}
```
### What code checks and computes
- Every Element belongs to a work package, and every work package to a component or a parent. A design cannot be left out of the plan.
- A work package's acceptance criteria are the Checks on its Elements. They are read from the Target architecture, not written again.
- Activities form no cycle, and every `after` names an activity that exists.
- **The plan is checked against the delivery window.** When `delivery.window_weeks` is set and the computed total exceeds it, the Deliverable says so and names the critical path; it does not shorten estimates to fit.
- **An assumption can hold up the plan.** An entry in `inferred_assumptions` that an activity depends on, such as a carrier's lead time, appears as a dependency the customer owns, so its `impact_if_invalid` is visible on the timeline.
- **Milestones.** With a `commercial_model` of fixed price by milestone, each top-level work package ends in a milestone.
- **The schedule is computed, not written.** From each activity's days and what it comes after, code works out its earliest start and finish, the critical path and the total. With `delivery.team_size` it limits how many activities run at once.
### What it renders
Three views, all written by code from the data file, as the architecture diagram is:
- **WBS dictionary:** the hierarchy, each work package with its scope, what it produces, its Elements, its acceptance Checks and the skills it needs.
- **Activity sequence list:** each activity with its days, what it follows, its start and finish day, and whether it is on the critical path.
- **Timeline** on a relative axis, Day 0 to Day N:
```
Activity Day 0 5 10 15 18
A-1 Create the VPC and subnet ██
A-2 Configure outbound access ██
A-3 Provision the database █████
A-4 Deploy the application ████
A-5 Load balancer and access ███
A-6 Monitoring and alerts ██
A-7 Validation in test ████
```
The chart format is to be settled when built: a Mermaid Gantt chart if it can show a relative day axis, and otherwise a drawn table like the one above.
### What it says about itself
Durations are a model's estimates. The Deliverable states their basis (working days, the team size from the spec) and that they are indicative. The timeline stays on a relative day axis; when `delivery.target_start_date` is set, the activity list also gives calendar dates.
## Order of work
| Stage | What | Usable on its own |
|---|---|---|
| 1 | The catalogue, validation, the gap list, the two files saved beside the Intent, the run options read from the spec | Yes: a complete spec starts a pre-approved Execution |
| 2 | The conversation that fills gaps; scoping as a Phase with its Deliverable and Approval; a spec drafted from a sentence | Yes |
| 3 | Requirements traced to spec fields; readiness in the guide | Yes |
| 4 | An edited spec as a Change request | Yes |
| 5 | The work breakdown structure | Yes, with stage 1 for `delivery` |
| 6 | Evaluation cases as specs | Yes |
## Not in this spec
- Attaching documents (a statement of work, a diagram) as a source for the spec.
- Reading the customer's cloud environment to fill `current_estate`. The `discovered` source is reserved for it.
- Delivery activities beyond the technical build, such as security review, user acceptance and cutover.
- Cost of the work, or named people.
## Open questions
1. The four tensions under **Where the two pull apart**: are the proposed resolutions right?
2. Which fields are essential for each engagement kind? The draft marks them; the owner's example suggests the delivery window and compliance frameworks matter more than budget for a foundation.
3. In `pre_approved` runs an essential gap refuses the start. Is that right, or should it default and warn?
4. Should the scoping Approval be skippable when the spec arrived complete and someone is present?
5. Is a platform foundation in scope now, with its own evaluation case, or should scoping be built against workloads first?
## Comments
- 2026-10-11, the owner: a platform foundation is in scope now; an engagement name and a Customer case are equivalent references; the implementation will eventually be provider-agnostic. The other tensions and open questions were taken as proposed: a stated choice binds the design; an essential gap refuses a pre-approved start; the scoping Approval is not skipped. Recorded in ADR-0008.

View File

@@ -19,13 +19,21 @@ A caller-supplied grouping of Executions. A Session never feeds earlier outputs
_Avoid_: conversation, context
**Customer case**:
A caller-supplied grouping of the Executions run for one customer workload, identified by a short slug. Every Execution belongs to one; an Execution started without one belongs to `unassigned`. A Customer case ties each request to the Deliverables generated for it. It feeds one Execution's outputs into another only when the caller starts a Change request and names the Baseline.
_Avoid_: engagement, project, account, tenant
A caller-supplied grouping of the Executions run for one customer engagement, identified by a short slug. An engagement and a Customer case are the same thing: a reference such as `ENG-2026-UK-084` names the case `eng-2026-uk-084`. Every Execution belongs to one; an Execution started without one belongs to `unassigned`. A Customer case ties each request to the Deliverables generated for it. It feeds one Execution's outputs into another only when the caller starts a Change request and names the Baseline.
_Avoid_: project, account, tenant
**Intent**:
What the customer wants from a Customer case, in their own words and at a high level. Each Workflow request made in the case is a statement of it, and the statements are kept in order as its history. The Intent is the customer's; requirements are what the agents derive from it.
_Avoid_: brief, ask, requirements (which are the derived Deliverable)
**Scope spec**:
The structured statement of what the customer wants and the facts a design depends on, as a file. It may arrive complete or partly filled. It sits beside the Intent, which stays in the customer's words, and a value it states is the customer's decision.
_Avoid_: brief, questionnaire, requirements (which are derived from it)
**Scoping**:
Completing the scope spec, from what was supplied, what the customer answers and what the agent is left to decide, and having the customer approve it before requirements begin.
_Avoid_: discovery (which reads a cloud environment), intake (which reads a message)
**Change request**:
An Execution started to say how the customer's Intent has changed. It begins from a Baseline and delivers the full updated Deliverables with an account of what changed and why.
_Avoid_: amendment, update run, iteration, re-run

View File

@@ -18,6 +18,8 @@ One Orchestrator coordinates an Execution and is the only agent that talks to th
Each Phase needs the user's explicit Approval before work that depends on it starts. A pre-approved mode serves batch and evaluation runs.
An Execution may start from a **scope spec** ([ADR-0008](docs/adr/0008-scoping-starts-from-a-scope-spec.md)): a YAML file of what the customer wants and the facts a design depends on, complete or partly filled. Code reads it against a catalogue, refuses a field it does not know, and writes a completed copy beside the customer's request with what was assumed and what is still unknown. `python -m app.scope template platform_foundation` prints one to fill in, and `python -m app.scope check <file>` says what a spec lacks. Pass it as `scope` to `/generate` or in A2A metadata.
A **Change request** ([ADR-0006](docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md)) starts from a completed Execution in the same Customer case, its Baseline, and delivers the full updated Deliverables with `changes.json`: what changed, what was only reworded, and anything changed that nobody asked for.
The design Phase ends with the **Target architecture**: one explicit graph of Elements, Relationships and Checks ([ADR-0005](docs/adr/0005-the-target-architecture-is-an-explicit-graph.md)). The user approves it before any infrastructure code is written. The diagram is drawn from it by code, the Terraform must account for every Check, and validation gives a verdict per Check. Requirements, components, products and recommendations each carry ids in a data file, so every requirement can be traced through to its verdict; a submission with a missing link is refused.

View File

@@ -13,6 +13,7 @@ Your task message carries JSON context:
- `working_copy`: what your own tools already hold from your earlier attempts, or `null`. Start from it; do not start again.
- `baseline`: set only in a Change request, where the user has said how their requirements changed since an architecture was delivered. `deliverable` is that delivered version of your own Deliverable, and `changes` is what has changed so far: the requirements added, changed and removed, with the reason, and everything already revised from them. `null` otherwise.
- `read_first`: references of your Skills that this task was judged to need, each as a Skill and a resource. Read these first with `load_skill_resource`. It is where to start, not a limit: read anything else you find you need.
- `scope`: what the customer's scope spec says, when they supplied one. `stated` is theirs and binds your Deliverable; `to_decide` is yours to choose and record as an assumption; `to_ask` is unknown. Load the scoping Skill for the rules. Honour every stated choice, build nothing under `stated.intent.exclusions`, and name the customer's own requirement id when your work serves one.
- `overlay_skills`: Skills that carry the user's organisation's standards or this engagement's constraints. Load each one and apply it. Where an overlay disagrees with the Cloud provider's general guidance, the overlay wins; where it disagrees with a requirement the user stated, follow the requirement and say so.
- `pre_approved`: `true` when no user is available.

View File

@@ -11,7 +11,8 @@ You produce the **requirements** Deliverable: the Skill's requirements-discovery
- Ask an open question only where no sensible default can be inferred from the request, such as the region, a compliance regime or a budget ceiling.
- Name every ambiguity or contradiction, explain why the conflicting goals can't both hold, and ask the user to choose. When they delegate the choice, propose one and ask them to approve it.
- When the user runs workloads in a cloud environment today, use your cloud discovery tools to describe the current state from what actually exists, and record anything you couldn't read as a discovery gap.
- Name no products, services or architectures. This Deliverable describes the problem, not the solution.
- Name no products, services or architectures of your own choosing. This Deliverable describes the problem, not the solution. A product or design choice the customer stated in `scope` is theirs: record it as a constraint, in their words.
- When `scope` is set, derive the requirements from it. Cover every entry of `stated.intent.requirements` and name its id in the requirement that covers it. Use what is `stated` as given and ask nothing it already answers. Ask the questions under `to_ask`, the most severe first, when you may ask the user; otherwise assume a default for each and list it under `Assumptions` and `Open questions`. Choose a default for each field under `to_decide` and list it under `Assumptions` with what it affects.
- Give every requirement an id and attach them all as `requirements.json`, in the format the requirements-chain Skill gives. Use the same ids in your content.
- Give every non-functional requirement its `conditions`: one entry for each separate thing that must be true for it to be met and that the configuration of infrastructure can show. "Data is encrypted in transit and at rest" has two. Each becomes something the design is checked against, so a condition left out is a condition nobody verifies.
- Put under `outside_code` what the requirement asks for that no infrastructure configuration can show: an outcome measured in operation (an availability percentage, a response time), behaviour of the application's own code, a process people follow, or a property of a region or product that only its provider publishes (a carbon rating, a service level). For the last, the condition is the choice configuration can show, such as which region is set. Those are recorded and reported, and are not checked against the infrastructure code.

View File

@@ -130,8 +130,9 @@ class SolutionArchitectExecutor(AgentExecutor):
baseline=options.get("baseline"),
overlay_skills=options.get("overlay_skills") or (),
terraform_modules=options.get("terraform_modules"),
scope=options.get("scope"),
execution_id=task_id,
pre_approved=bool(options.get("pre_approved", False)),
pre_approved=options.get("pre_approved"),
allow_cloud_read=options.get("allow_cloud_read"),
)
else:

View File

@@ -46,6 +46,7 @@ from app.nodes.router import (
)
from app.providers import ProviderBundle, load_provider
from app.recorder import DatabaseRecorder, ExecutionRecorder
from app.scope import ESSENTIAL, Catalogue, Scope, read_scope, slug
from app.skills.agent_skills import discover_skills, skill_version
from app.states.execution_state import check_customer_case
from app.states.ledger import Ledger
@@ -106,6 +107,9 @@ class ExecutionEngine:
assess_at_end: bool = False,
modules_available: bool = False,
default_modules: bool = False,
scope_catalogue: Catalogue | None = None,
scope_values: Mapping[str, Sequence[str]] | None = None,
scope_skill: str | None = None,
) -> None:
required = {
ORCHESTRATOR,
@@ -115,6 +119,11 @@ class ExecutionEngine:
if missing:
raise ValueError(f"Missing roles: {', '.join(missing)}")
self._max_revisions = max_revisions
# The catalogue a scope spec is read against (ADR-0008); None when this
# deployment's Cloud provider has none.
self._scope_catalogue = scope_catalogue
self._scope_values = dict(scope_values or {})
self._scope_skill = scope_skill
# Whether the Cloud provider approves any module sources, and whether an
# Execution uses them when its caller does not say.
self._modules_available = modules_available
@@ -154,13 +163,14 @@ class ExecutionEngine:
*,
session_id: str | None = None,
customer_case: str | None = None,
pre_approved: bool = False,
pre_approved: bool | None = None,
allow_cloud_read: bool | None = None,
execution_id: str | None = None,
scripted_answers: Sequence[Sequence[str]] = (),
baseline: str | None = None,
overlay_skills: Sequence[str] = (),
terraform_modules: bool | None = None,
scope: str | Mapping[str, Any] | None = None,
) -> Interaction:
"""Start an Execution. `scripted_answers` are (regex, answer[, asker]) entries a
pre-approved Execution uses to answer questions, for evaluation runs.
@@ -180,7 +190,50 @@ class ExecutionEngine:
does as its Baseline did and any other Execution takes the deployment's
default, which is plain resources. Opting in is refused when the provider
approves no sources.
`scope` is a scope spec (ADR-0008), as YAML text or already parsed. It may
be complete or partly filled. The start is refused when it names a field
the catalogue does not have or holds a value of the wrong type, and, for a
pre-approved Execution, when an essential field is not settled: nobody is
there to ask. What the spec states under `run` applies where the caller
gave no value of their own. Its statement of intent is the request when
none is given.
"""
read = self._read_scope(scope) if scope is not None else None
if read is not None:
run = read.values
if pre_approved is None:
pre_approved = bool(run.get("run.pre_approved", False))
if allow_cloud_read is None and "run.allow_cloud_read" in run:
allow_cloud_read = bool(run["run.allow_cloud_read"])
if terraform_modules is None and "run.terraform_modules" in run:
terraform_modules = bool(run["run.terraform_modules"])
overlay_skills = [*overlay_skills, *run.get("run.overlay_skills", [])]
if (
customer_case
and read.customer_case
and slug(customer_case) != (read.customer_case)
):
raise ValueError(
f"The scope spec is for {read.customer_case}, and this was "
f"started for {customer_case}."
)
customer_case = customer_case or read.customer_case
# The spec's statement is the customer's; it is what gets recorded.
workflow_request = read.statement or workflow_request
missing = read.gaps(ESSENTIAL)
if pre_approved and missing:
raise ValueError(
"The scope spec leaves essential fields unsettled, and a "
"pre-approved Execution has nobody to ask: "
+ ", ".join(gap["field"] for gap in missing)
+ ". Give each a value, or `decide` to leave it to the agent."
)
pre_approved = bool(pre_approved)
if not workflow_request.strip():
raise ValueError(
"A request is needed: text, or a scope spec with a statement."
)
customer_case = check_customer_case(customer_case)
execution_id = execution_id or str(uuid.uuid4())
if await self._exists(execution_id):
@@ -224,6 +277,8 @@ class ExecutionEngine:
session_id=session_id,
workflow_request=workflow_request,
baseline=baseline or None,
scope_supplied=scope if isinstance(scope, str) else None,
scope_completed=read.completed(read.status(pre_approved)) if read else None,
)
initial: dict[str, Any] = {
"execution_id": execution_id,
@@ -236,6 +291,15 @@ class ExecutionEngine:
# Overlay Skills this Execution loads, each with the version it had.
"overlay_skills": {name: self._overlays[name] for name in chosen},
"terraform_modules": terraform_modules,
# What the customer's scope spec settles and leaves open; None without one.
"scope": {
"skill": self._scope_skill,
"status": read.status(pre_approved),
"readiness": read.readiness(),
"context": read.for_context(),
}
if read
else None,
"risks": [],
"pre_approved": pre_approved,
"allow_cloud_read": allow_cloud_read,
@@ -274,6 +338,13 @@ class ExecutionEngine:
await self._graph.ainvoke(initial, self._config(execution_id))
return await self.interaction(execution_id)
def _read_scope(self, supplied: str | Mapping[str, Any]) -> Scope:
if self._scope_catalogue is None:
raise ValueError("This deployment has no scope spec catalogue.")
return read_scope(
supplied, self._scope_catalogue, provider_values=self._scope_values
)
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
current = await self.interaction(execution_id)
@@ -517,6 +588,9 @@ async def open_assembly(
else None,
skill_threshold=settings.JEV_SKILL_THRESHOLD,
assess_at_end=settings.JEV_ASSESS_EXECUTIONS,
scope_catalogue=bundle.scope_catalogue() if bundle.scope_skill else None,
scope_values=bundle.scope_values,
scope_skill=bundle.scope_skill,
modules_available=bool(bundle.terraform_module_namespaces),
default_modules=settings.TERRAFORM_MODULES,
overlays={

View File

@@ -14,15 +14,20 @@ nothing was delivered for it. history.json is the index of the case.
from __future__ import annotations
import json
from collections.abc import Mapping
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
import yaml
from app.states.execution_state import case_dir
INTENT_DIR = "intent"
HISTORY_FILENAME = "history.json"
IN_PROGRESS = "in_progress"
SCOPE_SUPPLIED = ".scope.yaml"
SCOPE_COMPLETED = ".scope.completed.yaml"
def intent_dir(output_root: Path, customer_case: str) -> Path:
@@ -50,6 +55,8 @@ def record_intent(
session_id: str,
workflow_request: str,
baseline: str | None = None,
scope_supplied: str | None = None,
scope_completed: Mapping[str, Any] | None = None,
) -> int:
"""Record a new statement of the customer's intent and return its version.
@@ -71,6 +78,19 @@ def record_intent(
f"## What the customer wants\n\n{workflow_request.strip()}\n",
encoding="utf-8",
)
scope_files = None
if scope_completed is not None:
# The spec as supplied, unaltered, and the completed copy beside it.
scope_files = {"completed": f"{INTENT_DIR}/{version:03d}{SCOPE_COMPLETED}"}
(folder / f"{version:03d}{SCOPE_COMPLETED}").write_text(
yaml.safe_dump(dict(scope_completed), sort_keys=False, allow_unicode=True),
encoding="utf-8",
)
if scope_supplied:
scope_files["supplied"] = f"{INTENT_DIR}/{version:03d}{SCOPE_SUPPLIED}"
(folder / f"{version:03d}{SCOPE_SUPPLIED}").write_text(
scope_supplied, encoding="utf-8"
)
history["intents"].append(
{
"version": version,
@@ -82,6 +102,7 @@ def record_intent(
"status": IN_PROGRESS,
"execution": None,
"baseline": baseline,
"scope": scope_files,
}
)
_write_history(folder, history)

View File

@@ -62,8 +62,10 @@ def create_app(
except ValueError:
body = {}
workflow_request = str(body.get("request") or "").strip()
if not workflow_request:
return JSONResponse({"error": "'request' is required"}, status_code=400)
if not workflow_request and not body.get("scope"):
return JSONResponse(
{"error": "'request' or a 'scope' spec is required"}, status_code=400
)
try:
interaction = await assembly.engine.start(
workflow_request,
@@ -72,6 +74,7 @@ def create_app(
baseline=body.get("baseline"),
overlay_skills=body.get("overlay_skills") or (),
terraform_modules=body.get("terraform_modules"),
scope=body.get("scope"),
pre_approved=True,
allow_cloud_read=bool(body.get("allow_cloud_read", False)),
)

View File

@@ -481,6 +481,10 @@ def make_execution_nodes(
role = dataclasses.replace(
role, skills=(*role.skills, *(n for n in named if n not in role.skills))
)
scope_skill = task_input.get("scope_skill")
if scope_skill and scope_skill not in role.skills:
# An Execution with a scope spec: every Specialist can read what it means.
role = dataclasses.replace(role, skills=(*role.skills, scope_skill))
opted_in = bool(task_input.get("terraform_modules"))
MODULES_OPTED_IN.set(opted_in)
if opted_in and role.modules_skill:
@@ -908,6 +912,7 @@ def _specialist_input(
"allow_cloud_read": state.get("allow_cloud_read"),
# Whether this Execution opted into published Terraform modules.
"terraform_modules": bool(state.get("terraform_modules")),
"scope_skill": (state.get("scope") or {}).get("skill"),
"working": dict(state.get("working", {}).get(kind.value, {})),
# Whether the user has been shown a draft of this Deliverable. Answering some
# other question about it, such as what to do after it stopped, is not that.
@@ -937,6 +942,9 @@ def _specialist_input(
# Skills carrying the organisation's standards or this engagement's
# constraints. Where one disagrees with general guidance, it wins.
"overlay_skills": sorted(state.get("overlay_skills") or {}),
# What the customer's scope spec states, leaves to the agent, and
# leaves unknown. None when no spec was supplied.
"scope": (state.get("scope") or {}).get("context"),
# False: infrastructure code is plain resources. True: a published
# module may be used where the modules Skill says one fits.
"terraform_modules": bool(state.get("terraform_modules")),
@@ -1357,6 +1365,12 @@ def _write_outputs(
"overlay_skills": dict(state.get("overlay_skills") or {}),
# Whether its infrastructure code was allowed published modules.
"terraform_modules": bool(state.get("terraform_modules")),
# How much of the scope spec the customer supplied; None without a spec.
"scope": {
key: (state.get("scope") or {}).get(key) for key in ("status", "readiness")
}
if state.get("scope")
else None,
"status": status,
"warnings": list(state.get("warnings", [])),
# Flags a person should look at; nothing was done about them.

View File

@@ -14,6 +14,7 @@ from typing import Any
import yaml
from app.scope import Catalogue, load_catalogue
from app.skills.agent_skills import SkillFormatError, SkillMeta, discover_skills
TIERS = ("general", "cloud_read")
@@ -85,6 +86,17 @@ class ProviderBundle:
# The Skill approved lessons are published in (ADR-0004); None when the provider
# has none. A Specialist is given it once it holds lessons for its Deliverable.
lessons_skill: str | None = None
# The Skill that holds the scope spec's catalogue (ADR-0008), and the values
# this provider allows for the fields the catalogue leaves to a provider.
scope_skill: str | None = None
scope_values: Mapping[str, tuple[str, ...]] = field(default_factory=dict)
def scope_catalogue(self) -> Catalogue:
if not self.scope_skill:
raise ProviderConfigError(f"The {self.name} bundle names no scope Skill.")
return load_catalogue(
self.skills[self.scope_skill].path / "assets" / "catalogue.yaml"
)
def mcp_servers_for_role(
self, role: str, include_preview: bool = True
@@ -233,6 +245,15 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
if lessons_skill is not None and lessons_skill not in skills:
raise fail(f"'lessons_skill' references unknown skill {lessons_skill!r}")
scope = config.get("scope") or {}
scope_skill = scope.get("skill") or None
if scope_skill is not None and scope_skill not in skills:
raise fail(f"'scope.skill' references unknown skill {scope_skill!r}")
scope_values = {
str(path): tuple(str(value) for value in values or ())
for path, values in (scope.get("values") or {}).items()
}
return ProviderBundle(
name=name,
skills_dirs=skills_dirs,
@@ -256,4 +277,6 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
or ()
),
lessons_skill=lessons_skill,
scope_skill=scope_skill,
scope_values=scope_values,
)

460
app/scope.py Normal file
View File

@@ -0,0 +1,460 @@
"""The scope spec: what the customer wants and the facts a design depends on, as a
file (ADR-0008).
A spec may arrive complete, partly filled or not at all. This reads one against the
catalogue of fields a Skill holds, and says exactly what is settled and what is not.
No model is involved: whether a field is filled is a fact.
a value the customer states it supplied
ask, or absent not known yet a gap, with its question
decide the customer leaves the choice to the agent an assumption
not_applicable does not apply nothing asked or assumed
The file supplied is never altered. `Scope.completed()` is the copy written beside
it, with the sections the agent writes: assumptions, gaps, where each value came
from, and a count of each.
.venv/bin/python -m app.scope check <spec.yaml> [provider]
.venv/bin/python -m app.scope template [kind]
"""
from __future__ import annotations
import datetime as dt
import re
import sys
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import yaml
ASK, DECIDE, NOT_APPLICABLE = "ask", "decide", "not_applicable"
MARKERS = (ASK, DECIDE, NOT_APPLICABLE)
SUPPLIED, ASSUMED, UNKNOWN = "supplied", "assumed", "unknown"
ESSENTIAL, IMPORTANT, OPTIONAL = "essential", "important", "optional"
_SEVERITY = {ESSENTIAL: "HIGH", IMPORTANT: "MEDIUM", OPTIONAL: "LOW"}
DRAFT, UNDER_REVIEW, APPROVED, LOCKED = (
"DRAFT",
"UNDER_REVIEW",
"APPROVED",
"BASELINE_LOCKED",
)
KIND_FIELD = "engagement.kind"
CASE_FIELD = "metadata.customer_case"
STATEMENT_FIELD = "intent.statement"
# What the agent writes into the completed copy. A supplied file may still carry
# them from an earlier run; they are read as that run's and replaced.
_AGENT_WRITTEN = (
"schema_version",
"inferred_assumptions",
"discovery_gaps",
"provenance",
"readiness",
)
_AGENT_FIELDS = ("metadata.status",)
_SLUG = re.compile(r"[^a-z0-9]+")
class ScopeError(ValueError):
"""A scope spec that cannot be used, with every problem found in it."""
def __init__(self, problems: Sequence[str]) -> None:
self.problems = list(problems)
super().__init__("scope spec: " + "; ".join(self.problems))
@dataclass(frozen=True)
class FieldSpec:
path: str
type: str = "string"
tier: Any = OPTIONAL # a tier, or {kind: tier}
question: str = ""
default: Any = None
affects: str = ""
values: tuple[str, ...] = ()
provider_values: bool = False
keys: tuple[str, ...] = ()
optional_keys: tuple[str, ...] = ()
aliases: tuple[str, ...] = ()
def tier_for(self, kind: str | None) -> str:
if isinstance(self.tier, Mapping):
return str(self.tier.get(kind, OPTIONAL))
return str(self.tier)
@dataclass(frozen=True)
class Catalogue:
schema_version: int
kinds: tuple[str, ...]
fields: Mapping[str, FieldSpec]
def aliases(self) -> dict[str, str]:
return {
alias: spec.path for spec in self.fields.values() for alias in spec.aliases
}
def load_catalogue(path: Path) -> Catalogue:
data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
fields = {}
for name, raw in (data.get("fields") or {}).items():
raw = raw or {}
fields[name] = FieldSpec(
path=name,
type=str(raw.get("type", "string")),
tier=raw.get("tier", OPTIONAL),
question=" ".join(str(raw.get("question", "")).split()),
default=raw.get("default"),
affects=str(raw.get("affects", "")),
values=tuple(str(v) for v in raw.get("values") or ()),
provider_values=bool(raw.get("provider_values")),
keys=tuple(raw.get("keys") or ()),
optional_keys=tuple(raw.get("optional_keys") or ()),
aliases=tuple(raw.get("aliases") or ()),
)
return Catalogue(
schema_version=int(data.get("schema_version", 1)),
kinds=tuple(data.get("kinds") or ()),
fields=fields,
)
@dataclass
class Scope:
"""A scope spec read against the catalogue."""
catalogue: Catalogue
kind: str | None
values: dict[str, Any] = field(default_factory=dict) # field -> stated value
states: dict[str, str] = field(default_factory=dict) # field -> marker
supplied_text: str = ""
@property
def customer_case(self) -> str | None:
value = self.values.get(CASE_FIELD)
return str(value) if value else None
@property
def statement(self) -> str:
return str(self.values.get(STATEMENT_FIELD) or "").strip()
def tier(self, path: str) -> str:
return self.catalogue.fields[path].tier_for(self.kind)
def gaps(self, *tiers: str) -> list[dict[str, str]]:
"""What is not known, most pressing first: each with the question to ask."""
wanted = tiers or (ESSENTIAL, IMPORTANT, OPTIONAL)
found = [
{
"field": path,
"prompt": self.catalogue.fields[path].question or f"What is {path}?",
"severity": _SEVERITY[self.tier(path)],
}
for path, state in self.states.items()
if state == ASK and self.tier(path) in wanted
]
order = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
return sorted(found, key=lambda gap: order[gap["severity"]])
def assumptions(self) -> list[dict[str, str]]:
"""What the customer left to the agent, each with the default to take and
what a wrong choice would change."""
found: list[dict[str, str]] = []
for path, state in self.states.items():
if state != DECIDE:
continue
spec = self.catalogue.fields[path]
found.append(
{
"id": f"ASSUMP-{len(found) + 1:02d}",
"field": path,
"statement": f"The customer left {path} to the agent. "
f"Default: {spec.default if spec.default is not None else 'the agent chooses'}.",
"impact_if_invalid": f"Changes {spec.affects}."
if spec.affects
else "Local to this field.",
}
)
return found
def readiness(self) -> dict[str, int]:
states = list(self.states.values())
return {
SUPPLIED: len(self.values),
ASSUMED: states.count(DECIDE),
UNKNOWN: states.count(ASK),
NOT_APPLICABLE: states.count(NOT_APPLICABLE),
}
def status(self, pre_approved: bool = False) -> str:
"""Where the spec stands. With nobody to ask, what is important and unknown
is defaulted and listed, so only an essential gap keeps it a draft."""
if pre_approved:
return DRAFT if self.gaps(ESSENTIAL) else APPROVED
return DRAFT if self.gaps(ESSENTIAL, IMPORTANT) else UNDER_REVIEW
def completed(self, status: str) -> dict[str, Any]:
"""The completed copy: every field of the catalogue, and the agent's
sections."""
document: dict[str, Any] = {"schema_version": self.catalogue.schema_version}
for path in self.catalogue.fields:
value = self.values.get(path, self.states.get(path, ASK))
_put(document, path, value)
_put(document, "metadata.status", status)
provenance = {path: SUPPLIED for path in self.values}
provenance |= {
path: {DECIDE: ASSUMED, ASK: UNKNOWN, NOT_APPLICABLE: NOT_APPLICABLE}[state]
for path, state in self.states.items()
}
document["inferred_assumptions"] = self.assumptions()
document["discovery_gaps"] = self.gaps()
document["provenance"] = dict(sorted(provenance.items()))
document["readiness"] = self.readiness()
return document
def for_context(self) -> dict[str, Any]:
"""What a Specialist is shown: what the customer stated, what is theirs to
choose, and what is unknown."""
stated: dict[str, Any] = {}
for path, value in self.values.items():
if not path.startswith(("run.", "metadata.")):
_put(stated, path, value)
return {
"kind": self.kind,
"stated": stated,
"to_decide": [
{
"field": path,
"default": self.catalogue.fields[path].default,
"affects": self.catalogue.fields[path].affects,
}
for path, state in self.states.items()
if state == DECIDE and not path.startswith("run.")
],
"to_ask": [
gap for gap in self.gaps() if not gap["field"].startswith("run.")
],
"not_applicable": sorted(
path for path, state in self.states.items() if state == NOT_APPLICABLE
),
}
def _put(document: dict[str, Any], path: str, value: Any) -> None:
*parents, last = path.split(".")
node = document
for part in parents:
node = node.setdefault(part, {})
node[last] = value
def slug(reference: str) -> str:
"""A Customer case id from an engagement reference: `ENG-2026-UK-084` and
`eng-2026-uk-084` name the same case."""
return _SLUG.sub("-", reference.strip().lower()).strip("-")[:64]
def read_scope(
supplied: str | Mapping[str, Any],
catalogue: Catalogue,
*,
provider_values: Mapping[str, Sequence[str]] | None = None,
) -> Scope:
"""Read a scope spec. Raises ScopeError naming every problem: YAML that does not
parse, a field the catalogue does not have, a value of the wrong type."""
text = supplied if isinstance(supplied, str) else yaml.safe_dump(dict(supplied))
try:
data = yaml.safe_load(text) if isinstance(supplied, str) else dict(supplied)
except yaml.YAMLError as exc:
raise ScopeError([f"not valid YAML ({exc})"]) from exc
if not isinstance(data, Mapping):
raise ScopeError(["the spec must be a mapping of sections"])
problems: list[str] = []
leaves: dict[str, Any] = {}
aliases = catalogue.aliases()
prefixes = {
".".join(path.split(".")[:depth])
for path in catalogue.fields
for depth in range(1, len(path.split(".")))
}
def walk(node: Any, path: str) -> None:
target = aliases.get(path, path)
if path in _AGENT_FIELDS or path in _AGENT_WRITTEN:
return
if target in catalogue.fields:
if target in leaves and leaves[target] != node and path != target:
problems.append(f"{path} and {target} name the same thing and differ")
leaves[target] = node
return
if path in prefixes:
if isinstance(node, Mapping):
for key, value in node.items():
walk(value, f"{path}.{key}")
elif node in MARKERS or node is None:
# A whole section marked at once applies to every field in it.
for name in catalogue.fields:
if name.startswith(path + "."):
leaves.setdefault(name, node)
else:
problems.append(f"{path} is a section, not a value")
return
problems.append(f"{path} is not a field of the scope spec")
for key, value in data.items():
walk(value, str(key))
kind = leaves.get(KIND_FIELD)
kind = str(kind) if kind in catalogue.kinds else None
scope = Scope(catalogue, kind, supplied_text=text)
extra = dict(provider_values or {})
for path, spec in catalogue.fields.items():
value = leaves.get(path)
if value is None or value == ASK:
scope.states[path] = ASK
elif value in (DECIDE, NOT_APPLICABLE):
scope.states[path] = value
else:
fault = _fault(spec, value, extra.get(path, ()))
if fault:
problems.append(f"{path}: {fault}")
else:
scope.values[path] = _plain(value)
case = scope.values.get(CASE_FIELD)
if case is not None:
scope.values[CASE_FIELD] = slug(str(case))
if not scope.values[CASE_FIELD]:
problems.append(f"{CASE_FIELD}: needs letters or digits")
if problems:
raise ScopeError(problems)
return scope
def _plain(value: Any) -> Any:
"""A value as plain data: YAML reads a date as a date, and state must be JSON."""
if isinstance(value, (dt.date, dt.datetime)):
return value.isoformat()
if isinstance(value, Mapping):
return {str(k): _plain(v) for k, v in value.items()}
if isinstance(value, (list, tuple)):
return [_plain(item) for item in value]
return value
def _fault(spec: FieldSpec, value: Any, provider_values: Sequence[str]) -> str:
"""Why a value does not fit its field; "" when it does."""
kind = spec.type
if kind in ("string", "text"):
return "" if isinstance(value, (str, int, float)) else "must be text"
if kind == "integer":
ok = isinstance(value, int) and not isinstance(value, bool)
return "" if ok else "must be a whole number"
if kind == "number":
ok = isinstance(value, (int, float)) and not isinstance(value, bool)
return "" if ok else "must be a number"
if kind == "boolean":
return "" if isinstance(value, bool) else "must be true or false"
if kind == "date":
if isinstance(value, (dt.date, dt.datetime)):
return ""
try:
dt.date.fromisoformat(str(value))
except ValueError:
return "must be a date, as YYYY-MM-DD"
return ""
if kind == "enum":
allowed = (*spec.values, *provider_values)
if not allowed or str(value) in allowed:
return ""
return f"must be one of {', '.join(allowed)}"
if kind == "list":
return "" if isinstance(value, list) else "must be a list"
if kind == "records":
if not isinstance(value, list):
return "must be a list of entries"
known = {*spec.keys, *spec.optional_keys}
for number, entry in enumerate(value, start=1):
if not isinstance(entry, Mapping):
return f"entry {number} must have named fields"
missing = [key for key in spec.keys if not entry.get(key)]
if missing:
return f"entry {number} needs {', '.join(missing)}"
unknown = sorted(set(map(str, entry)) - known)
if unknown:
return f"entry {number} has unknown fields {', '.join(unknown)}"
return ""
return f"has an unknown type {kind} in the catalogue"
def render_template(catalogue: Catalogue, kind: str | None = None) -> str:
"""A scope spec with every field set to ask, to fill in."""
document: dict[str, Any] = {}
for path, spec in catalogue.fields.items():
if path == KIND_FIELD and kind:
_put(document, path, kind)
elif spec.tier_for(kind) != OPTIONAL or kind is None:
_put(document, path, ASK)
header = (
"# Scope spec. Give each field a value, or: ask (not known yet), decide\n"
"# (you choose), not_applicable. A field left out is read as ask.\n"
)
return header + yaml.safe_dump(document, sort_keys=False, allow_unicode=True)
def render_reading(scope: Scope) -> str:
"""What a spec settles and what it leaves open, for a person."""
counts = scope.readiness()
lines = [
f"Engagement kind: {scope.kind or 'not stated'}",
(
f"{counts[SUPPLIED]} supplied, {counts[ASSUMED]} left to the agent, "
f"{counts[UNKNOWN]} unknown, {counts[NOT_APPLICABLE]} not applicable."
),
]
for severity, title in (
("HIGH", "Essential, needed before work can start"),
("MEDIUM", "Important, asked when someone is present"),
):
gaps = [gap for gap in scope.gaps() if gap["severity"] == severity]
if gaps:
lines += ["", f"{title}:"]
lines += [f" {gap['field']}: {gap['prompt']}" for gap in gaps]
return "\n".join(lines)
def main(arguments: Sequence[str]) -> int:
from app.config import get_settings
from app.providers import load_provider
command, rest = (arguments[0], arguments[1:]) if arguments else ("", [])
if command == "template":
bundle = load_provider(get_settings().CLOUD_PROVIDER)
print(render_template(bundle.scope_catalogue(), rest[0] if rest else None))
return 0
if command == "check" and rest:
bundle = load_provider(
rest[1] if len(rest) > 1 else get_settings().CLOUD_PROVIDER
)
try:
scope = read_scope(
Path(rest[0]).read_text(encoding="utf-8"),
bundle.scope_catalogue(),
provider_values=bundle.scope_values,
)
except ScopeError as refused:
print("This spec cannot be used:")
for problem in refused.problems:
print(f" {problem}")
return 1
print(render_reading(scope))
return 0
print(__doc__, file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

View File

@@ -59,6 +59,9 @@ class ExecutionState(TypedDict, total=False):
risks: list[dict[str, Any]] # flags on what the user wrote, for a person to see
overlay_skills: dict[str, str] # overlay Skills this Execution loads -> version
terraform_modules: bool # opted into published Terraform modules
# The customer's scope spec as read at the start: its Skill, status, readiness
# and what a Specialist is shown of it. None when no spec was supplied.
scope: dict[str, Any] | None
baseline: dict[str, Any] | None # a Change request's Baseline, read from its files
pre_approved: bool
allow_cloud_read: bool | None

View File

@@ -0,0 +1,50 @@
---
status: accepted
---
# Scoping starts from a scope spec
An Execution began from the customer's request, a sentence or two saved word for word, and the requirements analyst filled everything else with assumptions. Most of what went wrong downstream traced to what nobody had stated, and nothing told the customer how much of their design rested on assumption. We decide that an engagement is scoped from a **scope spec**: a structured file that states what the customer wants and the facts a design depends on, which may arrive complete, partly filled or not at all, and which the customer approves before requirements begin.
The Intent stays the customer's own words. The scope spec sits beside it as the agreed, structured form.
## What the spec is
- **A file of fields from a catalogue.** A Skill holds the catalogue: each field's type, when it must be settled, the question to ask for it, and guidance for a default. The catalogue names no Cloud provider. A provider's bundle supplies the allowed values for the fields it leaves to a provider, in that provider's terms.
- **Every field says one of four things.** A value is the customer's statement. `ask`, or the field left out, means not known. `decide` leaves the choice to the agent, recorded as an assumption. `not_applicable` asks and assumes nothing.
- **Whether a field is settled is read by code.** No model decides what is missing.
- **Fields are essential, important or optional,** and which is which depends on the kind of engagement: a workload, a platform foundation or a migration.
- **It carries the customer's own words and requirements:** the statement verbatim, the requirements they stated with their own ids, and what they excluded.
- **It carries how to run the agent,** so one file starts an Execution.
## Decisions
Accepted by the owner on 2026-10-11, with each of these:
1. **The file supplied is never altered.** The agent writes a completed copy beside it, with what it assumed, what is still unknown, where each value came from, and a status.
2. **A choice the customer states binds the design.** A product, a topology or a setting written in the spec is a constraint every Specialist honours. The rule that requirements name no products is about what the agents introduce, not what the customer has already decided.
3. **A platform foundation is in scope,** alongside the application workloads the agent was built on.
4. **An engagement and a Customer case are the same thing.** A spec may name either; an engagement reference is turned into the case's id.
5. **The spec is provider-agnostic in shape.** Provider vocabulary enters only as values a provider's bundle allows.
6. **An essential field that is not settled refuses a start that has nobody to ask.** With someone present it is asked for.
7. **Scoping becomes a Phase the customer approves,** before requirements discovery.
8. **Where the spec and a prose request disagree, the spec wins.**
## Considered Options
- **Judge the free-text request against a list of topics** with a model or Jev: the first proposal. Rejected once a spec was on the table: a field that is filled or not is a fact, and a judgement about prose is not.
- **One file the agent completes in place:** rejected. An author must be able to see that what they wrote is what was used.
- **Keep product names out of the spec, as out of requirements:** rejected. A customer who has decided on an interconnect or a named appliance has stated a constraint; refusing to record it would lose it.
- **A separate spec for platform foundations:** rejected. One catalogue with tiers by kind serves both, and a migration is usually some of each.
## Consequences
- The Requirements chain gains a start: a requirement traces to the customer's own requirement id or to a spec field.
- The requirements analyst stops being where questions are asked and assumptions made. That moves to scoping.
- A Change request may be an edited spec, compared field by field in code.
- An evaluation case may be a spec, in place of answers matched by pattern.
- The glossary gains **Scope spec** and **Scoping**, and Customer case says that an engagement is the same thing.
- A platform foundation is new work for the agents. It needs its own evaluation case before it is relied on.
- The spec holds client names and details. Like lesson candidates it is client data: kept with the Customer case, never in the repository, apart from the fictional benchmark specs.
This is built in stages; `.agents/plans/iter-09-scoping/` says which are done.

View File

@@ -545,5 +545,21 @@
]
],
"terraform_modules": true
},
{
"id": "case-08-landing-zone",
"name": "A regulated platform foundation, started from a scope spec",
"scope": "landing-zone.yaml",
"required_products": [
"Cloud Interconnect",
"Shared VPC",
"Cloud KMS"
],
"scripted_answers": [
[
".*",
"Proceed on the scope spec as supplied. Where it says decide, choose a sensible default and record it as an assumption."
]
]
}
]

View File

@@ -0,0 +1,117 @@
# A benchmark scope spec: a regulated platform foundation. A fictional customer.
schema_version: 1
metadata:
engagement_id: case-08-landing-zone
client_name: Example Payments UK
author: Evaluation
revision: "1.0.0"
engagement:
kind: platform_foundation
intent:
statement: |
We are a UK-regulated payment platform expanding into Google Cloud. We need an
enterprise-grade landing zone ready in under 10 weeks to support our core payments
engine. All payment processing must comply with DORA and PCI-DSS v4.0. We have
existing on-premises systems that require private, low-latency connectivity over a
10Gbps dedicated interconnect. We use Entra ID for corporate single sign-on and
GitHub Actions for our delivery pipelines. Security requires that all egress
traffic flows through centralised inspection appliances before touching the public
internet, and customer data must be encrypted with keys we control.
requirements:
- id: REQ-01
category: Timeline
statement: Platform foundation must be ready for workload ingestion within 10 weeks.
- id: REQ-02
category: Compliance
statement: Must achieve DORA resiliency alignment and PCI-DSS v4.0 perimeter isolation.
- id: REQ-03
category: Networking
statement: 10Gbps on-premises hybrid connectivity; centralised inspection appliances for egress.
- id: REQ-04
category: Identity and security
statement: Workload identity federation with GitHub Actions; Entra ID single sign-on; customer-managed encryption keys.
exclusions:
- Application container deployment; workloads are managed by application squads.
- An active-active secondary region in phase 1; warm standby is acceptable.
success_criteria: decide
estate:
cloud_provider: gcp
mode: greenfield
regions:
primary: europe-west2
secondary: europe-west1
resiliency_tier: warm_standby
data_residency: United Kingdom and European Union
existing_systems: On-premises payment systems that stay in place and connect over the interconnect.
workload:
users: not_applicable
availability:
target: decide
hours: always
recovery_time: decide
recovery_point: decide
compliance:
industry: financial_services
frameworks: [DORA, ISO27001, PCI_DSS_v4]
data_classification: confidential
encryption:
type: cmek
key_rotation_days: 90
audit_logging:
retention_days: 365
immutable_retention_policy: true
networking:
exposure: internal_only
topology: shared_vpc_hub_and_spoke
hybrid_connectivity:
enabled: true
type: cloud_interconnect
bandwidth_gbps: 10
egress_control:
model: centralized_appliance
appliance_vendor: decide
cidr_allocations: decide
governance:
identity_source_of_truth: Entra ID
pipeline_authentication: workload identity federation
environment_tiers:
- {name: development, archetype: corp_workload}
- {name: staging, archetype: corp_workload}
- {name: production, archetype: pci_regulated_workload}
operations:
operating_model: central platform team brokering access for application squads
support_hours: always
monitoring: decide
change_process: decide
integrations: decide
delivery:
window_weeks: 10
commercial_model: fixed_price_milestone
iac_tooling: terraform
ci_cd: GitHub Actions
team_size: 3
working_days_per_week: 5
cost:
monthly_budget: decide
currency: GBP
constraints:
products_required: decide
products_forbidden: decide
run:
pre_approved: true
allow_cloud_read: false
terraform_modules: false

View File

@@ -18,6 +18,8 @@ from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import yaml
from app.adk.runtime import ToolResult
from app.agent import Interaction
from app.tools.assessment import QUALITY_BAR, QUALITY_RUBRIC, checklist_question
@@ -73,6 +75,8 @@ class CaseSpec:
overlay_skills: tuple[str, ...] = ()
# Whether the case's Executions opt into published Terraform modules.
terraform_modules: bool = False
# A scope spec (ADR-0008), as YAML text, that the case's Execution starts from.
scope: str = ""
# What the infrastructure code must and must not contain, as regular
# expressions over all of it: how a case checks that standards from an overlay
# Skill reached the code. Each is {"name", "require"} or {"name", "forbid"}.
@@ -106,12 +110,27 @@ class CaseResult:
return self.score >= PASS_THRESHOLD
def _scope_of(path: Path, item: Mapping[str, Any]) -> str:
"""The text of the scope spec a case names, from the specs folder beside the
dataset; "" when it names none."""
name = item.get("scope")
return (path.parent / "specs" / str(name)).read_text("utf-8") if name else ""
def _request_of(item: Mapping[str, Any], scope: str) -> str:
"""A case's request: its own, or the statement of intent in its scope spec."""
if item.get("workflow_request"):
return str(item["workflow_request"])
spec = yaml.safe_load(scope) if scope else {}
return str(((spec or {}).get("intent") or {}).get("statement", "")).strip()
def load_cases(path: Path) -> list[CaseSpec]:
return [
CaseSpec(
id=item["id"],
name=item["name"],
workflow_request=item["workflow_request"],
workflow_request=_request_of(item, _scope_of(path, item)),
required_products=tuple(item["required_products"]),
scripted_answers=tuple(
tuple(pair) for pair in item.get("scripted_answers", [])
@@ -121,6 +140,7 @@ def load_cases(path: Path) -> list[CaseSpec]:
change=str(item.get("change", "")),
overlay_skills=tuple(item.get("overlay_skills", ())),
terraform_modules=bool(item.get("terraform_modules", False)),
scope=_scope_of(path, item),
code_standards=tuple(item.get("code_standards", ())),
)
for item in json.loads(path.read_text(encoding="utf-8"))

View File

@@ -133,6 +133,7 @@ async def run_with_scripted_user(
execution_id=execution_id,
overlay_skills=case.overlay_skills,
terraform_modules=case.terraform_modules,
scope=case.scope or None,
)
for _ in range(max_user_turns):
if interaction.status != "awaiting_user" or interaction.prompt is None:
@@ -196,6 +197,7 @@ async def run_case(
scripted_answers=case.scripted_answers,
overlay_skills=case.overlay_skills,
terraform_modules=case.terraform_modules,
scope=case.scope or None,
)
baseline_tokens = token_report(adk_runtime.usage)["total"].get("total")
baseline_seconds = round(time.monotonic() - started, 1)
@@ -211,6 +213,7 @@ async def run_case(
scripted_answers=case.scripted_answers,
overlay_skills=case.overlay_skills,
terraform_modules=case.terraform_modules,
scope=case.scope or None,
baseline=base.execution_id,
)
elif case.interactive:
@@ -227,6 +230,7 @@ async def run_case(
scripted_answers=case.scripted_answers,
overlay_skills=case.overlay_skills,
terraform_modules=case.terraform_modules,
scope=case.scope or None,
)
elapsed = time.monotonic() - started
write_progress()

View File

@@ -57,6 +57,17 @@ role_skills:
# once it holds lessons for that Specialist's Deliverable.
lessons_skill: google-cloud-lessons
# The scope spec (ADR-0008). `skill` holds its catalogue, which names no Cloud
# provider. `values` are what this provider allows for the fields the catalogue
# leaves to a provider, in this provider's own terms.
scope:
skill: scoping
values:
estate.cloud_provider: [gcp]
compliance.encryption.type: [cmek, cmek_hsm]
networking.topology: [single_vpc, shared_vpc, shared_vpc_hub_and_spoke]
networking.hybrid_connectivity.type: [cloud_vpn, cloud_interconnect, partner_interconnect]
# One advisor per architecture-framework pillar; each loads only its pillar's Skill.
pillar_skills:
security: google-cloud-waf-security

36
skills/scoping/SKILL.md Normal file
View File

@@ -0,0 +1,36 @@
---
name: scoping
description: >-
The scope spec: the structured statement of what a customer wants and the
facts a solution architecture depends on, with the catalogue of its fields.
Use when reading a scope spec in your context, to know what a field means,
whether the customer stated it or left it to you, and what binds your
Deliverable.
metadata:
scope: every Deliverable
---
# Scoping
A scope spec is a YAML file the customer's architect supplies, complete or partly filled. It sits beside the Intent, which stays in the customer's words. `assets/catalogue.yaml` lists every field it may hold: its type, when it must be settled, the question to ask for it, and guidance for a default.
## What a field says
| In the spec | Meaning |
| --- | --- |
| a value | The customer states this. It is theirs: use it as given. |
| `ask`, or left out | Not known yet. |
| `decide` | The customer leaves the choice to you. Choose, and record it as an assumption with what it affects. |
| `not_applicable` | It does not apply. Ask nothing and assume nothing. |
## Rules for every Specialist
- **A stated value binds you.** A region, a topology, a product, a setting or a number the spec states is the customer's decision. Honour it. If you believe it is wrong, say so with `needs_input`; do not quietly replace it.
- **A stated choice is not yours to generalise.** The rule that requirements name no products is about what you introduce. What the customer wrote stays as they wrote it.
- **Build nothing the customer excluded.** `intent.exclusions` is out of scope for every Deliverable.
- **The customer's own requirements carry their ids.** Each entry in `intent.requirements` must be met. When your work serves one, name its id.
- **Do not fill a gap silently.** A field under `to_ask` is unknown: ask for it when you may ask the user, and otherwise assume a default and say so. A field under `to_decide` is yours to choose: record the choice as an assumption.
## The kinds of engagement
`engagement.kind` is `workload` (an application), `platform_foundation` (a landing zone or shared platform that workloads will later run on) or `migration`. A platform foundation has no users or response times of its own: its subject is the organisation's structure, shared networking, identity, environments and guardrails.

View File

@@ -0,0 +1,318 @@
# The scope spec's catalogue: every field a scope spec may hold.
#
# A field is a dotted path. `type` is one of: string, text, integer, number, boolean,
# date, list, enum, records. `tier` says when it must be settled: essential (always,
# before any work), important (asked when someone is present) or optional. A tier
# may differ by the kind of engagement. `question` is what to ask the customer for
# it. `default` guides the choice when the customer says "decide". `affects` says
# what a wrong assumption would change.
#
# This catalogue names no Cloud provider. A provider's bundle adds allowed values
# for the fields marked `provider_values`.
schema_version: 1
kinds: [workload, platform_foundation, migration]
fields:
metadata.customer_case:
type: string
tier: essential
aliases: [metadata.engagement_id]
question: What short reference identifies this engagement?
metadata.engagement_reference:
type: string
metadata.client_name:
type: string
metadata.author:
type: string
metadata.revision:
type: string
engagement.kind:
type: enum
values: [workload, platform_foundation, migration]
tier: essential
question: >-
Is this an application workload, a platform foundation such as a landing zone,
or a migration of existing systems?
intent.statement:
type: text
tier: essential
question: In the customer's own words, what do they want?
intent.requirements:
type: records
keys: [id, statement]
optional_keys: [category]
question: Which requirements has the customer stated outright?
intent.exclusions:
type: list
tier: important
question: What has the customer said is out of scope?
default: Nothing is excluded beyond what the statement implies.
affects: what is designed and planned
intent.success_criteria:
type: list
question: How will the customer judge this a success?
default: The stated requirements are met.
estate.cloud_provider:
type: enum
provider_values: true
tier: essential
question: Which Cloud provider is this for?
estate.mode:
type: enum
values: [greenfield, brownfield_expansion, migration]
tier: essential
question: Is this a new estate, an extension of an existing one, or a migration?
estate.regions.primary:
type: string
tier: essential
question: Which region should it run in?
estate.regions.secondary:
type: string
question: Is there a second region, and which?
default: None, unless availability or recovery targets need one.
affects: resilience design and cost
estate.regions.resiliency_tier:
type: enum
values: [single_region, warm_standby, active_active]
tier: important
question: How should it survive the loss of a region?
default: single_region
affects: resilience design and cost
estate.data_residency:
type: string
tier: essential
question: Where must data stay?
estate.existing_systems:
type: text
tier: {migration: essential, platform_foundation: important, workload: important}
question: What exists today that this replaces, extends or must connect to?
default: Nothing exists today.
affects: connectivity, migration work and the plan
workload.users.who:
type: enum
values: [employees, customers, partners, public, systems]
tier: {workload: essential, migration: essential}
question: Who uses it?
workload.users.count:
type: integer
tier: {workload: essential, migration: essential}
question: How many users?
workload.users.peak_concurrent:
type: integer
tier: {workload: important, migration: important}
question: How many use it at once at peak?
default: A tenth of the user count.
affects: scaling limits and cost
workload.availability.target:
type: string
tier: {workload: essential, migration: essential, platform_foundation: important}
question: What availability is required, as a percentage?
default: "99.9%"
affects: resilience design and cost
workload.availability.hours:
type: enum
values: [business_hours, extended, always]
tier: important
question: During which hours must it be available?
default: always
affects: scaling to zero and support
workload.availability.recovery_time:
type: string
tier: important
question: How quickly must it recover from a failure?
default: 4 hours
affects: backup and standby design
workload.availability.recovery_point:
type: string
tier: important
question: How much recent data may be lost in a failure?
default: 1 hour
affects: backup frequency
compliance.industry:
type: string
question: "Which industry is the customer in?"
compliance.frameworks:
type: list
tier: essential
question: Which regulations or standards apply? An empty list if none.
compliance.data_classification:
type: enum
values: [public, internal, confidential, restricted]
tier: essential
question: How sensitive is the data?
compliance.encryption.type:
type: enum
values: [provider_managed, customer_managed]
provider_values: true
tier: important
question: Must the customer control the encryption keys?
default: provider_managed
affects: key management design
compliance.encryption.key_rotation_days:
type: integer
question: How often must keys rotate, in days?
default: 90
compliance.audit_logging.retention_days:
type: integer
tier: important
question: How long must audit logs be kept, in days?
default: 365
affects: storage cost and log design
compliance.audit_logging.immutable_retention_policy:
type: boolean
question: Must audit logs be undeletable for that time?
default: false
networking.exposure:
type: enum
values: [internal_only, internet_facing]
tier: important
question: Is it reached from the internet or only from inside?
default: internal_only
affects: perimeter design
networking.topology:
type: enum
provider_values: true
tier: {platform_foundation: important}
question: Is a network topology already decided?
default: The simplest topology that meets the requirements.
affects: the whole network design
networking.hybrid_connectivity.enabled:
type: boolean
tier: {platform_foundation: important, migration: important}
question: Must it connect privately to on-premises or another cloud?
default: false
affects: network design and lead time
networking.hybrid_connectivity.type:
type: enum
provider_values: true
question: By what kind of connection?
networking.hybrid_connectivity.bandwidth_gbps:
type: number
question: At what bandwidth, in Gbps?
networking.egress_control.model:
type: enum
values: [direct, provider_managed, centralized_appliance]
question: How must traffic to the internet be controlled?
default: provider_managed
affects: network design and cost
networking.egress_control.appliance_vendor:
type: string
question: Which inspection appliance, if one is mandated?
networking.cidr_allocations:
type: text
tier: {platform_foundation: important}
question: Which IP ranges are allocated, and how large per environment?
default: Private ranges that do not overlap, sized for growth.
affects: every network resource
governance.identity_source_of_truth:
type: string
tier: important
question: Which identity provider do people sign in with?
default: The Cloud provider's own identities.
affects: access design
governance.pipeline_authentication:
type: string
question: How do delivery pipelines authenticate?
default: Short-lived federated credentials, no stored keys.
governance.environment_tiers:
type: records
keys: [name]
optional_keys: [archetype]
tier: important
question: Which environments are needed?
default: development, test and production.
affects: the number of environments built and planned
operations.operating_model:
type: string
question: "Who runs the platform day to day?"
operations.support_hours:
type: enum
values: [business_hours, extended, always]
question: During which hours is it supported?
default: business_hours
operations.monitoring:
type: text
question: What monitoring and alerting is expected?
default: The Cloud provider's own monitoring, with alerts on availability and errors.
operations.change_process:
type: text
question: "How are changes approved and released?"
integrations:
type: records
keys: [name]
optional_keys: [direction, criticality]
question: Which other systems must it exchange data with?
default: None.
delivery.target_start_date:
type: date
question: "When should work start?"
delivery.window_weeks:
type: integer
tier: important
question: Is there a deadline, in weeks from the start?
default: No deadline; the plan gives the time the work needs.
affects: the plan and whether the scope fits
delivery.commercial_model:
type: enum
values: [time_and_materials, fixed_price_milestone, managed_service]
question: How is the work contracted?
delivery.iac_tooling:
type: string
question: "Which infrastructure-as-code tool?"
default: terraform
delivery.ci_cd:
type: string
question: "Which delivery pipeline tool?"
delivery.team_size:
type: integer
tier: important
question: How many engineers will build it?
default: 2
affects: the timeline
delivery.working_days_per_week:
type: integer
default: 5
cost.monthly_budget:
type: number
tier: {workload: essential, migration: essential, platform_foundation: important}
question: What is the monthly running budget?
default: No ceiling; cost is optimised within the requirements.
affects: product tiers and resilience choices
cost.currency:
type: string
default: "USD"
constraints.products_required:
type: list
question: Which products has the customer mandated?
default: None.
constraints.products_forbidden:
type: list
question: Which products are ruled out?
default: None.
constraints.other:
type: list
run.pre_approved:
type: boolean
default: false
run.allow_cloud_read:
type: boolean
default: false
run.overlay_skills:
type: list
default: []
run.terraform_modules:
type: boolean
default: false

View File

@@ -0,0 +1,198 @@
"""An Execution started from a scope spec (ADR-0008)."""
import json
import pytest
import yaml
from langgraph.checkpoint.memory import InMemorySaver
from app.adk.fake_runtime import FakeAgentRuntime, sample_submission
from app.adk.runtime import NoDecider, RoleSpec
from app.agent import ExecutionEngine
from app.nodes.router import Router, Thresholds
from app.providers import load_provider
from app.recorder import InMemoryRecorder
from app.workflows.deliverables import DEPENDENCY_GRAPH
from app.workflows.deliverables import DeliverableKind as K
ROLES = {
"orchestrator": RoleSpec("orchestrator", "Unused.", budget=40),
**{
spec.specialist: RoleSpec(
spec.specialist, "Produce.", budget=15, skills=("base",)
)
for spec in DEPENDENCY_GRAPH.values()
},
}
ANALYST = DEPENDENCY_GRAPH[K.REQUIREMENTS].specialist
SPEC = """
metadata: {engagement_id: ENG-77}
engagement: {kind: platform_foundation}
intent:
statement: Build us a landing zone.
exclusions: [application workloads]
estate:
cloud_provider: gcp
mode: greenfield
regions: {primary: europe-west2}
data_residency: United Kingdom
compliance:
frameworks: [DORA]
data_classification: confidential
networking: {topology: shared_vpc_hub_and_spoke, cidr_allocations: decide}
run: {pre_approved: true, terraform_modules: false}
"""
class Recording(FakeAgentRuntime):
def __init__(self, turns):
super().__init__(turns)
self.seen: dict[str, dict] = {}
async def run(self, role, task, tools):
self.seen[role.name] = {
"skills": role.skills,
"scope": task.context.get("scope"),
}
return await super().run(role, task, tools)
def engine(tmp_path) -> tuple[ExecutionEngine, Recording]:
bundle = load_provider("gcp")
runtime = Recording(
{
spec.specialist: [[sample_submission(kind)]]
for kind, spec in DEPENDENCY_GRAPH.items()
}
)
built = ExecutionEngine(
runtime=runtime,
roles=ROLES,
recorder=InMemoryRecorder(),
checkpointer=InMemorySaver(),
output_root=tmp_path,
orchestrator=Router(NoDecider(), None, Thresholds()),
scope_catalogue=bundle.scope_catalogue(),
scope_values=bundle.scope_values,
scope_skill=bundle.scope_skill,
)
return built, runtime
async def test_a_spec_alone_starts_an_execution_and_names_its_customer_case(tmp_path):
built, _ = engine(tmp_path)
result = await built.start("", scope=SPEC, execution_id="v1")
assert result.status == "completed"
state = result.state
# The spec said pre-approved, named the case, and gave the request.
assert state["pre_approved"] is True and state["customer_case"] == "eng-77"
assert state["workflow_request"] == "Build us a landing zone."
assert state["scope"]["status"] == "APPROVED"
async def test_the_supplied_file_is_kept_unaltered_beside_a_completed_copy(tmp_path):
built, _ = engine(tmp_path)
await built.start("", scope=SPEC, execution_id="v1")
intent = tmp_path / "deliverables" / "cases" / "eng-77" / "intent"
assert (intent / "001.scope.yaml").read_text() == SPEC
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
assert completed["metadata"]["status"] == "APPROVED"
assert completed["networking"]["cidr_allocations"] == "decide"
assert completed["provenance"]["estate.regions.primary"] == "supplied"
assert [a["field"] for a in completed["inferred_assumptions"]] == [
"networking.cidr_allocations"
]
history = json.loads((intent / "history.json").read_text())
assert history["intents"][0]["scope"] == {
"completed": "intent/001.scope.completed.yaml",
"supplied": "intent/001.scope.yaml",
}
manifest = json.loads(next(tmp_path.rglob("execution.json")).read_text())
assert manifest["scope"]["status"] == "APPROVED"
assert manifest["scope"]["readiness"]["assumed"] == 1
async def test_every_specialist_is_shown_the_spec_and_given_the_skill(tmp_path):
built, runtime = engine(tmp_path)
await built.start("", scope=SPEC, execution_id="v1")
for name, seen in runtime.seen.items():
assert seen["skills"] == ("base", "scoping"), name
assert seen["scope"]["kind"] == "platform_foundation"
stated = runtime.seen[ANALYST]["scope"]["stated"]
assert stated["networking"]["topology"] == "shared_vpc_hub_and_spoke"
assert stated["intent"]["exclusions"] == ["application workloads"]
assert runtime.seen[ANALYST]["scope"]["to_decide"][0]["field"] == (
"networking.cidr_allocations"
)
async def test_without_a_spec_nothing_changes(tmp_path):
built, runtime = engine(tmp_path)
result = await built.start("Ingest webhooks", pre_approved=True)
assert result.state["scope"] is None
assert runtime.seen[ANALYST] == {"skills": ("base",), "scope": None}
manifest = json.loads(next(tmp_path.rglob("execution.json")).read_text())
assert manifest["scope"] is None
assert not list(tmp_path.rglob("*.scope*.yaml"))
async def test_an_essential_gap_refuses_a_start_with_nobody_to_ask(tmp_path):
built, runtime = engine(tmp_path)
gappy = SPEC.replace(" data_residency: United Kingdom\n", "")
with pytest.raises(ValueError, match="estate.data_residency") as refused:
await built.start("", scope=gappy)
assert "nobody to ask" in str(refused.value)
assert runtime.seen == {} and not list(tmp_path.rglob("history.json"))
# With someone present the same spec starts, and the gap is theirs to answer.
waiting = await built.start("", scope=gappy, pre_approved=False, execution_id="v2")
assert waiting.state["scope"]["status"] == "DRAFT"
asked = waiting.state["scope"]["context"]["to_ask"]
assert asked[0] == {
"field": "estate.data_residency",
"prompt": "Where must data stay?",
"severity": "HIGH",
}
async def test_a_spec_that_cannot_be_read_refuses_the_start(tmp_path):
built, runtime = engine(tmp_path)
with pytest.raises(ValueError, match="networking.topolgy is not a field"):
await built.start("", scope=SPEC.replace("topology", "topolgy"))
with pytest.raises(ValueError, match="must be one of"):
await built.start(
"", scope=SPEC.replace("cloud_provider: gcp", "cloud_provider: aws")
)
with pytest.raises(ValueError, match="started for another-case"):
await built.start("", scope=SPEC, customer_case="another-case")
assert runtime.seen == {}
async def test_what_the_caller_says_outright_wins_over_the_specs_run_options(tmp_path):
built, _ = engine(tmp_path)
waiting = await built.start("", scope=SPEC, pre_approved=False, execution_id="v1")
assert waiting.state["pre_approved"] is False
assert waiting.status == "awaiting_user"
async def test_a_request_is_still_needed_when_the_spec_has_no_statement(tmp_path):
built, _ = engine(tmp_path)
with pytest.raises(ValueError, match="A request is needed"):
await built.start(
"",
scope=SPEC.replace(" statement: Build us a landing zone.\n", ""),
pre_approved=False,
)

290
tests/test_scope.py Normal file
View File

@@ -0,0 +1,290 @@
"""The scope spec (ADR-0008): read against the catalogue, with no model involved."""
from pathlib import Path
import pytest
import yaml
from app.providers import load_provider
from app.scope import (
APPROVED,
ASK,
DRAFT,
ESSENTIAL,
UNDER_REVIEW,
Catalogue,
FieldSpec,
ScopeError,
read_scope,
render_reading,
render_template,
slug,
)
CATALOGUE = Catalogue(
schema_version=1,
kinds=("workload", "platform_foundation"),
fields={
spec.path: spec
for spec in (
FieldSpec(
"metadata.customer_case",
tier=ESSENTIAL,
aliases=("metadata.engagement_id",),
),
FieldSpec(
"engagement.kind",
type="enum",
values=("workload", "platform_foundation"),
tier=ESSENTIAL,
),
FieldSpec("intent.statement", type="text", tier=ESSENTIAL),
FieldSpec(
"intent.requirements",
type="records",
keys=("id", "statement"),
optional_keys=("category",),
),
FieldSpec(
"users.count",
type="integer",
tier={"workload": ESSENTIAL},
question="How many users?",
),
FieldSpec(
"users.peak",
type="integer",
tier="important",
question="How many at once?",
default="a tenth of the users",
affects="scaling limits",
),
FieldSpec("estate.region", tier="important", question="Which region?"),
FieldSpec("network.topology", type="enum", provider_values=True),
FieldSpec("delivery.start", type="date"),
FieldSpec("run.pre_approved", type="boolean"),
)
},
)
COMPLETE = """
metadata: {customer_case: acme-bookings}
engagement: {kind: workload}
intent:
statement: Staff book rooms.
requirements:
- {id: REQ-01, statement: Rooms can be booked.}
users: {count: 400, peak: 40}
estate: {region: europe-west2}
"""
def read(text: str, **options):
return read_scope(text, CATALOGUE, **options)
def test_a_complete_spec_has_nothing_to_ask():
scope = read(COMPLETE)
assert scope.kind == "workload" and scope.customer_case == "acme-bookings"
assert scope.statement == "Staff book rooms."
assert scope.gaps(ESSENTIAL, "important") == []
assert scope.values["users.count"] == 400
assert (
scope.status() == UNDER_REVIEW and scope.status(pre_approved=True) == APPROVED
)
def test_a_field_left_out_or_marked_ask_is_a_gap_with_its_question():
scope = read(COMPLETE.replace("count: 400, peak: 40", "peak: ask"))
assert scope.gaps() == [
{"field": "users.count", "prompt": "How many users?", "severity": "HIGH"},
{"field": "users.peak", "prompt": "How many at once?", "severity": "MEDIUM"},
# Optional fields nobody filled are gaps too, the least pressing.
*[gap for gap in scope.gaps() if gap["severity"] == "LOW"],
]
assert scope.status(pre_approved=True) == DRAFT
def test_decide_is_an_assumption_and_not_applicable_is_neither():
scope = read(
COMPLETE.replace("peak: 40", "peak: decide").replace(
"estate: {region: europe-west2}", "estate: not_applicable"
)
)
(assumed,) = scope.assumptions()
assert assumed["id"] == "ASSUMP-01" and assumed["field"] == "users.peak"
assert "a tenth of the users" in assumed["statement"]
assert assumed["impact_if_invalid"] == "Changes scaling limits."
# A whole section marked at once covers every field in it.
assert scope.states["estate.region"] == "not_applicable"
assert all(gap["field"] != "estate.region" for gap in scope.gaps())
def test_which_fields_are_essential_depends_on_the_kind_of_engagement():
foundation = read(
COMPLETE.replace("workload", "platform_foundation").replace(
"users: {count: 400, peak: 40}", ""
)
)
assert foundation.gaps(ESSENTIAL) == []
assert foundation.tier("users.count") == "optional"
def test_an_unknown_field_is_named_never_ignored():
with pytest.raises(ScopeError) as refused:
read(COMPLETE + "userz: {count: 1}\nestate: {regoin: x}\n")
assert "userz is not a field of the scope spec" in refused.value.problems
# The second estate key replaced the first; its typo is reported too.
assert any("estate.regoin" in problem for problem in refused.value.problems)
@pytest.mark.parametrize(
("change", "fault"),
[
(("count: 400", "count: many"), "users.count: must be a whole number"),
(("kind: workload", "kind: holiday"), "must be one of workload"),
(
("{id: REQ-01, statement: Rooms can be booked.}", "{id: REQ-01}"),
"needs statement",
),
(("estate: {region: europe-west2}", "estate: europe"), "estate is a section"),
],
)
def test_a_value_of_the_wrong_kind_is_refused_with_the_reason(change, fault):
with pytest.raises(ScopeError) as refused:
read(COMPLETE.replace(*change))
assert any(fault in problem for problem in refused.value.problems)
def test_text_that_is_not_yaml_is_refused():
with pytest.raises(ScopeError, match="not valid YAML"):
read("intent: [unclosed")
with pytest.raises(ScopeError, match="must be a mapping"):
read("- just\n- a list\n")
def test_a_provider_supplies_the_values_the_catalogue_leaves_to_it():
spec = COMPLETE + "network: {topology: hub_and_spoke}\n"
with pytest.raises(ScopeError):
read(spec, provider_values={"network.topology": ["flat"]})
allowed = read(spec, provider_values={"network.topology": ["hub_and_spoke"]})
assert allowed.values["network.topology"] == "hub_and_spoke"
def test_an_engagement_reference_and_a_customer_case_name_the_same_thing():
by_engagement = read(
COMPLETE.replace(
"customer_case: acme-bookings", "engagement_id: ENG-2026-UK-084"
)
)
assert by_engagement.customer_case == "eng-2026-uk-084"
assert slug("ENG 2026/UK 084") == "eng-2026-uk-084"
with pytest.raises(ScopeError, match="name the same thing and differ"):
read(
COMPLETE.replace(
"{customer_case: acme-bookings}", "{customer_case: a, engagement_id: b}"
)
)
def test_the_completed_copy_holds_every_field_and_the_agents_sections():
scope = read(
COMPLETE.replace("peak: 40", "peak: decide") + "delivery: {start: 2026-11-01}\n"
)
completed = scope.completed(DRAFT)
assert completed["metadata"]["status"] == DRAFT
assert completed["users"] == {"count": 400, "peak": "decide"}
assert completed["delivery"]["start"] == "2026-11-01" # plain data, not a date
assert completed["network"]["topology"] == ASK
assert completed["provenance"]["users.count"] == "supplied"
assert completed["provenance"]["users.peak"] == "assumed"
assert completed["provenance"]["network.topology"] == "unknown"
assert completed["readiness"]["supplied"] == len(scope.values)
assert completed["inferred_assumptions"][0]["field"] == "users.peak"
assert yaml.safe_load(yaml.safe_dump(completed)) == completed
def test_a_completed_copy_can_be_read_back_as_a_spec():
"""The agent's own sections are replaced, not rejected, on the way back in."""
scope = read(COMPLETE.replace("peak: 40", "peak: decide"))
again = read(yaml.safe_dump(scope.completed(DRAFT)))
assert again.values == scope.values and again.states == scope.states
def test_a_specialist_is_shown_what_is_stated_and_what_is_open():
scope = read(
COMPLETE.replace("peak: 40", "peak: decide") + "run: {pre_approved: true}\n"
)
context = scope.for_context()
assert context["kind"] == "workload"
assert context["stated"]["users"] == {"count": 400}
assert context["stated"]["intent"]["requirements"][0]["id"] == "REQ-01"
assert context["to_decide"][0]["field"] == "users.peak"
# How to run the agent, and the case's name, are not the Specialists' business.
assert "run" not in context["stated"] and "metadata" not in context["stated"]
def test_a_template_and_a_reading_are_written_for_a_person():
template = yaml.safe_load(render_template(CATALOGUE, "workload"))
reading = render_reading(read(COMPLETE.replace("count: 400, peak: 40", "peak: 40")))
assert template["engagement"]["kind"] == "workload"
assert template["users"] == {"count": ASK, "peak": ASK}
assert "delivery" not in template # optional fields are left out of a template
assert "Essential, needed before work can start:" in reading
assert "users.count: How many users?" in reading
# --- the real catalogue and the benchmark spec ------------------------------------------
def test_the_gcp_bundle_reads_the_benchmark_landing_zone_spec():
bundle = load_provider("gcp")
text = (
Path(__file__).resolve().parent.parent
/ "eval"
/ "datasets"
/ "specs"
/ "landing-zone.yaml"
).read_text()
scope = read_scope(
text, bundle.scope_catalogue(), provider_values=bundle.scope_values
)
assert scope.kind == "platform_foundation"
assert scope.customer_case == "case-08-landing-zone"
# Complete enough to start with nobody to ask.
assert scope.gaps(ESSENTIAL) == []
assert scope.values["networking.topology"] == "shared_vpc_hub_and_spoke"
assert scope.states["workload.users.count"] == "not_applicable"
assert [item["id"] for item in scope.values["intent.requirements"]] == [
"REQ-01",
"REQ-02",
"REQ-03",
"REQ-04",
]
def test_every_essential_field_of_the_catalogue_has_a_question_to_ask():
catalogue = load_provider("gcp").scope_catalogue()
for kind in catalogue.kinds:
for path, spec in catalogue.fields.items():
if spec.tier_for(kind) != "optional":
assert spec.question, (
f"{path} is {spec.tier_for(kind)} and has no question"
)