Compare commits
6 Commits
feat/solut
...
feat/scopi
| Author | SHA1 | Date | |
|---|---|---|---|
| b66d18fe94 | |||
| f54af24169 | |||
| 0f80cb642f | |||
| c26b744b26 | |||
| 545097530b | |||
| afe78640f8 |
@@ -14,6 +14,8 @@ One directory per iteration of this agent. Each holds the plan for that iteratio
|
|||||||
| 6 | `iter-06-target-architecture-graph/` | complete: all four cases pass with real models, each run twice on 2026-10-08; thresholds set | The Target architecture as one explicit graph of Elements, Relationships and Checks, established before infrastructure code, with validation as a verdict per Check. |
|
| 6 | `iter-06-target-architecture-graph/` | complete: all four cases pass with real models, each run twice on 2026-10-08; thresholds set | The Target architecture as one explicit graph of Elements, Relationships and Checks, established before infrastructure code, with validation as a verdict per Check. |
|
||||||
| 7 | `iter-07-change-requests/` | complete (2026-10-09): the change case ran with real models and passed | A user restates a request to say how requirements changed; a new Execution starts from a Baseline Execution of the same Customer case and delivers the updated architecture with what changed. |
|
| 7 | `iter-07-change-requests/` | complete (2026-10-09): the change case ran with real models and passed | A user restates a request to say how requirements changed; a new Execution starts from a Baseline Execution of the same Customer case and delivers the updated architecture with what changed. |
|
||||||
| 8 | `iter-08-terraform-modules/` | steps 1–5 built and run once each way on 2026-10-09, both at 100%; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. |
|
| 8 | `iter-08-terraform-modules/` | steps 1–5 built and run once each way on 2026-10-09, both at 100%; steps 6 and 7 wait on Azure and AWS provider bundles | Infrastructure code is plain resources by default. An Execution may opt into published Terraform modules: only from registry namespaces the provider bundle lists, at one exact version, with a lookup of what a module accepts and a Skill of four Google modules tested against the pinned provider. |
|
||||||
|
| 9 | `iter-09-scoping/` | steps 1–3 and 6 built on 2026-10-11; cases 03 and 08 ran on steps 1 and 2; steps 4 and 5 planned; ADR-0008 accepted | An engagement is scoped from a scope spec: a structured file, complete or partly filled, read against a catalogue in code, completed in conversation and approved before requirements begin. Covers workloads and platform foundations. |
|
||||||
|
| 10 | `iter-10-work-breakdown/` | steps 1–4 built on 2026-10-11, without milestones; not run with real models | A work breakdown structure as a Deliverable: work packages traced to Elements and Checks, activities in sequence, and a timeline on a relative day axis computed and drawn by code. |
|
||||||
|
|
||||||
Numbers record when an iteration was planned, not the order to build in. Build iteration 6 before the remaining steps of iteration 4 (10 to 14) and before iteration 5: it changes the Deliverables, Phases and validation format they build on. Its `MAP.md` lists the effect on each of their steps.
|
Numbers record when an iteration was planned, not the order to build in. Build iteration 6 before the remaining steps of iteration 4 (10 to 14) and before iteration 5: it changes the Deliverables, Phases and validation format they build on. Its `MAP.md` lists the effect on each of their steps.
|
||||||
|
|
||||||
|
|||||||
34
.agents/plans/iter-09-scoping/MAP.md
Normal file
34
.agents/plans/iter-09-scoping/MAP.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
# Plan Map
|
||||||
|
|
||||||
|
**Iteration:** 9, Scoping
|
||||||
|
**Intent:** Scope an engagement from a scope spec: a structured file that may arrive complete, partly filled or not at all, completed in conversation and approved by the customer before requirements begin. So that a design's quality no longer rests on what the agents had to assume.
|
||||||
|
**Status:** steps 1, 2, 3 and 6 built on 2026-10-11; steps 1 and 2 ran in cases 03 and 08; step 3 not run; steps 4 and 5 planned
|
||||||
|
**This directory covers:** steps 1 to 6
|
||||||
|
|
||||||
|
## Sources of truth
|
||||||
|
|
||||||
|
- `docs/adr/0008-scoping-starts-from-a-scope-spec.md`: the decisions.
|
||||||
|
- `.scratch/scoping-and-work-breakdown/spec.md`: the design, including the draft spec, the owner's own example and how the two were combined.
|
||||||
|
- `skills/scoping/assets/catalogue.yaml`: every field a scope spec may hold.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
| Step | What | Status |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | The catalogue, reading and validating a spec, the gap list, the two files beside the Intent, run options from the spec, the spec shown to every Specialist | built 2026-10-11 (`steps/step-1.md`); not run with real models |
|
||||||
|
| 2 | The conversation that fills gaps and writes answers into the completed copy; scoping as a Phase with its Deliverable and Approval; a spec drafted from a sentence | built 2026-10-11 (`steps/step-2.md`); not run with real models |
|
||||||
|
| 3 | Requirements traced to the customer's requirement ids and to spec fields, in `requirements.json`, with a stated choice refused unless a requirement carries it; readiness in the solution guide | built 2026-10-11 (`steps/step-3.md`); not run with real models |
|
||||||
|
| 4 | An edited spec as a Change request, compared field by field | planned |
|
||||||
|
| 5 | Evaluation cases as specs, replacing answers matched by pattern | planned; `case-08-landing-zone` is the first |
|
||||||
|
| 6 | The status `BASELINE_LOCKED` set when the Execution completes | built with step 2 |
|
||||||
|
|
||||||
|
## Dependency graph
|
||||||
|
|
||||||
|
- Step 1: no dependencies
|
||||||
|
- Step 2: depends on step 1
|
||||||
|
- Step 3: depends on step 1
|
||||||
|
- Step 4: depends on step 1, iteration 7
|
||||||
|
- Step 5: depends on step 1
|
||||||
|
- Step 6: depends on step 2
|
||||||
|
|
||||||
|
The work breakdown structure is iteration 10 and uses the spec's `delivery` section from step 1.
|
||||||
20
.agents/plans/iter-09-scoping/steps/step-1.md
Normal file
20
.agents/plans/iter-09-scoping/steps/step-1.md
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
# Step 1 — Implementation Detail
|
||||||
|
|
||||||
|
Built on 2026-10-11. **Not run with real models.**
|
||||||
|
|
||||||
|
1. **The catalogue** is `skills/scoping/assets/catalogue.yaml`: 63 fields as dotted paths, each with a type, a tier (which may differ by kind of engagement), the question to ask and guidance for a default. It names no Cloud provider. `providers/gcp.yaml` adds, under `scope.values`, what Google Cloud allows for the four fields left to a provider: the provider itself, encryption type, network topology and the kind of hybrid connection.
|
||||||
|
2. **Reading a spec** is `read_scope` in `app/scope.py`, with no model. It refuses, naming every problem: YAML that does not parse, a field the catalogue does not have, a value of the wrong type, and an engagement reference and a customer case that disagree. A whole section may be marked `ask`, `decide` or `not_applicable` at once.
|
||||||
|
3. **What it yields:** the stated values; the gaps, most pressing first, each with its question and a severity (HIGH essential, MEDIUM important, LOW optional); the assumptions, one per `decide`, with the default to take and what a wrong one would change; a count of each; and a status.
|
||||||
|
4. **Starting from a spec:** `ExecutionEngine.start(scope=...)`, `scope` in the `/generate` body and in A2A metadata, and on an evaluation case. The spec's statement is the request when none is given, its engagement reference names the Customer case, and its `run` section applies where the caller gave nothing. A pre-approved start is refused when an essential field is not settled.
|
||||||
|
5. **The two files** are written beside the Intent: `NNN.scope.yaml`, the spec as supplied, byte for byte; and `NNN.scope.completed.yaml`, every field of the catalogue with the agent's sections (`inferred_assumptions`, `discovery_gaps`, `provenance`, `readiness`) and `metadata.status`. `history.json` names both. A completed copy can be supplied again as a spec; the agent's sections are replaced.
|
||||||
|
6. **Every Specialist is shown the spec** as `scope` in its context (what is stated, what is theirs to decide, what is unknown) and is given the scoping Skill. The protocol says a stated choice binds, an exclusion is out of scope, and the customer's requirement ids are to be named. The requirements analyst is told to derive from it and ask only for what it leaves unknown.
|
||||||
|
7. **For a person:** `python -m app.scope check <file>` says what a spec settles and what it lacks; `python -m app.scope template <kind>` prints one to fill in.
|
||||||
|
8. **A benchmark spec:** `eval/datasets/specs/landing-zone.yaml`, a fictional regulated payment platform, and `case-08-landing-zone` which starts from it.
|
||||||
|
|
||||||
|
Interim, until step 2 (now built; see `step-2.md`):
|
||||||
|
|
||||||
|
- **Gaps are asked by the requirements analyst,** who is handed them, not by a scoping Phase. Answers are not written back into the completed copy.
|
||||||
|
- **Status** is set once, at the start: `DRAFT` with gaps to ask, `UNDER_REVIEW` when complete and someone is present, `APPROVED` when pre-approved. `BASELINE_LOCKED` is not set.
|
||||||
|
- **A caller's `pre_approved` wins over the spec's** when both are given. `/generate` is always pre-approved.
|
||||||
|
|
||||||
|
Tests: `tests/test_scope.py` (18), `tests/execution/test_scope_spec.py` (8).
|
||||||
55
.agents/plans/iter-09-scoping/steps/step-2.md
Normal file
55
.agents/plans/iter-09-scoping/steps/step-2.md
Normal file
@@ -0,0 +1,55 @@
|
|||||||
|
# Step 2 — Implementation Detail
|
||||||
|
|
||||||
|
Built on 2026-10-11. **Not run with real models.**
|
||||||
|
|
||||||
|
1. **Scoping is the first Phase.** `PhaseId.SCOPING` comes before requirements discovery, with one Deliverable, `scope`, produced by a new Specialist, the scope analyst (`agents/scope-analyst.md`). Requirements depend on it. Every Execution now begins with it, whether or not a spec was supplied, and the customer approves it before requirements begin.
|
||||||
|
2. **The scope analyst completes the spec with two tools** (`app/tools/scoping.py`). `scope_read` shows what is settled and what is not, with the question to ask for each. `scope_set` records fields, each with where its value came from: `request` (the customer's request says it), `answered` (they said so in conversation) or `assumed` (the analyst chose, and must say why). Code checks every value against the catalogue as it is recorded.
|
||||||
|
3. **What the customer stated in the file stays.** The analyst cannot overwrite a supplied value by assumption; only the customer's own answer changes it.
|
||||||
|
4. **The conversation.** With someone present the analyst asks, through `needs_input`, for the essential fields and the important ones that most change a design, up to eight at a time. An answer is recorded as answered; "you decide" becomes an assumption; a skipped field stays unknown. With nobody present it assumes the essential and important fields from the catalogue's guidance and what the request implies.
|
||||||
|
5. **A spec drafted from a sentence.** With no spec supplied the analyst starts from an empty one, records what the request states and settles the rest the same way.
|
||||||
|
6. **A submission cannot leave an essential field unknown.** It is refused, naming the fields.
|
||||||
|
7. **The Deliverable is written by code from what was recorded.** The analyst writes the scope statement. Code writes the two files (`scope.yaml`, the completed spec; `scope.json`, what every later Specialist reads) and appends the information register, the assumptions with what each would change, and what is still to confirm.
|
||||||
|
8. **Every later Specialist reads the agreed scope**, as `scope` in its context: what the customer stated, what was assumed for them (shown apart), and what is unknown. The requirements analyst derives from it and no longer asks for what it settles.
|
||||||
|
9. **Status is kept current** in the Execution's `scope.yaml` and in the copy beside the Intent: `UNDER_REVIEW` once submitted, `APPROVED` when the Phase is approved, `BASELINE_LOCKED` when the Execution completes (which closes step 6). `execution.json` carries the status and the readiness count.
|
||||||
|
10. **A Change request starts from the scope that was delivered,** with where each value came from, and the analyst records only what the change reaches. A Baseline delivered before scoping existed has no scope; the analyst then completes one from the change and the delivered requirements.
|
||||||
|
11. **The Approval summary** for scoping says how much was assumed and what is unknown.
|
||||||
|
|
||||||
|
What it changed elsewhere:
|
||||||
|
|
||||||
|
- About 80 existing tests assumed requirements came first. `tests/scoped.py` holds helpers that run scoping ahead of a test's own script, so tests about later Phases start where they always did. Tests about order, counts and first prompts were updated to the new truth.
|
||||||
|
- `workflow.yaml` gains the Phase.
|
||||||
|
|
||||||
|
Not done here:
|
||||||
|
|
||||||
|
- **No real model has run the scope analyst.** How well it drafts from a sentence, how many questions it asks, and what it costs are unknown. Every evaluation case now begins with it.
|
||||||
|
- **Answers are parsed by the model**, not by code, before `scope_set` checks their type. Nothing reads a value back to the customer other than the draft they approve.
|
||||||
|
- A re-supplied file part-way through a conversation is not handled; the customer answers in chat.
|
||||||
|
|
||||||
|
Tests: `tests/execution/test_scope_spec.py` (12), and the existing suites as updated.
|
||||||
|
|
||||||
|
## First runs with real models, 2026-10-11
|
||||||
|
|
||||||
|
Two cases side by side (`20261011T172837Z` case 03, `20261011T172840Z` case 08). Both completed.
|
||||||
|
|
||||||
|
| | Case 03, a one-sentence request | Case 08, the landing-zone spec |
|
||||||
|
|---|---|---|
|
||||||
|
| Score | 100% | 91%; the one failure was the evaluation's, now fixed |
|
||||||
|
| Minutes | 38 | 55 |
|
||||||
|
| Cost, about | $5.42 | $8.76 |
|
||||||
|
| Revisions | 1 of the Target architecture | 2 of the Target architecture, 1 of the code |
|
||||||
|
| The scope analyst | 64 seconds, 12 requests, $0.07 | 96 seconds, 9 requests, $0.08 |
|
||||||
|
| The scope | 6 stated, 21 assumed | 44 stated, 11 assumed, 4 not applicable |
|
||||||
|
| The design | 22 Elements, 81 Checks | 49 Elements, 101 Checks, 718 lines of Terraform |
|
||||||
|
|
||||||
|
**Scoping works and is cheap:** about a minute and under ten cents. From one sentence the analyst recorded the six things the sentence says as the customer's and 21 as assumptions, each with a reason; nothing was presented as stated that was not. No essential field was left unknown in either.
|
||||||
|
|
||||||
|
**The landing zone, the agent's first platform foundation, came out as one:** folders, organisation policies, a service perimeter, an interconnect with four attachments, routers, customer-managed keys, workload identity federation for the pipelines, a workforce pool for sign-in, audit sinks. The customer's four requirement ids are each cited in the requirements. Nothing excluded was built: no container workloads. Regions and encryption are as stated. Sixteen questions for the customer were written out.
|
||||||
|
|
||||||
|
**What it shows is missing:**
|
||||||
|
|
||||||
|
- **A stated choice is not checked.** The spec says `shared_vpc_hub_and_spoke`. The design names a "Spoke Shared VPC" and builds a transit network and a spoke network, with no Shared VPC host project and nothing connecting hub to spoke. All 101 Checks passed, because no Check asks for the topology the customer stated. "A stated choice binds" is an instruction; nothing in code holds a design to it. This is what step 3 has to close: a stated field that shapes the design must be traceable to a Check.
|
||||||
|
- **The code brought in a provider the bundle does not pin.** A `github_repository` Element led to `integrations/github` in the Terraform. Validation downloads whatever provider the code names, as it once did modules.
|
||||||
|
- **The scope statement added an exclusion the customer did not state** (changes to on-premises systems). The analyst is told to use only what the spec holds.
|
||||||
|
- **The evaluation scored a stated choice as a leaked product.** "Shared VPC" in the requirements failed `requirements_name_no_products`, though the customer's spec names that topology. The check now counts what the spec states as the customer's.
|
||||||
|
|
||||||
|
Case 03's cost and its one Revision are in the range seen before scoping existed; one run cannot say what scoping changed downstream.
|
||||||
40
.agents/plans/iter-09-scoping/steps/step-3.md
Normal file
40
.agents/plans/iter-09-scoping/steps/step-3.md
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
# Step 3 — Implementation Detail
|
||||||
|
|
||||||
|
Built on 2026-10-11. **Not run with real models.**
|
||||||
|
|
||||||
|
## What a stated choice now has to reach
|
||||||
|
|
||||||
|
Case 08 showed a stated topology named in a design and not built, with every Check passed. A stated choice was an instruction; nothing held a design to it.
|
||||||
|
|
||||||
|
1. **The catalogue marks the fields that bind a design** (`binds: true`, 26 of them): regions and residency, availability and recovery targets, compliance frameworks and classification, encryption, audit logging, every networking field, identity, environments, and required and forbidden products.
|
||||||
|
2. **The agreed scope lists the binding fields the customer stated,** as `binding` in `scope.json`. A field assumed for them is not among them: an assumption is not the customer's decision.
|
||||||
|
3. **A requirement says what it comes from,** in `from`: the customer's own requirement ids and the scope fields it carries.
|
||||||
|
4. **A submission of requirements is refused** (`_carries_the_scope` in `app/workflows/chain.py`) when:
|
||||||
|
- one of the customer's own requirements is named by no requirement;
|
||||||
|
- a binding field is named by no requirement;
|
||||||
|
- a requirement that carries a binding field is not non-functional, or has no entry in `conditions`.
|
||||||
|
5. **From there the existing chain does the rest.** A non-functional requirement's conditions each need a Check on the Target architecture, and validation gives each Check a verdict. So a stated choice now reaches a Check, in code, at every link.
|
||||||
|
|
||||||
|
What this does not prove: that the condition the analyst writes says what the customer meant, or that the Check the architect writes verifies it. It guarantees a stated choice cannot be silently dropped; whether it is honoured well is still judged by the validator.
|
||||||
|
|
||||||
|
## Readiness in the guide
|
||||||
|
|
||||||
|
The packager is told to open the executive summary with what the design rests on: how much the customer stated, how much was assumed, how much is unknown. An instruction, not a check.
|
||||||
|
|
||||||
|
## Also closed, from the same run
|
||||||
|
|
||||||
|
- **Only the pinned providers.** `check_providers` in `app/tools/local.py` refuses, before Terraform runs, a resource or data source of a provider the bundle does not pin, and a `required_providers` entry for one. The target architect is told that something outside the Cloud provider's own provider is not an Element. A module's own providers are untouched: they are declared inside the module.
|
||||||
|
- **The scope statement names only the customer's exclusions.**
|
||||||
|
|
||||||
|
Not done: `intent.exclusions` are free text, so nothing in code checks that an excluded thing was not built. The Specialists are told; in case 08 they complied.
|
||||||
|
|
||||||
|
Tests: `tests/execution/test_requirements_chain.py` (6 more), `tests/tools/test_local.py` (5 more).
|
||||||
|
|
||||||
|
## First run with real models, 2026-10-11
|
||||||
|
|
||||||
|
`case-08-landing-zone` again, results `20261011T185421Z`: completed, 91%, 60 minutes, about $11.94.
|
||||||
|
|
||||||
|
- **The stated topology was built this time.** `networking.topology` is carried by NFR-2, whose first condition is that a Shared VPC host project is connected to service projects across the environment tiers. The code has a host project and the connections. The run before had neither and passed every Check.
|
||||||
|
- **Only the pinned provider is in the code.** The run before brought in the GitHub provider.
|
||||||
|
- **It cost more:** $11.94 against $8.76, with three Revisions of the Target architecture and three of the code, against two and one. Two of them were two Checks that contradicted each other on one key's role, sending the code back and forth before the design was corrected. That is the design's fault, not this step's, but a stricter chain gives a large design more to get wrong.
|
||||||
|
- **One evaluation check failed, and it is a real finding:** the requirements name "Cloud KMS". The customer asked for keys they control; the product was the analyst's addition.
|
||||||
23
.agents/plans/iter-10-work-breakdown/MAP.md
Normal file
23
.agents/plans/iter-10-work-breakdown/MAP.md
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
# Plan Map
|
||||||
|
|
||||||
|
**Iteration:** 10, Work breakdown structure
|
||||||
|
**Intent:** Deliver a plan for building the Target architecture: work packages traced to its Elements and Checks, activities in sequence, and a timeline on a relative day axis that code computes and draws.
|
||||||
|
**Status:** steps 1 to 4 built on 2026-10-11, without milestones; not run with real models
|
||||||
|
**This directory covers:** steps 1 to 4
|
||||||
|
|
||||||
|
## Sources of truth
|
||||||
|
|
||||||
|
- `.scratch/scoping-and-work-breakdown/spec.md`, section "The work breakdown structure".
|
||||||
|
- `docs/adr/0005-the-target-architecture-is-an-explicit-graph.md`: the ids it traces to.
|
||||||
|
- Iteration 9 step 1: the scope spec's `delivery` section (team size, window, commercial model).
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
| Step | What | Status |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | The Deliverable, its Specialist and its data file; every Element in a work package, checked in code | built 2026-10-11 (`steps/step-1-to-4.md`) |
|
||||||
|
| 2 | The schedule computed in code: start and finish day, critical path, team size as a limit | built 2026-10-11 (`steps/step-1-to-4.md`) |
|
||||||
|
| 3 | The three views written by code: WBS dictionary, activity sequence list, timeline on a relative axis | built 2026-10-11 (`steps/step-1-to-4.md`) |
|
||||||
|
| 4 | The delivery window checked; assumptions the plan depends on shown on the timeline; milestones | built 2026-10-11, without milestones (`steps/step-1-to-4.md`) |
|
||||||
|
|
||||||
|
Decisions the owner accepted on 2026-10-11: after validation, in the packaging Phase; grouped by component; the technical build only.
|
||||||
43
.agents/plans/iter-10-work-breakdown/steps/step-1-to-4.md
Normal file
43
.agents/plans/iter-10-work-breakdown/steps/step-1-to-4.md
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
# Steps 1 to 4 — Implementation Detail
|
||||||
|
|
||||||
|
Built on 2026-10-11. **Not run with real models.**
|
||||||
|
|
||||||
|
1. **A new Deliverable, `work_breakdown`,** in the packaging Phase, after validation, produced by a new Specialist, the delivery planner (`agents/delivery-planner.md`, on the larger model). It is given the Target architecture and the deployment guidance in full.
|
||||||
|
2. **The planner writes the plan as data,** `work-breakdown.json`: work packages (a hierarchy, each naming the Elements it delivers, its scope, what it produces and the skills it needs) and activities (work package, days, what each comes after, and whether the team or the customer owns it).
|
||||||
|
3. **Code checks the plan against the design** (`check_work_breakdown` in `app/workflows/work_breakdown.py`). A submission is refused when an Element belongs to no work package, a work package names an Element that does not exist, an activity follows one that does not exist, activities depend on each other in a circle, a duration is not above zero or is over 120 days, or a work package holds neither an activity nor another work package.
|
||||||
|
4. **Code computes the schedule** (`schedule`). An activity starts when everything it follows has finished. The team's activities also wait for a free person: no more run at once than the team has people, with the longest remaining chain placed first. What the customer owns takes nobody from the team. The critical path is every activity with no slack.
|
||||||
|
5. **Code writes the three views** into the Deliverable when it is submitted, as it writes the architecture's tables:
|
||||||
|
- the **WBS dictionary**, each work package with the Checks on its Elements as what makes it done, read from the Target architecture;
|
||||||
|
- the **activity sequence**, with each activity's start and finish day and whether it is critical;
|
||||||
|
- the **timeline**, a chart in text along a relative axis from Day 0, the team's work and the customer's drawn differently and the critical path marked.
|
||||||
|
6. **The delivery terms come from the agreed scope:** team size, working week, delivery window and start date. A team size the scope does not give is 2, and the Deliverable says it was assumed.
|
||||||
|
7. **The plan is checked against the delivery window.** Over it, the Deliverable says by how much and names the critical path. Nothing shortens an estimate to fit.
|
||||||
|
8. **What the customer owns is listed** with its lead time and the assumption behind it.
|
||||||
|
9. **With a start date in the scope** the activity sequence also gives calendar dates, skipping weekends, and a Mermaid Gantt chart is attached as `timeline.mmd`. Without one there are no dates to draw, and the text chart is the timeline.
|
||||||
|
10. **The guide carries the timeline.** The guide and the plan are produced side by side, so neither reads the other; when an Execution's files are written, the plan's timeline is added to the guide as "Delivery plan". `execution.json` records the plan's length, team, whether it fits the window and its critical path, and the packaging Approval says the same.
|
||||||
|
|
||||||
|
Differs from the spec:
|
||||||
|
|
||||||
|
- **Milestones for fixed-price work are not built.**
|
||||||
|
- **An assumption in the scope is not linked to an activity by id.** An activity the customer owns carries its own assumption as text.
|
||||||
|
- **The timeline chart is text.** A Mermaid Gantt chart needs dates, so one is drawn only when the scope gives a start date.
|
||||||
|
- **The guide does not depend on the plan.** Making it wait would have put the plan on the path of every Execution's last step; adding the timeline in code keeps them independent.
|
||||||
|
|
||||||
|
Durations are a model's estimates. The Deliverable says so.
|
||||||
|
|
||||||
|
Tests: `tests/execution/test_work_breakdown.py` (16).
|
||||||
|
|
||||||
|
## First run with real models, 2026-10-11
|
||||||
|
|
||||||
|
`case-08-landing-zone`, results `20261011T185421Z`. The delivery planner made one model request, about $0.32.
|
||||||
|
|
||||||
|
- **The plan:** 9 work packages under one parent and 15 activities. Every one of the design's Elements is in a work package, and each work package lists the Checks that make it done.
|
||||||
|
- **The schedule:** 36 working days, 7.2 weeks, with the team of 3 the scope states. Inside the 10-week window with 2.8 weeks to spare.
|
||||||
|
- **The critical path is the customer's:** ordering the interconnect circuits, 30 days, then the routers and attachments, then validation in the three environments the scope names. The team's own work is done by about Day 13 and waits.
|
||||||
|
- **The timeline was added to the solution guide** and its numbers are in `execution.json`.
|
||||||
|
|
||||||
|
To look at:
|
||||||
|
|
||||||
|
- **The estimates look short for an enterprise landing zone.** Two days for the hub and spoke networks, one for the pipeline and state, two for the resource hierarchy. The planner was told to estimate for one engineer who knows the provider and not to pad, and it did. Whether to ask for a contingency, or a range, is the owner's call.
|
||||||
|
- **The plan's length rests on one assumed number,** the 30-day carrier lead time, which the planner chose. It is shown as the customer's and as an assumption.
|
||||||
|
- No start date was in the spec, so there are no calendar dates and no Mermaid chart.
|
||||||
364
.scratch/scoping-and-work-breakdown/spec.md
Normal file
364
.scratch/scoping-and-work-breakdown/spec.md
Normal file
@@ -0,0 +1,364 @@
|
|||||||
|
# Spec: Scoping from a scope spec, and a work breakdown structure
|
||||||
|
|
||||||
|
Status: ready-for-agent
|
||||||
|
|
||||||
|
Accepted by the owner on 2026-10-11. Stage 1 is built; `.agents/plans/iter-09-scoping/` says what is done.
|
||||||
|
|
||||||
|
- Glossary: [`CONTEXT.md`](../../CONTEXT.md). New terms this would add are in **Language** below.
|
||||||
|
- Builds on: [ADR-0005](../../docs/adr/0005-the-target-architecture-is-an-explicit-graph.md) (ids along the Requirements chain), [ADR-0006](../../docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md) (Change requests), [ADR-0002](../../docs/adr/0002-skills-are-the-portable-unit-of-behaviour.md) (behaviour in Skills).
|
||||||
|
- Would need: one new ADR, for a new first Phase and a new start to the Requirements chain.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The Intent is the customer's request, saved word for word. For case 06 it is one sentence. Everything else a design needs (how many users, what availability, which region, what budget, which compliance regime, what exists today) is assumed by the requirements analyst or, in the evaluation, supplied by pattern-matched answers.
|
||||||
|
|
||||||
|
Three days of runs trace most downstream trouble to what nobody stated: Checks the validator asks for on topics the customer never raised, checklist items the request cannot settle, a "low-carbon region" no code can show. Nothing tells the customer at the start how much of their design rests on assumption, and a requirement does not record whether the customer stated it or the agent assumed it.
|
||||||
|
|
||||||
|
Separately, the agent delivers a design and its code but no plan for building it.
|
||||||
|
|
||||||
|
## Decisions taken
|
||||||
|
|
||||||
|
The owner accepted these defaults on 2026-10-11.
|
||||||
|
|
||||||
|
1. **Scoping is a Phase the customer approves,** before requirements discovery.
|
||||||
|
2. **Its centre is a scope spec:** a YAML file that may arrive complete, partly filled or not at all.
|
||||||
|
3. **Architects write the spec,** the owner and colleagues, not end customers. Field names may be technical.
|
||||||
|
4. **A small set of fields is essential** and always settled before work starts. Which they are depends on the kind of engagement.
|
||||||
|
5. **Two files.** The file supplied is never altered. A completed copy is written beside it.
|
||||||
|
6. **The run options live in the spec,** so one file starts an Execution.
|
||||||
|
7. **Where the spec and the prose request disagree, the spec wins** and the customer is shown the conflict.
|
||||||
|
8. **The work breakdown structure is a new Deliverable,** after validation, in the packaging Phase, grouped by component as the technical decomposition is, covering the technical build.
|
||||||
|
9. **Scoping is built before the work breakdown structure.**
|
||||||
|
|
||||||
|
## Language
|
||||||
|
|
||||||
|
Terms to add to `CONTEXT.md` when built:
|
||||||
|
|
||||||
|
- **Scope spec:** the structured statement of what the customer wants and the facts a design depends on, as a file. It sits beside the Intent, which stays in the customer's words.
|
||||||
|
- **Scoping:** the Phase that completes the scope spec and has the customer approve it.
|
||||||
|
- **Work breakdown structure:** the Deliverable that divides building the Target architecture into work packages and activities, with a timeline.
|
||||||
|
- **Work package:** a unit of work that delivers named Elements and is done when their Checks pass.
|
||||||
|
|
||||||
|
Already in the glossary and used here: Intent, Customer question, Change request, Baseline, Requirements chain.
|
||||||
|
|
||||||
|
## The scope spec
|
||||||
|
|
||||||
|
### What a field can say
|
||||||
|
|
||||||
|
Every field holds a value or one of three words:
|
||||||
|
|
||||||
|
| Written | Meaning | What the agent does |
|
||||||
|
|---|---|---|
|
||||||
|
| a value | The customer states this | Uses it. Recorded as `supplied`. |
|
||||||
|
| `ask`, or the field left out | Not known yet | Asks, when someone is present. Otherwise treats it as `decide` and lists it under `discovery_gaps`. |
|
||||||
|
| `decide` | You choose | Picks a default, recorded as `assumed` with the default and what it affects. |
|
||||||
|
| `not_applicable` | Does not apply | Asks nothing and assumes nothing. |
|
||||||
|
|
||||||
|
A field name the catalogue does not have is an error that names the field. Nothing is silently ignored.
|
||||||
|
|
||||||
|
### Tiers
|
||||||
|
|
||||||
|
| Tier | When it is settled |
|
||||||
|
|---|---|
|
||||||
|
| essential | Always, before any work: asked when someone is present, and refused with a list when nobody is and it is neither supplied nor `decide`. |
|
||||||
|
| important | Asked when someone is present; defaulted otherwise. |
|
||||||
|
| optional | Defaulted unless supplied. |
|
||||||
|
|
||||||
|
### The draft
|
||||||
|
|
||||||
|
Revised on 2026-10-11 to combine the first draft with the owner's own example, an engagement spec for a regulated landing zone. What each brought is listed under **How the two were combined**. Essential fields are marked; which are essential depends on `engagement.kind`. Values shown are from the owner's example.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
schema_version: 1 # the catalogue this file is written against
|
||||||
|
|
||||||
|
metadata:
|
||||||
|
customer_case: fintech-global-landing-zone # essential: groups this work's Executions
|
||||||
|
engagement_reference: ENG-2026-UK-084 # yours; carried through, never interpreted
|
||||||
|
client_name: FinTech Global UK
|
||||||
|
author: Pre-Sales Engineering
|
||||||
|
revision: "1.1.0" # yours: the version of this document
|
||||||
|
status: DRAFT # written by the agent; see Status below
|
||||||
|
|
||||||
|
engagement:
|
||||||
|
kind: platform_foundation # essential: workload | platform_foundation | migration
|
||||||
|
|
||||||
|
# 1. What the customer said. The source of truth; nothing here is rewritten.
|
||||||
|
intent:
|
||||||
|
statement: | # essential: verbatim, or consolidated from calls and documents
|
||||||
|
We are a UK-regulated payment platform expanding into Google Cloud. We need an
|
||||||
|
enterprise-grade landing zone ready in under 10 weeks ...
|
||||||
|
requirements: # what the customer stated outright, with their ids
|
||||||
|
- id: REQ-01
|
||||||
|
category: Timeline
|
||||||
|
statement: Platform foundation must be ready for workload ingestion within 10 weeks.
|
||||||
|
- id: REQ-03
|
||||||
|
category: Networking
|
||||||
|
statement: 10Gbps on-premises hybrid connectivity; centralized Palo Alto inspection for egress.
|
||||||
|
exclusions:
|
||||||
|
- Application container deployment (managed by app squads).
|
||||||
|
- Active-active secondary region in phase 1 (warm standby acceptable).
|
||||||
|
success_criteria: decide
|
||||||
|
|
||||||
|
# 2. Where it runs.
|
||||||
|
estate:
|
||||||
|
cloud_provider: gcp # essential; must be a provider this deployment has
|
||||||
|
mode: greenfield # essential: greenfield | brownfield_expansion | migration
|
||||||
|
regions:
|
||||||
|
primary: europe-west2 # essential
|
||||||
|
secondary: europe-west1
|
||||||
|
resiliency_tier: warm_standby
|
||||||
|
data_residency: United Kingdom # essential
|
||||||
|
existing_systems: ask
|
||||||
|
|
||||||
|
# 3. Who uses it and how hard. Essential for kind workload; not_applicable for a foundation.
|
||||||
|
workload:
|
||||||
|
users: {who: not_applicable, count: not_applicable, peak_concurrent: not_applicable}
|
||||||
|
availability: {target: ask, hours: always, recovery_time: ask, recovery_point: ask}
|
||||||
|
|
||||||
|
# 4. Regulation and governance.
|
||||||
|
compliance:
|
||||||
|
industry: financial_services
|
||||||
|
frameworks: [DORA, ISO27001, PCI_DSS_v4] # essential; an empty list means none apply
|
||||||
|
data_classification: confidential # essential
|
||||||
|
encryption: {type: cmek, key_rotation_days: 90}
|
||||||
|
audit_logging: {retention_days: 365, immutable_retention_policy: true}
|
||||||
|
|
||||||
|
# 5. Network.
|
||||||
|
networking:
|
||||||
|
exposure: internal_only # internal_only | internet_facing
|
||||||
|
topology: shared_vpc_hub_and_spoke # allowed values come from the Cloud provider's catalogue
|
||||||
|
hybrid_connectivity: {enabled: true, type: cloud_interconnect, bandwidth_gbps: 10}
|
||||||
|
egress_control: {model: centralized_appliance, appliance_vendor: palo_alto}
|
||||||
|
cidr_allocations: ask
|
||||||
|
|
||||||
|
# 6. Identity, environments and operations.
|
||||||
|
governance:
|
||||||
|
identity_source_of_truth: entra_id
|
||||||
|
pipeline_authentication: workload_identity_federation
|
||||||
|
environment_tiers:
|
||||||
|
- {name: development, archetype: corp_workload}
|
||||||
|
- {name: staging, archetype: corp_workload}
|
||||||
|
- {name: production, archetype: pci_regulated_workload}
|
||||||
|
operations:
|
||||||
|
operating_model: central_platform_broker
|
||||||
|
support_hours: decide
|
||||||
|
monitoring: decide
|
||||||
|
integrations: []
|
||||||
|
|
||||||
|
# 7. How it will be delivered. The work breakdown structure is built from this.
|
||||||
|
delivery:
|
||||||
|
target_start_date: 2026-11-01
|
||||||
|
window_weeks: 10 # essential when the customer has a deadline
|
||||||
|
commercial_model: fixed_price_milestone
|
||||||
|
iac_tooling: terraform
|
||||||
|
ci_cd: github_actions
|
||||||
|
team_size: ask
|
||||||
|
working_days_per_week: 5
|
||||||
|
|
||||||
|
cost:
|
||||||
|
monthly_budget: ask # essential for kind workload
|
||||||
|
currency: GBP
|
||||||
|
|
||||||
|
# 8. How to run the agent.
|
||||||
|
run:
|
||||||
|
pre_approved: false
|
||||||
|
allow_cloud_read: false
|
||||||
|
overlay_skills: []
|
||||||
|
terraform_modules: false
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sections the agent writes
|
||||||
|
|
||||||
|
Only in the completed copy. The supplied file never has to carry them, and if it does they are read as the last run's and replaced.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
inferred_assumptions:
|
||||||
|
- id: ASSUMP-01
|
||||||
|
field: networking.hybrid_connectivity
|
||||||
|
category: Network Telecom
|
||||||
|
statement: Assumes the client manages the carrier circuit lead time for the 10Gbps interconnect.
|
||||||
|
impact_if_invalid: Blocks the hybrid routing test; delays every activity after it.
|
||||||
|
|
||||||
|
discovery_gaps:
|
||||||
|
- field: networking.cidr_allocations
|
||||||
|
prompt: Specify IP subnet sizes for spoke workloads, for example /22 per environment.
|
||||||
|
severity: HIGH # HIGH is an essential field; MEDIUM important; LOW optional
|
||||||
|
|
||||||
|
provenance: # where every settled value came from
|
||||||
|
estate.regions.primary: supplied
|
||||||
|
cost.monthly_budget: answered
|
||||||
|
operations.monitoring: assumed
|
||||||
|
|
||||||
|
readiness: {supplied: 34, answered: 3, assumed: 6, unknown: 2}
|
||||||
|
```
|
||||||
|
|
||||||
|
`provenance` is one of `supplied`, `answered`, `discovered` (read from the customer's cloud environment), `assumed`, `unknown`. Every `assumed` field has an entry in `inferred_assumptions`, and every `unknown` one in `discovery_gaps`.
|
||||||
|
|
||||||
|
### Status
|
||||||
|
|
||||||
|
The owner's example carries a status. The agent sets it in the completed copy from what has happened; a status written by the author is not trusted.
|
||||||
|
|
||||||
|
| Status | Means |
|
||||||
|
|---|---|
|
||||||
|
| `DRAFT` | Gaps remain, or the conversation is still filling them. |
|
||||||
|
| `UNDER_REVIEW` | Complete, and waiting for the scoping Approval. |
|
||||||
|
| `APPROVED` | The customer approved it; requirements and the later Phases run from it. |
|
||||||
|
| `BASELINE_LOCKED` | Its Execution completed. This copy is the record a Change request starts from, and is never altered. |
|
||||||
|
|
||||||
|
### How the two were combined
|
||||||
|
|
||||||
|
| From the owner's example | How it is kept |
|
||||||
|
|---|---|
|
||||||
|
| The request, verbatim, inside the spec | `intent.statement`. It is also still saved as the Intent, as today. |
|
||||||
|
| The customer's stated requirements, with ids | `intent.requirements`. The agent's requirements trace to these ids, so the Requirements chain starts at what the customer wrote. |
|
||||||
|
| Stated exclusions | `intent.exclusions`. The design is checked for nothing built that is excluded. |
|
||||||
|
| Engagement metadata and status | `metadata`, with the status set by the agent. |
|
||||||
|
| Estate, compliance, networking and governance sections | Kept as written. |
|
||||||
|
| Start date, delivery window, commercial model, tooling | `delivery`. The window becomes a limit the plan is checked against. |
|
||||||
|
| Assumptions with their impact, and gaps with a prompt and a severity | The two agent-written sections, in place of the first draft's flat provenance table. Severity is the tier. |
|
||||||
|
|
||||||
|
| From the first draft | How it is kept |
|
||||||
|
|---|---|
|
||||||
|
| `ask`, `decide` and `not_applicable` | Unchanged. The example had no way to say "not known" other than omission. |
|
||||||
|
| Tiers | Kept, as the severity of a gap. |
|
||||||
|
| Two files | Kept. The author's file is untouched; the agent's sections go in the completed copy. |
|
||||||
|
| Where each value came from | Kept, as `provenance`. |
|
||||||
|
| Users, scale, availability target, budget | Kept under `workload` and `cost`, and essential only for a workload. |
|
||||||
|
| Run options | Kept under `run`. |
|
||||||
|
|
||||||
|
### Where the two pull apart
|
||||||
|
|
||||||
|
Four tensions, each with the resolution proposed. They are the owner's to confirm.
|
||||||
|
|
||||||
|
1. **The example states solution choices; the agent keeps requirements free of them.** A hub-and-spoke topology, an interconnect, customer-managed keys and a named inspection appliance are design decisions. The rule that requirements name no products exists so the agents do not pick a design before the design Phase. It should not stop a customer who has already decided. Proposed: a choice the spec states is a constraint the design must honour and may not change, recorded as supplied. The agents still add none of their own.
|
||||||
|
2. **The example is a landing zone; the agent has only designed workloads.** Every benchmark case is an application. A platform foundation has no users or response times, and has an organisation hierarchy, shared networking and environment archetypes the cases have never exercised. `engagement.kind` lets one spec serve both, but a platform foundation is a new kind of work for the agent and needs its own evaluation case before it is relied on.
|
||||||
|
3. **The example uses one Cloud provider's vocabulary in general fields.** `shared_vpc_hub_and_spoke`, `cloud_interconnect` and `cmek` are Google Cloud's terms. Proposed: the general catalogue says a field such as `networking.topology` exists; the Cloud provider's bundle supplies its allowed values. A second provider then brings its own list and the spec's shape does not change.
|
||||||
|
4. **"Engagement" is a word the glossary avoids.** The agent's term is Customer case. Proposed: `customer_case` stays the key the agent groups by, and `engagement_reference` carries the owner's own number untouched. If the owner would rather the agent's term became Engagement, that is a glossary change to make before this is built.
|
||||||
|
|
||||||
|
### Where the catalogue lives
|
||||||
|
|
||||||
|
A provider-neutral Skill holds the catalogue: each field's type, tier by engagement kind, the question to ask, and guidance for the default. A Cloud provider's bundle adds its allowed values and any fields of its own. An overlay Skill may raise a field's tier, so an organisation can make `compliance.encryption` essential. Code validates a spec against the catalogue; no model decides what is missing.
|
||||||
|
|
||||||
|
## How scoping runs
|
||||||
|
|
||||||
|
| What arrives | What happens |
|
||||||
|
|---|---|
|
||||||
|
| A complete spec | Validated. No gaps, no questions. With `pre_approved` the Phase passes without stopping. |
|
||||||
|
| A partly filled spec, someone present | Gaps are asked for, essential first, a few related fields at a time, with a count of how many are settled. |
|
||||||
|
| A partly filled spec, nobody present | Essential gaps refuse the start, with the list. Others are defaulted and recorded. |
|
||||||
|
| Only a sentence | A model drafts a spec from it, marking everything it did not find as `ask`. It is then a partly filled spec. |
|
||||||
|
|
||||||
|
In conversation an answer is a value, "you decide", or "skip". A skipped field becomes `unknown` and a Customer question in the output. A free-text answer is turned into the field's value and read back before it is written. The customer may supply an edited file at any point in place of answering.
|
||||||
|
|
||||||
|
The Phase ends when the customer approves the completed spec.
|
||||||
|
|
||||||
|
### What it writes
|
||||||
|
|
||||||
|
Under the Customer case, beside the Intent:
|
||||||
|
|
||||||
|
```
|
||||||
|
intent/
|
||||||
|
001.md # the request, in the customer's words (as today)
|
||||||
|
001.scope.yaml # the spec as supplied, unaltered; absent when none was
|
||||||
|
001.scope.completed.yaml # the completed copy: status, assumptions, gaps, provenance, readiness
|
||||||
|
```
|
||||||
|
|
||||||
|
The scoping Deliverable is a short document written from the completed copy: the scope statement, the information register as a table, and the readiness summary. The solution guide opens with that summary.
|
||||||
|
|
||||||
|
### What it changes downstream
|
||||||
|
|
||||||
|
- **The requirements analyst derives; it no longer asks or assumes.** Each requirement names the stated requirement (`REQ-03`) or the spec field it comes from, so the Requirements chain starts at what the customer wrote. A requirement from an `assumed` field says so.
|
||||||
|
- **A stated choice binds the design.** A product, topology or setting the spec states is carried to the design as a constraint, and a design that departs from it is refused.
|
||||||
|
- **Exclusions are checked.** Nothing in `intent.exclusions` may appear in the Target architecture.
|
||||||
|
- **Customer questions** from the pillar checklists (built 2026-10-10) are joined by the `unknown` fields.
|
||||||
|
- **A Change request may be an edited spec.** Code compares the two specs field by field, and that list is what changed. A sentence still works, as now.
|
||||||
|
- **An evaluation case becomes a spec,** replacing the pattern-matched answers.
|
||||||
|
|
||||||
|
## The work breakdown structure
|
||||||
|
|
||||||
|
A new Deliverable in the packaging Phase, produced by a new Specialist after validation, summarised in the solution guide.
|
||||||
|
|
||||||
|
### Its data file
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"work_packages": [
|
||||||
|
{"id": "WP-1", "name": "Network foundation", "parent": null,
|
||||||
|
"components": ["C-2"], "elements": ["E-1", "E-2"],
|
||||||
|
"scope": "The VPC, its subnet and outbound access.",
|
||||||
|
"produces": ["a private network the services attach to"],
|
||||||
|
"skills": ["network engineering"]}
|
||||||
|
],
|
||||||
|
"activities": [
|
||||||
|
{"id": "A-1", "work_package": "WP-1", "name": "Create the VPC and subnet",
|
||||||
|
"days": 1, "after": []},
|
||||||
|
{"id": "A-2", "work_package": "WP-1", "name": "Configure outbound access",
|
||||||
|
"days": 1, "after": ["A-1"]}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### What code checks and computes
|
||||||
|
|
||||||
|
- Every Element belongs to a work package, and every work package to a component or a parent. A design cannot be left out of the plan.
|
||||||
|
- A work package's acceptance criteria are the Checks on its Elements. They are read from the Target architecture, not written again.
|
||||||
|
- Activities form no cycle, and every `after` names an activity that exists.
|
||||||
|
- **The plan is checked against the delivery window.** When `delivery.window_weeks` is set and the computed total exceeds it, the Deliverable says so and names the critical path; it does not shorten estimates to fit.
|
||||||
|
- **An assumption can hold up the plan.** An entry in `inferred_assumptions` that an activity depends on, such as a carrier's lead time, appears as a dependency the customer owns, so its `impact_if_invalid` is visible on the timeline.
|
||||||
|
- **Milestones.** With a `commercial_model` of fixed price by milestone, each top-level work package ends in a milestone.
|
||||||
|
- **The schedule is computed, not written.** From each activity's days and what it comes after, code works out its earliest start and finish, the critical path and the total. With `delivery.team_size` it limits how many activities run at once.
|
||||||
|
|
||||||
|
### What it renders
|
||||||
|
|
||||||
|
Three views, all written by code from the data file, as the architecture diagram is:
|
||||||
|
|
||||||
|
- **WBS dictionary:** the hierarchy, each work package with its scope, what it produces, its Elements, its acceptance Checks and the skills it needs.
|
||||||
|
- **Activity sequence list:** each activity with its days, what it follows, its start and finish day, and whether it is on the critical path.
|
||||||
|
- **Timeline** on a relative axis, Day 0 to Day N:
|
||||||
|
|
||||||
|
```
|
||||||
|
Activity Day 0 5 10 15 18
|
||||||
|
A-1 Create the VPC and subnet ██
|
||||||
|
A-2 Configure outbound access ██
|
||||||
|
A-3 Provision the database █████
|
||||||
|
A-4 Deploy the application ████
|
||||||
|
A-5 Load balancer and access ███
|
||||||
|
A-6 Monitoring and alerts ██
|
||||||
|
A-7 Validation in test ████
|
||||||
|
```
|
||||||
|
|
||||||
|
The chart format is to be settled when built: a Mermaid Gantt chart if it can show a relative day axis, and otherwise a drawn table like the one above.
|
||||||
|
|
||||||
|
### What it says about itself
|
||||||
|
|
||||||
|
Durations are a model's estimates. The Deliverable states their basis (working days, the team size from the spec) and that they are indicative. The timeline stays on a relative day axis; when `delivery.target_start_date` is set, the activity list also gives calendar dates.
|
||||||
|
|
||||||
|
## Order of work
|
||||||
|
|
||||||
|
| Stage | What | Usable on its own |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | The catalogue, validation, the gap list, the two files saved beside the Intent, the run options read from the spec | Yes: a complete spec starts a pre-approved Execution |
|
||||||
|
| 2 | The conversation that fills gaps; scoping as a Phase with its Deliverable and Approval; a spec drafted from a sentence | Yes |
|
||||||
|
| 3 | Requirements traced to spec fields; readiness in the guide | Yes |
|
||||||
|
| 4 | An edited spec as a Change request | Yes |
|
||||||
|
| 5 | The work breakdown structure | Yes, with stage 1 for `delivery` |
|
||||||
|
| 6 | Evaluation cases as specs | Yes |
|
||||||
|
|
||||||
|
## Not in this spec
|
||||||
|
|
||||||
|
- Attaching documents (a statement of work, a diagram) as a source for the spec.
|
||||||
|
- Reading the customer's cloud environment to fill `current_estate`. The `discovered` source is reserved for it.
|
||||||
|
- Delivery activities beyond the technical build, such as security review, user acceptance and cutover.
|
||||||
|
- Cost of the work, or named people.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
1. The four tensions under **Where the two pull apart**: are the proposed resolutions right?
|
||||||
|
2. Which fields are essential for each engagement kind? The draft marks them; the owner's example suggests the delivery window and compliance frameworks matter more than budget for a foundation.
|
||||||
|
3. In `pre_approved` runs an essential gap refuses the start. Is that right, or should it default and warn?
|
||||||
|
4. Should the scoping Approval be skippable when the spec arrived complete and someone is present?
|
||||||
|
5. Is a platform foundation in scope now, with its own evaluation case, or should scoping be built against workloads first?
|
||||||
|
|
||||||
|
## Comments
|
||||||
|
|
||||||
|
- 2026-10-11, the owner: a platform foundation is in scope now; an engagement name and a Customer case are equivalent references; the implementation will eventually be provider-agnostic. The other tensions and open questions were taken as proposed: a stated choice binds the design; an essential gap refuses a pre-approved start; the scoping Approval is not skipped. Recorded in ADR-0008.
|
||||||
24
CONTEXT.md
24
CONTEXT.md
@@ -19,13 +19,21 @@ A caller-supplied grouping of Executions. A Session never feeds earlier outputs
|
|||||||
_Avoid_: conversation, context
|
_Avoid_: conversation, context
|
||||||
|
|
||||||
**Customer case**:
|
**Customer case**:
|
||||||
A caller-supplied grouping of the Executions run for one customer workload, identified by a short slug. Every Execution belongs to one; an Execution started without one belongs to `unassigned`. A Customer case ties each request to the Deliverables generated for it. It feeds one Execution's outputs into another only when the caller starts a Change request and names the Baseline.
|
A caller-supplied grouping of the Executions run for one customer engagement, identified by a short slug. An engagement and a Customer case are the same thing: a reference such as `ENG-2026-UK-084` names the case `eng-2026-uk-084`. Every Execution belongs to one; an Execution started without one belongs to `unassigned`. A Customer case ties each request to the Deliverables generated for it. It feeds one Execution's outputs into another only when the caller starts a Change request and names the Baseline.
|
||||||
_Avoid_: engagement, project, account, tenant
|
_Avoid_: project, account, tenant
|
||||||
|
|
||||||
**Intent**:
|
**Intent**:
|
||||||
What the customer wants from a Customer case, in their own words and at a high level. Each Workflow request made in the case is a statement of it, and the statements are kept in order as its history. The Intent is the customer's; requirements are what the agents derive from it.
|
What the customer wants from a Customer case, in their own words and at a high level. Each Workflow request made in the case is a statement of it, and the statements are kept in order as its history. The Intent is the customer's; requirements are what the agents derive from it.
|
||||||
_Avoid_: brief, ask, requirements (which are the derived Deliverable)
|
_Avoid_: brief, ask, requirements (which are the derived Deliverable)
|
||||||
|
|
||||||
|
**Scope spec**:
|
||||||
|
The structured statement of what the customer wants and the facts a design depends on, as a file. It may arrive complete or partly filled. It sits beside the Intent, which stays in the customer's words, and a value it states is the customer's decision.
|
||||||
|
_Avoid_: brief, questionnaire, requirements (which are derived from it)
|
||||||
|
|
||||||
|
**Scoping**:
|
||||||
|
Completing the scope spec, from what was supplied, what the customer answers and what the agent is left to decide, and having the customer approve it before requirements begin.
|
||||||
|
_Avoid_: discovery (which reads a cloud environment), intake (which reads a message)
|
||||||
|
|
||||||
**Change request**:
|
**Change request**:
|
||||||
An Execution started to say how the customer's Intent has changed. It begins from a Baseline and delivers the full updated Deliverables with an account of what changed and why.
|
An Execution started to say how the customer's Intent has changed. It begins from a Baseline and delivers the full updated Deliverables with an account of what changed and why.
|
||||||
_Avoid_: amendment, update run, iteration, re-run
|
_Avoid_: amendment, update run, iteration, re-run
|
||||||
@@ -35,7 +43,7 @@ The completed Execution, in the same Customer case, that a Change request starts
|
|||||||
_Avoid_: parent, previous run, source Execution
|
_Avoid_: parent, previous run, source Execution
|
||||||
|
|
||||||
**Phase**:
|
**Phase**:
|
||||||
A named grouping of Deliverables: requirements discovery, solution architecture (the design, ending with the Target architecture), solution implementation, solution validation, or packaging. A Phase is complete when all its Deliverables are submitted and the user has granted the Phase's Approval.
|
A named grouping of Deliverables: scoping, requirements discovery, solution architecture (the design, ending with the Target architecture), solution implementation, solution validation, or packaging. A Phase is complete when all its Deliverables are submitted and the user has granted the Phase's Approval.
|
||||||
_Avoid_: step, node, stage, pipeline
|
_Avoid_: step, node, stage, pipeline
|
||||||
|
|
||||||
**Approval**:
|
**Approval**:
|
||||||
@@ -81,7 +89,7 @@ _Avoid_: custom prompt, override, plugin
|
|||||||
### Outputs
|
### Outputs
|
||||||
|
|
||||||
**Deliverable**:
|
**Deliverable**:
|
||||||
A document an Execution produces: requirements, technical decomposition, product selection, design recommendations, the Target architecture, architecture description, deployment guidance, validation results, or the Solution guide. Each Deliverable depends on others being approved first.
|
A document an Execution produces: the scope, requirements, technical decomposition, product selection, design recommendations, the Target architecture, architecture description, deployment guidance, validation results, the Work breakdown structure, or the Solution guide. Each Deliverable depends on others being approved first.
|
||||||
_Avoid_: artifact, output file
|
_Avoid_: artifact, output file
|
||||||
|
|
||||||
**Dependency**:
|
**Dependency**:
|
||||||
@@ -112,6 +120,14 @@ _Avoid_: traceability matrix, lineage
|
|||||||
Something the architecture depends on that the customer's Intent does not say, recorded with the question to put to them. It marks missing information, not a fault in the design, and its answer may change a recommendation.
|
Something the architecture depends on that the customer's Intent does not say, recorded with the question to put to them. It marks missing information, not a fault in the design, and its answer may change a recommendation.
|
||||||
_Avoid_: gap, failure, open issue, assumption (which is a default the agents chose)
|
_Avoid_: gap, failure, open issue, assumption (which is a default the agents chose)
|
||||||
|
|
||||||
|
**Work breakdown structure**:
|
||||||
|
The Deliverable that divides building the Target architecture into Work packages and the activities within them, with a timeline on a relative axis of working days. Its schedule is computed from the activities' durations and dependencies, never written.
|
||||||
|
_Avoid_: project plan, roadmap, schedule (which is one view of it)
|
||||||
|
|
||||||
|
**Work package**:
|
||||||
|
A unit of the build that delivers named Elements, and is done when the Checks on those Elements pass.
|
||||||
|
_Avoid_: task, epic, phase (which groups Deliverables), workstream
|
||||||
|
|
||||||
**Solution guide**:
|
**Solution guide**:
|
||||||
The packaged Deliverable that assembles every other Deliverable into the Cloud provider's solution architecture template.
|
The packaged Deliverable that assembles every other Deliverable into the Cloud provider's solution architecture template.
|
||||||
_Avoid_: final report, package
|
_Avoid_: final report, package
|
||||||
|
|||||||
@@ -10,14 +10,19 @@ Vocabulary is defined in [CONTEXT.md](CONTEXT.md), decisions in [docs/adr/](docs
|
|||||||
|
|
||||||
### 1. Orchestrator and Specialists (`agents/`)
|
### 1. Orchestrator and Specialists (`agents/`)
|
||||||
One Orchestrator coordinates an Execution and is the only agent that talks to the user. It is a router ([ADR-0007](docs/adr/0007-jev-is-the-orchestrators-decision-point.md)): code reads the Execution's state, a decision model (Jev) answers the closed questions that need judgement, and a playbook in code picks the action. A language model only writes what the user reads. It delegates each Deliverable to one Specialist, and independent Deliverables run in parallel:
|
One Orchestrator coordinates an Execution and is the only agent that talks to the user. It is a router ([ADR-0007](docs/adr/0007-jev-is-the-orchestrators-decision-point.md)): code reads the Execution's state, a decision model (Jev) answers the closed questions that need judgement, and a playbook in code picks the action. A language model only writes what the user reads. It delegates each Deliverable to one Specialist, and independent Deliverables run in parallel:
|
||||||
|
- **Scoping:** `scope-analyst`, which completes the scope spec with the customer.
|
||||||
- **Requirements discovery:** `requirements-analyst`, `decomposer`.
|
- **Requirements discovery:** `requirements-analyst`, `decomposer`.
|
||||||
- **Solution architecture (design):** `product-selector`, `design-advisor` (which consults one `pillar-advisor` per architecture-framework pillar), `target-architect`.
|
- **Solution architecture (design):** `product-selector`, `design-advisor` (which consults one `pillar-advisor` per architecture-framework pillar), `target-architect`.
|
||||||
- **Solution implementation:** `describer`, `iac-author`.
|
- **Solution implementation:** `describer`, `iac-author`.
|
||||||
- **Solution validation:** `validator`.
|
- **Solution validation:** `validator`.
|
||||||
- **Packaging:** `packager`.
|
- **Packaging:** `delivery-planner`, which plans the build as a work breakdown structure, and `packager`.
|
||||||
|
|
||||||
Each Phase needs the user's explicit Approval before work that depends on it starts. A pre-approved mode serves batch and evaluation runs.
|
Each Phase needs the user's explicit Approval before work that depends on it starts. A pre-approved mode serves batch and evaluation runs.
|
||||||
|
|
||||||
|
An Execution may start from a **scope spec** ([ADR-0008](docs/adr/0008-scoping-starts-from-a-scope-spec.md)): a YAML file of what the customer wants and the facts a design depends on, complete or partly filled. Code reads it against a catalogue and refuses a field it does not know. Scoping, the first Phase, completes it: the scope analyst asks for what is missing when someone is present and assumes it, visibly, when nobody is. With no spec it drafts one from the request. The customer approves the result before requirements begin, and a completed copy is kept beside their request with what was assumed and what is still unknown. `python -m app.scope template platform_foundation` prints one to fill in, and `python -m app.scope check <file>` says what a spec lacks. Pass it as `scope` to `/generate` or in A2A metadata.
|
||||||
|
|
||||||
|
The **work breakdown structure** divides building the design into work packages and activities. The planner writes it as data; code checks that every Element belongs to a work package, computes each activity's start and finish from its days and what it follows, finds the critical path, and draws the timeline on a relative axis from Day 0. The plan is checked against the delivery window the scope gives.
|
||||||
|
|
||||||
A **Change request** ([ADR-0006](docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md)) starts from a completed Execution in the same Customer case, its Baseline, and delivers the full updated Deliverables with `changes.json`: what changed, what was only reworded, and anything changed that nobody asked for.
|
A **Change request** ([ADR-0006](docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md)) starts from a completed Execution in the same Customer case, its Baseline, and delivers the full updated Deliverables with `changes.json`: what changed, what was only reworded, and anything changed that nobody asked for.
|
||||||
|
|
||||||
The design Phase ends with the **Target architecture**: one explicit graph of Elements, Relationships and Checks ([ADR-0005](docs/adr/0005-the-target-architecture-is-an-explicit-graph.md)). The user approves it before any infrastructure code is written. The diagram is drawn from it by code, the Terraform must account for every Check, and validation gives a verdict per Check. Requirements, components, products and recommendations each carry ids in a data file, so every requirement can be traced through to its verdict; a submission with a missing link is refused.
|
The design Phase ends with the **Target architecture**: one explicit graph of Elements, Relationships and Checks ([ADR-0005](docs/adr/0005-the-target-architecture-is-an-explicit-graph.md)). The user approves it before any infrastructure code is written. The diagram is drawn from it by code, the Terraform must account for every Check, and validation gives a verdict per Check. Requirements, components, products and recommendations each carry ids in a data file, so every requirement can be traced through to its verdict; a submission with a missing link is refused.
|
||||||
|
|||||||
46
agents/delivery-planner.md
Normal file
46
agents/delivery-planner.md
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
---
|
||||||
|
role: delivery-planner
|
||||||
|
deliverable: work_breakdown
|
||||||
|
tools: []
|
||||||
|
budget: 20
|
||||||
|
includes: [specialist-protocol]
|
||||||
|
---
|
||||||
|
You produce the **work breakdown structure** Deliverable: a plan for building the Target architecture that has been designed and validated. You change nothing in the design. You say how the work divides, in what order it goes, and how long each piece takes.
|
||||||
|
|
||||||
|
You write the plan as data. The WBS dictionary, the activity sequence, the schedule and the timeline are computed and written from it for you when you submit, so write none of them yourself.
|
||||||
|
|
||||||
|
Attach `work-breakdown.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"work_packages": [
|
||||||
|
{"id": "WP-1", "name": "Network foundation", "parent": null,
|
||||||
|
"elements": ["E-1", "E-2"],
|
||||||
|
"scope": "The network, its subnets and outbound access.",
|
||||||
|
"produces": ["a private network the services attach to"],
|
||||||
|
"skills": ["network engineering"]}
|
||||||
|
],
|
||||||
|
"activities": [
|
||||||
|
{"id": "A-1", "work_package": "WP-1", "name": "Create the network and subnets", "days": 1, "after": []},
|
||||||
|
{"id": "A-2", "work_package": "WP-1", "name": "Order the carrier circuit", "days": 30, "after": [],
|
||||||
|
"owner": "customer", "assumption": "The customer manages the carrier's lead time."}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Work packages:
|
||||||
|
|
||||||
|
- Group them as the technical decomposition groups the design: one for each component, or for a few components that are built together, with a parent work package where several belong to one area.
|
||||||
|
- Every Element of the Target architecture belongs to exactly one work package. A submission is refused while any Element belongs to none.
|
||||||
|
- A work package is done when the Checks on its Elements pass. Do not write acceptance criteria: they are read from the Target architecture.
|
||||||
|
- Add a work package only for the technical build. Reviews, acceptance testing and cut-over are not part of this plan.
|
||||||
|
|
||||||
|
Activities:
|
||||||
|
|
||||||
|
- Each is one piece of work one person does, of between half a day and about ten days. Anything longer is several activities.
|
||||||
|
- `days` is working days of effort for one engineer who knows the Cloud provider. Estimate from what the infrastructure code actually contains: a resource written from a pattern is hours, a network joined to another site is days. Do not pad, and do not shorten to fit a deadline: the plan is checked against the delivery window after you submit, and it is more useful honest than fitting.
|
||||||
|
- `after` lists the activities that must finish first. Give a dependency only where one is real: what the code in one needs from another. Leaving independent work unordered is what lets it run in parallel.
|
||||||
|
- `owner` is `team`, the default, or `customer` for something the customer or their supplier must do that the team waits on, such as ordering a circuit or granting access. Give its lead time as `days` and say in `assumption` what you assumed.
|
||||||
|
- Include the activities that make the build usable: enabling the provider's APIs, the pipeline that applies the code, and running the validation in each environment the scope names.
|
||||||
|
|
||||||
|
Your content is one section, under its heading: the approach in a few sentences, saying how the work is ordered and why, what runs in parallel, what the plan waits on from the customer, and the assumptions behind the estimates. Read the delivery facts from `scope`: the team size, the delivery window, the environments and the commercial model.
|
||||||
@@ -10,6 +10,8 @@ You produce the **solution guide** Deliverable: the Skill's packaging of every a
|
|||||||
- Consolidate the content of every Deliverable in `dependencies`; rewrite for flow, but keep every requirement, product decision, citation, recommendation, command and validation verdict.
|
- Consolidate the content of every Deliverable in `dependencies`; rewrite for flow, but keep every requirement, product decision, citation, recommendation, command and validation verdict.
|
||||||
- Include the architecture diagram, which is the `architecture.mmd` attachment of the Target architecture, unchanged as a `mermaid` code block, and the Terraform as code blocks.
|
- Include the architecture diagram, which is the `architecture.mmd` attachment of the Target architecture, unchanged as a `mermaid` code block, and the Terraform as code blocks.
|
||||||
- List the Target architecture's Checks with their validation verdicts in the validation section.
|
- List the Target architecture's Checks with their validation verdicts in the validation section.
|
||||||
|
- Write no timeline, schedule or estimate of your own. The delivery plan is a Deliverable of its own, and its timeline is added to the guide for you.
|
||||||
|
- Open the executive summary with what the design rests on, from the scope: how many of its facts the customer stated, how many were assumed for them, and how many are still unknown, naming the assumptions that most shape the design. A reader must be able to tell an agreed design from an indicative one.
|
||||||
- When the design recommendations hold a "Questions for the customer" section, carry every question into the guide's design and configuration recommendations section, under the same heading and with the same wording. They are what the request did not say enough to settle; say so, and do not present them as faults or as recommendations.
|
- When the design recommendations hold a "Questions for the customer" section, carry every question into the guide's design and configuration recommendations section, under the same heading and with the same wording. They are what the request did not say enough to settle; say so, and do not present them as faults or as recommendations.
|
||||||
|
|
||||||
In a Change request (`baseline` is set), open the executive summary with what changed and why, from `baseline.changes`: the requirements added, changed and removed with their reasons, then what that changed in the architecture. Name separately anything listed under `unrequested`: it changed without a changed requirement reaching it.
|
In a Change request (`baseline` is set), open the executive summary with what changed and why, from `baseline.changes`: the requirements added, changed and removed with their reasons, then what that changed in the architecture. Name separately anything listed under `unrequested`: it changed without a changed requirement reaching it.
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ Your task message carries JSON context:
|
|||||||
- `working_copy`: what your own tools already hold from your earlier attempts, or `null`. Start from it; do not start again.
|
- `working_copy`: what your own tools already hold from your earlier attempts, or `null`. Start from it; do not start again.
|
||||||
- `baseline`: set only in a Change request, where the user has said how their requirements changed since an architecture was delivered. `deliverable` is that delivered version of your own Deliverable, and `changes` is what has changed so far: the requirements added, changed and removed, with the reason, and everything already revised from them. `null` otherwise.
|
- `baseline`: set only in a Change request, where the user has said how their requirements changed since an architecture was delivered. `deliverable` is that delivered version of your own Deliverable, and `changes` is what has changed so far: the requirements added, changed and removed, with the reason, and everything already revised from them. `null` otherwise.
|
||||||
- `read_first`: references of your Skills that this task was judged to need, each as a Skill and a resource. Read these first with `load_skill_resource`. It is where to start, not a limit: read anything else you find you need.
|
- `read_first`: references of your Skills that this task was judged to need, each as a Skill and a resource. Read these first with `load_skill_resource`. It is where to start, not a limit: read anything else you find you need.
|
||||||
|
- `scope`: what the customer's scope spec says, when they supplied one. `stated` is theirs and binds your Deliverable; `assumed` was chosen for them and may change when they answer; `to_ask` is unknown. Load the scoping Skill for the rules. Honour every stated choice, build nothing under `stated.intent.exclusions`, and name the customer's own requirement id when your work serves one.
|
||||||
- `overlay_skills`: Skills that carry the user's organisation's standards or this engagement's constraints. Load each one and apply it. Where an overlay disagrees with the Cloud provider's general guidance, the overlay wins; where it disagrees with a requirement the user stated, follow the requirement and say so.
|
- `overlay_skills`: Skills that carry the user's organisation's standards or this engagement's constraints. Load each one and apply it. Where an overlay disagrees with the Cloud provider's general guidance, the overlay wins; where it disagrees with a requirement the user stated, follow the requirement and say so.
|
||||||
- `pre_approved`: `true` when no user is available.
|
- `pre_approved`: `true` when no user is available.
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,8 @@ The context has `phase`, the Phase the user is being asked to approve, and `fact
|
|||||||
Write a summary a busy architect can approve from:
|
Write a summary a busy architect can approve from:
|
||||||
|
|
||||||
- Say what the Phase produced and what approving it commits them to. For the design Phase, say that no infrastructure code is written until they approve it.
|
- Say what the Phase produced and what approving it commits them to. For the design Phase, say that no infrastructure code is written until they approve it.
|
||||||
|
- For the scoping Phase, `facts.scope` says which fields were assumed for them and which are still unknown. Say how many of each, name the ones that most shape a design, and say that requirements are derived from this scope once they approve it.
|
||||||
|
- For the packaging Phase, `facts.delivery_plan` gives the plan's length in weeks, the team it assumes and whether it fits the delivery window. Say so plainly; when it is over the window, say by how much the plan is over and that the critical path is in the work breakdown structure.
|
||||||
- State every number in `facts` that bears on the decision, exactly as given. Add no fact that is not there.
|
- State every number in `facts` that bears on the decision, exactly as given. Add no fact that is not there.
|
||||||
- Name anything they should look at before approving: failed or unverifiable Checks, open gaps, recommendations treated as not applicable, Deliverables that were revised.
|
- Name anything they should look at before approving: failed or unverifiable Checks, open gaps, recommendations treated as not applicable, Deliverables that were revised.
|
||||||
- Use plain sentences and no headings. Keep it under 120 words.
|
- Use plain sentences and no headings. Keep it under 120 words.
|
||||||
|
|||||||
@@ -7,11 +7,13 @@ includes: [specialist-protocol]
|
|||||||
You produce the **requirements** Deliverable: the Skill's requirements-discovery gathering and its resolution of ambiguities and contradictions.
|
You produce the **requirements** Deliverable: the Skill's requirements-discovery gathering and its resolution of ambiguities and contradictions.
|
||||||
|
|
||||||
- Cover functional requirements, non-functional requirements (security, privacy, compliance, reliability, disaster recovery, cost, operations, performance, sustainability), current state, and dependencies.
|
- Cover functional requirements, non-functional requirements (security, privacy, compliance, reliability, disaster recovery, cost, operations, performance, sustainability), current state, and dependencies.
|
||||||
- The user states what they want at a high level; the detail is your job. When a non-functional requirement is missing, propose a default for it from the Cloud provider's architecture framework and record it under `Assumptions` with what it decides: availability tiers, scaling, cost budgets, resource types and security posture. Ask the user to confirm the whole set in one question.
|
- The scope has been agreed before you start: `scope` holds what the customer stated, what was assumed for them and what is still unknown. The detail is your job. Derive the requirements from it; do not ask again for anything it settles.
|
||||||
- Ask an open question only where no sensible default can be inferred from the request, such as the region, a compliance regime or a budget ceiling.
|
- Where a non-functional requirement needs something the scope does not hold, propose a default from the Cloud provider's architecture framework and record it under `Assumptions` with what it decides. Carry every entry of `scope.assumed` into `Assumptions` too, so a reader of the requirements sees what rests on one.
|
||||||
|
- Ask a question only for something the scope leaves unknown that you cannot write a requirement without.
|
||||||
- Name every ambiguity or contradiction, explain why the conflicting goals can't both hold, and ask the user to choose. When they delegate the choice, propose one and ask them to approve it.
|
- Name every ambiguity or contradiction, explain why the conflicting goals can't both hold, and ask the user to choose. When they delegate the choice, propose one and ask them to approve it.
|
||||||
- When the user runs workloads in a cloud environment today, use your cloud discovery tools to describe the current state from what actually exists, and record anything you couldn't read as a discovery gap.
|
- When the user runs workloads in a cloud environment today, use your cloud discovery tools to describe the current state from what actually exists, and record anything you couldn't read as a discovery gap.
|
||||||
- Name no products, services or architectures. This Deliverable describes the problem, not the solution.
|
- Name no products, services or architectures of your own choosing. This Deliverable describes the problem, not the solution. A product or design choice the customer stated in `scope` is theirs: record it as a constraint, in their words.
|
||||||
|
- Give each requirement its `from`, as the requirements-chain Skill describes. Cover every entry of `scope.stated.intent.requirements`, the customer's own requirements, by id. Carry every field under `scope.binding`, a choice the customer stated, in a non-functional requirement whose conditions say what must exist or be set for it to have been honoured. A stated choice that reaches no requirement reaches no Check, and can then be named in the design without being built. List what `scope.to_ask` still holds under `Open questions`.
|
||||||
- Give every requirement an id and attach them all as `requirements.json`, in the format the requirements-chain Skill gives. Use the same ids in your content.
|
- Give every requirement an id and attach them all as `requirements.json`, in the format the requirements-chain Skill gives. Use the same ids in your content.
|
||||||
- Give every non-functional requirement its `conditions`: one entry for each separate thing that must be true for it to be met and that the configuration of infrastructure can show. "Data is encrypted in transit and at rest" has two. Each becomes something the design is checked against, so a condition left out is a condition nobody verifies.
|
- Give every non-functional requirement its `conditions`: one entry for each separate thing that must be true for it to be met and that the configuration of infrastructure can show. "Data is encrypted in transit and at rest" has two. Each becomes something the design is checked against, so a condition left out is a condition nobody verifies.
|
||||||
- Put under `outside_code` what the requirement asks for that no infrastructure configuration can show: an outcome measured in operation (an availability percentage, a response time), behaviour of the application's own code, a process people follow, or a property of a region or product that only its provider publishes (a carbon rating, a service level). For the last, the condition is the choice configuration can show, such as which region is set. Those are recorded and reported, and are not checked against the infrastructure code.
|
- Put under `outside_code` what the requirement asks for that no infrastructure configuration can show: an outcome measured in operation (an availability percentage, a response time), behaviour of the application's own code, a process people follow, or a property of a region or product that only its provider publishes (a carbon rating, a service level). For the last, the condition is the choice configuration can show, such as which region is set. Those are recorded and reported, and are not checked against the infrastructure code.
|
||||||
|
|||||||
24
agents/scope-analyst.md
Normal file
24
agents/scope-analyst.md
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
---
|
||||||
|
role: scope-analyst
|
||||||
|
deliverable: scope
|
||||||
|
tools: []
|
||||||
|
budget: 30
|
||||||
|
includes: [specialist-protocol]
|
||||||
|
---
|
||||||
|
You produce the **scope** Deliverable: the agreed statement of what the customer wants and the facts a design depends on, as a completed scope spec. You design nothing and choose no products. Load the scoping Skill: it says what each field means.
|
||||||
|
|
||||||
|
The scope spec is kept for you. `scope_read` shows what is settled, what is not, and the question to ask for each field. `scope_set` records fields, several per call.
|
||||||
|
|
||||||
|
1. Call `scope_read`. When the customer supplied a spec, most of it is already there, and what they stated stays as they stated it.
|
||||||
|
2. Read `workflow_request`. Record what it states and the spec does not yet hold, with `source` `request`: the statement itself, word for word, under `intent.statement`; the kind of engagement; and any fact it gives. Record only what the words say. Do not infer a number or a choice the customer did not give.
|
||||||
|
3. Settle what is not settled, the essential fields first.
|
||||||
|
- **When you may ask the user** (`pre_approved` is `false`): call `needs_input` with the questions for the essential fields, and the important ones that most change a design, at most eight in one call, each in the catalogue's words. Tell them they may answer, say "you decide", or skip one. Record each answer with `source` `answered`. For "you decide", choose a value and record it with `source` `assumed` and a `note`. Leave a skipped field as it is. Ask a second batch only for what is still essential.
|
||||||
|
- **When you may not ask** (`pre_approved` is `true`): for each essential or important field not settled, choose a value from the catalogue's default guidance and what the request implies, and record it with `source` `assumed` and a `note` saying why. Leave optional fields alone.
|
||||||
|
- A field marked `decide` is yours: choose, and record it as assumed with a note.
|
||||||
|
- A field that cannot apply to this kind of engagement is `not_applicable`.
|
||||||
|
4. Call `scope_read` again. Every essential field must be settled before you can submit.
|
||||||
|
5. Submit. Your content is the scope statement under its heading: three to six sentences saying what the customer wants, what is in scope, what is out of scope, and what success looks like, in plain words and using only what the spec holds. Name as out of scope only what `intent.exclusions` lists; do not add an exclusion of your own. The completed spec, the information register, the assumptions and the questions still open are written for you from what you recorded. Pass no attachments.
|
||||||
|
|
||||||
|
Never record a product, a region, a number or a regulation the customer did not state as though they had: that is `assumed`, with its note. An assumption the customer can see is useful; one presented as their decision is a fault.
|
||||||
|
|
||||||
|
In a Change request (`baseline` is set), the spec you start from is the one that was delivered, and `workflow_request` says what has changed. Record only the fields the change reaches, with `source` `answered`: the customer has now said so. Leave every other field as it was. When no spec was delivered with the Baseline, complete one as above, from `workflow_request` and from the requirements in `baseline`.
|
||||||
@@ -8,6 +8,7 @@ includes: [specialist-protocol]
|
|||||||
You produce the **Target architecture** Deliverable: the solution as one explicit graph of Elements, Relationships and Checks. It is agreed before any infrastructure code is written, and the diagram, the description, the code and the validation are all derived from it.
|
You produce the **Target architecture** Deliverable: the solution as one explicit graph of Elements, Relationships and Checks. It is agreed before any infrastructure code is written, and the diagram, the description, the code and the validation are all derived from it.
|
||||||
|
|
||||||
- Load the requirements-chain Skill for the graph's format and how to derive it.
|
- Load the requirements-chain Skill for the graph's format and how to derive it.
|
||||||
|
- Every Element is a resource of the Cloud provider's own Terraform provider. Something outside it that the design depends on, such as a source repository or a third party's service, is not an Element: say in the design rationale that it is needed and who provides it.
|
||||||
- Build the graph with `graph_put` in a handful of calls of ten to twenty items each: all the Elements, then all the Relationships, then the Checks in batches. A call for every Element, or for every few Checks, uses up your tool calls before the graph is finished; a single call with everything is a reply long enough to time out.
|
- Build the graph with `graph_put` in a handful of calls of ten to twenty items each: all the Elements, then all the Relationships, then the Checks in batches. A call for every Element, or for every few Checks, uses up your tool calls before the graph is finished; a single call with everything is a reply long enough to time out.
|
||||||
- Give every component an Element with the product selected for it, and add the supporting Elements the design needs, such as identities, policies and networks.
|
- Give every component an Element with the product selected for it, and add the supporting Elements the design needs, such as identities, policies and networks.
|
||||||
- Give every connection between Elements a Relationship.
|
- Give every connection between Elements a Relationship.
|
||||||
|
|||||||
@@ -130,8 +130,9 @@ class SolutionArchitectExecutor(AgentExecutor):
|
|||||||
baseline=options.get("baseline"),
|
baseline=options.get("baseline"),
|
||||||
overlay_skills=options.get("overlay_skills") or (),
|
overlay_skills=options.get("overlay_skills") or (),
|
||||||
terraform_modules=options.get("terraform_modules"),
|
terraform_modules=options.get("terraform_modules"),
|
||||||
|
scope=options.get("scope"),
|
||||||
execution_id=task_id,
|
execution_id=task_id,
|
||||||
pre_approved=bool(options.get("pre_approved", False)),
|
pre_approved=options.get("pre_approved"),
|
||||||
allow_cloud_read=options.get("allow_cloud_read"),
|
allow_cloud_read=options.get("allow_cloud_read"),
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -37,6 +37,16 @@ Turn = Sequence[Call] | Callable[[AgentTask], Sequence[Call]]
|
|||||||
|
|
||||||
# One small, coherent Requirements chain, so sample submissions pass every link.
|
# One small, coherent Requirements chain, so sample submissions pass every link.
|
||||||
SAMPLE_DATA: dict[str, str] = {
|
SAMPLE_DATA: dict[str, str] = {
|
||||||
|
"scope.yaml": "engagement:\n kind: workload\nintent:\n statement: Accept events.\n",
|
||||||
|
"scope.json": json.dumps(
|
||||||
|
{
|
||||||
|
"kind": "workload",
|
||||||
|
"stated": {"intent": {"statement": "Accept events."}},
|
||||||
|
"to_decide": [],
|
||||||
|
"to_ask": [],
|
||||||
|
"not_applicable": [],
|
||||||
|
}
|
||||||
|
),
|
||||||
"requirements.json": json.dumps(
|
"requirements.json": json.dumps(
|
||||||
[
|
[
|
||||||
{"id": "FR-1", "kind": "functional", "text": "Accept events."},
|
{"id": "FR-1", "kind": "functional", "text": "Accept events."},
|
||||||
@@ -105,6 +115,37 @@ SAMPLE_DATA: dict[str, str] = {
|
|||||||
{"check": "K-2", "file": "terraform/main.tf", "address": "example.sa"},
|
{"check": "K-2", "file": "terraform/main.tf", "address": "example.sa"},
|
||||||
]
|
]
|
||||||
),
|
),
|
||||||
|
"work-breakdown.json": json.dumps(
|
||||||
|
{
|
||||||
|
"work_packages": [
|
||||||
|
{
|
||||||
|
"id": "WP-1",
|
||||||
|
"name": "Ingress",
|
||||||
|
"parent": None,
|
||||||
|
"elements": ["E-1", "E-2"],
|
||||||
|
"scope": "The service and its identity.",
|
||||||
|
"produces": ["a running service"],
|
||||||
|
"skills": ["platform engineering"],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"activities": [
|
||||||
|
{
|
||||||
|
"id": "A-1",
|
||||||
|
"work_package": "WP-1",
|
||||||
|
"name": "Create the identity",
|
||||||
|
"days": 1,
|
||||||
|
"after": [],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "A-2",
|
||||||
|
"work_package": "WP-1",
|
||||||
|
"name": "Deploy the service",
|
||||||
|
"days": 2,
|
||||||
|
"after": ["A-1"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
),
|
||||||
"verdicts.json": json.dumps(
|
"verdicts.json": json.dumps(
|
||||||
{
|
{
|
||||||
"dry_run": {"verdict": "pass", "evidence": "terraform validate passed."},
|
"dry_run": {"verdict": "pass", "evidence": "terraform validate passed."},
|
||||||
|
|||||||
82
app/agent.py
82
app/agent.py
@@ -46,6 +46,7 @@ from app.nodes.router import (
|
|||||||
)
|
)
|
||||||
from app.providers import ProviderBundle, load_provider
|
from app.providers import ProviderBundle, load_provider
|
||||||
from app.recorder import DatabaseRecorder, ExecutionRecorder
|
from app.recorder import DatabaseRecorder, ExecutionRecorder
|
||||||
|
from app.scope import ESSENTIAL, Catalogue, Scope, read_scope, slug
|
||||||
from app.skills.agent_skills import discover_skills, skill_version
|
from app.skills.agent_skills import discover_skills, skill_version
|
||||||
from app.states.execution_state import check_customer_case
|
from app.states.execution_state import check_customer_case
|
||||||
from app.states.ledger import Ledger
|
from app.states.ledger import Ledger
|
||||||
@@ -57,6 +58,7 @@ from app.tools.local import TerraformRunner, local_tools
|
|||||||
from app.tools.mcp import McpCatalog, auth_from_config
|
from app.tools.mcp import McpCatalog, auth_from_config
|
||||||
from app.tools.requirements_review import RequirementsReview
|
from app.tools.requirements_review import RequirementsReview
|
||||||
from app.tools.schema_review import Reviews, SchemaReview
|
from app.tools.schema_review import Reviews, SchemaReview
|
||||||
|
from app.tools.scoping import Scoping, to_working
|
||||||
from app.tools.skill_routing import routable
|
from app.tools.skill_routing import routable
|
||||||
from app.tools.toolkit import make_tool_factory
|
from app.tools.toolkit import make_tool_factory
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH
|
from app.workflows.deliverables import DEPENDENCY_GRAPH
|
||||||
@@ -106,6 +108,9 @@ class ExecutionEngine:
|
|||||||
assess_at_end: bool = False,
|
assess_at_end: bool = False,
|
||||||
modules_available: bool = False,
|
modules_available: bool = False,
|
||||||
default_modules: bool = False,
|
default_modules: bool = False,
|
||||||
|
scope_catalogue: Catalogue | None = None,
|
||||||
|
scope_values: Mapping[str, Sequence[str]] | None = None,
|
||||||
|
scope_skill: str | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
required = {
|
required = {
|
||||||
ORCHESTRATOR,
|
ORCHESTRATOR,
|
||||||
@@ -115,6 +120,11 @@ class ExecutionEngine:
|
|||||||
if missing:
|
if missing:
|
||||||
raise ValueError(f"Missing roles: {', '.join(missing)}")
|
raise ValueError(f"Missing roles: {', '.join(missing)}")
|
||||||
self._max_revisions = max_revisions
|
self._max_revisions = max_revisions
|
||||||
|
# The catalogue a scope spec is read against (ADR-0008); None when this
|
||||||
|
# deployment's Cloud provider has none.
|
||||||
|
self._scope_catalogue = scope_catalogue
|
||||||
|
self._scope_values = dict(scope_values or {})
|
||||||
|
self._scope_skill = scope_skill
|
||||||
# Whether the Cloud provider approves any module sources, and whether an
|
# Whether the Cloud provider approves any module sources, and whether an
|
||||||
# Execution uses them when its caller does not say.
|
# Execution uses them when its caller does not say.
|
||||||
self._modules_available = modules_available
|
self._modules_available = modules_available
|
||||||
@@ -146,6 +156,9 @@ class ExecutionEngine:
|
|||||||
skill_references=skill_references,
|
skill_references=skill_references,
|
||||||
skill_threshold=skill_threshold,
|
skill_threshold=skill_threshold,
|
||||||
assess_at_end=assess_at_end,
|
assess_at_end=assess_at_end,
|
||||||
|
scoping=Scoping(scope_catalogue, dict(scope_values or {}), scope_skill)
|
||||||
|
if scope_catalogue is not None
|
||||||
|
else None,
|
||||||
).compile(checkpointer=checkpointer)
|
).compile(checkpointer=checkpointer)
|
||||||
|
|
||||||
async def start(
|
async def start(
|
||||||
@@ -154,13 +167,14 @@ class ExecutionEngine:
|
|||||||
*,
|
*,
|
||||||
session_id: str | None = None,
|
session_id: str | None = None,
|
||||||
customer_case: str | None = None,
|
customer_case: str | None = None,
|
||||||
pre_approved: bool = False,
|
pre_approved: bool | None = None,
|
||||||
allow_cloud_read: bool | None = None,
|
allow_cloud_read: bool | None = None,
|
||||||
execution_id: str | None = None,
|
execution_id: str | None = None,
|
||||||
scripted_answers: Sequence[Sequence[str]] = (),
|
scripted_answers: Sequence[Sequence[str]] = (),
|
||||||
baseline: str | None = None,
|
baseline: str | None = None,
|
||||||
overlay_skills: Sequence[str] = (),
|
overlay_skills: Sequence[str] = (),
|
||||||
terraform_modules: bool | None = None,
|
terraform_modules: bool | None = None,
|
||||||
|
scope: str | Mapping[str, Any] | None = None,
|
||||||
) -> Interaction:
|
) -> Interaction:
|
||||||
"""Start an Execution. `scripted_answers` are (regex, answer[, asker]) entries a
|
"""Start an Execution. `scripted_answers` are (regex, answer[, asker]) entries a
|
||||||
pre-approved Execution uses to answer questions, for evaluation runs.
|
pre-approved Execution uses to answer questions, for evaluation runs.
|
||||||
@@ -180,7 +194,50 @@ class ExecutionEngine:
|
|||||||
does as its Baseline did and any other Execution takes the deployment's
|
does as its Baseline did and any other Execution takes the deployment's
|
||||||
default, which is plain resources. Opting in is refused when the provider
|
default, which is plain resources. Opting in is refused when the provider
|
||||||
approves no sources.
|
approves no sources.
|
||||||
|
|
||||||
|
`scope` is a scope spec (ADR-0008), as YAML text or already parsed. It may
|
||||||
|
be complete or partly filled. The start is refused when it names a field
|
||||||
|
the catalogue does not have or holds a value of the wrong type, and, for a
|
||||||
|
pre-approved Execution, when an essential field is not settled: nobody is
|
||||||
|
there to ask. What the spec states under `run` applies where the caller
|
||||||
|
gave no value of their own. Its statement of intent is the request when
|
||||||
|
none is given.
|
||||||
"""
|
"""
|
||||||
|
read = self._read_scope(scope) if scope is not None else None
|
||||||
|
if read is not None:
|
||||||
|
run = read.values
|
||||||
|
if pre_approved is None:
|
||||||
|
pre_approved = bool(run.get("run.pre_approved", False))
|
||||||
|
if allow_cloud_read is None and "run.allow_cloud_read" in run:
|
||||||
|
allow_cloud_read = bool(run["run.allow_cloud_read"])
|
||||||
|
if terraform_modules is None and "run.terraform_modules" in run:
|
||||||
|
terraform_modules = bool(run["run.terraform_modules"])
|
||||||
|
overlay_skills = [*overlay_skills, *run.get("run.overlay_skills", [])]
|
||||||
|
if (
|
||||||
|
customer_case
|
||||||
|
and read.customer_case
|
||||||
|
and slug(customer_case) != (read.customer_case)
|
||||||
|
):
|
||||||
|
raise ValueError(
|
||||||
|
f"The scope spec is for {read.customer_case}, and this was "
|
||||||
|
f"started for {customer_case}."
|
||||||
|
)
|
||||||
|
customer_case = customer_case or read.customer_case
|
||||||
|
# The spec's statement is the customer's; it is what gets recorded.
|
||||||
|
workflow_request = read.statement or workflow_request
|
||||||
|
missing = read.gaps(ESSENTIAL)
|
||||||
|
if pre_approved and missing:
|
||||||
|
raise ValueError(
|
||||||
|
"The scope spec leaves essential fields unsettled, and a "
|
||||||
|
"pre-approved Execution has nobody to ask: "
|
||||||
|
+ ", ".join(gap["field"] for gap in missing)
|
||||||
|
+ ". Give each a value, or `decide` to leave it to the agent."
|
||||||
|
)
|
||||||
|
pre_approved = bool(pre_approved)
|
||||||
|
if not workflow_request.strip():
|
||||||
|
raise ValueError(
|
||||||
|
"A request is needed: text, or a scope spec with a statement."
|
||||||
|
)
|
||||||
customer_case = check_customer_case(customer_case)
|
customer_case = check_customer_case(customer_case)
|
||||||
execution_id = execution_id or str(uuid.uuid4())
|
execution_id = execution_id or str(uuid.uuid4())
|
||||||
if await self._exists(execution_id):
|
if await self._exists(execution_id):
|
||||||
@@ -224,6 +281,8 @@ class ExecutionEngine:
|
|||||||
session_id=session_id,
|
session_id=session_id,
|
||||||
workflow_request=workflow_request,
|
workflow_request=workflow_request,
|
||||||
baseline=baseline or None,
|
baseline=baseline or None,
|
||||||
|
scope_supplied=scope if isinstance(scope, str) else None,
|
||||||
|
scope_completed=read.completed(read.status(pre_approved)) if read else None,
|
||||||
)
|
)
|
||||||
initial: dict[str, Any] = {
|
initial: dict[str, Any] = {
|
||||||
"execution_id": execution_id,
|
"execution_id": execution_id,
|
||||||
@@ -236,6 +295,17 @@ class ExecutionEngine:
|
|||||||
# Overlay Skills this Execution loads, each with the version it had.
|
# Overlay Skills this Execution loads, each with the version it had.
|
||||||
"overlay_skills": {name: self._overlays[name] for name in chosen},
|
"overlay_skills": {name: self._overlays[name] for name in chosen},
|
||||||
"terraform_modules": terraform_modules,
|
"terraform_modules": terraform_modules,
|
||||||
|
# What the customer's scope spec settles and leaves open; None without one.
|
||||||
|
"scope": {
|
||||||
|
"skill": self._scope_skill,
|
||||||
|
"status": read.status(pre_approved),
|
||||||
|
"readiness": read.readiness(),
|
||||||
|
"context": read.for_context(),
|
||||||
|
# What the scope analyst starts from and completes.
|
||||||
|
"working": to_working(read),
|
||||||
|
}
|
||||||
|
if read
|
||||||
|
else None,
|
||||||
"risks": [],
|
"risks": [],
|
||||||
"pre_approved": pre_approved,
|
"pre_approved": pre_approved,
|
||||||
"allow_cloud_read": allow_cloud_read,
|
"allow_cloud_read": allow_cloud_read,
|
||||||
@@ -274,6 +344,13 @@ class ExecutionEngine:
|
|||||||
await self._graph.ainvoke(initial, self._config(execution_id))
|
await self._graph.ainvoke(initial, self._config(execution_id))
|
||||||
return await self.interaction(execution_id)
|
return await self.interaction(execution_id)
|
||||||
|
|
||||||
|
def _read_scope(self, supplied: str | Mapping[str, Any]) -> Scope:
|
||||||
|
if self._scope_catalogue is None:
|
||||||
|
raise ValueError("This deployment has no scope spec catalogue.")
|
||||||
|
return read_scope(
|
||||||
|
supplied, self._scope_catalogue, provider_values=self._scope_values
|
||||||
|
)
|
||||||
|
|
||||||
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
|
async def resume(self, execution_id: str, reply: Mapping[str, Any]) -> Interaction:
|
||||||
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
|
"""Answer the Execution's current prompt: {"reply"} | {"approval"} | {"permission"}."""
|
||||||
current = await self.interaction(execution_id)
|
current = await self.interaction(execution_id)
|
||||||
@@ -517,6 +594,9 @@ async def open_assembly(
|
|||||||
else None,
|
else None,
|
||||||
skill_threshold=settings.JEV_SKILL_THRESHOLD,
|
skill_threshold=settings.JEV_SKILL_THRESHOLD,
|
||||||
assess_at_end=settings.JEV_ASSESS_EXECUTIONS,
|
assess_at_end=settings.JEV_ASSESS_EXECUTIONS,
|
||||||
|
scope_catalogue=bundle.scope_catalogue() if bundle.scope_skill else None,
|
||||||
|
scope_values=bundle.scope_values,
|
||||||
|
scope_skill=bundle.scope_skill,
|
||||||
modules_available=bool(bundle.terraform_module_namespaces),
|
modules_available=bool(bundle.terraform_module_namespaces),
|
||||||
default_modules=settings.TERRAFORM_MODULES,
|
default_modules=settings.TERRAFORM_MODULES,
|
||||||
overlays={
|
overlays={
|
||||||
|
|||||||
@@ -14,15 +14,20 @@ nothing was delivered for it. history.json is the index of the case.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
from collections.abc import Mapping
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
from app.states.execution_state import case_dir
|
from app.states.execution_state import case_dir
|
||||||
|
|
||||||
INTENT_DIR = "intent"
|
INTENT_DIR = "intent"
|
||||||
HISTORY_FILENAME = "history.json"
|
HISTORY_FILENAME = "history.json"
|
||||||
IN_PROGRESS = "in_progress"
|
IN_PROGRESS = "in_progress"
|
||||||
|
SCOPE_SUPPLIED = ".scope.yaml"
|
||||||
|
SCOPE_COMPLETED = ".scope.completed.yaml"
|
||||||
|
|
||||||
|
|
||||||
def intent_dir(output_root: Path, customer_case: str) -> Path:
|
def intent_dir(output_root: Path, customer_case: str) -> Path:
|
||||||
@@ -50,6 +55,8 @@ def record_intent(
|
|||||||
session_id: str,
|
session_id: str,
|
||||||
workflow_request: str,
|
workflow_request: str,
|
||||||
baseline: str | None = None,
|
baseline: str | None = None,
|
||||||
|
scope_supplied: str | None = None,
|
||||||
|
scope_completed: Mapping[str, Any] | None = None,
|
||||||
) -> int:
|
) -> int:
|
||||||
"""Record a new statement of the customer's intent and return its version.
|
"""Record a new statement of the customer's intent and return its version.
|
||||||
|
|
||||||
@@ -71,6 +78,19 @@ def record_intent(
|
|||||||
f"## What the customer wants\n\n{workflow_request.strip()}\n",
|
f"## What the customer wants\n\n{workflow_request.strip()}\n",
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
scope_files = None
|
||||||
|
if scope_completed is not None:
|
||||||
|
# The spec as supplied, unaltered, and the completed copy beside it.
|
||||||
|
scope_files = {"completed": f"{INTENT_DIR}/{version:03d}{SCOPE_COMPLETED}"}
|
||||||
|
(folder / f"{version:03d}{SCOPE_COMPLETED}").write_text(
|
||||||
|
yaml.safe_dump(dict(scope_completed), sort_keys=False, allow_unicode=True),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
if scope_supplied:
|
||||||
|
scope_files["supplied"] = f"{INTENT_DIR}/{version:03d}{SCOPE_SUPPLIED}"
|
||||||
|
(folder / f"{version:03d}{SCOPE_SUPPLIED}").write_text(
|
||||||
|
scope_supplied, encoding="utf-8"
|
||||||
|
)
|
||||||
history["intents"].append(
|
history["intents"].append(
|
||||||
{
|
{
|
||||||
"version": version,
|
"version": version,
|
||||||
@@ -82,6 +102,7 @@ def record_intent(
|
|||||||
"status": IN_PROGRESS,
|
"status": IN_PROGRESS,
|
||||||
"execution": None,
|
"execution": None,
|
||||||
"baseline": baseline,
|
"baseline": baseline,
|
||||||
|
"scope": scope_files,
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
_write_history(folder, history)
|
_write_history(folder, history)
|
||||||
|
|||||||
@@ -62,8 +62,10 @@ def create_app(
|
|||||||
except ValueError:
|
except ValueError:
|
||||||
body = {}
|
body = {}
|
||||||
workflow_request = str(body.get("request") or "").strip()
|
workflow_request = str(body.get("request") or "").strip()
|
||||||
if not workflow_request:
|
if not workflow_request and not body.get("scope"):
|
||||||
return JSONResponse({"error": "'request' is required"}, status_code=400)
|
return JSONResponse(
|
||||||
|
{"error": "'request' or a 'scope' spec is required"}, status_code=400
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
interaction = await assembly.engine.start(
|
interaction = await assembly.engine.start(
|
||||||
workflow_request,
|
workflow_request,
|
||||||
@@ -72,6 +74,7 @@ def create_app(
|
|||||||
baseline=body.get("baseline"),
|
baseline=body.get("baseline"),
|
||||||
overlay_skills=body.get("overlay_skills") or (),
|
overlay_skills=body.get("overlay_skills") or (),
|
||||||
terraform_modules=body.get("terraform_modules"),
|
terraform_modules=body.get("terraform_modules"),
|
||||||
|
scope=body.get("scope"),
|
||||||
pre_approved=True,
|
pre_approved=True,
|
||||||
allow_cloud_read=bool(body.get("allow_cloud_read", False)),
|
allow_cloud_read=bool(body.get("allow_cloud_read", False)),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ append outcomes that the next Orchestrator turn applies in order.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import copy
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
@@ -17,6 +18,7 @@ from dataclasses import dataclass
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, cast
|
from typing import Any, cast
|
||||||
|
|
||||||
|
import yaml
|
||||||
from langgraph.types import Command, Send, interrupt
|
from langgraph.types import Command, Send, interrupt
|
||||||
|
|
||||||
from app.adk.runtime import (
|
from app.adk.runtime import (
|
||||||
@@ -37,16 +39,22 @@ from app.adk.runtime import (
|
|||||||
TurnEnded,
|
TurnEnded,
|
||||||
)
|
)
|
||||||
from app.baseline import WORKING_FILENAME
|
from app.baseline import WORKING_FILENAME
|
||||||
from app.customer_case import IN_PROGRESS, close_intent
|
from app.customer_case import (
|
||||||
|
IN_PROGRESS,
|
||||||
|
SCOPE_COMPLETED,
|
||||||
|
close_intent,
|
||||||
|
intent_dir,
|
||||||
|
)
|
||||||
from app.nodes.router import ModelOrchestrator, OrchestratorPolicy, TurnView
|
from app.nodes.router import ModelOrchestrator, OrchestratorPolicy, TurnView
|
||||||
from app.recorder import ExecutionRecorder
|
from app.recorder import ExecutionRecorder
|
||||||
|
from app.scope import APPROVED, CASE_FIELD, LOCKED, UNDER_REVIEW
|
||||||
from app.states.execution_state import (
|
from app.states.execution_state import (
|
||||||
UNASSIGNED_CASE,
|
UNASSIGNED_CASE,
|
||||||
ExecutionState,
|
ExecutionState,
|
||||||
execution_dir,
|
execution_dir,
|
||||||
lessons_file,
|
lessons_file,
|
||||||
)
|
)
|
||||||
from app.states.ledger import DeliverableStatus, Ledger, LedgerRefusal
|
from app.states.ledger import DeliverableStatus, Ledger, LedgerRefusal, PhaseStatus
|
||||||
from app.states.lessons import collect_lesson_candidates, refusal
|
from app.states.lessons import collect_lesson_candidates, refusal
|
||||||
from app.tools import workspace
|
from app.tools import workspace
|
||||||
from app.tools.assessment import assess
|
from app.tools.assessment import assess
|
||||||
@@ -61,9 +69,12 @@ from app.tools.pillars import (
|
|||||||
customer_questions,
|
customer_questions,
|
||||||
pillars_touched,
|
pillars_touched,
|
||||||
)
|
)
|
||||||
|
from app.tools.scoping import SCOPE_KEY, Scoping, working_from_completed
|
||||||
from app.tools.skill_routing import read_first
|
from app.tools.skill_routing import read_first
|
||||||
from app.workflows.chain import (
|
from app.workflows.chain import (
|
||||||
IMPLEMENTATION_FILE,
|
IMPLEMENTATION_FILE,
|
||||||
|
SCOPE_DATA_FILE,
|
||||||
|
SCOPE_FILE,
|
||||||
VERDICTS_FILE,
|
VERDICTS_FILE,
|
||||||
unimplementable_checks,
|
unimplementable_checks,
|
||||||
validation_outcome,
|
validation_outcome,
|
||||||
@@ -73,9 +84,11 @@ from app.workflows.deliverables import (
|
|||||||
DEPENDENCY_GRAPH,
|
DEPENDENCY_GRAPH,
|
||||||
PHASE_ORDER,
|
PHASE_ORDER,
|
||||||
DeliverableKind,
|
DeliverableKind,
|
||||||
|
PhaseId,
|
||||||
dependencies_for,
|
dependencies_for,
|
||||||
)
|
)
|
||||||
from app.workflows.target_architecture import GRAPH_ATTACHMENT
|
from app.workflows.target_architecture import GRAPH_ATTACHMENT
|
||||||
|
from app.workflows.work_breakdown import summary as plan_summary
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -132,6 +145,7 @@ def make_execution_nodes(
|
|||||||
skill_references: Mapping[str, Mapping[str, Mapping[str, str]]] | None = None,
|
skill_references: Mapping[str, Mapping[str, Mapping[str, str]]] | None = None,
|
||||||
skill_threshold: float = 1.25,
|
skill_threshold: float = 1.25,
|
||||||
assess_at_end: bool = False,
|
assess_at_end: bool = False,
|
||||||
|
scoping: Scoping | None = None,
|
||||||
) -> ExecutionNodes:
|
) -> ExecutionNodes:
|
||||||
tool_factory = tool_factory or _no_tools
|
tool_factory = tool_factory or _no_tools
|
||||||
# How an Orchestrator turn is decided. The default drives the tools from the
|
# How an Orchestrator turn is decided. The default drives the tools from the
|
||||||
@@ -481,6 +495,10 @@ def make_execution_nodes(
|
|||||||
role = dataclasses.replace(
|
role = dataclasses.replace(
|
||||||
role, skills=(*role.skills, *(n for n in named if n not in role.skills))
|
role, skills=(*role.skills, *(n for n in named if n not in role.skills))
|
||||||
)
|
)
|
||||||
|
scope_skill = scoping.skill if scoping else task_input.get("scope_skill")
|
||||||
|
if scope_skill and scope_skill not in role.skills:
|
||||||
|
# An Execution with a scope spec: every Specialist can read what it means.
|
||||||
|
role = dataclasses.replace(role, skills=(*role.skills, scope_skill))
|
||||||
opted_in = bool(task_input.get("terraform_modules"))
|
opted_in = bool(task_input.get("terraform_modules"))
|
||||||
MODULES_OPTED_IN.set(opted_in)
|
MODULES_OPTED_IN.set(opted_in)
|
||||||
if opted_in and role.modules_skill:
|
if opted_in and role.modules_skill:
|
||||||
@@ -508,6 +526,16 @@ def make_execution_nodes(
|
|||||||
_start_from(kind, before, working, task_input.get("baseline_basis"))
|
_start_from(kind, before, working, task_input.get("baseline_basis"))
|
||||||
# And from its working material: the advice it was written from.
|
# And from its working material: the advice it was written from.
|
||||||
working.update(task_input.get("baseline_working") or {})
|
working.update(task_input.get("baseline_working") or {})
|
||||||
|
if kind is DeliverableKind.SCOPE and scoping and SCOPE_KEY not in working:
|
||||||
|
# The spec the customer supplied, or an empty one to draft into.
|
||||||
|
delivered = ((before or {}).get("attachments") or {}).get(SCOPE_FILE)
|
||||||
|
working[SCOPE_KEY] = (
|
||||||
|
# A Change request starts from the scope that was delivered.
|
||||||
|
(working_from_completed(delivered, scoping) if delivered else None)
|
||||||
|
or copy.deepcopy(
|
||||||
|
task_input.get("scope_working") or {"values": {}, "states": {}}
|
||||||
|
)
|
||||||
|
)
|
||||||
source = DEPENDENCY_GRAPH[kind].working_copy
|
source = DEPENDENCY_GRAPH[kind].working_copy
|
||||||
if source and source is not kind:
|
if source and source is not kind:
|
||||||
# The tools work on the files as they were last submitted.
|
# The tools work on the files as they were last submitted.
|
||||||
@@ -567,6 +595,7 @@ def make_execution_nodes(
|
|||||||
library=task_input.get("library"),
|
library=task_input.get("library"),
|
||||||
review=review if submission_review else None,
|
review=review if submission_review else None,
|
||||||
baseline=before,
|
baseline=before,
|
||||||
|
scoping=scoping,
|
||||||
# With a user present, their first sight of a Deliverable comes
|
# With a user present, their first sight of a Deliverable comes
|
||||||
# before it is submitted, whatever the Specialist intends.
|
# before it is submitted, whatever the Specialist intends.
|
||||||
confirm_first=confirm_drafts
|
confirm_first=confirm_drafts
|
||||||
@@ -908,6 +937,10 @@ def _specialist_input(
|
|||||||
"allow_cloud_read": state.get("allow_cloud_read"),
|
"allow_cloud_read": state.get("allow_cloud_read"),
|
||||||
# Whether this Execution opted into published Terraform modules.
|
# Whether this Execution opted into published Terraform modules.
|
||||||
"terraform_modules": bool(state.get("terraform_modules")),
|
"terraform_modules": bool(state.get("terraform_modules")),
|
||||||
|
"scope_skill": (state.get("scope") or {}).get("skill"),
|
||||||
|
# The spec as supplied, for the scope analyst to complete; with the Customer
|
||||||
|
# case the Execution belongs to, which is not the customer's to restate.
|
||||||
|
"scope_working": _scope_seed(state) if kind is DeliverableKind.SCOPE else None,
|
||||||
"working": dict(state.get("working", {}).get(kind.value, {})),
|
"working": dict(state.get("working", {}).get(kind.value, {})),
|
||||||
# Whether the user has been shown a draft of this Deliverable. Answering some
|
# Whether the user has been shown a draft of this Deliverable. Answering some
|
||||||
# other question about it, such as what to do after it stopped, is not that.
|
# other question about it, such as what to do after it stopped, is not that.
|
||||||
@@ -937,6 +970,10 @@ def _specialist_input(
|
|||||||
# Skills carrying the organisation's standards or this engagement's
|
# Skills carrying the organisation's standards or this engagement's
|
||||||
# constraints. Where one disagrees with general guidance, it wins.
|
# constraints. Where one disagrees with general guidance, it wins.
|
||||||
"overlay_skills": sorted(state.get("overlay_skills") or {}),
|
"overlay_skills": sorted(state.get("overlay_skills") or {}),
|
||||||
|
# What the customer's scope spec states, leaves to the agent, and
|
||||||
|
# leaves unknown. None when no spec was supplied.
|
||||||
|
"scope": _agreed_scope(deliverables)
|
||||||
|
or (state.get("scope") or {}).get("context"),
|
||||||
# False: infrastructure code is plain resources. True: a published
|
# False: infrastructure code is plain resources. True: a published
|
||||||
# module may be used where the modules Skill says one fits.
|
# module may be used where the modules Skill says one fits.
|
||||||
"terraform_modules": bool(state.get("terraform_modules")),
|
"terraform_modules": bool(state.get("terraform_modules")),
|
||||||
@@ -1000,6 +1037,32 @@ def _requirement_changes(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _scope_seed(state: ExecutionState) -> dict[str, Any]:
|
||||||
|
"""The scope spec the analyst starts from: what was supplied, and the Customer
|
||||||
|
case this Execution already belongs to."""
|
||||||
|
seed = copy.deepcopy(
|
||||||
|
(state.get("scope") or {}).get("working") or {"values": {}, "states": {}}
|
||||||
|
)
|
||||||
|
seed.setdefault("values", {}).setdefault(CASE_FIELD, _case(state))
|
||||||
|
return seed
|
||||||
|
|
||||||
|
|
||||||
|
def _agreed_scope(
|
||||||
|
deliverables: Mapping[str, Mapping[str, Any]],
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""The scope every later Specialist reads: the scope Deliverable's data file."""
|
||||||
|
text = (
|
||||||
|
(deliverables.get(DeliverableKind.SCOPE.value) or {})
|
||||||
|
.get("attachments", {})
|
||||||
|
.get(SCOPE_DATA_FILE, "")
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
data = json.loads(text) if text else None
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
return data if isinstance(data, dict) else None
|
||||||
|
|
||||||
|
|
||||||
async def _reading_hints(
|
async def _reading_hints(
|
||||||
state: ExecutionState,
|
state: ExecutionState,
|
||||||
ledger: Ledger,
|
ledger: Ledger,
|
||||||
@@ -1236,6 +1299,22 @@ def _case(state: ExecutionState) -> str:
|
|||||||
return state.get("customer_case") or UNASSIGNED_CASE
|
return state.get("customer_case") or UNASSIGNED_CASE
|
||||||
|
|
||||||
|
|
||||||
|
def _scope_with_status(text: str, status: str) -> tuple[str, dict[str, Any] | None]:
|
||||||
|
"""A completed scope spec with its status set, and what the Execution's record
|
||||||
|
says of it. Unchanged, with no record, when it cannot be read."""
|
||||||
|
try:
|
||||||
|
document = yaml.safe_load(text)
|
||||||
|
except yaml.YAMLError:
|
||||||
|
return text, None
|
||||||
|
if not isinstance(document, dict):
|
||||||
|
return text, None
|
||||||
|
document.setdefault("metadata", {})["status"] = status
|
||||||
|
return (
|
||||||
|
yaml.safe_dump(document, sort_keys=False, allow_unicode=True),
|
||||||
|
{"status": status, "readiness": document.get("readiness")},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _written_before(out_dir: Path) -> set[str]:
|
def _written_before(out_dir: Path) -> set[str]:
|
||||||
"""The files an earlier write of this Execution's folder recorded."""
|
"""The files an earlier write of this Execution's folder recorded."""
|
||||||
try:
|
try:
|
||||||
@@ -1335,6 +1414,39 @@ def _write_outputs(
|
|||||||
indent=2,
|
indent=2,
|
||||||
ensure_ascii=False,
|
ensure_ascii=False,
|
||||||
)
|
)
|
||||||
|
plan = deliverables.get(DeliverableKind.WORK_BREAKDOWN.value)
|
||||||
|
plan_record: dict[str, Any] | None = None
|
||||||
|
if plan:
|
||||||
|
plan_record = plan_summary(plan.get("attachments", {}), deliverables) or None
|
||||||
|
timeline = str(plan.get("content", "")).partition("\n## Timeline\n")[2]
|
||||||
|
if guide and timeline:
|
||||||
|
# The guide and the plan are written side by side, so neither holds the
|
||||||
|
# other. The plan's timeline is added to the guide here, from the plan.
|
||||||
|
appendix = (
|
||||||
|
"\n\n## Delivery plan\n\n"
|
||||||
|
f"From the work breakdown structure (`deliverables/"
|
||||||
|
f"{DeliverableKind.WORK_BREAKDOWN.value}.md`), which has the work "
|
||||||
|
"packages and the full activity sequence.\n\n" + timeline.strip() + "\n"
|
||||||
|
)
|
||||||
|
files[GUIDE_FILENAME] = files[GUIDE_FILENAME].rstrip() + appendix
|
||||||
|
guide_document = f"deliverables/{DeliverableKind.SOLUTION_GUIDE.value}.md"
|
||||||
|
files[guide_document] = files[guide_document].rstrip() + appendix
|
||||||
|
scope_record: dict[str, Any] | None = None
|
||||||
|
if SCOPE_FILE in files:
|
||||||
|
# The completed scope spec says where it stands, here and beside the Intent.
|
||||||
|
approved = ledger.phase_status(PhaseId.SCOPING) is PhaseStatus.APPROVED
|
||||||
|
files[SCOPE_FILE], scope_record = _scope_with_status(
|
||||||
|
files[SCOPE_FILE],
|
||||||
|
LOCKED if status == "completed" else APPROVED if approved else UNDER_REVIEW,
|
||||||
|
)
|
||||||
|
version = state.get("intent_version")
|
||||||
|
if version:
|
||||||
|
copy_path = (
|
||||||
|
intent_dir(output_root, customer_case)
|
||||||
|
/ f"{int(version):03d}{SCOPE_COMPLETED}"
|
||||||
|
)
|
||||||
|
copy_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
copy_path.write_text(files[SCOPE_FILE], encoding="utf-8")
|
||||||
# The folder is written more than once as an Execution goes. A file an earlier
|
# The folder is written more than once as an Execution goes. A file an earlier
|
||||||
# write left that this one does not have, such as a Terraform file removed in a
|
# write left that this one does not have, such as a Terraform file removed in a
|
||||||
# Revision, must not stay beside the files that replaced it.
|
# Revision, must not stay beside the files that replaced it.
|
||||||
@@ -1357,6 +1469,10 @@ def _write_outputs(
|
|||||||
"overlay_skills": dict(state.get("overlay_skills") or {}),
|
"overlay_skills": dict(state.get("overlay_skills") or {}),
|
||||||
# Whether its infrastructure code was allowed published modules.
|
# Whether its infrastructure code was allowed published modules.
|
||||||
"terraform_modules": bool(state.get("terraform_modules")),
|
"terraform_modules": bool(state.get("terraform_modules")),
|
||||||
|
# Where the agreed scope stands and how much of it the customer stated.
|
||||||
|
"scope": scope_record,
|
||||||
|
# The plan in a few numbers: its length, and whether it fits the window.
|
||||||
|
"delivery_plan": plan_record,
|
||||||
"status": status,
|
"status": status,
|
||||||
"warnings": list(state.get("warnings", [])),
|
"warnings": list(state.get("warnings", [])),
|
||||||
# Flags a person should look at; nothing was done about them.
|
# Flags a person should look at; nothing was done about them.
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ tools, so the Ledger refuses anything that is not allowed, whatever was decided.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
import logging
|
import logging
|
||||||
import re
|
import re
|
||||||
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
||||||
@@ -37,6 +38,7 @@ from app.workflows.chain import (
|
|||||||
IMPLEMENTATION_FILE,
|
IMPLEMENTATION_FILE,
|
||||||
RECOMMENDATIONS_FILE,
|
RECOMMENDATIONS_FILE,
|
||||||
REQUIREMENTS_FILE,
|
REQUIREMENTS_FILE,
|
||||||
|
SCOPE_DATA_FILE,
|
||||||
VERDICTS_FILE,
|
VERDICTS_FILE,
|
||||||
read_graph,
|
read_graph,
|
||||||
read_list,
|
read_list,
|
||||||
@@ -44,6 +46,7 @@ from app.workflows.chain import (
|
|||||||
validation_outcome,
|
validation_outcome,
|
||||||
)
|
)
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||||
|
from app.workflows.work_breakdown import summary as plan_summary
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -741,6 +744,30 @@ def approval_facts(phase: PhaseId, view: TurnView, r: Readings) -> dict[str, Any
|
|||||||
),
|
),
|
||||||
}
|
}
|
||||||
facts["counts"] = {name: count for name, count in counts.items() if count}
|
facts["counts"] = {name: count for name, count in counts.items() if count}
|
||||||
|
scope = (view.deliverables.get(DeliverableKind.SCOPE.value) or {}).get(
|
||||||
|
"attachments", {}
|
||||||
|
)
|
||||||
|
if phase is PhaseId.SCOPING and scope.get(SCOPE_DATA_FILE):
|
||||||
|
# What approving the scope commits to: how much the customer stated, how
|
||||||
|
# much was assumed for them, and what is still unknown.
|
||||||
|
try:
|
||||||
|
agreed = json.loads(scope[SCOPE_DATA_FILE])
|
||||||
|
except ValueError:
|
||||||
|
agreed = {}
|
||||||
|
facts["scope"] = {
|
||||||
|
"kind": agreed.get("kind"),
|
||||||
|
"assumed": [item.get("field") for item in agreed.get("assumed", [])],
|
||||||
|
"left_to_decide": [
|
||||||
|
item.get("field") for item in agreed.get("to_decide", [])
|
||||||
|
],
|
||||||
|
"unknown": [item.get("field") for item in agreed.get("to_ask", [])],
|
||||||
|
}
|
||||||
|
plan = view.deliverables.get(DeliverableKind.WORK_BREAKDOWN.value)
|
||||||
|
if plan and phase is PhaseId.PACKAGING:
|
||||||
|
# How long the build is planned to take, and whether that fits.
|
||||||
|
facts["delivery_plan"] = plan_summary(
|
||||||
|
plan.get("attachments", {}), view.deliverables
|
||||||
|
)
|
||||||
graph = read_graph(view.deliverables)
|
graph = read_graph(view.deliverables)
|
||||||
if graph is not None:
|
if graph is not None:
|
||||||
facts["target_architecture"] = graph.summary()
|
facts["target_architecture"] = graph.summary()
|
||||||
@@ -776,6 +803,29 @@ def assembled_summary(phase: PhaseId, facts: Mapping[str, Any]) -> str:
|
|||||||
lines.append(
|
lines.append(
|
||||||
", ".join(f"{count} {name}" for name, count in counts.items()) + "."
|
", ".join(f"{count} {name}" for name, count in counts.items()) + "."
|
||||||
)
|
)
|
||||||
|
plan = facts.get("delivery_plan")
|
||||||
|
if plan:
|
||||||
|
fits = plan.get("within_window")
|
||||||
|
lines.append(
|
||||||
|
f"Delivery plan: {plan['activities']} activities in "
|
||||||
|
f"{plan['work_packages']} work packages, {plan['weeks']} weeks with a "
|
||||||
|
f"team of {plan['team_size']}"
|
||||||
|
+ (
|
||||||
|
""
|
||||||
|
if fits is None
|
||||||
|
else f", {'inside' if fits else 'over'} the "
|
||||||
|
f"{plan['window_weeks']}-week window"
|
||||||
|
)
|
||||||
|
+ "."
|
||||||
|
)
|
||||||
|
scope = facts.get("scope")
|
||||||
|
if scope:
|
||||||
|
lines.append(
|
||||||
|
f"Scope of a {scope.get('kind') or 'workload'}: "
|
||||||
|
f"{len(scope['assumed'])} fields assumed, "
|
||||||
|
f"{len(scope['unknown'])} still unknown. Requirements are derived "
|
||||||
|
"from it once you approve."
|
||||||
|
)
|
||||||
graph = facts.get("target_architecture")
|
graph = facts.get("target_architecture")
|
||||||
if graph:
|
if graph:
|
||||||
lines.append(
|
lines.append(
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ from typing import Any
|
|||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
from app.scope import Catalogue, load_catalogue
|
||||||
from app.skills.agent_skills import SkillFormatError, SkillMeta, discover_skills
|
from app.skills.agent_skills import SkillFormatError, SkillMeta, discover_skills
|
||||||
|
|
||||||
TIERS = ("general", "cloud_read")
|
TIERS = ("general", "cloud_read")
|
||||||
@@ -85,6 +86,17 @@ class ProviderBundle:
|
|||||||
# The Skill approved lessons are published in (ADR-0004); None when the provider
|
# The Skill approved lessons are published in (ADR-0004); None when the provider
|
||||||
# has none. A Specialist is given it once it holds lessons for its Deliverable.
|
# has none. A Specialist is given it once it holds lessons for its Deliverable.
|
||||||
lessons_skill: str | None = None
|
lessons_skill: str | None = None
|
||||||
|
# The Skill that holds the scope spec's catalogue (ADR-0008), and the values
|
||||||
|
# this provider allows for the fields the catalogue leaves to a provider.
|
||||||
|
scope_skill: str | None = None
|
||||||
|
scope_values: Mapping[str, tuple[str, ...]] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def scope_catalogue(self) -> Catalogue:
|
||||||
|
if not self.scope_skill:
|
||||||
|
raise ProviderConfigError(f"The {self.name} bundle names no scope Skill.")
|
||||||
|
return load_catalogue(
|
||||||
|
self.skills[self.scope_skill].path / "assets" / "catalogue.yaml"
|
||||||
|
)
|
||||||
|
|
||||||
def mcp_servers_for_role(
|
def mcp_servers_for_role(
|
||||||
self, role: str, include_preview: bool = True
|
self, role: str, include_preview: bool = True
|
||||||
@@ -233,6 +245,15 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
|
|||||||
if lessons_skill is not None and lessons_skill not in skills:
|
if lessons_skill is not None and lessons_skill not in skills:
|
||||||
raise fail(f"'lessons_skill' references unknown skill {lessons_skill!r}")
|
raise fail(f"'lessons_skill' references unknown skill {lessons_skill!r}")
|
||||||
|
|
||||||
|
scope = config.get("scope") or {}
|
||||||
|
scope_skill = scope.get("skill") or None
|
||||||
|
if scope_skill is not None and scope_skill not in skills:
|
||||||
|
raise fail(f"'scope.skill' references unknown skill {scope_skill!r}")
|
||||||
|
scope_values = {
|
||||||
|
str(path): tuple(str(value) for value in values or ())
|
||||||
|
for path, values in (scope.get("values") or {}).items()
|
||||||
|
}
|
||||||
|
|
||||||
return ProviderBundle(
|
return ProviderBundle(
|
||||||
name=name,
|
name=name,
|
||||||
skills_dirs=skills_dirs,
|
skills_dirs=skills_dirs,
|
||||||
@@ -256,4 +277,6 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
|
|||||||
or ()
|
or ()
|
||||||
),
|
),
|
||||||
lessons_skill=lessons_skill,
|
lessons_skill=lessons_skill,
|
||||||
|
scope_skill=scope_skill,
|
||||||
|
scope_values=scope_values,
|
||||||
)
|
)
|
||||||
|
|||||||
501
app/scope.py
Normal file
501
app/scope.py
Normal file
@@ -0,0 +1,501 @@
|
|||||||
|
"""The scope spec: what the customer wants and the facts a design depends on, as a
|
||||||
|
file (ADR-0008).
|
||||||
|
|
||||||
|
A spec may arrive complete, partly filled or not at all. This reads one against the
|
||||||
|
catalogue of fields a Skill holds, and says exactly what is settled and what is not.
|
||||||
|
No model is involved: whether a field is filled is a fact.
|
||||||
|
|
||||||
|
a value the customer states it supplied
|
||||||
|
ask, or absent not known yet a gap, with its question
|
||||||
|
decide the customer leaves the choice to the agent an assumption
|
||||||
|
not_applicable does not apply nothing asked or assumed
|
||||||
|
|
||||||
|
The file supplied is never altered. `Scope.completed()` is the copy written beside
|
||||||
|
it, with the sections the agent writes: assumptions, gaps, where each value came
|
||||||
|
from, and a count of each.
|
||||||
|
|
||||||
|
.venv/bin/python -m app.scope check <spec.yaml> [provider]
|
||||||
|
.venv/bin/python -m app.scope template [kind]
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from collections.abc import Mapping, Sequence
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
ASK, DECIDE, NOT_APPLICABLE = "ask", "decide", "not_applicable"
|
||||||
|
MARKERS = (ASK, DECIDE, NOT_APPLICABLE)
|
||||||
|
SUPPLIED, ASSUMED, UNKNOWN = "supplied", "assumed", "unknown"
|
||||||
|
# Where a settled value came from: the spec file, the customer's request in prose,
|
||||||
|
# an answer they gave in conversation, or the agent's own choice.
|
||||||
|
REQUEST, ANSWERED = "request", "answered"
|
||||||
|
SOURCES = (SUPPLIED, REQUEST, ANSWERED, ASSUMED)
|
||||||
|
ESSENTIAL, IMPORTANT, OPTIONAL = "essential", "important", "optional"
|
||||||
|
_SEVERITY = {ESSENTIAL: "HIGH", IMPORTANT: "MEDIUM", OPTIONAL: "LOW"}
|
||||||
|
DRAFT, UNDER_REVIEW, APPROVED, LOCKED = (
|
||||||
|
"DRAFT",
|
||||||
|
"UNDER_REVIEW",
|
||||||
|
"APPROVED",
|
||||||
|
"BASELINE_LOCKED",
|
||||||
|
)
|
||||||
|
KIND_FIELD = "engagement.kind"
|
||||||
|
CASE_FIELD = "metadata.customer_case"
|
||||||
|
STATEMENT_FIELD = "intent.statement"
|
||||||
|
# What the agent writes into the completed copy. A supplied file may still carry
|
||||||
|
# them from an earlier run; they are read as that run's and replaced.
|
||||||
|
_AGENT_WRITTEN = (
|
||||||
|
"schema_version",
|
||||||
|
"inferred_assumptions",
|
||||||
|
"discovery_gaps",
|
||||||
|
"provenance",
|
||||||
|
"readiness",
|
||||||
|
)
|
||||||
|
_AGENT_FIELDS = ("metadata.status",)
|
||||||
|
_SLUG = re.compile(r"[^a-z0-9]+")
|
||||||
|
|
||||||
|
|
||||||
|
class ScopeError(ValueError):
|
||||||
|
"""A scope spec that cannot be used, with every problem found in it."""
|
||||||
|
|
||||||
|
def __init__(self, problems: Sequence[str]) -> None:
|
||||||
|
self.problems = list(problems)
|
||||||
|
super().__init__("scope spec: " + "; ".join(self.problems))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FieldSpec:
|
||||||
|
path: str
|
||||||
|
type: str = "string"
|
||||||
|
tier: Any = OPTIONAL # a tier, or {kind: tier}
|
||||||
|
question: str = ""
|
||||||
|
default: Any = None
|
||||||
|
affects: str = ""
|
||||||
|
values: tuple[str, ...] = ()
|
||||||
|
provider_values: bool = False
|
||||||
|
keys: tuple[str, ...] = ()
|
||||||
|
optional_keys: tuple[str, ...] = ()
|
||||||
|
aliases: tuple[str, ...] = ()
|
||||||
|
# A choice a design must honour. Stated by the customer, it has to be carried
|
||||||
|
# by a requirement, and so by a Check on the Target architecture.
|
||||||
|
binds: bool = False
|
||||||
|
|
||||||
|
def tier_for(self, kind: str | None) -> str:
|
||||||
|
if isinstance(self.tier, Mapping):
|
||||||
|
return str(self.tier.get(kind, OPTIONAL))
|
||||||
|
return str(self.tier)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Catalogue:
|
||||||
|
schema_version: int
|
||||||
|
kinds: tuple[str, ...]
|
||||||
|
fields: Mapping[str, FieldSpec]
|
||||||
|
|
||||||
|
def aliases(self) -> dict[str, str]:
|
||||||
|
return {
|
||||||
|
alias: spec.path for spec in self.fields.values() for alias in spec.aliases
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def load_catalogue(path: Path) -> Catalogue:
|
||||||
|
data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
|
||||||
|
fields = {}
|
||||||
|
for name, raw in (data.get("fields") or {}).items():
|
||||||
|
raw = raw or {}
|
||||||
|
fields[name] = FieldSpec(
|
||||||
|
path=name,
|
||||||
|
type=str(raw.get("type", "string")),
|
||||||
|
tier=raw.get("tier", OPTIONAL),
|
||||||
|
question=" ".join(str(raw.get("question", "")).split()),
|
||||||
|
default=raw.get("default"),
|
||||||
|
affects=str(raw.get("affects", "")),
|
||||||
|
values=tuple(str(v) for v in raw.get("values") or ()),
|
||||||
|
provider_values=bool(raw.get("provider_values")),
|
||||||
|
keys=tuple(raw.get("keys") or ()),
|
||||||
|
optional_keys=tuple(raw.get("optional_keys") or ()),
|
||||||
|
aliases=tuple(raw.get("aliases") or ()),
|
||||||
|
binds=bool(raw.get("binds")),
|
||||||
|
)
|
||||||
|
return Catalogue(
|
||||||
|
schema_version=int(data.get("schema_version", 1)),
|
||||||
|
kinds=tuple(data.get("kinds") or ()),
|
||||||
|
fields=fields,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Scope:
|
||||||
|
"""A scope spec read against the catalogue."""
|
||||||
|
|
||||||
|
catalogue: Catalogue
|
||||||
|
kind: str | None
|
||||||
|
values: dict[str, Any] = field(default_factory=dict) # field -> stated value
|
||||||
|
states: dict[str, str] = field(default_factory=dict) # field -> marker
|
||||||
|
supplied_text: str = ""
|
||||||
|
# Where each value came from, when it was not the spec file; and what was said
|
||||||
|
# about it: the reason for an assumption, the question an answer was given to.
|
||||||
|
sources: dict[str, str] = field(default_factory=dict)
|
||||||
|
notes: dict[str, str] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def source(self, path: str) -> str:
|
||||||
|
return self.sources.get(path, SUPPLIED)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def customer_case(self) -> str | None:
|
||||||
|
value = self.values.get(CASE_FIELD)
|
||||||
|
return str(value) if value else None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def statement(self) -> str:
|
||||||
|
return str(self.values.get(STATEMENT_FIELD) or "").strip()
|
||||||
|
|
||||||
|
def tier(self, path: str) -> str:
|
||||||
|
return self.catalogue.fields[path].tier_for(self.kind)
|
||||||
|
|
||||||
|
def gaps(self, *tiers: str) -> list[dict[str, str]]:
|
||||||
|
"""What is not known, most pressing first: each with the question to ask."""
|
||||||
|
wanted = tiers or (ESSENTIAL, IMPORTANT, OPTIONAL)
|
||||||
|
found = [
|
||||||
|
{
|
||||||
|
"field": path,
|
||||||
|
"prompt": self.catalogue.fields[path].question or f"What is {path}?",
|
||||||
|
"severity": _SEVERITY[self.tier(path)],
|
||||||
|
}
|
||||||
|
for path, state in self.states.items()
|
||||||
|
if state == ASK and self.tier(path) in wanted
|
||||||
|
]
|
||||||
|
order = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
|
||||||
|
return sorted(found, key=lambda gap: order[gap["severity"]])
|
||||||
|
|
||||||
|
def assumptions(self) -> list[dict[str, str]]:
|
||||||
|
"""What the customer left to the agent, each with the default to take and
|
||||||
|
what a wrong choice would change."""
|
||||||
|
found: list[dict[str, str]] = []
|
||||||
|
for path in self.catalogue.fields:
|
||||||
|
chosen = path in self.values and self.source(path) == ASSUMED
|
||||||
|
if not chosen and self.states.get(path) != DECIDE:
|
||||||
|
continue
|
||||||
|
spec = self.catalogue.fields[path]
|
||||||
|
if chosen:
|
||||||
|
statement = f"Assumed {path} is {self.values[path]}."
|
||||||
|
if self.notes.get(path):
|
||||||
|
statement += f" {self.notes[path]}"
|
||||||
|
else:
|
||||||
|
default = spec.default if spec.default is not None else "its choice"
|
||||||
|
statement = (
|
||||||
|
f"The customer left {path} to the agent. Default: {default}."
|
||||||
|
)
|
||||||
|
found.append(
|
||||||
|
{
|
||||||
|
"id": f"ASSUMP-{len(found) + 1:02d}",
|
||||||
|
"field": path,
|
||||||
|
"statement": statement,
|
||||||
|
"impact_if_invalid": f"Changes {spec.affects}."
|
||||||
|
if spec.affects
|
||||||
|
else "Local to this field.",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return found
|
||||||
|
|
||||||
|
def readiness(self) -> dict[str, int]:
|
||||||
|
states = list(self.states.values())
|
||||||
|
came = [self.source(path) for path in self.values]
|
||||||
|
return {
|
||||||
|
SUPPLIED: came.count(SUPPLIED) + came.count(REQUEST),
|
||||||
|
ANSWERED: came.count(ANSWERED),
|
||||||
|
ASSUMED: came.count(ASSUMED) + states.count(DECIDE),
|
||||||
|
UNKNOWN: states.count(ASK),
|
||||||
|
NOT_APPLICABLE: states.count(NOT_APPLICABLE),
|
||||||
|
}
|
||||||
|
|
||||||
|
def status(self, pre_approved: bool = False) -> str:
|
||||||
|
"""Where the spec stands. With nobody to ask, what is important and unknown
|
||||||
|
is defaulted and listed, so only an essential gap keeps it a draft."""
|
||||||
|
if pre_approved:
|
||||||
|
return DRAFT if self.gaps(ESSENTIAL) else APPROVED
|
||||||
|
return DRAFT if self.gaps(ESSENTIAL, IMPORTANT) else UNDER_REVIEW
|
||||||
|
|
||||||
|
def completed(self, status: str) -> dict[str, Any]:
|
||||||
|
"""The completed copy: every field of the catalogue, and the agent's
|
||||||
|
sections."""
|
||||||
|
document: dict[str, Any] = {"schema_version": self.catalogue.schema_version}
|
||||||
|
for path in self.catalogue.fields:
|
||||||
|
value = self.values.get(path, self.states.get(path, ASK))
|
||||||
|
_put(document, path, value)
|
||||||
|
_put(document, "metadata.status", status)
|
||||||
|
provenance = {path: self.source(path) for path in self.values}
|
||||||
|
provenance |= {
|
||||||
|
path: {DECIDE: ASSUMED, ASK: UNKNOWN, NOT_APPLICABLE: NOT_APPLICABLE}[state]
|
||||||
|
for path, state in self.states.items()
|
||||||
|
}
|
||||||
|
document["inferred_assumptions"] = self.assumptions()
|
||||||
|
document["discovery_gaps"] = self.gaps()
|
||||||
|
document["provenance"] = dict(sorted(provenance.items()))
|
||||||
|
document["readiness"] = self.readiness()
|
||||||
|
return document
|
||||||
|
|
||||||
|
def for_context(self) -> dict[str, Any]:
|
||||||
|
"""What a Specialist is shown: what the customer stated, what is theirs to
|
||||||
|
choose, and what is unknown."""
|
||||||
|
stated: dict[str, Any] = {}
|
||||||
|
for path, value in self.values.items():
|
||||||
|
if path.startswith(("run.", "metadata.")) or self.source(path) == ASSUMED:
|
||||||
|
continue
|
||||||
|
_put(stated, path, value)
|
||||||
|
return {
|
||||||
|
"kind": self.kind,
|
||||||
|
"stated": stated,
|
||||||
|
# The stated fields a design must honour: each has to be carried by a
|
||||||
|
# requirement, which the requirements are checked for.
|
||||||
|
"binding": [
|
||||||
|
path
|
||||||
|
for path in self.values
|
||||||
|
if self.catalogue.fields[path].binds and self.source(path) != ASSUMED
|
||||||
|
],
|
||||||
|
# What the agent chose where the customer did not say. Not the
|
||||||
|
# customer's decision: it may be revisited when they answer.
|
||||||
|
"assumed": [
|
||||||
|
{"field": path, "value": value, "why": self.notes.get(path, "")}
|
||||||
|
for path, value in self.values.items()
|
||||||
|
if self.source(path) == ASSUMED and not path.startswith("run.")
|
||||||
|
],
|
||||||
|
"to_decide": [
|
||||||
|
{
|
||||||
|
"field": path,
|
||||||
|
"default": self.catalogue.fields[path].default,
|
||||||
|
"affects": self.catalogue.fields[path].affects,
|
||||||
|
}
|
||||||
|
for path, state in self.states.items()
|
||||||
|
if state == DECIDE and not path.startswith("run.")
|
||||||
|
],
|
||||||
|
"to_ask": [
|
||||||
|
gap for gap in self.gaps() if not gap["field"].startswith("run.")
|
||||||
|
],
|
||||||
|
"not_applicable": sorted(
|
||||||
|
path for path, state in self.states.items() if state == NOT_APPLICABLE
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _put(document: dict[str, Any], path: str, value: Any) -> None:
|
||||||
|
*parents, last = path.split(".")
|
||||||
|
node = document
|
||||||
|
for part in parents:
|
||||||
|
node = node.setdefault(part, {})
|
||||||
|
node[last] = value
|
||||||
|
|
||||||
|
|
||||||
|
def slug(reference: str) -> str:
|
||||||
|
"""A Customer case id from an engagement reference: `ENG-2026-UK-084` and
|
||||||
|
`eng-2026-uk-084` name the same case."""
|
||||||
|
return _SLUG.sub("-", reference.strip().lower()).strip("-")[:64]
|
||||||
|
|
||||||
|
|
||||||
|
def read_scope(
|
||||||
|
supplied: str | Mapping[str, Any],
|
||||||
|
catalogue: Catalogue,
|
||||||
|
*,
|
||||||
|
provider_values: Mapping[str, Sequence[str]] | None = None,
|
||||||
|
) -> Scope:
|
||||||
|
"""Read a scope spec. Raises ScopeError naming every problem: YAML that does not
|
||||||
|
parse, a field the catalogue does not have, a value of the wrong type."""
|
||||||
|
text = supplied if isinstance(supplied, str) else yaml.safe_dump(dict(supplied))
|
||||||
|
try:
|
||||||
|
data = yaml.safe_load(text) if isinstance(supplied, str) else dict(supplied)
|
||||||
|
except yaml.YAMLError as exc:
|
||||||
|
raise ScopeError([f"not valid YAML ({exc})"]) from exc
|
||||||
|
if not isinstance(data, Mapping):
|
||||||
|
raise ScopeError(["the spec must be a mapping of sections"])
|
||||||
|
|
||||||
|
problems: list[str] = []
|
||||||
|
leaves: dict[str, Any] = {}
|
||||||
|
aliases = catalogue.aliases()
|
||||||
|
prefixes = {
|
||||||
|
".".join(path.split(".")[:depth])
|
||||||
|
for path in catalogue.fields
|
||||||
|
for depth in range(1, len(path.split(".")))
|
||||||
|
}
|
||||||
|
|
||||||
|
def walk(node: Any, path: str) -> None:
|
||||||
|
target = aliases.get(path, path)
|
||||||
|
if path in _AGENT_FIELDS or path in _AGENT_WRITTEN:
|
||||||
|
return
|
||||||
|
if target in catalogue.fields:
|
||||||
|
if target in leaves and leaves[target] != node and path != target:
|
||||||
|
problems.append(f"{path} and {target} name the same thing and differ")
|
||||||
|
leaves[target] = node
|
||||||
|
return
|
||||||
|
if path in prefixes:
|
||||||
|
if isinstance(node, Mapping):
|
||||||
|
for key, value in node.items():
|
||||||
|
walk(value, f"{path}.{key}")
|
||||||
|
elif node in MARKERS or node is None:
|
||||||
|
# A whole section marked at once applies to every field in it.
|
||||||
|
for name in catalogue.fields:
|
||||||
|
if name.startswith(path + "."):
|
||||||
|
leaves.setdefault(name, node)
|
||||||
|
else:
|
||||||
|
problems.append(f"{path} is a section, not a value")
|
||||||
|
return
|
||||||
|
problems.append(f"{path} is not a field of the scope spec")
|
||||||
|
|
||||||
|
for key, value in data.items():
|
||||||
|
walk(value, str(key))
|
||||||
|
|
||||||
|
kind = leaves.get(KIND_FIELD)
|
||||||
|
kind = str(kind) if kind in catalogue.kinds else None
|
||||||
|
scope = Scope(catalogue, kind, supplied_text=text)
|
||||||
|
extra = dict(provider_values or {})
|
||||||
|
for path, spec in catalogue.fields.items():
|
||||||
|
value = leaves.get(path)
|
||||||
|
if value is None or value == ASK:
|
||||||
|
scope.states[path] = ASK
|
||||||
|
elif value in (DECIDE, NOT_APPLICABLE):
|
||||||
|
scope.states[path] = value
|
||||||
|
else:
|
||||||
|
fault = _fault(spec, value, extra.get(path, ()))
|
||||||
|
if fault:
|
||||||
|
problems.append(f"{path}: {fault}")
|
||||||
|
else:
|
||||||
|
scope.values[path] = _plain(value)
|
||||||
|
case = scope.values.get(CASE_FIELD)
|
||||||
|
if case is not None:
|
||||||
|
scope.values[CASE_FIELD] = slug(str(case))
|
||||||
|
if not scope.values[CASE_FIELD]:
|
||||||
|
problems.append(f"{CASE_FIELD}: needs letters or digits")
|
||||||
|
if problems:
|
||||||
|
raise ScopeError(problems)
|
||||||
|
return scope
|
||||||
|
|
||||||
|
|
||||||
|
def _plain(value: Any) -> Any:
|
||||||
|
"""A value as plain data: YAML reads a date as a date, and state must be JSON."""
|
||||||
|
if isinstance(value, (dt.date, dt.datetime)):
|
||||||
|
return value.isoformat()
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {str(k): _plain(v) for k, v in value.items()}
|
||||||
|
if isinstance(value, (list, tuple)):
|
||||||
|
return [_plain(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _fault(spec: FieldSpec, value: Any, provider_values: Sequence[str]) -> str:
|
||||||
|
"""Why a value does not fit its field; "" when it does."""
|
||||||
|
kind = spec.type
|
||||||
|
if kind in ("string", "text"):
|
||||||
|
return "" if isinstance(value, (str, int, float)) else "must be text"
|
||||||
|
if kind == "integer":
|
||||||
|
ok = isinstance(value, int) and not isinstance(value, bool)
|
||||||
|
return "" if ok else "must be a whole number"
|
||||||
|
if kind == "number":
|
||||||
|
ok = isinstance(value, (int, float)) and not isinstance(value, bool)
|
||||||
|
return "" if ok else "must be a number"
|
||||||
|
if kind == "boolean":
|
||||||
|
return "" if isinstance(value, bool) else "must be true or false"
|
||||||
|
if kind == "date":
|
||||||
|
if isinstance(value, (dt.date, dt.datetime)):
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
dt.date.fromisoformat(str(value))
|
||||||
|
except ValueError:
|
||||||
|
return "must be a date, as YYYY-MM-DD"
|
||||||
|
return ""
|
||||||
|
if kind == "enum":
|
||||||
|
allowed = (*spec.values, *provider_values)
|
||||||
|
if not allowed or str(value) in allowed:
|
||||||
|
return ""
|
||||||
|
return f"must be one of {', '.join(allowed)}"
|
||||||
|
if kind == "list":
|
||||||
|
return "" if isinstance(value, list) else "must be a list"
|
||||||
|
if kind == "records":
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return "must be a list of entries"
|
||||||
|
known = {*spec.keys, *spec.optional_keys}
|
||||||
|
for number, entry in enumerate(value, start=1):
|
||||||
|
if not isinstance(entry, Mapping):
|
||||||
|
return f"entry {number} must have named fields"
|
||||||
|
missing = [key for key in spec.keys if not entry.get(key)]
|
||||||
|
if missing:
|
||||||
|
return f"entry {number} needs {', '.join(missing)}"
|
||||||
|
unknown = sorted(set(map(str, entry)) - known)
|
||||||
|
if unknown:
|
||||||
|
return f"entry {number} has unknown fields {', '.join(unknown)}"
|
||||||
|
return ""
|
||||||
|
return f"has an unknown type {kind} in the catalogue"
|
||||||
|
|
||||||
|
|
||||||
|
def render_template(catalogue: Catalogue, kind: str | None = None) -> str:
|
||||||
|
"""A scope spec with every field set to ask, to fill in."""
|
||||||
|
document: dict[str, Any] = {}
|
||||||
|
for path, spec in catalogue.fields.items():
|
||||||
|
if path == KIND_FIELD and kind:
|
||||||
|
_put(document, path, kind)
|
||||||
|
elif spec.tier_for(kind) != OPTIONAL or kind is None:
|
||||||
|
_put(document, path, ASK)
|
||||||
|
header = (
|
||||||
|
"# Scope spec. Give each field a value, or: ask (not known yet), decide\n"
|
||||||
|
"# (you choose), not_applicable. A field left out is read as ask.\n"
|
||||||
|
)
|
||||||
|
return header + yaml.safe_dump(document, sort_keys=False, allow_unicode=True)
|
||||||
|
|
||||||
|
|
||||||
|
def render_reading(scope: Scope) -> str:
|
||||||
|
"""What a spec settles and what it leaves open, for a person."""
|
||||||
|
counts = scope.readiness()
|
||||||
|
lines = [
|
||||||
|
f"Engagement kind: {scope.kind or 'not stated'}",
|
||||||
|
(
|
||||||
|
f"{counts[SUPPLIED]} supplied, {counts[ASSUMED]} left to the agent, "
|
||||||
|
f"{counts[UNKNOWN]} unknown, {counts[NOT_APPLICABLE]} not applicable."
|
||||||
|
),
|
||||||
|
]
|
||||||
|
for severity, title in (
|
||||||
|
("HIGH", "Essential, needed before work can start"),
|
||||||
|
("MEDIUM", "Important, asked when someone is present"),
|
||||||
|
):
|
||||||
|
gaps = [gap for gap in scope.gaps() if gap["severity"] == severity]
|
||||||
|
if gaps:
|
||||||
|
lines += ["", f"{title}:"]
|
||||||
|
lines += [f" {gap['field']}: {gap['prompt']}" for gap in gaps]
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def main(arguments: Sequence[str]) -> int:
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.providers import load_provider
|
||||||
|
|
||||||
|
command, rest = (arguments[0], arguments[1:]) if arguments else ("", [])
|
||||||
|
if command == "template":
|
||||||
|
bundle = load_provider(get_settings().CLOUD_PROVIDER)
|
||||||
|
print(render_template(bundle.scope_catalogue(), rest[0] if rest else None))
|
||||||
|
return 0
|
||||||
|
if command == "check" and rest:
|
||||||
|
bundle = load_provider(
|
||||||
|
rest[1] if len(rest) > 1 else get_settings().CLOUD_PROVIDER
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
scope = read_scope(
|
||||||
|
Path(rest[0]).read_text(encoding="utf-8"),
|
||||||
|
bundle.scope_catalogue(),
|
||||||
|
provider_values=bundle.scope_values,
|
||||||
|
)
|
||||||
|
except ScopeError as refused:
|
||||||
|
print("This spec cannot be used:")
|
||||||
|
for problem in refused.problems:
|
||||||
|
print(f" {problem}")
|
||||||
|
return 1
|
||||||
|
print(render_reading(scope))
|
||||||
|
return 0
|
||||||
|
print(__doc__, file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv[1:]))
|
||||||
@@ -59,6 +59,9 @@ class ExecutionState(TypedDict, total=False):
|
|||||||
risks: list[dict[str, Any]] # flags on what the user wrote, for a person to see
|
risks: list[dict[str, Any]] # flags on what the user wrote, for a person to see
|
||||||
overlay_skills: dict[str, str] # overlay Skills this Execution loads -> version
|
overlay_skills: dict[str, str] # overlay Skills this Execution loads -> version
|
||||||
terraform_modules: bool # opted into published Terraform modules
|
terraform_modules: bool # opted into published Terraform modules
|
||||||
|
# The customer's scope spec as read at the start: its Skill, status, readiness
|
||||||
|
# and what a Specialist is shown of it. None when no spec was supplied.
|
||||||
|
scope: dict[str, Any] | None
|
||||||
baseline: dict[str, Any] | None # a Change request's Baseline, read from its files
|
baseline: dict[str, Any] | None # a Change request's Baseline, read from its files
|
||||||
pre_approved: bool
|
pre_approved: bool
|
||||||
allow_cloud_read: bool | None
|
allow_cloud_read: bool | None
|
||||||
|
|||||||
@@ -22,6 +22,14 @@ from app.adk.runtime import (
|
|||||||
)
|
)
|
||||||
from app.states.ledger import Ledger, LedgerRefusal, PhaseStatus
|
from app.states.ledger import Ledger, LedgerRefusal, PhaseStatus
|
||||||
from app.tools.pillars import REVIEW_HEADING, render_checklist_review
|
from app.tools.pillars import REVIEW_HEADING, render_checklist_review
|
||||||
|
from app.tools.scoping import (
|
||||||
|
REGISTER_HEADING,
|
||||||
|
Scoping,
|
||||||
|
render_register,
|
||||||
|
scope_tools,
|
||||||
|
unsettled,
|
||||||
|
)
|
||||||
|
from app.tools.scoping import attachments as scope_attachments
|
||||||
from app.tools.workspace import (
|
from app.tools.workspace import (
|
||||||
FILES,
|
FILES,
|
||||||
basis_of,
|
basis_of,
|
||||||
@@ -53,6 +61,12 @@ from app.workflows.target_architecture import (
|
|||||||
render_mermaid,
|
render_mermaid,
|
||||||
render_tables,
|
render_tables,
|
||||||
)
|
)
|
||||||
|
from app.workflows.work_breakdown import (
|
||||||
|
DICTIONARY_HEADING,
|
||||||
|
TIMELINE_FILE,
|
||||||
|
)
|
||||||
|
from app.workflows.work_breakdown import mermaid as plan_mermaid
|
||||||
|
from app.workflows.work_breakdown import render as render_plan
|
||||||
|
|
||||||
ORCHESTRATOR = "orchestrator"
|
ORCHESTRATOR = "orchestrator"
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -284,6 +298,7 @@ def specialist_tools(
|
|||||||
review: Callable[[Mapping[str, str]], Awaitable[Sequence[str]]] | None = None,
|
review: Callable[[Mapping[str, str]], Awaitable[Sequence[str]]] | None = None,
|
||||||
confirm_first: bool = False,
|
confirm_first: bool = False,
|
||||||
baseline: Mapping[str, Any] | None = None,
|
baseline: Mapping[str, Any] | None = None,
|
||||||
|
scoping: Scoping | None = None,
|
||||||
) -> list[ToolSpec]:
|
) -> list[ToolSpec]:
|
||||||
"""A Specialist's tools: submit, ask, its working copy, and its role's tools with
|
"""A Specialist's tools: submit, ask, its working copy, and its role's tools with
|
||||||
the cloud-read gate.
|
the cloud-read gate.
|
||||||
@@ -298,6 +313,8 @@ def specialist_tools(
|
|||||||
this Deliverable: a submission is then refused until the Specialist has put its
|
this Deliverable: a submission is then refused until the Specialist has put its
|
||||||
draft to them. A rule, because an instruction alone was not always followed.
|
draft to them. A rule, because an instruction alone was not always followed.
|
||||||
`baseline` is the Baseline's version of this Deliverable in a Change request.
|
`baseline` is the Baseline's version of this Deliverable in a Change request.
|
||||||
|
`scoping` is the scope spec's catalogue; with it the scope Deliverable's two
|
||||||
|
files and its information register are written from what was recorded.
|
||||||
"""
|
"""
|
||||||
working = {} if working is None else working
|
working = {} if working is None else working
|
||||||
is_graph = kind is DeliverableKind.TARGET_ARCHITECTURE
|
is_graph = kind is DeliverableKind.TARGET_ARCHITECTURE
|
||||||
@@ -310,12 +327,30 @@ def specialist_tools(
|
|||||||
if is_validation:
|
if is_validation:
|
||||||
working_verdicts(working, basis) # drops verdicts given on other code
|
working_verdicts(working, basis) # drops verdicts given on other code
|
||||||
graph_now = read_graph(upstream)
|
graph_now = read_graph(upstream)
|
||||||
|
is_scope = kind is DeliverableKind.SCOPE and scoping is not None
|
||||||
|
|
||||||
async def submit_deliverable(content: str, attachments: Any = None) -> ToolResult:
|
async def submit_deliverable(content: str, attachments: Any = None) -> ToolResult:
|
||||||
try:
|
try:
|
||||||
attachments = files_from_argument(attachments)
|
attachments = files_from_argument(attachments)
|
||||||
except TypeError as exc:
|
except TypeError as exc:
|
||||||
return _refused(f"attachments: {exc}")
|
return _refused(f"attachments: {exc}")
|
||||||
|
if is_scope and scoping is not None:
|
||||||
|
# Nothing essential may be left unknown, and the files are written
|
||||||
|
# from what was recorded: they cannot disagree with it.
|
||||||
|
missing_fields = unsettled(working, scoping)
|
||||||
|
if missing_fields:
|
||||||
|
return ToolResult(
|
||||||
|
"Not submitted. Essential fields of the scope spec are not "
|
||||||
|
"settled: " + ", ".join(missing_fields) + ". Ask the user for "
|
||||||
|
"each with `needs_input`; when you may not ask, choose a value "
|
||||||
|
"and record it with `source` assumed and a note."
|
||||||
|
)
|
||||||
|
attachments = scope_attachments(working, scoping)
|
||||||
|
content = (
|
||||||
|
content.split(f"\n## {REGISTER_HEADING}")[0].rstrip()
|
||||||
|
+ "\n\n"
|
||||||
|
+ render_register(working, scoping)
|
||||||
|
)
|
||||||
# The working copy is the submission; a file passed here replaces its copy.
|
# The working copy is the submission; a file passed here replaces its copy.
|
||||||
if own_files:
|
if own_files:
|
||||||
attachments = {**working_files(working), **attachments}
|
attachments = {**working_files(working), **attachments}
|
||||||
@@ -375,6 +410,16 @@ def specialist_tools(
|
|||||||
content = content.split(f"\n## {REVIEW_HEADING}")[0].rstrip() + "\n"
|
content = content.split(f"\n## {REVIEW_HEADING}")[0].rstrip() + "\n"
|
||||||
if checklist_review:
|
if checklist_review:
|
||||||
content += "\n" + checklist_review
|
content += "\n" + checklist_review
|
||||||
|
if kind is DeliverableKind.WORK_BREAKDOWN:
|
||||||
|
# The dictionary, the sequence and the timeline are views of the plan,
|
||||||
|
# computed and written here: they cannot disagree with it.
|
||||||
|
views = render_plan(attachments, upstream)
|
||||||
|
content = content.split(f"\n## {DICTIONARY_HEADING}")[0].rstrip() + "\n"
|
||||||
|
if views:
|
||||||
|
content += "\n" + views
|
||||||
|
chart = plan_mermaid(attachments, upstream)
|
||||||
|
if chart:
|
||||||
|
attachments[TIMELINE_FILE] = chart
|
||||||
if is_graph:
|
if is_graph:
|
||||||
# The tables and the diagram are views of the graph, so the engine
|
# The tables and the diagram are views of the graph, so the engine
|
||||||
# writes them: they cannot disagree with it.
|
# writes them: they cannot disagree with it.
|
||||||
@@ -455,6 +500,7 @@ def specialist_tools(
|
|||||||
if library
|
if library
|
||||||
else ()
|
else ()
|
||||||
),
|
),
|
||||||
|
*(scope_tools(working, scoping) if is_scope and scoping else ()),
|
||||||
*(graph_tools(working) if is_graph else ()),
|
*(graph_tools(working) if is_graph else ()),
|
||||||
*(
|
*(
|
||||||
verdict_tools(working, basis, graph_now.check_ids if graph_now else ())
|
verdict_tools(working, basis, graph_now.check_ids if graph_now else ())
|
||||||
|
|||||||
@@ -143,8 +143,10 @@ class TerraformRunner:
|
|||||||
)
|
)
|
||||||
|
|
||||||
async def validate(self, files: Mapping[str, str]) -> ToolResult:
|
async def validate(self, files: Mapping[str, str]) -> ToolResult:
|
||||||
refusal = _check_files(files) or check_module_sources(
|
refusal = (
|
||||||
files, allowed_namespaces(self.module_namespaces)
|
_check_files(files)
|
||||||
|
or check_providers(files, self.providers)
|
||||||
|
or check_module_sources(files, allowed_namespaces(self.module_namespaces))
|
||||||
)
|
)
|
||||||
if refusal:
|
if refusal:
|
||||||
return refusal
|
return refusal
|
||||||
@@ -160,8 +162,10 @@ class TerraformRunner:
|
|||||||
async def plan(
|
async def plan(
|
||||||
self, files: Mapping[str, str], variables: Mapping[str, str] | None = None
|
self, files: Mapping[str, str], variables: Mapping[str, str] | None = None
|
||||||
) -> ToolResult:
|
) -> ToolResult:
|
||||||
refusal = _check_files(files) or check_module_sources(
|
refusal = (
|
||||||
files, allowed_namespaces(self.module_namespaces)
|
_check_files(files)
|
||||||
|
or check_providers(files, self.providers)
|
||||||
|
or check_module_sources(files, allowed_namespaces(self.module_namespaces))
|
||||||
)
|
)
|
||||||
if refusal:
|
if refusal:
|
||||||
return refusal
|
return refusal
|
||||||
@@ -363,6 +367,45 @@ class TerraformRunner:
|
|||||||
return process.returncode or 0, stdout.decode(errors="replace")
|
return process.returncode or 0, stdout.decode(errors="replace")
|
||||||
|
|
||||||
|
|
||||||
|
_BLOCK_TYPE = re.compile(
|
||||||
|
r'^\s*(?:resource|data)\s+"([a-z0-9]+)_[a-z0-9_]*"', re.MULTILINE
|
||||||
|
)
|
||||||
|
_PROVIDER_SOURCE = re.compile(r'^\s*source\s*=\s*"([\w.-]+/[\w.-]+)"\s*$', re.MULTILINE)
|
||||||
|
# Built into Terraform itself: terraform_data and the terraform_remote_state source.
|
||||||
|
_BUILT_IN = frozenset({"terraform"})
|
||||||
|
|
||||||
|
|
||||||
|
def check_providers(
|
||||||
|
files: Mapping[str, str], pinned: Mapping[str, Mapping[str, str]]
|
||||||
|
) -> ToolResult | None:
|
||||||
|
"""Refuse code that uses a provider the bundle does not pin.
|
||||||
|
|
||||||
|
`terraform init` downloads whatever provider the code names, at whatever
|
||||||
|
version is current. A design with a source repository in it brought the GitHub
|
||||||
|
provider into generated code that way on 2026-10-11. With nothing pinned there
|
||||||
|
is nothing to hold code to, and it passes.
|
||||||
|
"""
|
||||||
|
if not pinned:
|
||||||
|
return None
|
||||||
|
names = set(pinned) | _BUILT_IN
|
||||||
|
sources = {pin["source"] for pin in pinned.values()}
|
||||||
|
code = "\n".join(text for name, text in files.items() if name.endswith(".tf"))
|
||||||
|
foreign = sorted({m.group(1) for m in _BLOCK_TYPE.finditer(code)} - names)
|
||||||
|
undeclared = sorted({m.group(1) for m in _PROVIDER_SOURCE.finditer(code)} - sources)
|
||||||
|
if not foreign and not undeclared:
|
||||||
|
return None
|
||||||
|
found = [f"resources of the {name} provider" for name in foreign]
|
||||||
|
found += [f"the provider {source}" for source in undeclared]
|
||||||
|
return ToolResult(
|
||||||
|
"Refused: the code uses "
|
||||||
|
+ ", ".join(found)
|
||||||
|
+ ". Only these providers may be used: "
|
||||||
|
+ ", ".join(sorted(sources))
|
||||||
|
+ ". Write resources of those alone, and describe anything outside them in "
|
||||||
|
"the deployment guidance as a step the user takes."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _check_files(files: Mapping[str, str]) -> ToolResult | None:
|
def _check_files(files: Mapping[str, str]) -> ToolResult | None:
|
||||||
if not files:
|
if not files:
|
||||||
return ToolResult("Refused: provide at least one Terraform file.")
|
return ToolResult("Refused: provide at least one Terraform file.")
|
||||||
|
|||||||
322
app/tools/scoping.py
Normal file
322
app/tools/scoping.py
Normal file
@@ -0,0 +1,322 @@
|
|||||||
|
"""The scope analyst's tools: completing the scope spec (ADR-0008).
|
||||||
|
|
||||||
|
The spec is kept in the scope Deliverable's working memory as plain data, seeded
|
||||||
|
from what the customer supplied. The analyst reads what is settled and what is
|
||||||
|
not, and records each value it settles with where the value came from. Code checks
|
||||||
|
every value against the catalogue as it is recorded, and writes the completed copy
|
||||||
|
and the information register from what is there when the Deliverable is submitted:
|
||||||
|
the spec the customer approves cannot disagree with what was recorded.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from collections.abc import Mapping, MutableMapping, Sequence
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from app.adk.runtime import ToolResult, ToolSpec
|
||||||
|
from app.scope import (
|
||||||
|
ANSWERED,
|
||||||
|
ASK,
|
||||||
|
ASSUMED,
|
||||||
|
DECIDE,
|
||||||
|
ESSENTIAL,
|
||||||
|
IMPORTANT,
|
||||||
|
MARKERS,
|
||||||
|
NOT_APPLICABLE,
|
||||||
|
REQUEST,
|
||||||
|
SOURCES,
|
||||||
|
SUPPLIED,
|
||||||
|
UNDER_REVIEW,
|
||||||
|
Catalogue,
|
||||||
|
Scope,
|
||||||
|
ScopeError,
|
||||||
|
_fault,
|
||||||
|
_plain,
|
||||||
|
read_scope,
|
||||||
|
)
|
||||||
|
from app.workflows.chain import SCOPE_DATA_FILE, SCOPE_FILE
|
||||||
|
|
||||||
|
SCOPE_KEY = "scope_spec" # working memory: the spec as it is being completed
|
||||||
|
REGISTER_HEADING = "Information register"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Scoping:
|
||||||
|
"""What reading and completing a scope spec needs."""
|
||||||
|
|
||||||
|
catalogue: Catalogue
|
||||||
|
provider_values: Mapping[str, Sequence[str]] = field(default_factory=dict)
|
||||||
|
skill: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def to_working(scope: Scope) -> dict[str, Any]:
|
||||||
|
"""A spec as working memory: plain data that survives a checkpoint."""
|
||||||
|
return {
|
||||||
|
"kind": scope.kind,
|
||||||
|
"values": dict(scope.values),
|
||||||
|
"states": dict(scope.states),
|
||||||
|
"sources": dict(scope.sources),
|
||||||
|
"notes": dict(scope.notes),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def from_working(memory: Mapping[str, Any], catalogue: Catalogue) -> Scope:
|
||||||
|
values = dict(memory.get("values") or {})
|
||||||
|
states = {
|
||||||
|
path: (memory.get("states") or {}).get(path, ASK)
|
||||||
|
for path in catalogue.fields
|
||||||
|
if path not in values
|
||||||
|
}
|
||||||
|
kind = values.get("engagement.kind")
|
||||||
|
return Scope(
|
||||||
|
catalogue,
|
||||||
|
str(kind) if kind in catalogue.kinds else None,
|
||||||
|
values=values,
|
||||||
|
states=states,
|
||||||
|
sources=dict(memory.get("sources") or {}),
|
||||||
|
notes=dict(memory.get("notes") or {}),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def working_from_completed(text: str, scoping: Scoping) -> dict[str, Any] | None:
|
||||||
|
"""A completed scope spec as working memory, with where each value came from.
|
||||||
|
None when it cannot be read: a Change request then drafts the scope afresh."""
|
||||||
|
try:
|
||||||
|
scope = read_scope(
|
||||||
|
text, scoping.catalogue, provider_values=scoping.provider_values
|
||||||
|
)
|
||||||
|
document = yaml.safe_load(text) or {}
|
||||||
|
except (ScopeError, yaml.YAMLError):
|
||||||
|
return None
|
||||||
|
recorded = document.get("provenance") if isinstance(document, Mapping) else None
|
||||||
|
scope.sources = {
|
||||||
|
path: str(source)
|
||||||
|
for path, source in (recorded or {}).items()
|
||||||
|
if path in scope.values and source in SOURCES
|
||||||
|
}
|
||||||
|
return to_working(scope)
|
||||||
|
|
||||||
|
|
||||||
|
def working_scope(working: MutableMapping[str, Any]) -> dict[str, Any]:
|
||||||
|
return working.setdefault(SCOPE_KEY, {"values": {}, "states": {}})
|
||||||
|
|
||||||
|
|
||||||
|
def unsettled(working: Mapping[str, Any], scoping: Scoping) -> list[str]:
|
||||||
|
"""The essential fields a submission may not leave unknown."""
|
||||||
|
scope = from_working(working.get(SCOPE_KEY) or {}, scoping.catalogue)
|
||||||
|
return [gap["field"] for gap in scope.gaps(ESSENTIAL)]
|
||||||
|
|
||||||
|
|
||||||
|
def attachments(working: Mapping[str, Any], scoping: Scoping) -> dict[str, str]:
|
||||||
|
"""The Deliverable's two files, written from what was recorded."""
|
||||||
|
scope = from_working(working.get(SCOPE_KEY) or {}, scoping.catalogue)
|
||||||
|
return {
|
||||||
|
SCOPE_FILE: yaml.safe_dump(
|
||||||
|
scope.completed(UNDER_REVIEW), sort_keys=False, allow_unicode=True
|
||||||
|
),
|
||||||
|
SCOPE_DATA_FILE: json.dumps(scope.for_context(), indent=2, ensure_ascii=False),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_FROM = {
|
||||||
|
SUPPLIED: "the scope spec",
|
||||||
|
REQUEST: "the request",
|
||||||
|
ANSWERED: "the customer's answer",
|
||||||
|
ASSUMED: "assumed",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def render_register(working: Mapping[str, Any], scoping: Scoping) -> str:
|
||||||
|
"""The information register, readiness and open questions, as markdown written
|
||||||
|
by code from what was recorded."""
|
||||||
|
scope = from_working(working.get(SCOPE_KEY) or {}, scoping.catalogue)
|
||||||
|
counts = scope.readiness()
|
||||||
|
lines = [
|
||||||
|
f"## {REGISTER_HEADING}",
|
||||||
|
"",
|
||||||
|
(
|
||||||
|
f"{counts[SUPPLIED]} stated by the customer, {counts[ANSWERED]} answered "
|
||||||
|
f"in conversation, {counts[ASSUMED]} assumed, {counts['unknown']} "
|
||||||
|
f"unknown, {counts[NOT_APPLICABLE]} not applicable."
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
"| Field | Value | From |",
|
||||||
|
"| --- | --- | --- |",
|
||||||
|
]
|
||||||
|
for path, value in scope.values.items():
|
||||||
|
if path.startswith(("run.", "metadata.")):
|
||||||
|
continue
|
||||||
|
shown = (
|
||||||
|
json.dumps(value, ensure_ascii=False)
|
||||||
|
if isinstance(value, (list, dict))
|
||||||
|
else str(value)
|
||||||
|
)
|
||||||
|
shown = " ".join(shown.split()).replace("|", "\\|")
|
||||||
|
lines.append(f"| {path} | {shown[:200]} | {_FROM[scope.source(path)]} |")
|
||||||
|
assumed = scope.assumptions()
|
||||||
|
if assumed:
|
||||||
|
lines += [
|
||||||
|
"",
|
||||||
|
"### Assumptions",
|
||||||
|
"",
|
||||||
|
"| Id | Assumption | If it is wrong |",
|
||||||
|
"| --- | --- | --- |",
|
||||||
|
]
|
||||||
|
lines += [
|
||||||
|
f"| {item['id']} | {item['statement']} | {item['impact_if_invalid']} |"
|
||||||
|
for item in assumed
|
||||||
|
]
|
||||||
|
gaps = scope.gaps(ESSENTIAL, IMPORTANT)
|
||||||
|
if gaps:
|
||||||
|
lines += [
|
||||||
|
"",
|
||||||
|
"### Still to confirm with the customer",
|
||||||
|
"",
|
||||||
|
"| Field | Question | Severity |",
|
||||||
|
"| --- | --- | --- |",
|
||||||
|
]
|
||||||
|
lines += [
|
||||||
|
f"| {gap['field']} | {gap['prompt']} | {gap['severity']} |" for gap in gaps
|
||||||
|
]
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def scope_tools(working: MutableMapping[str, Any], scoping: Scoping) -> list[ToolSpec]:
|
||||||
|
catalogue = scoping.catalogue
|
||||||
|
|
||||||
|
async def scope_read() -> ToolResult:
|
||||||
|
scope = from_working(working_scope(working), catalogue)
|
||||||
|
counts = scope.readiness()
|
||||||
|
lines = [
|
||||||
|
f"Kind of engagement: {scope.kind or 'not settled'}",
|
||||||
|
(
|
||||||
|
f"{counts[SUPPLIED]} stated, {counts[ANSWERED]} answered, "
|
||||||
|
f"{counts[ASSUMED]} assumed, {counts['unknown']} unknown."
|
||||||
|
),
|
||||||
|
]
|
||||||
|
for severity, title in (("HIGH", "Essential"), ("MEDIUM", "Important")):
|
||||||
|
gaps = [gap for gap in scope.gaps() if gap["severity"] == severity]
|
||||||
|
if gaps:
|
||||||
|
lines.append(f"{title}, not settled:")
|
||||||
|
lines += [f" {gap['field']}: {gap['prompt']}" for gap in gaps]
|
||||||
|
left = [
|
||||||
|
path
|
||||||
|
for path, state in scope.states.items()
|
||||||
|
if state == DECIDE and not path.startswith("run.")
|
||||||
|
]
|
||||||
|
if left:
|
||||||
|
lines.append("Left to you to decide: " + ", ".join(left))
|
||||||
|
lines.append("Settled: " + (", ".join(scope.values) or "nothing yet"))
|
||||||
|
return ToolResult("\n".join(lines))
|
||||||
|
|
||||||
|
async def scope_set(entries: Any = None) -> ToolResult:
|
||||||
|
if not isinstance(entries, list) or not entries:
|
||||||
|
return ToolResult("Refused: `entries` is a list of fields to record.")
|
||||||
|
memory = working_scope(working)
|
||||||
|
values, states = (
|
||||||
|
memory.setdefault("values", {}),
|
||||||
|
memory.setdefault("states", {}),
|
||||||
|
)
|
||||||
|
sources, notes = (
|
||||||
|
memory.setdefault("sources", {}),
|
||||||
|
memory.setdefault("notes", {}),
|
||||||
|
)
|
||||||
|
problems, done = [], []
|
||||||
|
for entry in entries:
|
||||||
|
if not isinstance(entry, Mapping):
|
||||||
|
problems.append("each entry needs a field")
|
||||||
|
continue
|
||||||
|
path = str(entry.get("field", ""))
|
||||||
|
spec = catalogue.fields.get(path)
|
||||||
|
if spec is None:
|
||||||
|
problems.append(f"{path} is not a field of the scope spec")
|
||||||
|
continue
|
||||||
|
source, value = str(entry.get("source", "")), entry.get("value")
|
||||||
|
note = " ".join(str(entry.get("note") or "").split())
|
||||||
|
if value in MARKERS:
|
||||||
|
if path in values and sources.get(path, SUPPLIED) == SUPPLIED:
|
||||||
|
problems.append(f"{path} was stated in the scope spec; it stays")
|
||||||
|
continue
|
||||||
|
values.pop(path, None)
|
||||||
|
states[path] = value
|
||||||
|
done.append(path)
|
||||||
|
continue
|
||||||
|
if source not in SOURCES or source == SUPPLIED:
|
||||||
|
problems.append(
|
||||||
|
f"{path}: `source` is request (the customer's request says it), "
|
||||||
|
"answered (they told you) or assumed (you chose)"
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if (
|
||||||
|
path in values
|
||||||
|
and sources.get(path, SUPPLIED) == SUPPLIED
|
||||||
|
and source != ANSWERED
|
||||||
|
):
|
||||||
|
problems.append(
|
||||||
|
f"{path} was stated in the scope spec; only the customer's own "
|
||||||
|
"answer changes it"
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if source == ASSUMED and not note:
|
||||||
|
problems.append(f"{path}: an assumption needs a note saying why")
|
||||||
|
continue
|
||||||
|
fault = _fault(spec, value, scoping.provider_values.get(path, ()))
|
||||||
|
if fault:
|
||||||
|
problems.append(f"{path}: {fault}")
|
||||||
|
continue
|
||||||
|
values[path] = _plain(value)
|
||||||
|
states.pop(path, None)
|
||||||
|
sources[path] = source
|
||||||
|
if note:
|
||||||
|
notes[path] = note
|
||||||
|
done.append(path)
|
||||||
|
message = f"Recorded {len(done)}: {', '.join(done) or 'none'}."
|
||||||
|
if problems:
|
||||||
|
message += " Not recorded: " + "; ".join(problems) + "."
|
||||||
|
return ToolResult(message)
|
||||||
|
|
||||||
|
return [
|
||||||
|
ToolSpec(
|
||||||
|
"scope_read",
|
||||||
|
"What the scope spec settles so far, and what is not settled, with the "
|
||||||
|
"question to ask for each.",
|
||||||
|
{"type": "object", "properties": {}, "required": []},
|
||||||
|
scope_read,
|
||||||
|
tier="execution",
|
||||||
|
),
|
||||||
|
ToolSpec(
|
||||||
|
"scope_set",
|
||||||
|
"Record fields of the scope spec. Each entry has `field`, `value`, "
|
||||||
|
"`source` (request, answered or assumed) and, for an assumption, a "
|
||||||
|
"`note` saying why. A `value` of ask, decide or not_applicable marks the "
|
||||||
|
"field instead. Record several fields in one call.",
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"entries": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"field": {"type": "string"},
|
||||||
|
"value": {},
|
||||||
|
"source": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [REQUEST, ANSWERED, ASSUMED],
|
||||||
|
},
|
||||||
|
"note": {"type": "string"},
|
||||||
|
},
|
||||||
|
"required": ["field", "value"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["entries"],
|
||||||
|
},
|
||||||
|
scope_set,
|
||||||
|
tier="execution",
|
||||||
|
),
|
||||||
|
]
|
||||||
@@ -33,6 +33,8 @@ PRODUCTS_FILE = "products.json"
|
|||||||
RECOMMENDATIONS_FILE = "recommendations.json"
|
RECOMMENDATIONS_FILE = "recommendations.json"
|
||||||
IMPLEMENTATION_FILE = "implementation.json"
|
IMPLEMENTATION_FILE = "implementation.json"
|
||||||
VERDICTS_FILE = "verdicts.json"
|
VERDICTS_FILE = "verdicts.json"
|
||||||
|
SCOPE_FILE = "scope.yaml" # the completed scope spec
|
||||||
|
SCOPE_DATA_FILE = "scope.json" # what every Specialist reads of it
|
||||||
|
|
||||||
REQUIREMENT_KINDS = ("functional", "non_functional")
|
REQUIREMENT_KINDS = ("functional", "non_functional")
|
||||||
VERDICTS = ("pass", "fail", "not_verifiable")
|
VERDICTS = ("pass", "fail", "not_verifiable")
|
||||||
@@ -211,9 +213,87 @@ def check_requirements(
|
|||||||
problems += _lacking_conditions(
|
problems += _lacking_conditions(
|
||||||
[item for item in items if item["kind"] == "non_functional"], name
|
[item for item in items if item["kind"] == "non_functional"], name
|
||||||
)
|
)
|
||||||
|
problems += _carries_the_scope(items, name, upstream)
|
||||||
return problems
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def agreed_scope(upstream: Upstream | None) -> dict[str, Any]:
|
||||||
|
"""The agreed scope, as every Specialist reads it; empty when there is none."""
|
||||||
|
text = (
|
||||||
|
((upstream or {}).get("scope") or {})
|
||||||
|
.get("attachments", {})
|
||||||
|
.get(SCOPE_DATA_FILE, "")
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
data = json.loads(text) if text else {}
|
||||||
|
except ValueError:
|
||||||
|
return {}
|
||||||
|
return data if isinstance(data, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
def sources_of(item: Mapping[str, Any]) -> tuple[str, ...]:
|
||||||
|
"""What a requirement comes from: the customer's own requirement ids and the
|
||||||
|
scope spec's fields, as its `from` lists them."""
|
||||||
|
listed = item.get("from")
|
||||||
|
return tuple(str(s) for s in listed) if isinstance(listed, list) else ()
|
||||||
|
|
||||||
|
|
||||||
|
def _carries_the_scope(
|
||||||
|
items: Sequence[Mapping[str, Any]], name: str, upstream: Upstream | None
|
||||||
|
) -> list[str]:
|
||||||
|
"""What the customer stated has to reach the requirements, by name (ADR-0008).
|
||||||
|
|
||||||
|
Each requirement the customer wrote, and each choice they stated that a design
|
||||||
|
must honour, is named in the `from` of a requirement. A choice is carried by a
|
||||||
|
non-functional requirement with a condition, so the Target architecture owes it
|
||||||
|
a Check. Before this a stated topology could be named in a design and not built:
|
||||||
|
case 08 on 2026-10-11 passed all 101 Checks with no Check asking for it.
|
||||||
|
"""
|
||||||
|
scope = agreed_scope(upstream)
|
||||||
|
if not scope:
|
||||||
|
return []
|
||||||
|
stated = scope.get("stated") or {}
|
||||||
|
theirs = [
|
||||||
|
str(entry.get("id"))
|
||||||
|
for entry in ((stated.get("intent") or {}).get("requirements") or [])
|
||||||
|
if isinstance(entry, Mapping) and entry.get("id")
|
||||||
|
]
|
||||||
|
binding = [str(path) for path in scope.get("binding") or []]
|
||||||
|
cited = {source for item in items for source in sources_of(item)}
|
||||||
|
problems = [
|
||||||
|
f"{name}: the customer's requirement {reference} is covered by no "
|
||||||
|
"requirement; name it in the `from` of the one that covers it"
|
||||||
|
for reference in theirs
|
||||||
|
if reference not in cited
|
||||||
|
]
|
||||||
|
problems += [
|
||||||
|
f"{name}: the customer stated {path} and no requirement carries it; add a "
|
||||||
|
f"non-functional requirement whose `from` names {path}, with the condition "
|
||||||
|
"that shows it was honoured"
|
||||||
|
for path in binding
|
||||||
|
if path not in cited
|
||||||
|
]
|
||||||
|
for item in items:
|
||||||
|
carried = [source for source in sources_of(item) if source in binding]
|
||||||
|
if carried and (
|
||||||
|
item.get("kind") != "non_functional" or not conditions_of_listed(item)
|
||||||
|
):
|
||||||
|
problems.append(
|
||||||
|
f"{name}: {item.get('id')} carries {', '.join(carried)}, a choice "
|
||||||
|
"the customer stated; it must be non_functional with at least one "
|
||||||
|
"entry in `conditions`, so the design is checked against it"
|
||||||
|
)
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def conditions_of_listed(item: Mapping[str, Any]) -> tuple[str, ...]:
|
||||||
|
"""The conditions a requirement lists outright, with no fallback to its text."""
|
||||||
|
listed = item.get("conditions")
|
||||||
|
if not isinstance(listed, list):
|
||||||
|
return ()
|
||||||
|
return tuple(c.strip() for c in listed if isinstance(c, str) and c.strip())
|
||||||
|
|
||||||
|
|
||||||
def check_components(
|
def check_components(
|
||||||
attachments: Mapping[str, str], upstream: Upstream | None
|
attachments: Mapping[str, str], upstream: Upstream | None
|
||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
@@ -343,6 +423,19 @@ def _split_implementation(text: str, name: str) -> tuple[str, str]:
|
|||||||
return json.dumps(met), json.dumps(cannot)
|
return json.dumps(met), json.dumps(cannot)
|
||||||
|
|
||||||
|
|
||||||
|
def check_scope(attachments: Mapping[str, str], upstream: Upstream | None) -> list[str]:
|
||||||
|
"""The scope a Specialist reads: what the customer stated, what was left to the
|
||||||
|
agent, and what is unknown."""
|
||||||
|
name = SCOPE_DATA_FILE
|
||||||
|
try:
|
||||||
|
data = json.loads(attachments[name])
|
||||||
|
except ValueError as exc:
|
||||||
|
return [f"{name}: not valid JSON ({exc})"]
|
||||||
|
if not isinstance(data, dict) or not isinstance(data.get("stated"), dict):
|
||||||
|
return [f"{name}: must be an object with what the customer stated"]
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
def check_implementation(
|
def check_implementation(
|
||||||
attachments: Mapping[str, str], upstream: Upstream | None
|
attachments: Mapping[str, str], upstream: Upstream | None
|
||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
|
|||||||
@@ -19,6 +19,8 @@ from app.workflows.chain import (
|
|||||||
PRODUCTS_FILE,
|
PRODUCTS_FILE,
|
||||||
RECOMMENDATIONS_FILE,
|
RECOMMENDATIONS_FILE,
|
||||||
REQUIREMENTS_FILE,
|
REQUIREMENTS_FILE,
|
||||||
|
SCOPE_DATA_FILE,
|
||||||
|
SCOPE_FILE,
|
||||||
VERDICTS_FILE,
|
VERDICTS_FILE,
|
||||||
Upstream,
|
Upstream,
|
||||||
check_components,
|
check_components,
|
||||||
@@ -26,15 +28,20 @@ from app.workflows.chain import (
|
|||||||
check_products,
|
check_products,
|
||||||
check_recommendations,
|
check_recommendations,
|
||||||
check_requirements,
|
check_requirements,
|
||||||
|
check_scope,
|
||||||
check_target_architecture,
|
check_target_architecture,
|
||||||
check_verdicts,
|
check_verdicts,
|
||||||
graph_structure,
|
graph_structure,
|
||||||
)
|
)
|
||||||
from app.workflows.changes import check_requirement_changes
|
from app.workflows.changes import check_requirement_changes
|
||||||
from app.workflows.target_architecture import GRAPH_ATTACHMENT
|
from app.workflows.target_architecture import GRAPH_ATTACHMENT
|
||||||
|
from app.workflows.work_breakdown import WORK_BREAKDOWN_FILE, check_work_breakdown
|
||||||
|
|
||||||
|
|
||||||
class PhaseId(StrEnum):
|
class PhaseId(StrEnum):
|
||||||
|
# Scoping: what the customer wants and the facts a design depends on, agreed
|
||||||
|
# before requirements are derived from it (ADR-0008).
|
||||||
|
SCOPING = "scoping"
|
||||||
REQUIREMENTS_DISCOVERY = "requirements_discovery"
|
REQUIREMENTS_DISCOVERY = "requirements_discovery"
|
||||||
# Design: ends with the Target architecture, approved before any code is written.
|
# Design: ends with the Target architecture, approved before any code is written.
|
||||||
SOLUTION_ARCHITECTURE = "solution_architecture"
|
SOLUTION_ARCHITECTURE = "solution_architecture"
|
||||||
@@ -47,6 +54,7 @@ PHASE_ORDER: tuple[PhaseId, ...] = tuple(PhaseId)
|
|||||||
|
|
||||||
|
|
||||||
class DeliverableKind(StrEnum):
|
class DeliverableKind(StrEnum):
|
||||||
|
SCOPE = "scope"
|
||||||
REQUIREMENTS = "requirements"
|
REQUIREMENTS = "requirements"
|
||||||
TECHNICAL_DECOMPOSITION = "technical_decomposition"
|
TECHNICAL_DECOMPOSITION = "technical_decomposition"
|
||||||
PRODUCT_SELECTION = "product_selection"
|
PRODUCT_SELECTION = "product_selection"
|
||||||
@@ -55,6 +63,8 @@ class DeliverableKind(StrEnum):
|
|||||||
ARCHITECTURE_DESCRIPTION = "architecture_description"
|
ARCHITECTURE_DESCRIPTION = "architecture_description"
|
||||||
DEPLOYMENT_GUIDANCE = "deployment_guidance"
|
DEPLOYMENT_GUIDANCE = "deployment_guidance"
|
||||||
VALIDATION_RESULTS = "validation_results"
|
VALIDATION_RESULTS = "validation_results"
|
||||||
|
# The plan for building what was designed and validated (iteration 10).
|
||||||
|
WORK_BREAKDOWN = "work_breakdown"
|
||||||
SOLUTION_GUIDE = "solution_guide"
|
SOLUTION_GUIDE = "solution_guide"
|
||||||
|
|
||||||
|
|
||||||
@@ -92,10 +102,20 @@ _K = DeliverableKind
|
|||||||
DEPENDENCY_GRAPH: Mapping[DeliverableKind, DeliverableSpec] = {
|
DEPENDENCY_GRAPH: Mapping[DeliverableKind, DeliverableSpec] = {
|
||||||
spec.kind: spec
|
spec.kind: spec
|
||||||
for spec in (
|
for spec in (
|
||||||
|
DeliverableSpec(
|
||||||
|
_K.SCOPE,
|
||||||
|
PhaseId.SCOPING,
|
||||||
|
"scope-analyst",
|
||||||
|
sections=("Scope statement",),
|
||||||
|
# The completed scope spec, for people and for reuse; and what every
|
||||||
|
# later Specialist reads of it.
|
||||||
|
attachments=(SCOPE_FILE, SCOPE_DATA_FILE),
|
||||||
|
),
|
||||||
DeliverableSpec(
|
DeliverableSpec(
|
||||||
_K.REQUIREMENTS,
|
_K.REQUIREMENTS,
|
||||||
PhaseId.REQUIREMENTS_DISCOVERY,
|
PhaseId.REQUIREMENTS_DISCOVERY,
|
||||||
"requirements-analyst",
|
"requirements-analyst",
|
||||||
|
depends_on=(_K.SCOPE,),
|
||||||
sections=(
|
sections=(
|
||||||
"Functional requirements",
|
"Functional requirements",
|
||||||
"Non-functional requirements",
|
"Non-functional requirements",
|
||||||
@@ -183,6 +203,16 @@ DEPENDENCY_GRAPH: Mapping[DeliverableKind, DeliverableSpec] = {
|
|||||||
attachments=(VERDICTS_FILE,),
|
attachments=(VERDICTS_FILE,),
|
||||||
working_copy=_K.DEPLOYMENT_GUIDANCE,
|
working_copy=_K.DEPLOYMENT_GUIDANCE,
|
||||||
),
|
),
|
||||||
|
DeliverableSpec(
|
||||||
|
_K.WORK_BREAKDOWN,
|
||||||
|
PhaseId.PACKAGING,
|
||||||
|
"delivery-planner",
|
||||||
|
depends_on=(_K.VALIDATION_RESULTS,),
|
||||||
|
sections=("Approach",),
|
||||||
|
attachments=(WORK_BREAKDOWN_FILE,),
|
||||||
|
# A plan covers the design and how it is built: the graph and the code.
|
||||||
|
reads_in_full=(_K.TARGET_ARCHITECTURE, _K.DEPLOYMENT_GUIDANCE),
|
||||||
|
),
|
||||||
DeliverableSpec(
|
DeliverableSpec(
|
||||||
_K.SOLUTION_GUIDE,
|
_K.SOLUTION_GUIDE,
|
||||||
PhaseId.PACKAGING,
|
PhaseId.PACKAGING,
|
||||||
@@ -209,6 +239,7 @@ DATA_CHECKS: Mapping[
|
|||||||
DeliverableKind,
|
DeliverableKind,
|
||||||
tuple[str, Callable[[Mapping[str, str], Upstream | None], list[str]]],
|
tuple[str, Callable[[Mapping[str, str], Upstream | None], list[str]]],
|
||||||
] = {
|
] = {
|
||||||
|
_K.SCOPE: (SCOPE_DATA_FILE, check_scope),
|
||||||
_K.REQUIREMENTS: (REQUIREMENTS_FILE, check_requirements),
|
_K.REQUIREMENTS: (REQUIREMENTS_FILE, check_requirements),
|
||||||
_K.TECHNICAL_DECOMPOSITION: (COMPONENTS_FILE, check_components),
|
_K.TECHNICAL_DECOMPOSITION: (COMPONENTS_FILE, check_components),
|
||||||
_K.PRODUCT_SELECTION: (PRODUCTS_FILE, check_products),
|
_K.PRODUCT_SELECTION: (PRODUCTS_FILE, check_products),
|
||||||
@@ -216,6 +247,7 @@ DATA_CHECKS: Mapping[
|
|||||||
_K.TARGET_ARCHITECTURE: (GRAPH_ATTACHMENT, check_target_architecture),
|
_K.TARGET_ARCHITECTURE: (GRAPH_ATTACHMENT, check_target_architecture),
|
||||||
_K.DEPLOYMENT_GUIDANCE: (IMPLEMENTATION_FILE, check_implementation),
|
_K.DEPLOYMENT_GUIDANCE: (IMPLEMENTATION_FILE, check_implementation),
|
||||||
_K.VALIDATION_RESULTS: (VERDICTS_FILE, check_verdicts),
|
_K.VALIDATION_RESULTS: (VERDICTS_FILE, check_verdicts),
|
||||||
|
_K.WORK_BREAKDOWN: (WORK_BREAKDOWN_FILE, check_work_breakdown),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ from app.nodes.execution_nodes import make_execution_nodes
|
|||||||
from app.nodes.router import OrchestratorPolicy
|
from app.nodes.router import OrchestratorPolicy
|
||||||
from app.recorder import ExecutionRecorder
|
from app.recorder import ExecutionRecorder
|
||||||
from app.states.execution_state import ExecutionState
|
from app.states.execution_state import ExecutionState
|
||||||
|
from app.tools.scoping import Scoping
|
||||||
|
|
||||||
|
|
||||||
def build_execution_graph(
|
def build_execution_graph(
|
||||||
@@ -53,6 +54,7 @@ def build_execution_graph(
|
|||||||
skill_references: Mapping[str, Mapping[str, Mapping[str, str]]] | None = None,
|
skill_references: Mapping[str, Mapping[str, Mapping[str, str]]] | None = None,
|
||||||
skill_threshold: float = 1.25,
|
skill_threshold: float = 1.25,
|
||||||
assess_at_end: bool = False,
|
assess_at_end: bool = False,
|
||||||
|
scoping: Scoping | None = None,
|
||||||
) -> StateGraph:
|
) -> StateGraph:
|
||||||
nodes = make_execution_nodes(
|
nodes = make_execution_nodes(
|
||||||
runtime=runtime,
|
runtime=runtime,
|
||||||
@@ -74,6 +76,7 @@ def build_execution_graph(
|
|||||||
skill_references=skill_references,
|
skill_references=skill_references,
|
||||||
skill_threshold=skill_threshold,
|
skill_threshold=skill_threshold,
|
||||||
assess_at_end=assess_at_end,
|
assess_at_end=assess_at_end,
|
||||||
|
scoping=scoping,
|
||||||
)
|
)
|
||||||
|
|
||||||
graph = StateGraph(ExecutionState)
|
graph = StateGraph(ExecutionState)
|
||||||
|
|||||||
559
app/workflows/work_breakdown.py
Normal file
559
app/workflows/work_breakdown.py
Normal file
@@ -0,0 +1,559 @@
|
|||||||
|
"""The work breakdown structure: a plan for building the Target architecture.
|
||||||
|
|
||||||
|
A Specialist divides the build into work packages and activities and estimates each
|
||||||
|
activity's days. Everything after that is code, as it is for the architecture
|
||||||
|
diagram: the plan is checked against the design, the schedule is computed from the
|
||||||
|
days and the dependencies, and the three views a reader sees are written from the
|
||||||
|
result. So the timeline cannot disagree with the plan, and no part of the design can
|
||||||
|
be left out of it.
|
||||||
|
|
||||||
|
work packages a hierarchy; each delivers named Elements, and is done when the
|
||||||
|
Checks on those Elements pass
|
||||||
|
activities what is done, in which work package, for how many days, after
|
||||||
|
which other activities, and by whom
|
||||||
|
schedule each activity's start and finish day, on a relative axis from
|
||||||
|
Day 0, with no more of the team's activities at once than the
|
||||||
|
team has people; and the critical path
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
from collections.abc import Mapping, Sequence
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from app.workflows.chain import Upstream, agreed_scope, read_graph
|
||||||
|
|
||||||
|
WORK_BREAKDOWN_FILE = "work-breakdown.json"
|
||||||
|
TIMELINE_FILE = "timeline.mmd"
|
||||||
|
DICTIONARY_HEADING = "WBS dictionary"
|
||||||
|
TEAM, CUSTOMER = "team", "customer"
|
||||||
|
OWNERS = (TEAM, CUSTOMER)
|
||||||
|
DEFAULT_TEAM_SIZE = 2
|
||||||
|
DEFAULT_WEEK = 5
|
||||||
|
MAX_DAYS = 120 # one activity; anything longer is a work package, not an activity
|
||||||
|
_CHART_WIDTH = 60
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Scheduled:
|
||||||
|
id: str
|
||||||
|
start: float
|
||||||
|
finish: float
|
||||||
|
critical: bool
|
||||||
|
|
||||||
|
|
||||||
|
def _text(value: Any) -> str:
|
||||||
|
return " ".join(str(value or "").split())
|
||||||
|
|
||||||
|
|
||||||
|
def _list(value: Any) -> list[str]:
|
||||||
|
return [str(item) for item in value] if isinstance(value, list) else []
|
||||||
|
|
||||||
|
|
||||||
|
def parse(text: str) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[str]]:
|
||||||
|
"""(work packages, activities, problems) from the data file."""
|
||||||
|
name = WORK_BREAKDOWN_FILE
|
||||||
|
try:
|
||||||
|
data = json.loads(text)
|
||||||
|
except ValueError as exc:
|
||||||
|
return [], [], [f"{name}: not valid JSON ({exc})"]
|
||||||
|
if not isinstance(data, Mapping):
|
||||||
|
return [], [], [f"{name}: must be an object with work_packages and activities"]
|
||||||
|
packages = [p for p in data.get("work_packages") or [] if isinstance(p, Mapping)]
|
||||||
|
activities = [a for a in data.get("activities") or [] if isinstance(a, Mapping)]
|
||||||
|
return [dict(p) for p in packages], [dict(a) for a in activities], []
|
||||||
|
|
||||||
|
|
||||||
|
def check_work_breakdown(
|
||||||
|
attachments: Mapping[str, str], upstream: Upstream | None
|
||||||
|
) -> list[str]:
|
||||||
|
name = WORK_BREAKDOWN_FILE
|
||||||
|
packages, activities, problems = parse(attachments.get(name, ""))
|
||||||
|
if problems:
|
||||||
|
return problems
|
||||||
|
if not packages:
|
||||||
|
return [f"{name}: lists no work packages"]
|
||||||
|
if not activities:
|
||||||
|
return [f"{name}: lists no activities"]
|
||||||
|
|
||||||
|
ids = [str(p.get("id", "")) for p in packages]
|
||||||
|
problems += [f"{name}: a work package has no id"] if "" in ids else []
|
||||||
|
problems += [
|
||||||
|
f"{name}: work package id {dup} is used more than once"
|
||||||
|
for dup in sorted({i for i in ids if i and ids.count(i) > 1})
|
||||||
|
]
|
||||||
|
known = set(ids)
|
||||||
|
parents = {str(p.get("id")): p.get("parent") for p in packages}
|
||||||
|
for package in packages:
|
||||||
|
ident, parent = str(package.get("id")), package.get("parent")
|
||||||
|
if not _text(package.get("name")):
|
||||||
|
problems.append(f"{name}: {ident} has no name")
|
||||||
|
if parent is not None and str(parent) not in known:
|
||||||
|
problems.append(
|
||||||
|
f"{name}: {ident} has parent {parent}, which does not exist"
|
||||||
|
)
|
||||||
|
seen, node = {ident}, parent
|
||||||
|
while node is not None and str(node) in parents:
|
||||||
|
if str(node) in seen:
|
||||||
|
problems.append(f"{name}: {ident} is its own ancestor")
|
||||||
|
break
|
||||||
|
seen.add(str(node))
|
||||||
|
node = parents[str(node)]
|
||||||
|
|
||||||
|
graph = read_graph(upstream)
|
||||||
|
if graph is not None:
|
||||||
|
elements = {element.id for element in graph.elements}
|
||||||
|
covered: set[str] = set()
|
||||||
|
for package in packages:
|
||||||
|
listed = _list(package.get("elements"))
|
||||||
|
covered.update(listed)
|
||||||
|
problems += [
|
||||||
|
f"{name}: {package.get('id')} delivers {element}, which is not an "
|
||||||
|
"Element of the Target architecture"
|
||||||
|
for element in listed
|
||||||
|
if element not in elements
|
||||||
|
]
|
||||||
|
left = sorted(elements - covered)
|
||||||
|
if left:
|
||||||
|
problems.append(
|
||||||
|
f"{name}: no work package delivers {', '.join(left)}; every Element "
|
||||||
|
"of the Target architecture belongs to one"
|
||||||
|
)
|
||||||
|
|
||||||
|
activity_ids = [str(a.get("id", "")) for a in activities]
|
||||||
|
problems += [
|
||||||
|
f"{name}: activity id {dup} is used more than once"
|
||||||
|
for dup in sorted({i for i in activity_ids if i and activity_ids.count(i) > 1})
|
||||||
|
]
|
||||||
|
known_activities = set(activity_ids)
|
||||||
|
with_work = set()
|
||||||
|
for activity in activities:
|
||||||
|
ident = str(activity.get("id", ""))
|
||||||
|
if not ident or not _text(activity.get("name")):
|
||||||
|
problems.append(f"{name}: an activity needs an id and a name")
|
||||||
|
continue
|
||||||
|
within = str(activity.get("work_package", ""))
|
||||||
|
if within not in known:
|
||||||
|
problems.append(f"{name}: {ident} is in {within}, not a work package")
|
||||||
|
with_work.add(within)
|
||||||
|
days = activity.get("days")
|
||||||
|
if isinstance(days, bool) or not isinstance(days, (int, float)) or days <= 0:
|
||||||
|
problems.append(f"{name}: {ident} needs `days`, a number above 0")
|
||||||
|
elif days > MAX_DAYS:
|
||||||
|
problems.append(
|
||||||
|
f"{name}: {ident} is {days} days; split anything over {MAX_DAYS}"
|
||||||
|
)
|
||||||
|
problems += [
|
||||||
|
f"{name}: {ident} comes after {before}, which is not an activity"
|
||||||
|
for before in _list(activity.get("after"))
|
||||||
|
if before not in known_activities
|
||||||
|
]
|
||||||
|
if activity.get("owner", TEAM) not in OWNERS:
|
||||||
|
problems.append(f"{name}: {ident} owner must be one of {', '.join(OWNERS)}")
|
||||||
|
has_children = {str(parent) for parent in parents.values() if parent is not None}
|
||||||
|
problems += [
|
||||||
|
f"{name}: {ident} has no activity and no work package beneath it"
|
||||||
|
for ident in ids
|
||||||
|
if ident and ident not in with_work and ident not in has_children
|
||||||
|
]
|
||||||
|
if not problems and _cycle(activities):
|
||||||
|
problems.append(
|
||||||
|
f"{name}: activities depend on each other in a circle: "
|
||||||
|
+ " -> ".join(_cycle(activities))
|
||||||
|
)
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def _cycle(activities: Sequence[Mapping[str, Any]]) -> list[str]:
|
||||||
|
"""Activities that depend on each other in a circle; empty when none do."""
|
||||||
|
after = {str(a["id"]): _list(a.get("after")) for a in activities}
|
||||||
|
state: dict[str, int] = {}
|
||||||
|
|
||||||
|
def visit(node: str, path: list[str]) -> list[str]:
|
||||||
|
state[node] = 1
|
||||||
|
for before in after.get(node, ()):
|
||||||
|
if state.get(before) == 1:
|
||||||
|
return (
|
||||||
|
[*path[path.index(before) :], before]
|
||||||
|
if before in path
|
||||||
|
else [before, node]
|
||||||
|
)
|
||||||
|
if state.get(before, 0) == 0:
|
||||||
|
found = visit(before, [*path, before])
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
state[node] = 2
|
||||||
|
return []
|
||||||
|
|
||||||
|
for node in after:
|
||||||
|
if state.get(node, 0) == 0:
|
||||||
|
found = visit(node, [node])
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def schedule(
|
||||||
|
activities: Sequence[Mapping[str, Any]], team_size: int = DEFAULT_TEAM_SIZE
|
||||||
|
) -> dict[str, Scheduled]:
|
||||||
|
"""Each activity's start and finish day, from Day 0.
|
||||||
|
|
||||||
|
An activity starts when everything it comes after has finished. The team's own
|
||||||
|
activities also wait for a free person: no more run at once than `team_size`.
|
||||||
|
What the customer owns, such as a supplier's lead time, takes nobody from the
|
||||||
|
team. The critical path is every activity with no slack: delay it and the end
|
||||||
|
moves.
|
||||||
|
"""
|
||||||
|
team_size = max(1, int(team_size))
|
||||||
|
by_id = {str(a["id"]): a for a in activities}
|
||||||
|
after = {ident: _list(a.get("after")) for ident, a in by_id.items()}
|
||||||
|
days = {ident: float(a["days"]) for ident, a in by_id.items()}
|
||||||
|
# The longest chain of work that follows an activity: what to start first when
|
||||||
|
# people are short.
|
||||||
|
followers: dict[str, list[str]] = {ident: [] for ident in by_id}
|
||||||
|
for ident, befores in after.items():
|
||||||
|
for before in befores:
|
||||||
|
followers[before].append(ident)
|
||||||
|
tail: dict[str, float] = {}
|
||||||
|
|
||||||
|
def remaining(ident: str) -> float:
|
||||||
|
if ident not in tail:
|
||||||
|
tail[ident] = days[ident] + max(
|
||||||
|
(remaining(f) for f in followers[ident]), default=0.0
|
||||||
|
)
|
||||||
|
return tail[ident]
|
||||||
|
|
||||||
|
start: dict[str, float] = {}
|
||||||
|
finish: dict[str, float] = {}
|
||||||
|
busy: list[tuple[float, float]] = [] # the team's activities already placed
|
||||||
|
pending = set(by_id)
|
||||||
|
while pending:
|
||||||
|
ready = [i for i in pending if all(b in finish for b in after[i])]
|
||||||
|
ready.sort(key=lambda i: (-remaining(i), i))
|
||||||
|
ident = min(
|
||||||
|
ready,
|
||||||
|
key=lambda i: (
|
||||||
|
max((finish[b] for b in after[i]), default=0.0),
|
||||||
|
-remaining(i),
|
||||||
|
i,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
earliest = max((finish[b] for b in after[ident]), default=0.0)
|
||||||
|
if by_id[ident].get("owner", TEAM) == TEAM:
|
||||||
|
candidates = sorted({earliest, *(end for _, end in busy if end > earliest)})
|
||||||
|
for moment in candidates:
|
||||||
|
overlapping = sum(
|
||||||
|
1 for s, e in busy if s < moment + days[ident] and e > moment
|
||||||
|
)
|
||||||
|
if overlapping < team_size:
|
||||||
|
earliest = moment
|
||||||
|
break
|
||||||
|
busy.append((earliest, earliest + days[ident]))
|
||||||
|
start[ident], finish[ident] = earliest, earliest + days[ident]
|
||||||
|
pending.discard(ident)
|
||||||
|
|
||||||
|
end = max(finish.values(), default=0.0)
|
||||||
|
latest: dict[str, float] = {}
|
||||||
|
|
||||||
|
def latest_finish(ident: str) -> float:
|
||||||
|
if ident not in latest:
|
||||||
|
latest[ident] = min(
|
||||||
|
(latest_finish(f) - days[f] for f in followers[ident]), default=end
|
||||||
|
)
|
||||||
|
return latest[ident]
|
||||||
|
|
||||||
|
return {
|
||||||
|
ident: Scheduled(
|
||||||
|
ident,
|
||||||
|
start[ident],
|
||||||
|
finish[ident],
|
||||||
|
critical=latest_finish(ident) - finish[ident] < 1e-6,
|
||||||
|
)
|
||||||
|
for ident in by_id
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --- what the plan is checked against, from the agreed scope ----------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _scoped(scope: Mapping[str, Any], path: str) -> Any:
|
||||||
|
"""A field of the agreed scope, stated or assumed; None when it is neither."""
|
||||||
|
node: Any = scope.get("stated") or {}
|
||||||
|
for part in path.split("."):
|
||||||
|
node = node.get(part) if isinstance(node, Mapping) else None
|
||||||
|
if node is not None:
|
||||||
|
return node
|
||||||
|
for item in scope.get("assumed") or []:
|
||||||
|
if isinstance(item, Mapping) and item.get("field") == path:
|
||||||
|
return item.get("value")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def delivery_terms(upstream: Upstream | None) -> dict[str, Any]:
|
||||||
|
"""Team size, working week, window and start date, from the agreed scope."""
|
||||||
|
scope = agreed_scope(upstream)
|
||||||
|
|
||||||
|
def number(path: str, default: int) -> int:
|
||||||
|
value = _scoped(scope, path)
|
||||||
|
ok = (
|
||||||
|
isinstance(value, (int, float))
|
||||||
|
and not isinstance(value, bool)
|
||||||
|
and value > 0
|
||||||
|
)
|
||||||
|
return int(value) if ok else default
|
||||||
|
|
||||||
|
window = _scoped(scope, "delivery.window_weeks")
|
||||||
|
start = _scoped(scope, "delivery.target_start_date")
|
||||||
|
try:
|
||||||
|
start_date = dt.date.fromisoformat(str(start)) if start else None
|
||||||
|
except ValueError:
|
||||||
|
start_date = None
|
||||||
|
return {
|
||||||
|
"team_size": number("delivery.team_size", DEFAULT_TEAM_SIZE),
|
||||||
|
"team_size_stated": _scoped(scope, "delivery.team_size") is not None,
|
||||||
|
"week": number("delivery.working_days_per_week", DEFAULT_WEEK),
|
||||||
|
"window_weeks": window if isinstance(window, (int, float)) else None,
|
||||||
|
"start": start_date,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --- the three views --------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _day(value: float) -> str:
|
||||||
|
return str(int(value)) if float(value).is_integer() else f"{value:.1f}"
|
||||||
|
|
||||||
|
|
||||||
|
def _calendar(start: dt.date, day: float, week: int) -> dt.date:
|
||||||
|
"""The calendar date of a working day counted from `start`."""
|
||||||
|
weeks, days = divmod(math.floor(day), week)
|
||||||
|
return start + dt.timedelta(days=weeks * 7 + days)
|
||||||
|
|
||||||
|
|
||||||
|
def render(attachments: Mapping[str, str], upstream: Upstream | None) -> str:
|
||||||
|
"""The WBS dictionary, the activity sequence and the timeline, as markdown
|
||||||
|
written from the data file. "" when the file cannot be read."""
|
||||||
|
packages, activities, problems = parse(attachments.get(WORK_BREAKDOWN_FILE, ""))
|
||||||
|
if problems or not packages or not activities:
|
||||||
|
return ""
|
||||||
|
terms = delivery_terms(upstream)
|
||||||
|
plan = schedule(activities, terms["team_size"])
|
||||||
|
graph = read_graph(upstream)
|
||||||
|
checks: dict[str, list[str]] = {}
|
||||||
|
for check in graph.checks if graph else ():
|
||||||
|
checks.setdefault(check.applies_to, []).append(check.id)
|
||||||
|
|
||||||
|
parents = {str(p["id"]): p.get("parent") for p in packages}
|
||||||
|
|
||||||
|
def depth(ident: str) -> int:
|
||||||
|
level, node = 0, parents.get(ident)
|
||||||
|
while node is not None and str(node) in parents and level < 10:
|
||||||
|
level, node = level + 1, parents.get(str(node))
|
||||||
|
return level
|
||||||
|
|
||||||
|
effort: dict[str, float] = {}
|
||||||
|
for activity in activities:
|
||||||
|
effort[str(activity["work_package"])] = effort.get(
|
||||||
|
str(activity["work_package"]), 0.0
|
||||||
|
) + float(activity["days"])
|
||||||
|
|
||||||
|
def cell(value: Any) -> str:
|
||||||
|
return _text(value).replace("|", "\\|") or "-"
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
f"## {DICTIONARY_HEADING}",
|
||||||
|
"",
|
||||||
|
"Each work package, what it delivers, and the Checks that say it is done.",
|
||||||
|
"",
|
||||||
|
"| Id | Work package | Scope | Produces | Elements | Done when these pass | Skills | Days |",
|
||||||
|
"| --- | --- | --- | --- | --- | --- | --- | --- |",
|
||||||
|
]
|
||||||
|
for package in packages:
|
||||||
|
ident = str(package["id"])
|
||||||
|
elements = _list(package.get("elements"))
|
||||||
|
accepted = [c for element in elements for c in checks.get(element, [])]
|
||||||
|
lines.append(
|
||||||
|
"| "
|
||||||
|
+ " | ".join(
|
||||||
|
[
|
||||||
|
ident,
|
||||||
|
(" " * depth(ident)) + cell(package.get("name")),
|
||||||
|
cell(package.get("scope")),
|
||||||
|
cell("; ".join(_list(package.get("produces")))),
|
||||||
|
cell(", ".join(elements)),
|
||||||
|
cell(", ".join(accepted)),
|
||||||
|
cell(", ".join(_list(package.get("skills")))),
|
||||||
|
_day(effort[ident]) if ident in effort else "-",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
+ " |"
|
||||||
|
)
|
||||||
|
|
||||||
|
ordered = sorted(activities, key=lambda a: (plan[str(a["id"])].start, str(a["id"])))
|
||||||
|
dated = terms["start"] is not None
|
||||||
|
lines += [
|
||||||
|
"",
|
||||||
|
"## Activity sequence",
|
||||||
|
"",
|
||||||
|
"| Id | Activity | Work package | Days | After | Start | Finish |"
|
||||||
|
+ (" Dates |" if dated else "")
|
||||||
|
+ " Owner | Critical |",
|
||||||
|
"| --- | --- | --- | --- | --- | --- | --- |"
|
||||||
|
+ (" --- |" if dated else "")
|
||||||
|
+ " --- | --- |",
|
||||||
|
]
|
||||||
|
for activity in ordered:
|
||||||
|
item = plan[str(activity["id"])]
|
||||||
|
row = [
|
||||||
|
str(activity["id"]),
|
||||||
|
cell(activity.get("name")),
|
||||||
|
str(activity.get("work_package")),
|
||||||
|
_day(float(activity["days"])),
|
||||||
|
cell(", ".join(_list(activity.get("after")))),
|
||||||
|
f"Day {_day(item.start)}",
|
||||||
|
f"Day {_day(item.finish)}",
|
||||||
|
]
|
||||||
|
if dated:
|
||||||
|
begins = _calendar(terms["start"], item.start, terms["week"])
|
||||||
|
ends = _calendar(terms["start"], max(item.finish - 1e-9, 0), terms["week"])
|
||||||
|
row.append(f"{begins.isoformat()} to {ends.isoformat()}")
|
||||||
|
row += [str(activity.get("owner", TEAM)), "yes" if item.critical else ""]
|
||||||
|
lines.append("| " + " | ".join(row) + " |")
|
||||||
|
|
||||||
|
total = max(item.finish for item in plan.values())
|
||||||
|
weeks = total / terms["week"]
|
||||||
|
lines += [
|
||||||
|
"",
|
||||||
|
"## Timeline",
|
||||||
|
"",
|
||||||
|
"```text",
|
||||||
|
*_chart(ordered, plan, total),
|
||||||
|
"```",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
summary = (
|
||||||
|
f"{_day(total)} working days, about {weeks:.1f} weeks, with a team of "
|
||||||
|
f"{terms['team_size']}"
|
||||||
|
+ (
|
||||||
|
""
|
||||||
|
if terms["team_size_stated"]
|
||||||
|
else " (assumed; the scope gives no team size)"
|
||||||
|
)
|
||||||
|
+ f" and a {terms['week']}-day week."
|
||||||
|
)
|
||||||
|
lines.append(summary)
|
||||||
|
window = terms["window_weeks"]
|
||||||
|
if window is not None:
|
||||||
|
if weeks <= window + 1e-9:
|
||||||
|
lines.append(
|
||||||
|
f"That is inside the {window}-week delivery window, with "
|
||||||
|
f"{window - weeks:.1f} weeks to spare."
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
path = [str(a["id"]) for a in ordered if plan[str(a["id"])].critical]
|
||||||
|
lines.append(
|
||||||
|
f"**That is {weeks - window:.1f} weeks over the {window}-week delivery "
|
||||||
|
f"window.** The critical path is {' -> '.join(path)}: only shortening "
|
||||||
|
"or removing work on it, or adding people where it waits for them, "
|
||||||
|
"brings the end in."
|
||||||
|
)
|
||||||
|
theirs = [a for a in ordered if a.get("owner", TEAM) == CUSTOMER]
|
||||||
|
if theirs:
|
||||||
|
lines += ["", "The customer owns these, and the plan waits on them:"]
|
||||||
|
lines += [
|
||||||
|
f"- {a['id']} {_text(a.get('name'))}: {_day(float(a['days']))} days"
|
||||||
|
+ (f". {_text(a.get('assumption'))}" if _text(a.get("assumption")) else "")
|
||||||
|
for a in theirs
|
||||||
|
]
|
||||||
|
lines += [
|
||||||
|
"",
|
||||||
|
(
|
||||||
|
"Durations are estimates made without knowing the team, and are "
|
||||||
|
"indicative. Days are working days counted from Day 0, the day work "
|
||||||
|
"starts; each of the team's activities takes one person."
|
||||||
|
),
|
||||||
|
]
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def _chart(
|
||||||
|
ordered: Sequence[Mapping[str, Any]], plan: Mapping[str, Scheduled], total: float
|
||||||
|
) -> list[str]:
|
||||||
|
"""The timeline as text: one row per activity along a relative day axis."""
|
||||||
|
scale = min(1.0, _CHART_WIDTH / total) if total else 1.0
|
||||||
|
labels = [f"{a['id']} {_text(a.get('name'))}"[:38] for a in ordered]
|
||||||
|
width = max(len(label) for label in labels) + 2
|
||||||
|
step = max(1, math.ceil(total / 6 / 5.0) * 5)
|
||||||
|
axis = [" "] * (int(total * scale) + 8)
|
||||||
|
for day in range(0, int(total) + 1, step):
|
||||||
|
mark = str(day)
|
||||||
|
at = int(day * scale)
|
||||||
|
axis[at : at + len(mark)] = mark
|
||||||
|
rows = [" " * width + "Day " + "".join(axis).rstrip()]
|
||||||
|
for label, activity in zip(labels, ordered, strict=True):
|
||||||
|
item = plan[str(activity["id"])]
|
||||||
|
begin = round(item.start * scale)
|
||||||
|
length = max(1, round((item.finish - item.start) * scale))
|
||||||
|
block = "░" if activity.get("owner", TEAM) == CUSTOMER else "█"
|
||||||
|
flag = " *" if item.critical else ""
|
||||||
|
rows.append(f"{label.ljust(width)} {' ' * begin}{block * length}{flag}")
|
||||||
|
rows += ["", "█ the team ░ the customer * on the critical path"]
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def mermaid(attachments: Mapping[str, str], upstream: Upstream | None) -> str:
|
||||||
|
"""A Mermaid Gantt chart on calendar dates, when the scope gives a start date.
|
||||||
|
"" otherwise: a relative plan has no dates to draw."""
|
||||||
|
packages, activities, problems = parse(attachments.get(WORK_BREAKDOWN_FILE, ""))
|
||||||
|
terms = delivery_terms(upstream)
|
||||||
|
if problems or not activities or terms["start"] is None:
|
||||||
|
return ""
|
||||||
|
plan = schedule(activities, terms["team_size"])
|
||||||
|
names = {str(p["id"]): _text(p.get("name")) for p in packages}
|
||||||
|
lines = ["gantt", " dateFormat YYYY-MM-DD", " excludes weekends"]
|
||||||
|
for package in packages:
|
||||||
|
own = [
|
||||||
|
a for a in activities if str(a.get("work_package")) == str(package["id"])
|
||||||
|
]
|
||||||
|
if not own:
|
||||||
|
continue
|
||||||
|
lines.append(f" section {names[str(package['id'])].replace(':', ' ')}")
|
||||||
|
for activity in sorted(own, key=lambda a: plan[str(a["id"])].start):
|
||||||
|
item = plan[str(activity["id"])]
|
||||||
|
begins = _calendar(terms["start"], item.start, terms["week"])
|
||||||
|
tag = "crit, " if item.critical else ""
|
||||||
|
label = _text(activity.get("name")).replace(":", " ")
|
||||||
|
lines.append(
|
||||||
|
f" {label} :{tag}{activity['id']}, {begins.isoformat()}, "
|
||||||
|
f"{max(1, math.ceil(float(activity['days'])))}d"
|
||||||
|
)
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def summary(
|
||||||
|
attachments: Mapping[str, str], upstream: Upstream | None
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""The plan in a few numbers, for an Approval and for the Execution's record."""
|
||||||
|
packages, activities, problems = parse(attachments.get(WORK_BREAKDOWN_FILE, ""))
|
||||||
|
if problems or not activities:
|
||||||
|
return {}
|
||||||
|
terms = delivery_terms(upstream)
|
||||||
|
plan = schedule(activities, terms["team_size"])
|
||||||
|
total = max(item.finish for item in plan.values())
|
||||||
|
weeks = round(total / terms["week"], 1)
|
||||||
|
return {
|
||||||
|
"work_packages": len(packages),
|
||||||
|
"activities": len(activities),
|
||||||
|
"working_days": round(total, 1),
|
||||||
|
"weeks": weeks,
|
||||||
|
"team_size": terms["team_size"],
|
||||||
|
"window_weeks": terms["window_weeks"],
|
||||||
|
"within_window": None
|
||||||
|
if terms["window_weeks"] is None
|
||||||
|
else weeks <= terms["window_weeks"],
|
||||||
|
"critical_path": [i for i, item in plan.items() if item.critical],
|
||||||
|
}
|
||||||
50
docs/adr/0008-scoping-starts-from-a-scope-spec.md
Normal file
50
docs/adr/0008-scoping-starts-from-a-scope-spec.md
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
---
|
||||||
|
status: accepted
|
||||||
|
---
|
||||||
|
|
||||||
|
# Scoping starts from a scope spec
|
||||||
|
|
||||||
|
An Execution began from the customer's request, a sentence or two saved word for word, and the requirements analyst filled everything else with assumptions. Most of what went wrong downstream traced to what nobody had stated, and nothing told the customer how much of their design rested on assumption. We decide that an engagement is scoped from a **scope spec**: a structured file that states what the customer wants and the facts a design depends on, which may arrive complete, partly filled or not at all, and which the customer approves before requirements begin.
|
||||||
|
|
||||||
|
The Intent stays the customer's own words. The scope spec sits beside it as the agreed, structured form.
|
||||||
|
|
||||||
|
## What the spec is
|
||||||
|
|
||||||
|
- **A file of fields from a catalogue.** A Skill holds the catalogue: each field's type, when it must be settled, the question to ask for it, and guidance for a default. The catalogue names no Cloud provider. A provider's bundle supplies the allowed values for the fields it leaves to a provider, in that provider's terms.
|
||||||
|
- **Every field says one of four things.** A value is the customer's statement. `ask`, or the field left out, means not known. `decide` leaves the choice to the agent, recorded as an assumption. `not_applicable` asks and assumes nothing.
|
||||||
|
- **Whether a field is settled is read by code.** No model decides what is missing.
|
||||||
|
- **Fields are essential, important or optional,** and which is which depends on the kind of engagement: a workload, a platform foundation or a migration.
|
||||||
|
- **It carries the customer's own words and requirements:** the statement verbatim, the requirements they stated with their own ids, and what they excluded.
|
||||||
|
- **It carries how to run the agent,** so one file starts an Execution.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
Accepted by the owner on 2026-10-11, with each of these:
|
||||||
|
|
||||||
|
1. **The file supplied is never altered.** The agent writes a completed copy beside it, with what it assumed, what is still unknown, where each value came from, and a status.
|
||||||
|
2. **A choice the customer states binds the design.** A product, a topology or a setting written in the spec is a constraint every Specialist honours. The rule that requirements name no products is about what the agents introduce, not what the customer has already decided.
|
||||||
|
3. **A platform foundation is in scope,** alongside the application workloads the agent was built on.
|
||||||
|
4. **An engagement and a Customer case are the same thing.** A spec may name either; an engagement reference is turned into the case's id.
|
||||||
|
5. **The spec is provider-agnostic in shape.** Provider vocabulary enters only as values a provider's bundle allows.
|
||||||
|
6. **An essential field that is not settled refuses a start that has nobody to ask.** With someone present it is asked for.
|
||||||
|
7. **Scoping becomes a Phase the customer approves,** before requirements discovery.
|
||||||
|
8. **Where the spec and a prose request disagree, the spec wins.**
|
||||||
|
|
||||||
|
## Considered Options
|
||||||
|
|
||||||
|
- **Judge the free-text request against a list of topics** with a model or Jev: the first proposal. Rejected once a spec was on the table: a field that is filled or not is a fact, and a judgement about prose is not.
|
||||||
|
- **One file the agent completes in place:** rejected. An author must be able to see that what they wrote is what was used.
|
||||||
|
- **Keep product names out of the spec, as out of requirements:** rejected. A customer who has decided on an interconnect or a named appliance has stated a constraint; refusing to record it would lose it.
|
||||||
|
- **A separate spec for platform foundations:** rejected. One catalogue with tiers by kind serves both, and a migration is usually some of each.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The Requirements chain gains a start: a requirement traces to the customer's own requirement id or to a spec field.
|
||||||
|
- The requirements analyst stops being where questions are asked and assumptions made. That moves to scoping.
|
||||||
|
- A Change request may be an edited spec, compared field by field in code.
|
||||||
|
- An evaluation case may be a spec, in place of answers matched by pattern.
|
||||||
|
- The glossary gains **Scope spec** and **Scoping**, and Customer case says that an engagement is the same thing.
|
||||||
|
- A platform foundation is new work for the agents. It needs its own evaluation case before it is relied on.
|
||||||
|
- The spec holds client names and details. Like lesson candidates it is client data: kept with the Customer case, never in the repository, apart from the fictional benchmark specs.
|
||||||
|
|
||||||
|
This is built in stages; `.agents/plans/iter-09-scoping/` says which are done.
|
||||||
@@ -545,5 +545,21 @@
|
|||||||
]
|
]
|
||||||
],
|
],
|
||||||
"terraform_modules": true
|
"terraform_modules": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "case-08-landing-zone",
|
||||||
|
"name": "A regulated platform foundation, started from a scope spec",
|
||||||
|
"scope": "landing-zone.yaml",
|
||||||
|
"required_products": [
|
||||||
|
"Cloud Interconnect",
|
||||||
|
"Shared VPC",
|
||||||
|
"Cloud KMS"
|
||||||
|
],
|
||||||
|
"scripted_answers": [
|
||||||
|
[
|
||||||
|
".*",
|
||||||
|
"Proceed on the scope spec as supplied. Where it says decide, choose a sensible default and record it as an assumption."
|
||||||
|
]
|
||||||
|
]
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
|||||||
117
eval/datasets/specs/landing-zone.yaml
Normal file
117
eval/datasets/specs/landing-zone.yaml
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
# A benchmark scope spec: a regulated platform foundation. A fictional customer.
|
||||||
|
schema_version: 1
|
||||||
|
|
||||||
|
metadata:
|
||||||
|
engagement_id: case-08-landing-zone
|
||||||
|
client_name: Example Payments UK
|
||||||
|
author: Evaluation
|
||||||
|
revision: "1.0.0"
|
||||||
|
|
||||||
|
engagement:
|
||||||
|
kind: platform_foundation
|
||||||
|
|
||||||
|
intent:
|
||||||
|
statement: |
|
||||||
|
We are a UK-regulated payment platform expanding into Google Cloud. We need an
|
||||||
|
enterprise-grade landing zone ready in under 10 weeks to support our core payments
|
||||||
|
engine. All payment processing must comply with DORA and PCI-DSS v4.0. We have
|
||||||
|
existing on-premises systems that require private, low-latency connectivity over a
|
||||||
|
10Gbps dedicated interconnect. We use Entra ID for corporate single sign-on and
|
||||||
|
GitHub Actions for our delivery pipelines. Security requires that all egress
|
||||||
|
traffic flows through centralised inspection appliances before touching the public
|
||||||
|
internet, and customer data must be encrypted with keys we control.
|
||||||
|
requirements:
|
||||||
|
- id: REQ-01
|
||||||
|
category: Timeline
|
||||||
|
statement: Platform foundation must be ready for workload ingestion within 10 weeks.
|
||||||
|
- id: REQ-02
|
||||||
|
category: Compliance
|
||||||
|
statement: Must achieve DORA resiliency alignment and PCI-DSS v4.0 perimeter isolation.
|
||||||
|
- id: REQ-03
|
||||||
|
category: Networking
|
||||||
|
statement: 10Gbps on-premises hybrid connectivity; centralised inspection appliances for egress.
|
||||||
|
- id: REQ-04
|
||||||
|
category: Identity and security
|
||||||
|
statement: Workload identity federation with GitHub Actions; Entra ID single sign-on; customer-managed encryption keys.
|
||||||
|
exclusions:
|
||||||
|
- Application container deployment; workloads are managed by application squads.
|
||||||
|
- An active-active secondary region in phase 1; warm standby is acceptable.
|
||||||
|
success_criteria: decide
|
||||||
|
|
||||||
|
estate:
|
||||||
|
cloud_provider: gcp
|
||||||
|
mode: greenfield
|
||||||
|
regions:
|
||||||
|
primary: europe-west2
|
||||||
|
secondary: europe-west1
|
||||||
|
resiliency_tier: warm_standby
|
||||||
|
data_residency: United Kingdom and European Union
|
||||||
|
existing_systems: On-premises payment systems that stay in place and connect over the interconnect.
|
||||||
|
|
||||||
|
workload:
|
||||||
|
users: not_applicable
|
||||||
|
availability:
|
||||||
|
target: decide
|
||||||
|
hours: always
|
||||||
|
recovery_time: decide
|
||||||
|
recovery_point: decide
|
||||||
|
|
||||||
|
compliance:
|
||||||
|
industry: financial_services
|
||||||
|
frameworks: [DORA, ISO27001, PCI_DSS_v4]
|
||||||
|
data_classification: confidential
|
||||||
|
encryption:
|
||||||
|
type: cmek
|
||||||
|
key_rotation_days: 90
|
||||||
|
audit_logging:
|
||||||
|
retention_days: 365
|
||||||
|
immutable_retention_policy: true
|
||||||
|
|
||||||
|
networking:
|
||||||
|
exposure: internal_only
|
||||||
|
topology: shared_vpc_hub_and_spoke
|
||||||
|
hybrid_connectivity:
|
||||||
|
enabled: true
|
||||||
|
type: cloud_interconnect
|
||||||
|
bandwidth_gbps: 10
|
||||||
|
egress_control:
|
||||||
|
model: centralized_appliance
|
||||||
|
appliance_vendor: decide
|
||||||
|
cidr_allocations: decide
|
||||||
|
|
||||||
|
governance:
|
||||||
|
identity_source_of_truth: Entra ID
|
||||||
|
pipeline_authentication: workload identity federation
|
||||||
|
environment_tiers:
|
||||||
|
- {name: development, archetype: corp_workload}
|
||||||
|
- {name: staging, archetype: corp_workload}
|
||||||
|
- {name: production, archetype: pci_regulated_workload}
|
||||||
|
|
||||||
|
operations:
|
||||||
|
operating_model: central platform team brokering access for application squads
|
||||||
|
support_hours: always
|
||||||
|
monitoring: decide
|
||||||
|
change_process: decide
|
||||||
|
|
||||||
|
integrations: decide
|
||||||
|
|
||||||
|
delivery:
|
||||||
|
window_weeks: 10
|
||||||
|
commercial_model: fixed_price_milestone
|
||||||
|
iac_tooling: terraform
|
||||||
|
ci_cd: GitHub Actions
|
||||||
|
team_size: 3
|
||||||
|
working_days_per_week: 5
|
||||||
|
|
||||||
|
cost:
|
||||||
|
monthly_budget: decide
|
||||||
|
currency: GBP
|
||||||
|
|
||||||
|
constraints:
|
||||||
|
products_required: decide
|
||||||
|
products_forbidden: decide
|
||||||
|
|
||||||
|
run:
|
||||||
|
pre_approved: true
|
||||||
|
allow_cloud_read: false
|
||||||
|
terraform_modules: false
|
||||||
@@ -18,6 +18,8 @@ from dataclasses import dataclass, field
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
from app.adk.runtime import ToolResult
|
from app.adk.runtime import ToolResult
|
||||||
from app.agent import Interaction
|
from app.agent import Interaction
|
||||||
from app.tools.assessment import QUALITY_BAR, QUALITY_RUBRIC, checklist_question
|
from app.tools.assessment import QUALITY_BAR, QUALITY_RUBRIC, checklist_question
|
||||||
@@ -73,6 +75,8 @@ class CaseSpec:
|
|||||||
overlay_skills: tuple[str, ...] = ()
|
overlay_skills: tuple[str, ...] = ()
|
||||||
# Whether the case's Executions opt into published Terraform modules.
|
# Whether the case's Executions opt into published Terraform modules.
|
||||||
terraform_modules: bool = False
|
terraform_modules: bool = False
|
||||||
|
# A scope spec (ADR-0008), as YAML text, that the case's Execution starts from.
|
||||||
|
scope: str = ""
|
||||||
# What the infrastructure code must and must not contain, as regular
|
# What the infrastructure code must and must not contain, as regular
|
||||||
# expressions over all of it: how a case checks that standards from an overlay
|
# expressions over all of it: how a case checks that standards from an overlay
|
||||||
# Skill reached the code. Each is {"name", "require"} or {"name", "forbid"}.
|
# Skill reached the code. Each is {"name", "require"} or {"name", "forbid"}.
|
||||||
@@ -106,12 +110,27 @@ class CaseResult:
|
|||||||
return self.score >= PASS_THRESHOLD
|
return self.score >= PASS_THRESHOLD
|
||||||
|
|
||||||
|
|
||||||
|
def _scope_of(path: Path, item: Mapping[str, Any]) -> str:
|
||||||
|
"""The text of the scope spec a case names, from the specs folder beside the
|
||||||
|
dataset; "" when it names none."""
|
||||||
|
name = item.get("scope")
|
||||||
|
return (path.parent / "specs" / str(name)).read_text("utf-8") if name else ""
|
||||||
|
|
||||||
|
|
||||||
|
def _request_of(item: Mapping[str, Any], scope: str) -> str:
|
||||||
|
"""A case's request: its own, or the statement of intent in its scope spec."""
|
||||||
|
if item.get("workflow_request"):
|
||||||
|
return str(item["workflow_request"])
|
||||||
|
spec = yaml.safe_load(scope) if scope else {}
|
||||||
|
return str(((spec or {}).get("intent") or {}).get("statement", "")).strip()
|
||||||
|
|
||||||
|
|
||||||
def load_cases(path: Path) -> list[CaseSpec]:
|
def load_cases(path: Path) -> list[CaseSpec]:
|
||||||
return [
|
return [
|
||||||
CaseSpec(
|
CaseSpec(
|
||||||
id=item["id"],
|
id=item["id"],
|
||||||
name=item["name"],
|
name=item["name"],
|
||||||
workflow_request=item["workflow_request"],
|
workflow_request=_request_of(item, _scope_of(path, item)),
|
||||||
required_products=tuple(item["required_products"]),
|
required_products=tuple(item["required_products"]),
|
||||||
scripted_answers=tuple(
|
scripted_answers=tuple(
|
||||||
tuple(pair) for pair in item.get("scripted_answers", [])
|
tuple(pair) for pair in item.get("scripted_answers", [])
|
||||||
@@ -121,6 +140,7 @@ def load_cases(path: Path) -> list[CaseSpec]:
|
|||||||
change=str(item.get("change", "")),
|
change=str(item.get("change", "")),
|
||||||
overlay_skills=tuple(item.get("overlay_skills", ())),
|
overlay_skills=tuple(item.get("overlay_skills", ())),
|
||||||
terraform_modules=bool(item.get("terraform_modules", False)),
|
terraform_modules=bool(item.get("terraform_modules", False)),
|
||||||
|
scope=_scope_of(path, item),
|
||||||
code_standards=tuple(item.get("code_standards", ())),
|
code_standards=tuple(item.get("code_standards", ())),
|
||||||
)
|
)
|
||||||
for item in json.loads(path.read_text(encoding="utf-8"))
|
for item in json.loads(path.read_text(encoding="utf-8"))
|
||||||
@@ -265,11 +285,13 @@ async def score_execution(
|
|||||||
)
|
)
|
||||||
|
|
||||||
requirements = content(DeliverableKind.REQUIREMENTS)
|
requirements = content(DeliverableKind.REQUIREMENTS)
|
||||||
|
# What the customer named themselves, in the request or in their scope spec. A
|
||||||
|
# choice they stated is a constraint to record, not a product leaking in.
|
||||||
|
stated = case.workflow_request + "\n" + case.scope.replace("_", " ")
|
||||||
leaked = [
|
leaked = [
|
||||||
product
|
product
|
||||||
for product in case.required_products
|
for product in case.required_products
|
||||||
if _mentions(requirements, product)
|
if _mentions(requirements, product) and not _mentions(stated, product)
|
||||||
and not _mentions(case.workflow_request, product)
|
|
||||||
]
|
]
|
||||||
checks.append(
|
checks.append(
|
||||||
Check(
|
Check(
|
||||||
|
|||||||
@@ -133,6 +133,7 @@ async def run_with_scripted_user(
|
|||||||
execution_id=execution_id,
|
execution_id=execution_id,
|
||||||
overlay_skills=case.overlay_skills,
|
overlay_skills=case.overlay_skills,
|
||||||
terraform_modules=case.terraform_modules,
|
terraform_modules=case.terraform_modules,
|
||||||
|
scope=case.scope or None,
|
||||||
)
|
)
|
||||||
for _ in range(max_user_turns):
|
for _ in range(max_user_turns):
|
||||||
if interaction.status != "awaiting_user" or interaction.prompt is None:
|
if interaction.status != "awaiting_user" or interaction.prompt is None:
|
||||||
@@ -196,6 +197,7 @@ async def run_case(
|
|||||||
scripted_answers=case.scripted_answers,
|
scripted_answers=case.scripted_answers,
|
||||||
overlay_skills=case.overlay_skills,
|
overlay_skills=case.overlay_skills,
|
||||||
terraform_modules=case.terraform_modules,
|
terraform_modules=case.terraform_modules,
|
||||||
|
scope=case.scope or None,
|
||||||
)
|
)
|
||||||
baseline_tokens = token_report(adk_runtime.usage)["total"].get("total")
|
baseline_tokens = token_report(adk_runtime.usage)["total"].get("total")
|
||||||
baseline_seconds = round(time.monotonic() - started, 1)
|
baseline_seconds = round(time.monotonic() - started, 1)
|
||||||
@@ -211,6 +213,7 @@ async def run_case(
|
|||||||
scripted_answers=case.scripted_answers,
|
scripted_answers=case.scripted_answers,
|
||||||
overlay_skills=case.overlay_skills,
|
overlay_skills=case.overlay_skills,
|
||||||
terraform_modules=case.terraform_modules,
|
terraform_modules=case.terraform_modules,
|
||||||
|
scope=case.scope or None,
|
||||||
baseline=base.execution_id,
|
baseline=base.execution_id,
|
||||||
)
|
)
|
||||||
elif case.interactive:
|
elif case.interactive:
|
||||||
@@ -227,6 +230,7 @@ async def run_case(
|
|||||||
scripted_answers=case.scripted_answers,
|
scripted_answers=case.scripted_answers,
|
||||||
overlay_skills=case.overlay_skills,
|
overlay_skills=case.overlay_skills,
|
||||||
terraform_modules=case.terraform_modules,
|
terraform_modules=case.terraform_modules,
|
||||||
|
scope=case.scope or None,
|
||||||
)
|
)
|
||||||
elapsed = time.monotonic() - started
|
elapsed = time.monotonic() - started
|
||||||
write_progress()
|
write_progress()
|
||||||
|
|||||||
@@ -24,6 +24,8 @@ models:
|
|||||||
lesson-curator: gemini/gemini-3.1-pro-preview
|
lesson-curator: gemini/gemini-3.1-pro-preview
|
||||||
# The same reading the validator makes of a design's coverage, made earlier.
|
# The same reading the validator makes of a design's coverage, made earlier.
|
||||||
coverage-reviewer: gemini/gemini-3.1-pro-preview
|
coverage-reviewer: gemini/gemini-3.1-pro-preview
|
||||||
|
# Dividing and estimating the build is judgement about the whole design.
|
||||||
|
delivery-planner: gemini/gemini-3.1-pro-preview
|
||||||
# A lighter model for a role's first attempt, used when MODEL_TIERS is on. Only a
|
# A lighter model for a role's first attempt, used when MODEL_TIERS is on. Only a
|
||||||
# role whose work is checked by something else belongs here: the code is checked
|
# role whose work is checked by something else belongs here: the code is checked
|
||||||
# by terraform validate and by the validator. The validator is the check, and a
|
# by terraform validate and by the validator. The validator is the check, and a
|
||||||
@@ -39,10 +41,12 @@ models:
|
|||||||
validator: 300 # timed out five times at 180 s in a run of 2026-10-07
|
validator: 300 # timed out five times at 180 s in a run of 2026-10-07
|
||||||
packager: 300
|
packager: 300
|
||||||
coverage-reviewer: 300
|
coverage-reviewer: 300
|
||||||
|
delivery-planner: 300
|
||||||
|
|
||||||
# Skills each role may load. Role files stay provider-neutral; the bundle names Skills.
|
# Skills each role may load. Role files stay provider-neutral; the bundle names Skills.
|
||||||
role_skills:
|
role_skills:
|
||||||
orchestrator: [google-cloud-solution-architecture]
|
orchestrator: [google-cloud-solution-architecture]
|
||||||
|
scope-analyst: [scoping]
|
||||||
requirements-analyst: [google-cloud-solution-architecture, requirements-chain, cloud-discovery]
|
requirements-analyst: [google-cloud-solution-architecture, requirements-chain, cloud-discovery]
|
||||||
decomposer: [google-cloud-solution-architecture, requirements-chain]
|
decomposer: [google-cloud-solution-architecture, requirements-chain]
|
||||||
product-selector: [google-cloud-solution-architecture, requirements-chain]
|
product-selector: [google-cloud-solution-architecture, requirements-chain]
|
||||||
@@ -51,12 +55,24 @@ role_skills:
|
|||||||
target-architect: [google-cloud-solution-architecture, requirements-chain]
|
target-architect: [google-cloud-solution-architecture, requirements-chain]
|
||||||
iac-author: [google-cloud-solution-architecture, requirements-chain, iac-validation, google-cloud-iac-patterns]
|
iac-author: [google-cloud-solution-architecture, requirements-chain, iac-validation, google-cloud-iac-patterns]
|
||||||
validator: [google-cloud-solution-architecture, requirements-chain, iac-validation]
|
validator: [google-cloud-solution-architecture, requirements-chain, iac-validation]
|
||||||
|
delivery-planner: [requirements-chain]
|
||||||
packager: [google-cloud-solution-architecture]
|
packager: [google-cloud-solution-architecture]
|
||||||
|
|
||||||
# Where approved lessons are published (ADR-0004). A Specialist is given this Skill
|
# Where approved lessons are published (ADR-0004). A Specialist is given this Skill
|
||||||
# once it holds lessons for that Specialist's Deliverable.
|
# once it holds lessons for that Specialist's Deliverable.
|
||||||
lessons_skill: google-cloud-lessons
|
lessons_skill: google-cloud-lessons
|
||||||
|
|
||||||
|
# The scope spec (ADR-0008). `skill` holds its catalogue, which names no Cloud
|
||||||
|
# provider. `values` are what this provider allows for the fields the catalogue
|
||||||
|
# leaves to a provider, in this provider's own terms.
|
||||||
|
scope:
|
||||||
|
skill: scoping
|
||||||
|
values:
|
||||||
|
estate.cloud_provider: [gcp]
|
||||||
|
compliance.encryption.type: [cmek, cmek_hsm]
|
||||||
|
networking.topology: [single_vpc, shared_vpc, shared_vpc_hub_and_spoke]
|
||||||
|
networking.hybrid_connectivity.type: [cloud_vpn, cloud_interconnect, partner_interconnect]
|
||||||
|
|
||||||
# One advisor per architecture-framework pillar; each loads only its pillar's Skill.
|
# One advisor per architecture-framework pillar; each loads only its pillar's Skill.
|
||||||
pillar_skills:
|
pillar_skills:
|
||||||
security: google-cloud-waf-security
|
security: google-cloud-waf-security
|
||||||
|
|||||||
@@ -37,6 +37,19 @@ Every non-functional requirement lists its `conditions`: one entry for each sepa
|
|||||||
|
|
||||||
`kind` is `functional` or `non_functional`. Write one requirement per entry, so each can be checked on its own.
|
`kind` is `functional` or `non_functional`. Write one requirement per entry, so each can be checked on its own.
|
||||||
|
|
||||||
|
When the customer's scope is in your context, a requirement says what it comes from in `from`: the ids of the customer's own requirements, and the scope fields it carries.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"id": "NFR-4", "kind": "non_functional",
|
||||||
|
"text": "The network is a hub with spokes, joined to on-premises over a private 10Gbps connection.",
|
||||||
|
"conditions": ["A hub network is connected to every spoke network.", "A dedicated 10Gbps connection joins the hub to on-premises."],
|
||||||
|
"from": ["REQ-03", "networking.topology", "networking.hybrid_connectivity.type", "networking.hybrid_connectivity.bandwidth_gbps"]}
|
||||||
|
```
|
||||||
|
|
||||||
|
- Every requirement the customer wrote (`scope.stated.intent.requirements`) is named in the `from` of the requirement that covers it.
|
||||||
|
- Every field under `scope.binding` is a choice the customer stated that the design must honour. Each is named in the `from` of a non-functional requirement with at least one condition that shows it was honoured. Write the condition about what must exist or be set, in terms a Check can verify: "a hub network is connected to every spoke network", not "the topology is hub and spoke".
|
||||||
|
- One requirement may carry several fields that belong together. A submission is refused while any is carried by none.
|
||||||
|
|
||||||
### `components.json` (technical decomposition)
|
### `components.json` (technical decomposition)
|
||||||
|
|
||||||
A list of components and the requirement ids each serves. Every functional requirement must be served by at least one component.
|
A list of components and the requirement ids each serves. Every functional requirement must be served by at least one component.
|
||||||
|
|||||||
37
skills/scoping/SKILL.md
Normal file
37
skills/scoping/SKILL.md
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
---
|
||||||
|
name: scoping
|
||||||
|
description: >-
|
||||||
|
The scope spec: the structured statement of what a customer wants and the
|
||||||
|
facts a solution architecture depends on, with the catalogue of its fields.
|
||||||
|
Use when reading a scope spec in your context, to know what a field means,
|
||||||
|
whether the customer stated it or left it to you, and what binds your
|
||||||
|
Deliverable.
|
||||||
|
metadata:
|
||||||
|
scope: every Deliverable
|
||||||
|
---
|
||||||
|
|
||||||
|
# Scoping
|
||||||
|
|
||||||
|
A scope spec is a YAML file the customer's architect supplies, complete or partly filled. It sits beside the Intent, which stays in the customer's words. `assets/catalogue.yaml` lists every field it may hold: its type, when it must be settled, the question to ask for it, and guidance for a default.
|
||||||
|
|
||||||
|
## What a field says
|
||||||
|
|
||||||
|
| In the spec | Meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| a value | The customer states this. It is theirs: use it as given. |
|
||||||
|
| `ask`, or left out | Not known yet. |
|
||||||
|
| `decide` | The customer leaves the choice to you. Choose, and record it as an assumption with what it affects. |
|
||||||
|
| `not_applicable` | It does not apply. Ask nothing and assume nothing. |
|
||||||
|
|
||||||
|
## Rules for every Specialist
|
||||||
|
|
||||||
|
- **A stated value binds you.** A region, a topology, a product, a setting or a number the spec states is the customer's decision. Honour it. If you believe it is wrong, say so with `needs_input`; do not quietly replace it.
|
||||||
|
- **A stated choice is not yours to generalise.** The rule that requirements name no products is about what you introduce. What the customer wrote stays as they wrote it.
|
||||||
|
- **Build nothing the customer excluded.** `intent.exclusions` is out of scope for every Deliverable.
|
||||||
|
- **The customer's own requirements carry their ids.** Each entry in `intent.requirements` must be met. When your work serves one, name its id.
|
||||||
|
- **An assumption is not the customer's decision.** A field under `assumed` was chosen for them, with the reason given. Work from it, and say so where your Deliverable depends on it: it may change when they answer.
|
||||||
|
- **Do not fill a gap silently.** A field under `to_ask` is still unknown. When your Deliverable needs it, assume a default and say that you did.
|
||||||
|
|
||||||
|
## The kinds of engagement
|
||||||
|
|
||||||
|
`engagement.kind` is `workload` (an application), `platform_foundation` (a landing zone or shared platform that workloads will later run on) or `migration`. A platform foundation has no users or response times of its own: its subject is the organisation's structure, shared networking, identity, environments and guardrails.
|
||||||
345
skills/scoping/assets/catalogue.yaml
Normal file
345
skills/scoping/assets/catalogue.yaml
Normal file
@@ -0,0 +1,345 @@
|
|||||||
|
# The scope spec's catalogue: every field a scope spec may hold.
|
||||||
|
#
|
||||||
|
# A field is a dotted path. `type` is one of: string, text, integer, number, boolean,
|
||||||
|
# date, list, enum, records. `tier` says when it must be settled: essential (always,
|
||||||
|
# before any work), important (asked when someone is present) or optional. A tier
|
||||||
|
# may differ by the kind of engagement. `question` is what to ask the customer for
|
||||||
|
# it. `default` guides the choice when the customer says "decide". `affects` says
|
||||||
|
# what a wrong assumption would change. `binds: true` marks a choice a design must
|
||||||
|
# honour: stated by the customer, it has to reach a requirement and its Checks.
|
||||||
|
#
|
||||||
|
# This catalogue names no Cloud provider. A provider's bundle adds allowed values
|
||||||
|
# for the fields marked `provider_values`.
|
||||||
|
schema_version: 1
|
||||||
|
kinds: [workload, platform_foundation, migration]
|
||||||
|
|
||||||
|
fields:
|
||||||
|
metadata.customer_case:
|
||||||
|
type: string
|
||||||
|
tier: essential
|
||||||
|
aliases: [metadata.engagement_id]
|
||||||
|
question: What short reference identifies this engagement?
|
||||||
|
metadata.engagement_reference:
|
||||||
|
type: string
|
||||||
|
metadata.client_name:
|
||||||
|
type: string
|
||||||
|
metadata.author:
|
||||||
|
type: string
|
||||||
|
metadata.revision:
|
||||||
|
type: string
|
||||||
|
|
||||||
|
engagement.kind:
|
||||||
|
type: enum
|
||||||
|
values: [workload, platform_foundation, migration]
|
||||||
|
tier: essential
|
||||||
|
question: >-
|
||||||
|
Is this an application workload, a platform foundation such as a landing zone,
|
||||||
|
or a migration of existing systems?
|
||||||
|
|
||||||
|
intent.statement:
|
||||||
|
type: text
|
||||||
|
tier: essential
|
||||||
|
question: In the customer's own words, what do they want?
|
||||||
|
intent.requirements:
|
||||||
|
type: records
|
||||||
|
keys: [id, statement]
|
||||||
|
optional_keys: [category]
|
||||||
|
question: Which requirements has the customer stated outright?
|
||||||
|
intent.exclusions:
|
||||||
|
type: list
|
||||||
|
tier: important
|
||||||
|
question: What has the customer said is out of scope?
|
||||||
|
default: Nothing is excluded beyond what the statement implies.
|
||||||
|
affects: what is designed and planned
|
||||||
|
intent.success_criteria:
|
||||||
|
type: list
|
||||||
|
question: How will the customer judge this a success?
|
||||||
|
default: The stated requirements are met.
|
||||||
|
|
||||||
|
estate.cloud_provider:
|
||||||
|
type: enum
|
||||||
|
provider_values: true
|
||||||
|
tier: essential
|
||||||
|
question: Which Cloud provider is this for?
|
||||||
|
estate.mode:
|
||||||
|
type: enum
|
||||||
|
values: [greenfield, brownfield_expansion, migration]
|
||||||
|
tier: essential
|
||||||
|
question: Is this a new estate, an extension of an existing one, or a migration?
|
||||||
|
estate.regions.primary:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: essential
|
||||||
|
question: Which region should it run in?
|
||||||
|
estate.regions.secondary:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
question: Is there a second region, and which?
|
||||||
|
default: None, unless availability or recovery targets need one.
|
||||||
|
affects: resilience design and cost
|
||||||
|
estate.regions.resiliency_tier:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
values: [single_region, warm_standby, active_active]
|
||||||
|
tier: important
|
||||||
|
question: How should it survive the loss of a region?
|
||||||
|
default: single_region
|
||||||
|
affects: resilience design and cost
|
||||||
|
estate.data_residency:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: essential
|
||||||
|
question: Where must data stay?
|
||||||
|
estate.existing_systems:
|
||||||
|
type: text
|
||||||
|
tier: {migration: essential, platform_foundation: important, workload: important}
|
||||||
|
question: What exists today that this replaces, extends or must connect to?
|
||||||
|
default: Nothing exists today.
|
||||||
|
affects: connectivity, migration work and the plan
|
||||||
|
|
||||||
|
workload.users.who:
|
||||||
|
type: enum
|
||||||
|
values: [employees, customers, partners, public, systems]
|
||||||
|
tier: {workload: essential, migration: essential}
|
||||||
|
question: Who uses it?
|
||||||
|
workload.users.count:
|
||||||
|
type: integer
|
||||||
|
tier: {workload: essential, migration: essential}
|
||||||
|
question: How many users?
|
||||||
|
workload.users.peak_concurrent:
|
||||||
|
type: integer
|
||||||
|
tier: {workload: important, migration: important}
|
||||||
|
question: How many use it at once at peak?
|
||||||
|
default: A tenth of the user count.
|
||||||
|
affects: scaling limits and cost
|
||||||
|
workload.availability.target:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: {workload: essential, migration: essential, platform_foundation: important}
|
||||||
|
question: What availability is required, as a percentage?
|
||||||
|
default: "99.9%"
|
||||||
|
affects: resilience design and cost
|
||||||
|
workload.availability.hours:
|
||||||
|
type: enum
|
||||||
|
values: [business_hours, extended, always]
|
||||||
|
tier: important
|
||||||
|
question: During which hours must it be available?
|
||||||
|
default: always
|
||||||
|
affects: scaling to zero and support
|
||||||
|
workload.availability.recovery_time:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: important
|
||||||
|
question: How quickly must it recover from a failure?
|
||||||
|
default: 4 hours
|
||||||
|
affects: backup and standby design
|
||||||
|
workload.availability.recovery_point:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: important
|
||||||
|
question: How much recent data may be lost in a failure?
|
||||||
|
default: 1 hour
|
||||||
|
affects: backup frequency
|
||||||
|
|
||||||
|
compliance.industry:
|
||||||
|
type: string
|
||||||
|
question: "Which industry is the customer in?"
|
||||||
|
compliance.frameworks:
|
||||||
|
binds: true
|
||||||
|
type: list
|
||||||
|
tier: essential
|
||||||
|
question: Which regulations or standards apply? An empty list if none.
|
||||||
|
compliance.data_classification:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
values: [public, internal, confidential, restricted]
|
||||||
|
tier: essential
|
||||||
|
question: How sensitive is the data?
|
||||||
|
compliance.encryption.type:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
values: [provider_managed, customer_managed]
|
||||||
|
provider_values: true
|
||||||
|
tier: important
|
||||||
|
question: Must the customer control the encryption keys?
|
||||||
|
default: provider_managed
|
||||||
|
affects: key management design
|
||||||
|
compliance.encryption.key_rotation_days:
|
||||||
|
binds: true
|
||||||
|
type: integer
|
||||||
|
question: How often must keys rotate, in days?
|
||||||
|
default: 90
|
||||||
|
compliance.audit_logging.retention_days:
|
||||||
|
binds: true
|
||||||
|
type: integer
|
||||||
|
tier: important
|
||||||
|
question: How long must audit logs be kept, in days?
|
||||||
|
default: 365
|
||||||
|
affects: storage cost and log design
|
||||||
|
compliance.audit_logging.immutable_retention_policy:
|
||||||
|
binds: true
|
||||||
|
type: boolean
|
||||||
|
question: Must audit logs be undeletable for that time?
|
||||||
|
default: false
|
||||||
|
|
||||||
|
networking.exposure:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
values: [internal_only, internet_facing]
|
||||||
|
tier: important
|
||||||
|
question: Is it reached from the internet or only from inside?
|
||||||
|
default: internal_only
|
||||||
|
affects: perimeter design
|
||||||
|
networking.topology:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
provider_values: true
|
||||||
|
tier: {platform_foundation: important}
|
||||||
|
question: Is a network topology already decided?
|
||||||
|
default: The simplest topology that meets the requirements.
|
||||||
|
affects: the whole network design
|
||||||
|
networking.hybrid_connectivity.enabled:
|
||||||
|
binds: true
|
||||||
|
type: boolean
|
||||||
|
tier: {platform_foundation: important, migration: important}
|
||||||
|
question: Must it connect privately to on-premises or another cloud?
|
||||||
|
default: false
|
||||||
|
affects: network design and lead time
|
||||||
|
networking.hybrid_connectivity.type:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
provider_values: true
|
||||||
|
question: By what kind of connection?
|
||||||
|
networking.hybrid_connectivity.bandwidth_gbps:
|
||||||
|
binds: true
|
||||||
|
type: number
|
||||||
|
question: At what bandwidth, in Gbps?
|
||||||
|
networking.egress_control.model:
|
||||||
|
binds: true
|
||||||
|
type: enum
|
||||||
|
values: [direct, provider_managed, centralized_appliance]
|
||||||
|
question: How must traffic to the internet be controlled?
|
||||||
|
default: provider_managed
|
||||||
|
affects: network design and cost
|
||||||
|
networking.egress_control.appliance_vendor:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
question: Which inspection appliance, if one is mandated?
|
||||||
|
networking.cidr_allocations:
|
||||||
|
binds: true
|
||||||
|
type: text
|
||||||
|
tier: {platform_foundation: important}
|
||||||
|
question: Which IP ranges are allocated, and how large per environment?
|
||||||
|
default: Private ranges that do not overlap, sized for growth.
|
||||||
|
affects: every network resource
|
||||||
|
|
||||||
|
governance.identity_source_of_truth:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
tier: important
|
||||||
|
question: Which identity provider do people sign in with?
|
||||||
|
default: The Cloud provider's own identities.
|
||||||
|
affects: access design
|
||||||
|
governance.pipeline_authentication:
|
||||||
|
binds: true
|
||||||
|
type: string
|
||||||
|
question: How do delivery pipelines authenticate?
|
||||||
|
default: Short-lived federated credentials, no stored keys.
|
||||||
|
governance.environment_tiers:
|
||||||
|
binds: true
|
||||||
|
type: records
|
||||||
|
keys: [name]
|
||||||
|
optional_keys: [archetype]
|
||||||
|
tier: important
|
||||||
|
question: Which environments are needed?
|
||||||
|
default: development, test and production.
|
||||||
|
affects: the number of environments built and planned
|
||||||
|
|
||||||
|
operations.operating_model:
|
||||||
|
type: string
|
||||||
|
question: "Who runs the platform day to day?"
|
||||||
|
operations.support_hours:
|
||||||
|
type: enum
|
||||||
|
values: [business_hours, extended, always]
|
||||||
|
question: During which hours is it supported?
|
||||||
|
default: business_hours
|
||||||
|
operations.monitoring:
|
||||||
|
type: text
|
||||||
|
question: What monitoring and alerting is expected?
|
||||||
|
default: The Cloud provider's own monitoring, with alerts on availability and errors.
|
||||||
|
operations.change_process:
|
||||||
|
type: text
|
||||||
|
question: "How are changes approved and released?"
|
||||||
|
|
||||||
|
integrations:
|
||||||
|
type: records
|
||||||
|
keys: [name]
|
||||||
|
optional_keys: [direction, criticality]
|
||||||
|
question: Which other systems must it exchange data with?
|
||||||
|
default: None.
|
||||||
|
|
||||||
|
delivery.target_start_date:
|
||||||
|
type: date
|
||||||
|
question: "When should work start?"
|
||||||
|
delivery.window_weeks:
|
||||||
|
type: integer
|
||||||
|
tier: important
|
||||||
|
question: Is there a deadline, in weeks from the start?
|
||||||
|
default: No deadline; the plan gives the time the work needs.
|
||||||
|
affects: the plan and whether the scope fits
|
||||||
|
delivery.commercial_model:
|
||||||
|
type: enum
|
||||||
|
values: [time_and_materials, fixed_price_milestone, managed_service]
|
||||||
|
question: How is the work contracted?
|
||||||
|
delivery.iac_tooling:
|
||||||
|
type: string
|
||||||
|
question: "Which infrastructure-as-code tool?"
|
||||||
|
default: terraform
|
||||||
|
delivery.ci_cd:
|
||||||
|
type: string
|
||||||
|
question: "Which delivery pipeline tool?"
|
||||||
|
delivery.team_size:
|
||||||
|
type: integer
|
||||||
|
tier: important
|
||||||
|
question: How many engineers will build it?
|
||||||
|
default: 2
|
||||||
|
affects: the timeline
|
||||||
|
delivery.working_days_per_week:
|
||||||
|
type: integer
|
||||||
|
default: 5
|
||||||
|
|
||||||
|
cost.monthly_budget:
|
||||||
|
type: number
|
||||||
|
tier: {workload: essential, migration: essential, platform_foundation: important}
|
||||||
|
question: What is the monthly running budget?
|
||||||
|
default: No ceiling; cost is optimised within the requirements.
|
||||||
|
affects: product tiers and resilience choices
|
||||||
|
cost.currency:
|
||||||
|
type: string
|
||||||
|
default: "USD"
|
||||||
|
|
||||||
|
constraints.products_required:
|
||||||
|
binds: true
|
||||||
|
type: list
|
||||||
|
question: Which products has the customer mandated?
|
||||||
|
default: None.
|
||||||
|
constraints.products_forbidden:
|
||||||
|
binds: true
|
||||||
|
type: list
|
||||||
|
question: Which products are ruled out?
|
||||||
|
default: None.
|
||||||
|
constraints.other:
|
||||||
|
type: list
|
||||||
|
|
||||||
|
run.pre_approved:
|
||||||
|
type: boolean
|
||||||
|
default: false
|
||||||
|
run.allow_cloud_read:
|
||||||
|
type: boolean
|
||||||
|
default: false
|
||||||
|
run.overlay_skills:
|
||||||
|
type: list
|
||||||
|
default: []
|
||||||
|
run.terraform_modules:
|
||||||
|
type: boolean
|
||||||
|
default: false
|
||||||
@@ -23,6 +23,8 @@ from app.nodes.intake import Intake
|
|||||||
from app.providers import load_provider
|
from app.providers import load_provider
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine # noqa: F811
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
|
||||||
AGENTS_DIR = Path(__file__).parents[2] / "agents"
|
AGENTS_DIR = Path(__file__).parents[2] / "agents"
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
|
|||||||
@@ -18,7 +18,6 @@ from app.agent import ExecutionEngine
|
|||||||
from app.providers import load_provider
|
from app.providers import load_provider
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.tools.toolkit import make_tool_factory
|
from app.tools.toolkit import make_tool_factory
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
|
||||||
|
|
||||||
AGENTS_DIR = Path(__file__).parents[2] / "agents"
|
AGENTS_DIR = Path(__file__).parents[2] / "agents"
|
||||||
|
|
||||||
@@ -42,11 +41,6 @@ def call(tool: str, **args) -> list[types.Part]:
|
|||||||
return [types.Part(function_call=types.FunctionCall(name=tool, args=args))]
|
return [types.Part(function_call=types.FunctionCall(name=tool, args=args))]
|
||||||
|
|
||||||
|
|
||||||
def requirements_markdown() -> str:
|
|
||||||
sections = DEPENDENCY_GRAPH[DeliverableKind.REQUIREMENTS].sections
|
|
||||||
return "\n".join(f"## {section}\nDetails." for section in sections)
|
|
||||||
|
|
||||||
|
|
||||||
def task_context(request: LlmRequest) -> dict:
|
def task_context(request: LlmRequest) -> dict:
|
||||||
text = (request.contents[0].parts or [])[0].text or ""
|
text = (request.contents[0].parts or [])[0].text or ""
|
||||||
return json.loads(text.split("```json\n", 1)[1].rsplit("\n```", 1)[0])
|
return json.loads(text.split("```json\n", 1)[1].rsplit("\n```", 1)[0])
|
||||||
@@ -58,26 +52,24 @@ async def test_engine_runs_role_files_through_adk_with_one_voice(tmp_path):
|
|||||||
models = {
|
models = {
|
||||||
"orchestrator": RoleScriptedLlm(
|
"orchestrator": RoleScriptedLlm(
|
||||||
script=[
|
script=[
|
||||||
call("delegate", kinds=["requirements"]),
|
call("delegate", kinds=["scope"]),
|
||||||
call("ask_user", questions=[]),
|
call("ask_user", questions=[]),
|
||||||
call("delegate", kinds=["requirements"]),
|
call("delegate", kinds=["scope"]),
|
||||||
call("ask_user", questions=["Anything to add?"]),
|
call("ask_user", questions=["Anything to add?"]),
|
||||||
]
|
]
|
||||||
),
|
),
|
||||||
"requirements-analyst": RoleScriptedLlm(
|
"scope-analyst": RoleScriptedLlm(
|
||||||
script=[
|
script=[
|
||||||
call("load_skill", skill_name="google-cloud-solution-architecture"),
|
call("load_skill", skill_name="scoping"),
|
||||||
call(
|
call(
|
||||||
"needs_input", questions=["What availability target do you need?"]
|
"needs_input", questions=["What availability target do you need?"]
|
||||||
),
|
),
|
||||||
call(
|
call(
|
||||||
"submit_deliverable",
|
"submit_deliverable",
|
||||||
content=requirements_markdown(),
|
content="# Scope\n## Scope statement\nRegional event ingestion.\n",
|
||||||
attachments=[
|
attachments=[
|
||||||
{
|
{"path": name, "content": SAMPLE_DATA[name]}
|
||||||
"path": "requirements.json",
|
for name in ("scope.yaml", "scope.json")
|
||||||
"content": SAMPLE_DATA["requirements.json"],
|
|
||||||
}
|
|
||||||
],
|
],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
@@ -101,7 +93,7 @@ async def test_engine_runs_role_files_through_adk_with_one_voice(tmp_path):
|
|||||||
assert asked.prompt == {
|
assert asked.prompt == {
|
||||||
"questions": [
|
"questions": [
|
||||||
{
|
{
|
||||||
"from": "requirements",
|
"from": "scope",
|
||||||
"question": "What availability target do you need?",
|
"question": "What availability target do you need?",
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
@@ -109,11 +101,8 @@ async def test_engine_runs_role_files_through_adk_with_one_voice(tmp_path):
|
|||||||
|
|
||||||
answered = await engine.resume(asked.execution_id, {"reply": "99.9% in one region"})
|
answered = await engine.resume(asked.execution_id, {"reply": "99.9% in one region"})
|
||||||
|
|
||||||
assert (
|
assert answered.state["ledger"]["deliverables"]["scope"]["status"] == "submitted"
|
||||||
answered.state["ledger"]["deliverables"]["requirements"]["status"]
|
analyst_second_run = models["scope-analyst"].requests[-1]
|
||||||
== "submitted"
|
|
||||||
)
|
|
||||||
analyst_second_run = models["requirements-analyst"].requests[-1]
|
|
||||||
assert task_context(analyst_second_run)["answers"] == [
|
assert task_context(analyst_second_run)["answers"] == [
|
||||||
{
|
{
|
||||||
"questions": ["What availability target do you need?"],
|
"questions": ["What availability target do you need?"],
|
||||||
@@ -122,7 +111,5 @@ async def test_engine_runs_role_files_through_adk_with_one_voice(tmp_path):
|
|||||||
]
|
]
|
||||||
orchestrator_prompt = models["orchestrator"].requests[0].config.system_instruction
|
orchestrator_prompt = models["orchestrator"].requests[0].config.system_instruction
|
||||||
assert "You coordinate one solution-architecture Execution" in orchestrator_prompt
|
assert "You coordinate one solution-architecture Execution" in orchestrator_prompt
|
||||||
analyst_prompt = (
|
analyst_prompt = models["scope-analyst"].requests[0].config.system_instruction
|
||||||
models["requirements-analyst"].requests[0].config.system_instruction
|
|
||||||
)
|
|
||||||
assert "## How you work" in analyst_prompt
|
assert "## How you work" in analyst_prompt
|
||||||
|
|||||||
@@ -9,6 +9,8 @@ from app.adk.runtime import ToolResult
|
|||||||
from app.agent import Interaction
|
from app.agent import Interaction
|
||||||
from app.workflows.target_architecture import parse_graph, render_mermaid
|
from app.workflows.target_architecture import parse_graph, render_mermaid
|
||||||
from eval.scoring import CaseSpec, score_execution, template_headings
|
from eval.scoring import CaseSpec, score_execution, template_headings
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime
|
||||||
|
|
||||||
TEMPLATE = (
|
TEMPLATE = (
|
||||||
Path(__file__).parents[2]
|
Path(__file__).parents[2]
|
||||||
@@ -427,9 +429,8 @@ async def test_a_scripted_user_drives_an_interactive_execution(tmp_path):
|
|||||||
|
|
||||||
from langgraph.checkpoint.memory import InMemorySaver
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
|
|
||||||
from app.adk.fake_runtime import Call, FakeAgentRuntime, sample_submission
|
from app.adk.fake_runtime import Call, sample_submission
|
||||||
from app.adk.runtime import RoleSpec
|
from app.adk.runtime import RoleSpec
|
||||||
from app.agent import ExecutionEngine
|
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||||
from eval.solution_eval import run_with_scripted_user
|
from eval.solution_eval import run_with_scripted_user
|
||||||
@@ -819,3 +820,29 @@ async def test_a_case_with_no_standards_has_no_such_check():
|
|||||||
result = await score_execution(CASE, execution(), HEADINGS, passing_terraform)
|
result = await score_execution(CASE, execution(), HEADINGS, passing_terraform)
|
||||||
|
|
||||||
assert all(c.name != "code_follows_standards" for c in result.checks)
|
assert all(c.name != "code_follows_standards" for c in result.checks)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_product_the_customers_scope_spec_names_may_appear_in_requirements():
|
||||||
|
"""Case 08: the spec said shared_vpc_hub_and_spoke, and "Shared VPC" in the
|
||||||
|
requirements was scored as a product leaking in."""
|
||||||
|
named = {
|
||||||
|
"content": "## Functional requirements\nUse the Shared VPC topology stated.\n"
|
||||||
|
"## Current state\nNone.",
|
||||||
|
"attachments": {"requirements.json": SAMPLE_DATA["requirements.json"]},
|
||||||
|
}
|
||||||
|
with_spec = dataclasses.replace(
|
||||||
|
CASE,
|
||||||
|
required_products=("Shared VPC",),
|
||||||
|
scope="networking:\n topology: shared_vpc_hub_and_spoke\n",
|
||||||
|
)
|
||||||
|
without = dataclasses.replace(CASE, required_products=("Shared VPC",))
|
||||||
|
|
||||||
|
stated = await score_execution(
|
||||||
|
with_spec, execution(requirements=named), HEADINGS, passing_terraform
|
||||||
|
)
|
||||||
|
leaked = await score_execution(
|
||||||
|
without, execution(requirements=named), HEADINGS, passing_terraform
|
||||||
|
)
|
||||||
|
|
||||||
|
assert check(stated, "requirements_name_no_products").passed
|
||||||
|
assert not check(leaked, "requirements_name_no_products").passed
|
||||||
|
|||||||
@@ -24,6 +24,8 @@ from app.baseline import BaselineError, load_baseline
|
|||||||
from app.nodes.router import Router, Thresholds
|
from app.nodes.router import Router, Thresholds
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine # noqa: F811
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
CASE = "acme-webhooks"
|
CASE = "acme-webhooks"
|
||||||
|
|||||||
@@ -45,6 +45,7 @@ def test_dependencies_never_point_to_a_later_phase_and_form_no_cycle():
|
|||||||
|
|
||||||
def test_every_phase_groups_at_least_one_deliverable_with_a_distinct_specialist():
|
def test_every_phase_groups_at_least_one_deliverable_with_a_distinct_specialist():
|
||||||
assert PHASE_ORDER == (
|
assert PHASE_ORDER == (
|
||||||
|
PhaseId.SCOPING,
|
||||||
PhaseId.REQUIREMENTS_DISCOVERY,
|
PhaseId.REQUIREMENTS_DISCOVERY,
|
||||||
PhaseId.SOLUTION_ARCHITECTURE,
|
PhaseId.SOLUTION_ARCHITECTURE,
|
||||||
PhaseId.SOLUTION_IMPLEMENTATION,
|
PhaseId.SOLUTION_IMPLEMENTATION,
|
||||||
@@ -61,7 +62,8 @@ def test_the_target_architecture_sits_between_the_design_and_the_code():
|
|||||||
depends = {kind: set(spec.depends_on) for kind, spec in DEPENDENCY_GRAPH.items()}
|
depends = {kind: set(spec.depends_on) for kind, spec in DEPENDENCY_GRAPH.items()}
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
|
|
||||||
assert depends[K.REQUIREMENTS] == set()
|
assert depends[K.SCOPE] == set()
|
||||||
|
assert depends[K.REQUIREMENTS] == {K.SCOPE}
|
||||||
assert depends[K.TECHNICAL_DECOMPOSITION] == {K.REQUIREMENTS}
|
assert depends[K.TECHNICAL_DECOMPOSITION] == {K.REQUIREMENTS}
|
||||||
assert depends[K.PRODUCT_SELECTION] == {K.TECHNICAL_DECOMPOSITION}
|
assert depends[K.PRODUCT_SELECTION] == {K.TECHNICAL_DECOMPOSITION}
|
||||||
assert depends[K.DESIGN_RECOMMENDATIONS] == {K.PRODUCT_SELECTION, K.REQUIREMENTS}
|
assert depends[K.DESIGN_RECOMMENDATIONS] == {K.PRODUCT_SELECTION, K.REQUIREMENTS}
|
||||||
@@ -89,19 +91,26 @@ def test_the_design_is_approved_before_code_is_written():
|
|||||||
|
|
||||||
def test_inputs_are_earlier_phases_plus_indirect_dependencies_in_graph_order():
|
def test_inputs_are_earlier_phases_plus_indirect_dependencies_in_graph_order():
|
||||||
design = (
|
design = (
|
||||||
|
DeliverableKind.SCOPE,
|
||||||
DeliverableKind.REQUIREMENTS,
|
DeliverableKind.REQUIREMENTS,
|
||||||
DeliverableKind.TECHNICAL_DECOMPOSITION,
|
DeliverableKind.TECHNICAL_DECOMPOSITION,
|
||||||
DeliverableKind.PRODUCT_SELECTION,
|
DeliverableKind.PRODUCT_SELECTION,
|
||||||
DeliverableKind.DESIGN_RECOMMENDATIONS,
|
DeliverableKind.DESIGN_RECOMMENDATIONS,
|
||||||
DeliverableKind.TARGET_ARCHITECTURE,
|
DeliverableKind.TARGET_ARCHITECTURE,
|
||||||
)
|
)
|
||||||
assert inputs_of(DeliverableKind.REQUIREMENTS) == ()
|
assert inputs_of(DeliverableKind.SCOPE) == ()
|
||||||
|
assert inputs_of(DeliverableKind.REQUIREMENTS) == (DeliverableKind.SCOPE,)
|
||||||
assert inputs_of(DeliverableKind.DEPLOYMENT_GUIDANCE) == design
|
assert inputs_of(DeliverableKind.DEPLOYMENT_GUIDANCE) == design
|
||||||
assert inputs_of(DeliverableKind.ARCHITECTURE_DESCRIPTION) == design
|
assert inputs_of(DeliverableKind.ARCHITECTURE_DESCRIPTION) == design
|
||||||
assert inputs_of(DeliverableKind.TARGET_ARCHITECTURE) == design[:4]
|
assert inputs_of(DeliverableKind.TARGET_ARCHITECTURE) == design[:5]
|
||||||
|
# The plan is produced beside the guide, not before it: neither reads the other.
|
||||||
assert set(inputs_of(DeliverableKind.SOLUTION_GUIDE)) == set(DeliverableKind) - {
|
assert set(inputs_of(DeliverableKind.SOLUTION_GUIDE)) == set(DeliverableKind) - {
|
||||||
DeliverableKind.SOLUTION_GUIDE
|
DeliverableKind.SOLUTION_GUIDE,
|
||||||
|
DeliverableKind.WORK_BREAKDOWN,
|
||||||
}
|
}
|
||||||
|
assert DeliverableKind.SOLUTION_GUIDE not in inputs_of(
|
||||||
|
DeliverableKind.WORK_BREAKDOWN
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def sections_missing(kind: DeliverableKind, content: str) -> list[str]:
|
def sections_missing(kind: DeliverableKind, content: str) -> list[str]:
|
||||||
|
|||||||
@@ -10,11 +10,13 @@ import pytest
|
|||||||
from langgraph.checkpoint.memory import InMemorySaver
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||||
|
|
||||||
from app.adk.fake_runtime import Call, FakeAgentRuntime, sample_submission
|
from app.adk.fake_runtime import Call, sample_submission
|
||||||
from app.adk.runtime import RoleSpec, ToolResult, ToolSpec
|
from app.adk.runtime import RoleSpec, ToolResult, ToolSpec
|
||||||
from app.agent import ExecutionEngine
|
from app.agent import ExecutionEngine
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||||
|
from tests.scoped import ScopedEngine
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime
|
||||||
|
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
P = PhaseId
|
P = PhaseId
|
||||||
@@ -71,7 +73,7 @@ def make_engine(tmp_path: Path, runtime, **overrides) -> ExecutionEngine:
|
|||||||
"output_root": tmp_path,
|
"output_root": tmp_path,
|
||||||
}
|
}
|
||||||
options.update(overrides)
|
options.update(overrides)
|
||||||
return ExecutionEngine(**options)
|
return ScopedEngine(**options)
|
||||||
|
|
||||||
|
|
||||||
def happy_path_turns(**extra_specialist_turns) -> dict:
|
def happy_path_turns(**extra_specialist_turns) -> dict:
|
||||||
@@ -132,7 +134,7 @@ async def test_pre_approved_execution_completes_and_writes_deliverables(tmp_path
|
|||||||
assert recorder.statuses[result.execution_id][-1]["status"] == "completed"
|
assert recorder.statuses[result.execution_id][-1]["status"] == "completed"
|
||||||
assert {s["kind"] for s in recorder.submissions[result.execution_id]} == set(K)
|
assert {s["kind"] for s in recorder.submissions[result.execution_id]} == set(K)
|
||||||
recorded_types = [e.type for e in recorder.events[result.execution_id]]
|
recorded_types = [e.type for e in recorder.events[result.execution_id]]
|
||||||
assert recorded_types.count("approved") == 5
|
assert recorded_types.count("approved") == 6 # scoping, then the five after it
|
||||||
|
|
||||||
|
|
||||||
async def test_independent_specialists_run_together_between_orchestrator_turns(
|
async def test_independent_specialists_run_together_between_orchestrator_turns(
|
||||||
@@ -161,6 +163,7 @@ async def test_specialist_receives_its_dependencies_content(tmp_path):
|
|||||||
)
|
)
|
||||||
dependencies = describer_run.task.context["dependencies"]
|
dependencies = describer_run.task.context["dependencies"]
|
||||||
assert list(dependencies) == [
|
assert list(dependencies) == [
|
||||||
|
"scope",
|
||||||
"requirements",
|
"requirements",
|
||||||
"technical_decomposition",
|
"technical_decomposition",
|
||||||
"product_selection",
|
"product_selection",
|
||||||
@@ -718,7 +721,7 @@ async def test_pre_approved_questions_use_matching_scripted_answers(tmp_path):
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
engine = make_engine(tmp_path, runtime, max_orchestrator_turns=2)
|
engine = make_engine(tmp_path, runtime, max_orchestrator_turns=3)
|
||||||
|
|
||||||
result = await engine.start(
|
result = await engine.start(
|
||||||
"Ingestion",
|
"Ingestion",
|
||||||
|
|||||||
@@ -25,6 +25,14 @@ def approve(ledger: Ledger, phase: P) -> None:
|
|||||||
ledger.approve(phase, actor="user")
|
ledger.approve(phase, actor="user")
|
||||||
|
|
||||||
|
|
||||||
|
def scoped(**options) -> Ledger:
|
||||||
|
"""A Ledger whose scope is agreed: where requirements discovery begins."""
|
||||||
|
ledger = Ledger(**options)
|
||||||
|
produce(ledger, K.SCOPE)
|
||||||
|
approve(ledger, P.SCOPING)
|
||||||
|
return ledger
|
||||||
|
|
||||||
|
|
||||||
def through_phase_one(ledger: Ledger) -> None:
|
def through_phase_one(ledger: Ledger) -> None:
|
||||||
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
||||||
approve(ledger, P.REQUIREMENTS_DISCOVERY)
|
approve(ledger, P.REQUIREMENTS_DISCOVERY)
|
||||||
@@ -49,22 +57,29 @@ def through_phase_three(ledger: Ledger) -> None:
|
|||||||
# --- ready work and the two acceptance thresholds -------------------------------
|
# --- ready work and the two acceptance thresholds -------------------------------
|
||||||
|
|
||||||
|
|
||||||
def test_new_execution_has_only_requirements_ready():
|
def test_a_new_execution_has_only_the_scope_ready():
|
||||||
ledger = Ledger()
|
ledger = Ledger()
|
||||||
|
|
||||||
assert ledger.ready_work() == (K.REQUIREMENTS,)
|
assert ledger.ready_work() == (K.SCOPE,)
|
||||||
assert all(ledger.phase_status(p) is PhaseStatus.OPEN for p in P)
|
assert all(ledger.phase_status(p) is PhaseStatus.OPEN for p in P)
|
||||||
|
|
||||||
|
|
||||||
|
def test_once_the_scope_is_approved_requirements_are_ready():
|
||||||
|
ledger = scoped()
|
||||||
|
|
||||||
|
assert ledger.ready_work() == (K.REQUIREMENTS,)
|
||||||
|
assert ledger.phase_status(P.SCOPING) is PhaseStatus.APPROVED
|
||||||
|
|
||||||
|
|
||||||
def test_same_phase_dependency_is_met_once_submitted():
|
def test_same_phase_dependency_is_met_once_submitted():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
|
|
||||||
assert ledger.ready_work() == (K.TECHNICAL_DECOMPOSITION,)
|
assert ledger.ready_work() == (K.TECHNICAL_DECOMPOSITION,)
|
||||||
|
|
||||||
|
|
||||||
def test_cross_phase_dependency_needs_the_earlier_phase_approved():
|
def test_cross_phase_dependency_needs_the_earlier_phase_approved():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
||||||
|
|
||||||
assert K.PRODUCT_SELECTION not in ledger.ready_work()
|
assert K.PRODUCT_SELECTION not in ledger.ready_work()
|
||||||
@@ -75,7 +90,7 @@ def test_cross_phase_dependency_needs_the_earlier_phase_approved():
|
|||||||
|
|
||||||
|
|
||||||
def test_independent_work_becomes_ready_together():
|
def test_independent_work_becomes_ready_together():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
through_phase_one(ledger)
|
through_phase_one(ledger)
|
||||||
produce(
|
produce(
|
||||||
ledger, K.PRODUCT_SELECTION, K.DESIGN_RECOMMENDATIONS, K.TARGET_ARCHITECTURE
|
ledger, K.PRODUCT_SELECTION, K.DESIGN_RECOMMENDATIONS, K.TARGET_ARCHITECTURE
|
||||||
@@ -93,7 +108,7 @@ def test_independent_work_becomes_ready_together():
|
|||||||
|
|
||||||
|
|
||||||
def test_ready_phases_lists_open_phases_with_every_deliverable_submitted():
|
def test_ready_phases_lists_open_phases_with_every_deliverable_submitted():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
assert ledger.phases_ready_for_approval() == ()
|
assert ledger.phases_ready_for_approval() == ()
|
||||||
|
|
||||||
@@ -106,7 +121,7 @@ def test_ready_phases_lists_open_phases_with_every_deliverable_submitted():
|
|||||||
|
|
||||||
|
|
||||||
def test_start_refuses_work_that_is_not_ready_and_changes_nothing():
|
def test_start_refuses_work_that_is_not_ready_and_changes_nothing():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="technical_decomposition"):
|
with pytest.raises(LedgerRefusal, match="technical_decomposition"):
|
||||||
ledger.start([K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION], actor="orchestrator")
|
ledger.start([K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION], actor="orchestrator")
|
||||||
@@ -115,7 +130,7 @@ def test_start_refuses_work_that_is_not_ready_and_changes_nothing():
|
|||||||
|
|
||||||
|
|
||||||
def test_start_moves_ready_work_in_progress_and_returns_attempts():
|
def test_start_moves_ready_work_in_progress_and_returns_attempts():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
|
|
||||||
attempts = ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
attempts = ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
||||||
|
|
||||||
@@ -125,7 +140,7 @@ def test_start_moves_ready_work_in_progress_and_returns_attempts():
|
|||||||
|
|
||||||
|
|
||||||
def test_submit_requires_the_current_attempt():
|
def test_submit_requires_the_current_attempt():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
attempt = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
attempt = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="attempt"):
|
with pytest.raises(LedgerRefusal, match="attempt"):
|
||||||
@@ -136,7 +151,7 @@ def test_submit_requires_the_current_attempt():
|
|||||||
|
|
||||||
|
|
||||||
def test_late_submission_from_a_superseded_attempt_is_refused_after_restart():
|
def test_late_submission_from_a_superseded_attempt_is_refused_after_restart():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
first = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
first = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
||||||
ledger.release(K.REQUIREMENTS, first, reason="needs input")
|
ledger.release(K.REQUIREMENTS, first, reason="needs input")
|
||||||
second = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
second = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
||||||
@@ -149,14 +164,14 @@ def test_late_submission_from_a_superseded_attempt_is_refused_after_restart():
|
|||||||
|
|
||||||
|
|
||||||
def test_submit_refuses_work_not_in_progress():
|
def test_submit_refuses_work_not_in_progress():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="in_progress"):
|
with pytest.raises(LedgerRefusal, match="in_progress"):
|
||||||
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
||||||
|
|
||||||
|
|
||||||
def test_release_returns_work_to_pending_without_a_revision():
|
def test_release_returns_work_to_pending_without_a_revision():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
attempt = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
attempt = ledger.start([K.REQUIREMENTS], actor="orchestrator")[K.REQUIREMENTS]
|
||||||
|
|
||||||
ledger.release(K.REQUIREMENTS, attempt, reason="needs input")
|
ledger.release(K.REQUIREMENTS, attempt, reason="needs input")
|
||||||
@@ -172,7 +187,7 @@ def test_release_returns_work_to_pending_without_a_revision():
|
|||||||
|
|
||||||
|
|
||||||
def test_request_approval_refuses_until_every_deliverable_is_submitted():
|
def test_request_approval_refuses_until_every_deliverable_is_submitted():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="technical_decomposition"):
|
with pytest.raises(LedgerRefusal, match="technical_decomposition"):
|
||||||
@@ -182,7 +197,7 @@ def test_request_approval_refuses_until_every_deliverable_is_submitted():
|
|||||||
|
|
||||||
|
|
||||||
def test_approval_moves_through_awaiting_to_approved_by_the_user():
|
def test_approval_moves_through_awaiting_to_approved_by_the_user():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
||||||
|
|
||||||
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
||||||
@@ -198,14 +213,14 @@ def test_approval_moves_through_awaiting_to_approved_by_the_user():
|
|||||||
|
|
||||||
|
|
||||||
def test_approve_refuses_a_phase_not_awaiting_approval():
|
def test_approve_refuses_a_phase_not_awaiting_approval():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="awaiting_approval"):
|
with pytest.raises(LedgerRefusal, match="awaiting_approval"):
|
||||||
ledger.approve(P.REQUIREMENTS_DISCOVERY, actor="user")
|
ledger.approve(P.REQUIREMENTS_DISCOVERY, actor="user")
|
||||||
|
|
||||||
|
|
||||||
def test_pre_approved_ledger_grants_approval_immediately():
|
def test_pre_approved_ledger_grants_approval_immediately():
|
||||||
ledger = Ledger(pre_approved=True)
|
ledger = scoped(pre_approved=True)
|
||||||
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
||||||
|
|
||||||
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
||||||
@@ -216,7 +231,7 @@ def test_pre_approved_ledger_grants_approval_immediately():
|
|||||||
|
|
||||||
|
|
||||||
def test_request_changes_reopens_the_phase_for_further_work():
|
def test_request_changes_reopens_the_phase_for_further_work():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
produce(ledger, K.REQUIREMENTS, K.TECHNICAL_DECOMPOSITION)
|
||||||
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
ledger.request_approval(P.REQUIREMENTS_DISCOVERY, actor="orchestrator")
|
||||||
|
|
||||||
@@ -228,11 +243,11 @@ def test_request_changes_reopens_the_phase_for_further_work():
|
|||||||
|
|
||||||
|
|
||||||
def test_finish_is_allowed_only_once_packaging_is_approved():
|
def test_finish_is_allowed_only_once_packaging_is_approved():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
through_phase_three(ledger)
|
through_phase_three(ledger)
|
||||||
assert not ledger.can_finish()
|
assert not ledger.can_finish()
|
||||||
|
|
||||||
produce(ledger, K.SOLUTION_GUIDE)
|
produce(ledger, K.WORK_BREAKDOWN, K.SOLUTION_GUIDE)
|
||||||
approve(ledger, P.PACKAGING)
|
approve(ledger, P.PACKAGING)
|
||||||
|
|
||||||
assert ledger.can_finish()
|
assert ledger.can_finish()
|
||||||
@@ -242,7 +257,7 @@ def test_finish_is_allowed_only_once_packaging_is_approved():
|
|||||||
|
|
||||||
|
|
||||||
def test_reopen_cascades_to_dependents_and_withdraws_affected_approvals():
|
def test_reopen_cascades_to_dependents_and_withdraws_affected_approvals():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
through_phase_three(ledger)
|
through_phase_three(ledger)
|
||||||
|
|
||||||
result = ledger.reopen(
|
result = ledger.reopen(
|
||||||
@@ -274,7 +289,7 @@ def test_reopen_cascades_to_dependents_and_withdraws_affected_approvals():
|
|||||||
|
|
||||||
|
|
||||||
def test_reopen_invalidates_attempts_of_dependents_already_in_progress():
|
def test_reopen_invalidates_attempts_of_dependents_already_in_progress():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
through_phase_one(ledger)
|
through_phase_one(ledger)
|
||||||
produce(ledger, K.PRODUCT_SELECTION)
|
produce(ledger, K.PRODUCT_SELECTION)
|
||||||
attempt = ledger.start([K.DESIGN_RECOMMENDATIONS], actor="orchestrator")[
|
attempt = ledger.start([K.DESIGN_RECOMMENDATIONS], actor="orchestrator")[
|
||||||
@@ -289,14 +304,14 @@ def test_reopen_invalidates_attempts_of_dependents_already_in_progress():
|
|||||||
|
|
||||||
|
|
||||||
def test_reopen_refuses_deliverables_that_were_never_submitted():
|
def test_reopen_refuses_deliverables_that_were_never_submitted():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
|
|
||||||
with pytest.raises(LedgerRefusal, match="submitted"):
|
with pytest.raises(LedgerRefusal, match="submitted"):
|
||||||
ledger.reopen(K.REQUIREMENTS, "why", actor="orchestrator")
|
ledger.reopen(K.REQUIREMENTS, "why", actor="orchestrator")
|
||||||
|
|
||||||
|
|
||||||
def test_reopen_escalates_at_the_revision_cap_without_changing_state():
|
def test_reopen_escalates_at_the_revision_cap_without_changing_state():
|
||||||
ledger = Ledger(max_revisions=2)
|
ledger = scoped(max_revisions=2)
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
for _ in range(2):
|
for _ in range(2):
|
||||||
ledger.reopen(K.REQUIREMENTS, "again", actor="orchestrator")
|
ledger.reopen(K.REQUIREMENTS, "again", actor="orchestrator")
|
||||||
@@ -313,7 +328,7 @@ def test_reopen_escalates_at_the_revision_cap_without_changing_state():
|
|||||||
|
|
||||||
|
|
||||||
def test_user_can_override_the_revision_cap():
|
def test_user_can_override_the_revision_cap():
|
||||||
ledger = Ledger(max_revisions=1)
|
ledger = scoped(max_revisions=1)
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
ledger.reopen(K.REQUIREMENTS, "first", actor="orchestrator")
|
ledger.reopen(K.REQUIREMENTS, "first", actor="orchestrator")
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.REQUIREMENTS)
|
||||||
@@ -327,7 +342,7 @@ def test_user_can_override_the_revision_cap():
|
|||||||
|
|
||||||
|
|
||||||
def test_default_revision_cap_is_three():
|
def test_default_revision_cap_is_three():
|
||||||
assert Ledger().max_revisions == 3
|
assert scoped().max_revisions == 3
|
||||||
|
|
||||||
|
|
||||||
# --- records and persistence -------------------------------------------------------
|
# --- records and persistence -------------------------------------------------------
|
||||||
@@ -335,16 +350,16 @@ def test_default_revision_cap_is_three():
|
|||||||
|
|
||||||
def test_transitions_are_recorded_as_ordered_events():
|
def test_transitions_are_recorded_as_ordered_events():
|
||||||
ledger = Ledger()
|
ledger = Ledger()
|
||||||
produce(ledger, K.REQUIREMENTS)
|
produce(ledger, K.SCOPE)
|
||||||
|
|
||||||
assert [(e.sequence, e.type, e.subject) for e in ledger.events] == [
|
assert [(e.sequence, e.type, e.subject) for e in ledger.events] == [
|
||||||
(1, "started", "requirements"),
|
(1, "started", "scope"),
|
||||||
(2, "submitted", "requirements"),
|
(2, "submitted", "scope"),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def test_round_trips_through_plain_data():
|
def test_round_trips_through_plain_data():
|
||||||
ledger = Ledger(max_revisions=5)
|
ledger = scoped(max_revisions=5)
|
||||||
through_phase_one(ledger)
|
through_phase_one(ledger)
|
||||||
produce(ledger, K.PRODUCT_SELECTION)
|
produce(ledger, K.PRODUCT_SELECTION)
|
||||||
ledger.reopen(K.PRODUCT_SELECTION, "cost", actor="user")
|
ledger.reopen(K.PRODUCT_SELECTION, "cost", actor="user")
|
||||||
@@ -358,7 +373,7 @@ def test_round_trips_through_plain_data():
|
|||||||
|
|
||||||
|
|
||||||
def test_revising_the_target_architecture_returns_the_code_to_draft():
|
def test_revising_the_target_architecture_returns_the_code_to_draft():
|
||||||
ledger = Ledger()
|
ledger = scoped()
|
||||||
through_phase_three(ledger)
|
through_phase_three(ledger)
|
||||||
|
|
||||||
result = ledger.reopen(
|
result = ledger.reopen(
|
||||||
|
|||||||
@@ -10,6 +10,8 @@ from app.agent import ExecutionEngine
|
|||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.states.lessons import refusal
|
from app.states.lessons import refusal
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine # noqa: F811
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
P = PhaseId
|
P = PhaseId
|
||||||
@@ -195,9 +197,9 @@ async def test_an_execution_that_ends_with_warnings_still_records_lessons(tmp_pa
|
|||||||
"reason": "stopped",
|
"reason": "stopped",
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
assert not (
|
# Only the scope had been produced, so only the scope was written.
|
||||||
tmp_path / "deliverables" / "cases" / "unassigned" / "executions"
|
written = {path.name for path in tmp_path.rglob("deliverables/*.md")}
|
||||||
).exists()
|
assert written == {"scope.md"}
|
||||||
|
|
||||||
|
|
||||||
async def test_a_cloud_read_refusal_keeps_only_its_leading_words(tmp_path):
|
async def test_a_cloud_read_refusal_keeps_only_its_leading_words(tmp_path):
|
||||||
|
|||||||
@@ -48,14 +48,14 @@ async def test_an_execution_waiting_for_its_user_has_written_what_exists(tmp_pat
|
|||||||
|
|
||||||
waiting = await engine.start("Ingest", customer_case="acme", execution_id="v1")
|
waiting = await engine.start("Ingest", customer_case="acme", execution_id="v1")
|
||||||
|
|
||||||
# It stopped for the first Approval: only the first Phase has been produced.
|
# It stopped for the first Approval: only the scope has been produced.
|
||||||
assert waiting.status == "awaiting_user"
|
assert waiting.status == "awaiting_user"
|
||||||
out = folder(tmp_path, "v1")
|
out = folder(tmp_path, "v1")
|
||||||
manifest = json.loads((out / "execution.json").read_text())
|
manifest = json.loads((out / "execution.json").read_text())
|
||||||
assert manifest["status"] == "in_progress"
|
assert manifest["status"] == "in_progress"
|
||||||
assert set(manifest["deliverables"]) == {"requirements", "technical_decomposition"}
|
assert set(manifest["deliverables"]) == {"scope"}
|
||||||
assert (out / "deliverables" / "requirements.md").read_text()
|
assert (out / "deliverables" / "scope.md").read_text()
|
||||||
assert (out / "requirements.json").is_file()
|
assert (out / "scope.json").is_file() and (out / "scope.yaml").is_file()
|
||||||
assert manifest["solution_guide"] is None
|
assert manifest["solution_guide"] is None
|
||||||
# Unfinished work is never something a Change request can start from.
|
# Unfinished work is never something a Change request can start from.
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ from app.adk.roles import PILLAR_ADVISOR
|
|||||||
from app.adk.runtime import RoleSpec, ToolResult, ToolSpec
|
from app.adk.runtime import RoleSpec, ToolResult, ToolSpec
|
||||||
from app.tools.pillars import CONSULT_TOOL, pillar_consultation_tool
|
from app.tools.pillars import CONSULT_TOOL, pillar_consultation_tool
|
||||||
from app.tools.toolkit import make_tool_factory
|
from app.tools.toolkit import make_tool_factory
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
|
||||||
PILLARS = {
|
PILLARS = {
|
||||||
"security": "waf-security",
|
"security": "waf-security",
|
||||||
|
|||||||
@@ -28,6 +28,8 @@ from app.workflows.target_architecture import (
|
|||||||
parse_graph,
|
parse_graph,
|
||||||
render_mermaid,
|
render_mermaid,
|
||||||
)
|
)
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine # noqa: F811
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
P = PhaseId
|
P = PhaseId
|
||||||
@@ -765,3 +767,113 @@ def test_a_constraint_with_conditions_still_needs_a_check():
|
|||||||
)
|
)
|
||||||
|
|
||||||
assert gaps == ("Requirement NFR-2 has no Check",)
|
assert gaps == ("Requirement NFR-2 has no Check",)
|
||||||
|
|
||||||
|
|
||||||
|
# --- what the customer stated reaches the requirements (ADR-0008) ------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def scoped_upstream(**scope) -> dict:
|
||||||
|
import json
|
||||||
|
|
||||||
|
return {"scope": {"attachments": {"scope.json": json.dumps(scope)}}}
|
||||||
|
|
||||||
|
|
||||||
|
def requirement_problems(requirements: list[dict], **scope) -> list[str]:
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.workflows.chain import check_requirements
|
||||||
|
|
||||||
|
return check_requirements(
|
||||||
|
{"requirements.json": json.dumps(requirements)}, scoped_upstream(**scope)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
STATED = {
|
||||||
|
"stated": {
|
||||||
|
"intent": {
|
||||||
|
"requirements": [{"id": "REQ-01", "statement": "Ready in 10 weeks."}]
|
||||||
|
},
|
||||||
|
"networking": {"topology": "hub_and_spoke"},
|
||||||
|
},
|
||||||
|
"binding": ["networking.topology"],
|
||||||
|
}
|
||||||
|
FUNCTIONAL = {"id": "FR-1", "kind": "functional", "text": "Accept events."}
|
||||||
|
CARRIED = {
|
||||||
|
"id": "NFR-1",
|
||||||
|
"kind": "non_functional",
|
||||||
|
"text": "The network is hub and spoke, ready in ten weeks.",
|
||||||
|
"conditions": ["A hub network is connected to each spoke network."],
|
||||||
|
"from": ["REQ-01", "networking.topology"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_requirements_that_carry_everything_the_customer_stated_are_accepted():
|
||||||
|
assert requirement_problems([FUNCTIONAL, CARRIED], **STATED) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_customers_own_requirement_must_be_covered_by_name():
|
||||||
|
uncited = {**CARRIED, "from": ["networking.topology"]}
|
||||||
|
|
||||||
|
(problem,) = requirement_problems([FUNCTIONAL, uncited], **STATED)
|
||||||
|
|
||||||
|
assert "the customer's requirement REQ-01 is covered by no requirement" in problem
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_stated_choice_must_be_carried_by_a_requirement():
|
||||||
|
"""Case 08: a stated topology was named in the design and not built, and every
|
||||||
|
Check passed, because nothing asked for it."""
|
||||||
|
uncarried = {**CARRIED, "from": ["REQ-01"]}
|
||||||
|
|
||||||
|
(problem,) = requirement_problems([FUNCTIONAL, uncarried], **STATED)
|
||||||
|
|
||||||
|
assert "the customer stated networking.topology and no requirement carries it" in (
|
||||||
|
problem
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_stated_choice_is_carried_as_a_constraint_the_design_is_checked_against():
|
||||||
|
as_functional = {**CARRIED, "kind": "functional", "conditions": []}
|
||||||
|
outside_code_only = {
|
||||||
|
**CARRIED,
|
||||||
|
"conditions": [],
|
||||||
|
"outside_code": ["The topology is reviewed by the network team."],
|
||||||
|
}
|
||||||
|
|
||||||
|
for weak in (as_functional, outside_code_only):
|
||||||
|
problems = requirement_problems([FUNCTIONAL, weak], **STATED)
|
||||||
|
assert any(
|
||||||
|
"must be non_functional with at least one entry in `conditions`" in p
|
||||||
|
for p in problems
|
||||||
|
), weak
|
||||||
|
|
||||||
|
|
||||||
|
def test_with_no_scope_or_nothing_binding_requirements_are_as_before():
|
||||||
|
from app.workflows.chain import check_requirements
|
||||||
|
|
||||||
|
plain = [FUNCTIONAL]
|
||||||
|
import json
|
||||||
|
|
||||||
|
assert check_requirements({"requirements.json": json.dumps(plain)}, None) == []
|
||||||
|
assert requirement_problems(plain, stated={"intent": {"statement": "x"}}) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_what_was_assumed_for_the_customer_does_not_bind_the_design():
|
||||||
|
from app.scope import Catalogue, FieldSpec, Scope
|
||||||
|
|
||||||
|
catalogue = Catalogue(
|
||||||
|
1,
|
||||||
|
("workload",),
|
||||||
|
{
|
||||||
|
"net.topology": FieldSpec("net.topology", binds=True),
|
||||||
|
"net.region": FieldSpec("net.region", binds=True),
|
||||||
|
"notes": FieldSpec("notes"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
scope = Scope(
|
||||||
|
catalogue,
|
||||||
|
"workload",
|
||||||
|
values={"net.topology": "hub", "net.region": "x", "notes": "n"},
|
||||||
|
sources={"net.region": "assumed"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert scope.for_context()["binding"] == ["net.topology"]
|
||||||
|
|||||||
@@ -25,6 +25,9 @@ from app.nodes.router import (
|
|||||||
)
|
)
|
||||||
from app.recorder import InMemoryRecorder
|
from app.recorder import InMemoryRecorder
|
||||||
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind
|
||||||
|
from tests.scoped import ScopedEngine as ExecutionEngine # noqa: F811
|
||||||
|
from tests.scoped import ScopedRuntime as FakeAgentRuntime # noqa: F811
|
||||||
|
from tests.scoped import scoped_ledger
|
||||||
|
|
||||||
K = DeliverableKind
|
K = DeliverableKind
|
||||||
|
|
||||||
@@ -122,6 +125,8 @@ async def test_an_execution_completes_with_no_model_choosing_an_action(tmp_path)
|
|||||||
assert result.status == "completed"
|
assert result.status == "completed"
|
||||||
assert "orchestrator" not in {run.role for run in runtime.runs}
|
assert "orchestrator" not in {run.role for run in runtime.runs}
|
||||||
assert actions(result) == [
|
assert actions(result) == [
|
||||||
|
"delegate", # the scope
|
||||||
|
"request_approval",
|
||||||
"delegate", # requirements
|
"delegate", # requirements
|
||||||
"delegate", # technical decomposition
|
"delegate", # technical decomposition
|
||||||
"request_approval",
|
"request_approval",
|
||||||
@@ -198,14 +203,18 @@ async def test_the_approval_summary_is_assembled_from_the_data(tmp_path):
|
|||||||
async def test_a_model_writes_the_summary_but_decides_nothing(tmp_path):
|
async def test_a_model_writes_the_summary_but_decides_nothing(tmp_path):
|
||||||
turns = {
|
turns = {
|
||||||
**once(),
|
**once(),
|
||||||
REPLY_WRITER: [[Call("write_reply", {"text": "Two requirements, ready."})]],
|
REPLY_WRITER: [
|
||||||
|
[Call("write_reply", {"text": "The scope, ready."})],
|
||||||
|
[Call("write_reply", {"text": "Two requirements, ready."})],
|
||||||
|
],
|
||||||
}
|
}
|
||||||
engine, runtime = engine_for(tmp_path, turns, writer=True)
|
engine, runtime = engine_for(tmp_path, turns, writer=True)
|
||||||
|
|
||||||
first = await engine.start("Ingest webhooks")
|
first = await engine.start("Ingest webhooks")
|
||||||
|
|
||||||
assert first.prompt["approval"]["summary"] == "Two requirements, ready."
|
assert first.prompt["approval"]["summary"] == "Two requirements, ready."
|
||||||
writer_run = next(run for run in runtime.runs if run.role == REPLY_WRITER)
|
# The writer ran once for the scope and once for the requirements.
|
||||||
|
writer_run = [run for run in runtime.runs if run.role == REPLY_WRITER][-1]
|
||||||
assert writer_run.tools == ("write_reply",)
|
assert writer_run.tools == ("write_reply",)
|
||||||
assert writer_run.task.context["facts"]["deliverables"] == [
|
assert writer_run.task.context["facts"]["deliverables"] == [
|
||||||
"requirements",
|
"requirements",
|
||||||
@@ -478,6 +487,7 @@ async def test_a_requested_change_is_routed_to_the_deliverable_it_affects(tmp_pa
|
|||||||
[(state, questions, _)] = decider.asked
|
[(state, questions, _)] = decider.asked
|
||||||
assert "The user's message: Use customer-managed keys on the topic." in state
|
assert "The user's message: Use customer-managed keys on the topic." in state
|
||||||
assert questions["change_target"].options == (
|
assert questions["change_target"].options == (
|
||||||
|
"scope",
|
||||||
"requirements",
|
"requirements",
|
||||||
"technical_decomposition",
|
"technical_decomposition",
|
||||||
"product_selection",
|
"product_selection",
|
||||||
@@ -689,9 +699,8 @@ def test_a_quoted_question_is_removed_from_what_the_user_said():
|
|||||||
|
|
||||||
async def test_the_questions_asked_are_given_to_the_decider_as_background(tmp_path):
|
async def test_the_questions_asked_are_given_to_the_decider_as_background(tmp_path):
|
||||||
from app.nodes.router import TurnView
|
from app.nodes.router import TurnView
|
||||||
from app.states.ledger import Ledger
|
|
||||||
|
|
||||||
ledger = Ledger()
|
ledger = scoped_ledger()
|
||||||
ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
||||||
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
||||||
decider = FakeDecider(
|
decider = FakeDecider(
|
||||||
@@ -765,9 +774,8 @@ async def test_a_reply_that_restates_what_is_agreed_is_an_answer_not_a_change(
|
|||||||
):
|
):
|
||||||
"""Evaluation case 04: a requirement said again in reply to a later question."""
|
"""Evaluation case 04: a requirement said again in reply to a later question."""
|
||||||
from app.nodes.router import Readings, TurnView
|
from app.nodes.router import Readings, TurnView
|
||||||
from app.states.ledger import Ledger
|
|
||||||
|
|
||||||
ledger = Ledger()
|
ledger = scoped_ledger()
|
||||||
ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
ledger.start([K.REQUIREMENTS], actor="orchestrator")
|
||||||
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
ledger.submit(K.REQUIREMENTS, 1, actor="requirements-analyst")
|
||||||
view = TurnView(
|
view = TurnView(
|
||||||
|
|||||||
431
tests/execution/test_scope_spec.py
Normal file
431
tests/execution/test_scope_spec.py
Normal file
@@ -0,0 +1,431 @@
|
|||||||
|
"""An Execution started from a scope spec (ADR-0008)."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
|
|
||||||
|
from app.adk.fake_runtime import Call, FakeAgentRuntime, sample_submission
|
||||||
|
from app.adk.runtime import NoDecider, RoleSpec
|
||||||
|
from app.agent import ExecutionEngine
|
||||||
|
from app.nodes.router import Router, Thresholds
|
||||||
|
from app.providers import load_provider
|
||||||
|
from app.recorder import InMemoryRecorder
|
||||||
|
from app.workflows.deliverables import DEPENDENCY_GRAPH
|
||||||
|
from app.workflows.deliverables import DeliverableKind as K
|
||||||
|
from tests.scoped import carrying
|
||||||
|
|
||||||
|
ROLES = {
|
||||||
|
"orchestrator": RoleSpec("orchestrator", "Unused.", budget=40),
|
||||||
|
**{
|
||||||
|
spec.specialist: RoleSpec(
|
||||||
|
spec.specialist, "Produce.", budget=15, skills=("base",)
|
||||||
|
)
|
||||||
|
for spec in DEPENDENCY_GRAPH.values()
|
||||||
|
},
|
||||||
|
}
|
||||||
|
ANALYST = DEPENDENCY_GRAPH[K.REQUIREMENTS].specialist
|
||||||
|
SCOPE_ANALYST = DEPENDENCY_GRAPH[K.SCOPE].specialist
|
||||||
|
SPEC = """
|
||||||
|
metadata: {engagement_id: ENG-77}
|
||||||
|
engagement: {kind: platform_foundation}
|
||||||
|
intent:
|
||||||
|
statement: Build us a landing zone.
|
||||||
|
exclusions: [application workloads]
|
||||||
|
estate:
|
||||||
|
cloud_provider: gcp
|
||||||
|
mode: greenfield
|
||||||
|
regions: {primary: europe-west2}
|
||||||
|
data_residency: United Kingdom
|
||||||
|
compliance:
|
||||||
|
frameworks: [DORA]
|
||||||
|
data_classification: confidential
|
||||||
|
networking: {topology: shared_vpc_hub_and_spoke, cidr_allocations: decide}
|
||||||
|
run: {pre_approved: true, terraform_modules: false}
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
STATED_CHOICES = [
|
||||||
|
"estate.regions.primary",
|
||||||
|
"estate.data_residency",
|
||||||
|
"compliance.frameworks",
|
||||||
|
"compliance.data_classification",
|
||||||
|
"networking.topology",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Recording(FakeAgentRuntime):
|
||||||
|
def __init__(self, turns):
|
||||||
|
super().__init__(turns)
|
||||||
|
self.seen: dict[str, dict] = {}
|
||||||
|
|
||||||
|
async def run(self, role, task, tools):
|
||||||
|
self.seen[role.name] = {
|
||||||
|
"skills": role.skills,
|
||||||
|
"scope": task.context.get("scope"),
|
||||||
|
}
|
||||||
|
return await super().run(role, task, tools)
|
||||||
|
|
||||||
|
|
||||||
|
def engine(tmp_path) -> tuple[ExecutionEngine, Recording]:
|
||||||
|
bundle = load_provider("gcp")
|
||||||
|
runtime = Recording(
|
||||||
|
{
|
||||||
|
**{
|
||||||
|
spec.specialist: [[sample_submission(kind)]]
|
||||||
|
for kind, spec in DEPENDENCY_GRAPH.items()
|
||||||
|
},
|
||||||
|
# The requirements carry every choice SPEC states, and a Check holds
|
||||||
|
# the design to them.
|
||||||
|
**carrying(STATED_CHOICES),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
built = ExecutionEngine(
|
||||||
|
runtime=runtime,
|
||||||
|
roles=ROLES,
|
||||||
|
recorder=InMemoryRecorder(),
|
||||||
|
checkpointer=InMemorySaver(),
|
||||||
|
output_root=tmp_path,
|
||||||
|
orchestrator=Router(NoDecider(), None, Thresholds()),
|
||||||
|
scope_catalogue=bundle.scope_catalogue(),
|
||||||
|
scope_values=bundle.scope_values,
|
||||||
|
scope_skill=bundle.scope_skill,
|
||||||
|
)
|
||||||
|
return built, runtime
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_spec_alone_starts_an_execution_and_names_its_customer_case(tmp_path):
|
||||||
|
built, _ = engine(tmp_path)
|
||||||
|
|
||||||
|
result = await built.start("", scope=SPEC, execution_id="v1")
|
||||||
|
|
||||||
|
assert result.status == "completed"
|
||||||
|
state = result.state
|
||||||
|
# The spec said pre-approved, named the case, and gave the request.
|
||||||
|
assert state["pre_approved"] is True and state["customer_case"] == "eng-77"
|
||||||
|
assert state["workflow_request"] == "Build us a landing zone."
|
||||||
|
assert state["scope"]["status"] == "APPROVED"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_the_supplied_file_is_kept_unaltered_beside_a_completed_copy(tmp_path):
|
||||||
|
built, _ = engine(tmp_path)
|
||||||
|
|
||||||
|
await built.start("", scope=SPEC, execution_id="v1")
|
||||||
|
|
||||||
|
intent = tmp_path / "deliverables" / "cases" / "eng-77" / "intent"
|
||||||
|
assert (intent / "001.scope.yaml").read_text() == SPEC
|
||||||
|
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
# Its Execution completed, so this copy is the record a change starts from.
|
||||||
|
assert completed["metadata"]["status"] == "BASELINE_LOCKED"
|
||||||
|
assert completed["networking"]["cidr_allocations"] == "decide"
|
||||||
|
assert completed["provenance"]["estate.regions.primary"] == "supplied"
|
||||||
|
assert [a["field"] for a in completed["inferred_assumptions"]] == [
|
||||||
|
"networking.cidr_allocations"
|
||||||
|
]
|
||||||
|
history = json.loads((intent / "history.json").read_text())
|
||||||
|
assert history["intents"][0]["scope"] == {
|
||||||
|
"completed": "intent/001.scope.completed.yaml",
|
||||||
|
"supplied": "intent/001.scope.yaml",
|
||||||
|
}
|
||||||
|
manifest = json.loads(next(tmp_path.rglob("execution.json")).read_text())
|
||||||
|
assert manifest["scope"]["status"] == "BASELINE_LOCKED"
|
||||||
|
assert manifest["scope"]["readiness"]["assumed"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
async def test_every_specialist_is_shown_the_spec_and_given_the_skill(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
|
||||||
|
await built.start("", scope=SPEC, execution_id="v1")
|
||||||
|
|
||||||
|
for name, seen in runtime.seen.items():
|
||||||
|
assert seen["skills"] == ("base", "scoping"), name
|
||||||
|
assert seen["scope"]["kind"] == "platform_foundation"
|
||||||
|
stated = runtime.seen[ANALYST]["scope"]["stated"]
|
||||||
|
assert stated["networking"]["topology"] == "shared_vpc_hub_and_spoke"
|
||||||
|
assert stated["intent"]["exclusions"] == ["application workloads"]
|
||||||
|
assert runtime.seen[ANALYST]["scope"]["to_decide"][0]["field"] == (
|
||||||
|
"networking.cidr_allocations"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def settle(**values) -> Call:
|
||||||
|
"""What a scope analyst records when it must assume what nobody stated."""
|
||||||
|
return Call(
|
||||||
|
"scope_set",
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"field": field,
|
||||||
|
"value": value,
|
||||||
|
"source": "assumed",
|
||||||
|
"note": "A default.",
|
||||||
|
}
|
||||||
|
for field, value in values.items()
|
||||||
|
]
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
DRAFTED = [
|
||||||
|
Call(
|
||||||
|
"scope_set",
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"field": "intent.statement",
|
||||||
|
"value": "Ingest webhooks",
|
||||||
|
"source": "request",
|
||||||
|
},
|
||||||
|
{"field": "engagement.kind", "value": "workload", "source": "request"},
|
||||||
|
]
|
||||||
|
},
|
||||||
|
),
|
||||||
|
settle(
|
||||||
|
**{
|
||||||
|
"estate.cloud_provider": "gcp",
|
||||||
|
"estate.mode": "greenfield",
|
||||||
|
"estate.regions.primary": "europe-west2",
|
||||||
|
"estate.data_residency": "United Kingdom",
|
||||||
|
"workload.users.who": "systems",
|
||||||
|
"workload.users.count": 10,
|
||||||
|
"workload.availability.target": "99.9%",
|
||||||
|
"compliance.frameworks": [],
|
||||||
|
"compliance.data_classification": "internal",
|
||||||
|
"cost.monthly_budget": 500,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_without_a_spec_the_scope_analyst_drafts_one_from_the_request(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
runtime._turns[SCOPE_ANALYST].clear()
|
||||||
|
runtime._turns[SCOPE_ANALYST].append([*DRAFTED, sample_submission(K.SCOPE)])
|
||||||
|
|
||||||
|
result = await built.start(
|
||||||
|
"Ingest webhooks", customer_case="acme", pre_approved=True, execution_id="v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.status == "completed"
|
||||||
|
intent = tmp_path / "deliverables" / "cases" / "acme" / "intent"
|
||||||
|
# Nothing was supplied, so there is no supplied file; the completed copy is
|
||||||
|
# what the analyst drafted.
|
||||||
|
assert not (intent / "001.scope.yaml").exists()
|
||||||
|
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
assert completed["intent"]["statement"] == "Ingest webhooks"
|
||||||
|
assert completed["metadata"]["customer_case"] == "acme"
|
||||||
|
assert completed["provenance"]["intent.statement"] == "request"
|
||||||
|
assert completed["provenance"]["estate.regions.primary"] == "assumed"
|
||||||
|
assert completed["readiness"]["assumed"] == 10
|
||||||
|
assumed = {a["field"] for a in completed["inferred_assumptions"]}
|
||||||
|
assert "cost.monthly_budget" in assumed
|
||||||
|
# What was assumed is shown apart from what the customer stated.
|
||||||
|
scope = runtime.seen[ANALYST]["scope"]
|
||||||
|
assert scope["stated"]["intent"]["statement"] == "Ingest webhooks"
|
||||||
|
assert "estate" not in scope["stated"]
|
||||||
|
assert {"field": "cost.monthly_budget", "value": 500, "why": "A default."} in (
|
||||||
|
scope["assumed"]
|
||||||
|
)
|
||||||
|
document = tmp_path / "deliverables" / "cases" / "acme" / "executions" / "v1"
|
||||||
|
register = (document / "deliverables" / "scope.md").read_text()
|
||||||
|
assert "## Information register" in register and "### Assumptions" in register
|
||||||
|
assert "| intent.statement | Ingest webhooks | the request |" in register
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_scope_with_an_essential_field_unsettled_cannot_be_submitted(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
runtime._turns[SCOPE_ANALYST].clear()
|
||||||
|
# It records the statement and nothing else, then tries to submit, twice.
|
||||||
|
runtime._turns[SCOPE_ANALYST].append(
|
||||||
|
[DRAFTED[0], sample_submission(K.SCOPE), sample_submission(K.SCOPE)]
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await built.start("Ingest webhooks", pre_approved=True)
|
||||||
|
|
||||||
|
run = next(r for r in runtime.runs if r.role == SCOPE_ANALYST)
|
||||||
|
refused = run.results[1][1]
|
||||||
|
assert refused.startswith("Not submitted. Essential fields of the scope spec")
|
||||||
|
assert "estate.regions.primary" in refused
|
||||||
|
assert result.status == "completed_with_warnings"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_what_the_customer_stated_is_not_overwritten_by_the_analyst(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
runtime._turns[SCOPE_ANALYST].clear()
|
||||||
|
runtime._turns[SCOPE_ANALYST].append(
|
||||||
|
[
|
||||||
|
Call("scope_read"),
|
||||||
|
settle(**{"estate.regions.primary": "us-central1", "nonsense.field": 1}),
|
||||||
|
Call(
|
||||||
|
"scope_set",
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"field": "estate.regions.primary",
|
||||||
|
"value": "europe-west1",
|
||||||
|
"source": "answered",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
),
|
||||||
|
Call(
|
||||||
|
"scope_set",
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"field": "cost.monthly_budget",
|
||||||
|
"value": "lots",
|
||||||
|
"source": "answered",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
),
|
||||||
|
sample_submission(K.SCOPE),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
await built.start("", scope=SPEC, execution_id="v1")
|
||||||
|
|
||||||
|
run = next(r for r in runtime.runs if r.role == SCOPE_ANALYST)
|
||||||
|
read, assumed, answered, mistyped = (reply for _, reply in run.results[:4])
|
||||||
|
assert "Kind of engagement: platform_foundation" in read
|
||||||
|
assert "Left to you to decide: networking.cidr_allocations" in read
|
||||||
|
assert "estate.regions.primary was stated in the scope spec" in assumed
|
||||||
|
assert "nonsense.field is not a field" in assumed
|
||||||
|
# The customer's own answer does change what the file said.
|
||||||
|
assert answered == "Recorded 1: estate.regions.primary."
|
||||||
|
assert "cost.monthly_budget: must be a number" in mistyped
|
||||||
|
intent = tmp_path / "deliverables" / "cases" / "eng-77" / "intent"
|
||||||
|
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
assert completed["estate"]["regions"]["primary"] == "europe-west1"
|
||||||
|
assert completed["provenance"]["estate.regions.primary"] == "answered"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_the_scope_is_approved_before_requirements_begin(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
|
||||||
|
waiting = await built.start("", scope=SPEC, pre_approved=False, execution_id="v1")
|
||||||
|
|
||||||
|
# Only the scope analyst has run, and the customer is asked to approve the scope.
|
||||||
|
assert {run.role for run in runtime.runs} == {SCOPE_ANALYST}
|
||||||
|
assert waiting.prompt["approval"]["phase"] == "scoping"
|
||||||
|
intent = tmp_path / "deliverables" / "cases" / "eng-77" / "intent"
|
||||||
|
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
assert completed["metadata"]["status"] == "UNDER_REVIEW"
|
||||||
|
|
||||||
|
approved = await built.resume(
|
||||||
|
"v1", {"approval": {"phase": "scoping", "decision": "approve"}}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert ANALYST in {run.role for run in runtime.runs}
|
||||||
|
assert approved.prompt["approval"]["phase"] == "requirements_discovery"
|
||||||
|
completed = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
assert completed["metadata"]["status"] == "APPROVED"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_an_essential_gap_refuses_a_start_with_nobody_to_ask(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
gappy = SPEC.replace(" data_residency: United Kingdom\n", "")
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="estate.data_residency") as refused:
|
||||||
|
await built.start("", scope=gappy)
|
||||||
|
|
||||||
|
assert "nobody to ask" in str(refused.value)
|
||||||
|
assert runtime.seen == {} and not list(tmp_path.rglob("history.json"))
|
||||||
|
# With someone present the same spec starts, and the gap is theirs to answer.
|
||||||
|
waiting = await built.start("", scope=gappy, pre_approved=False, execution_id="v2")
|
||||||
|
assert waiting.state["scope"]["status"] == "DRAFT"
|
||||||
|
asked = waiting.state["scope"]["context"]["to_ask"]
|
||||||
|
assert asked[0] == {
|
||||||
|
"field": "estate.data_residency",
|
||||||
|
"prompt": "Where must data stay?",
|
||||||
|
"severity": "HIGH",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_spec_that_cannot_be_read_refuses_the_start(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="networking.topolgy is not a field"):
|
||||||
|
await built.start("", scope=SPEC.replace("topology", "topolgy"))
|
||||||
|
with pytest.raises(ValueError, match="must be one of"):
|
||||||
|
await built.start(
|
||||||
|
"", scope=SPEC.replace("cloud_provider: gcp", "cloud_provider: aws")
|
||||||
|
)
|
||||||
|
with pytest.raises(ValueError, match="started for another-case"):
|
||||||
|
await built.start("", scope=SPEC, customer_case="another-case")
|
||||||
|
assert runtime.seen == {}
|
||||||
|
|
||||||
|
|
||||||
|
async def test_what_the_caller_says_outright_wins_over_the_specs_run_options(tmp_path):
|
||||||
|
built, _ = engine(tmp_path)
|
||||||
|
|
||||||
|
waiting = await built.start("", scope=SPEC, pre_approved=False, execution_id="v1")
|
||||||
|
|
||||||
|
assert waiting.state["pre_approved"] is False
|
||||||
|
assert waiting.status == "awaiting_user"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_request_is_still_needed_when_the_spec_has_no_statement(tmp_path):
|
||||||
|
built, _ = engine(tmp_path)
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="A request is needed"):
|
||||||
|
await built.start(
|
||||||
|
"",
|
||||||
|
scope=SPEC.replace(" statement: Build us a landing zone.\n", ""),
|
||||||
|
pre_approved=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_change_request_starts_from_the_scope_that_was_delivered(tmp_path):
|
||||||
|
built, runtime = engine(tmp_path)
|
||||||
|
await built.start("", scope=SPEC, execution_id="v1")
|
||||||
|
for name, turns in runtime._turns.items():
|
||||||
|
kind = next(
|
||||||
|
k for k, spec in DEPENDENCY_GRAPH.items() if spec.specialist == name
|
||||||
|
)
|
||||||
|
turns.append([sample_submission(kind)])
|
||||||
|
runtime._turns[SCOPE_ANALYST].clear()
|
||||||
|
runtime._turns[SCOPE_ANALYST].append(
|
||||||
|
[
|
||||||
|
Call("scope_read"),
|
||||||
|
Call(
|
||||||
|
"scope_set",
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"field": "estate.regions.secondary",
|
||||||
|
"value": "europe-west1",
|
||||||
|
"source": "answered",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
),
|
||||||
|
sample_submission(K.SCOPE),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
await built.start(
|
||||||
|
"We now need a second region.",
|
||||||
|
customer_case="eng-77",
|
||||||
|
pre_approved=True,
|
||||||
|
execution_id="v2",
|
||||||
|
baseline="v1",
|
||||||
|
)
|
||||||
|
|
||||||
|
run = [r for r in runtime.runs if r.role == SCOPE_ANALYST][-1]
|
||||||
|
# The delivered spec was there to read before anything was recorded.
|
||||||
|
assert "Kind of engagement: platform_foundation" in run.results[0][1]
|
||||||
|
intent = tmp_path / "deliverables" / "cases" / "eng-77" / "intent"
|
||||||
|
changed = yaml.safe_load((intent / "002.scope.completed.yaml").read_text())
|
||||||
|
assert changed["estate"]["regions"] == {
|
||||||
|
"primary": "europe-west2",
|
||||||
|
"secondary": "europe-west1",
|
||||||
|
"resiliency_tier": "ask",
|
||||||
|
}
|
||||||
|
assert changed["provenance"]["estate.regions.primary"] == "supplied"
|
||||||
|
assert changed["provenance"]["estate.regions.secondary"] == "answered"
|
||||||
|
# The Baseline's own copy is untouched.
|
||||||
|
first = yaml.safe_load((intent / "001.scope.completed.yaml").read_text())
|
||||||
|
assert first["estate"]["regions"]["secondary"] == "ask"
|
||||||
308
tests/execution/test_work_breakdown.py
Normal file
308
tests/execution/test_work_breakdown.py
Normal file
@@ -0,0 +1,308 @@
|
|||||||
|
"""The work breakdown structure: a plan checked, scheduled and drawn by code."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
|
|
||||||
|
from app.adk.fake_runtime import SAMPLE_DATA, Call, FakeAgentRuntime, sample_submission
|
||||||
|
from app.adk.runtime import NoDecider, RoleSpec
|
||||||
|
from app.agent import ExecutionEngine
|
||||||
|
from app.nodes.router import Router, Thresholds
|
||||||
|
from app.recorder import InMemoryRecorder
|
||||||
|
from app.workflows.deliverables import DEPENDENCY_GRAPH
|
||||||
|
from app.workflows.deliverables import DeliverableKind as K
|
||||||
|
from app.workflows.work_breakdown import (
|
||||||
|
check_work_breakdown,
|
||||||
|
mermaid,
|
||||||
|
render,
|
||||||
|
schedule,
|
||||||
|
summary,
|
||||||
|
)
|
||||||
|
|
||||||
|
GRAPH = SAMPLE_DATA["target-architecture.json"] # Elements E-1 and E-2
|
||||||
|
PLANNER = DEPENDENCY_GRAPH[K.WORK_BREAKDOWN].specialist
|
||||||
|
|
||||||
|
|
||||||
|
def upstream(**delivery) -> dict:
|
||||||
|
scope = {"stated": {"delivery": delivery}, "assumed": []}
|
||||||
|
return {
|
||||||
|
"target_architecture": {"attachments": {"target-architecture.json": GRAPH}},
|
||||||
|
"scope": {"attachments": {"scope.json": json.dumps(scope)}},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def plan(packages=None, activities=None) -> dict[str, str]:
|
||||||
|
data = json.loads(SAMPLE_DATA["work-breakdown.json"])
|
||||||
|
if packages is not None:
|
||||||
|
data["work_packages"] = packages
|
||||||
|
if activities is not None:
|
||||||
|
data["activities"] = activities
|
||||||
|
return {"work-breakdown.json": json.dumps(data)}
|
||||||
|
|
||||||
|
|
||||||
|
def activity(ident: str, days: float, *after: str, **more) -> dict:
|
||||||
|
return {
|
||||||
|
"id": ident,
|
||||||
|
"work_package": "WP-1",
|
||||||
|
"name": f"Do {ident}",
|
||||||
|
"days": days,
|
||||||
|
"after": list(after),
|
||||||
|
**more,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
PACKAGE = json.loads(SAMPLE_DATA["work-breakdown.json"])["work_packages"][0]
|
||||||
|
|
||||||
|
|
||||||
|
# --- the plan is checked against the design ---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_plan_that_covers_the_design_is_accepted():
|
||||||
|
assert check_work_breakdown(plan(), upstream()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_every_element_of_the_design_belongs_to_a_work_package():
|
||||||
|
only_one = [{**PACKAGE, "elements": ["E-1"]}]
|
||||||
|
invented = [{**PACKAGE, "elements": ["E-1", "E-2", "E-9"]}]
|
||||||
|
|
||||||
|
(left_out,) = check_work_breakdown(plan(only_one), upstream())
|
||||||
|
(unknown,) = check_work_breakdown(plan(invented), upstream())
|
||||||
|
|
||||||
|
assert "no work package delivers E-2" in left_out
|
||||||
|
assert "WP-1 delivers E-9, which is not an Element" in unknown
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_plan_that_cannot_be_scheduled_is_refused_with_the_reason():
|
||||||
|
def problems(*activities):
|
||||||
|
return check_work_breakdown(plan(activities=list(activities)), upstream())
|
||||||
|
|
||||||
|
assert any(
|
||||||
|
"depend on each other in a circle" in p
|
||||||
|
for p in problems(activity("A-1", 1, "A-2"), activity("A-2", 1, "A-1"))
|
||||||
|
)
|
||||||
|
assert any(
|
||||||
|
"comes after A-9, which is not an activity" in p
|
||||||
|
for p in problems(activity("A-1", 1, "A-9"))
|
||||||
|
)
|
||||||
|
assert any(
|
||||||
|
"needs `days`, a number above 0" in p for p in problems(activity("A-1", 0))
|
||||||
|
)
|
||||||
|
assert any("split anything over" in p for p in problems(activity("A-1", 400)))
|
||||||
|
assert any(
|
||||||
|
"owner must be one of" in p
|
||||||
|
for p in problems(activity("A-1", 1, owner="vendor"))
|
||||||
|
)
|
||||||
|
assert any(
|
||||||
|
"is in WP-7, not a work package" in p
|
||||||
|
for p in problems({**activity("A-1", 1), "work_package": "WP-7"})
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_work_package_holds_work_or_other_work_packages():
|
||||||
|
parent = {**PACKAGE, "id": "WP-0", "elements": []}
|
||||||
|
child = {**PACKAGE, "parent": "WP-0"}
|
||||||
|
idle = {**PACKAGE, "id": "WP-2", "elements": []}
|
||||||
|
|
||||||
|
assert check_work_breakdown(plan([parent, child]), upstream()) == []
|
||||||
|
assert any(
|
||||||
|
"WP-2 has no activity and no work package beneath it" in p
|
||||||
|
for p in check_work_breakdown(
|
||||||
|
plan([child | {"parent": None}, idle]), upstream()
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert any(
|
||||||
|
"has parent WP-9, which does not exist" in p
|
||||||
|
for p in check_work_breakdown(plan([{**PACKAGE, "parent": "WP-9"}]), upstream())
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --- the schedule is computed -----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_activity_starts_when_what_it_follows_has_finished():
|
||||||
|
placed = schedule(
|
||||||
|
[activity("A-1", 2), activity("A-2", 3, "A-1"), activity("A-3", 1, "A-2")]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert [(p.start, p.finish) for p in placed.values()] == [(0, 2), (2, 5), (5, 6)]
|
||||||
|
assert all(p.critical for p in placed.values())
|
||||||
|
|
||||||
|
|
||||||
|
def test_independent_work_runs_at_once_up_to_the_size_of_the_team():
|
||||||
|
work = [activity("A-1", 4), activity("A-2", 4), activity("A-3", 4)]
|
||||||
|
|
||||||
|
two = schedule(work, team_size=2)
|
||||||
|
three = schedule(work, team_size=3)
|
||||||
|
|
||||||
|
assert max(p.finish for p in two.values()) == 8
|
||||||
|
assert sorted(p.start for p in two.values()) == [0, 0, 4]
|
||||||
|
assert max(p.finish for p in three.values()) == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_what_the_customer_owns_takes_nobody_from_the_team():
|
||||||
|
work = [
|
||||||
|
activity("A-1", 10, owner="customer"),
|
||||||
|
activity("A-2", 3),
|
||||||
|
activity("A-3", 2, "A-1", "A-2"),
|
||||||
|
]
|
||||||
|
|
||||||
|
placed = schedule(work, team_size=1)
|
||||||
|
|
||||||
|
# The lead time and the team's work run together; the end waits on the longer.
|
||||||
|
assert placed["A-2"].start == 0 and placed["A-3"].start == 10
|
||||||
|
assert placed["A-1"].critical and placed["A-3"].critical
|
||||||
|
assert not placed["A-2"].critical
|
||||||
|
|
||||||
|
|
||||||
|
def test_when_people_are_short_the_longest_chain_goes_first():
|
||||||
|
work = [activity("A-1", 1), activity("A-2", 1), activity("A-3", 5, "A-2")]
|
||||||
|
|
||||||
|
placed = schedule(work, team_size=1)
|
||||||
|
|
||||||
|
assert placed["A-2"].start == 0 and placed["A-1"].start > 0
|
||||||
|
|
||||||
|
|
||||||
|
# --- the views are written from the plan ------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_three_views_are_written_from_the_data():
|
||||||
|
text = render(plan(), upstream(team_size=2))
|
||||||
|
|
||||||
|
assert "## WBS dictionary" in text
|
||||||
|
# Acceptance is the Checks on the work package's Elements, read from the design.
|
||||||
|
assert (
|
||||||
|
"| WP-1 | Ingress | The service and its identity. | a running service | E-1, E-2 | K-1, K-2 |"
|
||||||
|
in text
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
"| A-2 | Deploy the service | WP-1 | 2 | A-1 | Day 1 | Day 3 | team | yes |"
|
||||||
|
in text
|
||||||
|
)
|
||||||
|
assert "## Timeline" in text and "Day 0" in text
|
||||||
|
assert "3 working days, about 0.6 weeks, with a team of 2 and a 5-day week." in text
|
||||||
|
assert "indicative" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_plan_over_the_delivery_window_says_so_and_names_the_critical_path():
|
||||||
|
long = plan(activities=[activity("A-1", 20), activity("A-2", 15, "A-1")])
|
||||||
|
|
||||||
|
over = render(long, upstream(team_size=2, window_weeks=4))
|
||||||
|
inside = render(long, upstream(team_size=2, window_weeks=10))
|
||||||
|
|
||||||
|
assert "**That is 3.0 weeks over the 4-week delivery window.**" in over
|
||||||
|
assert "The critical path is A-1 -> A-2" in over
|
||||||
|
assert "inside the 10-week delivery window, with 3.0 weeks to spare" in inside
|
||||||
|
assert summary(long, upstream(window_weeks=4))["within_window"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_assumed_team_size_is_said_to_be_assumed():
|
||||||
|
assert "(assumed; the scope gives no team size)" in render(plan(), upstream())
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_customers_lead_time_is_shown_as_theirs():
|
||||||
|
waiting = plan(
|
||||||
|
activities=[
|
||||||
|
activity(
|
||||||
|
"A-1", 30, owner="customer", assumption="The carrier takes six weeks."
|
||||||
|
),
|
||||||
|
activity("A-2", 2, "A-1"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
text = render(waiting, upstream(team_size=2))
|
||||||
|
|
||||||
|
assert "The customer owns these, and the plan waits on them:" in text
|
||||||
|
assert "- A-1 Do A-1: 30 days. The carrier takes six weeks." in text
|
||||||
|
assert "░" in text and "█" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_calendar_dates_and_a_chart_appear_only_with_a_start_date():
|
||||||
|
dated = upstream(team_size=2, target_start_date="2026-11-02") # a Monday
|
||||||
|
|
||||||
|
assert mermaid(plan(), upstream(team_size=2)) == ""
|
||||||
|
assert "Dates" not in render(plan(), upstream(team_size=2))
|
||||||
|
chart = mermaid(plan(), dated)
|
||||||
|
assert chart.startswith("gantt\n dateFormat YYYY-MM-DD")
|
||||||
|
assert "Deploy the service :crit, A-2, 2026-11-03, 2d" in chart
|
||||||
|
assert "| 2026-11-03 to 2026-11-04 |" in render(plan(), dated)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_week_of_working_days_skips_the_weekend():
|
||||||
|
week = plan(activities=[activity("A-1", 5), activity("A-2", 1, "A-1")])
|
||||||
|
|
||||||
|
text = render(week, upstream(team_size=1, target_start_date="2026-11-02"))
|
||||||
|
|
||||||
|
# Five working days from Monday end on Friday; the next starts on Monday.
|
||||||
|
assert "| 2026-11-02 to 2026-11-06 |" in text
|
||||||
|
assert "| 2026-11-09 to 2026-11-09 |" in text
|
||||||
|
|
||||||
|
|
||||||
|
# --- in an Execution --------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_an_execution_delivers_a_plan_and_adds_its_timeline_to_the_guide(
|
||||||
|
tmp_path,
|
||||||
|
):
|
||||||
|
roles = {
|
||||||
|
"orchestrator": RoleSpec("orchestrator", "Unused.", budget=40),
|
||||||
|
**{
|
||||||
|
spec.specialist: RoleSpec(spec.specialist, "Produce.", budget=15)
|
||||||
|
for spec in DEPENDENCY_GRAPH.values()
|
||||||
|
},
|
||||||
|
}
|
||||||
|
runtime = FakeAgentRuntime(
|
||||||
|
{
|
||||||
|
spec.specialist: [[sample_submission(kind)]]
|
||||||
|
for kind, spec in DEPENDENCY_GRAPH.items()
|
||||||
|
}
|
||||||
|
)
|
||||||
|
engine = ExecutionEngine(
|
||||||
|
runtime=runtime,
|
||||||
|
roles=roles,
|
||||||
|
recorder=InMemoryRecorder(),
|
||||||
|
checkpointer=InMemorySaver(),
|
||||||
|
output_root=tmp_path,
|
||||||
|
orchestrator=Router(NoDecider(), None, Thresholds()),
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await engine.start("Ingest", customer_case="acme", pre_approved=True)
|
||||||
|
|
||||||
|
assert result.status == "completed"
|
||||||
|
out = next(tmp_path.rglob("execution.json")).parent
|
||||||
|
breakdown = (out / "deliverables" / "work_breakdown.md").read_text()
|
||||||
|
assert "## WBS dictionary" in breakdown and "## Activity sequence" in breakdown
|
||||||
|
assert (out / "work-breakdown.json").is_file()
|
||||||
|
guide = (out / "solution-architecture-guide.md").read_text()
|
||||||
|
assert "## Delivery plan" in guide and "A-2 Deploy the service" in guide
|
||||||
|
manifest = json.loads((out / "execution.json").read_text())
|
||||||
|
assert manifest["delivery_plan"]["working_days"] == 3
|
||||||
|
assert manifest["delivery_plan"]["critical_path"] == ["A-1", "A-2"]
|
||||||
|
# The planner was given the design and the code to plan from, in full.
|
||||||
|
planner = next(run for run in runtime.runs if run.role == PLANNER)
|
||||||
|
given = planner.task.context["dependencies"]
|
||||||
|
assert "content" in given["target_architecture"]
|
||||||
|
assert "content" in given["deployment_guidance"]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_plan_that_leaves_an_element_out_is_returned_to_the_planner(tmp_path):
|
||||||
|
from app.tools.execution import specialist_tools
|
||||||
|
|
||||||
|
(submit,) = (
|
||||||
|
tool
|
||||||
|
for tool in specialist_tools(
|
||||||
|
K.WORK_BREAKDOWN, (), allow_cloud_read=False, upstream=upstream()
|
||||||
|
)
|
||||||
|
if tool.name == "submit_deliverable"
|
||||||
|
)
|
||||||
|
partial = Call(
|
||||||
|
"submit_deliverable",
|
||||||
|
{
|
||||||
|
"content": "## Approach\nIn order.\n",
|
||||||
|
"attachments": plan([{**PACKAGE, "elements": ["E-1"]}]),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
refused = await submit.handler(**partial.args)
|
||||||
|
|
||||||
|
assert refused.outcome is None
|
||||||
|
assert "no work package delivers E-2" in refused.message
|
||||||
@@ -386,7 +386,8 @@ def test_the_validator_reads_the_code_whole_and_the_graph_as_data():
|
|||||||
def test_the_solution_guide_is_assembled_from_every_deliverable_whole():
|
def test_the_solution_guide_is_assembled_from_every_deliverable_whole():
|
||||||
given = dependencies_for(K.SOLUTION_GUIDE, submitted())
|
given = dependencies_for(K.SOLUTION_GUIDE, submitted())
|
||||||
|
|
||||||
assert len(given) == len(DEPENDENCY_GRAPH) - 1
|
# Every Deliverable but the guide itself and the plan produced beside it.
|
||||||
|
assert len(given) == len(DEPENDENCY_GRAPH) - 2
|
||||||
assert all("content" in deliverable for deliverable in given.values())
|
assert all("content" in deliverable for deliverable in given.values())
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
167
tests/scoped.py
Normal file
167
tests/scoped.py
Normal file
@@ -0,0 +1,167 @@
|
|||||||
|
"""Helpers for tests about what comes after scoping.
|
||||||
|
|
||||||
|
Every Execution now begins with the scoping Phase (ADR-0008). A test about
|
||||||
|
requirements, design or validation is not about that, so these run scoping ahead
|
||||||
|
of the test's own script and leave the test where it always started: requirements
|
||||||
|
ready, nothing recorded of how the scope was agreed.
|
||||||
|
|
||||||
|
The work breakdown structure (iteration 10) is produced beside the solution guide.
|
||||||
|
A script written before it existed delegates the guide alone; here the plan is
|
||||||
|
delegated with it, and planned without a script.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from app.adk.fake_runtime import FakeAgentRuntime, sample_submission
|
||||||
|
from app.adk.runtime import Submitted
|
||||||
|
from app.agent import ExecutionEngine, Interaction
|
||||||
|
from app.workflows.deliverables import DEPENDENCY_GRAPH, DeliverableKind, PhaseId
|
||||||
|
|
||||||
|
SCOPE_ANALYST = DEPENDENCY_GRAPH[DeliverableKind.SCOPE].specialist
|
||||||
|
PLANNER = DEPENDENCY_GRAPH[DeliverableKind.WORK_BREAKDOWN].specialist
|
||||||
|
ORCHESTRATOR = "orchestrator"
|
||||||
|
|
||||||
|
|
||||||
|
def _with_the_plan(tool): # type: ignore[no-untyped-def]
|
||||||
|
"""`delegate`, delegating the work breakdown structure whenever the solution
|
||||||
|
guide is delegated."""
|
||||||
|
if tool.name != "delegate":
|
||||||
|
return tool
|
||||||
|
guide, plan = (
|
||||||
|
DeliverableKind.SOLUTION_GUIDE.value,
|
||||||
|
DeliverableKind.WORK_BREAKDOWN.value,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def delegate(kinds, **rest): # type: ignore[no-untyped-def]
|
||||||
|
if guide in kinds and plan not in kinds:
|
||||||
|
kinds = [*kinds, plan]
|
||||||
|
return await tool.handler(kinds=kinds, **rest)
|
||||||
|
|
||||||
|
return dataclasses.replace(tool, handler=delegate)
|
||||||
|
|
||||||
|
|
||||||
|
class ScopedRuntime(FakeAgentRuntime):
|
||||||
|
"""A scripted runtime that first has the scope produced and its Approval asked
|
||||||
|
for, outside the script and outside `runs`."""
|
||||||
|
|
||||||
|
def __init__(self, turns: Any) -> None:
|
||||||
|
super().__init__(turns)
|
||||||
|
self._next = "delegate"
|
||||||
|
|
||||||
|
async def run(self, role, task, tools): # type: ignore[no-untyped-def]
|
||||||
|
by_name = {tool.name: tool for tool in tools}
|
||||||
|
if role.name == PLANNER and not self.remaining(PLANNER):
|
||||||
|
call = sample_submission(DeliverableKind.WORK_BREAKDOWN)
|
||||||
|
return Submitted(call.args["content"], call.args["attachments"])
|
||||||
|
if role.name == ORCHESTRATOR and "delegate" in by_name:
|
||||||
|
tools = [_with_the_plan(tool) for tool in tools]
|
||||||
|
if role.name == SCOPE_ANALYST and not self.remaining(SCOPE_ANALYST):
|
||||||
|
# Submitted as it stands: a test about a later Deliverable's draft is
|
||||||
|
# not asking whether the scope's draft was shown first.
|
||||||
|
call = sample_submission(DeliverableKind.SCOPE)
|
||||||
|
return Submitted(call.args["content"], call.args["attachments"])
|
||||||
|
if role.name == ORCHESTRATOR and self._next == "delegate":
|
||||||
|
self._next = "approve"
|
||||||
|
asked = await by_name["delegate"].handler(
|
||||||
|
kinds=[DeliverableKind.SCOPE.value]
|
||||||
|
)
|
||||||
|
return asked.outcome
|
||||||
|
if role.name == ORCHESTRATOR and self._next == "approve":
|
||||||
|
self._next = "done"
|
||||||
|
asked = await by_name["request_approval"].handler(
|
||||||
|
phase=PhaseId.SCOPING.value, summary="The scope is ready."
|
||||||
|
)
|
||||||
|
if asked.outcome is not None:
|
||||||
|
return asked.outcome # a user is present: their Approval is awaited
|
||||||
|
return await super().run(role, task, tools)
|
||||||
|
|
||||||
|
|
||||||
|
class ScopedEngine(ExecutionEngine):
|
||||||
|
"""An engine whose `start` grants the scoping Approval when it is asked for, so
|
||||||
|
an interactive test meets the first prompt it was written about."""
|
||||||
|
|
||||||
|
async def start(self, *args: Any, **kwargs: Any) -> Interaction:
|
||||||
|
interaction = await super().start(*args, **kwargs)
|
||||||
|
approval = (interaction.prompt or {}).get("approval") or {}
|
||||||
|
if approval.get("phase") == PhaseId.SCOPING.value:
|
||||||
|
interaction = await self.resume(
|
||||||
|
interaction.execution_id,
|
||||||
|
{"approval": {"phase": PhaseId.SCOPING.value, "decision": "approve"}},
|
||||||
|
)
|
||||||
|
return interaction
|
||||||
|
|
||||||
|
|
||||||
|
def scoped_ledger(**options: Any): # type: ignore[no-untyped-def]
|
||||||
|
"""A Ledger whose scope is agreed: where requirements discovery begins."""
|
||||||
|
from app.states.ledger import Ledger, PhaseStatus
|
||||||
|
|
||||||
|
ledger = Ledger(**options)
|
||||||
|
attempt = ledger.start([DeliverableKind.SCOPE], actor=ORCHESTRATOR)
|
||||||
|
ledger.submit(
|
||||||
|
DeliverableKind.SCOPE, attempt[DeliverableKind.SCOPE], actor=SCOPE_ANALYST
|
||||||
|
)
|
||||||
|
ledger.request_approval(PhaseId.SCOPING, actor=ORCHESTRATOR)
|
||||||
|
if ledger.phase_status(PhaseId.SCOPING) is PhaseStatus.AWAITING_APPROVAL:
|
||||||
|
ledger.approve(PhaseId.SCOPING, actor="user")
|
||||||
|
return ledger
|
||||||
|
|
||||||
|
|
||||||
|
def carrying(sources: list[str]) -> dict[str, list]:
|
||||||
|
"""Scripted turns for the Specialists along the Requirements chain, in which one
|
||||||
|
more requirement, NFR-2, carries what the customer stated (`sources`), and a
|
||||||
|
Check, K-3, holds the design to it."""
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.adk.fake_runtime import SAMPLE_DATA, Call
|
||||||
|
|
||||||
|
def submission(kind: DeliverableKind, **files: str) -> Call:
|
||||||
|
call = sample_submission(kind)
|
||||||
|
return Call(
|
||||||
|
call.tool,
|
||||||
|
{**call.args, "attachments": {**call.args["attachments"], **files}},
|
||||||
|
)
|
||||||
|
|
||||||
|
requirements = json.loads(SAMPLE_DATA["requirements.json"])
|
||||||
|
requirements.append(
|
||||||
|
{
|
||||||
|
"id": "NFR-2",
|
||||||
|
"kind": "non_functional",
|
||||||
|
"text": "The design honours what the customer stated.",
|
||||||
|
"conditions": ["Each stated choice is configured as stated."],
|
||||||
|
"from": sources,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
components = json.loads(SAMPLE_DATA["components.json"])
|
||||||
|
components[0]["requirements"].append("NFR-2")
|
||||||
|
graph = json.loads(SAMPLE_DATA["target-architecture.json"])
|
||||||
|
graph["checks"].append(
|
||||||
|
{
|
||||||
|
"id": "K-3",
|
||||||
|
"applies_to": "E-1",
|
||||||
|
"source": "NFR-2",
|
||||||
|
"statement": "the stated choices are configured",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
implementation = json.loads(SAMPLE_DATA["implementation.json"])
|
||||||
|
implementation.append(
|
||||||
|
{"check": "K-3", "file": "terraform/main.tf", "address": "example.run"}
|
||||||
|
)
|
||||||
|
verdicts = json.loads(SAMPLE_DATA["verdicts.json"])
|
||||||
|
verdicts["checks"].append(
|
||||||
|
{"check": "K-3", "verdict": "pass", "evidence": "Configured as stated."}
|
||||||
|
)
|
||||||
|
K = DeliverableKind
|
||||||
|
files = {
|
||||||
|
K.REQUIREMENTS: {"requirements.json": json.dumps(requirements)},
|
||||||
|
K.TECHNICAL_DECOMPOSITION: {"components.json": json.dumps(components)},
|
||||||
|
K.TARGET_ARCHITECTURE: {"target-architecture.json": json.dumps(graph)},
|
||||||
|
K.DEPLOYMENT_GUIDANCE: {"implementation.json": json.dumps(implementation)},
|
||||||
|
K.VALIDATION_RESULTS: {"verdicts.json": json.dumps(verdicts)},
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
DEPENDENCY_GRAPH[kind].specialist: [[submission(kind, **attached)]]
|
||||||
|
for kind, attached in files.items()
|
||||||
|
}
|
||||||
@@ -57,12 +57,11 @@ async def test_preview_mcp_servers_follow_the_setting(tmp_path):
|
|||||||
|
|
||||||
async def test_the_assembly_routes_by_default_and_runs_without_a_jev_key(tmp_path):
|
async def test_the_assembly_routes_by_default_and_runs_without_a_jev_key(tmp_path):
|
||||||
"""With no Jev key the router follows its playbook on rules alone."""
|
"""With no Jev key the router follows its playbook on rules alone."""
|
||||||
from app.adk.fake_runtime import sample_submission
|
from app.adk.fake_runtime import Call
|
||||||
from app.adk.runtime import NoDecider
|
from app.adk.runtime import NoDecider
|
||||||
from app.workflows.deliverables import DeliverableKind
|
|
||||||
|
|
||||||
runtime = FakeAgentRuntime(
|
runtime = FakeAgentRuntime(
|
||||||
{"requirements-analyst": [[sample_submission(DeliverableKind.REQUIREMENTS)]]}
|
{"scope-analyst": [[Call("needs_input", {"questions": ["Which region?"]})]]}
|
||||||
)
|
)
|
||||||
|
|
||||||
async with open_assembly(
|
async with open_assembly(
|
||||||
@@ -74,9 +73,10 @@ async def test_the_assembly_routes_by_default_and_runs_without_a_jev_key(tmp_pat
|
|||||||
interaction = await assembly.engine.start("Design ingestion")
|
interaction = await assembly.engine.start("Design ingestion")
|
||||||
|
|
||||||
assert isinstance(assembly.decider, NoDecider)
|
assert isinstance(assembly.decider, NoDecider)
|
||||||
# The router delegated requirements with no Orchestrator model, then the
|
# The router delegated the scope with no Orchestrator model, and the scope
|
||||||
# decomposer (unscripted here) stopped and its question reached the user.
|
# analyst's question reached the user.
|
||||||
assert [run.role for run in runtime.runs] == ["requirements-analyst"]
|
assert [run.role for run in runtime.runs] == ["scope-analyst"]
|
||||||
|
assert "scope_set" in runtime.runs[0].tools
|
||||||
assert interaction.status == "awaiting_user"
|
assert interaction.status == "awaiting_user"
|
||||||
assert "reply-writer" in assembly.roles
|
assert "reply-writer" in assembly.roles
|
||||||
|
|
||||||
|
|||||||
290
tests/test_scope.py
Normal file
290
tests/test_scope.py
Normal file
@@ -0,0 +1,290 @@
|
|||||||
|
"""The scope spec (ADR-0008): read against the catalogue, with no model involved."""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from app.providers import load_provider
|
||||||
|
from app.scope import (
|
||||||
|
APPROVED,
|
||||||
|
ASK,
|
||||||
|
DRAFT,
|
||||||
|
ESSENTIAL,
|
||||||
|
UNDER_REVIEW,
|
||||||
|
Catalogue,
|
||||||
|
FieldSpec,
|
||||||
|
ScopeError,
|
||||||
|
read_scope,
|
||||||
|
render_reading,
|
||||||
|
render_template,
|
||||||
|
slug,
|
||||||
|
)
|
||||||
|
|
||||||
|
CATALOGUE = Catalogue(
|
||||||
|
schema_version=1,
|
||||||
|
kinds=("workload", "platform_foundation"),
|
||||||
|
fields={
|
||||||
|
spec.path: spec
|
||||||
|
for spec in (
|
||||||
|
FieldSpec(
|
||||||
|
"metadata.customer_case",
|
||||||
|
tier=ESSENTIAL,
|
||||||
|
aliases=("metadata.engagement_id",),
|
||||||
|
),
|
||||||
|
FieldSpec(
|
||||||
|
"engagement.kind",
|
||||||
|
type="enum",
|
||||||
|
values=("workload", "platform_foundation"),
|
||||||
|
tier=ESSENTIAL,
|
||||||
|
),
|
||||||
|
FieldSpec("intent.statement", type="text", tier=ESSENTIAL),
|
||||||
|
FieldSpec(
|
||||||
|
"intent.requirements",
|
||||||
|
type="records",
|
||||||
|
keys=("id", "statement"),
|
||||||
|
optional_keys=("category",),
|
||||||
|
),
|
||||||
|
FieldSpec(
|
||||||
|
"users.count",
|
||||||
|
type="integer",
|
||||||
|
tier={"workload": ESSENTIAL},
|
||||||
|
question="How many users?",
|
||||||
|
),
|
||||||
|
FieldSpec(
|
||||||
|
"users.peak",
|
||||||
|
type="integer",
|
||||||
|
tier="important",
|
||||||
|
question="How many at once?",
|
||||||
|
default="a tenth of the users",
|
||||||
|
affects="scaling limits",
|
||||||
|
),
|
||||||
|
FieldSpec("estate.region", tier="important", question="Which region?"),
|
||||||
|
FieldSpec("network.topology", type="enum", provider_values=True),
|
||||||
|
FieldSpec("delivery.start", type="date"),
|
||||||
|
FieldSpec("run.pre_approved", type="boolean"),
|
||||||
|
)
|
||||||
|
},
|
||||||
|
)
|
||||||
|
COMPLETE = """
|
||||||
|
metadata: {customer_case: acme-bookings}
|
||||||
|
engagement: {kind: workload}
|
||||||
|
intent:
|
||||||
|
statement: Staff book rooms.
|
||||||
|
requirements:
|
||||||
|
- {id: REQ-01, statement: Rooms can be booked.}
|
||||||
|
users: {count: 400, peak: 40}
|
||||||
|
estate: {region: europe-west2}
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def read(text: str, **options):
|
||||||
|
return read_scope(text, CATALOGUE, **options)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_complete_spec_has_nothing_to_ask():
|
||||||
|
scope = read(COMPLETE)
|
||||||
|
|
||||||
|
assert scope.kind == "workload" and scope.customer_case == "acme-bookings"
|
||||||
|
assert scope.statement == "Staff book rooms."
|
||||||
|
assert scope.gaps(ESSENTIAL, "important") == []
|
||||||
|
assert scope.values["users.count"] == 400
|
||||||
|
assert (
|
||||||
|
scope.status() == UNDER_REVIEW and scope.status(pre_approved=True) == APPROVED
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_field_left_out_or_marked_ask_is_a_gap_with_its_question():
|
||||||
|
scope = read(COMPLETE.replace("count: 400, peak: 40", "peak: ask"))
|
||||||
|
|
||||||
|
assert scope.gaps() == [
|
||||||
|
{"field": "users.count", "prompt": "How many users?", "severity": "HIGH"},
|
||||||
|
{"field": "users.peak", "prompt": "How many at once?", "severity": "MEDIUM"},
|
||||||
|
# Optional fields nobody filled are gaps too, the least pressing.
|
||||||
|
*[gap for gap in scope.gaps() if gap["severity"] == "LOW"],
|
||||||
|
]
|
||||||
|
assert scope.status(pre_approved=True) == DRAFT
|
||||||
|
|
||||||
|
|
||||||
|
def test_decide_is_an_assumption_and_not_applicable_is_neither():
|
||||||
|
scope = read(
|
||||||
|
COMPLETE.replace("peak: 40", "peak: decide").replace(
|
||||||
|
"estate: {region: europe-west2}", "estate: not_applicable"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
(assumed,) = scope.assumptions()
|
||||||
|
assert assumed["id"] == "ASSUMP-01" and assumed["field"] == "users.peak"
|
||||||
|
assert "a tenth of the users" in assumed["statement"]
|
||||||
|
assert assumed["impact_if_invalid"] == "Changes scaling limits."
|
||||||
|
# A whole section marked at once covers every field in it.
|
||||||
|
assert scope.states["estate.region"] == "not_applicable"
|
||||||
|
assert all(gap["field"] != "estate.region" for gap in scope.gaps())
|
||||||
|
|
||||||
|
|
||||||
|
def test_which_fields_are_essential_depends_on_the_kind_of_engagement():
|
||||||
|
foundation = read(
|
||||||
|
COMPLETE.replace("workload", "platform_foundation").replace(
|
||||||
|
"users: {count: 400, peak: 40}", ""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert foundation.gaps(ESSENTIAL) == []
|
||||||
|
assert foundation.tier("users.count") == "optional"
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_field_is_named_never_ignored():
|
||||||
|
with pytest.raises(ScopeError) as refused:
|
||||||
|
read(COMPLETE + "userz: {count: 1}\nestate: {regoin: x}\n")
|
||||||
|
|
||||||
|
assert "userz is not a field of the scope spec" in refused.value.problems
|
||||||
|
# The second estate key replaced the first; its typo is reported too.
|
||||||
|
assert any("estate.regoin" in problem for problem in refused.value.problems)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("change", "fault"),
|
||||||
|
[
|
||||||
|
(("count: 400", "count: many"), "users.count: must be a whole number"),
|
||||||
|
(("kind: workload", "kind: holiday"), "must be one of workload"),
|
||||||
|
(
|
||||||
|
("{id: REQ-01, statement: Rooms can be booked.}", "{id: REQ-01}"),
|
||||||
|
"needs statement",
|
||||||
|
),
|
||||||
|
(("estate: {region: europe-west2}", "estate: europe"), "estate is a section"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_a_value_of_the_wrong_kind_is_refused_with_the_reason(change, fault):
|
||||||
|
with pytest.raises(ScopeError) as refused:
|
||||||
|
read(COMPLETE.replace(*change))
|
||||||
|
|
||||||
|
assert any(fault in problem for problem in refused.value.problems)
|
||||||
|
|
||||||
|
|
||||||
|
def test_text_that_is_not_yaml_is_refused():
|
||||||
|
with pytest.raises(ScopeError, match="not valid YAML"):
|
||||||
|
read("intent: [unclosed")
|
||||||
|
with pytest.raises(ScopeError, match="must be a mapping"):
|
||||||
|
read("- just\n- a list\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_provider_supplies_the_values_the_catalogue_leaves_to_it():
|
||||||
|
spec = COMPLETE + "network: {topology: hub_and_spoke}\n"
|
||||||
|
|
||||||
|
with pytest.raises(ScopeError):
|
||||||
|
read(spec, provider_values={"network.topology": ["flat"]})
|
||||||
|
allowed = read(spec, provider_values={"network.topology": ["hub_and_spoke"]})
|
||||||
|
|
||||||
|
assert allowed.values["network.topology"] == "hub_and_spoke"
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_engagement_reference_and_a_customer_case_name_the_same_thing():
|
||||||
|
by_engagement = read(
|
||||||
|
COMPLETE.replace(
|
||||||
|
"customer_case: acme-bookings", "engagement_id: ENG-2026-UK-084"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert by_engagement.customer_case == "eng-2026-uk-084"
|
||||||
|
assert slug("ENG 2026/UK 084") == "eng-2026-uk-084"
|
||||||
|
with pytest.raises(ScopeError, match="name the same thing and differ"):
|
||||||
|
read(
|
||||||
|
COMPLETE.replace(
|
||||||
|
"{customer_case: acme-bookings}", "{customer_case: a, engagement_id: b}"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_completed_copy_holds_every_field_and_the_agents_sections():
|
||||||
|
scope = read(
|
||||||
|
COMPLETE.replace("peak: 40", "peak: decide") + "delivery: {start: 2026-11-01}\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
completed = scope.completed(DRAFT)
|
||||||
|
|
||||||
|
assert completed["metadata"]["status"] == DRAFT
|
||||||
|
assert completed["users"] == {"count": 400, "peak": "decide"}
|
||||||
|
assert completed["delivery"]["start"] == "2026-11-01" # plain data, not a date
|
||||||
|
assert completed["network"]["topology"] == ASK
|
||||||
|
assert completed["provenance"]["users.count"] == "supplied"
|
||||||
|
assert completed["provenance"]["users.peak"] == "assumed"
|
||||||
|
assert completed["provenance"]["network.topology"] == "unknown"
|
||||||
|
assert completed["readiness"]["supplied"] == len(scope.values)
|
||||||
|
assert completed["inferred_assumptions"][0]["field"] == "users.peak"
|
||||||
|
assert yaml.safe_load(yaml.safe_dump(completed)) == completed
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_completed_copy_can_be_read_back_as_a_spec():
|
||||||
|
"""The agent's own sections are replaced, not rejected, on the way back in."""
|
||||||
|
scope = read(COMPLETE.replace("peak: 40", "peak: decide"))
|
||||||
|
again = read(yaml.safe_dump(scope.completed(DRAFT)))
|
||||||
|
|
||||||
|
assert again.values == scope.values and again.states == scope.states
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_specialist_is_shown_what_is_stated_and_what_is_open():
|
||||||
|
scope = read(
|
||||||
|
COMPLETE.replace("peak: 40", "peak: decide") + "run: {pre_approved: true}\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
context = scope.for_context()
|
||||||
|
|
||||||
|
assert context["kind"] == "workload"
|
||||||
|
assert context["stated"]["users"] == {"count": 400}
|
||||||
|
assert context["stated"]["intent"]["requirements"][0]["id"] == "REQ-01"
|
||||||
|
assert context["to_decide"][0]["field"] == "users.peak"
|
||||||
|
# How to run the agent, and the case's name, are not the Specialists' business.
|
||||||
|
assert "run" not in context["stated"] and "metadata" not in context["stated"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_template_and_a_reading_are_written_for_a_person():
|
||||||
|
template = yaml.safe_load(render_template(CATALOGUE, "workload"))
|
||||||
|
reading = render_reading(read(COMPLETE.replace("count: 400, peak: 40", "peak: 40")))
|
||||||
|
|
||||||
|
assert template["engagement"]["kind"] == "workload"
|
||||||
|
assert template["users"] == {"count": ASK, "peak": ASK}
|
||||||
|
assert "delivery" not in template # optional fields are left out of a template
|
||||||
|
assert "Essential, needed before work can start:" in reading
|
||||||
|
assert "users.count: How many users?" in reading
|
||||||
|
|
||||||
|
|
||||||
|
# --- the real catalogue and the benchmark spec ------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_gcp_bundle_reads_the_benchmark_landing_zone_spec():
|
||||||
|
bundle = load_provider("gcp")
|
||||||
|
text = (
|
||||||
|
Path(__file__).resolve().parent.parent
|
||||||
|
/ "eval"
|
||||||
|
/ "datasets"
|
||||||
|
/ "specs"
|
||||||
|
/ "landing-zone.yaml"
|
||||||
|
).read_text()
|
||||||
|
|
||||||
|
scope = read_scope(
|
||||||
|
text, bundle.scope_catalogue(), provider_values=bundle.scope_values
|
||||||
|
)
|
||||||
|
|
||||||
|
assert scope.kind == "platform_foundation"
|
||||||
|
assert scope.customer_case == "case-08-landing-zone"
|
||||||
|
# Complete enough to start with nobody to ask.
|
||||||
|
assert scope.gaps(ESSENTIAL) == []
|
||||||
|
assert scope.values["networking.topology"] == "shared_vpc_hub_and_spoke"
|
||||||
|
assert scope.states["workload.users.count"] == "not_applicable"
|
||||||
|
assert [item["id"] for item in scope.values["intent.requirements"]] == [
|
||||||
|
"REQ-01",
|
||||||
|
"REQ-02",
|
||||||
|
"REQ-03",
|
||||||
|
"REQ-04",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_every_essential_field_of_the_catalogue_has_a_question_to_ask():
|
||||||
|
catalogue = load_provider("gcp").scope_catalogue()
|
||||||
|
|
||||||
|
for kind in catalogue.kinds:
|
||||||
|
for path, spec in catalogue.fields.items():
|
||||||
|
if spec.tier_for(kind) != "optional":
|
||||||
|
assert spec.question, (
|
||||||
|
f"{path} is {spec.tier_for(kind)} and has no question"
|
||||||
|
)
|
||||||
@@ -383,3 +383,72 @@ def test_the_cache_setting_resolves_under_the_repository_and_can_be_disabled():
|
|||||||
).terraform_plugin_cache()
|
).terraform_plugin_cache()
|
||||||
== "/var/cache/tf"
|
== "/var/cache/tf"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --- only the providers the bundle pins ---------------------------------------------------
|
||||||
|
|
||||||
|
PINNED = {"cloud": {"source": "example/cloud", "version": "1.2.3"}}
|
||||||
|
|
||||||
|
|
||||||
|
def provider_refusal(code: str, pinned=PINNED) -> str:
|
||||||
|
from app.tools.local import check_providers
|
||||||
|
|
||||||
|
result = check_providers({"terraform/main.tf": code}, pinned)
|
||||||
|
return result.message if result else ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_code_of_the_pinned_provider_alone_is_accepted():
|
||||||
|
code = """
|
||||||
|
terraform {
|
||||||
|
required_providers {
|
||||||
|
cloud = {
|
||||||
|
source = "example/cloud"
|
||||||
|
version = "1.2.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resource "cloud_service" "web" {}
|
||||||
|
data "cloud_project" "this" {}
|
||||||
|
resource "terraform_data" "marker" {}
|
||||||
|
module "network" {
|
||||||
|
source = "example-modules/network/cloud"
|
||||||
|
version = "1.0.0"
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
assert provider_refusal(code) == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_resource_of_a_provider_that_is_not_pinned_is_refused():
|
||||||
|
message = provider_refusal('resource "sourcehost_repository" "infra" {}')
|
||||||
|
|
||||||
|
assert message.startswith("Refused: the code uses resources of the sourcehost")
|
||||||
|
assert "Only these providers may be used: example/cloud" in message
|
||||||
|
|
||||||
|
|
||||||
|
def test_declaring_another_provider_is_refused():
|
||||||
|
declared = """
|
||||||
|
terraform {
|
||||||
|
required_providers {
|
||||||
|
sourcehost = {
|
||||||
|
source = "integrations/sourcehost"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
assert "the provider integrations/sourcehost" in provider_refusal(declared)
|
||||||
|
|
||||||
|
|
||||||
|
def test_with_nothing_pinned_there_is_nothing_to_hold_code_to():
|
||||||
|
assert provider_refusal('resource "anything_at_all" "x" {}', pinned={}) == ""
|
||||||
|
|
||||||
|
|
||||||
|
async def test_validation_refuses_a_foreign_provider_before_terraform_runs():
|
||||||
|
runner = TerraformRunner(binary="no-such-binary", providers=PINNED)
|
||||||
|
|
||||||
|
result = await runner.validate(
|
||||||
|
{"terraform/main.tf": 'resource "sourcehost_repository" "infra" {}'}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.message.startswith("Refused: the code uses")
|
||||||
|
|||||||
@@ -5,8 +5,13 @@ repository: gcp_solution_architecture_agent
|
|||||||
# for the dependency graph. A Phase completes when every Deliverable is submitted and the
|
# for the dependency graph. A Phase completes when every Deliverable is submitted and the
|
||||||
# user grants its Approval. Outputs are written to deliverables/executions/<execution_id>/.
|
# user grants its Approval. Outputs are written to deliverables/executions/<execution_id>/.
|
||||||
phases:
|
phases:
|
||||||
|
- id: scoping
|
||||||
|
input: [workflow_request, scope_spec]
|
||||||
|
deliverables: [scope]
|
||||||
|
attachments: [scope.yaml, scope.json]
|
||||||
|
approval: required
|
||||||
- id: requirements_discovery
|
- id: requirements_discovery
|
||||||
input: workflow_request
|
input: scope
|
||||||
deliverables: [requirements, technical_decomposition]
|
deliverables: [requirements, technical_decomposition]
|
||||||
product_selection_deferred: true
|
product_selection_deferred: true
|
||||||
approval: required
|
approval: required
|
||||||
@@ -32,7 +37,8 @@ phases:
|
|||||||
approval: required
|
approval: required
|
||||||
- id: packaging
|
- id: packaging
|
||||||
input: [validation_results]
|
input: [validation_results]
|
||||||
deliverables: [solution_guide]
|
deliverables: [work_breakdown, solution_guide]
|
||||||
|
attachments: [work-breakdown.json]
|
||||||
output: solution-architecture-guide.md
|
output: solution-architecture-guide.md
|
||||||
approval: required
|
approval: required
|
||||||
verification:
|
verification:
|
||||||
|
|||||||
Reference in New Issue
Block a user