Case 08 failed requirements_name_no_products for "Shared VPC", which the spec's stated topology names. Records cases 03 and 08 on the scoping branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6.5 KiB
Step 2 — Implementation Detail
Built on 2026-10-11. Not run with real models.
- Scoping is the first Phase.
PhaseId.SCOPINGcomes before requirements discovery, with one Deliverable,scope, produced by a new Specialist, the scope analyst (agents/scope-analyst.md). Requirements depend on it. Every Execution now begins with it, whether or not a spec was supplied, and the customer approves it before requirements begin. - The scope analyst completes the spec with two tools (
app/tools/scoping.py).scope_readshows what is settled and what is not, with the question to ask for each.scope_setrecords fields, each with where its value came from:request(the customer's request says it),answered(they said so in conversation) orassumed(the analyst chose, and must say why). Code checks every value against the catalogue as it is recorded. - What the customer stated in the file stays. The analyst cannot overwrite a supplied value by assumption; only the customer's own answer changes it.
- The conversation. With someone present the analyst asks, through
needs_input, for the essential fields and the important ones that most change a design, up to eight at a time. An answer is recorded as answered; "you decide" becomes an assumption; a skipped field stays unknown. With nobody present it assumes the essential and important fields from the catalogue's guidance and what the request implies. - A spec drafted from a sentence. With no spec supplied the analyst starts from an empty one, records what the request states and settles the rest the same way.
- A submission cannot leave an essential field unknown. It is refused, naming the fields.
- The Deliverable is written by code from what was recorded. The analyst writes the scope statement. Code writes the two files (
scope.yaml, the completed spec;scope.json, what every later Specialist reads) and appends the information register, the assumptions with what each would change, and what is still to confirm. - Every later Specialist reads the agreed scope, as
scopein its context: what the customer stated, what was assumed for them (shown apart), and what is unknown. The requirements analyst derives from it and no longer asks for what it settles. - Status is kept current in the Execution's
scope.yamland in the copy beside the Intent:UNDER_REVIEWonce submitted,APPROVEDwhen the Phase is approved,BASELINE_LOCKEDwhen the Execution completes (which closes step 6).execution.jsoncarries the status and the readiness count. - A Change request starts from the scope that was delivered, with where each value came from, and the analyst records only what the change reaches. A Baseline delivered before scoping existed has no scope; the analyst then completes one from the change and the delivered requirements.
- The Approval summary for scoping says how much was assumed and what is unknown.
What it changed elsewhere:
- About 80 existing tests assumed requirements came first.
tests/scoped.pyholds helpers that run scoping ahead of a test's own script, so tests about later Phases start where they always did. Tests about order, counts and first prompts were updated to the new truth. workflow.yamlgains the Phase.
Not done here:
- No real model has run the scope analyst. How well it drafts from a sentence, how many questions it asks, and what it costs are unknown. Every evaluation case now begins with it.
- Answers are parsed by the model, not by code, before
scope_setchecks their type. Nothing reads a value back to the customer other than the draft they approve. - A re-supplied file part-way through a conversation is not handled; the customer answers in chat.
Tests: tests/execution/test_scope_spec.py (12), and the existing suites as updated.
First runs with real models, 2026-10-11
Two cases side by side (20261011T172837Z case 03, 20261011T172840Z case 08). Both completed.
| Case 03, a one-sentence request | Case 08, the landing-zone spec | |
|---|---|---|
| Score | 100% | 91%; the one failure was the evaluation's, now fixed |
| Minutes | 38 | 55 |
| Cost, about | $5.42 | $8.76 |
| Revisions | 1 of the Target architecture | 2 of the Target architecture, 1 of the code |
| The scope analyst | 64 seconds, 12 requests, $0.07 | 96 seconds, 9 requests, $0.08 |
| The scope | 6 stated, 21 assumed | 44 stated, 11 assumed, 4 not applicable |
| The design | 22 Elements, 81 Checks | 49 Elements, 101 Checks, 718 lines of Terraform |
Scoping works and is cheap: about a minute and under ten cents. From one sentence the analyst recorded the six things the sentence says as the customer's and 21 as assumptions, each with a reason; nothing was presented as stated that was not. No essential field was left unknown in either.
The landing zone, the agent's first platform foundation, came out as one: folders, organisation policies, a service perimeter, an interconnect with four attachments, routers, customer-managed keys, workload identity federation for the pipelines, a workforce pool for sign-in, audit sinks. The customer's four requirement ids are each cited in the requirements. Nothing excluded was built: no container workloads. Regions and encryption are as stated. Sixteen questions for the customer were written out.
What it shows is missing:
- A stated choice is not checked. The spec says
shared_vpc_hub_and_spoke. The design names a "Spoke Shared VPC" and builds a transit network and a spoke network, with no Shared VPC host project and nothing connecting hub to spoke. All 101 Checks passed, because no Check asks for the topology the customer stated. "A stated choice binds" is an instruction; nothing in code holds a design to it. This is what step 3 has to close: a stated field that shapes the design must be traceable to a Check. - The code brought in a provider the bundle does not pin. A
github_repositoryElement led tointegrations/githubin the Terraform. Validation downloads whatever provider the code names, as it once did modules. - The scope statement added an exclusion the customer did not state (changes to on-premises systems). The analyst is told to use only what the spec holds.
- The evaluation scored a stated choice as a leaked product. "Shared VPC" in the requirements failed
requirements_name_no_products, though the customer's spec names that topology. The check now counts what the spec states as the customer's.
Case 03's cost and its one Revision are in the range seen before scoping existed; one run cannot say what scoping changed downstream.