Files
gcp_solution_architecture_a…/.agents/plans/iter-09-scoping/steps/step-2.md
Jonathan Boniface c26b744b26 fix: a product the customer's scope spec names is not a leak; record the first scoping runs
Case 08 failed requirements_name_no_products for "Shared VPC", which the
spec's stated topology names. Records cases 03 and 08 on the scoping branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 19:27:51 +01:00

6.5 KiB

Step 2 — Implementation Detail

Built on 2026-10-11. Not run with real models.

  1. Scoping is the first Phase. PhaseId.SCOPING comes before requirements discovery, with one Deliverable, scope, produced by a new Specialist, the scope analyst (agents/scope-analyst.md). Requirements depend on it. Every Execution now begins with it, whether or not a spec was supplied, and the customer approves it before requirements begin.
  2. The scope analyst completes the spec with two tools (app/tools/scoping.py). scope_read shows what is settled and what is not, with the question to ask for each. scope_set records fields, each with where its value came from: request (the customer's request says it), answered (they said so in conversation) or assumed (the analyst chose, and must say why). Code checks every value against the catalogue as it is recorded.
  3. What the customer stated in the file stays. The analyst cannot overwrite a supplied value by assumption; only the customer's own answer changes it.
  4. The conversation. With someone present the analyst asks, through needs_input, for the essential fields and the important ones that most change a design, up to eight at a time. An answer is recorded as answered; "you decide" becomes an assumption; a skipped field stays unknown. With nobody present it assumes the essential and important fields from the catalogue's guidance and what the request implies.
  5. A spec drafted from a sentence. With no spec supplied the analyst starts from an empty one, records what the request states and settles the rest the same way.
  6. A submission cannot leave an essential field unknown. It is refused, naming the fields.
  7. The Deliverable is written by code from what was recorded. The analyst writes the scope statement. Code writes the two files (scope.yaml, the completed spec; scope.json, what every later Specialist reads) and appends the information register, the assumptions with what each would change, and what is still to confirm.
  8. Every later Specialist reads the agreed scope, as scope in its context: what the customer stated, what was assumed for them (shown apart), and what is unknown. The requirements analyst derives from it and no longer asks for what it settles.
  9. Status is kept current in the Execution's scope.yaml and in the copy beside the Intent: UNDER_REVIEW once submitted, APPROVED when the Phase is approved, BASELINE_LOCKED when the Execution completes (which closes step 6). execution.json carries the status and the readiness count.
  10. A Change request starts from the scope that was delivered, with where each value came from, and the analyst records only what the change reaches. A Baseline delivered before scoping existed has no scope; the analyst then completes one from the change and the delivered requirements.
  11. The Approval summary for scoping says how much was assumed and what is unknown.

What it changed elsewhere:

  • About 80 existing tests assumed requirements came first. tests/scoped.py holds helpers that run scoping ahead of a test's own script, so tests about later Phases start where they always did. Tests about order, counts and first prompts were updated to the new truth.
  • workflow.yaml gains the Phase.

Not done here:

  • No real model has run the scope analyst. How well it drafts from a sentence, how many questions it asks, and what it costs are unknown. Every evaluation case now begins with it.
  • Answers are parsed by the model, not by code, before scope_set checks their type. Nothing reads a value back to the customer other than the draft they approve.
  • A re-supplied file part-way through a conversation is not handled; the customer answers in chat.

Tests: tests/execution/test_scope_spec.py (12), and the existing suites as updated.

First runs with real models, 2026-10-11

Two cases side by side (20261011T172837Z case 03, 20261011T172840Z case 08). Both completed.

Case 03, a one-sentence request Case 08, the landing-zone spec
Score 100% 91%; the one failure was the evaluation's, now fixed
Minutes 38 55
Cost, about $5.42 $8.76
Revisions 1 of the Target architecture 2 of the Target architecture, 1 of the code
The scope analyst 64 seconds, 12 requests, $0.07 96 seconds, 9 requests, $0.08
The scope 6 stated, 21 assumed 44 stated, 11 assumed, 4 not applicable
The design 22 Elements, 81 Checks 49 Elements, 101 Checks, 718 lines of Terraform

Scoping works and is cheap: about a minute and under ten cents. From one sentence the analyst recorded the six things the sentence says as the customer's and 21 as assumptions, each with a reason; nothing was presented as stated that was not. No essential field was left unknown in either.

The landing zone, the agent's first platform foundation, came out as one: folders, organisation policies, a service perimeter, an interconnect with four attachments, routers, customer-managed keys, workload identity federation for the pipelines, a workforce pool for sign-in, audit sinks. The customer's four requirement ids are each cited in the requirements. Nothing excluded was built: no container workloads. Regions and encryption are as stated. Sixteen questions for the customer were written out.

What it shows is missing:

  • A stated choice is not checked. The spec says shared_vpc_hub_and_spoke. The design names a "Spoke Shared VPC" and builds a transit network and a spoke network, with no Shared VPC host project and nothing connecting hub to spoke. All 101 Checks passed, because no Check asks for the topology the customer stated. "A stated choice binds" is an instruction; nothing in code holds a design to it. This is what step 3 has to close: a stated field that shapes the design must be traceable to a Check.
  • The code brought in a provider the bundle does not pin. A github_repository Element led to integrations/github in the Terraform. Validation downloads whatever provider the code names, as it once did modules.
  • The scope statement added an exclusion the customer did not state (changes to on-premises systems). The analyst is told to use only what the spec holds.
  • The evaluation scored a stated choice as a leaked product. "Shared VPC" in the requirements failed requirements_name_no_products, though the customer's spec names that topology. The check now counts what the spec states as the customer's.

Case 03's cost and its one Revision are in the range seen before scoping existed; one run cannot say what scoping changed downstream.