Every Execution now begins by agreeing the scope, before requirements.
- A new Phase and Deliverable, produced by a scope analyst with two tools:
scope_read shows what is settled and what is not; scope_set records fields
with where each value came from (the request, the customer's answer, or an
assumption with its reason). Code checks each value against the catalogue.
- With someone present the analyst asks for what is missing; with nobody
present it assumes it, visibly. With no spec it drafts one from the request.
A value the customer stated in the file is changed only by their own answer.
- A submission cannot leave an essential field unknown. Code writes the
completed spec, the data every later Specialist reads, and the information
register, from what was recorded.
- The spec's status follows the Execution: under review, approved, then
locked as a Baseline. A Change request starts from the delivered scope.
- The requirements analyst derives from the agreed scope and no longer asks
for what it settles.
About 80 tests assumed requirements came first; tests/scoped.py runs scoping
ahead of a test's own script, and tests about order and counts are updated.
Not run with real models.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pillar's checklist item the request gives no information about was marked
"not addressed", which reads as a fault in the design. It is missing
information. The checklist review gains a fourth outcome, to confirm with the
customer, which must carry the question to ask. The questions are written
into the design recommendations and the guide, and to customer-questions.md;
the evaluation's judged check counts such an item as accounted for.
Records the owner's decisions of 2026-10-10 and the vendor's answers on Jev's
retention, training, context and rate limits (iteration 5 step 1), and the
owner's labels, which showed the judged question was wrong, not the judges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A graded question per pillar (not at all, only indirectly, directly) picks
the pillar advisors to consult again on a Revision or a Change request; the
design advisor is told which, and keeps its earlier advice for the rest. As
a yes or no the answer was yes for nearly every pillar of every change.
A completed Execution now writes working.json with each pillar's advice
and checklist review, and a Change request starts from its Baseline's, so
only the pillars a change touches are consulted afresh.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Case 04 (interactive)
- A scripted answer may name who it is for, so facts written for
requirements discovery are not given to a Specialist confirming a draft.
- A reply that restates what the agreed requirements already say is read
as an answer, not a request for changes.
- With a user present, a first submission is refused until the user has
been shown a draft of that Deliverable (CONFIRM_DRAFTS).
Routing and reviews
- Intake: a message that starts something is read first. A question is
answered and starts nothing; a change to delivered work starts as a new
request with a note, until Change requests are built.
- Requirements and recommendations list what infrastructure code can show
(conditions) apart from what it cannot (outside_code); the latter needs
no Check and validation proposes nothing for it.
- A requirements review, "names no products", behind its own switch (off).
- Each pillar advisor reviews its pillar's checklist; the engine writes the
reviews into the design recommendations. The judged checklist check went
from 9-45% a pillar to 100% in one run of case 04.
Evaluation
- Per-case thresholds for Revisions, seconds and tokens, set from every
completed run on this build.
- A progress file after every agent run, so an interrupted run keeps its
time and tokens.
- eval/calibrate_judge.py with a 32-question set awaiting a person's
labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91.
Smaller
- An Execution that ends with warnings writes its Deliverables, with the
warnings and each status in the manifest.
- CI installs the application's dependencies before linting and testing.
- A load-balancer pattern in the IaC patterns Skill.
Plans record nine runs of 2026-10-08. Iteration 6 is complete.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.
Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/
Unproven with a real model: no evaluation run has completed on this design.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Increment 6 of .scratch/solution-architect-agent/spec.md, test-first.
- app/tools/mcp.py: MCP client and catalog exposing only allowlisted,
readOnlyHint tools as framework-neutral ToolSpecs; unreachable servers
and failed calls become discovery gaps; ADC auth with quota project.
- app/tools/local.py: Mermaid structure check and Terraform fmt,
validate and plan in a scratch copy; apply is never constructed.
- Tool factory gives roles their named local tools and bundle MCP
servers; pillar advisors get general-tier documentation tools.
- providers/gcp.yaml: 42 allowlisted tools reviewed against live
tools/list of 12 Google Cloud MCP servers (snapshot in
providers/reviews/), excluding data-reading and secret-capable tools.
- scripts/list-mcp-tools.py, docs/operations/mcp-access.md (IAM deny
policy, service account roles), cloud-discovery and iac-validation
Skills, MCP and Terraform settings, pytest asyncio auto mode.
- 31 new tests; full suite 189 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Increment 5 of .scratch/solution-architect-agent/spec.md, test-first.
- agents/*.md: provider-neutral role files for the Orchestrator, nine
Specialists and the pillar advisor, with a shared Specialist protocol.
- app/agents/roles.py: load and validate role files; Skills, MCP
servers and models come from the provider bundle.
- app/agents/adk_runtime.py: AgentRuntime on google-adk 2.9 with a
per-run LlmAgent, role-scoped SkillToolset (no run_skill_script),
step budgets, early stop on turn-ending tools, LiteLLM models.
- app/execution/pillars.py + toolkit.py: design advisor fans out to one
advisor per WAF pillar in parallel.
- Specialists receive every earlier-Phase Deliverable as input.
- Bundle gains role_skills, pillar_skills and -latest Gemini aliases;
pin google-adk[extensions]==2.9.0.
- 37 new tests; full suite 158 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>