feat: establish the Target architecture as a graph before code

Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 13:59:09 +01:00
parent 7ee49ab352
commit e9246828b3
586 changed files with 5166 additions and 22432 deletions

View File

@@ -0,0 +1,14 @@
# Step 1 — Implementation Detail
ADR-0005 was accepted by the owner on 2026-10-07 ("lets move forward with this", after reviewing the proposed case structure), with its three points as written:
1. The user approves the Target architecture before code is written; the solution architecture Phase is split, making five Phases.
2. A new Specialist produces the Target architecture.
3. Code renders the diagram from the graph; the diagrammer role is removed.
ADR-0006 (Change requests) was accepted in the same reply; it is built in iteration 7.
Clarifications the owner gave before accepting, now in the ADR and the role files:
- The graph is the Target architecture itself, not a build plan beside it.
- The user states Intent at a high level in the Workflow request. The agents derive every Element, Relationship and Check from the architecture framework and best practice; the user never writes the graph.

View File

@@ -0,0 +1,12 @@
# Step 10 — Implementation Detail
1. The Orchestrator's context carries `validation`: the outcome of the submitted validation, or null before there is one.
2. `agents/orchestrator.md` step 3, "Validation decided", routes on it:
- failed Checks, or a failed dry-run: reopen the deployment guidance with each failed Check's id and evidence.
- otherwise, not-verifiable Checks or proposed additions: the Target architecture has a gap. Interactive: ask the user whether to revise it. Pre-approved: reopen it only for items whose reason names a requirement or recommendation id.
The routing is the Orchestrator's choice, as ADR-0003 requires; the engine supplies the facts and the Ledger enforces what a reopen does.
On "the list cannot grow": the verdict list is fixed by the graph, because every verdict must be on a Check and every Check must have one. With an unchanged graph, two validation passes cover the same ids by construction (`test_validation_gives_exactly_one_verdict_per_check`).
Not verified: whether a real Orchestrator model follows the new routing. `test_the_orchestrator_is_told_what_validation_decided` covers the context and the Revision, with the reopen scripted.

View File

@@ -0,0 +1,20 @@
# Step 11 — Implementation Detail
Partly done. The checks are built; the evaluation run that this step exists for has not been made.
Built in `eval/scoring.py`:
- `requirements_chain_complete`: the delivered data files form a chain with no gap.
- `every_check_passes`: validation ended with no Check failed or not verifiable, and the dry-run not failed.
- `diagram_matches_graph`: the delivered diagram is exactly the rendering of the delivered graph.
These replaced `components_trace_to_requirements` and `no_blocking_findings` from iteration 4 step 8.
Outstanding, and why:
- **No evaluation run.** The owner stopped the last run on 2026-10-06 and asked that no further cases start until they advise. Nothing in this iteration has run against a real model.
- **The acceptance criteria are therefore unmet:** all cases scoring at least 0.8, the failing-Check set never growing between passes, and at most one Revision of the deployment guidance in two of three cases.
- **Not built:** the per-case record of validation passes and the Check ids failing at each. The evaluation reads only the final verdicts today.
- **Thresholds** from iteration 4 step 8 are still unset.
Do not mark this step complete until a run meets the criteria. (`legacy-baseline` was removed on 2026-10-07 on the owner's instruction, ahead of this.)

View File

@@ -0,0 +1,17 @@
# Step 2 — Implementation Detail
1. `app/workflows/chain.py` defines the data files and a checker for each. Each checker returns one entry per fault, so a Specialist can fix and resubmit.
2. New required attachments (`DATA_CHECKS` in `app/workflows/deliverables.py`):
- `requirements.json` on requirements: `{id, kind, text}`, kind functional or non_functional, ids unique.
- `components.json` on the technical decomposition: `{id, name, requirements}`. This replaced the component table contract from iteration 4 step 4.
- `products.json` on product selection: `{component, product}`, one for every component.
- `recommendations.json` on design recommendations: `{id, pillar, text, components}`.
3. `check_submission` takes the Deliverables being built on (`upstream`), so links are checked across Deliverables: a component may only serve a real requirement, a product may only be selected for a real component, a recommendation may only name real components.
4. The formats live in a new Skill, `skills/requirements-chain/SKILL.md`, assigned in `providers/gcp.yaml` to every role that writes or reads one.
Changed from the plan:
- `products.json` was added. The plan's rule "a selected product appears on no Element" needs the selection as data.
- Only functional requirements must be served by a component at the decomposition. A non-functional requirement often constrains the whole workload, so it is held to a stricter rule later: it must be the source of a Check (step 5).
Tests: `tests/execution/test_requirements_chain.py`, "the links before the graph".

View File

@@ -0,0 +1,8 @@
# Step 3 — Implementation Detail
1. `app/workflows/target_architecture.py`, pure and without I/O: `Element`, `Relationship`, `Check`, `NotApplicable`, `TargetArchitecture`, and `parse_graph`, which returns the graph or raises `GraphError` listing every fault.
2. The file is `target-architecture.json` with `elements`, `relationships`, `checks` and optional `not_applicable`.
3. Refused, with the id named: a repeated id; a Relationship whose `from` or `to` is not an Element; a Check that applies to nothing in the graph; an Element with no Check; a missing or empty field.
4. An Element's `component` is optional, for supporting Elements such as identities, keys and networks.
Tests: `tests/execution/test_requirements_chain.py`, "the graph is well-formed".

View File

@@ -0,0 +1,8 @@
# Step 4 — Implementation Detail
1. `DeliverableKind.TARGET_ARCHITECTURE`, in the design Phase, depending on product selection and design recommendations. Its content needs `Elements`, `Relationships` and `Checks` sections; its attachment is the graph.
2. Role file `agents/target-architect.md`. In the bundle it loads the solution-architecture and requirements-chain Skills, reaches the documentation MCP server, and runs on the Pro model tier.
3. `skills/requirements-chain/SKILL.md` gives the graph's format and how to write a Check that can be verified from code, with verifiable and non-verifiable examples.
4. The role file states that implementation detail is the agent's job and the user stays at the level of Intent.
Not verified: no real model has produced a Target architecture yet. Whether one can write Checks specific enough to verify, in one run and inside its tool budget, is the main risk of this iteration and is what step 11 measures.

View File

@@ -0,0 +1,18 @@
# Step 5 — Implementation Detail
`chain_gaps` in `app/workflows/target_architecture.py` lists every place the Requirements chain stops; `check_target_architecture` runs it at submission, so a Target architecture with a gap is refused with each gap named.
A gap is:
- a component with no Element;
- a non-functional requirement that is the source of no Check;
- a functional requirement that is the source of no Check and is served by no component with an Element;
- a selected product that does not appear on its component's Element;
- a recommendation that is the source of no Check and is not listed under `not_applicable` with a reason;
- a Check whose `source` is not a requirement or recommendation, or an Element that realises something that is not a component.
Recommendations marked not applicable are carried in the graph's summary and shown to the user at the design Approval.
The rule checks that each link exists, not that it is right. A vague Check passes here and is caught later as `not_verifiable`, or by the user at Approval.
Tests: `tests/execution/test_requirements_chain.py`, "the Requirements chain has no gap", including one through `submit_deliverable`.

View File

@@ -0,0 +1,10 @@
# Step 6 — Implementation Detail
1. `PhaseId` has five members. `solution_architecture` is now the design Phase (product selection, design recommendations, Target architecture). The new `solution_implementation` holds the architecture description and the deployment guidance. The id `solution_architecture` was kept so recorded Executions and callers do not break.
2. The Ledger's existing rule does the gating: a Dependency across Phases is met only on Approval, so nothing in implementation is ready until design is approved.
3. The design Approval carries a summary of the graph (counts of Elements, Relationships and Checks, and recommendations treated as not applicable). `describe_prompt` in `app/presentation.py` states that no code is written until it is approved.
4. `workflow.yaml` lists the five Phases; its test checks it against the dependency graph. The A2A card derives from the graph and needed no change.
Revision behaviour that follows: reopening the deployment guidance withdraws the implementation and validation Approvals and leaves the design approved; reopening the Target architecture withdraws design, implementation and validation, and the code cannot restart until the design is approved again.
Tests: `tests/execution/test_ledger.py`, `test_the_user_approves_the_graph_before_any_code_is_written` in `tests/execution/test_requirements_chain.py`, `tests/execution/test_presentation.py`.

View File

@@ -0,0 +1,10 @@
# Step 7 — Implementation Detail
1. `render_mermaid` draws a left-to-right flowchart with every Element (name and product) and every Relationship (its kind) exactly once. Node ids are generated, and characters Mermaid reserves are stripped from labels, so any graph step 3 accepts renders.
2. `submit_deliverable` adds `architecture.mmd` to the Target architecture's attachments when it accepts the submission. No Specialist draws it.
3. Removed: the `architecture_diagram` Deliverable, `agents/diagrammer.md` and its bundle entries.
4. `/generate` keeps its `mermaid_diagram` key, now read from the Target architecture. The packager is told to embed that attachment unchanged.
The local `mermaid_check` tool stays in the bundle; the evaluation still uses it.
Tests: `tests/execution/test_requirements_chain.py`, "the diagram is a view of the graph".

View File

@@ -0,0 +1,9 @@
# Step 8 — Implementation Detail
1. The deployment guidance depends on the Target architecture and requires `implementation.json`: for each Check, the attached file and resource address that satisfies it.
2. Refused: a Check not accounted for, an id that is not a Check, an entry naming a file that is not attached.
3. `agents/iac-author.md` now says to implement the Target architecture and nothing it does not ask for, and on a Revision to change only what the failed Checks need.
Changed from the plan: the IaC author's context still carries every earlier Deliverable in full, prose recommendations included, because `inputs_of` is unchanged. Narrowing it to the graph is iteration 4 step 13.
Tests: `test_deployment_guidance_must_account_for_every_check`.

View File

@@ -0,0 +1,12 @@
# Step 9 — Implementation Detail
1. The validation results require `verdicts.json` in place of `findings.json`: a `dry_run` result, one verdict per Check (`pass`, `fail` or `not_verifiable`) with evidence, and a separate list of `proposed_additions`.
2. Refused: a Check with no verdict, a Check with two, a verdict on something that is not a Check, a verdict without evidence, a missing dry-run result.
3. A proposed addition is never a failure. This is the rule that stops validation findings growing between passes: the validator cannot add to what the code is judged against.
4. `validation_outcome` in `app/workflows/chain.py` separates faults in the code (failed Checks) from gaps in the graph (not verifiable, proposed additions). Lesson candidates and the evaluation use it.
Changed from the plan: the dry-run is its own `dry_run` entry, not a Check in the graph, so the Target architecture stays free of tooling concerns.
`check_findings` and `FINDINGS_ATTACHMENT` from iteration 4 step 1 are gone. Lesson candidate types are now `failed_check`, `unverifiable_check` and `proposed_addition`.
Tests: `tests/execution/test_requirements_chain.py`, "code and validation answer the graph"; `tests/execution/test_lessons.py`.