feat: Change requests against a Baseline (ADR-0006)

A caller restates a request to say how requirements have changed, naming a
completed Execution of the same Customer case as the Baseline.

- app/baseline.py reads a Baseline from its files and refuses one that
  does not exist, did not complete or belongs to another case, before
  anything runs or is recorded. A Baseline is never modified.
- Requirements carry a change (unchanged, added, changed, removed) and a
  reason. Submission is refused when a Baseline requirement disappears, an
  id is reused, or an unchanged requirement's text differs. A removed
  requirement stays on record and asks nothing downstream.
- Each Specialist starts from its delivered version; the graph and the code
  are its working copy, and the description is carried over when the
  design's shape is the same.
- app/workflows/changes.py compares versions by id and traces what each
  changed requirement reaches; anything else that changed is listed as
  unrequested. Approvals show the change for their Phase, and changes.json
  is written with the full Deliverables.
- Intake no longer starts a message read as a change: it names the
  Execution to give as the Baseline.
- A fifth evaluation case runs a request and then a change to it, with
  four checks that the change is recorded, reaches Checks, stays contained
  and costs less than its Baseline.

Tested with scripted agents only; no Change request has run with real
models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 11:38:15 +01:00
parent db28d02f56
commit 5ef434d00e
32 changed files with 1824 additions and 40 deletions

View File

@@ -2,12 +2,12 @@
**Iteration:** 7, Change requests
**Intent:** Let a user restate a request to say how their requirements have changed, at a high level, and receive the delivered architecture updated from where it stood, with an account of what changed and why.
**Status:** planned. ADR-0006 was accepted on 2026-10-07 and the glossary updated (step 1). Build after iteration 6 step 11 has a passing evaluation run
**Status:** steps 1 to 6 and 8 built (2026-10-09), covered by the deterministic suite; no Change request has run with real models. Step 7's evaluation case and checks are built and its run is outstanding
**This directory covers:** steps 1 to 8
## Sources of truth
- `docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md`: the decision and its four points to confirm. Status `proposed`.
- `docs/adr/0006-a-change-request-starts-from-a-baseline-execution.md`: the decision and its four points. Accepted 2026-10-07.
- `docs/adr/0005-the-target-architecture-is-an-explicit-graph.md` and `../iter-06-target-architecture-graph/`: the ids along the Requirements chain that a change is traced through. Build iteration 6 first.
- `CONTEXT.md`: today it says an Execution starts clean and a Customer case never feeds one Execution into another. Step 1 changes those entries; until then they stand.
- `iter-04-learning-and-skills/steps/step-15.md`: the Customer case layout, `execution.json` and the Intent history (`intent/history.json`) that a Baseline is found through. A Change request is the next statement in that history.
@@ -17,12 +17,12 @@
| Step | What | Size | Status |
|---|---|---|---|
| 1 | Accept ADR-0006 and update the glossary | S | complete: accepted 2026-10-07 |
| 2 | Start an Execution against a Baseline | M | planned |
| 3 | Requirements revised with a change record | M | planned |
| 4 | Later Specialists revise from the Baseline | M | planned |
| 5 | Approvals show the change | S | planned |
| 6 | Change summary and version line in the case | S | planned |
| 7 | Evaluation: a change case | M | planned |
| 2 | Start an Execution against a Baseline | M | built; not run with real models |
| 3 | Requirements revised with a change record | M | built; not run with real models |
| 4 | Later Specialists revise from the Baseline | M | built; not run with real models |
| 5 | Approvals show the change | S | built; not run with real models |
| 6 | Change summary and version line in the case | S | built; not run with real models |
| 7 | Evaluation: a change case | M | case and checks built; run outstanding |
| 8 | Remove the `legacy-baseline` case | S | complete: removed early on the owner's instruction, 2026-10-07 |
## Dependency graph

View File

@@ -0,0 +1,14 @@
# Step 2 — Implementation Detail
Built on 2026-10-09.
1. `app/baseline.py`: `load_baseline(output_root, customer_case, execution_id)` reads a Baseline from its files under the Customer case, through `execution.json`: its id, its request, the Baseline it was itself changed from, and every Deliverable with its attachments. It never needs the Baseline's checkpoint.
2. It refuses, with `BaselineError` (a `ValueError`), an id that is not an Execution id, an Execution with no files in that case, one that belongs to another Customer case (without saying which), one whose manifest status is not `completed`, and a Change request with no Customer case.
3. `ExecutionEngine.start(baseline=...)` loads the Baseline before the Execution's id is taken or its intent recorded, so a refusal leaves nothing behind. The loaded Baseline is in the state as `baseline`.
4. `record_intent` puts the Baseline on the new statement in `intent/history.json` and in its `NNN.md`; `execution.json` records it too. An Execution started without one has `"baseline": null` in both.
5. Callers: `baseline` in the `/generate` body and in the A2A message metadata. A bad Baseline is a 400 from `/generate` and a failed task over A2A.
6. Intake (iteration 5 step 12) no longer starts a message read as a change. It answers with the latest completed Execution of the case and how to name it as the Baseline, and starts nothing; `"new_request": true` in the metadata says it is new work and skips the reading. A caller who names a Baseline is not read at all.
Differs from the plan: nothing.
Tests: `tests/execution/test_change_requests.py` "step 2" (10), `tests/a2a/test_server.py` (3), `tests/execution/test_intake.py`.

View File

@@ -0,0 +1,17 @@
# Step 3 — Implementation Detail
Built on 2026-10-09.
1. In a Change request every requirement in `requirements.json` carries a `change`: `unchanged`, `added`, `changed` or `removed`. Everything but `unchanged` carries a `reason`.
2. `check_requirement_changes` in `app/workflows/changes.py`, run by `check_submission` when the Deliverable has a Baseline, refuses:
- a requirement with no `change`, or a change with no reason;
- a Baseline requirement missing from the list (it must stay, marked `removed`);
- a requirement marked `added` whose id the Baseline already has, or marked anything else whose id it does not;
- a requirement marked `unchanged` whose text differs from the Baseline's;
- a list in which nothing is added, changed or removed.
3. A removed requirement stays in the file as the record of what was dropped. `read_list` leaves it out, so nothing downstream is asked to serve or check it.
4. `agents/requirements-analyst.md` says how to mark each requirement and to add a `Changes` heading. With a user present the draft is confirmed before submission by the rule of iteration 4 step 27.
Differs from the plan: the check that an `unchanged` requirement keeps its text was added. Without it a requirement could change quietly.
Tests: `tests/execution/test_change_requests.py` "step 3" (6).

View File

@@ -0,0 +1,14 @@
# Step 4 — Implementation Detail
Built on 2026-10-09.
1. Each Specialist's context has `baseline` in a Change request: the Baseline's id and original request, the Baseline's version of its own Deliverable (not for the solution guide, which is assembled afresh), and `changes`, what has changed so far upstream.
2. Working copies start from the Baseline (`_start_from` in `app/nodes/execution_nodes.py`): the target architect's working graph is the delivered graph, and the IaC author's working copy is the delivered code. A Deliverable with `written_from` is seeded as standing, so the architecture description is carried over without running the describer when the Elements and Relationships are the same as the Baseline's.
3. `agents/partials/specialist-protocol.md` tells every Specialist how to revise: start from the delivered version, change only what the changed requirements reach, keep every id that still holds, never renumber.
4. `app/workflows/changes.py` compares two versions by id (`change_summary`) and traces `reach`: a changed requirement reaches the components that serve it, the Elements that realise them, the Relationships at those Elements, and every Check on any of them or with the requirement as its source; a changed recommendation reaches the Checks that come from it. Reach is traced through both versions. Anything that changed outside it is listed under `unrequested`.
Differs from the plan: the plan's test was that an unreached Element or Check is "identical to the Baseline's". The engine cannot make an agent leave something alone; it reports what changed outside the reach, and step 7's check fails a run where much did.
**Not run with real models**, so whether Specialists do keep ids and leave unreached parts alone is unknown.
Tests: `tests/execution/test_change_requests.py` "step 4" and "what changed" (6).

View File

@@ -0,0 +1,9 @@
# Step 5 — Implementation Detail
Built on 2026-10-09.
1. `OrchestratorToolbox` is given the Baseline. In a Change request `request_approval` adds `baseline` and `changes` to the Approval: the part of the change summary that belongs to that Phase's Deliverables (`changes_for`), computed from the ids and never written by an agent.
2. `describe_prompt` in `app/presentation.py` renders it: each requirement added, changed or removed with its reason; ids added, changed and removed for components, recommendations, Elements, Relationships and Checks; and, apart, anything changed that no changed requirement reaches. A Phase that changed nothing says so.
3. An Approval outside a Change request has no `changes` key.
Tests: `tests/execution/test_change_requests.py` "step 5" (2).

View File

@@ -0,0 +1,12 @@
# Step 6 — Implementation Detail
Built on 2026-10-09.
1. A Change request writes `changes.json` beside its Deliverables: the requirements' own account, then components, recommendations, Elements, Relationships and Checks added, changed and removed by id, and `unrequested`. `execution.json` names it under `changes` and names the Baseline.
2. Every Deliverable is written in full, as for any Execution, so the latest Execution of a case stands on its own.
3. `intent/history.json` lists each statement with its Baseline (step 2), so a case's Executions read as a line of versions.
4. `agents/packager.md`: the guide's executive summary opens with what changed and why, and names unrequested changes separately.
Differs from the plan: `changes.json` records ids, and reasons for requirements only. The plan asked for a reason on every item; a reason for a changed Check is whichever changed requirement or recommendation is its source, which a reader can follow from the ids.
Tests: `tests/execution/test_change_requests.py` "step 6" (1).

View File

@@ -0,0 +1,16 @@
# Step 7 — Implementation Detail
Built on 2026-10-09: the case and its checks. **The run is outstanding.**
1. A case may carry `change`, a statement of what has changed. The evaluation then runs the case's request to completion, pre-approved, and runs `change` as a Change request against it in the same Customer case. The Change request is what is scored.
2. `case-05-change-retention`: case 01's request, then "raw events must now be kept for 7 years instead of 30 days ... and must not be deletable during that time. Everything else stays as it is."
3. `score_change` adds four checks to the usual ones:
- `change_is_recorded`: at least one requirement is marked added, changed or removed;
- `changed_requirements_reach_checks`: each added or changed requirement is the source of a Check that was added or changed, and a removed one's Checks are gone;
- `change_is_contained`: no more than two items, or a tenth of the Baseline's graph, changed without being reached;
- `change_costs_less_than_baseline`: the Change request used fewer tokens than the Execution it changed.
4. The report records the Baseline's id, seconds and tokens.
Acceptance criteria: not yet tested. The case needs two Executions, roughly an hour and $8 to $10.
Tests: `tests/eval/test_scoring.py`, "a Change request stays contained" (5).