feat: Change requests against a Baseline (ADR-0006)
A caller restates a request to say how requirements have changed, naming a completed Execution of the same Customer case as the Baseline. - app/baseline.py reads a Baseline from its files and refuses one that does not exist, did not complete or belongs to another case, before anything runs or is recorded. A Baseline is never modified. - Requirements carry a change (unchanged, added, changed, removed) and a reason. Submission is refused when a Baseline requirement disappears, an id is reused, or an unchanged requirement's text differs. A removed requirement stays on record and asks nothing downstream. - Each Specialist starts from its delivered version; the graph and the code are its working copy, and the description is carried over when the design's shape is the same. - app/workflows/changes.py compares versions by id and traces what each changed requirement reaches; anything else that changed is listed as unrequested. Approvals show the change for their Phase, and changes.json is written with the full Deliverables. - Intake no longer starts a message read as a change: it names the Execution to give as the Baseline. - A fifth evaluation case runs a request and then a change to it, with four checks that the change is recorded, reaches Checks, stays contained and costs less than its Baseline. Tested with scripted agents only; no Change request has run with real models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
14
.agents/plans/iter-07-change-requests/steps/step-2.md
Normal file
14
.agents/plans/iter-07-change-requests/steps/step-2.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Step 2 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09.
|
||||
|
||||
1. `app/baseline.py`: `load_baseline(output_root, customer_case, execution_id)` reads a Baseline from its files under the Customer case, through `execution.json`: its id, its request, the Baseline it was itself changed from, and every Deliverable with its attachments. It never needs the Baseline's checkpoint.
|
||||
2. It refuses, with `BaselineError` (a `ValueError`), an id that is not an Execution id, an Execution with no files in that case, one that belongs to another Customer case (without saying which), one whose manifest status is not `completed`, and a Change request with no Customer case.
|
||||
3. `ExecutionEngine.start(baseline=...)` loads the Baseline before the Execution's id is taken or its intent recorded, so a refusal leaves nothing behind. The loaded Baseline is in the state as `baseline`.
|
||||
4. `record_intent` puts the Baseline on the new statement in `intent/history.json` and in its `NNN.md`; `execution.json` records it too. An Execution started without one has `"baseline": null` in both.
|
||||
5. Callers: `baseline` in the `/generate` body and in the A2A message metadata. A bad Baseline is a 400 from `/generate` and a failed task over A2A.
|
||||
6. Intake (iteration 5 step 12) no longer starts a message read as a change. It answers with the latest completed Execution of the case and how to name it as the Baseline, and starts nothing; `"new_request": true` in the metadata says it is new work and skips the reading. A caller who names a Baseline is not read at all.
|
||||
|
||||
Differs from the plan: nothing.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py` "step 2" (10), `tests/a2a/test_server.py` (3), `tests/execution/test_intake.py`.
|
||||
17
.agents/plans/iter-07-change-requests/steps/step-3.md
Normal file
17
.agents/plans/iter-07-change-requests/steps/step-3.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Step 3 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09.
|
||||
|
||||
1. In a Change request every requirement in `requirements.json` carries a `change`: `unchanged`, `added`, `changed` or `removed`. Everything but `unchanged` carries a `reason`.
|
||||
2. `check_requirement_changes` in `app/workflows/changes.py`, run by `check_submission` when the Deliverable has a Baseline, refuses:
|
||||
- a requirement with no `change`, or a change with no reason;
|
||||
- a Baseline requirement missing from the list (it must stay, marked `removed`);
|
||||
- a requirement marked `added` whose id the Baseline already has, or marked anything else whose id it does not;
|
||||
- a requirement marked `unchanged` whose text differs from the Baseline's;
|
||||
- a list in which nothing is added, changed or removed.
|
||||
3. A removed requirement stays in the file as the record of what was dropped. `read_list` leaves it out, so nothing downstream is asked to serve or check it.
|
||||
4. `agents/requirements-analyst.md` says how to mark each requirement and to add a `Changes` heading. With a user present the draft is confirmed before submission by the rule of iteration 4 step 27.
|
||||
|
||||
Differs from the plan: the check that an `unchanged` requirement keeps its text was added. Without it a requirement could change quietly.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py` "step 3" (6).
|
||||
14
.agents/plans/iter-07-change-requests/steps/step-4.md
Normal file
14
.agents/plans/iter-07-change-requests/steps/step-4.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Step 4 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09.
|
||||
|
||||
1. Each Specialist's context has `baseline` in a Change request: the Baseline's id and original request, the Baseline's version of its own Deliverable (not for the solution guide, which is assembled afresh), and `changes`, what has changed so far upstream.
|
||||
2. Working copies start from the Baseline (`_start_from` in `app/nodes/execution_nodes.py`): the target architect's working graph is the delivered graph, and the IaC author's working copy is the delivered code. A Deliverable with `written_from` is seeded as standing, so the architecture description is carried over without running the describer when the Elements and Relationships are the same as the Baseline's.
|
||||
3. `agents/partials/specialist-protocol.md` tells every Specialist how to revise: start from the delivered version, change only what the changed requirements reach, keep every id that still holds, never renumber.
|
||||
4. `app/workflows/changes.py` compares two versions by id (`change_summary`) and traces `reach`: a changed requirement reaches the components that serve it, the Elements that realise them, the Relationships at those Elements, and every Check on any of them or with the requirement as its source; a changed recommendation reaches the Checks that come from it. Reach is traced through both versions. Anything that changed outside it is listed under `unrequested`.
|
||||
|
||||
Differs from the plan: the plan's test was that an unreached Element or Check is "identical to the Baseline's". The engine cannot make an agent leave something alone; it reports what changed outside the reach, and step 7's check fails a run where much did.
|
||||
|
||||
**Not run with real models**, so whether Specialists do keep ids and leave unreached parts alone is unknown.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py` "step 4" and "what changed" (6).
|
||||
9
.agents/plans/iter-07-change-requests/steps/step-5.md
Normal file
9
.agents/plans/iter-07-change-requests/steps/step-5.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Step 5 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09.
|
||||
|
||||
1. `OrchestratorToolbox` is given the Baseline. In a Change request `request_approval` adds `baseline` and `changes` to the Approval: the part of the change summary that belongs to that Phase's Deliverables (`changes_for`), computed from the ids and never written by an agent.
|
||||
2. `describe_prompt` in `app/presentation.py` renders it: each requirement added, changed or removed with its reason; ids added, changed and removed for components, recommendations, Elements, Relationships and Checks; and, apart, anything changed that no changed requirement reaches. A Phase that changed nothing says so.
|
||||
3. An Approval outside a Change request has no `changes` key.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py` "step 5" (2).
|
||||
12
.agents/plans/iter-07-change-requests/steps/step-6.md
Normal file
12
.agents/plans/iter-07-change-requests/steps/step-6.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Step 6 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09.
|
||||
|
||||
1. A Change request writes `changes.json` beside its Deliverables: the requirements' own account, then components, recommendations, Elements, Relationships and Checks added, changed and removed by id, and `unrequested`. `execution.json` names it under `changes` and names the Baseline.
|
||||
2. Every Deliverable is written in full, as for any Execution, so the latest Execution of a case stands on its own.
|
||||
3. `intent/history.json` lists each statement with its Baseline (step 2), so a case's Executions read as a line of versions.
|
||||
4. `agents/packager.md`: the guide's executive summary opens with what changed and why, and names unrequested changes separately.
|
||||
|
||||
Differs from the plan: `changes.json` records ids, and reasons for requirements only. The plan asked for a reason on every item; a reason for a changed Check is whichever changed requirement or recommendation is its source, which a reader can follow from the ids.
|
||||
|
||||
Tests: `tests/execution/test_change_requests.py` "step 6" (1).
|
||||
16
.agents/plans/iter-07-change-requests/steps/step-7.md
Normal file
16
.agents/plans/iter-07-change-requests/steps/step-7.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# Step 7 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09: the case and its checks. **The run is outstanding.**
|
||||
|
||||
1. A case may carry `change`, a statement of what has changed. The evaluation then runs the case's request to completion, pre-approved, and runs `change` as a Change request against it in the same Customer case. The Change request is what is scored.
|
||||
2. `case-05-change-retention`: case 01's request, then "raw events must now be kept for 7 years instead of 30 days ... and must not be deletable during that time. Everything else stays as it is."
|
||||
3. `score_change` adds four checks to the usual ones:
|
||||
- `change_is_recorded`: at least one requirement is marked added, changed or removed;
|
||||
- `changed_requirements_reach_checks`: each added or changed requirement is the source of a Check that was added or changed, and a removed one's Checks are gone;
|
||||
- `change_is_contained`: no more than two items, or a tenth of the Baseline's graph, changed without being reached;
|
||||
- `change_costs_less_than_baseline`: the Change request used fewer tokens than the Execution it changed.
|
||||
4. The report records the Baseline's id, seconds and tokens.
|
||||
|
||||
Acceptance criteria: not yet tested. The case needs two Executions, roughly an hour and $8 to $10.
|
||||
|
||||
Tests: `tests/eval/test_scoring.py`, "a Change request stays contained" (5).
|
||||
Reference in New Issue
Block a user