feat: Change requests against a Baseline (ADR-0006)

A caller restates a request to say how requirements have changed, naming a
completed Execution of the same Customer case as the Baseline.

- app/baseline.py reads a Baseline from its files and refuses one that
  does not exist, did not complete or belongs to another case, before
  anything runs or is recorded. A Baseline is never modified.
- Requirements carry a change (unchanged, added, changed, removed) and a
  reason. Submission is refused when a Baseline requirement disappears, an
  id is reused, or an unchanged requirement's text differs. A removed
  requirement stays on record and asks nothing downstream.
- Each Specialist starts from its delivered version; the graph and the code
  are its working copy, and the description is carried over when the
  design's shape is the same.
- app/workflows/changes.py compares versions by id and traces what each
  changed requirement reaches; anything else that changed is listed as
  unrequested. Approvals show the change for their Phase, and changes.json
  is written with the full Deliverables.
- Intake no longer starts a message read as a change: it names the
  Execution to give as the Baseline.
- A fifth evaluation case runs a request and then a change to it, with
  four checks that the change is recorded, reaches Checks, stays contained
  and costs less than its Baseline.

Tested with scripted agents only; no Change request has run with real
models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 11:38:15 +01:00
parent db28d02f56
commit 5ef434d00e
32 changed files with 1824 additions and 40 deletions

View File

@@ -0,0 +1,16 @@
# Step 7 — Implementation Detail
Built on 2026-10-09: the case and its checks. **The run is outstanding.**
1. A case may carry `change`, a statement of what has changed. The evaluation then runs the case's request to completion, pre-approved, and runs `change` as a Change request against it in the same Customer case. The Change request is what is scored.
2. `case-05-change-retention`: case 01's request, then "raw events must now be kept for 7 years instead of 30 days ... and must not be deletable during that time. Everything else stays as it is."
3. `score_change` adds four checks to the usual ones:
- `change_is_recorded`: at least one requirement is marked added, changed or removed;
- `changed_requirements_reach_checks`: each added or changed requirement is the source of a Check that was added or changed, and a removed one's Checks are gone;
- `change_is_contained`: no more than two items, or a tenth of the Baseline's graph, changed without being reached;
- `change_costs_less_than_baseline`: the Change request used fewer tokens than the Execution it changed.
4. The report records the Baseline's id, seconds and tokens.
Acceptance criteria: not yet tested. The case needs two Executions, roughly an hour and $8 to $10.
Tests: `tests/eval/test_scoring.py`, "a Change request stays contained" (5).