Commit Graph

6 Commits

Author SHA1 Message Date
545097530b feat: scoping is the first Phase, completed in conversation (iteration 9 step 2)
Some checks failed
validation / verify (push) Failing after 10s
Every Execution now begins by agreeing the scope, before requirements.

- A new Phase and Deliverable, produced by a scope analyst with two tools:
  scope_read shows what is settled and what is not; scope_set records fields
  with where each value came from (the request, the customer's answer, or an
  assumption with its reason). Code checks each value against the catalogue.
- With someone present the analyst asks for what is missing; with nobody
  present it assumes it, visibly. With no spec it drafts one from the request.
  A value the customer stated in the file is changed only by their own answer.
- A submission cannot leave an essential field unknown. Code writes the
  completed spec, the data every later Specialist reads, and the information
  register, from what was recorded.
- The spec's status follows the Execution: under review, approved, then
  locked as a Baseline. A Change request starts from the delivered scope.
- The requirements analyst derives from the agreed scope and no longer asks
  for what it settles.

About 80 tests assumed requirements came first; tests/scoped.py runs scoping
ahead of a test's own script, and tests about order and counts are updated.
Not run with real models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:17:32 +01:00
5ef434d00e feat: Change requests against a Baseline (ADR-0006)
A caller restates a request to say how requirements have changed, naming a
completed Execution of the same Customer case as the Baseline.

- app/baseline.py reads a Baseline from its files and refuses one that
  does not exist, did not complete or belongs to another case, before
  anything runs or is recorded. A Baseline is never modified.
- Requirements carry a change (unchanged, added, changed, removed) and a
  reason. Submission is refused when a Baseline requirement disappears, an
  id is reused, or an unchanged requirement's text differs. A removed
  requirement stays on record and asks nothing downstream.
- Each Specialist starts from its delivered version; the graph and the code
  are its working copy, and the description is carried over when the
  design's shape is the same.
- app/workflows/changes.py compares versions by id and traces what each
  changed requirement reaches; anything else that changed is listed as
  unrequested. Approvals show the change for their Phase, and changes.json
  is written with the full Deliverables.
- Intake no longer starts a message read as a change: it names the
  Execution to give as the Baseline.
- A fifth evaluation case runs a request and then a change to it, with
  four checks that the change is recorded, reaches Checks, stays contained
  and costs less than its Baseline.

Tested with scripted agents only; no Change request has run with real
models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:38:15 +01:00
654370bb46 feat: all four evaluation cases pass; intake, checklist review, thresholds
Case 04 (interactive)
- A scripted answer may name who it is for, so facts written for
  requirements discovery are not given to a Specialist confirming a draft.
- A reply that restates what the agreed requirements already say is read
  as an answer, not a request for changes.
- With a user present, a first submission is refused until the user has
  been shown a draft of that Deliverable (CONFIRM_DRAFTS).

Routing and reviews
- Intake: a message that starts something is read first. A question is
  answered and starts nothing; a change to delivered work starts as a new
  request with a note, until Change requests are built.
- Requirements and recommendations list what infrastructure code can show
  (conditions) apart from what it cannot (outside_code); the latter needs
  no Check and validation proposes nothing for it.
- A requirements review, "names no products", behind its own switch (off).
- Each pillar advisor reviews its pillar's checklist; the engine writes the
  reviews into the design recommendations. The judged checklist check went
  from 9-45% a pillar to 100% in one run of case 04.

Evaluation
- Per-case thresholds for Revisions, seconds and tokens, set from every
  completed run on this build.
- A progress file after every agent run, so an interrupted run keeps its
  time and tokens.
- eval/calibrate_judge.py with a 32-question set awaiting a person's
  labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91.

Smaller
- An Execution that ends with warnings writes its Deliverables, with the
  warnings and each status in the manifest.
- CI installs the application's dependencies before linting and testing.
- A load-balancer pattern in the IaC patterns Skill.

Plans record nine runs of 2026-10-08. Iteration 6 is complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:32:57 +01:00
5adc911a82 feat: keep an unchanged description, list conditions, review coverage with Jev
After case 02 (completed at 100%, but with two Revisions of the Target
architecture for gaps validation found one pass at a time):

- A Deliverable stands while what it was written from is unchanged. The
  architecture description is written from the Elements and Relationships,
  so a Revision that only adds or rewords Checks no longer runs the describer.
- Non-functional requirements and recommendations list their conditions,
  one entry per separate thing that must be true; submission requires them.
- A coverage review at submission (JEV_COVERAGE_REVIEW, off by default):
  for each condition, Jev chooses which Check verifies it, or none. Asked as
  one broad yes-or-no per requirement it could not tell a complete design
  from one with known gaps; asked as a choice per condition it raised all
  four known gaps of case 02 and nothing on the complete graph. An opt-in
  live test (LIVE_JEV=1) repeats that.
- The validator is told to report every gap in its first pass.

Plans record the case 02 run and both forms of the review with figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:58:38 +01:00
5bb22d984e fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s
2026-10-07 17:33:36 +01:00
e9246828b3 feat: establish the Target architecture as a graph before code
Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:59:09 +01:00