Commit Graph

10 Commits

Author SHA1 Message Date
5ef434d00e feat: Change requests against a Baseline (ADR-0006)
A caller restates a request to say how requirements have changed, naming a
completed Execution of the same Customer case as the Baseline.

- app/baseline.py reads a Baseline from its files and refuses one that
  does not exist, did not complete or belongs to another case, before
  anything runs or is recorded. A Baseline is never modified.
- Requirements carry a change (unchanged, added, changed, removed) and a
  reason. Submission is refused when a Baseline requirement disappears, an
  id is reused, or an unchanged requirement's text differs. A removed
  requirement stays on record and asks nothing downstream.
- Each Specialist starts from its delivered version; the graph and the code
  are its working copy, and the description is carried over when the
  design's shape is the same.
- app/workflows/changes.py compares versions by id and traces what each
  changed requirement reaches; anything else that changed is listed as
  unrequested. Approvals show the change for their Phase, and changes.json
  is written with the full Deliverables.
- Intake no longer starts a message read as a change: it names the
  Execution to give as the Baseline.
- A fifth evaluation case runs a request and then a change to it, with
  four checks that the change is recorded, reaches Checks, stays contained
  and costs less than its Baseline.

Tested with scripted agents only; no Change request has run with real
models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:38:15 +01:00
654370bb46 feat: all four evaluation cases pass; intake, checklist review, thresholds
Case 04 (interactive)
- A scripted answer may name who it is for, so facts written for
  requirements discovery are not given to a Specialist confirming a draft.
- A reply that restates what the agreed requirements already say is read
  as an answer, not a request for changes.
- With a user present, a first submission is refused until the user has
  been shown a draft of that Deliverable (CONFIRM_DRAFTS).

Routing and reviews
- Intake: a message that starts something is read first. A question is
  answered and starts nothing; a change to delivered work starts as a new
  request with a note, until Change requests are built.
- Requirements and recommendations list what infrastructure code can show
  (conditions) apart from what it cannot (outside_code); the latter needs
  no Check and validation proposes nothing for it.
- A requirements review, "names no products", behind its own switch (off).
- Each pillar advisor reviews its pillar's checklist; the engine writes the
  reviews into the design recommendations. The judged checklist check went
  from 9-45% a pillar to 100% in one run of case 04.

Evaluation
- Per-case thresholds for Revisions, seconds and tokens, set from every
  completed run on this build.
- A progress file after every agent run, so an interrupted run keeps its
  time and tokens.
- eval/calibrate_judge.py with a 32-question set awaiting a person's
  labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91.

Smaller
- An Execution that ends with warnings writes its Deliverables, with the
  warnings and each status in the manifest.
- CI installs the application's dependencies before linting and testing.
- A load-balancer pattern in the IaC patterns Skill.

Plans record nine runs of 2026-10-08. Iteration 6 is complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:32:57 +01:00
d41607f505 fix: read only the user's own words, and stop the router looping at a limit
Case 04, the first interactive evaluation, stopped at 100 Orchestrator
turns on two router faults:

- A reply that quoted the agent's question back was judged on that question.
  A reply now reaches the router with quoted questions and Q:/A: labels
  removed, and the questions are given to the decider as background.
- At a Revision limit the reply was read as another change request for the
  same Deliverable, and the refusal repeated. The user is now told once,
  the reply is taken as the answer, that Deliverable is not reopened again,
  and the Execution carries on with it as it stands.
- Any run that asks the user four times running with no work done in
  between now ends with a warning.

The validator is told to confirm with the user in an interactive Execution.

Plans record the fixes and the rerun, which did not finish: both fixes
held, and a third fault appeared that is not fixed here. A scripted answer
restating a requirement, given to a draft-confirmation question, reopens
requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:05:03 +01:00
5bb22d984e fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s
2026-10-07 17:33:36 +01:00
e9246828b3 feat: establish the Target architecture as a graph before code
Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:59:09 +01:00
7ee49ab352 latest refactoring
Some checks failed
validation / verify (push) Failing after 10s
2026-10-06 20:22:37 +01:00
69012b2656 chore: stop tracking __pycache__ bytecode
These 30 .pyc files were committed before .gitignore covered them and
changed on every test run. Files remain on disk; .gitignore already
excludes __pycache__/ and *.pyc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 18:13:48 +01:00
a24a44e28c refactored: to utilise the google adk and production grade agent
Some checks failed
validation / verify (push) Failing after 10s
2026-09-02 21:42:10 +01:00
b9a924cf4a fix: refactored manually
Some checks failed
validation / verify (push) Failing after 15s
2026-09-02 16:08:12 +01:00
43f0f5bc79 decomposer: scaffold gcp_solution_architecture_agent 2026-09-01 19:47:04 +00:00