Commit Graph

8 Commits

Author SHA1 Message Date
545097530b feat: scoping is the first Phase, completed in conversation (iteration 9 step 2)
Some checks failed
validation / verify (push) Failing after 10s
Every Execution now begins by agreeing the scope, before requirements.

- A new Phase and Deliverable, produced by a scope analyst with two tools:
  scope_read shows what is settled and what is not; scope_set records fields
  with where each value came from (the request, the customer's answer, or an
  assumption with its reason). Code checks each value against the catalogue.
- With someone present the analyst asks for what is missing; with nobody
  present it assumes it, visibly. With no spec it drafts one from the request.
  A value the customer stated in the file is changed only by their own answer.
- A submission cannot leave an essential field unknown. Code writes the
  completed spec, the data every later Specialist reads, and the information
  register, from what was recorded.
- The spec's status follows the Execution: under review, approved, then
  locked as a Baseline. A Change request starts from the delivered scope.
- The requirements analyst derives from the agreed scope and no longer asks
  for what it settles.

About 80 tests assumed requirements came first; tests/scoped.py runs scoping
ahead of a test's own script, and tests about order and counts are updated.
Not run with real models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:17:32 +01:00
567aede5ca feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)
Iteration 4:
- Overlay Skills (step 12): an Execution names an organisation's standards
  Skill from the folders the deployment configures; recorded with its version.
  An example standards Skill and an evaluation case that uses it.
- Lessons (step 10): a prove command that compares a run with lessons against
  one without, and triage of evidence an existing lesson already covers.

Iteration 5:
- Step 8: the references of a Specialist's Skills its task needs are named in
  its context as read_first.
- Step 14: a remote MCP server's reply that instructs the agent is withheld;
  the user's own text is flagged as a risk and nothing more.
- Step 15: in a Change request, items only reworded are listed apart from
  real changes.
- Step 19: each Execution's design recommendations are judged when it ends and
  the scores kept in its record. The evaluation takes its rubric from the same
  place.

Each question was calibrated against live Jev; none has run in a case. Steps
6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:45:56 +01:00
844987587c feat: consult only the pillars a change affects directly (iteration 5 step 3)
A graded question per pillar (not at all, only indirectly, directly) picks
the pillar advisors to consult again on a Revision or a Change request; the
design advisor is told which, and keeps its earlier advice for the rest. As
a yes or no the answer was yes for nearly every pillar of every change.

A completed Execution now writes working.json with each pillar's advice
and checklist review, and a Change request starts from its Baseline's, so
only the pillars a change touches are consulted afresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:22:57 +01:00
654370bb46 feat: all four evaluation cases pass; intake, checklist review, thresholds
Case 04 (interactive)
- A scripted answer may name who it is for, so facts written for
  requirements discovery are not given to a Specialist confirming a draft.
- A reply that restates what the agreed requirements already say is read
  as an answer, not a request for changes.
- With a user present, a first submission is refused until the user has
  been shown a draft of that Deliverable (CONFIRM_DRAFTS).

Routing and reviews
- Intake: a message that starts something is read first. A question is
  answered and starts nothing; a change to delivered work starts as a new
  request with a note, until Change requests are built.
- Requirements and recommendations list what infrastructure code can show
  (conditions) apart from what it cannot (outside_code); the latter needs
  no Check and validation proposes nothing for it.
- A requirements review, "names no products", behind its own switch (off).
- Each pillar advisor reviews its pillar's checklist; the engine writes the
  reviews into the design recommendations. The judged checklist check went
  from 9-45% a pillar to 100% in one run of case 04.

Evaluation
- Per-case thresholds for Revisions, seconds and tokens, set from every
  completed run on this build.
- A progress file after every agent run, so an interrupted run keeps its
  time and tokens.
- eval/calibrate_judge.py with a 32-question set awaiting a person's
  labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91.

Smaller
- An Execution that ends with warnings writes its Deliverables, with the
  warnings and each status in the manifest.
- CI installs the application's dependencies before linting and testing.
- A load-balancer pattern in the IaC patterns Skill.

Plans record nine runs of 2026-10-08. Iteration 6 is complete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:32:57 +01:00
5adc911a82 feat: keep an unchanged description, list conditions, review coverage with Jev
After case 02 (completed at 100%, but with two Revisions of the Target
architecture for gaps validation found one pass at a time):

- A Deliverable stands while what it was written from is unchanged. The
  architecture description is written from the Elements and Relationships,
  so a Revision that only adds or rewords Checks no longer runs the describer.
- Non-functional requirements and recommendations list their conditions,
  one entry per separate thing that must be true; submission requires them.
- A coverage review at submission (JEV_COVERAGE_REVIEW, off by default):
  for each condition, Jev chooses which Check verifies it, or none. Asked as
  one broad yes-or-no per requirement it could not tell a complete design
  from one with known gaps; asked as a choice per condition it raised all
  four known gaps of case 02 and nothing on the complete graph. An opt-in
  live test (LIVE_JEV=1) repeats that.
- The validator is told to report every gap in its first pass.

Plans record the case 02 run and both forms of the review with figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:58:38 +01:00
5bb22d984e fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s
2026-10-07 17:33:36 +01:00
e9246828b3 feat: establish the Target architecture as a graph before code
Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:59:09 +01:00
7ee49ab352 latest refactoring
Some checks failed
validation / verify (push) Failing after 10s
2026-10-06 20:22:37 +01:00