feat: establish the Target architecture as a graph before code

Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 13:59:09 +01:00
parent 7ee49ab352
commit e9246828b3
586 changed files with 5166 additions and 22432 deletions

View File

@@ -2,8 +2,8 @@
**Iteration:** 5, Jev decisions
**Intent:** Use Jev, a hosted decision model, for the agent's closed questions (yes or no, pick one, rate on a scale) so that large-model calls are kept for writing and reasoning. The aims are fewer tokens, faster turns, and checks the engine cannot make with string matching.
**Status:** planned; gated on step 1
**This directory covers:** steps 1 to 14
**Status:** planned; gated on step 1. Build `../iter-06-target-architecture-graph/` first: it removes the need for step 5 here and changes what step 4 checks.
**This directory covers:** steps 1 to 15
## What Jev is
@@ -26,10 +26,29 @@ Jev will be deployed separately and reached with an API key (owner, 2026-10-06).
## Sources of truth
- https://www.langchain.com/blog/building-a-harness-with-jev and https://www.jevtypesafeai.com/use-cases (and `/docs` on that site).
- https://vercel.com/i/jev-agent-control: the agent-control pattern step 12 follows (decision model proposes, code checks and executes, a generative model explains).
- Review report, section "Where Jev fits": https://claude.ai/code/artifact/16b37169-1554-4eb7-bfcb-254315fa0518. `Report ref` names its J-number.
- `iter-04-learning-and-skills/`: several steps here build on steps there.
- `docs/adr/0003-langgraph-holds-state-adk-agents-reason.md`: step 12 changes what it says about the Orchestrator.
## The capabilities the owner wants from Jev
On 2026-10-07 the owner named the nine use cases at https://vercel.com/i/jev-use-cases as the capabilities to use. Each maps to this agent as follows.
| Jev use case | Question type | What it is here | Step |
|---|---|---|---|
| Form routing | Choice | Read the user's message: an answer, a request for changes, or a new statement of Intent, and which Deliverable a change affects | 12 |
| Tool-call review (`clear` or `caution`) | Choice | Review each cloud-read call and `terraform_plan` before it runs; `caution` pauses for the user | 13 |
| Response model selection | Choice | Pick the model tier for each delegation from the bundle's list | 6 |
| Response evaluation | Boolean, Score | Judge a Deliverable at submission (does it address its task; is each Check verifiable as written), and score the evaluation's rubric | 4, 10 |
| Content moderation | Boolean | Flag text that breaks a written policy: requirements that name products; request or tool text that tries to instruct the agent | 4, 14 |
| UI copy translation review (did an edit change the meaning?) | Boolean | In a Change request, flag a requirement, Element or Check whose meaning changed against the Baseline, so only real changes are followed through | 15 |
| Ticket prioritization | Score | Rate how much a proposed addition or a failed Check matters, to order Revisions and lesson candidates | 10 |
| Document categorization | Choice | Sort lesson candidates by Deliverable for the curator | 10 |
| CSV column mapping | Choice | Suggest which requirement or recommendation a proposed addition serves, for confirmation; code still checks the link exists | 10 |
The page's own cautions apply to every step: Jev does not write replies, so a generative model drafts messages; exact conditions (a Dependency is met, a file is attached, an id exists) stay deterministic checks in code; and a typed answer can still be wrong, so start where outcomes can be inspected and corrected, and record every decision.
## Boundaries
Jev answers closed questions. Writing Deliverables, grounding product choices and producing infrastructure code stay with the large models. Dependencies and Approvals stay rules in the Ledger.
@@ -48,10 +67,11 @@ Jev answers closed questions. Writing Deliverables, grounding product choices an
| 8 | Skill routing | J9 | planned |
| 9 | In-run context compaction | J10 | planned |
| 10 | Evaluation rubric and lesson triage | J8 | planned |
| 11 | ADR: Orchestrator routing | none | planned, owner's decision |
| 12 | Orchestrator routing | J1 | planned |
| 11 | ADR: Jev is the Orchestrator's decision point | none | drafted as `docs/adr/0007-jev-is-the-orchestrators-decision-point.md`, status proposed; awaits the owner |
| 12 | Jev proposes the Orchestrator's next action; a model only writes replies | J1 | planned; the owner's priority for this iteration (2026-10-07) |
| 13 | Cloud-read call guard | J7 | planned |
| 14 | Input guardrail | J11 | planned |
| 15 | Changes of meaning in a Change request | none | planned; needs iteration 7 |
## Dependency graph
@@ -65,9 +85,10 @@ Jev answers closed questions. Writing Deliverables, grounding product choices an
- Step 8: depends on step 2, iteration 4 step 12
- Step 9: depends on step 2
- Step 10: depends on step 2, iteration 4 step 7, iteration 4 step 8
- Step 11: depends on step 4 having evaluation results
- Step 12: depends on step 2, step 11
- Step 11: no dependencies
- Step 12: depends on step 2, step 11. Build it straight after step 2, ahead of steps 3 to 10.
- Step 13: depends on step 2
- Step 14: depends on step 2
- Step 15: depends on step 2, iteration 7 step 4
Steps 3 and 4 come first: they are small, need no ADR change, and target Revisions, where the tokens are. The Orchestrator is 11% of case 03's tokens; the IaC author and validator together used 1.43 million over 7 runs.