Files
gcp_solution_architecture_a…/docs/adr/0007-jev-is-the-orchestrators-decision-point.md
Jonathan Boniface e9246828b3 feat: establish the Target architecture as a graph before code
Design is now approved before any infrastructure code is written (ADR-0005).
A new target-architect Specialist produces one graph of Elements,
Relationships and Checks; the diagram is rendered from it, deployment
guidance must account for every Check, and validation gives a verdict per
Check. Requirements, components, products and recommendations carry ids in
data files, and a submission with a gap in the Requirements chain is refused.

Also in this commit:
- Customer cases group Executions, with an intent history per case
- lesson candidates are captured when an Execution ends
- evaluation gains chain, verdict, judged and interactive checks, and
  per-case thresholds
- Terraform providers are cached between checks
- the requirements analyst proposes defaults instead of asking
- the legacy-baseline case and the checks that only read it are removed
- ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed
- plans for iterations 4 to 7 under .agents/plans/

Unproven with a real model: no evaluation run has completed on this design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:59:09 +01:00

5.6 KiB

status
status
proposed

Jev is the Orchestrator's decision point

ADR-0003 made the Orchestrator a reasoning agent on a large language model. In practice its turns are slow and mostly mechanical: in the evaluation run of 2026-10-06 it took about 7 of 35 minutes, and stalled four times before doing the only thing available. We decide that the Orchestrator follows the agent-control pattern Vercel describes for Jev (Jev in AI agent loops): Jev, a decision model that answers typed questions with probabilities, proposes the next action; application code checks that it is allowed, carries it out and records it; a generative model only writes what the user reads. Specialists are unchanged: they still reason and write on language models.

Who owns what

Responsibility Owner in the pattern Here
Interpret the request A model with supplied context The requirements analyst and the Specialists after it
Propose the next action The decision model Jev, asked typed questions about the Execution's state
Check permission Application code The Ledger: Dependencies, Approvals, the Revision limit, the cloud-read gate
Execute the action The tool runner The Orchestrator's existing tools, called by code
Update state Application code The Execution graph and its checkpoint
Explain the result A generative model One small model call that writes an Approval summary or a question, from data it is given

How a turn is decided

  1. Code lists the allowed actions from the Ledger and the Execution's state: relay waiting questions, act on a validation outcome, request an Approval, delegate ready work, finish. This is the permission check, and it holds whatever Jev answers, because a typed decision can still be wrong.
  2. Jev proposes the next action from that list, and answers the other closed questions the turn needs, in one request where they rest on the same evidence:
    • Did the user's last message ask for a change to submitted work? (yes or no)
    • Which Deliverable does that change affect? (one of the Deliverables)
    • What should happen after this validation: revise the code, revise the Target architecture, or proceed? (one of three) A question whose options depend on another answer, or on evidence not yet fetched, is a second request.
  3. Code carries the decision out through the existing tools and records it: the questions, the answers, their probabilities and what was done. A resumed Execution reuses a recorded decision when its input is unchanged.
  4. A generative model writes the reply, only where the user reads prose: the summary they approve a Phase from, and any question the Orchestrator itself asks. It is given the decision and the data and decides nothing.

Two guards from the pattern are kept in code. Repeated actions are limited, so the Orchestrator cannot loop on the same step: the turn limit and the Revision limit already do this. A failed call to Jev is recorded as a failure, distinct from an answer, so "Jev was unreachable" is never read as "nothing to do".

Decisions to confirm

This ADR is proposed. Accepting it means agreeing to each of these:

  1. No language model decides what the Orchestrator does. Jev proposes and code disposes. A generative model writes user-facing text only. This replaces the part of ADR-0003 that makes the Orchestrator a reasoning agent.
  2. Low confidence goes to the user, not to another model. Below a decision's threshold, an interactive Execution asks the user in plain terms; a pre-approved one takes the safest allowed action and records a warning.
  3. Jev being unreachable does not stop an Execution. After a fixed retry budget the Orchestrator falls back to the fixed order of its seven steps, in code, and records that it did.
  4. What is sent to Jev is the Execution's state, not its content: Deliverable names and statuses, Check ids and verdicts, and the user's latest message. Deliverable text and discovered cloud data are not sent for an Orchestrator decision.
  5. Jev does not choose Specialists. Which Specialist produces a Deliverable is fixed by the dependency graph, and all ready work is delegated together. The pattern's specialist routing has nothing to route here.

Considered Options

  • A smaller language model as Orchestrator: rejected by the owner. It keeps the slow, open-ended turn and the stalls, at lower cost.
  • No generative model at all, with fixed wording for summaries and questions: this ADR's first draft. Rejected to follow the pattern, which pairs Jev with a generative model for replies: an Approval summary is what a person decides from and reads better written than assembled.
  • Code only, with fixed rules: rejected. It cannot tell what a user's free-text reply is asking for.
  • Code for the mechanical steps and the existing model for judgement: rejected in favour of one decision point. It keeps two ways of deciding and the large model in the loop.

Consequences

  • The Orchestrator's instructions in agents/orchestrator.md become the wording of Jev's questions and the brief for the reply writer, held in a Skill or the provider bundle so they stay reviewable and portable.
  • Jev becomes a dependency of every interactive Execution. Its availability, latency, rate limits and data retention must be known before this is built; none is documented publicly.
  • Decisions become measurable: each has a confidence, so thresholds can be tuned from recorded Executions.
  • An unusual request that fits none of the questions is handled by asking the user, where a language model might have improvised.