The reference agent's subject, cloud solution architecture, as an agent in the operating model agent's pattern: the Execution engine (from orchestration_engine) and one Workflow in app/solution_architecture, built for the settings and provider bundle by assembly.workflow_for. Copied from gcp_solution_architecture_agent at 8b397cd without its own engine: its role files, Skills, provider bundle, ADRs, vocabulary, scripts and evaluation datasets. The image takes the engine from a named build context and installs Terraform. It is called through the Orchestration engine, and its files are what the operating model agent digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9.9 KiB
status
| status |
|---|
| accepted |
Jev is the Orchestrator's decision point, and the Orchestrator is a router
ADR-0003 made the Orchestrator a reasoning agent on a large language model. In practice its turns are slow and mostly mechanical: in the evaluation run of 2026-10-06 it took about 7 of 35 minutes, and stalled four times before doing the only thing available. We decide that the Orchestrator follows the agent-control pattern Vercel describes for Jev (Jev in AI agent loops): Jev, a decision model that answers typed questions with probabilities, proposes the next action; application code checks that it is allowed, carries it out and records it; a generative model only writes what the user reads. Specialists are unchanged: they still reason and write on language models.
Who owns what
| Responsibility | Owner in the pattern | Here |
|---|---|---|
| Interpret the request | A model with supplied context | The requirements analyst and the Specialists after it |
| Propose the next action | The decision model | Jev, asked typed questions about the Execution's state |
| Check permission | Application code | The Ledger: Dependencies, Approvals, the Revision limit, the cloud-read gate |
| Execute the action | The tool runner | The Orchestrator's existing tools, called by code |
| Update state | Application code | The Execution graph and its checkpoint |
| Explain the result | A generative model | One small model call that writes an Approval summary or a question, from data it is given |
The Orchestrator is a router
With Jev as its decision point, the Orchestrator's job is routing: it reads what has arrived and the state of the Execution, asks Jev which route applies, and dispatches. It does no work of its own. There are four routes to choose.
| What is routed | The routes Jev chooses between | Who is dispatched |
|---|---|---|
| A message that starts something | a new Workflow request; a Change request against a Baseline; a question about work already delivered; not something this agent does | a full Execution; an Execution from the Baseline; the reply writer, from the delivered files; a polite refusal |
| A message inside an Execution | an answer to a waiting question; a request to change submitted work, and which Deliverable; anything else | the Specialist that asked; a Revision of that Deliverable; a question back to the user |
| A validation outcome | revise the code; revise the Target architecture; proceed | the IaC author; the target architect; the Approval |
| Ready work, in a Change request | which Deliverables the change reaches, and which carry over from the Baseline unchanged | only the Specialists whose Deliverables are reached |
Which Specialist produces a given Deliverable is still fixed by the dependency graph: Jev chooses the route, never the worker for it. And the Ledger still refuses a route whose Dependencies or Approvals are not met.
How a turn is decided: readings, disposition, playbook
The owner's model for this is a four-stage pipeline: readings of what arrived, a disposition worked out in code, further readings of the state, and a playbook that takes the first group that applies and, within it, the strongest action whose conditions hold, escalating when no group applies.
-
Readings. Facts about the turn. Most are read from the Ledger and the Execution's data by code: are questions waiting, is a Phase complete, is work ready, which Checks failed, did a Check fail twice running. The ones that need judgement are typed answers from Jev: is the user asking for a change (yes or no), which Deliverable would have to change first (a choice), how specific are they (a score), does a gap in the Target architecture leave a requirement unmet (yes or no). Jev reads; it never picks the action.
-
Disposition. Code turns a judged reading into act, queue or close by threshold: a probability of 0.75 or more is acted on, 0.15 or less is dropped, and anything between is put to the user. A choice with confidence under 0.6 is not relied on. The thresholds are settings.
-
Playbook. Ordered groups; the first that applies is taken, and within it the strongest action whose conditions hold:
Group Actions, strongest first The user asked for a change revise the Deliverable it affects; else ask them to confirm which and what Questions are waiting relay them Validation decided revise the Target architecture (a Check failed twice running, or a gap that matters); else revise the code (a Check failed); else ask about the gaps; else proceed A Phase is complete request its Approval Work is ready delegate all of it Packaging is approved finish No group applies escalate: ask the user, or end with warnings when there is none -
Permission. Every action goes through the Orchestrator's tools, so the Ledger refuses what is not allowed whatever was decided, and the playbook then takes the next action that is. A typed answer can still be wrong.
-
Record. Each decision is kept in the Execution's state: the turn, the group, the action and what Jev answered with its confidence.
-
Reply. A generative model writes only what the user reads, the summary they approve a Phase from, out of data it is given. It runs after the decision and cannot change it. When it fails, the summary is assembled from the same data.
A failed call to Jev is not an answer. The client returns nothing, and the reading falls back to a rule that acts only on what is certain: a change is acted on when the Deliverable is named outright, and a gap is revised for when it names a real requirement or cannot be verified.
Decisions
Accepted by the owner on 2026-10-07, with each of these:
- No language model decides what the Orchestrator does. Jev reads, and the playbook in code decides. A generative model writes user-facing text only. This replaces the part of ADR-0003 that makes the Orchestrator a reasoning agent.
- Low confidence goes to the user, not to another model. Below a decision's threshold, an interactive Execution asks the user in plain terms; a pre-approved one takes the safest allowed action and records a warning.
- Jev being unreachable does not stop an Execution. After a fixed retry budget the Orchestrator falls back to the fixed order of its seven steps, in code, and records that it did.
- What is sent to Jev is the Execution's state, not its content: Deliverable names and statuses, Check ids and verdicts, and the user's latest message. Deliverable text and discovered cloud data are not sent for an Orchestrator decision.
- The Orchestrator routes; Jev chooses the route. That covers what kind of work a message starts, where a message inside an Execution goes, where a validation outcome goes, and, in a Change request, which Deliverables are reached. Jev never chooses which Specialist produces a Deliverable: the dependency graph fixes that.
- In a new Execution every Deliverable is still produced. Routing does not skip a Specialist for a new Workflow request, because each later Deliverable is checked against the ones before it. Skipping happens only in a Change request, where the Baseline supplies what is carried over.
Considered Options
- A smaller language model as Orchestrator: rejected by the owner. It keeps the slow, open-ended turn and the stalls, at lower cost.
- No generative model at all, with fixed wording for summaries and questions: this ADR's first draft. Rejected to follow the pattern, which pairs Jev with a generative model for replies: an Approval summary is what a person decides from and reads better written than assembled.
- Code only, with fixed rules: rejected. It cannot tell what a user's free-text reply is asking for.
- Code for the mechanical steps and the existing model for judgement: rejected in favour of one decision point. It keeps two ways of deciding and the large model in the loop.
Consequences
-
Jev is reached through TypeSafe AI's Python SDK,
typesafe-sdk, pinned exactly (owner, 2026-10-07).app/tools/jev.pyis the only module that imports it. -
The Orchestrator as a language-model agent is kept behind the setting
ORCHESTRATOR=model, for the tests that drive the Ledger's rules in arbitrary order and for comparing the two in an evaluation. The default isrouter. Remove the setting once the router is proven. -
The Orchestrator's instructions in
agents/orchestrator.mdbecome the wording of Jev's questions and the brief for the reply writer, held in a Skill or the provider bundle so they stay reviewable and portable. -
Jev becomes a dependency of every interactive Execution. Its availability, latency, rate limits and data retention must be known before this is built; none is documented publicly.
-
Decisions become measurable: each has a confidence, so thresholds can be tuned from recorded Executions.
-
An unusual request that fits none of the questions is handled by asking the user, where a language model might have improvised.
As built (2026-10-09)
- Jev is used beyond the Orchestrator. It also chooses the pillars a change touches and the Skill references a Specialist reads first, checks remote MCP replies and the user's text for instructions aimed at the agent, tells a rewording from a change, and scores each Execution's design recommendations. Each has a fallback in code and is a setting.
- Decision 4 covers Orchestrator decisions only. These other uses do send content: design recommendations, Check and Element text, remote MCP replies and the Workflow request. Jev's data retention is therefore a question about Deliverable content, and it is still unanswered by the vendor.
- Jev did not become a hard dependency. Without a key, or when a call fails, every decision takes its fallback and the Execution continues.
- The fourth route, which Deliverables a change reaches, is decided in code from the changed requirements and the Requirements chain, not by Jev.