fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s

This commit is contained in:
2026-10-07 17:33:36 +01:00
parent 4c4d44ef87
commit 5bb22d984e
76 changed files with 5799 additions and 179 deletions

View File

@@ -13,7 +13,7 @@ An Execution is a hierarchy of ReAct agents shaped by the dependencies between D
The dependency graph is declared as data in code. The Orchestrator picks what to do next through tools that refuse work whose dependencies aren't met. Dependencies are guaranteed by code, while ordering and parallelism are the model's choice.
> **Under review (2026-10-07).** [ADR-0007](0007-jev-is-the-orchestrators-decision-point.md), proposed, would have the Orchestrator decide with Jev and run no language model. If accepted, it replaces "the Orchestrator ... is an ADK `LlmAgent`" above and reverses the third option rejected below. Specialists are unaffected.
> **Amended (2026-10-07) by [ADR-0007](0007-jev-is-the-orchestrators-decision-point.md), accepted.** The Orchestrator is now a router: it decides by readings and a playbook in code, with Jev for the judgements, and runs no language model to choose an action. That replaces "the Orchestrator ... is an ADK `LlmAgent`" above and reverses the third option rejected below. Specialists are unaffected and still reason on language models.
## Considered Options

View File

@@ -1,8 +1,8 @@
---
status: proposed
status: accepted
---
# Jev is the Orchestrator's decision point
# Jev is the Orchestrator's decision point, and the Orchestrator is a router
[ADR-0003](0003-langgraph-holds-state-adk-agents-reason.md) made the Orchestrator a reasoning agent on a large language model. In practice its turns are slow and mostly mechanical: in the evaluation run of 2026-10-06 it took about 7 of 35 minutes, and stalled four times before doing the only thing available. We decide that the Orchestrator follows the agent-control pattern Vercel describes for Jev ([Jev in AI agent loops](https://vercel.com/i/jev-agent-control)): Jev, a decision model that answers typed questions with probabilities, proposes the next action; application code checks that it is allowed, carries it out and records it; a generative model only writes what the user reads. Specialists are unchanged: they still reason and write on language models.
@@ -17,28 +17,53 @@ status: proposed
| Update state | Application code | The Execution graph and its checkpoint |
| Explain the result | A generative model | One small model call that writes an Approval summary or a question, from data it is given |
## How a turn is decided
## The Orchestrator is a router
1. **Code lists the allowed actions** from the Ledger and the Execution's state: relay waiting questions, act on a validation outcome, request an Approval, delegate ready work, finish. This is the permission check, and it holds whatever Jev answers, because a typed decision can still be wrong.
2. **Jev proposes the next action** from that list, and answers the other closed questions the turn needs, in one request where they rest on the same evidence:
- Did the user's last message ask for a change to submitted work? (yes or no)
- Which Deliverable does that change affect? (one of the Deliverables)
- What should happen after this validation: revise the code, revise the Target architecture, or proceed? (one of three)
A question whose options depend on another answer, or on evidence not yet fetched, is a second request.
3. **Code carries the decision out** through the existing tools and records it: the questions, the answers, their probabilities and what was done. A resumed Execution reuses a recorded decision when its input is unchanged.
4. **A generative model writes the reply**, only where the user reads prose: the summary they approve a Phase from, and any question the Orchestrator itself asks. It is given the decision and the data and decides nothing.
With Jev as its decision point, the Orchestrator's job is routing: it reads what has arrived and the state of the Execution, asks Jev which route applies, and dispatches. It does no work of its own. There are four routes to choose.
Two guards from the pattern are kept in code. Repeated actions are limited, so the Orchestrator cannot loop on the same step: the turn limit and the Revision limit already do this. A failed call to Jev is recorded as a failure, distinct from an answer, so "Jev was unreachable" is never read as "nothing to do".
| What is routed | The routes Jev chooses between | Who is dispatched |
|---|---|---|
| A message that starts something | a new Workflow request; a Change request against a Baseline; a question about work already delivered; not something this agent does | a full Execution; an Execution from the Baseline; the reply writer, from the delivered files; a polite refusal |
| A message inside an Execution | an answer to a waiting question; a request to change submitted work, and which Deliverable; anything else | the Specialist that asked; a Revision of that Deliverable; a question back to the user |
| A validation outcome | revise the code; revise the Target architecture; proceed | the IaC author; the target architect; the Approval |
| Ready work, in a Change request | which Deliverables the change reaches, and which carry over from the Baseline unchanged | only the Specialists whose Deliverables are reached |
## Decisions to confirm
Which Specialist produces a given Deliverable is still fixed by the dependency graph: Jev chooses the route, never the worker for it. And the Ledger still refuses a route whose Dependencies or Approvals are not met.
This ADR is proposed. Accepting it means agreeing to each of these:
## How a turn is decided: readings, disposition, playbook
1. **No language model decides what the Orchestrator does.** Jev proposes and code disposes. A generative model writes user-facing text only. This replaces the part of ADR-0003 that makes the Orchestrator a reasoning agent.
The owner's model for this is a four-stage pipeline: readings of what arrived, a disposition worked out in code, further readings of the state, and a playbook that takes the first group that applies and, within it, the strongest action whose conditions hold, escalating when no group applies.
1. **Readings.** Facts about the turn. Most are read from the Ledger and the Execution's data by code: are questions waiting, is a Phase complete, is work ready, which Checks failed, did a Check fail twice running. The ones that need judgement are typed answers from Jev: is the user asking for a change (yes or no), which Deliverable would have to change first (a choice), how specific are they (a score), does a gap in the Target architecture leave a requirement unmet (yes or no). Jev reads; it never picks the action.
2. **Disposition.** Code turns a judged reading into act, queue or close by threshold: a probability of 0.75 or more is acted on, 0.15 or less is dropped, and anything between is put to the user. A choice with confidence under 0.6 is not relied on. The thresholds are settings.
3. **Playbook.** Ordered groups; the first that applies is taken, and within it the strongest action whose conditions hold:
| Group | Actions, strongest first |
|---|---|
| The user asked for a change | revise the Deliverable it affects; else ask them to confirm which and what |
| Questions are waiting | relay them |
| Validation decided | revise the Target architecture (a Check failed twice running, or a gap that matters); else revise the code (a Check failed); else ask about the gaps; else proceed |
| A Phase is complete | request its Approval |
| Work is ready | delegate all of it |
| Packaging is approved | finish |
| No group applies | escalate: ask the user, or end with warnings when there is none |
4. **Permission.** Every action goes through the Orchestrator's tools, so the Ledger refuses what is not allowed whatever was decided, and the playbook then takes the next action that is. A typed answer can still be wrong.
5. **Record.** Each decision is kept in the Execution's state: the turn, the group, the action and what Jev answered with its confidence.
6. **Reply.** A generative model writes only what the user reads, the summary they approve a Phase from, out of data it is given. It runs after the decision and cannot change it. When it fails, the summary is assembled from the same data.
A failed call to Jev is not an answer. The client returns nothing, and the reading falls back to a rule that acts only on what is certain: a change is acted on when the Deliverable is named outright, and a gap is revised for when it names a real requirement or cannot be verified.
## Decisions
Accepted by the owner on 2026-10-07, with each of these:
1. **No language model decides what the Orchestrator does.** Jev reads, and the playbook in code decides. A generative model writes user-facing text only. This replaces the part of ADR-0003 that makes the Orchestrator a reasoning agent.
2. **Low confidence goes to the user, not to another model.** Below a decision's threshold, an interactive Execution asks the user in plain terms; a pre-approved one takes the safest allowed action and records a warning.
3. **Jev being unreachable does not stop an Execution.** After a fixed retry budget the Orchestrator falls back to the fixed order of its seven steps, in code, and records that it did.
4. **What is sent to Jev is the Execution's state, not its content:** Deliverable names and statuses, Check ids and verdicts, and the user's latest message. Deliverable text and discovered cloud data are not sent for an Orchestrator decision.
5. **Jev does not choose Specialists.** Which Specialist produces a Deliverable is fixed by the dependency graph, and all ready work is delegated together. The pattern's specialist routing has nothing to route here.
5. **The Orchestrator routes; Jev chooses the route.** That covers what kind of work a message starts, where a message inside an Execution goes, where a validation outcome goes, and, in a Change request, which Deliverables are reached. Jev never chooses which Specialist produces a Deliverable: the dependency graph fixes that.
6. **In a new Execution every Deliverable is still produced.** Routing does not skip a Specialist for a new Workflow request, because each later Deliverable is checked against the ones before it. Skipping happens only in a Change request, where the Baseline supplies what is carried over.
## Considered Options
@@ -49,6 +74,9 @@ This ADR is proposed. Accepting it means agreeing to each of these:
## Consequences
- Jev is reached through TypeSafe AI's Python SDK, `typesafe-sdk`, pinned exactly (owner, 2026-10-07). `app/tools/jev.py` is the only module that imports it.
- The Orchestrator as a language-model agent is kept behind the setting `ORCHESTRATOR=model`, for the tests that drive the Ledger's rules in arbitrary order and for comparing the two in an evaluation. The default is `router`. Remove the setting once the router is proven.
- The Orchestrator's instructions in `agents/orchestrator.md` become the wording of Jev's questions and the brief for the reply writer, held in a Skill or the provider bundle so they stay reviewable and portable.
- Jev becomes a dependency of every interactive Execution. Its availability, latency, rate limits and data retention must be known before this is built; none is documented publicly.
- Decisions become measurable: each has a confidence, so thresholds can be tuned from recorded Executions.