fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s

This commit is contained in:
2026-10-07 17:33:36 +01:00
parent 4c4d44ef87
commit 5bb22d984e
76 changed files with 5799 additions and 179 deletions

View File

@@ -0,0 +1,9 @@
# Step 11 — Implementation Detail
ADR-0007 was accepted by the owner on 2026-10-07 ("this is exactly what I want, I approve ADR-0007"), after three refinements made the same day and recorded in it:
1. Jev is the decision point, in place of a different language model.
2. The agent-control pattern at https://vercel.com/i/jev-agent-control is followed: a decision model reads, application code checks permission and executes, a generative model writes replies.
3. The Orchestrator is a router, and decides by readings, a disposition and a playbook, on the model of a triage pipeline the owner supplied: the first group that applies, the strongest action whose conditions hold, escalate when none applies.
Jev is reached through TypeSafe AI's Python SDK and API endpoint (owner, 2026-10-07). The API key is still to be supplied.

View File

@@ -0,0 +1,36 @@
# Step 12 — Implementation Detail
Built against a scripted decider. No call has been made to the live Jev service, and no evaluation has run on it: the owner holds evaluation runs until the Jev work is in.
1. **`app/nodes/router.py`.** `Router` decides an Orchestrator turn in three stages: readings (`_read_ledger` from the Ledger and data; `_read_message` and `_read_gap` from the decider), a disposition by `Thresholds` (act at 0.75, close at 0.15, choice confidence 0.6), and the playbook in `_choose`: ordered groups, the strongest action whose conditions hold, escalate when none applies.
2. **Permission stays with the tools.** The router calls the same `OrchestratorToolbox` tools the model did. A refused action is skipped for the rest of the turn and the next one that holds is taken. Past the Revision limit the user is asked.
3. **What Jev is asked** (`Question` objects, batched when they rest on the same evidence):
- of the user's message: `asks_change` (noul), `change_target` (choice over submitted Deliverables), `change_clarity` (score: vague, partial, specific);
- of validation gaps: `gap_material` (noul);
- of the user's reply about gaps: `wants_revision` (noul).
What is sent is Deliverable names, the count of waiting questions, the user's message, and for gaps the proposed Check statements and unverifiable Check ids.
4. **Rules when Jev gives no answer.** A requested change is acted on only when one Deliverable is named outright, otherwise the user is asked which. A free-text reply is treated as an answer. A gap is revised for when a Check cannot be verified or an addition names a real requirement or recommendation id.
5. **A new reading from the run of 2026-10-07:** a Check that fails in two validations running is sent to the Target architecture, not the code. That run retried one unimplementable Check as code three times.
6. **Decisions are recorded** in the Execution's state under `decisions`: turn, group, action and what was judged, with confidence.
7. **The reply writer.** A new role, `agents/reply-writer.md`, writes the Approval summary from facts (`approval_facts`) through one tool. When it fails or is absent, `assembled_summary` builds the summary from the same facts.
8. **Wiring.** `ORCHESTRATOR=router` (the default) or `model`; `open_assembly` builds the router with the decider and the thresholds (`JEV_ACT_THRESHOLD`, `JEV_CLOSE_THRESHOLD`, `JEV_CONFIDENCE_THRESHOLD`). State gained `last_reply`, `pending_change`, `awaiting_gap_decision`, `validation_settled` and `decisions`.
Changed from the plan and the ADR's first wording:
- **Jev does not propose the next action.** It supplies readings, and the playbook in code picks the action. This follows the owner's triage model and keeps every decision explainable by a rule and a threshold.
- **The model Orchestrator is kept** behind `ORCHESTRATOR=model`. The 40 or so engine tests drive the Ledger's rules by scripting its tool calls in arbitrary order, which the router cannot do. `agents/orchestrator.md` is therefore unchanged and unused in router mode.
- **An unclear change is confirmed once.** If the reply still does not settle it, a named Deliverable is acted on and anything else is dropped.
- **A pre-approved Execution's automatic replies are not routed.** There is no user message to read.
Not built from the ADR's routes:
- **A message that starts something** (new request, Change request, question about delivered work, out of scope). It needs an entry point that can answer from a case's files, and Change requests (iteration 7).
- **Which Deliverables a Change request reaches.** Iteration 7.
Tests: `tests/execution/test_router.py` (18), `tests/test_assembly.py`.
Unverified:
- Whether Jev's answers to these five questions are good enough at these thresholds. The thresholds are the ones in the owner's example and are untuned.
- Whether the reply writer's summaries are accurate. It is instructed to state only the facts it is given; nothing checks that it does.
- Run time. The Orchestrator made 31 model requests in the last run; the router makes none, and one small writer call per Approval. No run has measured it.

View File

@@ -0,0 +1,44 @@
# Step 2 — Implementation Detail
Built ahead of step 1's answers, on the owner's instruction of 2026-10-07 that no evaluation be rerun until the Jev work is in. Nothing here has called the live service.
1. **The seam** (`app/adk/runtime.py`): `Question` (kind choice, score or noul, with instructions and options), `Answer` (value, confidence, probabilities) and the `Decider` protocol. `decide` returns the answers, or `None` when no decision could be made, and the caller then takes its fallback. `NoDecider` always returns `None`.
2. **The client** (`app/tools/jev.py`): `JevDecider` calls Jev through TypeSafe AI's official Python SDK, `typesafe-sdk` 0.7.2 (`AsyncTypeSafeClient.system_one`). The owner chose the SDK and its API endpoint on 2026-10-07. It is the only module that imports the SDK.
3. **Wiring:** `open_decider` in `app/agent.py` returns the client when `TYPESAFE_API_KEY` is set and `NoDecider` otherwise; the `Assembly` carries it.
4. **A scripted decider** for tests, `FakeDecider` in `app/adk/fake_runtime.py`.
Which package: three names exist on the index. `typesafe-sdk` is the vendor's (author TypeSafe AI, repository `typesafe-ai/typesafe-sdk-python`). `typesafe` is an unrelated type-checking library, and `typesafe-ai` is a third party's redirect shim; neither is used.
What the SDK settled, read from the installed package and a stand-in transport:
- The request is `POST /v1/systemone` with a bearer key; default endpoint `https://api.typesafe.ai`, default model `jev-latest`.
- Questions are `Noul(instructions)`, `Choice(instructions, criteria={option: description})` and `Score(instructions, criteria=[levels])`.
- Answers are `noul` (a probability, with no confidence of its own), `choice` with `confidence` and `probabilities`, and `score` with `confidence`. Usage reports `input_tokens` only: no cost and no remaining credit.
Limits kept, each with a test in `tests/tools/test_jev.py`:
| Practice | Built as |
|---|---|
| Size | Refused locally above 48,000 estimated tokens in all, or 24,000 for the state plus the longest question. Never truncated. |
| Batching | Every question about one state goes in one request. |
| Timeout | 5 seconds a request, 15 for a decision with its retries (the SDK's). |
| Retries | 2, by the SDK: on 408, 429, 5xx, connection and timeout errors, with jittered backoff, honouring `Retry-After`. |
| Rate | A token bucket, 5 a second with a burst of 10; the rate halves when a rate limit outlasts the retries. |
| Concurrency | At most 8 requests in flight. |
| Circuit breaker | Opens after 5 failures in 30 seconds for 60 seconds, then lets one probe through. |
| Cache | Answers kept by a hash of the state and questions, up to 1,024. |
| Budget | 500 calls or 1,000,000 input tokens per scope (the Execution), then fallbacks with one warning. |
| Records | Each decision's outcome, seconds, rate limiting, input tokens and confidence; `summary()` totals them. |
| Key | From `TYPESAFE_API_KEY` only; never logged. |
Settings: `TYPESAFE_API_KEY`, `TYPESAFE_BASE_URL`, `TYPESAFE_DEFAULT_MODEL` (the SDK's own names), and `JEV_TIMEOUT_SECONDS`, `JEV_REQUESTS_PER_SECOND`, `JEV_MAX_IN_FLIGHT`, `JEV_BUDGET_CALLS`, `JEV_BUDGET_TOKENS`.
Changed from the plan:
- The cache is per client, not per Execution. A decision depends only on its state and questions, so two Executions asking the same thing can share the answer.
- A reply that leaves any question unanswered, or picks a choice that was not offered, counts as no decision for the whole request.
- The budget is in calls and input tokens, not dollars, and there is no low-credit warning: the SDK reports neither cost nor balance.
- A yes-or-no answer's confidence is derived as its distance from an even probability, because Jev returns none.
- Retries and their backoff are the SDK's, configured here, not reimplemented.
Unverified: the client has run only against a stand-in transport, with the real SDK in the loop. The first call with a real key is still to be made, and the service's actual rate limits, latency and data retention are still unknown.