Files
gcp_solution_architecture_a…/.agents/plans/iter-05-jev-decisions/steps/step-2.md
Jonathan Boniface 5bb22d984e
Some checks failed
validation / verify (push) Failing after 8s
fixed multile issues
2026-10-07 17:33:36 +01:00

4.0 KiB

Step 2 — Implementation Detail

Built ahead of step 1's answers, on the owner's instruction of 2026-10-07 that no evaluation be rerun until the Jev work is in. Nothing here has called the live service.

  1. The seam (app/adk/runtime.py): Question (kind choice, score or noul, with instructions and options), Answer (value, confidence, probabilities) and the Decider protocol. decide returns the answers, or None when no decision could be made, and the caller then takes its fallback. NoDecider always returns None.
  2. The client (app/tools/jev.py): JevDecider calls Jev through TypeSafe AI's official Python SDK, typesafe-sdk 0.7.2 (AsyncTypeSafeClient.system_one). The owner chose the SDK and its API endpoint on 2026-10-07. It is the only module that imports the SDK.
  3. Wiring: open_decider in app/agent.py returns the client when TYPESAFE_API_KEY is set and NoDecider otherwise; the Assembly carries it.
  4. A scripted decider for tests, FakeDecider in app/adk/fake_runtime.py.

Which package: three names exist on the index. typesafe-sdk is the vendor's (author TypeSafe AI, repository typesafe-ai/typesafe-sdk-python). typesafe is an unrelated type-checking library, and typesafe-ai is a third party's redirect shim; neither is used.

What the SDK settled, read from the installed package and a stand-in transport:

  • The request is POST /v1/systemone with a bearer key; default endpoint https://api.typesafe.ai, default model jev-latest.
  • Questions are Noul(instructions), Choice(instructions, criteria={option: description}) and Score(instructions, criteria=[levels]).
  • Answers are noul (a probability, with no confidence of its own), choice with confidence and probabilities, and score with confidence. Usage reports input_tokens only: no cost and no remaining credit.

Limits kept, each with a test in tests/tools/test_jev.py:

Practice Built as
Size Refused locally above 48,000 estimated tokens in all, or 24,000 for the state plus the longest question. Never truncated.
Batching Every question about one state goes in one request.
Timeout 5 seconds a request, 15 for a decision with its retries (the SDK's).
Retries 2, by the SDK: on 408, 429, 5xx, connection and timeout errors, with jittered backoff, honouring Retry-After.
Rate A token bucket, 5 a second with a burst of 10; the rate halves when a rate limit outlasts the retries.
Concurrency At most 8 requests in flight.
Circuit breaker Opens after 5 failures in 30 seconds for 60 seconds, then lets one probe through.
Cache Answers kept by a hash of the state and questions, up to 1,024.
Budget 500 calls or 1,000,000 input tokens per scope (the Execution), then fallbacks with one warning.
Records Each decision's outcome, seconds, rate limiting, input tokens and confidence; summary() totals them.
Key From TYPESAFE_API_KEY only; never logged.

Settings: TYPESAFE_API_KEY, TYPESAFE_BASE_URL, TYPESAFE_DEFAULT_MODEL (the SDK's own names), and JEV_TIMEOUT_SECONDS, JEV_REQUESTS_PER_SECOND, JEV_MAX_IN_FLIGHT, JEV_BUDGET_CALLS, JEV_BUDGET_TOKENS.

Changed from the plan:

  • The cache is per client, not per Execution. A decision depends only on its state and questions, so two Executions asking the same thing can share the answer.
  • A reply that leaves any question unanswered, or picks a choice that was not offered, counts as no decision for the whole request.
  • The budget is in calls and input tokens, not dollars, and there is no low-credit warning: the SDK reports neither cost nor balance.
  • A yes-or-no answer's confidence is derived as its distance from an even probability, because Jev returns none.
  • Retries and their backoff are the SDK's, configured here, not reimplemented.

Unverified: the client has run only against a stand-in transport, with the real SDK in the loop. The first call with a real key is still to be made, and the service's actual rate limits, latency and data retention are still unknown.