Some checks failed
check agents / check (push) Failing after 12s
- Critic 0.2.0 / 0.2.1: K-04 (risk), K-05 (confidence), K-06 (TCO), K-07 (routing) and K-11 (schema and 7R enum) run in code in the critic's hooks (app/arithmetic.py, app/business_logic.py); post_process sets those checks, the rejected dispositions, the sign-off queue and the verdict. The model keeps the judgement checks; a different reading of a code check is kept in unverified_claims. K-07 checks the disposition agent's own queue. Tests on the records of run exec_4d88027a4d26, whose model sums were wrong. - A represents edge is not a dependency (ADR 0012): risk scorer 0.1.1 and disposition 0.1.2 leave it out of dependents and dependencies; the disposition and critic prompts carry the risk scorer's rule. Run exec_1f518b04d6e4: APP-001's risk 1.00 -> 0.00, out of the tail. - The scaffold declares the critic's hooks, copies app/ in its Dockerfile, and keeps its app/ and tests/ when regenerating. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.5 KiB
3.5 KiB
0011. The critic's arithmetic, routing and schema checks run in code; the model judges the rest
| Status | Accepted |
| Date | 2026-11-05 |
| Decided by | Product owner, 2026-11-05: "yes" (do the arithmetic in code; leave the judgement to the model) |
| Requirements changed | ALZ-ASM-06: K-04, K-05, K-06, K-07, K-11 and the verdict are set in code. Refines ADR 0001: the critic is generated from its story but carries hooks, as the intake does (ADR 0008) |
Context
- ADR 0001 keeps the scoring rules as prompt rules, applied by the producing agents and recalculated by the critic.
- Run
exec_1779863642d6showed that the critic had none of the rules it recalculates with; critic 0.1.6 copies them into its prompt. Runexec_4d88027a4d26then showed the critic applying the right rules and getting the sums wrong:- K-04: it expected a risk score of 0.70; with one factor available (fan-in 1.0, weight 0.25), the renormalised score is 1.00, as the risk scorer recorded;
- K-05: it listed 0.90 - 0.10 - 0.20 - 0.10 and called it 0.60; it is 0.50, as the disposition agent recorded.
- The cluster's other model,
gpt-4o, is weaker, andgpt-5.6-solruns the executors (K-12, ADR 0010). - These checks are sums, lookups and set equality over the producers' own records. They need no judgement.
Decision
- Code checks, in the critic's hooks (
app/arithmetic.py,app/business_logic.py):- K-04: each risk score from its factors and the fixed weights, renormalised over the available ones; each
contribution;
partial;missing_factors; the 0.70 tail rule;high_riskand the high-risk tail. - K-05: each confidence from its workings: the 0.90 base, each adjustment's prescribed delta (P2 at -0.10 a
fact, -0.20 at most), the C1 cap for repurchase and relocate, the result,
signoff_required. - K-06: each line item's rate id in the rate card at its annual rate; current, target and delta sums; a missing
cost named;
indicative. - K-07: the sign-off queue is the dispositions below 0.80 confidence, less the rejected ones.
- K-11: the input contract validated every record on arrival; the 7R enum is checked again.
- K-04: each risk score from its factors and the fixed weights, renormalised over the available ones; each
contribution;
pre_processworks these out before the model runs and shows them to it aschecked_in_code.post_processsets those checks, the rejected dispositions (K-05, K-06, K-11), the sign-off queue and the verdict:failif any check fails or any check other than K-10 is unverified (ADR 0006).- The model keeps the judgement: K-01 to K-03, K-08 to K-10 and K-12, and, inside the code checks, whether a
factor's value or an adjustment was due. Where its reading of a code check differs, the reading is kept in
unverified_claims, so the Engagement Lead sees it. - When the planner refused to start (
preconditions.failed), the hooks change nothing: the model returnsfailon K-01 alone. - The scaffold keeps the critic's
app/andtests/when it regenerates, and declares the hooks.
Consequences
- The verdict follows from the checks in code, so a sum the model gets wrong no longer fails a run, and one the producer gets wrong always does.
- Whether a rule applied is still a model judgement. The code checks that P2 is -0.10 a fact, not that the facts were unknown, and that a rate id exists at its rate, not that it is the right equivalent.
- The critic is no longer purely declarative. Its code is small, standard library only, and tested on the
records of a live run (
tests/test_arithmetic.py).