feat: prove the lessons, check overlays in evaluation, and settle the cost steps
Some checks failed
validation / verify (push) Failing after 13s
Some checks failed
validation / verify (push) Failing after 13s
- Schema review: a dotted path is judged only when it starts at a name the resource type has, so a database flag or a service name is no longer read as a missing setting. Lesson L-04, which worked around that, is retired. - Skill routing: in a Change request the question says what changed. - Evaluation: a case may list code_standards the infrastructure code must meet; case 06 checks its overlay's regions, naming and labels. - Model tiers (iteration 5 step 6): a lighter model for a role's first attempt, named in the provider bundle. Measured on case 03 and left off: Flash saved five cents on the IaC author and took twice the requests. - Steps 7 and 9 decided against on the numbers, recorded in the plans. - Plans: the runs of cases 05 and 06, and the first proof of the lessons on case 03 (no Revision, 22 minutes against 28, quality held). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
11
.agents/plans/iter-05-jev-decisions/steps/step-9.md
Normal file
11
.agents/plans/iter-05-jev-decisions/steps/step-9.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Step 9 — Decision
|
||||
|
||||
Decided on 2026-10-09: **not to be built as planned.** It would be likely to raise cost, not lower it.
|
||||
|
||||
The plan drops or truncates old tool results inside one agent run before each model request.
|
||||
|
||||
- **The prompt is already mostly cached.** In case 06 the validator's prompt was 82% cached, the target architect's 69% and the IaC author's 60%, and a cached token costs a tenth of an uncached one.
|
||||
- **Caching depends on the prompt's beginning staying the same.** Removing or shortening an earlier tool result changes everything after it, so the request that follows is paid at the full price.
|
||||
- **The arithmetic.** A request of 46 thousand tokens, 82% cached, costs about $0.024 on the Pro model. Cut to 30 thousand with the cache lost, it costs $0.060.
|
||||
|
||||
It could still help where a run nears the model's context limit, which no run has. The cost that remains in these roles is output and thinking tokens, and the number of requests; neither is what this step addresses.
|
||||
Reference in New Issue
Block a user