feat: prove the lessons, check overlays in evaluation, and settle the cost steps
Some checks failed
validation / verify (push) Failing after 13s

- Schema review: a dotted path is judged only when it starts at a name the
  resource type has, so a database flag or a service name is no longer read
  as a missing setting. Lesson L-04, which worked around that, is retired.
- Skill routing: in a Change request the question says what changed.
- Evaluation: a case may list code_standards the infrastructure code must
  meet; case 06 checks its overlay's regions, naming and labels.
- Model tiers (iteration 5 step 6): a lighter model for a role's first
  attempt, named in the provider bundle. Measured on case 03 and left off:
  Flash saved five cents on the IaC author and took twice the requests.
- Steps 7 and 9 decided against on the numbers, recorded in the plans.
- Plans: the runs of cases 05 and 06, and the first proof of the lessons on
  case 03 (no Revision, 22 minutes against 28, quality held).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 21:27:09 +01:00
parent 567aede5ca
commit e5e1647ff1
35 changed files with 497 additions and 39 deletions

View File

@@ -45,6 +45,9 @@ class ModelConfig:
# Seconds one model request may take, for the roles whose replies are long. A
# role not named here gets the runtime's default.
timeouts: Mapping[str, float] = field(default_factory=dict)
# A lighter model for a role's first attempt at a Deliverable. Only for a role
# whose work something else checks; an attempt after feedback takes its own.
first_attempt: Mapping[str, str] = field(default_factory=dict)
def for_role(self, role: str) -> str:
return self.roles.get(role, self.default)
@@ -131,6 +134,10 @@ def load_provider(name: str, root: Path | None = None) -> ProviderBundle:
default=models["default"],
roles=dict(models.get("roles") or {}),
timeouts=timeouts,
first_attempt={
str(role): str(model)
for role, model in (models.get("first_attempt") or {}).items()
},
)
terraform_providers = {}