feat: prove the lessons, check overlays in evaluation, and settle the cost steps
Some checks failed
validation / verify (push) Failing after 13s

- Schema review: a dotted path is judged only when it starts at a name the
  resource type has, so a database flag or a service name is no longer read
  as a missing setting. Lesson L-04, which worked around that, is retired.
- Skill routing: in a Change request the question says what changed.
- Evaluation: a case may list code_standards the infrastructure code must
  meet; case 06 checks its overlay's regions, naming and labels.
- Model tiers (iteration 5 step 6): a lighter model for a role's first
  attempt, named in the provider bundle. Measured on case 03 and left off:
  Flash saved five cents on the IaC author and took twice the requests.
- Steps 7 and 9 decided against on the numbers, recorded in the plans.
- Plans: the runs of cases 05 and 06, and the first proof of the lessons on
  case 03 (no Revision, 22 minutes against 28, quality held).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 21:27:09 +01:00
parent 567aede5ca
commit e5e1647ff1
35 changed files with 497 additions and 39 deletions

View File

@@ -20,7 +20,15 @@ Seen in 14 of 21 Executions, 58 times in all: validation proposed a Check the de
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments

View File

@@ -20,7 +20,15 @@ Seen in 13 of 21 Executions, 13 times in all: submit_deliverable refused: Not su
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments

View File

@@ -20,7 +20,15 @@ Seen in 3 of 21 Executions, 3 times in all: revised because a Check named someth
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments

View File

@@ -20,8 +20,18 @@ Seen in 2 of 21 Executions, 4 times in all: submit_deliverable refused: Not subm
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments
Approved by the owner on 2026-10-09.
- 2026-10-09: retired. The schema review no longer reads a value written with dots (a database flag, a service name) as a setting, so the rule has nothing left to work around.

View File

@@ -20,7 +20,15 @@ Seen in 2 of 21 Executions, 2 times in all: terraform_validate refused: terrafor
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments

View File

@@ -20,7 +20,15 @@ Seen in 2 of 21 Executions, 2 times in all: submit_deliverable refused: Not subm
## Proof
Not yet measured.
Evaluation without lessons: `20261009T195603Z`; with: `20261009T195607Z`.
| Case | Score | Revisions | Minutes | Tokens (M) |
| --- | --- | --- | --- | --- |
| case-03-internal-web-app | 1.00 → 1.00 | 1 → 0 | 28 → 22 | 3.00 → 2.32 |
Quality held and revisions, seconds, tokens fell: kept.
This compares every published lesson together, on one run each way. Runs of the same case vary, so one pair suggests a direction and does not settle it.
## Comments