# Step 8 — Implementation Detail This step has no entry in the spec's comments; this file is its record. 1. Added `eval/solution_eval.py`: runs each case as a pre-approved Execution through the full assembly with real models, counting agent runs, outcomes, seconds and tokens per role. 2. Added `eval/scoring.py` with 8 checks: completed, guide follows the template, required products selected, alternatives listed, documentation cited, requirements name no products, diagram is valid Mermaid, Terraform validates. 3. Added 3 cases in `eval/datasets/solution_cases.json`. Results are written to `eval/results//` (gitignored). Recorded runs: | Run | Case | Score | Seconds | Revisions | |---|---|---|---|---| | 2026-09-16 10:57Z | case-01-event-driven-http | 1.0 | 2,932 | design_recommendations × 2 | | 2026-09-17 08:44Z | case-03-internal-web-app | 1.0 | 2,623 | deployment_guidance × 3 | No scored run of `case-02-high-scale-ingestion` is on record. Case 03 used 3.27 million tokens over 127 model requests. By role: IaC author 812 thousand, validator 618 thousand, describer 513 thousand, Orchestrator 356 thousand, product selector 345 thousand. These figures are the evidence base for iterations 4 and 5. All 8 checks are structural, so a better or worse Skill scores the same. Iteration 4 step 8 addresses that. Suite at the end of this step: 226 passed.