Files
gcp_solution_architecture_a…/.agents/plans/iter-05-jev-decisions/steps/step-3.md
Jonathan Boniface e5e1647ff1
Some checks failed
validation / verify (push) Failing after 13s
feat: prove the lessons, check overlays in evaluation, and settle the cost steps
- Schema review: a dotted path is judged only when it starts at a name the
  resource type has, so a database flag or a service name is no longer read
  as a missing setting. Lesson L-04, which worked around that, is retired.
- Skill routing: in a Change request the question says what changed.
- Evaluation: a case may list code_standards the infrastructure code must
  meet; case 06 checks its overlay's regions, naming and labels.
- Model tiers (iteration 5 step 6): a lighter model for a role's first
  attempt, named in the provider bundle. Measured on case 03 and left off:
  Flash saved five cents on the IaC author and took twice the requests.
- Steps 7 and 9 decided against on the numbers, recorded in the plans.
- Plans: the runs of cases 05 and 06, and the first proof of the lessons on
  case 03 (no Revision, 22 minutes against 28, quality held).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:27:09 +01:00

2.5 KiB

Step 3 — Implementation Detail

Built on 2026-10-09. Not run in an Execution with real models.

  1. pillars_touched in app/tools/pillars.py asks one graded question per pillar: how directly does this change affect the pillar's recommendations (not at all, only indirectly, directly). A pillar at JEV_PILLAR_THRESHOLD (1.5 on that 0 to 2 scale) or above is consulted again. An empty list means the change touches none; no decision means consult them all.
  2. When the design advisor is started on a Revision, or in a Change request, and advice from an earlier consultation exists, the engine asks which pillars the change touches (_pillar_hints in app/nodes/execution_nodes.py) and puts the answer in its context as pillars_to_consult. The change is the Deliverable's feedback, or in a Change request the requirements added, changed and removed.
  3. consult_pillar_advisors with an empty list consults nobody and returns the advice already given.
  4. A completed Execution writes working.json: each pillar's advice and checklist review. A Change request starts from its Baseline's, so the pillars a change does not touch are not consulted again. In the first Change request, before this, the design advisor consulted all six afresh and that was its longest step.

Differs from the plan: the plan asked a yes or no per pillar. Measured against live Jev on six changes, that was yes for nearly every pillar every time (0.39 to 0.96 across the five real changes). The graded form separates them:

Change Pillars at "directly"
Retention to 7 years, undeletable security, reliability, operations, cost, sustainability
Survive the loss of a region reliability, operations, cost
Budget halved operations, cost, performance, sustainability
Page load 1 s to 300 ms operations, cost, performance
A page title renamed none
Customer-managed keys for one bucket security

Operations and cost are picked for almost every real change, so the saving is two to five advisors of six, not five of six.

Tests: tests/execution/test_pillars.py (3), tests/execution/test_change_requests.py (2).

Run on 2026-10-09

case-05-change-retention (results 20261009T171336Z): the Change request consulted 5 pillar advisors, not 6, which is what calibration predicted for this change. The Change request took about 18 minutes and 1.30 million tokens against the Baseline's 30 minutes and 3.09 million (42%), passed every change check, and changed nothing unrequested (0 of 89).