feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)

Iteration 4:
- Overlay Skills (step 12): an Execution names an organisation's standards
  Skill from the folders the deployment configures; recorded with its version.
  An example standards Skill and an evaluation case that uses it.
- Lessons (step 10): a prove command that compares a run with lessons against
  one without, and triage of evidence an existing lesson already covers.

Iteration 5:
- Step 8: the references of a Specialist's Skills its task needs are named in
  its context as read_first.
- Step 14: a remote MCP server's reply that instructs the agent is withheld;
  the user's own text is flagged as a risk and nothing more.
- Step 15: in a Change request, items only reworded are listed apart from
  real changes.
- Step 19: each Execution's design recommendations are judged when it ends and
  the scores kept in its record. The evaluation takes its rubric from the same
  place.

Each question was calibrated against live Jev; none has run in a case. Steps
6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 17:45:56 +01:00
parent 844987587c
commit 567aede5ca
38 changed files with 1956 additions and 58 deletions

View File

@@ -0,0 +1,27 @@
# Step 8 — Implementation Detail
Built on 2026-10-09, on by default (`JEV_SKILL_ROUTING`). **No Execution has run with it.**
1. `app/tools/skill_routing.py`. A Skill with four or more reference files is one worth routing within; each reference is described by its title and first paragraph. Today that is `google-cloud-iac-patterns` (14 references), which only the IaC author loads.
2. When a Specialist whose Skills include such a Skill is started, the engine asks one graded question per reference: how much does this task need a reference on this (not needed, might help, needed). The task is the Deliverable, any feedback, and the Target architecture's Elements with their products and resource types.
3. References at `JEV_SKILL_THRESHOLD` (1.25 on that 0 to 2 scale) or above are named in the Specialist's context as `read_first`. The protocol says to read those first, and that it is a place to start, not a limit.
4. With no decision nothing is named, and the Specialist searches its Skills as before.
Measured against live Jev on four tasks, each reference scored 0 to 2:
| Task | Named at 1.25 | Highest not named |
|---|---|---|
| Web app: Cloud Run, Cloud SQL, Secret Manager, load balancer, IAM, VPC | service, SQL, secrets, load balancer, IAM, networking, project services, Artifact Registry | KMS 0.72 |
| Nightly batch: Cloud Run job, Scheduler, Storage, IAM | job, scheduler, storage, IAM, project services, Artifact Registry | Secret Manager 0.90 |
| Events: Pub/Sub, Cloud Run, Artifact Registry, KMS, Monitoring | all seven it needs | Secret Manager 1.09 |
| A Revision: one failed Check on a bucket's retention | storage only | project services 0.96 |
The Cloud Run service pattern scored 1.39 and 1.42 where it was needed, which is why the threshold is 1.25 and not 1.5. The cost of that is the Artifact Registry pattern being named for two designs that run containers and did not list a registry.
Differs from the plan:
- **A graded question per reference, not one Choice.** A task needs several references, and a Choice names one.
- **Overlay Skills are not selected by Jev.** The caller or the deployment names them (iteration 4 step 12): which organisation's standards apply is not something to infer.
- **Not measured against its acceptance criterion.** Whether `load_skill_resource` calls fall needs a case run with it and one without.
Tests: `tests/execution/test_skill_routing.py` (5).