feat: overlay Skills, and five more Jev decisions (iterations 4 and 5)
Iteration 4: - Overlay Skills (step 12): an Execution names an organisation's standards Skill from the folders the deployment configures; recorded with its version. An example standards Skill and an evaluation case that uses it. - Lessons (step 10): a prove command that compares a run with lessons against one without, and triage of evidence an existing lesson already covers. Iteration 5: - Step 8: the references of a Specialist's Skills its task needs are named in its context as read_first. - Step 14: a remote MCP server's reply that instructs the agent is withheld; the user's own text is flagged as a risk and nothing more. - Step 15: in a Change request, items only reworded are listed apart from real changes. - Step 19: each Execution's design recommendations are judged when it ends and the scores kept in its record. The evaluation takes its rubric from the same place. Each question was calibrated against live Jev; none has run in a case. Steps 6, 7, 9, 13 and 18 stay planned, with the reasons in the plan map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
19
.agents/plans/iter-04-learning-and-skills/steps/step-12.md
Normal file
19
.agents/plans/iter-04-learning-and-skills/steps/step-12.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Step 12 — Implementation Detail
|
||||
|
||||
Built on 2026-10-09. **Not run with real models.**
|
||||
|
||||
1. An overlay Skill is an ordinary Agent Skill in a folder the deployment names: `OVERLAY_SKILLS_DIRS` (default `skills/overlays`). `discover_overlays` in `app/agent.py` reads them with the same validation as the provider's Skills and refuses one that reuses a provider Skill's name.
|
||||
2. `ExecutionEngine.start(overlay_skills=[...])` names the overlays for one Execution. `DEFAULT_OVERLAY_SKILLS` names the ones every Execution loads. An unknown name, or anything but a list of names, refuses the start before anything runs or is recorded. A caller names an overlay; nothing lets it supply one.
|
||||
3. Every Specialist is given the Execution's overlays on top of its own Skills, and its context lists them as `overlay_skills`. The protocol says an overlay wins over the provider's general guidance, and a requirement the user stated wins over an overlay.
|
||||
4. The Execution records each overlay with the content hash it had: in its state, in `execution.json` and in its lesson candidates.
|
||||
5. Callers: `overlay_skills` in the `/generate` body, in A2A message metadata, and on an evaluation case.
|
||||
6. `skills/overlays/example-organisation-standards/` is an example: naming, four mandatory labels, approved regions, three prohibitions. It is marked as an example, is not loaded by default, and is not any real organisation's policy.
|
||||
|
||||
Differs from the plan:
|
||||
|
||||
- **Skills are still loaded once at start-up.** The runtime loads every provider Skill and every overlay in the configured folders when it starts; an Execution chooses among them. An overlay added to the folder needs a restart.
|
||||
- **No organisation standards Skill is loaded by default.** The plan's criterion "loaded by the roles the bundle lists" is met by naming it, not by the bundle: whose standards apply is the deployment's and the caller's to say.
|
||||
|
||||
`case-06-organisation-standards` in the evaluation dataset runs case 03's request under the example overlay. It has not been run.
|
||||
|
||||
Tests: `tests/execution/test_overlay_skills.py` (8).
|
||||
Reference in New Issue
Block a user