feat: lessons from what recurs about one setting, and three more approved
Some checks failed
validation / verify (push) Failing after 10s

Evidence was grouped only by the kind of problem, and a group gets one lesson,
so what kept recurring inside "validation proposed a Check" could not be
seen. A proposed or unimplementable Check is now also counted under each
provider setting or resource type it names; a subject seen in two or more
Executions is its own group and reaches the curator past triage.

Over 32 Executions that gave five proposals. The owner approved three, now
published: L-07 location Checks, L-08 Cloud SQL backup encryption, L-09
backend service capabilities. L-10 is held for more evidence and L-11
rejected.

Also: retiring a lesson marks its proposal, so publish no longer brings a
retired lesson back. L-04 had returned to the Skill that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-10 12:47:18 +01:00
parent ee1ba38867
commit 5feaaa6271
17 changed files with 354 additions and 13 deletions

View File

@@ -81,3 +81,17 @@ Neither run raised the schema review's false alarm, which had been raised in cas
- **`status` says "compared", not "proved",** and shows a retired lesson as retired. One pair of runs is a comparison.
Tests: `tests/test_lessons_loop.py`.
## Evidence is also grouped by what it is about (2026-10-10)
Running `propose` over 32 Executions gave nothing useful. Evidence was grouped by the kind of problem, and a group gets one lesson: "validation proposed a Check the design lacked" was 20 of 32 Executions and already had L-01. What recurred inside it could not be seen.
- A proposed Check, and a Check sent back as unimplementable, is now also counted under each provider setting or resource type its text names (`_subjects` in `app/learning/evidence.py`). A subject seen in two or more Executions is its own group, with its own key. A setting's name is the provider's vocabulary, not a client's.
- These narrower groups skip triage: each recurred with the general lessons published. The curator is shown the existing lessons and may still decline.
- The curator's instructions say such a lesson may name the setting or resource type.
Over the same 32 Executions this gave 22 groups and five proposals, L-07 to L-11, all awaiting the owner: location Checks, Cloud SQL backup encryption, backend service capabilities, Pub/Sub persistence regions, and customer-managed keys.
A first run of `propose` before this change produced one proposal, about giving `edit_file` enough context. It was removed to run again cleanly and was not proposed the second time.
Tests: `tests/test_lessons_loop.py` (3 more).