feat: working TOM agent

This commit is contained in:
2026-10-21 19:47:12 +01:00
commit efe543e6e2
83 changed files with 11779 additions and 0 deletions

View File

@@ -0,0 +1,38 @@
# Steps T3, T4, T11 and T14: the digest, the guide, and the first runs with real models
Built 2026-10-13 and 14, with the engine's digest step (its `steps/step-6.md`).
## What was built
- **The architecture adapter** (`app/operating_model/digest.py`, `python -m app.operating_model.digest <folder>`). An Item is a component of the architecture, with the Elements that realise it; a Condition is what its requirements and recommendations say can only be shown in operation (`outside_code`); a Check that did not pass is an inherited risk. Only what the customer stated in the architecture's scope is carried across. The provider bundle's `operating.item_kinds` says what kind of thing each product is. Tested against a real completed landing-zone Execution, copied into `tests/fixtures/architecture/`.
- **Documents code cannot read** go to the scope analyst, who records what it takes from them with the passage quoted (T4, as instructions; no model has run it).
- **The guide** (`app/operating_model/guide.py`, `skills/operating-guide/`). The packager writes the summary; code writes every other section and the key figures from the data.
- **The evaluation harness** (`eval/`): two cases, structural scoring, tokens per role.
## Differed from the plan
- An Item is a component, not an Element. The real architecture had 60 Elements in 9 components; an organisation operates the components.
- The architecture's 104 Checks on infrastructure settings are not Conditions. Keeping them true is configuration management. Its 12 statements of what can only be shown in operation are.
## The runs with real models, 2026-10-14
| Case | Result | Minutes | Revisions | Capabilities | Roles | Checks | Scenarios | People | Cost a month |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 01, written spec (4 Items) | pass | 25.8 | operating model ×2, capability model ×1 | 17 | 4 | 8 | 13 of 13 | 5.2 | 25,355 GBP |
| 02, digested architecture (9 Items) | pass | 16.8 | operating model ×1 | 17 | 5 | 36 | 31 of 31 | 11.6 | 67,298 GBP |
Costs are from the placeholder rate card. An earlier run of case 01 passed every structural check in 30.6 minutes, over its limit then of 30.
What they showed:
- **Most time goes on the architect placing a Check on the wrong Capability**, such as a restore-time Check on service-level reporting. Case 01 revised the same way in both runs. Not yet acted on.
- **The router's reasons still speak of code.** A Check that could not be verified reopens the capability model with "could not be verified from the code", and a Check that failed twice is said to be possibly "not implementable". These are the engine's words from the architecture's subject.
- **The capability modeller takes all 17 catalogue entries** whatever the size of the solution, as its instructions say.
- **The architect is most of the cost:** about 380,000 to 490,000 of each case's 780,000 to 940,000 prompt tokens.
- One request was retried after Vertex AI's tokens-per-minute quota was reached; it did not affect the result.
Nobody has yet read the outputs for quality: these checks are structural only.
## Verified
The engine's 529 tests and this repository's 140 pass, with `ruff` and `mypy` clean in both.