feat: working orhcestration engine
This commit is contained in:
12
.agents/plans/iter-01-execution-engine/steps/step-1.md
Normal file
12
.agents/plans/iter-01-execution-engine/steps/step-1.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Step 1: the repository
|
||||
|
||||
Built 2026-10-13.
|
||||
|
||||
- `pyproject.toml`: the package `execution-engine`, importable as `execution_engine`, with the Reference's dependency pins and a `dev` extra. Tool caches go under `.cache/`.
|
||||
- `CONTEXT.md`, `CLAUDE.md`, `README.md`.
|
||||
- `docs/adr/0001` and `0002`, both `proposed`. The owner has not accepted them.
|
||||
- `scripts/bring.py` and `REFERENCE.lock.json`.
|
||||
|
||||
Differed from the plan: nothing.
|
||||
|
||||
Verified: `pip install -e .` succeeds in `.venv`; `import execution_engine` works.
|
||||
32
.agents/plans/iter-01-execution-engine/steps/step-2.md
Normal file
32
.agents/plans/iter-01-execution-engine/steps/step-2.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Step 2: the modules that depend on nothing about a subject
|
||||
|
||||
Built 2026-10-13, from the Reference at `8b397cd`.
|
||||
|
||||
## Brought
|
||||
|
||||
Twenty-five modules, 4,468 lines, copied by `scripts/bring.py` with imports renamed, then put through `ruff check --fix` and `ruff format` because the longer package name re-wraps import lines. Nothing else in them was changed.
|
||||
|
||||
`config`, `checkpointing`, `database`, `customer_case`, `baseline`, `scope`, `providers`, `adk/runtime`, `adk/adk_runtime`, `skills/agent_skills`, `states/execution_state`, `tools/jev`, `tools/guardrail`, `tools/workspace`, `tools/skill_routing`, `tools/mcp`, `learning/proposals`, `learning/proof`, `learning/skill`, and the package `__init__` files.
|
||||
|
||||
## Differed from the plan
|
||||
|
||||
The plan listed more modules as usable as they are. Reading their imports showed these depend on the Reference's Deliverable graph or its file formats, so they wait for step 3: `states/ledger`, `recorder` (through the Ledger), `adk/fake_runtime`, `states/lessons`, `nodes/intake` (its tests need the fake runtime), `workflows/execution_graph`, `tools/scoping`, `tools/meaning`, `learning/curator`, `learning/evidence`, `main`.
|
||||
|
||||
## Tests
|
||||
|
||||
Six test files brought, 85 tests. Changed after bringing:
|
||||
|
||||
| File | Change | Why |
|
||||
| --- | --- | --- |
|
||||
| `tests/agents/test_adk_runtime.py` | Skills come from `tests/fixtures/skills/` (`toy-design`, `toy-security`) in place of the Reference's bundle; two assertions on Skill text follow | The engine has no Skills of its own |
|
||||
| `tests/test_scope.py` | The two tests of the Reference's real catalogue and benchmark spec removed | They test the Reference's subject; they return with its example Workflow in step 5 |
|
||||
| `tests/tools/test_jev.py` | The two `open_decider` tests removed | `open_decider` is in `agent.py`; they return in step 3 |
|
||||
| `tests/tools/test_mcp.py` | The `gate_cloud_read` test removed | `gate_cloud_read` is in `tools/execution.py`; it returns in step 3 |
|
||||
|
||||
## Left for step 3
|
||||
|
||||
These brought modules still carry names from the Reference's subject (Terraform, pillars, Google Cloud, its Deliverables), counted by line: `providers` 26, `config` 18, `adk/adk_runtime` 18, `tools/mcp` 9, `adk/runtime` 4, `tools/workspace` 4, `baseline` 2, `states/execution_state` 2, `scope` 1, `database` 1. Some are comments and examples; `providers` and `config` hold real fields that belong to a Workflow. `config.BASE_DIR` and `providers._REPO_ROOT` resolve paths from the package's own location, which is wrong once the package is installed into an agent.
|
||||
|
||||
## Verified
|
||||
|
||||
`pytest -q`: 85 passed. `ruff check .`, `ruff format --check .` and `mypy .`: clean. `git status` in the Reference shows the same nine modified files as before this work began, none under `app/` or `tests/`.
|
||||
69
.agents/plans/iter-01-execution-engine/steps/step-3.md
Normal file
69
.agents/plans/iter-01-execution-engine/steps/step-3.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# Steps 3 and 5: the Workflow, and the Reference's workflow as its first user
|
||||
|
||||
Built 2026-10-13, from the Reference at `8b397cd`. The owner has not yet accepted ADR-0001, which this step builds.
|
||||
|
||||
## What was built
|
||||
|
||||
**The Workflow value** (`execution_engine/workflows/workflow.py`, new) and the generic Deliverable graph (`workflows/deliverables.py`, rewritten from the Reference's). The engine takes a Workflow as a value everywhere it used the Reference's graph: `Ledger(workflow, ...)`, `Ledger.from_dict(workflow, data)`, `specialist_tools(workflow, kind, ...)`, `make_tool_factory(workflow, runtime, roles, ...)`, `load_roles(agents_dir, bundle, workflow, ...)`, `sample_submission(workflow, kind)`, `ExecutionEngine(workflow=...)`, `open_assembly(settings, workflow, root=...)`, `create_app(workflow, ...)`.
|
||||
|
||||
What a Workflow supplies, beyond its Phases and Deliverables:
|
||||
|
||||
| Part | What it replaces in the Reference |
|
||||
| --- | --- |
|
||||
| `data_checks`, `baseline_checks`, `for_readers`, `spec_deliverable`, `final`, `final_file` | `app/workflows/deliverables.py` |
|
||||
| `validation` (`ValidationPolicy`: the kind, its file, what a failure reopens, `check_ids`, `unimplementable`, `known_ids`) | The router's validation group and `_gap_by_rule` |
|
||||
| `on_submit` | The three branches at the end of `submit_deliverable` |
|
||||
| `GraphShape` on a Deliverable | `graph_tools` and the hard-coded list names in `tools/workspace.py` |
|
||||
| `approval_details`, `approval_facts`, `summary_lines`, `describe_approval` | `request_approval`, `approval_facts`, `assembled_summary`, `describe_prompt` |
|
||||
| `agreed`, `describe_design`, `subject` | The requirements shown to the decider; the reading hints |
|
||||
| `changes` (`ChangePolicy`) | `change_summary`, `is_empty`, `settle_meaning`, `changes.json` |
|
||||
| `role_tools`, `roles`, `optional_roles`, `adjust_roles`, `prepare_role` | Pillar consultation; the pillar advisor and coverage reviewer; the modules Skill |
|
||||
| `task_entries`, `task_hints` | `terraform_modules` and `pillars_to_consult` in a Specialist's task |
|
||||
| `options` (`OptionSpec`) | `terraform_modules` on `start`, in state, in the manifest and in a Baseline |
|
||||
| `kept_for_changes`, `outputs`, `assess` | `working.json`, the customer questions file, the plan appended to the guide, the manifest's own entries, the assessment |
|
||||
| `local_tools`, `reviews` (given an `Assembling`) | The Terraform runner and the reviews in `open_assembly` |
|
||||
| `card`, `done`, `response`, `about` | The agent card, the completion message, `/generate`'s reply |
|
||||
|
||||
**The example** (`tests/reference_workflow/`): the Reference's own subject as a Workflow. `workflow.py` is the value, `kinds.py` its enums, `hooks.py` and `assembly.py` its code at the points above, and fourteen of the Reference's modules brought unchanged (`chain`, `target_architecture`, `changes`, `work_breakdown`, `pillars`, `modules`, `meaning`, `local`, `coverage`, `design_review`, `requirements_review`, `schema_review`, `assessment`, `presentation`).
|
||||
|
||||
**Two scripts.** `scripts/bring.py` now sends the Reference's subject modules to the example. `scripts/repoint.py` points a brought test at the example Workflow.
|
||||
|
||||
## Differed from the plan
|
||||
|
||||
- **Steps 3 and 5 were built as one, and before the toy Workflow.** With the Reference's workflow as a Workflow, its tests come across almost unchanged, which is a far stronger check of the seam than a toy. Step 4 remains: it shows the engine needs nothing of the Reference's subject.
|
||||
- **Kinds and Phases are plain strings inside the engine.** A Workflow may declare them as a `StrEnum`; the Workflow turns them into strings when it is built, because they go into checkpointed state. A review therefore receives a string, and one brought test compares with `==` where it used `is`.
|
||||
- **The manifest changed.** `execution.json` gains `final_document`, `options` and `files`; the Reference's `solution_guide`, `terraform_modules`, `delivery_plan` and `customer_questions` are written by the example's `outputs` hook. Stale files are found from `files`, not from named keys.
|
||||
- **`tools/meaning.py` went to the example,** not the engine: it reads the Reference's graph.
|
||||
- **The spec's two file names are the engine's** (`tools/scoping.py`), no longer the Reference's chain module's.
|
||||
|
||||
## Tests
|
||||
|
||||
454 pass. Brought in this step: `tests/scoped.py` and 21 files under `tests/execution/` and `tests/tools/`, among them `test_engine` (whole Executions on the scripted runtime), `test_router`, `test_change_requests`, `test_working_copy`, `test_requirements_chain`, `test_work_breakdown` and `test_pillars`.
|
||||
|
||||
Changed after bringing, beyond what `repoint.py` does:
|
||||
|
||||
| File | Change |
|
||||
| --- | --- |
|
||||
| `tests/scoped.py` | `ScopedEngine` defaults to the example Workflow |
|
||||
| `tests/execution/test_pillars.py`, `test_working_copy.py`, `tools/test_guardrail.py` | `make_tool_factory` is given a Workflow carrying the pillar Skills; `graph_tools` is given the shape |
|
||||
| `tests/execution/test_coverage_review.py` | `kind ==` where it was `kind is` |
|
||||
| `tests/execution/test_deliverables.py`, `test_engine.py`, `test_working_copy.py` | Three annotations and one dict key take `str` |
|
||||
|
||||
## Not brought, and why
|
||||
|
||||
Six test files need the Reference's own role files, provider bundle and Skills, which this repository does not hold: `execution/test_overlay_skills`, `test_design_review`, `test_scope_spec`, `test_model_tiers`, `test_terraform_modules`, and `a2a/test_server`. So `adk/roles.load_roles`, `open_assembly`, `card`, `a2a_server` and `main` are converted and type-check but have no test here. The same holds for `tests/agents/test_roles`, `test_adk_engine`, `tests/test_lessons_loop`, `test_providers` and `test_assembly`, and for the four tests removed in step 2.
|
||||
|
||||
They need a small agent root under `tests/`: role files, a bundle and stand-in Skills. That is the first thing to build next.
|
||||
|
||||
## Left in the engine from the Reference's subject
|
||||
|
||||
No engine module names a Deliverable or imports the example. What remains is wording and a few fields:
|
||||
|
||||
- `providers.py` (26 lines) still parses `terraform` and `pillar_skills`; `config.py` (18) still holds the Terraform and review settings; `adk/runtime.RoleSpec` still has `modules_skill`. These are read only by the example. They should move to where a Workflow reads its own sections of a bundle and its own settings.
|
||||
- `config.BASE_DIR` and `providers._REPO_ROOT` still resolve from the package's own location. `open_assembly` no longer uses them; `scope.py`'s command line still does.
|
||||
- The router's action names (`revise_code`, `revise_target_architecture`) and some messages ("the deployment dry-run failed", "Scope of a workload") are the Reference's words. `nodes/intake.py` still speaks of "the architecture" and reads the Baseline's `solution_guide`.
|
||||
- `workflows/work_breakdown.py` is whole in the example; its scheduler has not been separated for the engine.
|
||||
|
||||
## Verified
|
||||
|
||||
`pytest -q`: 454 passed. `ruff check .`, `ruff format --check .`, `mypy .` (103 files): clean. `grep` finds no import of `tests` and no `DeliverableKind`, `PhaseId`, `DEPENDENCY_GRAPH` or `PHASE_ORDER` in `execution_engine/`. `git status` in the Reference shows the same nine modified files as before this work began.
|
||||
50
.agents/plans/iter-01-execution-engine/steps/step-4.md
Normal file
50
.agents/plans/iter-01-execution-engine/steps/step-4.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Step 4: a toy Workflow, and an agent root to test the rest of the engine on
|
||||
|
||||
Built 2026-10-13.
|
||||
|
||||
## What was built
|
||||
|
||||
**The toy Workflow** (`tests/toy/workflow.py`): planning a picnic. Four Deliverables in three Phases: a menu, a shopping list written from it, a check that every dish can be made from the list, and the plan that puts them together. It has no spec, no graph and no cloud. It uses each kind of part a Workflow can supply at least once: data checks along a chain, code that adds a view on submission, a validation that reopens the list or the menu, an account of what a Change request changed, an option a caller may set, a local tool of its own, its own facts at an Approval, its own entries in a Specialist's task and in the manifest, and its card.
|
||||
|
||||
**Its agent root** (`tests/toy/root/`): a provider bundle, six role files with a shared partial, one Skill and one overlay Skill. All are stand-ins.
|
||||
|
||||
**Its tests** (`tests/toy/`, 62 tests):
|
||||
|
||||
| File | What it covers |
|
||||
| --- | --- |
|
||||
| `test_workflow.py` | A Workflow is checked when built; names written as an enum are held as strings |
|
||||
| `test_execution.py` | Whole Executions: pre-approved to the end; each Phase waiting for its Approval; a Specialist's question and its answer; a failed Check reopening the list; a Check failing twice reopening the menu; a user's request for changes; a Change request and its account; options |
|
||||
| `test_roles.py` | The role loader on a Workflow and bundle that are not the Reference's |
|
||||
| `test_assembly.py` | `open_assembly` from settings, the agent's folder and a Workflow |
|
||||
| `test_server.py` | The card, `/health`, `/generate`, A2A tasks, and messages that start nothing |
|
||||
|
||||
`tests/test_independence.py` keeps three things true: the engine imports nothing from its tests, it names none of the Reference's Deliverables, and the toy imports nothing from the Reference's example.
|
||||
|
||||
## What the toy found in the engine
|
||||
|
||||
Each was a place the engine still assumed the Reference, and each is fixed.
|
||||
|
||||
- **A provider bundle had to name an output template.** `providers.py` now takes it as optional.
|
||||
- **Overlay Skills were looked for under the engine's own folder.** `Settings.overlay_dirs`, `discover_overlays` and `skill_dirs` now take the agent's root.
|
||||
- **A whole number in an A2A message arrived as a float.** A message's metadata holds every number as a double, so `guests: 9` reached the option's check as `9.0` and was refused. `a2a_server._options` now returns whole numbers as ints.
|
||||
- **Message intake spoke of architectures.** `nodes/intake.py` no longer holds what the agent says it does (the Workflow's `about` is used), its notes and its questions to the decider say "work", and `DeliveredAnswers` is told the Workflow's final Deliverable in place of reading `solution_guide`.
|
||||
- **Settings defaulted to the Reference's names** for the agent and its databases. They are now `agent`, `agent_db` and `agent_local.db`.
|
||||
|
||||
## Differed from the plan
|
||||
|
||||
- Four Deliverables in three Phases, where the spec said three in two: with the check as the final Deliverable the toy could not show that a section written for readers is kept from the next Specialist and given to the final one.
|
||||
- The three tests removed in step 2 (`open_decider` twice, `gate_cloud_read`) are back, now that the modules they need are here.
|
||||
|
||||
## Still not tested here
|
||||
|
||||
- The six of the Reference's test files set aside in step 3 are still not brought. What they covered in the engine is now covered on the toy, except: starting from a spec through `ExecutionEngine.start` (the toy has no spec), the lighter model for a first attempt, and the lessons Skill being given to a role. `tests/test_lessons_loop` and `tests/agents/test_adk_engine` are also not brought.
|
||||
- `main.serve` and `create_app` with a real `open_assembly` (it builds the ADK runtime).
|
||||
- Nothing has run with real models.
|
||||
|
||||
## Left in the engine from the Reference's subject
|
||||
|
||||
Unchanged from step 3, less what this step fixed: `providers.py` still parses `terraform` and `pillar_skills`; `config.py` still holds the Terraform and review settings and `BASE_DIR`; `RoleSpec.modules_skill`; the router's action names and a few messages; `recorder.SESSION_AGENT_NAME` is `solution-architect`; the scheduler is still whole in the example.
|
||||
|
||||
## Verified
|
||||
|
||||
`pytest -q`: 521 passed. `ruff check .`, `ruff format --check .` and `mypy .` (112 files): clean. `git status` in the Reference shows the same nine modified files as before this work began.
|
||||
29
.agents/plans/iter-01-execution-engine/steps/step-6.md
Normal file
29
.agents/plans/iter-01-execution-engine/steps/step-6.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Step 6: the digest step
|
||||
|
||||
Built 2026-10-13.
|
||||
|
||||
## What was built
|
||||
|
||||
- **`sources` on `ExecutionEngine.start`**, and on `/generate` and A2A message metadata: files by name. The Workflow's `digest` reads the ones it knows and returns a `Digested`: the spec's fields, the sources it used, and a note per field of where it was read from. A Workflow with no `digest` reads none.
|
||||
- **`scope.merged`**: the digested spec with what the caller supplied laid over it, field by field. What the caller wrote wins.
|
||||
- **A new provenance, `digested`.** It stands as a supplied value does: the scope analyst cannot overwrite it, and only the customer's own answer changes it. The completed spec's `provenance` and `readiness`, and the information register, count digested values apart. `readiness` has the `digested` key only when something was digested, so a spec that was not is recorded as before.
|
||||
- **Sources are kept as supplied,** under the Customer case's `intent/NNN.sources/`, each with its hash in `history.json`, in the Execution's state and in `execution.json`.
|
||||
- **A source code did not read is a document** for the Specialist that completes the spec, in its context as `source_documents`, up to 60,000 characters each.
|
||||
- **Refused before anything runs or is written:** a source whose name is not a safe relative path, sources that are not text by name, and a source of a kind the Workflow knows and cannot read.
|
||||
|
||||
## Differed from the plan
|
||||
|
||||
- The plan had the inventory travel as a separate data file to the Specialists that use it. It still travels in the spec's data file, to every Specialist. That needs the engine to let a Workflow say which of a spec's fields are shown to whom, and is not built.
|
||||
- A Workflow still cannot name the spec's two files; they are `scope.yaml` and `scope.json`.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/toy/test_digest.py`: eight tests on the toy Workflow with a small catalogue and a digest of a made-up format. 529 pass in all.
|
||||
|
||||
## Verified
|
||||
|
||||
`pytest -q`: 529 passed. `ruff check .`, `ruff format --check .`, `mypy .`: clean.
|
||||
|
||||
## A guard added after, 2026-10-14
|
||||
|
||||
In an Execution with nobody to answer, a Specialist that ends three runs running without submitting its Deliverable now stops the Execution with a warning (`MAX_UNFINISHED_RUNS` in `nodes/execution_nodes.py`). Each such run is answered automatically and run again; with no guard it went on until the Orchestrator's turn limit. An operating model evaluation case looped this way for about two and a half hours. A submission resets the count. Two tests on the toy Workflow; 531 pass.
|
||||
Reference in New Issue
Block a user