Files
solution_architecture_agent/.agents/plans/iter-01-on-the-engine/steps/step-2.md

1.6 KiB

Step 2: a run with real models through the Orchestration engine, into the operating model agent

Run 2026-10-21.

One pre-approved request to the Orchestration engine for architecture_and_operating_model, Customer case case-08-landing-zone, from the benchmark spec eval/datasets/specs/landing-zone.yaml, the operating model agent's own fields under operating_model. Models: Gemini on Vertex AI; Google Cloud's remote MCP servers on.

Result

  • This agent's Execution (6c316f05…) completed in about 50 minutes on the shared Execution engine: its first run with real models since it was built from the reference. Fifteen files kept: the solution guide, the scope, requirements, components, products, recommendations, the Target architecture and its diagram, implementation, verdicts, the work breakdown structure, and Terraform. The reference took 55 to 60 minutes on the same spec on its own engine.
  • The operating model agent was given those files as sources with the Orchestration engine's record that the work completed, digested them, and completed: 24 Operational requirements, 21 Capabilities, 8 Roles, 26 Checks, 44 of 44 scenarios, 12.2 people at 87,959 GBP a month.
  • 74.9 minutes end to end. Both Executions are kept in the Orchestration engine's store and were fetched through their links.

The record of the run, its report and every file: ../tom_analysis/eval/results/20261021T203623Z-handoff/ (not in a repository).

Not done

  • The evaluation harness (step 3): this agent's own checks on what it delivered were not run; the run was judged by the operating model agent completing from it.