- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
45 lines
2.6 KiB
Markdown
45 lines
2.6 KiB
Markdown
# 0008. Models
|
|
|
|
| | |
|
|
|---|---|
|
|
| Status | Proposed |
|
|
| Date | 2026-10-29, changed 2026-11-01 and 2026-11-03 |
|
|
| Decided by | Built as a default; awaiting product owner and platform confirmation |
|
|
| Requirements changed | DSC-08's note on sharing a model with any executor |
|
|
|
|
## Decision
|
|
|
|
- **All eight agents use `gpt-5.6-sol`,** as the CP4 agents do, with eval cases waiting 330 seconds.
|
|
- **The critic shares a model with the executors.** It was to use `claude-sonnet-5-5`, but the cluster's LiteLLM doesn't serve that (checked 2026-11-01), so the product owner moved it to `gpt-5.6-sol`.
|
|
|
|
## Change, 2026-11-03
|
|
|
|
**The critic now runs on `gpt-5.4`,** which the cluster's LiteLLM serves, and the eight other agents stay on
|
|
`gpt-5.6-sol`. The product owner asked for this after the first live GCP run (`exec_0f8629066e13`), whose critic
|
|
noted that it shared the executors' model. The critic states the models it and the executors use in `note`.
|
|
|
|
## Change, 2026-11-04
|
|
|
|
**The network agent also runs on `gpt-5.4`.** Two live Azure runs on an 81-record subscription
|
|
(`exec_c57354b25f8e`, `exec_beb8879632b8`) failed at the network step: `gpt-5.6-sol` returned HTTP 429 for its token
|
|
rate limit, with no fallback configured. The network agent sends the whole inventory on each of up to 25 model calls
|
|
while it pages through topology, which uses the per-minute budget in one step. `gpt-5.4` has its own quota. The
|
|
product owner approved this. As a result the critic shares a model with the network agent, and says so in `note`.
|
|
Follow-up: send the network agent only network-relevant records, which fixes the cause.
|
|
|
|
## Consequences
|
|
|
|
- **The critic can share blind spots with the agents it judges.** It states this in its `note`, as DSC-08 requires, and the Engagement Lead's approval still gates its verdict (ADR 0006).
|
|
- **Revisit this when another model is on the gateway.** A different model for the critic remains the better design.
|
|
|
|
## Change, 2026-11-04: output limit 80,000 on the `gpt-5.6-sol` agents
|
|
|
|
At the product owner's direction ("do not change the model; increase the token limits within the agents to 80,000"),
|
|
`LITELLM_MAX_TOKENS` is `80000` (was `32000`) on the eight `gpt-5.6-sol` agents. Models are unchanged.
|
|
|
|
- The network agent and the critic stay at `16384`: `gpt-5.4` refuses a larger value ("max_tokens is too large",
|
|
`exec_b929f06cbb0c`).
|
|
- Not yet confirmed that `gpt-5.6-sol` accepts 80,000. A refusal would fail the planner, the first model step, at once.
|
|
- Azure OpenAI counts a request's `max_tokens` against the deployment's tokens-per-minute quota before the reply is
|
|
written, so a higher limit can bring the 429 sooner if `gpt-5.6-sol` is served from Azure.
|