Case 04 (interactive) - A scripted answer may name who it is for, so facts written for requirements discovery are not given to a Specialist confirming a draft. - A reply that restates what the agreed requirements already say is read as an answer, not a request for changes. - With a user present, a first submission is refused until the user has been shown a draft of that Deliverable (CONFIRM_DRAFTS). Routing and reviews - Intake: a message that starts something is read first. A question is answered and starts nothing; a change to delivered work starts as a new request with a note, until Change requests are built. - Requirements and recommendations list what infrastructure code can show (conditions) apart from what it cannot (outside_code); the latter needs no Check and validation proposes nothing for it. - A requirements review, "names no products", behind its own switch (off). - Each pillar advisor reviews its pillar's checklist; the engine writes the reviews into the design recommendations. The judged checklist check went from 9-45% a pillar to 100% in one run of case 04. Evaluation - Per-case thresholds for Revisions, seconds and tokens, set from every completed run on this build. - A progress file after every agent run, so an interrupted run keeps its time and tokens. - eval/calibrate_judge.py with a 32-question set awaiting a person's labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91. Smaller - An Execution that ends with warnings writes its Deliverables, with the warnings and each status in the manifest. - CI installs the application's dependencies before linting and testing. - A load-balancer pattern in the IaC patterns Skill. Plans record nine runs of 2026-10-08. Iteration 6 is complete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
123 lines
4.7 KiB
Python
123 lines
4.7 KiB
Python
"""Intake: what a message that starts something is asking for (ADR-0007).
|
|
|
|
Inside an Execution the router reads the user's replies. This is the same reading
|
|
one step earlier, for a message that arrives with no Execution in progress:
|
|
|
|
new request design this workload start an Execution
|
|
change request change what was delivered (ADR-0006; not built yet)
|
|
question what can you do, how long... answer; start nothing
|
|
|
|
Without a decision, and whenever the reading is not confident, the message is taken
|
|
as a new request: that is what every message was before, and starting work nobody
|
|
asked for is cheaper to put right than refusing work somebody did.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from collections.abc import Mapping, Sequence
|
|
from dataclasses import dataclass
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
from app.adk.runtime import Decider, Question
|
|
from app.customer_case import IN_PROGRESS, read_history
|
|
from app.states.execution_state import UNASSIGNED_CASE
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
NEW_REQUEST = "new_request"
|
|
CHANGE_REQUEST = "change_request"
|
|
QUESTION = "question"
|
|
|
|
ABOUT = (
|
|
"I design cloud solution architectures. Tell me, at a high level, what you want "
|
|
"built, and I will work through requirements, a target architecture, "
|
|
"infrastructure code, validation and a packaged guide, asking for your approval "
|
|
"at each Phase. Nothing was started for this message."
|
|
)
|
|
CHANGE_NOTE = (
|
|
"This reads as a change to work already delivered for this Customer case. "
|
|
"Changing an earlier architecture in place is not available yet, so this is "
|
|
"being designed as a new request."
|
|
)
|
|
|
|
_MESSAGE_LIMIT = 8_000
|
|
_EARLIER_SHOWN = 5
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class IntakeReading:
|
|
kind: str # new_request | change_request | question
|
|
confidence: float = 0.0
|
|
judged: bool = False # False when no decision was available
|
|
earlier: int = 0 # finished Executions of the same Customer case
|
|
|
|
|
|
def finished_intents(output_root: Path, customer_case: str) -> list[Mapping[str, Any]]:
|
|
"""The case's earlier statements of intent whose Executions delivered something."""
|
|
if not customer_case or customer_case == UNASSIGNED_CASE:
|
|
return []
|
|
return [
|
|
item
|
|
for item in read_history(output_root, customer_case)["intents"]
|
|
if isinstance(item, Mapping)
|
|
and str(item.get("status", IN_PROGRESS)).startswith("completed")
|
|
]
|
|
|
|
|
|
class Intake:
|
|
def __init__(
|
|
self,
|
|
decider: Decider,
|
|
output_root: Path,
|
|
*,
|
|
act: float = 0.75,
|
|
) -> None:
|
|
self._decider = decider
|
|
self._output_root = output_root
|
|
# How sure the reading must be before anything but a new Execution is done.
|
|
self._act = act
|
|
|
|
async def read(
|
|
self, message: str, customer_case: str | None = None, scope: str = ""
|
|
) -> IntakeReading:
|
|
earlier = finished_intents(self._output_root, customer_case or "")
|
|
text = message.strip()
|
|
if not text:
|
|
return IntakeReading(NEW_REQUEST, earlier=len(earlier))
|
|
answers = await self._decider.decide(
|
|
_state(text, earlier),
|
|
{
|
|
"intake": Question(
|
|
"choice",
|
|
"What is the user's message? new_request: they describe "
|
|
"something they want designed or built. change_request: they "
|
|
"ask to change or extend an architecture already delivered to "
|
|
"them. question: they ask about the service or about earlier "
|
|
"work, and want an answer, not new work.",
|
|
(NEW_REQUEST, CHANGE_REQUEST, QUESTION),
|
|
)
|
|
},
|
|
scope=scope,
|
|
)
|
|
answer = (answers or {}).get("intake")
|
|
if answer is None:
|
|
return IntakeReading(NEW_REQUEST, earlier=len(earlier))
|
|
kind, confidence = str(answer.value), float(answer.confidence)
|
|
if kind != NEW_REQUEST and confidence < self._act:
|
|
kind = NEW_REQUEST # not sure enough to do anything but start
|
|
if kind == CHANGE_REQUEST and not earlier:
|
|
kind = NEW_REQUEST # there is nothing delivered to change
|
|
logger.info("[%s] intake: %s (%.2f)", scope, kind, confidence)
|
|
return IntakeReading(kind, confidence, judged=True, earlier=len(earlier))
|
|
|
|
|
|
def _state(message: str, earlier: Sequence[Mapping[str, Any]]) -> str:
|
|
lines = [f"Architectures already delivered to this user: {len(earlier)}."]
|
|
for item in list(earlier)[-_EARLIER_SHOWN:]:
|
|
request = " ".join(str(item.get("workflow_request", "")).split())[:300]
|
|
lines.append(f"- {request}")
|
|
lines.append(f"The user's message: {message[:_MESSAGE_LIMIT]}")
|
|
return "\n".join(lines)
|