Case 04 (interactive) - A scripted answer may name who it is for, so facts written for requirements discovery are not given to a Specialist confirming a draft. - A reply that restates what the agreed requirements already say is read as an answer, not a request for changes. - With a user present, a first submission is refused until the user has been shown a draft of that Deliverable (CONFIRM_DRAFTS). Routing and reviews - Intake: a message that starts something is read first. A question is answered and starts nothing; a change to delivered work starts as a new request with a note, until Change requests are built. - Requirements and recommendations list what infrastructure code can show (conditions) apart from what it cannot (outside_code); the latter needs no Check and validation proposes nothing for it. - A requirements review, "names no products", behind its own switch (off). - Each pillar advisor reviews its pillar's checklist; the engine writes the reviews into the design recommendations. The judged checklist check went from 9-45% a pillar to 100% in one run of case 04. Evaluation - Per-case thresholds for Revisions, seconds and tokens, set from every completed run on this build. - A progress file after every agent run, so an interrupted run keeps its time and tokens. - eval/calibrate_judge.py with a 32-question set awaiting a person's labels. Unlabelled: Jev stable at 1.00, Gemini 0.92, agreeing on 0.91. Smaller - An Execution that ends with warnings writes its Deliverables, with the warnings and each status in the manifest. - CI installs the application's dependencies before linting and testing. - A load-balancer pattern in the IaC patterns Skill. Plans record nine runs of 2026-10-08. Iteration 6 is complete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
59 lines
2.3 KiB
Python
59 lines
2.3 KiB
Python
"""Intake: what a message that starts something is asking for."""
|
|
|
|
import json
|
|
|
|
from app.adk.fake_runtime import FakeDecider
|
|
from app.adk.runtime import Answer, NoDecider
|
|
from app.nodes.intake import CHANGE_REQUEST, NEW_REQUEST, QUESTION, Intake
|
|
|
|
|
|
def delivered(tmp_path, case: str, statuses: list[str]) -> None:
|
|
folder = tmp_path / "deliverables" / "cases" / case / "intent"
|
|
folder.mkdir(parents=True)
|
|
intents = [
|
|
{"version": n, "workflow_request": f"Design thing {n}.", "status": status}
|
|
for n, status in enumerate(statuses, start=1)
|
|
]
|
|
(folder / "history.json").write_text(
|
|
json.dumps({"customer_case": case, "intents": intents})
|
|
)
|
|
|
|
|
|
async def test_without_a_decision_every_message_is_a_new_request(tmp_path):
|
|
reading = await Intake(NoDecider(), tmp_path).read("What can you do?")
|
|
|
|
assert reading.kind == NEW_REQUEST and not reading.judged
|
|
|
|
|
|
async def test_a_confident_question_is_read_as_one(tmp_path):
|
|
decider = FakeDecider({"intake": Answer(QUESTION, 0.98)})
|
|
|
|
reading = await Intake(decider, tmp_path).read("How long does it take?", scope="t1")
|
|
|
|
state, questions, scope = decider.asked[0]
|
|
assert reading.kind == QUESTION and reading.judged
|
|
assert scope == "t1"
|
|
assert questions["intake"].options == (NEW_REQUEST, CHANGE_REQUEST, QUESTION)
|
|
assert state.endswith("The user's message: How long does it take?")
|
|
|
|
|
|
async def test_a_reading_that_is_not_sure_enough_starts_new_work(tmp_path):
|
|
decider = FakeDecider({"intake": Answer(QUESTION, 0.6)})
|
|
|
|
assert (await Intake(decider, tmp_path, act=0.75).read("hello")).kind == NEW_REQUEST
|
|
|
|
|
|
async def test_a_change_needs_something_delivered_to_change(tmp_path):
|
|
decider = FakeDecider({"intake": Answer(CHANGE_REQUEST, 0.99)})
|
|
delivered(tmp_path, "acme", ["completed", "in_progress"])
|
|
intake = Intake(decider, tmp_path)
|
|
|
|
with_history = await intake.read("Raise availability to 99.99%.", "acme")
|
|
without = await intake.read("Raise availability to 99.99%.", "new-client")
|
|
no_case = await intake.read("Raise availability to 99.99%.")
|
|
|
|
assert with_history.kind == CHANGE_REQUEST and with_history.earlier == 1
|
|
assert "- Design thing 1." in decider.asked[0][0]
|
|
assert "Design thing 2." not in decider.asked[0][0] # still in progress
|
|
assert without.kind == NEW_REQUEST and no_case.kind == NEW_REQUEST
|