feat: what a request does not say is a question for the customer
Some checks failed
validation / verify (push) Failing after 11s
Some checks failed
validation / verify (push) Failing after 11s
A pillar's checklist item the request gives no information about was marked "not addressed", which reads as a fault in the design. It is missing information. The checklist review gains a fourth outcome, to confirm with the customer, which must carry the question to ask. The questions are written into the design recommendations and the guide, and to customer-questions.md; the evaluation's judged check counts such an item as accounted for. Records the owner's decisions of 2026-10-10 and the vendor's answers on Jev's retention, training, context and rate limits (iteration 5 step 1), and the owner's labels, which showed the judged question was wrong, not the judges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -32,6 +32,21 @@ QUALITY_RUBRIC: Mapping[str, str] = {
|
||||
_JUDGED = "design_recommendations"
|
||||
|
||||
|
||||
def checklist_question(pillar: str, item: str) -> str:
|
||||
"""Whether the recommendations account for one item of a pillar's checklist.
|
||||
|
||||
An item is accounted for in three ways. The third matters: a customer's request
|
||||
seldom says enough to settle every item, and what it leaves unsaid is a
|
||||
question for them, not a fault in the design (the owner, 2026-10-10).
|
||||
"""
|
||||
return (
|
||||
f"Do the recommendations account for this {pillar} checklist item in any "
|
||||
"of these ways: by recommending it, by explaining why it does not apply, "
|
||||
"or by recording it as something to confirm with the customer? "
|
||||
f"{item}"
|
||||
)
|
||||
|
||||
|
||||
async def assess(
|
||||
decider: Decider,
|
||||
deliverables: Mapping[str, Mapping[str, Any]],
|
||||
|
||||
Reference in New Issue
Block a user