feat: what a request does not say is a question for the customer
Some checks failed
validation / verify (push) Failing after 11s
Some checks failed
validation / verify (push) Failing after 11s
A pillar's checklist item the request gives no information about was marked "not addressed", which reads as a fault in the design. It is missing information. The checklist review gains a fourth outcome, to confirm with the customer, which must carry the question to ask. The questions are written into the design recommendations and the guide, and to customer-questions.md; the evaluation's judged check counts such an item as accounted for. Records the owner's decisions of 2026-10-10 and the vendor's answers on Jev's retention, training, context and rate limits (iteration 5 step 1), and the owner's labels, which showed the judged question was wrong, not the judges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -20,7 +20,7 @@ from typing import Any
|
||||
|
||||
from app.adk.runtime import ToolResult
|
||||
from app.agent import Interaction
|
||||
from app.tools.assessment import QUALITY_BAR, QUALITY_RUBRIC
|
||||
from app.tools.assessment import QUALITY_BAR, QUALITY_RUBRIC, checklist_question
|
||||
from app.tools.local import mermaid_check
|
||||
from app.workflows.chain import (
|
||||
COMPONENTS_FILE,
|
||||
@@ -417,8 +417,7 @@ async def _checklist_coverage(
|
||||
questions = {
|
||||
f"{pillar}.{index}": {
|
||||
"kind": "yes_no",
|
||||
"text": f"Do the recommendations address this {pillar} checklist item, "
|
||||
f"either by recommending it or by explaining why it does not apply? {item}",
|
||||
"text": checklist_question(pillar, item),
|
||||
}
|
||||
for pillar, items in checklists.items()
|
||||
for index, item in enumerate(items)
|
||||
|
||||
Reference in New Issue
Block a user