feat: a design-time coverage review, measured and left off
A reviewer role reads the requirements and recommendations against the Checks when the Target architecture is first submitted. Three runs of case 03: it did not stop the validator sending the design back, and the designs it reviewed ended with 87 and 93 Checks against 52, so they cost more. It stays behind DESIGN_COVERAGE_REVIEW, off (iteration 6 step 13). Also: the schema review no longer reads `true` or `false` in backticks as a setting name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -22,7 +22,7 @@ from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import BaseCheckpointSaver
|
||||
from langgraph.types import Command
|
||||
|
||||
from app.adk.roles import load_roles
|
||||
from app.adk.roles import COVERAGE_REVIEWER, load_roles
|
||||
from app.adk.runtime import (
|
||||
AgentRuntime,
|
||||
Decider,
|
||||
@@ -50,6 +50,7 @@ from app.skills.agent_skills import discover_skills, skill_version
|
||||
from app.states.execution_state import check_customer_case
|
||||
from app.states.ledger import Ledger
|
||||
from app.tools.coverage import CoverageReview
|
||||
from app.tools.design_review import DesignCoverageReview
|
||||
from app.tools.execution import ORCHESTRATOR
|
||||
from app.tools.jev import JevDecider, JevLimits
|
||||
from app.tools.local import TerraformRunner, local_tools
|
||||
@@ -481,6 +482,10 @@ async def open_assembly(
|
||||
reviews.append(
|
||||
RequirementsReview(decider, act=settings.JEV_REQUIREMENTS_THRESHOLD)
|
||||
)
|
||||
if settings.DESIGN_COVERAGE_REVIEW and COVERAGE_REVIEWER in roles:
|
||||
# A model reads the requirements against the Checks, once, before the
|
||||
# design is approved: the reading the validator would make after the code.
|
||||
reviews.append(DesignCoverageReview(runtime, roles[COVERAGE_REVIEWER]))
|
||||
if settings.JEV_COVERAGE_REVIEW and isinstance(decider, JevDecider):
|
||||
# Optional, and off by default: Jev says which Check verifies each condition.
|
||||
reviews.append(CoverageReview(decider, none_at=settings.JEV_COVERAGE_THRESHOLD))
|
||||
|
||||
Reference in New Issue
Block a user