feat: the solution architecture agent on the shared Execution engine
The reference agent's subject, cloud solution architecture, as an agent in the operating model agent's pattern: the Execution engine (from orchestration_engine) and one Workflow in app/solution_architecture, built for the settings and provider bundle by assembly.workflow_for. Copied from gcp_solution_architecture_agent at 8b397cd without its own engine: its role files, Skills, provider bundle, ADRs, vocabulary, scripts and evaluation datasets. The image takes the engine from a named build context and installs Terraform. It is called through the Orchestration engine, and its files are what the operating model agent digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
20
eval/datasets/judge/calibration.json
Normal file
20
eval/datasets/judge/calibration.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"runs": 3,
|
||||
"questions": 32,
|
||||
"labelled": 0,
|
||||
"judges": {
|
||||
"jev": {
|
||||
"answered": 32,
|
||||
"stability": 1.0,
|
||||
"agreement_with_labels": null,
|
||||
"mean_answer": 0.37
|
||||
},
|
||||
"gemini": {
|
||||
"answered": 32,
|
||||
"stability": 0.92,
|
||||
"agreement_with_labels": null,
|
||||
"mean_answer": 0.32
|
||||
}
|
||||
},
|
||||
"between_judges": 0.91
|
||||
}
|
||||
Reference in New Issue
Block a user