feat: establish the Target architecture as a graph before code
Design is now approved before any infrastructure code is written (ADR-0005). A new target-architect Specialist produces one graph of Elements, Relationships and Checks; the diagram is rendered from it, deployment guidance must account for every Check, and validation gives a verdict per Check. Requirements, components, products and recommendations carry ids in data files, and a submission with a gap in the Requirements chain is refused. Also in this commit: - Customer cases group Executions, with an intent history per case - lesson candidates are captured when an Execution ends - evaluation gains chain, verdict, judged and interactive checks, and per-case thresholds - Terraform providers are cached between checks - the requirements analyst proposes defaults instead of asking - the legacy-baseline case and the checks that only read it are removed - ADR-0006 (Change requests) accepted; ADR-0004 and ADR-0007 proposed - plans for iterations 4 to 7 under .agents/plans/ Unproven with a real model: no evaluation run has completed on this design. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -3,19 +3,22 @@
|
||||
Run from the repository root without credentials:
|
||||
|
||||
```bash
|
||||
# 1. Static artifact & guide section validation
|
||||
python3 scripts/validate_artifacts.py
|
||||
|
||||
# 2. Pytest unit & integration test suite
|
||||
# 1. Pytest unit & integration test suite
|
||||
pytest -v
|
||||
|
||||
# 3. Real-model evaluation of the engine (needs model credentials; costs usage)
|
||||
# 2. Real-model evaluation of the engine (needs model credentials; costs usage)
|
||||
python3 -m eval.solution_eval
|
||||
|
||||
# 4. Optional Terraform formatting & validation
|
||||
# 3. Optional Terraform formatting & validation
|
||||
terraform -chdir=terraform fmt -check -recursive
|
||||
terraform -chdir=terraform init -backend=false
|
||||
terraform -chdir=terraform validate
|
||||
```
|
||||
|
||||
`init -backend=false` may download the pinned provider constraint but does not create infrastructure. Do not use `terraform apply` as part of validation. A deployment candidate additionally requires an operator-supplied `terraform plan -out=tfplan` and human review.
|
||||
|
||||
## Evaluation options
|
||||
|
||||
- `python3 -m eval.solution_eval --judge` adds the judged checks: coverage of each pillar Skill's validation checklist, and a quality rubric for the design recommendations.
|
||||
- `python3 -m eval.solution_eval case-04-interactive-web-app` runs one case with a scripted user in the loop.
|
||||
- A case's `thresholds` in `eval/datasets/solution_cases.json` bound its revisions, seconds and tokens; exceeding one is reported as a regression.
|
||||
|
||||
Reference in New Issue
Block a user