ALZ-F01 Discovery: package step, value scorecard, application grouping, cases

- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part
  hashed (sha256 over canonical JSON), with a manifest and a value scorecard.
- Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021):
  CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them.
- Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum
  narrower than its producer's.
- Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic
  agents; the report and Well-Architected review agents; ADRs 0012-0021.
- Workflow cases with value bars and run limits; run evaluation and usage collection scripts;
  the run context template and the field-by-field review for ADR 0016.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-05 22:13:25 +00:00
parent f659564208
commit 9e503885cc
211 changed files with 18170 additions and 158 deletions

View File

@@ -0,0 +1,5 @@
# cp1-well-architected-review-agent
CP1 Well-Architected review (ALZ-DSC-10): one step of the CP1 Discovery workflow. It answers from code, like the
report agent (ADR 0014); keep it that way. Every tool it calls is read-only and declared in `agent.yaml` under the
gateway's display names. Add a check in `app/waf.py` with a test in `tests/`, and a row in README.md.

View File

@@ -0,0 +1,6 @@
ARG RUNTIME_IMAGE=cruciblecoredevacr.azurecr.io/kab/agent-runtime-adk:0.35.7
FROM ${RUNTIME_IMAGE}
COPY agent.yaml prompt.md /agent/
COPY schemas/ /agent/schemas/
COPY eval/ /agent/eval/
COPY app/ /agent/app/

View File

@@ -0,0 +1,33 @@
# CP1 - Well-Architected Review Agent
Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML.
A step of the workflow "CP1 - Agentic Landing Zone - Discovery", after the critic and before the report. Like the
report agent, it answers from code (`app/waf.py`, called by the `pre_process` hook), so no model is involved and every
figure can be recomputed from the evidence. With `provider` `sample` it replies "not applicable" without calling a tool.
## What it reads
Only read-only tools on the existing servers, through the MCP gateway, with the project's stored connection:
`Azure Discovery` or `Google Cloud Discovery` (servers, workloads, clusters, data stores, firewall or NSG rules,
organisation policies, Azure resource locks, every page of the provider's recommendations, and one guidance reference
per pillar from the server's own `<provider>_search_well_architected`, which takes only fixed pillars and topics). The checks judge only the scope entries in `records`.
## What it writes
`waf_markdown` (the file), `waf_html` (the draft page) and `final_waf_html` (the page released after approval),
plus the structured `pillars`, `checks` and `recommendations`. A pillar with no evidence is **not assessed**, never a
pass. Calls that failed are listed in `call_issues` and in the review.
## Checks
| Pillar | Checks |
|---|---|
| Security | SEC-01 no admin or database port open to the internet; SEC-02 no public server address; SEC-03 Secure Boot on; SEC-04 storage blocks public access; SEC-05 guardrail policies at the scope root |
| Reliability | REL-01 compute not confined to one zone; REL-02 deletion protection (GCP) or resource locks (Azure); REL-03 database backups; REL-04 object versioning (GCP) |
| Cost | COST-01 no stopped server holding paid resources; COST-02 cost-allocation labels |
| Operations | OPS-01 owner or environment labels; OPS-02 storage access logging (GCP) |
| Sustainability | SUS-01 no idle or stopped resources |
| Performance | Provider recommendations only (no telemetry) |
Tests: `python3 -m pytest tests/` (no runtime needed).

View File

@@ -0,0 +1,11 @@
## [0.2.0] - 2026-11-04
### Changed
- Guidance comes from the provider's own discovery server (`azure_search_well_architected`, `gcp_search_well_architected`), not the separate `cloud-knowledge` server, at the product owner's direction.
## [0.1.0] - 2026-11-03
### Added
- First release (ALZ-DSC-10, ADR 0014): Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML.

View File

@@ -0,0 +1,46 @@
name: "cp1-well-architected-review-agent"
display_name: "CP1 - Well-Architected Review Agent"
description: "Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML."
version: "0.2.0"
framework: "adk"
tags:
- "cp1"
- "public-cloud"
- "discovery"
capabilities:
streaming: false
push_notifications: false
prompt: "prompt.md"
contract:
input: "schemas/input.json"
output: "schemas/output.json"
hooks: app.business_logic
mcp:
"Azure Discovery":
- "azure_list_servers"
- "azure_list_workloads"
- "azure_list_kubernetes_clusters"
- "azure_list_data_stores"
- "azure_get_network_topology"
- "azure_list_organization_policies"
- "azure_list_resource_policies"
- "azure_list_recommendations"
- "azure_search_well_architected"
"Google Cloud Discovery":
- "gcp_list_servers"
- "gcp_list_workloads"
- "gcp_list_kubernetes_clusters"
- "gcp_list_data_stores"
- "gcp_get_network_topology"
- "gcp_list_organization_policies"
- "gcp_list_recommendations"
- "gcp_search_well_architected"
skills:
- id: "cp1_well_architected_review_agent"
name: "CP1 - Well-Architected Review Agent"
description: "Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML."
tags:
- "cp1"
- "public-cloud"
- "discovery"
examples: []

View File

@@ -0,0 +1,13 @@
"""Answers from pre_process: the review comes from app/waf.py, so no model is called (ADR 0014)."""
import json
from typing import Any, Optional
from kyndryl_agent_runtime.request_cycle import Reply
from kyndryl_agent_runtime.tools import call_tool
from app import waf
def pre_process(payload: dict) -> Optional[Any]:
"""Collect the evidence through the declared read-only tools, judge it, and render the review."""
return Reply(json.dumps(waf.review(payload, call_tool)))

View File

@@ -0,0 +1,487 @@
"""The Well-Architected review: evidence collected and judged in code, rendered as Markdown and HTML (ADR 0014).
`review(payload, call)` reads the live estate through the cloud discovery server's read-only tools, applies fixed
checks per pillar, rolls up the provider's own recommendations by pillar and impact, and adds guidance references
from the provider's own discovery server. No model is involved, so every figure can be recomputed from the evidence. `call(name, args)`
is the runtime's call_tool; tests pass a stand-in. A pillar with no evidence is "not assessed", never a pass.
"""
from __future__ import annotations
import html
import re
from collections import Counter, defaultdict
from datetime import date
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple
PILLARS = ("security", "reliability", "cost", "operations", "performance", "sustainability")
TITLES = {"security": "Security", "reliability": "Reliability", "cost": "Cost optimisation",
"operations": "Operational excellence", "performance": "Performance efficiency", "sustainability": "Sustainability"}
# The provider's own discovery server accepts only these fixed topics (mcp-servers/_common/knowledge_support.py).
TOPICS = {"security": "network segmentation", "reliability": "high availability", "cost": "cost allocation",
"operations": "logging and monitoring", "performance": "scaling", "sustainability": "virtual machines"}
ADMIN_PORTS = {22, 3389, 21, 1433, 3306, 5432}
OPEN = {"0.0.0.0/0", "::/0", "*", "internet", "any"}
OWNER_KEYS = ("owner", "team", "cost-center", "cost_center", "costcenter", "environment", "env", "application", "app")
PAGE_SIZE = 200
MAX_PAGES = 40 # per call: 8,000 items is far beyond the estates seen so far, and bounds a runaway
KINDS = {
"gcp": {"servers": ("gcp_list_servers", ["instances"]), "workloads": ("gcp_list_workloads", ["cloud_run", "cloud_functions", "app_engine"]),
"clusters": ("gcp_list_kubernetes_clusters", ["clusters"]), "stores": ("gcp_list_data_stores", ["cloud_sql", "firestore", "cloud_storage"]),
"rules": ("gcp_get_network_topology", ["firewall_rules"]),
"org_policies": ("gcp_list_organization_policies", ["organization_policies"])},
"azure": {"servers": ("azure_list_servers", ["virtual_machines", "scale_sets"]),
"workloads": ("azure_list_workloads", ["app_service", "container_apps", "container_instances"]),
"clusters": ("azure_list_kubernetes_clusters", [None]),
"stores": ("azure_list_data_stores", ["sql", "cosmos_db", "storage_accounts", "postgresql", "mysql", "redis"]),
"rules": ("azure_get_network_topology", ["network_security_groups"]),
"public_ips": ("azure_get_network_topology", ["public_ips"]),
"org_policies": ("azure_list_organization_policies", ["assignments"]),
"locks": ("azure_list_resource_policies", ["resource_locks"])},
}
GCP_RECOMMENDERS = ("google.compute.instance.MachineTypeRecommender", "google.compute.instance.IdleResourceRecommender",
"google.compute.disk.IdleResourceRecommender", "google.iam.policy.Recommender")
# ---------------------------------------------------------------- evidence ----------------------------------------------
def _data(result: Any) -> Dict[str, Any]:
"""The servers' envelope, {success, data} or {success: false, error: {code, message}}; a refusal raises."""
if not isinstance(result, dict):
raise ValueError(f"unexpected reply {type(result).__name__}")
if result.get("success") is False or (result.get("error") and "data" not in result):
error = result.get("error") or {}
raise ValueError(f"{error.get('code', 'refused')}: {error.get('message', '')}"[:200] if isinstance(error, dict) else str(error)[:200])
return result.get("data") if isinstance(result.get("data"), dict) else result
def _items(result: Any) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[str]]:
"""A tool reply's items, next page token and evidence reference; a refusal raises."""
data = _data(result)
return list(data.get("items") or []), data.get("next_page_token"), data.get("evidence_ref")
def _all_pages(call: Callable, tool: str, args: Dict[str, Any], issues: List[str], refs: List[str]) -> List[Dict[str, Any]]:
out, token = [], None
for _ in range(MAX_PAGES):
try:
items, token, ref = _items(call(tool, {**args, "page_size": PAGE_SIZE, **({"page_token": token} if token else {})}))
except Exception as error: # noqa: BLE001 - a refusal or failure is evidence of a gap, not a crash
issues.append(f"{tool}({', '.join(f'{k}={v}' for k, v in args.items())}): {str(error)[:160]}")
return out
out += items
if ref:
refs.append(ref)
if not token:
return out
issues.append(f"{tool}: stopped after {MAX_PAGES} pages")
return out
def _resource(item: Dict[str, Any]) -> Dict[str, Any]:
"""The provider's own resource body: GCP versionedResources[0].resource.data, or an Azure row's properties."""
versions = item.get("versionedResources") or []
if versions:
body = (versions[0] or {}).get("resource") or {}
return body.get("data") or body
return item.get("properties") or item.get("resource", {}).get("data") or {}
def _name(item: Dict[str, Any]) -> str:
return str(item.get("name") or item.get("id") or "")
def _short(name: str) -> str:
return name.rstrip("/").rsplit("/", 1)[-1]
def collect(provider: str, call: Callable, scope: Iterable[str]) -> Dict[str, Any]:
"""Every piece of evidence the checks use, with the calls that failed and the evidence references returned."""
issues: List[str] = []
refs: List[str] = []
ev: Dict[str, Any] = {}
failed = set()
for key, (tool, kinds) in KINDS[provider].items():
ev[key] = []
for kind in kinds:
before = len(issues)
ev[key] += _all_pages(call, tool, {"kind": kind} if kind else {}, issues, refs)
if len(issues) > before:
failed.add(key) # this evidence wasn't fully read: its checks say so instead of passing
scope = set(scope)
if scope: # the checks judge the scope the plan accounted for; structure and rules stay as listed
for key in ("servers", "workloads", "clusters", "stores"):
ev[key] = [i for i in ev[key] if _name(i) in scope]
if provider == "gcp":
projects = sorted({m.group(1) for n in scope for m in [re.search(r"/projects/([^/]+)", n)] if m})
zones = sorted({m.group(1) for n in scope for m in [re.search(r"/zones/([^/]+)", n)] if m})
recs = []
for project in projects:
for recommender in GCP_RECOMMENDERS:
for location in (["global"] if recommender.startswith("google.iam") else zones or ["global"]):
recs += _all_pages(call, "gcp_list_recommendations",
{"project_id": project, "recommender": recommender, "location": location}, issues, refs)
ev["recommendations"] = recs
else:
ev["recommendations"] = _all_pages(call, "azure_list_recommendations", {}, issues, refs)
ev["issues"], ev["evidence_refs"], ev["failed"] = issues, refs, failed
return ev
# ---------------------------------------------------------------- checks ------------------------------------------------
def _labels(item: Dict[str, Any]) -> Dict[str, Any]:
body = _resource(item)
return item.get("labels") or item.get("tags") or body.get("labels") or body.get("tags") or {}
def _ports(spec: Any) -> set:
out = set()
for part in re.split(r"[,\s]+", str(spec or "")):
if not part or part == "*":
out |= ADMIN_PORTS if part == "*" else set()
continue
lo, _, hi = part.partition("-")
if lo.isdigit():
out |= {p for p in ADMIN_PORTS if int(lo) <= p <= int(hi or lo)}
return out
def _exposures(provider: str, rules: List[Dict[str, Any]]) -> List[str]:
"""Inbound allow rules open to the internet on an administrative or database port."""
found = []
for item in rules:
r = _resource(item)
if provider == "gcp":
if r.get("disabled") or str(r.get("direction", "INGRESS")).upper() != "INGRESS":
continue
if not set(r.get("sourceRanges") or []) & OPEN:
continue
for allowed in r.get("allowed") or []:
proto = str(allowed.get("IPProtocol", "")).lower()
ports = set().union(*[_ports(p) for p in allowed.get("ports") or []]) if allowed.get("ports") else (
ADMIN_PORTS if proto in ("all", "tcp") else set())
if ports:
found.append(f"{_short(_name(item))}: {', '.join(str(p) for p in sorted(ports))}")
else:
for rule in r.get("securityRules") or []:
p = rule.get("properties") or rule
if str(p.get("access")).lower() != "allow" or str(p.get("direction")).lower() != "inbound":
continue
sources = {str(p.get("sourceAddressPrefix", "")).lower(), *[str(s).lower() for s in p.get("sourceAddressPrefixes") or []]}
if not sources & OPEN:
continue
ports = set().union(_ports(p.get("destinationPortRange")), *[_ports(x) for x in p.get("destinationPortRanges") or []])
if ports:
found.append(f"{_short(_name(item))}/{rule.get('name')}: {', '.join(str(x) for x in sorted(ports))}")
return found
def _check(pillar: str, cid: str, title: str, affected: Optional[List[str]], basis: str) -> Dict[str, Any]:
"""`affected` None means the evidence wasn't available; [] means checked and clean."""
result = "no_evidence" if affected is None else ("fail" if affected else "pass")
return {"pillar": pillar, "check_id": cid, "title": title, "result": result, "affected": affected or [], "basis": basis}
def checks(provider: str, ev: Dict[str, Any]) -> List[Dict[str, Any]]:
servers, stores, rules = ev.get("servers") or [], ev.get("stores") or [], ev.get("rules") or []
compute = servers + (ev.get("workloads") or []) + (ev.get("clusters") or [])
everything = compute + stores
gcp = provider == "gcp"
failed = ev.get("failed") or set()
def read(*keys): # True when every piece of evidence a check rests on was read in full
return not (set(keys) & failed)
out = []
# Security
out.append(_check("security", "SEC-01", "No administrative or database port is open to the internet",
_exposures(provider, rules) if read("rules") else None,
f"{len(rules)} firewall or NSG rule sets read"))
if gcp:
public = [_short(_name(s)) for s in servers
if any(ni.get("accessConfigs") for ni in _resource(s).get("networkInterfaces") or [])]
boot = [_short(_name(s)) for s in servers
if not (_resource(s).get("shieldedInstanceConfig") or {}).get("enableSecureBoot")]
buckets = [s for s in stores if "storage" in str(s.get("assetType", "")).lower()]
open_buckets = [_short(_name(b)) for b in buckets
if str((_resource(b).get("iamConfiguration") or {}).get("publicAccessPrevention", "")).lower() != "enforced"]
else:
public = [_short(_name(p)) for p in ev.get("public_ips") or []]
boot = [_short(_name(s)) for s in servers
if not ((_resource(s).get("securityProfile") or {}).get("uefiSettings") or {}).get("secureBootEnabled")]
buckets = [s for s in stores if "storageaccounts" in str(s.get("type", "")).lower()]
open_buckets = [_short(_name(b)) for b in buckets if _resource(b).get("allowBlobPublicAccess") is not False]
out.append(_check("security", "SEC-02", "Servers have no public address",
public if read("servers", *([] if gcp else ["public_ips"])) and (servers or not gcp) else None,
f"{len(servers)} servers read" + ("" if gcp else f", {len(ev.get('public_ips') or [])} public addresses")))
out.append(_check("security", "SEC-03", "Secure Boot is on for every server", boot if servers else None, f"{len(servers)} servers read"))
out.append(_check("security", "SEC-04", "Storage blocks public access", open_buckets if buckets else None, f"{len(buckets)} storage resources read"))
pol = ev.get("org_policies")
out.append(_check("security", "SEC-05", "Guardrail policies are set at the scope root",
(["no organisation policy or policy assignment found"] if not pol else []) if read("org_policies") else None,
f"{len(pol or [])} policies read"))
# Reliability
zones = sorted({z for s in compute for z in [_zone(s)] if z})
out.append(_check("reliability", "REL-01", "Compute isn't confined to a single zone",
([f"all compute in {zones[0]}"] if len(zones) == 1 and len(compute) > 1 else []) if compute else None,
f"zones: {', '.join(zones) or 'none recorded'}"))
if gcp:
unprotected = [_short(_name(s)) for s in servers if not _resource(s).get("deletionProtection")]
no_backup = [_short(_name(s)) for s in stores if "sqladmin" in str(s.get("assetType", "")).lower()
and not ((_resource(s).get("settings") or {}).get("backupConfiguration") or {}).get("enabled")]
sql = [s for s in stores if "sqladmin" in str(s.get("assetType", "")).lower()]
no_versioning = [_short(_name(b)) for b in buckets if not (_resource(b).get("versioning") or {}).get("enabled")]
else:
locked = {str(l.get("properties", {}).get("scope") or _name(l)).lower() for l in ev.get("locks") or []}
unprotected = [_short(_name(s)) for s in servers if not any(_name(s).lower().startswith(x.rsplit("/providers/microsoft.authorization", 1)[0]) for x in locked)]
sql = [s for s in stores if "sql" in str(s.get("type", "")).lower()]
no_backup = [] # Azure SQL backups are on by default and not exposed by this server's kinds
no_versioning = []
out.append(_check("reliability", "REL-02", "Servers are protected from accidental deletion", unprotected if servers else None,
"deletion protection" if gcp else "resource locks"))
out.append(_check("reliability", "REL-03", "Databases have automated backups", no_backup if sql else None, f"{len(sql)} databases read"))
if gcp:
out.append(_check("reliability", "REL-04", "Storage keeps object versions", no_versioning if buckets else None, f"{len(buckets)} buckets read"))
# Cost and sustainability
stopped = [_short(_name(s)) for s in servers if str(_resource(s).get("status") or s.get("state") or
((_resource(s).get("extended") or {}).get("instanceView") or {}).get("powerState", {}).get("code", "")).upper()
in ("TERMINATED", "STOPPED", "SUSPENDED", "POWERSTATE/DEALLOCATED", "POWERSTATE/STOPPED")]
out.append(_check("cost", "COST-01", "No stopped server is still holding paid resources", stopped if servers else None,
"stopped servers keep their disks and addresses, which are still billed"))
unlabelled = [_short(_name(i)) for i in everything if not any(k.lower() in OWNER_KEYS for k in _labels(i))]
out.append(_check("cost", "COST-02", "Resources carry cost-allocation labels", unlabelled if everything else None,
f"labels looked for: {', '.join(OWNER_KEYS[:4])}…"))
out.append(_check("sustainability", "SUS-01", "No idle or stopped resources", stopped if servers else None,
"the same stopped servers, as wasted capacity"))
# Operations
out.append(_check("operations", "OPS-01", "Resources name an owner or environment", unlabelled if everything else None,
"the same labels identify who runs a resource"))
if gcp and buckets:
no_logs = [_short(_name(b)) for b in buckets if not _resource(b).get("logging")]
out.append(_check("operations", "OPS-02", "Storage access is logged", no_logs, f"{len(buckets)} buckets read"))
return out
def _zone(item: Dict[str, Any]) -> Optional[str]:
m = re.search(r"/zones/([^/]+)", _name(item))
if m:
return m.group(1)
zones = item.get("zones") or _resource(item).get("zones")
return ",".join(zones) if zones else None
# ---------------------------------------------------------------- roll-up ------------------------------------------------
def _impact(provider: str, rec: Dict[str, Any]) -> str:
if provider == "gcp":
return {"P1": "high", "P2": "high", "P3": "medium", "P4": "low"}.get(str(rec.get("priority")), "unknown")
return str((rec.get("properties") or {}).get("impact") or "unknown").lower()
def _summary(provider: str, rec: Dict[str, Any]) -> str:
if provider == "gcp":
return str(rec.get("description") or rec.get("recommenderSubtype") or "")[:160]
p = rec.get("properties") or {}
return str((p.get("shortDescription") or {}).get("problem") or p.get("description") or "")[:160]
def roll_up(provider: str, recs: List[Dict[str, Any]]) -> Dict[str, Any]:
by_pillar: Dict[str, Counter] = defaultdict(Counter)
top: Dict[str, Counter] = defaultdict(Counter)
for r in recs:
pillar = r.get("pillar") or "other"
impact = _impact(provider, r)
by_pillar[pillar][impact] += 1
top[pillar][(impact, _summary(provider, r))] += 1
rank = {"high": 0, "medium": 1, "low": 2}
return {"total": len(recs),
"by_pillar": {p: dict(c) for p, c in by_pillar.items()},
"top": {p: [{"impact": i, "summary": s, "count": n}
for (i, s), n in sorted(c.items(), key=lambda kv: (rank.get(kv[0][0], 3), -kv[1]))[:5]]
for p, c in top.items()}}
def guidance(provider: str, call: Callable, issues: List[str]) -> Dict[str, Dict[str, Any]]:
out = {}
for pillar in PILLARS:
try:
data = _data(call(f"{provider}_search_well_architected", {"pillar": pillar, "topic": TOPICS[pillar]}))
results = data.get("results") or []
first = results[0] if results and isinstance(results[0], dict) else {}
text = first.get("text") or first.get("content") or first.get("snippet") or (results[0] if results else "")
title, link = first.get("title"), first.get("url") or first.get("link")
out[pillar] = {"topic": TOPICS[pillar], "title": title, "url": link, "excerpt": " ".join(str(text).split())[:400],
"evidence_refs": [data["evidence_ref"]] if data.get("evidence_ref") else []}
except Exception as error: # noqa: BLE001
issues.append(f"{provider}_search_well_architected({pillar}): {str(error)[:120]}")
return out
def pillars(chks: List[Dict[str, Any]], recs: Dict[str, Any], guide: Dict[str, Dict[str, Any]]) -> List[Dict[str, Any]]:
out = []
for p in PILLARS:
mine = [c for c in chks if c["pillar"] == p]
counted = recs["by_pillar"].get(p, {})
evidenced = [c for c in mine if c["result"] != "no_evidence"]
failed = [c for c in evidenced if c["result"] == "fail"]
status = "not_assessed" if not evidenced and not counted else ("attention" if failed or counted.get("high") else "reviewed")
out.append({"pillar": p, "title": TITLES[p], "status": status, "checks_passed": len(evidenced) - len(failed),
"checks_failed": len(failed), "checks_no_evidence": len(mine) - len(evidenced),
"recommendations": counted, "top_recommendations": recs["top"].get(p, []),
"guidance": guide.get(p)})
return out
def review(payload: Dict[str, Any], call: Callable) -> Dict[str, Any]:
provider = payload.get("provider") or "sample"
base = {"engagement_id": payload.get("engagement_id"), "run_id": payload.get("run_id"), "provider": provider}
if provider not in KINDS:
result = {**base, "status": "not_applicable", "pillars": [], "checks": [], "recommendations": {"total": 0, "by_pillar": {}, "top": {}},
"call_issues": [], "evidence_refs": [],
"note": "Not applicable: the sample estate has no cloud configuration to review against the Well-Architected Framework."}
else:
scope = [r.get("scope_entry_id") for r in payload.get("records") or [] if r.get("scope_entry_id")]
ev = collect(provider, call, scope)
chks = checks(provider, ev)
recs = roll_up(provider, ev["recommendations"])
guide = guidance(provider, call, ev["issues"])
result = {**base, "status": "assessed", "pillars": pillars(chks, recs, guide), "checks": chks, "recommendations": recs,
"call_issues": ev["issues"], "evidence_refs": ev["evidence_refs"][:200],
"note": (f"{len(scope)} in-scope resources, {recs['total']} provider recommendations. Configuration and "
"recommendations only: no utilisation telemetry, so performance and sustainability are partial.")}
result["waf_markdown"] = render(result, payload)
result["waf_html"] = render_html(result, payload, "Draft. Not for release until the Engagement Lead approves it.")
result["final_waf_html"] = render_html(result, payload)
return result
# ---------------------------------------------------------------- rendering ----------------------------------------------
STATUS_TEXT = {"reviewed": "Reviewed", "attention": "Needs attention", "not_assessed": "Not assessed"}
RESULT_TEXT = {"pass": "Pass", "fail": "Fail", "no_evidence": "No evidence"}
def _md(value: Any) -> str:
if value is None or value == "" or value == []:
return "-"
if isinstance(value, (list, tuple)):
value = ", ".join(str(v) for v in value)
text = " ".join(str(value).split())
for char in ("\\", "|", "`", "*", "_", "<", ">", "[", "]"):
text = text.replace(char, f"\\{char}")
return text
def _table(headers: List[str], rows: List[List[Any]]) -> List[str]:
return ["| " + " | ".join(headers) + " |", "|" + "---|" * len(headers)] + ["| " + " | ".join(_md(v) for v in r) + " |" for r in rows]
def render(r: Dict[str, Any], payload: Dict[str, Any]) -> str:
lines = [f"# Well-Architected review: {_md(r.get('engagement_id'))}", "",
*_table(["Item", "Value"], [["Run", r.get("run_id")], ["Provider", r.get("provider")], ["Status", r.get("status")],
["Discovery critic verdict", payload.get("verdict") or "not supplied"]]), "",
f"> {_md(r.get('note'))}", ""]
if r["status"] != "assessed":
return "\n".join(lines)
lines += ["## Pillars", "", *_table(["Pillar", "Status", "Checks passed", "Checks failed", "No evidence", "Recommendations"],
[[p["title"], STATUS_TEXT[p["status"]], p["checks_passed"], p["checks_failed"], p["checks_no_evidence"],
sum(p["recommendations"].values())] for p in r["pillars"]]), ""]
for p in r["pillars"]:
lines += [f"## {p['title']}", ""]
mine = [c for c in r["checks"] if c["pillar"] == p["pillar"]]
lines += (_table(["Check", "Result", "Affected", "Basis"],
[[f"{c['check_id']} {c['title']}", RESULT_TEXT[c["result"]], c["affected"], c["basis"]] for c in mine])
if mine else ["No fixed check covers this pillar; see the provider's recommendations."]) + [""]
if p["top_recommendations"]:
lines += ["Provider recommendations:", "", *_table(["Impact", "Recommendation", "Count"],
[[t["impact"], t["summary"], t["count"]] for t in p["top_recommendations"]]), ""]
if p.get("guidance") and p["guidance"].get("excerpt"):
lines += [f"Guidance ({_md(p['guidance']['topic'])}): {_md(p['guidance']['excerpt'])}", ""]
if p["status"] == "not_assessed":
lines += ["**Not assessed:** no evidence was available for this pillar. This is not a pass.", ""]
gaps = payload.get("source_gaps") or []
lines += ["## What this review doesn't cover", "",
"- Utilisation telemetry: performance and sustainability rest on configuration and recommendations only."]
lines += [f"- {_md(g.get('source'))}: {_md(g.get('impact'))}" for g in gaps]
if r["call_issues"]:
lines += ["", "## Evidence that couldn't be read", "", *[f"- {_md(i)}" for i in r["call_issues"]]]
return "\n".join(lines) + "\n"
def _h(value: Any) -> str:
if value is None or value == "" or value == []:
return "-"
if isinstance(value, (list, tuple)):
value = ", ".join(str(v) for v in value)
return html.escape(str(value))
TONE = {"Fail": "bad", "Needs attention": "bad", "No evidence": "warn", "Not assessed": "warn", "Pass": "ok", "Reviewed": "ok", "high": "bad",
"medium": "warn", "low": "ok"}
STYLE = """
:root { --ground:#f4f6f8; --surface:#fff; --ink:#16202c; --muted:#566373; --line:#dbe1e8; --band:#10283f; --band-ink:#f2f6fa;
--band-muted:#b5c4d3; --bad:#9f2229; --bad-bg:#fbe8e9; --warn:#855700; --warn-bg:#fcf0d5; --ok:#1a6640; --ok-bg:#e1f2e8;
--sans:"Segoe UI",system-ui,-apple-system,Roboto,Arial,sans-serif; --serif:Georgia,"Times New Roman",serif; }
@media (prefers-color-scheme: dark) { :root:not([data-theme="light"]) { --ground:#10161d; --surface:#182029; --ink:#e6ebf1;
--muted:#9dabb9; --line:#2e3945; --band:#0b1f33; --bad:#ff9fa3; --bad-bg:#40191d; --warn:#f0c46e; --warn-bg:#3a2c10; --ok:#84d9a8;
--ok-bg:#153223; color-scheme: dark; } }
:root[data-theme="dark"] { --ground:#10161d; --surface:#182029; --ink:#e6ebf1; --muted:#9dabb9; --line:#2e3945; --band:#0b1f33;
--bad:#ff9fa3; --bad-bg:#40191d; --warn:#f0c46e; --warn-bg:#3a2c10; --ok:#84d9a8; --ok-bg:#153223; color-scheme: dark; }
* { box-sizing: border-box; } body { margin:0; background:var(--ground); color:var(--ink); font:16px/1.55 var(--sans); }
.wrap { max-width:68rem; margin:0 auto; padding-left:max(16px,env(safe-area-inset-left,0px)); padding-right:max(16px,env(safe-area-inset-right,0px)); }
.cover { background:var(--band); color:var(--band-ink); } .cover .wrap { padding-block:3rem 2.5rem; display:grid; gap:.6rem; }
.eyebrow { margin:0; color:var(--band-muted); font-size:.8rem; letter-spacing:.08em; text-transform:uppercase; }
h1 { margin:0; font:400 2.6rem/1.15 var(--serif); } h2 { margin:0 0 .75rem; font:400 1.5rem/1.25 var(--serif); }
main.wrap { padding-block:2rem 4rem; display:grid; gap:2.5rem; } main > * { min-width:0; }
.notice { margin:0; padding:.75rem 1rem; border-radius:4px; background:var(--warn-bg); color:var(--warn); }
.scroll { overflow-x:auto; background:var(--surface); border:1px solid var(--line); }
table { border-collapse:collapse; width:100%; font-size:.9rem; } th { text-align:left; color:var(--muted); font-size:.8rem; }
th, td { padding:.55rem .8rem; border-bottom:1px solid var(--line); vertical-align:top; } tr:last-child td { border-bottom:0; }
.pill { display:inline-block; padding:0 .45rem; border-radius:3px; font-size:.8rem; font-weight:600; white-space:nowrap; }
.pill.bad { background:var(--bad-bg); color:var(--bad); } .pill.warn { background:var(--warn-bg); color:var(--warn); }
.pill.ok { background:var(--ok-bg); color:var(--ok); } .muted { color:var(--muted); margin:.5rem 0 0; font-size:.9rem; }
@media (max-width:30rem) { h1 { font-size:2rem; } }
@media print { body { background:#fff; } .cover, .pill, .notice { print-color-adjust:exact; -webkit-print-color-adjust:exact; } section { break-inside:avoid-page; } }
"""
def _cell(v: Any) -> str:
tone = TONE.get(str(v))
return f'<span class="pill {tone}">{_h(v)}</span>' if tone else _h(v)
def _htable(headers: List[str], rows: List[List[Any]]) -> str:
head = "".join(f'<th scope="col">{_h(h)}</th>' for h in headers)
body = "".join("<tr>" + "".join(f"<td>{_cell(v)}</td>" for v in row) + "</tr>" for row in rows)
return f'<div class="scroll"><table><thead><tr>{head}</tr></thead><tbody>{body}</tbody></table></div>'
def render_html(r: Dict[str, Any], payload: Dict[str, Any], notice: Optional[str] = None) -> str:
parts = [f'<p class="notice">{_h(notice)}</p>'] if notice else []
parts.append(f'<section><h2>Overview</h2>{_htable(["Item", "Value"], [["Provider", r.get("provider")], ["Status", r.get("status")], ["Discovery critic verdict", payload.get("verdict") or "not supplied"]])}'
f'<p class="muted">{_h(r.get("note"))}</p></section>')
if r["status"] == "assessed":
parts.append("<section><h2>Pillars</h2>" + _htable(["Pillar", "Status", "Passed", "Failed", "No evidence", "Recommendations"],
[[p["title"], STATUS_TEXT[p["status"]], p["checks_passed"], p["checks_failed"], p["checks_no_evidence"],
sum(p["recommendations"].values())] for p in r["pillars"]]) + "</section>")
for p in r["pillars"]:
mine = [c for c in r["checks"] if c["pillar"] == p["pillar"]]
body = _htable(["Check", "Result", "Affected", "Basis"], [[c["title"], RESULT_TEXT[c["result"]], c["affected"], c["basis"]] for c in mine]) if mine else ""
if p["top_recommendations"]:
body += '<p class="muted">Provider recommendations</p>' + _htable(["Impact", "Recommendation", "Count"],
[[t["impact"], t["summary"], t["count"]] for t in p["top_recommendations"]])
if p["status"] == "not_assessed":
body += '<p class="muted">Not assessed: no evidence was available for this pillar. This is not a pass.</p>'
parts.append(f"<section><h2>{_h(p['title'])}</h2>{body}</section>")
gaps = "".join(f"<li>{_h(g.get('source'))}: {_h(g.get('impact'))}</li>" for g in payload.get("source_gaps") or [])
parts.append("<section><h2>What this review doesn't cover</h2><ul><li>Utilisation telemetry: performance and sustainability "
f"rest on configuration and recommendations only.</li>{gaps}</ul></section>")
cover = (f'<header class="cover"><div class="wrap"><p class="eyebrow">Well-Architected review</p><h1>{_h(r.get("engagement_id"))}</h1>'
f'<p class="eyebrow">Run {_h(r.get("run_id"))} · {date.today():%-d %B %Y}</p></div></header>')
return (f'<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">'
f'<title>Well-Architected review {_h(r.get("engagement_id"))}</title><style>{STYLE}</style></head>'
f'<body>{cover}<main class="wrap">{"".join(parts)}</main></body></html>')

View File

@@ -0,0 +1,9 @@
# Decisions behind ALZ-DSC-10's requirements
Its story, ALZ-DSC-10, is a local draft (`drafts/cp1/agentic-landing-zone/ALZ-DSC-10-story-draft.md` in the engineering hub workspace), added by ADR 0014. The decisions are recorded once, for all agents, in the Feature's [docs/adr/](../../../docs/adr/README.md). These apply to this agent:
- [ADR 0013](../../../docs/adr/0013.md)
- [ADR 0014](../../../docs/adr/0014.md)
- [ADR 0016](../../../docs/adr/0016.md)
A change that affects only this agent gets its own record here, numbered `10-NNNN`.

View File

@@ -0,0 +1,17 @@
# Eval cases for CP1 - Well-Architected Review Agent.
schema_version: "1.0"
agent: "cp1-well-architected-review-agent"
defaults:
timeout: 330
cases:
- id: "sample-is-not-applicable"
description: "The Meridian sample estate has no cloud configuration: the review is not applicable and calls no tool."
input:
json:
engagement_id: "ENG-MERIDIAN-001"
run_id: "RUN-EVAL-01"
provider: "sample"
checks:
- schema: true
- field: "status"
equals: "not_applicable"

View File

@@ -0,0 +1,111 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
data:
# This file is the agent's configuration contract. deployment.yaml mounts it
# with `envFrom: configMapRef`, so every key below arrives in the container as
# an environment variable. ./run-local.sh reads the same file, so local runs
# and deployed runs share one source of truth.
#
# Values are read once at process start. Changing a key requires a restart
# (`kubectl rollout restart deploy/<agent>`), not just a ConfigMap apply.
#
# Secrets do NOT belong here — see kubernetes/secretnames.yaml.
# =============================================================================
# Platform Variables (set by Kustomize/Deployment)
# =============================================================================
PORT: "8080"
LOG_LEVEL: "INFO" # DEBUG adds prompts, payloads and replies to the log
# json: one object per line, for a log store. console: aligned columns, for
# reading by eye (set it in .env on your machine).
LOG_FORMAT: "json"
# =============================================================================
# Required Agent Configuration
# =============================================================================
# The agent's name, description, version and framework -- and its tags,
# capabilities and skills -- live in agent.yaml, the source of truth for all
# of them. They are not read from here.
# LLM model (required) - MUST use *LLM_MODEL or *SLM_MODEL pattern for watcher
# Any model the platform's gateway offers, e.g. gpt-5.4, claude-sonnet-4-6.
LITELLM_MODEL: "gpt-5.6-sol"
# =============================================================================
# Optional Model Settings
# =============================================================================
# These MUST carry the LITELLM_ prefix. The runtime configures the model via
# build_model(), which reads LITELLM_TEMPERATURE / LITELLM_MAX_TOKENS
# (defaults 0.0 / 16384). The unprefixed LLM_TEMPERATURE / LLM_MAX_TOKENS names
# are read by a different SDK code path that does not configure this agent's
# model, so setting those is silently ignored — the agent keeps running at the
# defaults with no error to chase.
LITELLM_TEMPERATURE: "1"
LITELLM_MAX_TOKENS: "80000"
# =============================================================================
# Agent Loop Guard Rails
# =============================================================================
# An agent with tools makes as many model calls as the model decides it
# needs. These bound that. Any one of them ends the loop, and the caller
# receives an answer explaining why rather than an error.
#
# The defaults sit well above normal work, so they catch runaways — most
# often a tool bug that keeps telling the model to call it again — without
# affecting an agent that is behaving. Measure your agent's typical call
# count in the `agent_loop_completed` log line before tightening them.
#
# An agent with no tools cannot loop: one request is one model call, and
# these settings have nothing to bound.
#
AGENT_GUARDRAILS_ENABLED: "true" # "false" removes the bounds; the run is still counted and logged
AGENT_MAX_ITERATIONS: "25" # model calls per run: the agent's, or one sub-agent's
# Model calls in the whole request, the sub-agents the model calls included
# (runtime 0.28.2 or later). Default 4 x AGENT_MAX_ITERATIONS.
# AGENT_MAX_REQUEST_MODEL_CALLS: "100"
AGENT_MAX_DURATION_SECONDS: "1140" # wall clock per request (19 minutes): inside the workflow step's 20, so the agent replies before the step is cut off
AGENT_MAX_TOKENS: "0" # total tokens per request; 0 = no limit
# =============================================================================
# Connections: set by the platform, not here
# =============================================================================
# The model gateway (LITELLM_PROXY_BASE, GATEWAY_TYPE), the MCP gateway
# (MCP_GATEWAY_URL) and where telemetry goes (OBSERVABILITY_TYPE,
# OTEL_EXPORTER_OTLP_ENDPOINT) are the same for every agent in an environment.
# The platform supplies them in the agents namespace (ConfigMap
# agent-platform-connections), and the Deployment reads them last, so a value
# here would be overridden. Telemetry goes to the platform's OpenTelemetry
# Collector, which holds any keys. Locally, .env holds them (.env.example).
# =============================================================================
# Inbound checks (runtime 0.25.0 or later)
# =============================================================================
# What the agent accepts before any of its work starts.
AGENT_MAX_CONCURRENT_REQUESTS: "100" # one more at once gets HTTP 503, which Temporal retries
# AGENT_MAX_REQUEST_BYTES: "67108864" # 64 MiB; a larger request body gets HTTP 413
# The name of a guardrail configured on the LLM gateway. Each request's text
# is screened by it before the agent runs; a blocked request fails with the
# gateway's reason, with no model call and no cost.
# AGENT_INPUT_GUARDRAIL: ""
# AGENT_INBOUND_TOKEN is a secret, so it is not set here (see
# kubernetes/secretnames.yaml). The platform does not send it yet: leave it unset.
# =============================================================================
# Outbound connections (runtime 0.25.0 or later)
# =============================================================================
# The agent may connect to its own endpoints: the LLM gateway, the MCP
# gateway, its telemetry, localhost, and the hosts in AGENT_EGRESS_ALLOW.
# enforce (default): a connection to any other host is refused and logged
# as egress_blocked, with the file and line that made it.
# log: such a connection goes through and is logged as egress_unlisted.
# off: not watched.
# Reach outside systems through the MCP gateway rather than adding hosts here.
# The cluster's network policy remains the real boundary.
AGENT_EGRESS: "enforce"
# AGENT_EGRESS_ALLOW: "" # comma-separated hosts; *.<domain> for a domain and its subdomains
# The review handles a client's estate data: keep prompts, replies and tool data out of traces.
OBSERVABILITY_CAPTURE_CONTENT: "false"

View File

@@ -0,0 +1,81 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
labels:
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
spec:
replicas: 1
selector:
matchLabels:
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
template:
metadata:
labels:
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
spec:
imagePullSecrets:
- name: jfrog-secret
containers:
- name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
image: <placeholder_for_image> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
imagePullPolicy: Always
ports:
- containerPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
name: http
# Load all environment variables from ConfigMap and Secrets
envFrom:
- secretRef:
name: <placeholder_for_agent_name>-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
optional: true
- secretRef:
name: <placeholder_for_agent_name>-dotenv-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
optional: true
- configMapRef:
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
# The platform's connections, the same for every agent: the model
# gateway, the MCP gateway and where telemetry goes. Last, so they win
# over anything above; an agent does not choose them. Created by the
# platform's Helm chart in this namespace.
- configMapRef:
name: agent-platform-connections
optional: true
# Platform-provided credentials
env:
- name: LITELLM_MASTER_KEY
valueFrom:
secretKeyRef:
name: agent-builder-secrets
key: litellm-master-key
volumeMounts:
- name: execution-storage
mountPath: /data/executions
resources:
requests:
memory: "128Mi"
cpu: "10m"
limits:
memory: "1024Mi"
cpu: "1000m"
# Health checks use A2A agent card endpoint
livenessProbe:
httpGet:
path: /.well-known/agent-card.json
port: <placeholder_for_port>
initialDelaySeconds: 30
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 5
readinessProbe:
httpGet:
path: /.well-known/agent-card.json
port: <placeholder_for_port>
initialDelaySeconds: 30
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 5
volumes:
- name: execution-storage
persistentVolumeClaim:
claimName: execution-storage-pvc # Same PVC name as core-components

View File

@@ -0,0 +1,20 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- deployment.yaml
- service.yaml
- configmap.yaml
- secretnames.yaml
# Optionally add common labels
# Namespace for all resources
namespace: agents
labels:
- includeSelectors: true
pairs:
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
environment: production
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline

View File

@@ -0,0 +1,7 @@
# Do not modify this file, otherwise CI/CD pipeline will fail
apiVersion: v1
kind: Namespace
metadata:
name: agents
labels:
name: agents

View File

@@ -0,0 +1,34 @@
apiVersion: v1
kind: Secret
metadata:
name: <placeholder_for_agent_name>-secrets
type: Opaque
data:
# Base64 encoded values (NOT RECOMMENDED for sensitive data)
# To encode: echo -n 'your-secret-value' | base64
# Example: api-key: eW91ci1hcGkta2V5LXZhbHVl
stringData:
# Plain text values saved on GitHub Secrets (RECOMMENDED)
# Example:
#
# DB_PASSWORD: ${DB_PASSWORD_GITHUB_SECRET}
# ANOTHER_SECRET: ${ANOTHER_GITHUB_SECRET}
#
# ${DB_PASSWORD_GITHUB_SECRET} and ${ANOTHER_GITHUB_SECRET} must exist as GitHub Secrets
#
# AGENT_INBOUND_TOKEN (runtime 0.25.0 or later): when set, every call to the
# agent must carry it as a bearer token, and other calls get HTTP 401. It is
# a secret, so it goes here, never in the ConfigMap. The platform does not
# send this token to agents yet, so leave it unset for now.
# AGENT_INBOUND_TOKEN: ${AGENT_INBOUND_TOKEN_GITHUB_SECRET}
#
# Telemetry keys are not here: agents send to the platform's OpenTelemetry
# Collector, which holds them.
#
# AGENT_CONTINUATION_KEY (runtime 0.28.0 or later): signs the conversation the
# agent hands the platform when it pauses for a person's approval, so it can
# carry on from there. A long random string, different for each agent. Unset,
# each process makes its own at start-up: an answer that reaches another
# replica, or comes after a restart, runs the step again from the start.
# AGENT_CONTINUATION_KEY: ${AGENT_CONTINUATION_KEY_GITHUB_SECRET}

View File

@@ -0,0 +1,33 @@
apiVersion: v1
kind: Service
metadata:
name: <placeholder_for_agent_name>
labels:
app: <placeholder_for_agent_name>
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
spec:
type: ClusterIP
ports:
- port: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
targetPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
protocol: TCP
name: http
selector:
app: <placeholder_for_agent_name>
---
# Optional: LoadBalancer service for external access
# apiVersion: v1
# kind: Service
# metadata:
# name: <placeholder_for_agent_name>-external
# labels:
# app: <placeholder_for_agent_name>
# spec:
# type: LoadBalancer
# ports:
# - port: 80
# targetPort: 8080
# protocol: TCP
# name: http
# selector:
# app: <placeholder_for_agent_name>

View File

@@ -0,0 +1,3 @@
# CP1 - Well-Architected Review Agent
The review is collected, judged and rendered in `app/waf.py` by the `pre_process` hook, which replies without calling the model. If this prompt is ever reached, reply with only: `{"error": "review not rendered"}`. Never call a tool from here.

View File

@@ -0,0 +1,50 @@
{
"type": "object",
"title": "Well-Architected review input",
"required": [
"engagement_id",
"run_id"
],
"properties": {
"engagement_id": {
"type": "string"
},
"run_id": {
"type": "string"
},
"provider": {
"type": "string",
"enum": [
"sample",
"azure",
"gcp"
],
"default": "sample",
"description": "Which estate this run reads: `sample` (not applicable here) or a live cloud, `azure` or `gcp`."
},
"verdict": {
"type": [
"string",
"null"
],
"default": null,
"description": "The Discovery critic's verdict, shown on the review."
},
"records": {
"type": "array",
"default": [],
"items": {
"type": "object"
},
"description": "The canonical inventory: its scope entry ids bound what is judged."
},
"source_gaps": {
"type": "array",
"default": [],
"items": {
"type": "object"
},
"description": "Sources the run had no connection to."
}
}
}

View File

@@ -0,0 +1,74 @@
{
"type": "object",
"title": "Well-Architected review",
"required": [
"engagement_id",
"run_id",
"provider",
"status",
"pillars",
"checks",
"recommendations",
"call_issues",
"waf_markdown",
"waf_html",
"final_waf_html"
],
"properties": {
"engagement_id": {
"type": "string"
},
"run_id": {
"type": "string"
},
"provider": {
"type": "string"
},
"status": {
"type": "string",
"enum": [
"assessed",
"not_applicable"
]
},
"note": {
"type": "string"
},
"pillars": {
"type": "array",
"items": {
"type": "object"
}
},
"checks": {
"type": "array",
"items": {
"type": "object"
}
},
"recommendations": {
"type": "object"
},
"call_issues": {
"type": "array",
"items": {
"type": "string"
}
},
"evidence_refs": {
"type": "array",
"items": {
"type": "string"
}
},
"waf_markdown": {
"type": "string"
},
"waf_html": {
"type": "string"
},
"final_waf_html": {
"type": "string"
}
}
}

View File

@@ -0,0 +1,109 @@
"""The Well-Architected review in code: checks, roll-up and rendering, against stand-in tool replies (no runtime)."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from app import waf # noqa: E402
VM = "//compute.googleapis.com/projects/p1/zones/europe-west2-b/instances/vm-1"
BUCKET = "//storage.googleapis.com/bucket-1"
def ok(items, token=None):
return {"success": True, "data": {"items": items, "next_page_token": token, "evidence_ref": "sha256:x"}}
def gcp_call(calls):
vm = {"name": VM, "assetType": "compute.googleapis.com/Instance", "labels": {},
"versionedResources": [{"resource": {"data": {"status": "TERMINATED", "deletionProtection": False,
"shieldedInstanceConfig": {"enableSecureBoot": False},
"networkInterfaces": [{"network": "n"}]}}}]}
bucket = {"name": BUCKET, "assetType": "storage.googleapis.com/Bucket", "labels": {"owner": "platform"},
"versionedResources": [{"resource": {"data": {"iamConfiguration": {"publicAccessPrevention": "inherited"}}}}]}
def call(tool, args):
calls.append((tool, args))
if tool == "gcp_list_servers":
return ok([vm])
if tool == "gcp_list_data_stores":
return ok([bucket] if args.get("kind") == "cloud_storage" else [])
if tool == "gcp_list_organization_policies":
return {"success": False, "error": {"code": "upstream_failed", "message": "403"}}
if tool == "gcp_list_recommendations":
if args["recommender"] == "google.compute.instance.IdleResourceRecommender":
return ok([{"pillar": "cost", "priority": "P2", "description": "Delete idle VM vm-1"}])
return ok([])
if tool == "gcp_search_well_architected":
return {"success": True, "data": {"results": [{"title": "Design", "text": "Guidance text."}], "evidence_ref": "sha256:g"}}
return ok([])
return call
def by_id(result):
return {c["check_id"]: c for c in result["checks"]}
def test_sample_is_not_applicable_and_calls_nothing():
calls = []
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "sample"}, lambda t, a: calls.append(t))
assert r["status"] == "not_applicable" and calls == [] and "not applicable" in r["waf_markdown"].lower()
def test_gcp_checks_match_the_live_estate():
calls = []
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "gcp",
"records": [{"scope_entry_id": VM}, {"scope_entry_id": BUCKET}]}, gcp_call(calls))
c = by_id(r)
assert r["status"] == "assessed"
assert c["SEC-01"]["result"] == "pass" # no firewall rule opens a port
assert c["SEC-03"]["affected"] == ["vm-1"] # Secure Boot off
assert c["SEC-04"]["affected"] == ["bucket-1"] # public access not enforced
assert c["SEC-05"]["result"] == "no_evidence" # the policy call was refused: not a pass
assert c["REL-02"]["affected"] == ["vm-1"] # deletion protection off
assert c["COST-01"]["affected"] == ["vm-1"] # TERMINATED
assert c["COST-02"]["affected"] == ["vm-1"] # the bucket has an owner label
assert r["recommendations"]["by_pillar"] == {"cost": {"high": 1}}
assert any("gcp_list_organization_policies" in i for i in r["call_issues"])
assert not any(t.startswith("azure_") for t, _ in calls) # one cloud only
zones = {a["location"] for t, a in calls if t == "gcp_list_recommendations" and "instance" in a["recommender"]}
assert zones == {"europe-west2-b"}
def test_pillar_with_no_evidence_is_not_assessed():
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "gcp", "records": []}, lambda t, a: ok([]))
perf = next(p for p in r["pillars"] if p["pillar"] == "performance")
assert perf["status"] == "not_assessed"
assert "This is not a pass" in r["waf_markdown"]
def test_azure_open_rdp_and_public_storage():
nsg = {"name": "nsg-1", "id": "/subscriptions/s/resourceGroups/g/providers/Microsoft.Network/networkSecurityGroups/nsg-1",
"properties": {"securityRules": [{"name": "rdp", "properties": {"access": "Allow", "direction": "Inbound",
"sourceAddressPrefix": "*", "destinationPortRange": "3389"}}]}}
sa = {"id": "/subscriptions/s/resourceGroups/g/providers/Microsoft.Storage/storageAccounts/sa1",
"type": "microsoft.storage/storageaccounts", "properties": {"allowBlobPublicAccess": True}}
def call(tool, args):
if tool == "azure_get_network_topology" and args.get("kind") == "network_security_groups":
return ok([nsg])
if tool == "azure_list_data_stores" and args.get("kind") == "storage_accounts":
return ok([sa])
if tool == "azure_list_recommendations":
return ok([{"pillar": "security", "properties": {"impact": "High", "shortDescription": {"problem": "Enable MFA"}}}])
return ok([])
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "azure", "records": [{"scope_entry_id": sa["id"]}]}, call)
c = by_id(r)
assert c["SEC-01"]["affected"] == ["nsg-1/rdp: 3389"]
assert c["SEC-04"]["affected"] == ["sa1"]
assert r["recommendations"]["by_pillar"]["security"] == {"high": 1}
def test_paging_reads_every_page():
pages = {None: ok([{"name": "a"}], "t2"), "t2": ok([{"name": "b"}])}
seen = waf._all_pages(lambda t, a: pages[a.get("page_token")], "x", {}, [], [])
assert [i["name"] for i in seen] == ["a", "b"]
def test_html_escapes_values():
r = waf.review({"engagement_id": "<script>", "run_id": "R", "provider": "sample"}, lambda t, a: None)
assert "<script>" not in r["final_waf_html"] and "&lt;script&gt;" in r["final_waf_html"]