ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
# cp1-well-architected-review-agent
|
||||
|
||||
CP1 Well-Architected review (ALZ-DSC-10): one step of the CP1 Discovery workflow. It answers from code, like the
|
||||
report agent (ADR 0014); keep it that way. Every tool it calls is read-only and declared in `agent.yaml` under the
|
||||
gateway's display names. Add a check in `app/waf.py` with a test in `tests/`, and a row in README.md.
|
||||
@@ -0,0 +1,6 @@
|
||||
ARG RUNTIME_IMAGE=cruciblecoredevacr.azurecr.io/kab/agent-runtime-adk:0.35.7
|
||||
FROM ${RUNTIME_IMAGE}
|
||||
COPY agent.yaml prompt.md /agent/
|
||||
COPY schemas/ /agent/schemas/
|
||||
COPY eval/ /agent/eval/
|
||||
COPY app/ /agent/app/
|
||||
@@ -0,0 +1,33 @@
|
||||
# CP1 - Well-Architected Review Agent
|
||||
|
||||
Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML.
|
||||
|
||||
A step of the workflow "CP1 - Agentic Landing Zone - Discovery", after the critic and before the report. Like the
|
||||
report agent, it answers from code (`app/waf.py`, called by the `pre_process` hook), so no model is involved and every
|
||||
figure can be recomputed from the evidence. With `provider` `sample` it replies "not applicable" without calling a tool.
|
||||
|
||||
## What it reads
|
||||
|
||||
Only read-only tools on the existing servers, through the MCP gateway, with the project's stored connection:
|
||||
`Azure Discovery` or `Google Cloud Discovery` (servers, workloads, clusters, data stores, firewall or NSG rules,
|
||||
organisation policies, Azure resource locks, every page of the provider's recommendations, and one guidance reference
|
||||
per pillar from the server's own `<provider>_search_well_architected`, which takes only fixed pillars and topics). The checks judge only the scope entries in `records`.
|
||||
|
||||
## What it writes
|
||||
|
||||
`waf_markdown` (the file), `waf_html` (the draft page) and `final_waf_html` (the page released after approval),
|
||||
plus the structured `pillars`, `checks` and `recommendations`. A pillar with no evidence is **not assessed**, never a
|
||||
pass. Calls that failed are listed in `call_issues` and in the review.
|
||||
|
||||
## Checks
|
||||
|
||||
| Pillar | Checks |
|
||||
|---|---|
|
||||
| Security | SEC-01 no admin or database port open to the internet; SEC-02 no public server address; SEC-03 Secure Boot on; SEC-04 storage blocks public access; SEC-05 guardrail policies at the scope root |
|
||||
| Reliability | REL-01 compute not confined to one zone; REL-02 deletion protection (GCP) or resource locks (Azure); REL-03 database backups; REL-04 object versioning (GCP) |
|
||||
| Cost | COST-01 no stopped server holding paid resources; COST-02 cost-allocation labels |
|
||||
| Operations | OPS-01 owner or environment labels; OPS-02 storage access logging (GCP) |
|
||||
| Sustainability | SUS-01 no idle or stopped resources |
|
||||
| Performance | Provider recommendations only (no telemetry) |
|
||||
|
||||
Tests: `python3 -m pytest tests/` (no runtime needed).
|
||||
@@ -0,0 +1,11 @@
|
||||
## [0.2.0] - 2026-11-04
|
||||
|
||||
### Changed
|
||||
|
||||
- Guidance comes from the provider's own discovery server (`azure_search_well_architected`, `gcp_search_well_architected`), not the separate `cloud-knowledge` server, at the product owner's direction.
|
||||
|
||||
## [0.1.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- First release (ALZ-DSC-10, ADR 0014): Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML.
|
||||
@@ -0,0 +1,46 @@
|
||||
name: "cp1-well-architected-review-agent"
|
||||
display_name: "CP1 - Well-Architected Review Agent"
|
||||
description: "Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML."
|
||||
version: "0.2.0"
|
||||
framework: "adk"
|
||||
tags:
|
||||
- "cp1"
|
||||
- "public-cloud"
|
||||
- "discovery"
|
||||
capabilities:
|
||||
streaming: false
|
||||
push_notifications: false
|
||||
prompt: "prompt.md"
|
||||
contract:
|
||||
input: "schemas/input.json"
|
||||
output: "schemas/output.json"
|
||||
hooks: app.business_logic
|
||||
mcp:
|
||||
"Azure Discovery":
|
||||
- "azure_list_servers"
|
||||
- "azure_list_workloads"
|
||||
- "azure_list_kubernetes_clusters"
|
||||
- "azure_list_data_stores"
|
||||
- "azure_get_network_topology"
|
||||
- "azure_list_organization_policies"
|
||||
- "azure_list_resource_policies"
|
||||
- "azure_list_recommendations"
|
||||
- "azure_search_well_architected"
|
||||
"Google Cloud Discovery":
|
||||
- "gcp_list_servers"
|
||||
- "gcp_list_workloads"
|
||||
- "gcp_list_kubernetes_clusters"
|
||||
- "gcp_list_data_stores"
|
||||
- "gcp_get_network_topology"
|
||||
- "gcp_list_organization_policies"
|
||||
- "gcp_list_recommendations"
|
||||
- "gcp_search_well_architected"
|
||||
skills:
|
||||
- id: "cp1_well_architected_review_agent"
|
||||
name: "CP1 - Well-Architected Review Agent"
|
||||
description: "Reviews a live cloud estate against the six Well-Architected pillars in code: fixed checks on its configuration, the provider's own recommendations by pillar and impact, and guidance references, rendered as Markdown and HTML."
|
||||
tags:
|
||||
- "cp1"
|
||||
- "public-cloud"
|
||||
- "discovery"
|
||||
examples: []
|
||||
@@ -0,0 +1,13 @@
|
||||
"""Answers from pre_process: the review comes from app/waf.py, so no model is called (ADR 0014)."""
|
||||
import json
|
||||
from typing import Any, Optional
|
||||
|
||||
from kyndryl_agent_runtime.request_cycle import Reply
|
||||
from kyndryl_agent_runtime.tools import call_tool
|
||||
|
||||
from app import waf
|
||||
|
||||
|
||||
def pre_process(payload: dict) -> Optional[Any]:
|
||||
"""Collect the evidence through the declared read-only tools, judge it, and render the review."""
|
||||
return Reply(json.dumps(waf.review(payload, call_tool)))
|
||||
@@ -0,0 +1,487 @@
|
||||
"""The Well-Architected review: evidence collected and judged in code, rendered as Markdown and HTML (ADR 0014).
|
||||
|
||||
`review(payload, call)` reads the live estate through the cloud discovery server's read-only tools, applies fixed
|
||||
checks per pillar, rolls up the provider's own recommendations by pillar and impact, and adds guidance references
|
||||
from the provider's own discovery server. No model is involved, so every figure can be recomputed from the evidence. `call(name, args)`
|
||||
is the runtime's call_tool; tests pass a stand-in. A pillar with no evidence is "not assessed", never a pass.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import html
|
||||
import re
|
||||
from collections import Counter, defaultdict
|
||||
from datetime import date
|
||||
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple
|
||||
|
||||
PILLARS = ("security", "reliability", "cost", "operations", "performance", "sustainability")
|
||||
TITLES = {"security": "Security", "reliability": "Reliability", "cost": "Cost optimisation",
|
||||
"operations": "Operational excellence", "performance": "Performance efficiency", "sustainability": "Sustainability"}
|
||||
# The provider's own discovery server accepts only these fixed topics (mcp-servers/_common/knowledge_support.py).
|
||||
TOPICS = {"security": "network segmentation", "reliability": "high availability", "cost": "cost allocation",
|
||||
"operations": "logging and monitoring", "performance": "scaling", "sustainability": "virtual machines"}
|
||||
ADMIN_PORTS = {22, 3389, 21, 1433, 3306, 5432}
|
||||
OPEN = {"0.0.0.0/0", "::/0", "*", "internet", "any"}
|
||||
OWNER_KEYS = ("owner", "team", "cost-center", "cost_center", "costcenter", "environment", "env", "application", "app")
|
||||
PAGE_SIZE = 200
|
||||
MAX_PAGES = 40 # per call: 8,000 items is far beyond the estates seen so far, and bounds a runaway
|
||||
|
||||
KINDS = {
|
||||
"gcp": {"servers": ("gcp_list_servers", ["instances"]), "workloads": ("gcp_list_workloads", ["cloud_run", "cloud_functions", "app_engine"]),
|
||||
"clusters": ("gcp_list_kubernetes_clusters", ["clusters"]), "stores": ("gcp_list_data_stores", ["cloud_sql", "firestore", "cloud_storage"]),
|
||||
"rules": ("gcp_get_network_topology", ["firewall_rules"]),
|
||||
"org_policies": ("gcp_list_organization_policies", ["organization_policies"])},
|
||||
"azure": {"servers": ("azure_list_servers", ["virtual_machines", "scale_sets"]),
|
||||
"workloads": ("azure_list_workloads", ["app_service", "container_apps", "container_instances"]),
|
||||
"clusters": ("azure_list_kubernetes_clusters", [None]),
|
||||
"stores": ("azure_list_data_stores", ["sql", "cosmos_db", "storage_accounts", "postgresql", "mysql", "redis"]),
|
||||
"rules": ("azure_get_network_topology", ["network_security_groups"]),
|
||||
"public_ips": ("azure_get_network_topology", ["public_ips"]),
|
||||
"org_policies": ("azure_list_organization_policies", ["assignments"]),
|
||||
"locks": ("azure_list_resource_policies", ["resource_locks"])},
|
||||
}
|
||||
GCP_RECOMMENDERS = ("google.compute.instance.MachineTypeRecommender", "google.compute.instance.IdleResourceRecommender",
|
||||
"google.compute.disk.IdleResourceRecommender", "google.iam.policy.Recommender")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- evidence ----------------------------------------------
|
||||
|
||||
def _data(result: Any) -> Dict[str, Any]:
|
||||
"""The servers' envelope, {success, data} or {success: false, error: {code, message}}; a refusal raises."""
|
||||
if not isinstance(result, dict):
|
||||
raise ValueError(f"unexpected reply {type(result).__name__}")
|
||||
if result.get("success") is False or (result.get("error") and "data" not in result):
|
||||
error = result.get("error") or {}
|
||||
raise ValueError(f"{error.get('code', 'refused')}: {error.get('message', '')}"[:200] if isinstance(error, dict) else str(error)[:200])
|
||||
return result.get("data") if isinstance(result.get("data"), dict) else result
|
||||
|
||||
|
||||
def _items(result: Any) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[str]]:
|
||||
"""A tool reply's items, next page token and evidence reference; a refusal raises."""
|
||||
data = _data(result)
|
||||
return list(data.get("items") or []), data.get("next_page_token"), data.get("evidence_ref")
|
||||
|
||||
|
||||
def _all_pages(call: Callable, tool: str, args: Dict[str, Any], issues: List[str], refs: List[str]) -> List[Dict[str, Any]]:
|
||||
out, token = [], None
|
||||
for _ in range(MAX_PAGES):
|
||||
try:
|
||||
items, token, ref = _items(call(tool, {**args, "page_size": PAGE_SIZE, **({"page_token": token} if token else {})}))
|
||||
except Exception as error: # noqa: BLE001 - a refusal or failure is evidence of a gap, not a crash
|
||||
issues.append(f"{tool}({', '.join(f'{k}={v}' for k, v in args.items())}): {str(error)[:160]}")
|
||||
return out
|
||||
out += items
|
||||
if ref:
|
||||
refs.append(ref)
|
||||
if not token:
|
||||
return out
|
||||
issues.append(f"{tool}: stopped after {MAX_PAGES} pages")
|
||||
return out
|
||||
|
||||
|
||||
def _resource(item: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""The provider's own resource body: GCP versionedResources[0].resource.data, or an Azure row's properties."""
|
||||
versions = item.get("versionedResources") or []
|
||||
if versions:
|
||||
body = (versions[0] or {}).get("resource") or {}
|
||||
return body.get("data") or body
|
||||
return item.get("properties") or item.get("resource", {}).get("data") or {}
|
||||
|
||||
|
||||
def _name(item: Dict[str, Any]) -> str:
|
||||
return str(item.get("name") or item.get("id") or "")
|
||||
|
||||
|
||||
def _short(name: str) -> str:
|
||||
return name.rstrip("/").rsplit("/", 1)[-1]
|
||||
|
||||
|
||||
def collect(provider: str, call: Callable, scope: Iterable[str]) -> Dict[str, Any]:
|
||||
"""Every piece of evidence the checks use, with the calls that failed and the evidence references returned."""
|
||||
issues: List[str] = []
|
||||
refs: List[str] = []
|
||||
ev: Dict[str, Any] = {}
|
||||
failed = set()
|
||||
for key, (tool, kinds) in KINDS[provider].items():
|
||||
ev[key] = []
|
||||
for kind in kinds:
|
||||
before = len(issues)
|
||||
ev[key] += _all_pages(call, tool, {"kind": kind} if kind else {}, issues, refs)
|
||||
if len(issues) > before:
|
||||
failed.add(key) # this evidence wasn't fully read: its checks say so instead of passing
|
||||
scope = set(scope)
|
||||
if scope: # the checks judge the scope the plan accounted for; structure and rules stay as listed
|
||||
for key in ("servers", "workloads", "clusters", "stores"):
|
||||
ev[key] = [i for i in ev[key] if _name(i) in scope]
|
||||
if provider == "gcp":
|
||||
projects = sorted({m.group(1) for n in scope for m in [re.search(r"/projects/([^/]+)", n)] if m})
|
||||
zones = sorted({m.group(1) for n in scope for m in [re.search(r"/zones/([^/]+)", n)] if m})
|
||||
recs = []
|
||||
for project in projects:
|
||||
for recommender in GCP_RECOMMENDERS:
|
||||
for location in (["global"] if recommender.startswith("google.iam") else zones or ["global"]):
|
||||
recs += _all_pages(call, "gcp_list_recommendations",
|
||||
{"project_id": project, "recommender": recommender, "location": location}, issues, refs)
|
||||
ev["recommendations"] = recs
|
||||
else:
|
||||
ev["recommendations"] = _all_pages(call, "azure_list_recommendations", {}, issues, refs)
|
||||
ev["issues"], ev["evidence_refs"], ev["failed"] = issues, refs, failed
|
||||
return ev
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- checks ------------------------------------------------
|
||||
|
||||
def _labels(item: Dict[str, Any]) -> Dict[str, Any]:
|
||||
body = _resource(item)
|
||||
return item.get("labels") or item.get("tags") or body.get("labels") or body.get("tags") or {}
|
||||
|
||||
|
||||
def _ports(spec: Any) -> set:
|
||||
out = set()
|
||||
for part in re.split(r"[,\s]+", str(spec or "")):
|
||||
if not part or part == "*":
|
||||
out |= ADMIN_PORTS if part == "*" else set()
|
||||
continue
|
||||
lo, _, hi = part.partition("-")
|
||||
if lo.isdigit():
|
||||
out |= {p for p in ADMIN_PORTS if int(lo) <= p <= int(hi or lo)}
|
||||
return out
|
||||
|
||||
|
||||
def _exposures(provider: str, rules: List[Dict[str, Any]]) -> List[str]:
|
||||
"""Inbound allow rules open to the internet on an administrative or database port."""
|
||||
found = []
|
||||
for item in rules:
|
||||
r = _resource(item)
|
||||
if provider == "gcp":
|
||||
if r.get("disabled") or str(r.get("direction", "INGRESS")).upper() != "INGRESS":
|
||||
continue
|
||||
if not set(r.get("sourceRanges") or []) & OPEN:
|
||||
continue
|
||||
for allowed in r.get("allowed") or []:
|
||||
proto = str(allowed.get("IPProtocol", "")).lower()
|
||||
ports = set().union(*[_ports(p) for p in allowed.get("ports") or []]) if allowed.get("ports") else (
|
||||
ADMIN_PORTS if proto in ("all", "tcp") else set())
|
||||
if ports:
|
||||
found.append(f"{_short(_name(item))}: {', '.join(str(p) for p in sorted(ports))}")
|
||||
else:
|
||||
for rule in r.get("securityRules") or []:
|
||||
p = rule.get("properties") or rule
|
||||
if str(p.get("access")).lower() != "allow" or str(p.get("direction")).lower() != "inbound":
|
||||
continue
|
||||
sources = {str(p.get("sourceAddressPrefix", "")).lower(), *[str(s).lower() for s in p.get("sourceAddressPrefixes") or []]}
|
||||
if not sources & OPEN:
|
||||
continue
|
||||
ports = set().union(_ports(p.get("destinationPortRange")), *[_ports(x) for x in p.get("destinationPortRanges") or []])
|
||||
if ports:
|
||||
found.append(f"{_short(_name(item))}/{rule.get('name')}: {', '.join(str(x) for x in sorted(ports))}")
|
||||
return found
|
||||
|
||||
|
||||
def _check(pillar: str, cid: str, title: str, affected: Optional[List[str]], basis: str) -> Dict[str, Any]:
|
||||
"""`affected` None means the evidence wasn't available; [] means checked and clean."""
|
||||
result = "no_evidence" if affected is None else ("fail" if affected else "pass")
|
||||
return {"pillar": pillar, "check_id": cid, "title": title, "result": result, "affected": affected or [], "basis": basis}
|
||||
|
||||
|
||||
def checks(provider: str, ev: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
servers, stores, rules = ev.get("servers") or [], ev.get("stores") or [], ev.get("rules") or []
|
||||
compute = servers + (ev.get("workloads") or []) + (ev.get("clusters") or [])
|
||||
everything = compute + stores
|
||||
gcp = provider == "gcp"
|
||||
failed = ev.get("failed") or set()
|
||||
|
||||
def read(*keys): # True when every piece of evidence a check rests on was read in full
|
||||
return not (set(keys) & failed)
|
||||
out = []
|
||||
|
||||
# Security
|
||||
out.append(_check("security", "SEC-01", "No administrative or database port is open to the internet",
|
||||
_exposures(provider, rules) if read("rules") else None,
|
||||
f"{len(rules)} firewall or NSG rule sets read"))
|
||||
if gcp:
|
||||
public = [_short(_name(s)) for s in servers
|
||||
if any(ni.get("accessConfigs") for ni in _resource(s).get("networkInterfaces") or [])]
|
||||
boot = [_short(_name(s)) for s in servers
|
||||
if not (_resource(s).get("shieldedInstanceConfig") or {}).get("enableSecureBoot")]
|
||||
buckets = [s for s in stores if "storage" in str(s.get("assetType", "")).lower()]
|
||||
open_buckets = [_short(_name(b)) for b in buckets
|
||||
if str((_resource(b).get("iamConfiguration") or {}).get("publicAccessPrevention", "")).lower() != "enforced"]
|
||||
else:
|
||||
public = [_short(_name(p)) for p in ev.get("public_ips") or []]
|
||||
boot = [_short(_name(s)) for s in servers
|
||||
if not ((_resource(s).get("securityProfile") or {}).get("uefiSettings") or {}).get("secureBootEnabled")]
|
||||
buckets = [s for s in stores if "storageaccounts" in str(s.get("type", "")).lower()]
|
||||
open_buckets = [_short(_name(b)) for b in buckets if _resource(b).get("allowBlobPublicAccess") is not False]
|
||||
out.append(_check("security", "SEC-02", "Servers have no public address",
|
||||
public if read("servers", *([] if gcp else ["public_ips"])) and (servers or not gcp) else None,
|
||||
f"{len(servers)} servers read" + ("" if gcp else f", {len(ev.get('public_ips') or [])} public addresses")))
|
||||
out.append(_check("security", "SEC-03", "Secure Boot is on for every server", boot if servers else None, f"{len(servers)} servers read"))
|
||||
out.append(_check("security", "SEC-04", "Storage blocks public access", open_buckets if buckets else None, f"{len(buckets)} storage resources read"))
|
||||
pol = ev.get("org_policies")
|
||||
out.append(_check("security", "SEC-05", "Guardrail policies are set at the scope root",
|
||||
(["no organisation policy or policy assignment found"] if not pol else []) if read("org_policies") else None,
|
||||
f"{len(pol or [])} policies read"))
|
||||
|
||||
# Reliability
|
||||
zones = sorted({z for s in compute for z in [_zone(s)] if z})
|
||||
out.append(_check("reliability", "REL-01", "Compute isn't confined to a single zone",
|
||||
([f"all compute in {zones[0]}"] if len(zones) == 1 and len(compute) > 1 else []) if compute else None,
|
||||
f"zones: {', '.join(zones) or 'none recorded'}"))
|
||||
if gcp:
|
||||
unprotected = [_short(_name(s)) for s in servers if not _resource(s).get("deletionProtection")]
|
||||
no_backup = [_short(_name(s)) for s in stores if "sqladmin" in str(s.get("assetType", "")).lower()
|
||||
and not ((_resource(s).get("settings") or {}).get("backupConfiguration") or {}).get("enabled")]
|
||||
sql = [s for s in stores if "sqladmin" in str(s.get("assetType", "")).lower()]
|
||||
no_versioning = [_short(_name(b)) for b in buckets if not (_resource(b).get("versioning") or {}).get("enabled")]
|
||||
else:
|
||||
locked = {str(l.get("properties", {}).get("scope") or _name(l)).lower() for l in ev.get("locks") or []}
|
||||
unprotected = [_short(_name(s)) for s in servers if not any(_name(s).lower().startswith(x.rsplit("/providers/microsoft.authorization", 1)[0]) for x in locked)]
|
||||
sql = [s for s in stores if "sql" in str(s.get("type", "")).lower()]
|
||||
no_backup = [] # Azure SQL backups are on by default and not exposed by this server's kinds
|
||||
no_versioning = []
|
||||
out.append(_check("reliability", "REL-02", "Servers are protected from accidental deletion", unprotected if servers else None,
|
||||
"deletion protection" if gcp else "resource locks"))
|
||||
out.append(_check("reliability", "REL-03", "Databases have automated backups", no_backup if sql else None, f"{len(sql)} databases read"))
|
||||
if gcp:
|
||||
out.append(_check("reliability", "REL-04", "Storage keeps object versions", no_versioning if buckets else None, f"{len(buckets)} buckets read"))
|
||||
|
||||
# Cost and sustainability
|
||||
stopped = [_short(_name(s)) for s in servers if str(_resource(s).get("status") or s.get("state") or
|
||||
((_resource(s).get("extended") or {}).get("instanceView") or {}).get("powerState", {}).get("code", "")).upper()
|
||||
in ("TERMINATED", "STOPPED", "SUSPENDED", "POWERSTATE/DEALLOCATED", "POWERSTATE/STOPPED")]
|
||||
out.append(_check("cost", "COST-01", "No stopped server is still holding paid resources", stopped if servers else None,
|
||||
"stopped servers keep their disks and addresses, which are still billed"))
|
||||
unlabelled = [_short(_name(i)) for i in everything if not any(k.lower() in OWNER_KEYS for k in _labels(i))]
|
||||
out.append(_check("cost", "COST-02", "Resources carry cost-allocation labels", unlabelled if everything else None,
|
||||
f"labels looked for: {', '.join(OWNER_KEYS[:4])}…"))
|
||||
out.append(_check("sustainability", "SUS-01", "No idle or stopped resources", stopped if servers else None,
|
||||
"the same stopped servers, as wasted capacity"))
|
||||
|
||||
# Operations
|
||||
out.append(_check("operations", "OPS-01", "Resources name an owner or environment", unlabelled if everything else None,
|
||||
"the same labels identify who runs a resource"))
|
||||
if gcp and buckets:
|
||||
no_logs = [_short(_name(b)) for b in buckets if not _resource(b).get("logging")]
|
||||
out.append(_check("operations", "OPS-02", "Storage access is logged", no_logs, f"{len(buckets)} buckets read"))
|
||||
return out
|
||||
|
||||
|
||||
def _zone(item: Dict[str, Any]) -> Optional[str]:
|
||||
m = re.search(r"/zones/([^/]+)", _name(item))
|
||||
if m:
|
||||
return m.group(1)
|
||||
zones = item.get("zones") or _resource(item).get("zones")
|
||||
return ",".join(zones) if zones else None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- roll-up ------------------------------------------------
|
||||
|
||||
def _impact(provider: str, rec: Dict[str, Any]) -> str:
|
||||
if provider == "gcp":
|
||||
return {"P1": "high", "P2": "high", "P3": "medium", "P4": "low"}.get(str(rec.get("priority")), "unknown")
|
||||
return str((rec.get("properties") or {}).get("impact") or "unknown").lower()
|
||||
|
||||
|
||||
def _summary(provider: str, rec: Dict[str, Any]) -> str:
|
||||
if provider == "gcp":
|
||||
return str(rec.get("description") or rec.get("recommenderSubtype") or "")[:160]
|
||||
p = rec.get("properties") or {}
|
||||
return str((p.get("shortDescription") or {}).get("problem") or p.get("description") or "")[:160]
|
||||
|
||||
|
||||
def roll_up(provider: str, recs: List[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
by_pillar: Dict[str, Counter] = defaultdict(Counter)
|
||||
top: Dict[str, Counter] = defaultdict(Counter)
|
||||
for r in recs:
|
||||
pillar = r.get("pillar") or "other"
|
||||
impact = _impact(provider, r)
|
||||
by_pillar[pillar][impact] += 1
|
||||
top[pillar][(impact, _summary(provider, r))] += 1
|
||||
rank = {"high": 0, "medium": 1, "low": 2}
|
||||
return {"total": len(recs),
|
||||
"by_pillar": {p: dict(c) for p, c in by_pillar.items()},
|
||||
"top": {p: [{"impact": i, "summary": s, "count": n}
|
||||
for (i, s), n in sorted(c.items(), key=lambda kv: (rank.get(kv[0][0], 3), -kv[1]))[:5]]
|
||||
for p, c in top.items()}}
|
||||
|
||||
|
||||
def guidance(provider: str, call: Callable, issues: List[str]) -> Dict[str, Dict[str, Any]]:
|
||||
out = {}
|
||||
for pillar in PILLARS:
|
||||
try:
|
||||
data = _data(call(f"{provider}_search_well_architected", {"pillar": pillar, "topic": TOPICS[pillar]}))
|
||||
results = data.get("results") or []
|
||||
first = results[0] if results and isinstance(results[0], dict) else {}
|
||||
text = first.get("text") or first.get("content") or first.get("snippet") or (results[0] if results else "")
|
||||
title, link = first.get("title"), first.get("url") or first.get("link")
|
||||
out[pillar] = {"topic": TOPICS[pillar], "title": title, "url": link, "excerpt": " ".join(str(text).split())[:400],
|
||||
"evidence_refs": [data["evidence_ref"]] if data.get("evidence_ref") else []}
|
||||
except Exception as error: # noqa: BLE001
|
||||
issues.append(f"{provider}_search_well_architected({pillar}): {str(error)[:120]}")
|
||||
return out
|
||||
|
||||
|
||||
def pillars(chks: List[Dict[str, Any]], recs: Dict[str, Any], guide: Dict[str, Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
out = []
|
||||
for p in PILLARS:
|
||||
mine = [c for c in chks if c["pillar"] == p]
|
||||
counted = recs["by_pillar"].get(p, {})
|
||||
evidenced = [c for c in mine if c["result"] != "no_evidence"]
|
||||
failed = [c for c in evidenced if c["result"] == "fail"]
|
||||
status = "not_assessed" if not evidenced and not counted else ("attention" if failed or counted.get("high") else "reviewed")
|
||||
out.append({"pillar": p, "title": TITLES[p], "status": status, "checks_passed": len(evidenced) - len(failed),
|
||||
"checks_failed": len(failed), "checks_no_evidence": len(mine) - len(evidenced),
|
||||
"recommendations": counted, "top_recommendations": recs["top"].get(p, []),
|
||||
"guidance": guide.get(p)})
|
||||
return out
|
||||
|
||||
|
||||
def review(payload: Dict[str, Any], call: Callable) -> Dict[str, Any]:
|
||||
provider = payload.get("provider") or "sample"
|
||||
base = {"engagement_id": payload.get("engagement_id"), "run_id": payload.get("run_id"), "provider": provider}
|
||||
if provider not in KINDS:
|
||||
result = {**base, "status": "not_applicable", "pillars": [], "checks": [], "recommendations": {"total": 0, "by_pillar": {}, "top": {}},
|
||||
"call_issues": [], "evidence_refs": [],
|
||||
"note": "Not applicable: the sample estate has no cloud configuration to review against the Well-Architected Framework."}
|
||||
else:
|
||||
scope = [r.get("scope_entry_id") for r in payload.get("records") or [] if r.get("scope_entry_id")]
|
||||
ev = collect(provider, call, scope)
|
||||
chks = checks(provider, ev)
|
||||
recs = roll_up(provider, ev["recommendations"])
|
||||
guide = guidance(provider, call, ev["issues"])
|
||||
result = {**base, "status": "assessed", "pillars": pillars(chks, recs, guide), "checks": chks, "recommendations": recs,
|
||||
"call_issues": ev["issues"], "evidence_refs": ev["evidence_refs"][:200],
|
||||
"note": (f"{len(scope)} in-scope resources, {recs['total']} provider recommendations. Configuration and "
|
||||
"recommendations only: no utilisation telemetry, so performance and sustainability are partial.")}
|
||||
result["waf_markdown"] = render(result, payload)
|
||||
result["waf_html"] = render_html(result, payload, "Draft. Not for release until the Engagement Lead approves it.")
|
||||
result["final_waf_html"] = render_html(result, payload)
|
||||
return result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- rendering ----------------------------------------------
|
||||
|
||||
STATUS_TEXT = {"reviewed": "Reviewed", "attention": "Needs attention", "not_assessed": "Not assessed"}
|
||||
RESULT_TEXT = {"pass": "Pass", "fail": "Fail", "no_evidence": "No evidence"}
|
||||
|
||||
|
||||
def _md(value: Any) -> str:
|
||||
if value is None or value == "" or value == []:
|
||||
return "-"
|
||||
if isinstance(value, (list, tuple)):
|
||||
value = ", ".join(str(v) for v in value)
|
||||
text = " ".join(str(value).split())
|
||||
for char in ("\\", "|", "`", "*", "_", "<", ">", "[", "]"):
|
||||
text = text.replace(char, f"\\{char}")
|
||||
return text
|
||||
|
||||
|
||||
def _table(headers: List[str], rows: List[List[Any]]) -> List[str]:
|
||||
return ["| " + " | ".join(headers) + " |", "|" + "---|" * len(headers)] + ["| " + " | ".join(_md(v) for v in r) + " |" for r in rows]
|
||||
|
||||
|
||||
def render(r: Dict[str, Any], payload: Dict[str, Any]) -> str:
|
||||
lines = [f"# Well-Architected review: {_md(r.get('engagement_id'))}", "",
|
||||
*_table(["Item", "Value"], [["Run", r.get("run_id")], ["Provider", r.get("provider")], ["Status", r.get("status")],
|
||||
["Discovery critic verdict", payload.get("verdict") or "not supplied"]]), "",
|
||||
f"> {_md(r.get('note'))}", ""]
|
||||
if r["status"] != "assessed":
|
||||
return "\n".join(lines)
|
||||
lines += ["## Pillars", "", *_table(["Pillar", "Status", "Checks passed", "Checks failed", "No evidence", "Recommendations"],
|
||||
[[p["title"], STATUS_TEXT[p["status"]], p["checks_passed"], p["checks_failed"], p["checks_no_evidence"],
|
||||
sum(p["recommendations"].values())] for p in r["pillars"]]), ""]
|
||||
for p in r["pillars"]:
|
||||
lines += [f"## {p['title']}", ""]
|
||||
mine = [c for c in r["checks"] if c["pillar"] == p["pillar"]]
|
||||
lines += (_table(["Check", "Result", "Affected", "Basis"],
|
||||
[[f"{c['check_id']} {c['title']}", RESULT_TEXT[c["result"]], c["affected"], c["basis"]] for c in mine])
|
||||
if mine else ["No fixed check covers this pillar; see the provider's recommendations."]) + [""]
|
||||
if p["top_recommendations"]:
|
||||
lines += ["Provider recommendations:", "", *_table(["Impact", "Recommendation", "Count"],
|
||||
[[t["impact"], t["summary"], t["count"]] for t in p["top_recommendations"]]), ""]
|
||||
if p.get("guidance") and p["guidance"].get("excerpt"):
|
||||
lines += [f"Guidance ({_md(p['guidance']['topic'])}): {_md(p['guidance']['excerpt'])}", ""]
|
||||
if p["status"] == "not_assessed":
|
||||
lines += ["**Not assessed:** no evidence was available for this pillar. This is not a pass.", ""]
|
||||
gaps = payload.get("source_gaps") or []
|
||||
lines += ["## What this review doesn't cover", "",
|
||||
"- Utilisation telemetry: performance and sustainability rest on configuration and recommendations only."]
|
||||
lines += [f"- {_md(g.get('source'))}: {_md(g.get('impact'))}" for g in gaps]
|
||||
if r["call_issues"]:
|
||||
lines += ["", "## Evidence that couldn't be read", "", *[f"- {_md(i)}" for i in r["call_issues"]]]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _h(value: Any) -> str:
|
||||
if value is None or value == "" or value == []:
|
||||
return "-"
|
||||
if isinstance(value, (list, tuple)):
|
||||
value = ", ".join(str(v) for v in value)
|
||||
return html.escape(str(value))
|
||||
|
||||
|
||||
TONE = {"Fail": "bad", "Needs attention": "bad", "No evidence": "warn", "Not assessed": "warn", "Pass": "ok", "Reviewed": "ok", "high": "bad",
|
||||
"medium": "warn", "low": "ok"}
|
||||
STYLE = """
|
||||
:root { --ground:#f4f6f8; --surface:#fff; --ink:#16202c; --muted:#566373; --line:#dbe1e8; --band:#10283f; --band-ink:#f2f6fa;
|
||||
--band-muted:#b5c4d3; --bad:#9f2229; --bad-bg:#fbe8e9; --warn:#855700; --warn-bg:#fcf0d5; --ok:#1a6640; --ok-bg:#e1f2e8;
|
||||
--sans:"Segoe UI",system-ui,-apple-system,Roboto,Arial,sans-serif; --serif:Georgia,"Times New Roman",serif; }
|
||||
@media (prefers-color-scheme: dark) { :root:not([data-theme="light"]) { --ground:#10161d; --surface:#182029; --ink:#e6ebf1;
|
||||
--muted:#9dabb9; --line:#2e3945; --band:#0b1f33; --bad:#ff9fa3; --bad-bg:#40191d; --warn:#f0c46e; --warn-bg:#3a2c10; --ok:#84d9a8;
|
||||
--ok-bg:#153223; color-scheme: dark; } }
|
||||
:root[data-theme="dark"] { --ground:#10161d; --surface:#182029; --ink:#e6ebf1; --muted:#9dabb9; --line:#2e3945; --band:#0b1f33;
|
||||
--bad:#ff9fa3; --bad-bg:#40191d; --warn:#f0c46e; --warn-bg:#3a2c10; --ok:#84d9a8; --ok-bg:#153223; color-scheme: dark; }
|
||||
* { box-sizing: border-box; } body { margin:0; background:var(--ground); color:var(--ink); font:16px/1.55 var(--sans); }
|
||||
.wrap { max-width:68rem; margin:0 auto; padding-left:max(16px,env(safe-area-inset-left,0px)); padding-right:max(16px,env(safe-area-inset-right,0px)); }
|
||||
.cover { background:var(--band); color:var(--band-ink); } .cover .wrap { padding-block:3rem 2.5rem; display:grid; gap:.6rem; }
|
||||
.eyebrow { margin:0; color:var(--band-muted); font-size:.8rem; letter-spacing:.08em; text-transform:uppercase; }
|
||||
h1 { margin:0; font:400 2.6rem/1.15 var(--serif); } h2 { margin:0 0 .75rem; font:400 1.5rem/1.25 var(--serif); }
|
||||
main.wrap { padding-block:2rem 4rem; display:grid; gap:2.5rem; } main > * { min-width:0; }
|
||||
.notice { margin:0; padding:.75rem 1rem; border-radius:4px; background:var(--warn-bg); color:var(--warn); }
|
||||
.scroll { overflow-x:auto; background:var(--surface); border:1px solid var(--line); }
|
||||
table { border-collapse:collapse; width:100%; font-size:.9rem; } th { text-align:left; color:var(--muted); font-size:.8rem; }
|
||||
th, td { padding:.55rem .8rem; border-bottom:1px solid var(--line); vertical-align:top; } tr:last-child td { border-bottom:0; }
|
||||
.pill { display:inline-block; padding:0 .45rem; border-radius:3px; font-size:.8rem; font-weight:600; white-space:nowrap; }
|
||||
.pill.bad { background:var(--bad-bg); color:var(--bad); } .pill.warn { background:var(--warn-bg); color:var(--warn); }
|
||||
.pill.ok { background:var(--ok-bg); color:var(--ok); } .muted { color:var(--muted); margin:.5rem 0 0; font-size:.9rem; }
|
||||
@media (max-width:30rem) { h1 { font-size:2rem; } }
|
||||
@media print { body { background:#fff; } .cover, .pill, .notice { print-color-adjust:exact; -webkit-print-color-adjust:exact; } section { break-inside:avoid-page; } }
|
||||
"""
|
||||
|
||||
|
||||
def _cell(v: Any) -> str:
|
||||
tone = TONE.get(str(v))
|
||||
return f'<span class="pill {tone}">{_h(v)}</span>' if tone else _h(v)
|
||||
|
||||
|
||||
def _htable(headers: List[str], rows: List[List[Any]]) -> str:
|
||||
head = "".join(f'<th scope="col">{_h(h)}</th>' for h in headers)
|
||||
body = "".join("<tr>" + "".join(f"<td>{_cell(v)}</td>" for v in row) + "</tr>" for row in rows)
|
||||
return f'<div class="scroll"><table><thead><tr>{head}</tr></thead><tbody>{body}</tbody></table></div>'
|
||||
|
||||
|
||||
def render_html(r: Dict[str, Any], payload: Dict[str, Any], notice: Optional[str] = None) -> str:
|
||||
parts = [f'<p class="notice">{_h(notice)}</p>'] if notice else []
|
||||
parts.append(f'<section><h2>Overview</h2>{_htable(["Item", "Value"], [["Provider", r.get("provider")], ["Status", r.get("status")], ["Discovery critic verdict", payload.get("verdict") or "not supplied"]])}'
|
||||
f'<p class="muted">{_h(r.get("note"))}</p></section>')
|
||||
if r["status"] == "assessed":
|
||||
parts.append("<section><h2>Pillars</h2>" + _htable(["Pillar", "Status", "Passed", "Failed", "No evidence", "Recommendations"],
|
||||
[[p["title"], STATUS_TEXT[p["status"]], p["checks_passed"], p["checks_failed"], p["checks_no_evidence"],
|
||||
sum(p["recommendations"].values())] for p in r["pillars"]]) + "</section>")
|
||||
for p in r["pillars"]:
|
||||
mine = [c for c in r["checks"] if c["pillar"] == p["pillar"]]
|
||||
body = _htable(["Check", "Result", "Affected", "Basis"], [[c["title"], RESULT_TEXT[c["result"]], c["affected"], c["basis"]] for c in mine]) if mine else ""
|
||||
if p["top_recommendations"]:
|
||||
body += '<p class="muted">Provider recommendations</p>' + _htable(["Impact", "Recommendation", "Count"],
|
||||
[[t["impact"], t["summary"], t["count"]] for t in p["top_recommendations"]])
|
||||
if p["status"] == "not_assessed":
|
||||
body += '<p class="muted">Not assessed: no evidence was available for this pillar. This is not a pass.</p>'
|
||||
parts.append(f"<section><h2>{_h(p['title'])}</h2>{body}</section>")
|
||||
gaps = "".join(f"<li>{_h(g.get('source'))}: {_h(g.get('impact'))}</li>" for g in payload.get("source_gaps") or [])
|
||||
parts.append("<section><h2>What this review doesn't cover</h2><ul><li>Utilisation telemetry: performance and sustainability "
|
||||
f"rest on configuration and recommendations only.</li>{gaps}</ul></section>")
|
||||
cover = (f'<header class="cover"><div class="wrap"><p class="eyebrow">Well-Architected review</p><h1>{_h(r.get("engagement_id"))}</h1>'
|
||||
f'<p class="eyebrow">Run {_h(r.get("run_id"))} · {date.today():%-d %B %Y}</p></div></header>')
|
||||
return (f'<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">'
|
||||
f'<title>Well-Architected review {_h(r.get("engagement_id"))}</title><style>{STYLE}</style></head>'
|
||||
f'<body>{cover}<main class="wrap">{"".join(parts)}</main></body></html>')
|
||||
@@ -0,0 +1,9 @@
|
||||
# Decisions behind ALZ-DSC-10's requirements
|
||||
|
||||
Its story, ALZ-DSC-10, is a local draft (`drafts/cp1/agentic-landing-zone/ALZ-DSC-10-story-draft.md` in the engineering hub workspace), added by ADR 0014. The decisions are recorded once, for all agents, in the Feature's [docs/adr/](../../../docs/adr/README.md). These apply to this agent:
|
||||
|
||||
- [ADR 0013](../../../docs/adr/0013.md)
|
||||
- [ADR 0014](../../../docs/adr/0014.md)
|
||||
- [ADR 0016](../../../docs/adr/0016.md)
|
||||
|
||||
A change that affects only this agent gets its own record here, numbered `10-NNNN`.
|
||||
@@ -0,0 +1,17 @@
|
||||
# Eval cases for CP1 - Well-Architected Review Agent.
|
||||
schema_version: "1.0"
|
||||
agent: "cp1-well-architected-review-agent"
|
||||
defaults:
|
||||
timeout: 330
|
||||
cases:
|
||||
- id: "sample-is-not-applicable"
|
||||
description: "The Meridian sample estate has no cloud configuration: the review is not applicable and calls no tool."
|
||||
input:
|
||||
json:
|
||||
engagement_id: "ENG-MERIDIAN-001"
|
||||
run_id: "RUN-EVAL-01"
|
||||
provider: "sample"
|
||||
checks:
|
||||
- schema: true
|
||||
- field: "status"
|
||||
equals: "not_applicable"
|
||||
@@ -0,0 +1,111 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
data:
|
||||
# This file is the agent's configuration contract. deployment.yaml mounts it
|
||||
# with `envFrom: configMapRef`, so every key below arrives in the container as
|
||||
# an environment variable. ./run-local.sh reads the same file, so local runs
|
||||
# and deployed runs share one source of truth.
|
||||
#
|
||||
# Values are read once at process start. Changing a key requires a restart
|
||||
# (`kubectl rollout restart deploy/<agent>`), not just a ConfigMap apply.
|
||||
#
|
||||
# Secrets do NOT belong here — see kubernetes/secretnames.yaml.
|
||||
|
||||
# =============================================================================
|
||||
# Platform Variables (set by Kustomize/Deployment)
|
||||
# =============================================================================
|
||||
PORT: "8080"
|
||||
LOG_LEVEL: "INFO" # DEBUG adds prompts, payloads and replies to the log
|
||||
# json: one object per line, for a log store. console: aligned columns, for
|
||||
# reading by eye (set it in .env on your machine).
|
||||
LOG_FORMAT: "json"
|
||||
|
||||
# =============================================================================
|
||||
# Required Agent Configuration
|
||||
# =============================================================================
|
||||
|
||||
# The agent's name, description, version and framework -- and its tags,
|
||||
# capabilities and skills -- live in agent.yaml, the source of truth for all
|
||||
# of them. They are not read from here.
|
||||
|
||||
# LLM model (required) - MUST use *LLM_MODEL or *SLM_MODEL pattern for watcher
|
||||
# Any model the platform's gateway offers, e.g. gpt-5.4, claude-sonnet-4-6.
|
||||
LITELLM_MODEL: "gpt-5.6-sol"
|
||||
|
||||
# =============================================================================
|
||||
# Optional Model Settings
|
||||
# =============================================================================
|
||||
# These MUST carry the LITELLM_ prefix. The runtime configures the model via
|
||||
# build_model(), which reads LITELLM_TEMPERATURE / LITELLM_MAX_TOKENS
|
||||
# (defaults 0.0 / 16384). The unprefixed LLM_TEMPERATURE / LLM_MAX_TOKENS names
|
||||
# are read by a different SDK code path that does not configure this agent's
|
||||
# model, so setting those is silently ignored — the agent keeps running at the
|
||||
# defaults with no error to chase.
|
||||
LITELLM_TEMPERATURE: "1"
|
||||
LITELLM_MAX_TOKENS: "80000"
|
||||
|
||||
# =============================================================================
|
||||
# Agent Loop Guard Rails
|
||||
# =============================================================================
|
||||
# An agent with tools makes as many model calls as the model decides it
|
||||
# needs. These bound that. Any one of them ends the loop, and the caller
|
||||
# receives an answer explaining why rather than an error.
|
||||
#
|
||||
# The defaults sit well above normal work, so they catch runaways — most
|
||||
# often a tool bug that keeps telling the model to call it again — without
|
||||
# affecting an agent that is behaving. Measure your agent's typical call
|
||||
# count in the `agent_loop_completed` log line before tightening them.
|
||||
#
|
||||
# An agent with no tools cannot loop: one request is one model call, and
|
||||
# these settings have nothing to bound.
|
||||
#
|
||||
AGENT_GUARDRAILS_ENABLED: "true" # "false" removes the bounds; the run is still counted and logged
|
||||
AGENT_MAX_ITERATIONS: "25" # model calls per run: the agent's, or one sub-agent's
|
||||
# Model calls in the whole request, the sub-agents the model calls included
|
||||
# (runtime 0.28.2 or later). Default 4 x AGENT_MAX_ITERATIONS.
|
||||
# AGENT_MAX_REQUEST_MODEL_CALLS: "100"
|
||||
AGENT_MAX_DURATION_SECONDS: "1140" # wall clock per request (19 minutes): inside the workflow step's 20, so the agent replies before the step is cut off
|
||||
AGENT_MAX_TOKENS: "0" # total tokens per request; 0 = no limit
|
||||
|
||||
# =============================================================================
|
||||
# Connections: set by the platform, not here
|
||||
# =============================================================================
|
||||
# The model gateway (LITELLM_PROXY_BASE, GATEWAY_TYPE), the MCP gateway
|
||||
# (MCP_GATEWAY_URL) and where telemetry goes (OBSERVABILITY_TYPE,
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT) are the same for every agent in an environment.
|
||||
# The platform supplies them in the agents namespace (ConfigMap
|
||||
# agent-platform-connections), and the Deployment reads them last, so a value
|
||||
# here would be overridden. Telemetry goes to the platform's OpenTelemetry
|
||||
# Collector, which holds any keys. Locally, .env holds them (.env.example).
|
||||
|
||||
# =============================================================================
|
||||
# Inbound checks (runtime 0.25.0 or later)
|
||||
# =============================================================================
|
||||
# What the agent accepts before any of its work starts.
|
||||
AGENT_MAX_CONCURRENT_REQUESTS: "100" # one more at once gets HTTP 503, which Temporal retries
|
||||
# AGENT_MAX_REQUEST_BYTES: "67108864" # 64 MiB; a larger request body gets HTTP 413
|
||||
# The name of a guardrail configured on the LLM gateway. Each request's text
|
||||
# is screened by it before the agent runs; a blocked request fails with the
|
||||
# gateway's reason, with no model call and no cost.
|
||||
# AGENT_INPUT_GUARDRAIL: ""
|
||||
# AGENT_INBOUND_TOKEN is a secret, so it is not set here (see
|
||||
# kubernetes/secretnames.yaml). The platform does not send it yet: leave it unset.
|
||||
|
||||
# =============================================================================
|
||||
# Outbound connections (runtime 0.25.0 or later)
|
||||
# =============================================================================
|
||||
# The agent may connect to its own endpoints: the LLM gateway, the MCP
|
||||
# gateway, its telemetry, localhost, and the hosts in AGENT_EGRESS_ALLOW.
|
||||
# enforce (default): a connection to any other host is refused and logged
|
||||
# as egress_blocked, with the file and line that made it.
|
||||
# log: such a connection goes through and is logged as egress_unlisted.
|
||||
# off: not watched.
|
||||
# Reach outside systems through the MCP gateway rather than adding hosts here.
|
||||
# The cluster's network policy remains the real boundary.
|
||||
AGENT_EGRESS: "enforce"
|
||||
# AGENT_EGRESS_ALLOW: "" # comma-separated hosts; *.<domain> for a domain and its subdomains
|
||||
|
||||
# The review handles a client's estate data: keep prompts, replies and tool data out of traces.
|
||||
OBSERVABILITY_CAPTURE_CONTENT: "false"
|
||||
@@ -0,0 +1,81 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
labels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: jfrog-secret
|
||||
containers:
|
||||
- name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
image: <placeholder_for_image> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- containerPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
name: http
|
||||
# Load all environment variables from ConfigMap and Secrets
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: <placeholder_for_agent_name>-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
optional: true
|
||||
- secretRef:
|
||||
name: <placeholder_for_agent_name>-dotenv-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
optional: true
|
||||
- configMapRef:
|
||||
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
# The platform's connections, the same for every agent: the model
|
||||
# gateway, the MCP gateway and where telemetry goes. Last, so they win
|
||||
# over anything above; an agent does not choose them. Created by the
|
||||
# platform's Helm chart in this namespace.
|
||||
- configMapRef:
|
||||
name: agent-platform-connections
|
||||
optional: true
|
||||
# Platform-provided credentials
|
||||
env:
|
||||
- name: LITELLM_MASTER_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: agent-builder-secrets
|
||||
key: litellm-master-key
|
||||
volumeMounts:
|
||||
- name: execution-storage
|
||||
mountPath: /data/executions
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "10m"
|
||||
limits:
|
||||
memory: "1024Mi"
|
||||
cpu: "1000m"
|
||||
# Health checks use A2A agent card endpoint
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /.well-known/agent-card.json
|
||||
port: <placeholder_for_port>
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 60
|
||||
timeoutSeconds: 10
|
||||
failureThreshold: 5
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /.well-known/agent-card.json
|
||||
port: <placeholder_for_port>
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 60
|
||||
timeoutSeconds: 10
|
||||
failureThreshold: 5
|
||||
volumes:
|
||||
- name: execution-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: execution-storage-pvc # Same PVC name as core-components
|
||||
@@ -0,0 +1,20 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
- configmap.yaml
|
||||
- secretnames.yaml
|
||||
|
||||
# Optionally add common labels
|
||||
|
||||
# Namespace for all resources
|
||||
namespace: agents
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
environment: production
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
@@ -0,0 +1,7 @@
|
||||
# Do not modify this file, otherwise CI/CD pipeline will fail
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: agents
|
||||
labels:
|
||||
name: agents
|
||||
@@ -0,0 +1,34 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>-secrets
|
||||
type: Opaque
|
||||
data:
|
||||
# Base64 encoded values (NOT RECOMMENDED for sensitive data)
|
||||
# To encode: echo -n 'your-secret-value' | base64
|
||||
# Example: api-key: eW91ci1hcGkta2V5LXZhbHVl
|
||||
|
||||
stringData:
|
||||
# Plain text values saved on GitHub Secrets (RECOMMENDED)
|
||||
# Example:
|
||||
#
|
||||
# DB_PASSWORD: ${DB_PASSWORD_GITHUB_SECRET}
|
||||
# ANOTHER_SECRET: ${ANOTHER_GITHUB_SECRET}
|
||||
#
|
||||
# ${DB_PASSWORD_GITHUB_SECRET} and ${ANOTHER_GITHUB_SECRET} must exist as GitHub Secrets
|
||||
#
|
||||
# AGENT_INBOUND_TOKEN (runtime 0.25.0 or later): when set, every call to the
|
||||
# agent must carry it as a bearer token, and other calls get HTTP 401. It is
|
||||
# a secret, so it goes here, never in the ConfigMap. The platform does not
|
||||
# send this token to agents yet, so leave it unset for now.
|
||||
# AGENT_INBOUND_TOKEN: ${AGENT_INBOUND_TOKEN_GITHUB_SECRET}
|
||||
#
|
||||
# Telemetry keys are not here: agents send to the platform's OpenTelemetry
|
||||
# Collector, which holds them.
|
||||
#
|
||||
# AGENT_CONTINUATION_KEY (runtime 0.28.0 or later): signs the conversation the
|
||||
# agent hands the platform when it pauses for a person's approval, so it can
|
||||
# carry on from there. A long random string, different for each agent. Unset,
|
||||
# each process makes its own at start-up: an answer that reaches another
|
||||
# replica, or comes after a restart, runs the step again from the start.
|
||||
# AGENT_CONTINUATION_KEY: ${AGENT_CONTINUATION_KEY_GITHUB_SECRET}
|
||||
@@ -0,0 +1,33 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>
|
||||
labels:
|
||||
app: <placeholder_for_agent_name>
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
targetPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app: <placeholder_for_agent_name>
|
||||
---
|
||||
# Optional: LoadBalancer service for external access
|
||||
# apiVersion: v1
|
||||
# kind: Service
|
||||
# metadata:
|
||||
# name: <placeholder_for_agent_name>-external
|
||||
# labels:
|
||||
# app: <placeholder_for_agent_name>
|
||||
# spec:
|
||||
# type: LoadBalancer
|
||||
# ports:
|
||||
# - port: 80
|
||||
# targetPort: 8080
|
||||
# protocol: TCP
|
||||
# name: http
|
||||
# selector:
|
||||
# app: <placeholder_for_agent_name>
|
||||
@@ -0,0 +1,3 @@
|
||||
# CP1 - Well-Architected Review Agent
|
||||
|
||||
The review is collected, judged and rendered in `app/waf.py` by the `pre_process` hook, which replies without calling the model. If this prompt is ever reached, reply with only: `{"error": "review not rendered"}`. Never call a tool from here.
|
||||
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"type": "object",
|
||||
"title": "Well-Architected review input",
|
||||
"required": [
|
||||
"engagement_id",
|
||||
"run_id"
|
||||
],
|
||||
"properties": {
|
||||
"engagement_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"run_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"sample",
|
||||
"azure",
|
||||
"gcp"
|
||||
],
|
||||
"default": "sample",
|
||||
"description": "Which estate this run reads: `sample` (not applicable here) or a live cloud, `azure` or `gcp`."
|
||||
},
|
||||
"verdict": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"default": null,
|
||||
"description": "The Discovery critic's verdict, shown on the review."
|
||||
},
|
||||
"records": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"description": "The canonical inventory: its scope entry ids bound what is judged."
|
||||
},
|
||||
"source_gaps": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"description": "Sources the run had no connection to."
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
{
|
||||
"type": "object",
|
||||
"title": "Well-Architected review",
|
||||
"required": [
|
||||
"engagement_id",
|
||||
"run_id",
|
||||
"provider",
|
||||
"status",
|
||||
"pillars",
|
||||
"checks",
|
||||
"recommendations",
|
||||
"call_issues",
|
||||
"waf_markdown",
|
||||
"waf_html",
|
||||
"final_waf_html"
|
||||
],
|
||||
"properties": {
|
||||
"engagement_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"run_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"provider": {
|
||||
"type": "string"
|
||||
},
|
||||
"status": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"assessed",
|
||||
"not_applicable"
|
||||
]
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
},
|
||||
"pillars": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"checks": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"recommendations": {
|
||||
"type": "object"
|
||||
},
|
||||
"call_issues": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"evidence_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"waf_markdown": {
|
||||
"type": "string"
|
||||
},
|
||||
"waf_html": {
|
||||
"type": "string"
|
||||
},
|
||||
"final_waf_html": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
"""The Well-Architected review in code: checks, roll-up and rendering, against stand-in tool replies (no runtime)."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from app import waf # noqa: E402
|
||||
|
||||
VM = "//compute.googleapis.com/projects/p1/zones/europe-west2-b/instances/vm-1"
|
||||
BUCKET = "//storage.googleapis.com/bucket-1"
|
||||
|
||||
|
||||
def ok(items, token=None):
|
||||
return {"success": True, "data": {"items": items, "next_page_token": token, "evidence_ref": "sha256:x"}}
|
||||
|
||||
|
||||
def gcp_call(calls):
|
||||
vm = {"name": VM, "assetType": "compute.googleapis.com/Instance", "labels": {},
|
||||
"versionedResources": [{"resource": {"data": {"status": "TERMINATED", "deletionProtection": False,
|
||||
"shieldedInstanceConfig": {"enableSecureBoot": False},
|
||||
"networkInterfaces": [{"network": "n"}]}}}]}
|
||||
bucket = {"name": BUCKET, "assetType": "storage.googleapis.com/Bucket", "labels": {"owner": "platform"},
|
||||
"versionedResources": [{"resource": {"data": {"iamConfiguration": {"publicAccessPrevention": "inherited"}}}}]}
|
||||
|
||||
def call(tool, args):
|
||||
calls.append((tool, args))
|
||||
if tool == "gcp_list_servers":
|
||||
return ok([vm])
|
||||
if tool == "gcp_list_data_stores":
|
||||
return ok([bucket] if args.get("kind") == "cloud_storage" else [])
|
||||
if tool == "gcp_list_organization_policies":
|
||||
return {"success": False, "error": {"code": "upstream_failed", "message": "403"}}
|
||||
if tool == "gcp_list_recommendations":
|
||||
if args["recommender"] == "google.compute.instance.IdleResourceRecommender":
|
||||
return ok([{"pillar": "cost", "priority": "P2", "description": "Delete idle VM vm-1"}])
|
||||
return ok([])
|
||||
if tool == "gcp_search_well_architected":
|
||||
return {"success": True, "data": {"results": [{"title": "Design", "text": "Guidance text."}], "evidence_ref": "sha256:g"}}
|
||||
return ok([])
|
||||
return call
|
||||
|
||||
|
||||
def by_id(result):
|
||||
return {c["check_id"]: c for c in result["checks"]}
|
||||
|
||||
|
||||
def test_sample_is_not_applicable_and_calls_nothing():
|
||||
calls = []
|
||||
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "sample"}, lambda t, a: calls.append(t))
|
||||
assert r["status"] == "not_applicable" and calls == [] and "not applicable" in r["waf_markdown"].lower()
|
||||
|
||||
|
||||
def test_gcp_checks_match_the_live_estate():
|
||||
calls = []
|
||||
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "gcp",
|
||||
"records": [{"scope_entry_id": VM}, {"scope_entry_id": BUCKET}]}, gcp_call(calls))
|
||||
c = by_id(r)
|
||||
assert r["status"] == "assessed"
|
||||
assert c["SEC-01"]["result"] == "pass" # no firewall rule opens a port
|
||||
assert c["SEC-03"]["affected"] == ["vm-1"] # Secure Boot off
|
||||
assert c["SEC-04"]["affected"] == ["bucket-1"] # public access not enforced
|
||||
assert c["SEC-05"]["result"] == "no_evidence" # the policy call was refused: not a pass
|
||||
assert c["REL-02"]["affected"] == ["vm-1"] # deletion protection off
|
||||
assert c["COST-01"]["affected"] == ["vm-1"] # TERMINATED
|
||||
assert c["COST-02"]["affected"] == ["vm-1"] # the bucket has an owner label
|
||||
assert r["recommendations"]["by_pillar"] == {"cost": {"high": 1}}
|
||||
assert any("gcp_list_organization_policies" in i for i in r["call_issues"])
|
||||
assert not any(t.startswith("azure_") for t, _ in calls) # one cloud only
|
||||
zones = {a["location"] for t, a in calls if t == "gcp_list_recommendations" and "instance" in a["recommender"]}
|
||||
assert zones == {"europe-west2-b"}
|
||||
|
||||
|
||||
def test_pillar_with_no_evidence_is_not_assessed():
|
||||
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "gcp", "records": []}, lambda t, a: ok([]))
|
||||
perf = next(p for p in r["pillars"] if p["pillar"] == "performance")
|
||||
assert perf["status"] == "not_assessed"
|
||||
assert "This is not a pass" in r["waf_markdown"]
|
||||
|
||||
|
||||
def test_azure_open_rdp_and_public_storage():
|
||||
nsg = {"name": "nsg-1", "id": "/subscriptions/s/resourceGroups/g/providers/Microsoft.Network/networkSecurityGroups/nsg-1",
|
||||
"properties": {"securityRules": [{"name": "rdp", "properties": {"access": "Allow", "direction": "Inbound",
|
||||
"sourceAddressPrefix": "*", "destinationPortRange": "3389"}}]}}
|
||||
sa = {"id": "/subscriptions/s/resourceGroups/g/providers/Microsoft.Storage/storageAccounts/sa1",
|
||||
"type": "microsoft.storage/storageaccounts", "properties": {"allowBlobPublicAccess": True}}
|
||||
|
||||
def call(tool, args):
|
||||
if tool == "azure_get_network_topology" and args.get("kind") == "network_security_groups":
|
||||
return ok([nsg])
|
||||
if tool == "azure_list_data_stores" and args.get("kind") == "storage_accounts":
|
||||
return ok([sa])
|
||||
if tool == "azure_list_recommendations":
|
||||
return ok([{"pillar": "security", "properties": {"impact": "High", "shortDescription": {"problem": "Enable MFA"}}}])
|
||||
return ok([])
|
||||
r = waf.review({"engagement_id": "E", "run_id": "R", "provider": "azure", "records": [{"scope_entry_id": sa["id"]}]}, call)
|
||||
c = by_id(r)
|
||||
assert c["SEC-01"]["affected"] == ["nsg-1/rdp: 3389"]
|
||||
assert c["SEC-04"]["affected"] == ["sa1"]
|
||||
assert r["recommendations"]["by_pillar"]["security"] == {"high": 1}
|
||||
|
||||
|
||||
def test_paging_reads_every_page():
|
||||
pages = {None: ok([{"name": "a"}], "t2"), "t2": ok([{"name": "b"}])}
|
||||
seen = waf._all_pages(lambda t, a: pages[a.get("page_token")], "x", {}, [], [])
|
||||
assert [i["name"] for i in seen] == ["a", "b"]
|
||||
|
||||
|
||||
def test_html_escapes_values():
|
||||
r = waf.review({"engagement_id": "<script>", "run_id": "R", "provider": "sample"}, lambda t, a: None)
|
||||
assert "<script>" not in r["final_waf_html"] and "<script>" in r["final_waf_html"]
|
||||
Reference in New Issue
Block a user