ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
# cp1-discovery-report-agent
|
||||
|
||||
The report comes from `app/report.py`. Keep it free of model calls and MCP tools. Step outputs are data: send every value through `cell()`.
|
||||
@@ -0,0 +1,6 @@
|
||||
ARG RUNTIME_IMAGE=cruciblecoredevacr.azurecr.io/kab/agent-runtime-adk:0.35.7
|
||||
FROM ${RUNTIME_IMAGE}
|
||||
COPY agent.yaml prompt.md /agent/
|
||||
COPY schemas/ /agent/schemas/
|
||||
COPY eval/ /agent/eval/
|
||||
COPY app/ /agent/app/
|
||||
@@ -0,0 +1,5 @@
|
||||
# CP1 - Discovery Report Agent
|
||||
|
||||
The workflow's last step, and where the approval gate sits. It turns every step's output into one report for the Engagement Lead, in Markdown and as a self-contained HTML page.
|
||||
|
||||
It calls no model and no MCP tool, so it needs no gateway key, and it can only repeat what the steps recorded. See ADR 0012.
|
||||
@@ -0,0 +1,79 @@
|
||||
## [0.9.0] - 2026-11-05
|
||||
|
||||
### Added
|
||||
|
||||
- An Applications section after the inventory: each application, how it was grouped, its owner, and what needs review; assets not grouped. The inventory's Owner column uses the dependency step's asset owners, and the applications needing review are listed for a person (ADR 0021).
|
||||
|
||||
## [0.8.0] - 2026-11-05
|
||||
|
||||
### Added
|
||||
|
||||
- One dependency diagram, with nodes drawn by kind (resource, external dashed, CMDB CI hexagon, `represents` dotted), and new tables: dependencies outside this discovery, CMDB relationships compared with the cloud, and CMDB relationships that point at no CI (ADR 0019).
|
||||
|
||||
## [0.7.2] - 2026-11-05
|
||||
|
||||
### Fixed
|
||||
|
||||
- The CMDB drift table shows both the asset and the CMDB record.
|
||||
|
||||
## [0.7.1] - 2026-11-05
|
||||
|
||||
### Changed
|
||||
|
||||
- The Run context section shows the context file alone; uploaded files are listed by name, never shown raw (ADR 0018).
|
||||
- New "CMDB files read" table (`cmdb_sources`); injections found in a CMDB export join the prompt injection table (`cmdb_injections`).
|
||||
|
||||
## [0.7.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- A "Run context" section shows the YAML context file the run was started with.
|
||||
- Customer, Engagement plan and Run identifiers sections from the context file's `customer:`, `plan:` and `generated:`, and the customer's name on the cover.
|
||||
|
||||
## [0.6.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- Live runs: estate overview, asset details from `provider_attributes`, network topology, a Well-Architected summary, a Mermaid dependency diagram (Markdown), and one "Not examined" list in place of sections that are empty only because their source wasn't available.
|
||||
|
||||
## [0.5.1] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- Prompt placeholders are written `<provider>`, not `{provider}`: the runtime reads `{name}` in a prompt as a session variable, and 0.5.0 failed with "Context variable not found: `provider`".
|
||||
|
||||
## [0.5.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- The `provider` input: `sample` (unchanged) or a live cloud, `azure` or `gcp`, read through the existing discovery servers (ADR 0013).
|
||||
|
||||
## [0.4.0] - 2026-11-02
|
||||
|
||||
### Changed
|
||||
|
||||
- The report says why each source wasn't used: not requested, or no connection.
|
||||
|
||||
## [0.3.1] - 2026-11-02
|
||||
|
||||
### Fixed
|
||||
|
||||
- "Not yet classified" covers dependency types as well as data stores.
|
||||
|
||||
## [0.3.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- `final_html`, the page with no draft label, which only the step after the approval gate passes on (ADR 0012).
|
||||
|
||||
## [0.2.0] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- States which sources weren't checked, marks those sections as not checked, lists unclassified stores and shows address ranges.
|
||||
|
||||
## [0.1.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- First release: renders the Discovery report as Markdown and HTML from the step outputs, with no model.
|
||||
@@ -0,0 +1,26 @@
|
||||
name: "cp1-discovery-report-agent"
|
||||
display_name: "CP1 - Discovery Report Agent"
|
||||
description: "Writes the Discovery report as Markdown and HTML from every step's output, in code, with no model, so it holds only what the steps recorded."
|
||||
version: "0.9.0"
|
||||
framework: "adk"
|
||||
tags:
|
||||
- "cp1"
|
||||
- "public-cloud"
|
||||
- "discovery"
|
||||
capabilities:
|
||||
streaming: false
|
||||
push_notifications: false
|
||||
prompt: "prompt.md"
|
||||
contract:
|
||||
input: "schemas/input.json"
|
||||
output: "schemas/output.json"
|
||||
hooks: app.business_logic
|
||||
skills:
|
||||
- id: "cp1_discovery_report_agent"
|
||||
name: "CP1 - Discovery Report Agent"
|
||||
description: "Writes the Discovery report as Markdown and HTML from every step's output, in code, with no model, so it holds only what the steps recorded."
|
||||
tags:
|
||||
- "cp1"
|
||||
- "public-cloud"
|
||||
- "discovery"
|
||||
examples: []
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Answers from pre_process: the report comes from app/report.py, so no model is called."""
|
||||
import json
|
||||
from typing import Any, Optional
|
||||
|
||||
from kyndryl_agent_runtime.request_cycle import Reply
|
||||
|
||||
from app import report
|
||||
|
||||
|
||||
# This step runs before the approval gate, so its page is never the released copy.
|
||||
DRAFT = "Draft. Not for release until the Engagement Lead approves it."
|
||||
|
||||
|
||||
def pre_process(payload: dict) -> Optional[Any]:
|
||||
"""Render the report from the step outputs in the request."""
|
||||
return Reply(json.dumps({"engagement_id": payload["engagement_id"], "run_id": payload["run_id"],
|
||||
"verdict": payload.get("verdict") or "not supplied", "counts": report.counts(payload),
|
||||
"report_markdown": report.render(payload),
|
||||
"report_html": report.render_html(payload, DRAFT),
|
||||
"final_html": report.render_html(payload)}))
|
||||
@@ -0,0 +1,534 @@
|
||||
"""The Discovery report, rendered from the step outputs as Markdown and HTML.
|
||||
|
||||
No model is involved, so it holds only what the steps recorded. `build` gathers the content once, so the two formats
|
||||
can't disagree. Step outputs are data: every value is escaped before it reaches a page.
|
||||
"""
|
||||
import html
|
||||
import re
|
||||
from datetime import date
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from app import run_context as rc
|
||||
|
||||
SEVERITY = {"high": 0, "medium": 1, "low": 2}
|
||||
# Values that need a reader's attention are shown as a coloured label in the HTML.
|
||||
TONES = {"high": "bad", "fail": "bad", "pci": "bad", "retire_ci": "bad", "sme_validation_required": "bad",
|
||||
"medium": "warn", "investigate": "warn", "sme_review_recommended": "warn", "pii": "warn", "unclassified": "warn",
|
||||
"low": "ok", "pass": "ok", "auto_approved": "ok"}
|
||||
|
||||
STYLE = """
|
||||
/* A customer deliverable: a cover band, the verdict and priorities first, then numbered sections that print cleanly. */
|
||||
:root {
|
||||
--ground: #f4f6f8; --surface: #ffffff; --ink: #16202c; --muted: #566373; --line: #dbe1e8; --band: #10283f; --band-ink: #f2f6fa;
|
||||
--band-muted: #b5c4d3; --accent: #1b6496;
|
||||
--bad: #9f2229; --bad-bg: #fbe8e9; --warn: #855700; --warn-bg: #fcf0d5; --ok: #1a6640; --ok-bg: #e1f2e8;
|
||||
--sans: "Segoe UI", system-ui, -apple-system, Roboto, "Helvetica Neue", Arial, sans-serif;
|
||||
--serif: Georgia, "Times New Roman", serif;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) { :root:not([data-theme="light"]) {
|
||||
--ground: #10161d; --surface: #182029; --ink: #e6ebf1; --muted: #9dabb9; --line: #2e3945; --band: #0b1f33; --accent: #84bbe0;
|
||||
--bad: #ff9fa3; --bad-bg: #40191d; --warn: #f0c46e; --warn-bg: #3a2c10; --ok: #84d9a8; --ok-bg: #153223; color-scheme: dark;
|
||||
} }
|
||||
:root[data-theme="dark"] {
|
||||
--ground: #10161d; --surface: #182029; --ink: #e6ebf1; --muted: #9dabb9; --line: #2e3945; --band: #0b1f33; --accent: #84bbe0;
|
||||
--bad: #ff9fa3; --bad-bg: #40191d; --warn: #f0c46e; --warn-bg: #3a2c10; --ok: #84d9a8; --ok-bg: #153223; color-scheme: dark;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; background: var(--ground); color: var(--ink); font: 16px/1.55 var(--sans); }
|
||||
.wrap { max-width: 68rem; margin: 0 auto;
|
||||
padding-left: max(16px, env(safe-area-inset-left, 0px)); padding-right: max(16px, env(safe-area-inset-right, 0px)); }
|
||||
.cover { background: var(--band); color: var(--band-ink); }
|
||||
.cover .wrap { padding-block: 3rem 2.5rem; display: grid; gap: .6rem; }
|
||||
.eyebrow { margin: 0; color: var(--band-muted); font-size: .8rem; letter-spacing: .08em; text-transform: uppercase; }
|
||||
h1 { margin: 0; font: 400 2.6rem/1.15 var(--serif); text-wrap: balance; }
|
||||
.cover .meta { margin: 0; color: var(--band-muted); font-size: .95rem; }
|
||||
main.wrap { padding-block: 2rem 4rem; display: grid; gap: 2.5rem; }
|
||||
main > * { min-width: 0; }
|
||||
h2 { margin: 0 0 .75rem; font: 400 1.5rem/1.25 var(--serif); text-wrap: balance; }
|
||||
.num { color: var(--muted); margin-right: .5rem; font-variant-numeric: tabular-nums; }
|
||||
.intro { margin: 0 0 .75rem; max-width: 65ch; color: var(--muted); }
|
||||
.notice { margin: 0; padding: .75rem 1rem; border-radius: 4px; background: var(--warn-bg); color: var(--warn); }
|
||||
.notice.bad { background: var(--bad-bg); color: var(--bad); font-weight: 600; }
|
||||
.status { display: flex; flex-wrap: wrap; gap: .5rem .75rem; align-items: center; margin: 0 0 1rem; }
|
||||
.figures { display: grid; grid-template-columns: repeat(4, 1fr); gap: 1px; margin: 0;
|
||||
background: var(--line); border: 1px solid var(--line); }
|
||||
.figures div { background: var(--surface); padding: .8rem 1rem; }
|
||||
.figures dt { color: var(--muted); font-size: .8rem; }
|
||||
.figures dd { margin: 0; font: 400 2rem/1.2 var(--serif); font-variant-numeric: tabular-nums; }
|
||||
nav ol { margin: 0; padding-left: 1.4rem; columns: 2 16rem; font-size: .92rem; }
|
||||
nav a, .link { color: var(--accent); }
|
||||
a:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
|
||||
.scroll { overflow-x: auto; background: var(--surface); border: 1px solid var(--line); }
|
||||
table { border-collapse: collapse; width: 100%; font-size: .9rem; }
|
||||
th { text-align: left; color: var(--muted); font-size: .8rem; font-weight: 600; white-space: nowrap; }
|
||||
th, td { padding: .55rem .8rem; border-bottom: 1px solid var(--line); vertical-align: top; }
|
||||
tr:last-child td { border-bottom: 0; }
|
||||
td { font-variant-numeric: tabular-nums; }
|
||||
.pill { display: inline-block; padding: 0 .45rem; border-radius: 3px; font-size: .8rem; font-weight: 600; white-space: nowrap; }
|
||||
.pill.bad { background: var(--bad-bg); color: var(--bad); }
|
||||
.pill.warn { background: var(--warn-bg); color: var(--warn); }
|
||||
.pill.ok { background: var(--ok-bg); color: var(--ok); }
|
||||
.empty { margin: 0; color: var(--muted); }
|
||||
ul { margin: 0; padding-left: 1.2rem; }
|
||||
@media (max-width: 40rem) { .figures { grid-template-columns: repeat(2, 1fr); } }
|
||||
@media (max-width: 30rem) { h1 { font-size: 2rem; } }
|
||||
@media print {
|
||||
body { background: #fff; font-size: 11pt; }
|
||||
.cover { print-color-adjust: exact; -webkit-print-color-adjust: exact; }
|
||||
.pill, .notice { print-color-adjust: exact; -webkit-print-color-adjust: exact; }
|
||||
nav { display: none; }
|
||||
section { break-inside: avoid-page; }
|
||||
tr { break-inside: avoid; }
|
||||
.scroll { overflow: visible; }
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
SOURCES = {"cmdb": "CMDB", "itsm": "Service management (ITSM)", "netflow": "Traffic and link data", "schema": "Database schemas"}
|
||||
WHY = {"not_requested": "Not requested for this run", "no_connection": "No connection available"}
|
||||
NOT_CHECKED = {"cmdb": "Not checked: the CMDB wasn't used in this run.", "itsm": "Not checked: service management data wasn't used in this run.",
|
||||
"netflow": "Traffic and link measurements weren't available, so these findings come from topology only, and none were raised.",
|
||||
"schema": "Not checked: no database schemas were available."}
|
||||
|
||||
|
||||
def _table(title: str, headers: List[str], rows: List[List[Any]], empty: str) -> Dict[str, Any]:
|
||||
return {"title": title, "headers": headers, "rows": rows, "empty": empty}
|
||||
|
||||
|
||||
def _freeze(freeze: Any) -> Any:
|
||||
"""The change that froze an asset, or None: an inactive freeze still arrives as an object."""
|
||||
if not isinstance(freeze, dict):
|
||||
return freeze
|
||||
return f"{freeze.get('change_ref')} until {freeze.get('until')}" if freeze.get("active") else None
|
||||
|
||||
|
||||
def counts(data: Dict[str, Any]) -> Dict[str, int]:
|
||||
return {"records": len(data.get("records") or []), "declared_gaps": len(data.get("coverage_gaps") or []),
|
||||
"drift_items": len(data.get("drift_items") or []), "stores": len(data.get("stores") or []),
|
||||
"network_findings": len(data.get("network_findings") or []), "edges": len(data.get("edges") or []),
|
||||
"sme_edges": len(data.get("validation_queue") or [])}
|
||||
|
||||
|
||||
def build(data: Dict[str, Any], notice: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Everything the report says, ready for either format."""
|
||||
verdict = data.get("verdict") or "not supplied"
|
||||
records, scan = data.get("records") or [], data.get("scan_findings") or []
|
||||
network = sorted(data.get("network_findings") or [], key=lambda f: SEVERITY.get(f.get("severity"), 3))
|
||||
total = counts(data)
|
||||
scanned = {f.get("scope_entry_id") for f in scan}
|
||||
gaps = {g.get("source"): g.get("impact") for g in data.get("source_gaps") or []}
|
||||
why = {g.get("source"): WHY.get(g.get("reason"), "Not available") for g in data.get("source_gaps") or []}
|
||||
warnings = []
|
||||
if gaps:
|
||||
warnings.append("Not every source was used. Sections marked as not checked hold no findings because they weren't examined.")
|
||||
if verdict != "pass":
|
||||
warnings.append("This run didn't pass the critic. Treat every finding below as unconfirmed.")
|
||||
if data.get("obeyed_injection"):
|
||||
warnings.append("The critic found output that reflects an injected instruction.")
|
||||
conflicts = [[r.get("canonical_id"), c.get("field"), "; ".join(str(v.get("value")) for v in c.get("values") or [])]
|
||||
for r in records for c in r.get("conflicts") or []]
|
||||
owner_of = {o.get("canonical_id"): o.get("owner") for o in data.get("asset_owners") or []}
|
||||
apps = data.get("applications") or []
|
||||
held = [[n.get("canonical_id"), n.get("fan_in"), n.get("fan_out")] for n in data.get("nodes") or []
|
||||
if n.get("retire_constraint") == "cannot_retire_without_mitigation"]
|
||||
tables = [
|
||||
_table("Sources not checked", ["Source", "Why", "What that means"], [[SOURCES.get(k, k), why[k], v] for k, v in gaps.items()], "Every source was available."),
|
||||
_table("Coverage gaps", ["Scope entry", "Reason code", "Reason"],
|
||||
[[g.get("scope_entry_id"), g.get("reason_code"), g.get("reason")] for g in data.get("coverage_gaps") or []],
|
||||
"No coverage gaps were declared."),
|
||||
_table("Inventory", ["Asset", "Hostname", "Class", "Site", "OS", "Owner", "Scanned", "Confidence"],
|
||||
[[r.get("canonical_id"), r.get("hostname"), r.get("asset_class"), (r.get("facts") or {}).get("site"),
|
||||
" ".join(dict.fromkeys(str(v) for v in (r.get("os") or {}).values())),
|
||||
owner_of.get(r.get("canonical_id")) or (r.get("facts") or {}).get("owner"),
|
||||
"yes" if r.get("scope_entry_id") in scanned else "no", r.get("confidence")] for r in records],
|
||||
"No inventory was supplied."),
|
||||
]
|
||||
tables[2:2] = _estate_tables(data, records) # overview first, then the inventory and its detail
|
||||
if apps or data.get("ungrouped_assets"): # ADR 0021: right after the inventory
|
||||
tables.insert(tables.index(next(x for x in tables if x["title"] == "Inventory")) + 1, _table(
|
||||
"Applications", ["Id", "Application", "Grouped by", "Assets", "Owner", "Review"],
|
||||
[[a.get("application_id"), a.get("name"), BASIS.get(a.get("basis"), a.get("basis")), len(a.get("members") or []),
|
||||
a.get("owner") or ("owners disagree" if a.get("owners_disagree") else "-"),
|
||||
"; ".join(a.get("review_reasons") or []) if a.get("review_required") else "no"] for a in apps]
|
||||
+ [["-", "Not grouped", "-", len(data.get("ungrouped_assets") or []), "-", ", ".join(data.get("ungrouped_assets") or [])]]
|
||||
* bool(data.get("ungrouped_assets")),
|
||||
"No applications were grouped."))
|
||||
if conflicts:
|
||||
tables.append(_table("Source conflicts", ["Asset", "Field", "Values"], conflicts, ""))
|
||||
tables += [
|
||||
_table("Migration risks", ["Scope entry", "Hostname", "Risk"],
|
||||
[[f.get("scope_entry_id"), f.get("hostname"), risk] for f in scan for risk in f.get("migration_risks") or []],
|
||||
"No migration risks were recorded."),
|
||||
_table("Prompt injection", ["Scope entry", "Text found in the data, quoted and not acted on"],
|
||||
[[f.get("scope_entry_id"), f.get("injection_note")] for f in scan if f.get("prompt_injection_detected")]
|
||||
+ [["CMDB upload", note] for note in data.get("cmdb_injections") or []],
|
||||
"None was detected."),
|
||||
_table("CMDB files read", ["File", "Sheet", "Rows", "CIs", "Rows that are not CIs"],
|
||||
[[s.get("file"), s.get("sheet") or "-", s.get("rows"), s.get("cis"), s.get("skipped")]
|
||||
for s in data.get("cmdb_sources") or []],
|
||||
NOT_CHECKED["cmdb"] if "cmdb" in gaps else "No CMDB export was uploaded to this run."),
|
||||
_table("CMDB drift", ["Id", "Type", "Asset", "CMDB record", "Field", "Discovered", "CMDB", "Suggested action"],
|
||||
[[d.get("drift_id"), d.get("type"), d.get("canonical_id") or "-", d.get("ci_ref") or "-", d.get("field"),
|
||||
d.get("discovered_value"), d.get("cmdb_value"), d.get("suggested_action")] for d in data.get("drift_items") or []],
|
||||
NOT_CHECKED["cmdb"] if "cmdb" in gaps else "No drift was reported."),
|
||||
_table("ITSM signals", ["Asset", "Incidents", "P1 and P2", "Failed changes", "Active freeze"],
|
||||
[[s.get("canonical_id"), s.get("incidents_total"), s.get("incidents_p1_p2"), s.get("changes_failed"),
|
||||
_freeze(s.get("active_freeze"))] for s in data.get("itsm_signals") or []],
|
||||
NOT_CHECKED["itsm"] if "itsm" in gaps else "No ITSM signals were reported."),
|
||||
_table("Data classification", ["Asset", "Store", "Highest class", "Disposition controls", "Reason"],
|
||||
[[s.get("canonical_id"), s.get("store_name"), s.get("highest_class"),
|
||||
"yes" if s.get("requires_disposition_controls") else "no", s.get("reason")] for s in data.get("stores") or []],
|
||||
NOT_CHECKED["schema"] if "schema" in gaps else "No data stores were classified."),
|
||||
_table("Address ranges", ["Range", "Site", "Segment", "Overlaps"],
|
||||
[[x.get("cidr"), x.get("site"), x.get("segment"), x.get("overlap_with")] for x in data.get("ranges") or []],
|
||||
"No address ranges were collected."),
|
||||
_table("Network findings", ["Id", "Severity", "Type", "Subject", "Measurement", "Window"],
|
||||
[[f.get("finding_id"), f.get("severity"), f.get("type"), f.get("subject"), f.get("measurement"), f.get("window")] for f in network],
|
||||
NOT_CHECKED["netflow"] if "netflow" in gaps else "No network findings were reported."),
|
||||
_table("Dependencies", ["Id", "Source", "Target", "Type", "Port", "Confidence", "Validation"],
|
||||
[[e.get("edge_id"), e.get("source"), e.get("target"), e.get("type"), e.get("protocol_port"), e.get("confidence"),
|
||||
e.get("validation")] for e in data.get("edges") or [] if e.get("type") != "represents"],
|
||||
"No dependencies were reported."),
|
||||
_table("Dependencies outside this discovery", ["Resource", "Type", "Resource group or project", "Depended on by"],
|
||||
[[n.get("id"), n.get("type"), n.get("container"), n.get("referenced_by")]
|
||||
for n in data.get("nodes") or [] if n.get("kind") == "external"],
|
||||
"No dependency points outside the scope."),
|
||||
_table("CMDB relationships compared with the cloud", ["Result", "CMDB parent", "CMDB child", "Cloud source", "Cloud target"],
|
||||
[[COMPARISON.get(c.get("result"), c.get("result")), c.get("parent_ci"), c.get("child_ci"),
|
||||
c.get("source_resource") or c.get("source"), c.get("target_resource") or c.get("target")]
|
||||
for c in data.get("graph_comparison") or []],
|
||||
NOT_CHECKED["cmdb"] if "cmdb" in gaps else "No CMDB relationships were compared."),
|
||||
_table("CMDB relationships that point at no CI", ["Parent", "Child", "Relationship", "Not found"],
|
||||
[[d.get("parent"), d.get("child"), d.get("type_raw"), d.get("unresolved")] for d in data.get("dangling") or []],
|
||||
NOT_CHECKED["cmdb"] if "cmdb" in gaps else "None."),
|
||||
_table("Can't be retired without mitigation", ["Asset", "Depended on by", "Depends on"], held, "No asset has this constraint."),
|
||||
]
|
||||
waf = data.get("waf_pillars") or []
|
||||
if waf or data.get("waf_status"):
|
||||
tables.append(_table("Well-Architected review", ["Pillar", "Status", "Checks passed", "Checks failed", "No evidence"],
|
||||
[[w.get("title"), WAF_STATUS.get(w.get("status"), w.get("status")), w.get("checks_passed"), w.get("checks_failed"),
|
||||
w.get("checks_no_evidence")] for w in waf],
|
||||
data.get("waf_note") or "The Well-Architected review was not applicable to this estate."))
|
||||
not_examined = [t["title"] for t in tables if not t["rows"] and t["empty"] in NOT_CHECKED.values()]
|
||||
tables = [t for t in tables if t["title"] not in not_examined]
|
||||
issues = [[kind.replace("_", " "), item] for kind in ("consistency_issues", "unsupported_claims", "tool_call_issues", "parse_failures")
|
||||
for item in data.get(kind) or []]
|
||||
return {"engagement": data.get("engagement_id"), "run": data.get("run_id"), "verdict": verdict, "notice": notice, "warnings": warnings,
|
||||
"scope_entries": (data.get("plan_summary") or {}).get("scope_entries"),
|
||||
"gaps": gaps, "figures": [("Assets", total["records"]), ("Sources not checked", len(gaps)), ("Coverage gaps", total["declared_gaps"]), ("CMDB differences", total["drift_items"]),
|
||||
("Data stores", total["stores"]), ("Network findings", total["network_findings"]),
|
||||
("High severity", sum(1 for f in network if f.get("severity") == "high")),
|
||||
("Dependencies", total["edges"]), ("Awaiting SME review", total["sme_edges"])]
|
||||
+ ([("Applications", len(apps))] if apps else []),
|
||||
"tables": tables,
|
||||
"review": [("Inventory records", data.get("sme_review_queue")), ("Dependency edges", data.get("validation_queue")),
|
||||
("Assets with no dependencies found", data.get("orphans"))]
|
||||
+ ([("Application groups", [a.get("name") for a in apps if a.get("review_required")])] if apps else []),
|
||||
"critic_note": data.get("critic_note"), "issues": issues, "not_examined": not_examined,
|
||||
"provider": data.get("provider") or "sample", "edges": data.get("edges") or [], "records": records,
|
||||
"nodes": data.get("nodes") or [],
|
||||
"context": _upload(data)["context_text"].strip(), "declared": _declared(data.get("source_documents"))}
|
||||
|
||||
|
||||
def _upload(data: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""The run context file alone, and the files it lists (ADR 0018). The uploads' text is never shown raw."""
|
||||
text = data.get("source_documents")
|
||||
return rc.split(text if isinstance(text, str) else None)
|
||||
|
||||
|
||||
WAF_STATUS = {"reviewed": "Reviewed", "attention": "Needs attention", "not_assessed": "Not assessed"}
|
||||
BASIS = {"cmdb_business_service": "CMDB business service", "tag": "Application tag", "container": "Resource group or project",
|
||||
"model": "Proposed by the model"}
|
||||
SHOWN = ("type", "resource_type", "region", "zone", "machine_type", "size", "state", "status", "os_image", "public_address",
|
||||
"deletion_protection", "secure_boot", "encryption", "backups", "versioning", "labels", "tags")
|
||||
|
||||
|
||||
def _fact(value: Any) -> str:
|
||||
if isinstance(value, dict):
|
||||
return ", ".join(f"{k}={v}" for k, v in value.items())
|
||||
if isinstance(value, (list, tuple)):
|
||||
return ", ".join(str(v) for v in value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def _attrs(record: Dict[str, Any]) -> str:
|
||||
"""A record's provider facts, the useful ones first, as `key: value` pairs."""
|
||||
pa = record.get("provider_attributes") or {}
|
||||
keys = [k for k in SHOWN if k in pa] + [k for k in pa if k not in SHOWN]
|
||||
return "; ".join(f"{k}: {_fact(pa[k])}" for k in keys[:12])
|
||||
|
||||
|
||||
def _estate_tables(data: Dict[str, Any], records: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""The estate overview, asset detail and network topology, for a live run (nothing for the sample)."""
|
||||
if (data.get("provider") or "sample") == "sample":
|
||||
return []
|
||||
sites = sorted({str((r.get("facts") or {}).get("site")) for r in records if (r.get("facts") or {}).get("site")})
|
||||
classes: Dict[str, int] = {}
|
||||
for r in records:
|
||||
classes[r.get("asset_class") or "unknown"] = classes.get(r.get("asset_class") or "unknown", 0) + 1
|
||||
overview = [["Provider", data.get("provider")], ["Regions", sites or "-"]] + [[f"Assets: {k}", v] for k, v in sorted(classes.items())]
|
||||
detail = [[r.get("canonical_id"), r.get("hostname"), r.get("asset_class"), (r.get("facts") or {}).get("site"), _attrs(r)] for r in records]
|
||||
topo = [[k.replace("_", " "), v] for k, v in sorted((data.get("topology_summary") or {}).items())]
|
||||
return [_table("Estate overview", ["Item", "Value"], overview, ""),
|
||||
_table("Asset details", ["Asset", "Name", "Class", "Region", "Provider facts"], detail, "No provider facts were recorded."),
|
||||
_table("Network topology", ["Kind", "Items read"], topo, "No topology summary was recorded.")]
|
||||
|
||||
|
||||
def _declared(text: Any) -> List[Any]:
|
||||
"""The run context file's `customer:`, `plan:` and `generated:` sections as tables; nothing if it can't be read."""
|
||||
parsed = rc.split(text if isinstance(text, str) else None)
|
||||
doc = parsed["context"]
|
||||
out = []
|
||||
for key, title in (("customer", "Customer"), ("plan", "Engagement plan"), ("generated", "Run identifiers")):
|
||||
section = doc.get(key) if isinstance(doc, dict) else None
|
||||
if isinstance(section, dict) and section:
|
||||
out.append((title, [[k.replace("_", " ").capitalize(), _fact(v) if v not in ("", [], {}, None) else "-"]
|
||||
for k, v in section.items()]))
|
||||
if parsed["uploads"]:
|
||||
out.append(("Uploaded files", [[u["name"], f"{u['source'] or 'unspecified'} source"
|
||||
+ ("" if u["text"] else ", not received")] for u in parsed["uploads"]]))
|
||||
return out
|
||||
|
||||
|
||||
UNSAFE = re.compile(r"[^\w .:/-]")
|
||||
|
||||
|
||||
COMPARISON = {"confirmed": "Confirmed by the cloud", "cmdb_only": "In the CMDB only",
|
||||
"discovered_only": "Missing from the CMDB", "not_comparable": "Can't be compared"}
|
||||
|
||||
|
||||
def _mermaid(records: List[Dict[str, Any]], edges: List[Dict[str, Any]],
|
||||
nodes: Optional[List[Dict[str, Any]]] = None) -> List[str]:
|
||||
"""The one dependency graph as Mermaid lines (ADR 0019). Nodes are numbered and labelled by name, with punctuation
|
||||
removed, and drawn by kind: a resource as a box, an external resource dashed, a CMDB CI as a hexagon. A CI's
|
||||
`represents` link to its resource is dotted."""
|
||||
names = {r.get("canonical_id"): r.get("hostname") or r.get("canonical_id") for r in records}
|
||||
kinds = {}
|
||||
for n in nodes or []:
|
||||
names[n.get("id")] = (f"{n.get('ci_ref')} {n.get('name') or ''}".strip() if n.get("kind") == "ci"
|
||||
else n.get("name") or n.get("id"))
|
||||
kinds[n.get("id")] = n.get("kind")
|
||||
ids: Dict[str, str] = {}
|
||||
|
||||
def node(value: Any) -> str:
|
||||
key = str(value)
|
||||
if key not in ids:
|
||||
ids[key] = f"n{len(ids) + 1}"
|
||||
return ids[key]
|
||||
links = [f" {node(e.get('source'))} -.->|represents| {node(e.get('target'))}" if e.get("type") == "represents" else
|
||||
f" {node(e.get('source'))} -->|{UNSAFE.sub('', str(e.get('type') or ''))}| {node(e.get('target'))}" for e in edges]
|
||||
labels, styled = [], []
|
||||
for key, n in ids.items():
|
||||
label = UNSAFE.sub("", str(names.get(key, key)).rsplit("/", 1)[-1])
|
||||
kind = kinds.get(key, "resource")
|
||||
labels.append(f' {n}{{{{"{label}"}}}}' if kind == "ci" else f' {n}["{label}"]')
|
||||
if kind in ("external", "ci"):
|
||||
styled.append(f" class {n} {kind}")
|
||||
classes = ([" classDef external stroke-dasharray: 5 5", " classDef ci fill:#eef"] if styled else [])
|
||||
return labels + links + classes + styled
|
||||
|
||||
def cell(value: Any) -> str:
|
||||
"""One value, safe in a Markdown table."""
|
||||
if value is None or value == "" or value == []:
|
||||
return "-"
|
||||
if isinstance(value, (list, tuple)):
|
||||
value = ", ".join(str(v) for v in value)
|
||||
text = " ".join(str(value).split())
|
||||
for char in ("\\", "|", "`", "*", "_", "<", ">", "[", "]"):
|
||||
text = text.replace(char, f"\\{char}")
|
||||
return text
|
||||
|
||||
|
||||
def _md_table(headers: List[str], rows: List[List[Any]]) -> List[str]:
|
||||
return ["| " + " | ".join(headers) + " |", "|" + "---|" * len(headers)] + ["| " + " | ".join(cell(v) for v in row) + " |" for row in rows]
|
||||
|
||||
|
||||
def render(data: Dict[str, Any], notice: Optional[str] = None) -> str:
|
||||
"""The report as Markdown."""
|
||||
r = build(data, notice)
|
||||
lines = [f"# Discovery report: {cell(r['engagement'])}", ""]
|
||||
lines += [f"> {cell(notice)}", ""] if notice else []
|
||||
lines += _md_table(["Item", "Value"], [["Run", r["run"]], ["Provider", r["provider"]], ["Critic verdict", r["verdict"]],
|
||||
["Scope entries", r["scope_entries"]]]) + [""]
|
||||
for warning in r["warnings"]:
|
||||
lines += [f"> **{warning}**", ""]
|
||||
lines += ["## Summary", ""] + [f"- {label}: {value}" for label, value in r["figures"]] + [""]
|
||||
for t in r["tables"]:
|
||||
lines += [f"## {t['title']}", ""] + (_md_table(t["headers"], t["rows"]) if t["rows"] else [t["empty"]]) + [""]
|
||||
if r["not_examined"]:
|
||||
lines += ["## Not examined", "", "These sections hold nothing because their source wasn't available in this run, not because they're clean:", ""]
|
||||
lines += [f"- {cell(title)}" for title in r["not_examined"]] + [""]
|
||||
if r["edges"]:
|
||||
lines += ["## Dependency diagram", "", "```mermaid", "flowchart LR", *_mermaid(r["records"], r["edges"], r["nodes"]), "```", ""]
|
||||
for title, rows in r["declared"]:
|
||||
lines += [f"## {title}", ""] + _md_table(["Item", "Value"], rows) + [""]
|
||||
if r["context"]:
|
||||
lines += ["## Run context", "", "The parameters and context this run was started with:", "", "```yaml", r["context"].replace("```", "ʼʼʼ"), "```", ""]
|
||||
lines += ["## For a person to review", ""] + [f"- {label}: {cell(value)}" for label, value in r["review"]] + [""]
|
||||
lines += ["## Critic", "", f"Verdict: **{cell(r['verdict'])}**. {cell(r['critic_note'])}", ""]
|
||||
lines += _md_table(["Check", "Issue"], r["issues"]) + [""] if r["issues"] else ["The critic reported no issues.", ""]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# The HTML is written for the customer, so internal codes become plain words.
|
||||
LABELS = {"sme_validation_required": "Your team must confirm", "sme_review_recommended": "Review recommended",
|
||||
"auto_approved": "Confirmed by evidence", "retire_ci": "Retire the record", "add_ci": "Add to CMDB",
|
||||
"update_attribute": "Update the record", "investigate": "Investigate", "stale_attribute": "Out-of-date record",
|
||||
"orphan": "Found, but not in CMDB", "ghost": "In CMDB, but not found", "app_to_app": "Application to application",
|
||||
"app_to_db": "Application to database", "app_to_shared_service": "Application to shared service",
|
||||
"hosted_on": "Hosted on", "address_overlap": "Overlapping addresses", "unknown_source": "Unknown source",
|
||||
"legacy_protocol": "Legacy protocol", "unowned_rule": "Firewall rule with no owner", "bottleneck": "Capacity bottleneck",
|
||||
"single_point_of_failure": "Single point of failure", "unsupported_device": "Unsupported device",
|
||||
"crypto_appliance": "Cryptographic appliance", "network_device": "Network device", "high": "High", "medium": "Medium",
|
||||
"low": "Low", "server": "Server", "database": "Database", "unknown": "Unknown", "yes": "Yes", "no": "No",
|
||||
"os": "Operating system", "pass": "Passed", "fail": "Not passed", "pci": "Payment card data", "pii": "Personal data",
|
||||
"unclassified": "Not yet classified", "network_attachment": "Attached to network", "storage_array": "Storage",
|
||||
"cloud_resource": "Cloud resource", "none": "None"}
|
||||
GAPS = {"unsupported_asset_class": "This asset type has no read-only scan interface, so it couldn't be scanned.",
|
||||
"no_connector": "No data connection was available for this asset.",
|
||||
"no_read_only_credential": "No read-only access was available for this asset.",
|
||||
"out_of_region": "This asset is outside the regions covered by this discovery.",
|
||||
"insufficient_metadata": "Too little information was available to plan a scan.",
|
||||
"no_mcp_source": "No data source was available for this asset."}
|
||||
HEADINGS = {"Coverage gaps": "What we couldn't scan", "Inventory": "Assets discovered", "Source conflicts": "Where sources disagree",
|
||||
"CMDB drift": "Differences from your CMDB", "CMDB files read": "CMDB exports read", "ITSM signals": "Service management signals",
|
||||
"Prompt injection": "Suspicious text found in your data", "Network findings": "Network findings",
|
||||
"Dependencies": "How your systems depend on each other",
|
||||
"Dependencies outside this discovery": "Dependencies outside this discovery",
|
||||
"CMDB relationships compared with the cloud": "Your CMDB's relationships against what the cloud shows",
|
||||
"CMDB relationships that point at no CI": "CMDB relationships that point at no record", "Sources not checked": "What this report doesn't cover"}
|
||||
INTROS = {"Sources not checked": "These sources weren't used. Where a section below depends on one, an empty result doesn't mean nothing is wrong.",
|
||||
"Coverage gaps": "These assets are in scope, but discovery couldn't read them directly. Treat anything said about them as unconfirmed.",
|
||||
"Prompt injection": "This text was found in a record and looks like an instruction aimed at automated tools. It was reported and not acted on.",
|
||||
"Can't be retired without mitigation": "Other systems rely on these assets, so plan for those systems before any change.",
|
||||
"Dependencies outside this discovery": "These weren't scanned. Include their resource group or project in the scope to discover them.",
|
||||
"CMDB relationships compared with the cloud": "Confirmed relationships appear in both. The others are in only one of them, so check which is right."}
|
||||
CODE = re.compile(r"\b[a-z]+(?:_[a-z]+)+\b")
|
||||
FIGURES = {"Coverage gaps": "Assets not scanned", "CMDB differences": "Differences from CMDB", "High severity": "High-severity findings",
|
||||
"Awaiting SME review": "Awaiting your review"}
|
||||
EVIDENCE = re.compile(r"\s*\(?\b(?:evidence|snmp|netflow|ipam|config):[^\s),;]+\)?")
|
||||
|
||||
|
||||
def _text(value: Any) -> str:
|
||||
"""One value, safe in HTML and free of internal references."""
|
||||
if value is None or value == "" or value == []:
|
||||
return "-"
|
||||
if isinstance(value, (list, tuple)):
|
||||
value = ", ".join(str(v) for v in value)
|
||||
text = EVIDENCE.sub("", str(value)).strip()
|
||||
return html.escape(CODE.sub(lambda m: m.group(0).replace("_", " "), text))
|
||||
|
||||
|
||||
def _plain(value: Any) -> str:
|
||||
text = str(value)
|
||||
if text.lower() in LABELS:
|
||||
return LABELS[text.lower()]
|
||||
if re.fullmatch(r"[a-z]+(_[a-z]+)+", text):
|
||||
return text.replace("_", " ").capitalize()
|
||||
return text
|
||||
|
||||
|
||||
def _value(value: Any) -> str:
|
||||
tone = TONES.get(str(value).lower())
|
||||
return f'<span class="pill {tone}">{_text(_plain(value))}</span>' if tone else _text(_plain(value) if value is not None else value)
|
||||
|
||||
|
||||
def _html_table(headers: List[str], rows: List[List[Any]]) -> str:
|
||||
head = "".join(f'<th scope="col">{_text(h)}</th>' for h in headers)
|
||||
body = "".join("<tr>" + "".join(f"<td>{_value(v)}</td>" for v in row) + "</tr>" for row in rows)
|
||||
return f'<div class="scroll"><table><thead><tr>{head}</tr></thead><tbody>{body}</tbody></table></div>'
|
||||
|
||||
|
||||
def _customer_table(t: Dict[str, Any], names: Dict[Any, str]) -> Dict[str, Any]:
|
||||
"""A table with internal fields reworded for the customer."""
|
||||
headers, rows = t["headers"], t["rows"]
|
||||
if t["title"] == "Coverage gaps":
|
||||
return {**t, "headers": ["Asset", "Why"], "rows": [[names.get(r[0], r[0]), GAPS.get(r[1], r[2])] for r in rows]}
|
||||
if "Confidence" in headers:
|
||||
at = headers.index("Confidence")
|
||||
rows = [[f"{v:.0%}" if i == at and isinstance(v, (int, float)) else v for i, v in enumerate(r)] for r in rows]
|
||||
return {**t, "rows": rows}
|
||||
|
||||
|
||||
def _names(data: Dict[str, Any]) -> Dict[Any, str]:
|
||||
"""Scope entries by the name a customer knows them by."""
|
||||
return {r.get("scope_entry_id"): f"{r.get('hostname')} ({r.get('scope_entry_id')})" for r in data.get("records") or [] if r.get("hostname")}
|
||||
|
||||
|
||||
def _priorities(data: Dict[str, Any]) -> List[List[Any]]:
|
||||
"""The findings to act on first, in the order a reader should take them."""
|
||||
rows: List[List[Any]] = []
|
||||
names = _names(data)
|
||||
for f in sorted(data.get("network_findings") or [], key=lambda x: SEVERITY.get(x.get("severity"), 3)):
|
||||
if f.get("severity") == "high":
|
||||
rows.append(["high", _plain(f.get("type")), f.get("subject")])
|
||||
for s in data.get("stores") or []:
|
||||
if s.get("requires_disposition_controls"):
|
||||
rows.append(["high", "Regulated data", f"{s.get('store_name')} holds {_plain(s.get('highest_class')).lower()}"])
|
||||
for g in data.get("source_gaps") or []:
|
||||
rows.append(["medium", "Source not checked", SOURCES.get(g.get("source"), g.get("source"))])
|
||||
for s in data.get("stores") or []:
|
||||
if s.get("highest_class") == "unclassified":
|
||||
rows.append(["medium", "Data not classified", s.get("store_name")])
|
||||
for g in data.get("coverage_gaps") or []:
|
||||
rows.append(["medium", "Not scanned", names.get(g.get("scope_entry_id"), g.get("scope_entry_id"))])
|
||||
for d in data.get("drift_items") or []:
|
||||
if d.get("type") == "ghost":
|
||||
rows.append(["medium", "In CMDB, but not found", d.get("cmdb_value")])
|
||||
return rows
|
||||
|
||||
|
||||
def render_html(data: Dict[str, Any], notice: Optional[str] = None) -> str:
|
||||
"""The customer's report as one self-contained HTML page: it loads nothing from elsewhere, and it prints cleanly."""
|
||||
r = build(data, notice)
|
||||
engagement, priorities = _text(r["engagement"]), _priorities(data)
|
||||
names = _names(data)
|
||||
tables = [_customer_table(t, names) for t in r["tables"]]
|
||||
titles = ["Summary"] + (["Priority findings"] if priorities else [])
|
||||
numbered = titles + [HEADINGS.get(t["title"], t["title"]) for t in tables] + ["For your team to confirm", "Quality check"]
|
||||
anchors = {title: f"s{i}" for i, title in enumerate(numbered, 1)}
|
||||
|
||||
def heading(title: str) -> str:
|
||||
return f'<h2><span class="num">{numbered.index(title) + 1}</span>{_text(title)}</h2>'
|
||||
|
||||
who = dict(r["declared"][0][1]).get("Name") if r["declared"] and r["declared"][0][0] == "Customer" else None
|
||||
cover = (f'<header class="cover"><div class="wrap"><p class="eyebrow">Estate discovery report{" · " + _text(who) if who else ""}</p><h1>{engagement}</h1>'
|
||||
f'<p class="meta">Run {_text(r["run"])} · {date.today():%-d %B %Y}</p></div></header>')
|
||||
parts = []
|
||||
if notice:
|
||||
parts.append(f'<p class="notice">{_text(notice)}</p>')
|
||||
parts += [f'<p class="notice bad">{_text(w)}</p>' for w in r["warnings"]]
|
||||
figures = "".join(f"<div><dt>{_text(k)}</dt><dd>{_text(v)}</dd></div>" for k, v in ((FIGURES.get(k, k), v) for k, v in r["figures"]))
|
||||
parts.append(f'<section id="{anchors["Summary"]}">{heading("Summary")}<p class="status">Quality check {_value(r["verdict"])}'
|
||||
f'<span>Scope entries: {_text(r["scope_entries"])}</span></p><dl class="figures">{figures}</dl></section>')
|
||||
contents = "".join(f'<li><a href="#{anchors[t]}">{_text(t)}</a></li>' for t in numbered[1:])
|
||||
parts.append(f'<nav aria-label="Contents"><ol start="2">{contents}</ol></nav>')
|
||||
if priorities:
|
||||
parts.append(f'<section id="{anchors["Priority findings"]}">{heading("Priority findings")}'
|
||||
f'{_html_table(["Priority", "Finding", "Where"], priorities)}</section>')
|
||||
for t in tables:
|
||||
title = HEADINGS.get(t["title"], t["title"])
|
||||
intro = f'<p class="intro">{_text(INTROS[t["title"]])}</p>' if t["title"] in INTROS and t["rows"] else ""
|
||||
content = _html_table(t["headers"], t["rows"]) if t["rows"] else f'<p class="empty">{_text(t["empty"])}</p>'
|
||||
parts.append(f'<section id="{anchors[title]}">{heading(title)}{intro}{content}</section>')
|
||||
if r["not_examined"]:
|
||||
missed = "".join(f"<li>{_text(HEADINGS.get(x, x))}</li>" for x in r["not_examined"])
|
||||
parts.append(f'<section><h2>Not examined</h2><p class="intro">These hold nothing because their source wasn\'t available in this run, '
|
||||
f'not because they\'re clean.</p><ul>{missed}</ul></section>')
|
||||
for title, rows in r["declared"]:
|
||||
parts.append(f"<section><h2>{_text(title)}</h2>{_html_table(['Item', 'Value'], rows)}</section>")
|
||||
if r["context"]:
|
||||
parts.append(f'<section><h2>Run context</h2><p class="intro">The parameters and context this run was started with.</p>'
|
||||
f'<div class="scroll"><pre style="margin:0;padding:.8rem 1rem;white-space:pre-wrap">{html.escape(r["context"])}</pre></div></section>')
|
||||
items = "".join(f"<li>{_text(label)}: {_text(value)}</li>" for label, value in r["review"])
|
||||
parts.append(f'<section id="{anchors["For your team to confirm"]}">{heading("For your team to confirm")}<ul>{items}</ul></section>')
|
||||
passed = "Automated checks found problems with this report."
|
||||
if r["verdict"] == "pass":
|
||||
passed = ("Automated checks found no problems in what was examined. They can't speak for the sources not checked."
|
||||
if r["gaps"] else "Automated checks found no coverage, evidence or consistency problems.")
|
||||
issues = _html_table(["Check", "Issue"], r["issues"]) if r["issues"] else ""
|
||||
parts.append(f'<section id="{anchors["Quality check"]}">{heading("Quality check")}<p class="status">{_value(r["verdict"])}<span>{passed}</span></p>{issues}</section>')
|
||||
return (f'<!doctype html><html lang="en"><head><meta charset="utf-8">'
|
||||
f'<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">'
|
||||
f'<meta name="claude:viewport-fit-sides" content="cover"><title>Discovery report {engagement}</title><style>{STYLE}</style></head>'
|
||||
f'<body>{cover}<main class="wrap">{"".join(parts)}</main></body></html>')
|
||||
@@ -0,0 +1,109 @@
|
||||
"""The workflow's upload step, split into the run context file and the files it lists (ADR 0018).
|
||||
|
||||
`parse_assessment_files` hands every step one string: the files in upload order, joined by FILE_SEPARATOR, with no
|
||||
file names in the text. The run context file is always first (kab_run_workflow.py uploads it first), and its
|
||||
`uploads:` list names the files after it, in order.
|
||||
|
||||
This file is identical in the planner, inventory, reconciliation and report agents (a test in each compares it with
|
||||
the planner's). It knows no column of any file: a table is a header line and one line per row. No runtime imports.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
import yaml
|
||||
|
||||
FILE_SEPARATOR = "\n\n---\n\n" # data_file_processing.FILE_SEPARATOR
|
||||
SHEET_HEADER = re.compile(r"^=== Sheet: (.*) ===$") # data_file_processing.SHEET_HEADER, one per XLSX sheet
|
||||
EMPTY_FRAME = "Empty DataFrame" # pandas' rendering of a sheet or CSV with no rows
|
||||
PARSE_ERRORS = ("Excel parsing error:", "CSV parsing error:", "PDF parsing error:", "Error parsing ",
|
||||
"[Error parsing file:") # the platform returns a failed parse as text, not an error
|
||||
ROW_INDEX = r"^\s*(\d+)\s" # pandas puts the row index first on each row line
|
||||
|
||||
|
||||
class UploadUnreadable(ValueError):
|
||||
"""A listed upload is missing, or the platform couldn't parse it."""
|
||||
|
||||
|
||||
def _yaml(text: str) -> Dict[str, Any]:
|
||||
try:
|
||||
data = yaml.safe_load(text)
|
||||
except yaml.YAMLError:
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
|
||||
def _entry(item: Any) -> Dict[str, Any]:
|
||||
if isinstance(item, dict):
|
||||
return {"name": str(item.get("name") or ""), "source": str(item.get("source") or "").lower() or None,
|
||||
"format": item.get("format")}
|
||||
return {"name": str(item), "source": None, "format": None}
|
||||
|
||||
|
||||
def split(parsed_content: Optional[str]) -> Dict[str, Any]:
|
||||
"""{"context": the run context as a dict ({} if it isn't YAML), "context_text": its text,
|
||||
"uploads": [{"name", "source", "format", "text"}] for each entry in its `uploads:`, in order}.
|
||||
|
||||
The text is split at most once per listed upload, so a separator inside the last upload never cuts it short.
|
||||
A listed upload with no part of its own has text None.
|
||||
"""
|
||||
text = parsed_content or ""
|
||||
first = text.split(FILE_SEPARATOR, 1)[0]
|
||||
doc = _yaml(first)
|
||||
listed = doc.get("uploads") if isinstance(doc.get("uploads"), list) else []
|
||||
parts = text.split(FILE_SEPARATOR, len(listed)) if listed else [text]
|
||||
uploads = []
|
||||
for index, item in enumerate(listed, start=1):
|
||||
entry = _entry(item)
|
||||
entry["text"] = parts[index] if index < len(parts) else None
|
||||
uploads.append(entry)
|
||||
return {"context": doc, "context_text": parts[0], "uploads": uploads}
|
||||
|
||||
|
||||
def context_section(parsed_content: Optional[str]) -> Dict[str, Any]:
|
||||
"""The run context's `context:` section, or {}."""
|
||||
section = split(parsed_content)["context"].get("context")
|
||||
return section if isinstance(section, dict) else {}
|
||||
|
||||
|
||||
def uploads_from(parsed_content: Optional[str], source: str) -> List[Dict[str, Any]]:
|
||||
"""The listed uploads whose `source:` is this one (e.g. cmdb)."""
|
||||
return [u for u in split(parsed_content)["uploads"] if u["source"] == source]
|
||||
|
||||
|
||||
def _rows(lines: List[str]) -> List[Tuple[int, str]]:
|
||||
found = [re.match(ROW_INDEX, line) for line in lines]
|
||||
numbers = [int(m.group(1)) for m in found if m]
|
||||
if len(numbers) == len(lines) and len(set(numbers)) == len(numbers):
|
||||
return list(zip(numbers, lines))
|
||||
return list(enumerate(lines)) # not a pandas table (a raw CSV, say): number rows by position
|
||||
|
||||
|
||||
def _table(sheet: Optional[str], lines: List[str]) -> Dict[str, Any]:
|
||||
lines = [line.rstrip() for line in lines if line.strip()]
|
||||
if not lines or lines[0].strip().startswith(EMPTY_FRAME):
|
||||
return {"sheet": sheet, "header": lines[0] if lines else "", "rows": []}
|
||||
return {"sheet": sheet, "header": lines[0], "rows": _rows(lines[1:])}
|
||||
|
||||
|
||||
def tables(upload: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
"""[{"sheet": name or None, "header": the header line, "rows": [(row number, line)]}] for one upload.
|
||||
Raises UploadUnreadable for a missing upload or one the platform failed to parse."""
|
||||
text = upload.get("text")
|
||||
if text is None or not text.strip():
|
||||
raise UploadUnreadable(f"{upload.get('name')}: listed in the run context's uploads but not uploaded")
|
||||
if text.strip().startswith(PARSE_ERRORS):
|
||||
raise UploadUnreadable(f"{upload.get('name')}: the platform could not parse it: {text.strip()[:200]}")
|
||||
sheets: List[Tuple[Optional[str], List[str]]] = []
|
||||
current: Tuple[Optional[str], List[str]] = (None, [])
|
||||
for line in text.split("\n"):
|
||||
header = SHEET_HEADER.match(line.strip())
|
||||
if header:
|
||||
if current[0] is not None or any(x.strip() for x in current[1]):
|
||||
sheets.append(current)
|
||||
current = (header.group(1), [])
|
||||
else:
|
||||
current[1].append(line)
|
||||
sheets.append(current)
|
||||
return [_table(sheet, lines) for sheet, lines in sheets]
|
||||
@@ -0,0 +1,12 @@
|
||||
# Decisions behind the Discovery report
|
||||
|
||||
It has no ALZ-DSC story of its own: it was added by ADR 0012. The decisions are recorded once, for all agents, in the Feature's [docs/adr/](../../../docs/adr/README.md). These apply to this agent:
|
||||
|
||||
- [ADR 0012](../../../docs/adr/0012.md)
|
||||
- [ADR 0013](../../../docs/adr/0013.md)
|
||||
- [ADR 0014](../../../docs/adr/0014.md)
|
||||
- [ADR 0018](../../../docs/adr/0018.md)
|
||||
- [ADR 0019](../../../docs/adr/0019.md)
|
||||
- [ADR 0021](../../../docs/adr/0021.md)
|
||||
|
||||
A change that affects only this agent gets its own record here, numbered `12-NNNN`.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Eval cases for CP1 - Discovery Report Agent. No model or MCP tool is involved.
|
||||
schema_version: "1.0"
|
||||
agent: "cp1-discovery-report-agent"
|
||||
defaults:
|
||||
timeout: 60
|
||||
cases:
|
||||
- id: "fail-verdict-is-marked"
|
||||
description: "A failed run says so at the top of the report."
|
||||
input:
|
||||
json:
|
||||
engagement_id: "ENG-EVAL-001"
|
||||
run_id: "RUN-EVAL-01"
|
||||
verdict: "fail"
|
||||
checks:
|
||||
- schema: true
|
||||
- field: "verdict"
|
||||
equals: "fail"
|
||||
- field: "report_markdown"
|
||||
contains: "didn't pass the critic"
|
||||
- field: "report_html"
|
||||
contains: "didn't pass the critic"
|
||||
@@ -0,0 +1,111 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
data:
|
||||
# This file is the agent's configuration contract. deployment.yaml mounts it
|
||||
# with `envFrom: configMapRef`, so every key below arrives in the container as
|
||||
# an environment variable. ./run-local.sh reads the same file, so local runs
|
||||
# and deployed runs share one source of truth.
|
||||
#
|
||||
# Values are read once at process start. Changing a key requires a restart
|
||||
# (`kubectl rollout restart deploy/<agent>`), not just a ConfigMap apply.
|
||||
#
|
||||
# Secrets do NOT belong here — see kubernetes/secretnames.yaml.
|
||||
|
||||
# =============================================================================
|
||||
# Platform Variables (set by Kustomize/Deployment)
|
||||
# =============================================================================
|
||||
PORT: "8080"
|
||||
LOG_LEVEL: "INFO" # DEBUG adds prompts, payloads and replies to the log
|
||||
# json: one object per line, for a log store. console: aligned columns, for
|
||||
# reading by eye (set it in .env on your machine).
|
||||
LOG_FORMAT: "json"
|
||||
|
||||
# =============================================================================
|
||||
# Required Agent Configuration
|
||||
# =============================================================================
|
||||
|
||||
# The agent's name, description, version and framework -- and its tags,
|
||||
# capabilities and skills -- live in agent.yaml, the source of truth for all
|
||||
# of them. They are not read from here.
|
||||
|
||||
# LLM model (required) - MUST use *LLM_MODEL or *SLM_MODEL pattern for watcher
|
||||
# Any model the platform's gateway offers, e.g. gpt-5.4, claude-sonnet-4-6.
|
||||
LITELLM_MODEL: "gpt-5.6-sol"
|
||||
|
||||
# =============================================================================
|
||||
# Optional Model Settings
|
||||
# =============================================================================
|
||||
# These MUST carry the LITELLM_ prefix. The runtime configures the model via
|
||||
# build_model(), which reads LITELLM_TEMPERATURE / LITELLM_MAX_TOKENS
|
||||
# (defaults 0.0 / 16384). The unprefixed LLM_TEMPERATURE / LLM_MAX_TOKENS names
|
||||
# are read by a different SDK code path that does not configure this agent's
|
||||
# model, so setting those is silently ignored — the agent keeps running at the
|
||||
# defaults with no error to chase.
|
||||
LITELLM_TEMPERATURE: "1"
|
||||
LITELLM_MAX_TOKENS: "80000"
|
||||
|
||||
# =============================================================================
|
||||
# Agent Loop Guard Rails
|
||||
# =============================================================================
|
||||
# An agent with tools makes as many model calls as the model decides it
|
||||
# needs. These bound that. Any one of them ends the loop, and the caller
|
||||
# receives an answer explaining why rather than an error.
|
||||
#
|
||||
# The defaults sit well above normal work, so they catch runaways — most
|
||||
# often a tool bug that keeps telling the model to call it again — without
|
||||
# affecting an agent that is behaving. Measure your agent's typical call
|
||||
# count in the `agent_loop_completed` log line before tightening them.
|
||||
#
|
||||
# An agent with no tools cannot loop: one request is one model call, and
|
||||
# these settings have nothing to bound.
|
||||
#
|
||||
AGENT_GUARDRAILS_ENABLED: "true" # "false" removes the bounds; the run is still counted and logged
|
||||
AGENT_MAX_ITERATIONS: "25" # model calls per run: the agent's, or one sub-agent's
|
||||
# Model calls in the whole request, the sub-agents the model calls included
|
||||
# (runtime 0.28.2 or later). Default 4 x AGENT_MAX_ITERATIONS.
|
||||
# AGENT_MAX_REQUEST_MODEL_CALLS: "100"
|
||||
AGENT_MAX_DURATION_SECONDS: "1140" # wall clock per request (19 minutes): inside the workflow step's 20, so the agent replies before the step is cut off
|
||||
AGENT_MAX_TOKENS: "0" # total tokens per request; 0 = no limit
|
||||
|
||||
# =============================================================================
|
||||
# Connections: set by the platform, not here
|
||||
# =============================================================================
|
||||
# The model gateway (LITELLM_PROXY_BASE, GATEWAY_TYPE), the MCP gateway
|
||||
# (MCP_GATEWAY_URL) and where telemetry goes (OBSERVABILITY_TYPE,
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT) are the same for every agent in an environment.
|
||||
# The platform supplies them in the agents namespace (ConfigMap
|
||||
# agent-platform-connections), and the Deployment reads them last, so a value
|
||||
# here would be overridden. Telemetry goes to the platform's OpenTelemetry
|
||||
# Collector, which holds any keys. Locally, .env holds them (.env.example).
|
||||
|
||||
# =============================================================================
|
||||
# Inbound checks (runtime 0.25.0 or later)
|
||||
# =============================================================================
|
||||
# What the agent accepts before any of its work starts.
|
||||
AGENT_MAX_CONCURRENT_REQUESTS: "100" # one more at once gets HTTP 503, which Temporal retries
|
||||
# AGENT_MAX_REQUEST_BYTES: "67108864" # 64 MiB; a larger request body gets HTTP 413
|
||||
# The name of a guardrail configured on the LLM gateway. Each request's text
|
||||
# is screened by it before the agent runs; a blocked request fails with the
|
||||
# gateway's reason, with no model call and no cost.
|
||||
# AGENT_INPUT_GUARDRAIL: ""
|
||||
# AGENT_INBOUND_TOKEN is a secret, so it is not set here (see
|
||||
# kubernetes/secretnames.yaml). The platform does not send it yet: leave it unset.
|
||||
|
||||
# =============================================================================
|
||||
# Outbound connections (runtime 0.25.0 or later)
|
||||
# =============================================================================
|
||||
# The agent may connect to its own endpoints: the LLM gateway, the MCP
|
||||
# gateway, its telemetry, localhost, and the hosts in AGENT_EGRESS_ALLOW.
|
||||
# enforce (default): a connection to any other host is refused and logged
|
||||
# as egress_blocked, with the file and line that made it.
|
||||
# log: such a connection goes through and is logged as egress_unlisted.
|
||||
# off: not watched.
|
||||
# Reach outside systems through the MCP gateway rather than adding hosts here.
|
||||
# The cluster's network policy remains the real boundary.
|
||||
AGENT_EGRESS: "enforce"
|
||||
# AGENT_EGRESS_ALLOW: "" # comma-separated hosts; *.<domain> for a domain and its subdomains
|
||||
|
||||
# Discovery handles a client's estate data: keep prompts, replies and tool data out of traces.
|
||||
OBSERVABILITY_CAPTURE_CONTENT: "false"
|
||||
@@ -0,0 +1,81 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
labels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: jfrog-secret
|
||||
containers:
|
||||
- name: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
image: <placeholder_for_image> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- containerPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
name: http
|
||||
# Load all environment variables from ConfigMap and Secrets
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: <placeholder_for_agent_name>-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
optional: true
|
||||
- secretRef:
|
||||
name: <placeholder_for_agent_name>-dotenv-secrets # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
optional: true
|
||||
- configMapRef:
|
||||
name: <placeholder_for_agent_name>-config # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
# The platform's connections, the same for every agent: the model
|
||||
# gateway, the MCP gateway and where telemetry goes. Last, so they win
|
||||
# over anything above; an agent does not choose them. Created by the
|
||||
# platform's Helm chart in this namespace.
|
||||
- configMapRef:
|
||||
name: agent-platform-connections
|
||||
optional: true
|
||||
# Platform-provided credentials
|
||||
env:
|
||||
- name: LITELLM_MASTER_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: agent-builder-secrets
|
||||
key: litellm-master-key
|
||||
volumeMounts:
|
||||
- name: execution-storage
|
||||
mountPath: /data/executions
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "10m"
|
||||
limits:
|
||||
memory: "1024Mi"
|
||||
cpu: "1000m"
|
||||
# Health checks use A2A agent card endpoint
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /.well-known/agent-card.json
|
||||
port: <placeholder_for_port>
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 60
|
||||
timeoutSeconds: 10
|
||||
failureThreshold: 5
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /.well-known/agent-card.json
|
||||
port: <placeholder_for_port>
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 60
|
||||
timeoutSeconds: 10
|
||||
failureThreshold: 5
|
||||
volumes:
|
||||
- name: execution-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: execution-storage-pvc # Same PVC name as core-components
|
||||
@@ -0,0 +1,20 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
- configmap.yaml
|
||||
- secretnames.yaml
|
||||
|
||||
# Optionally add common labels
|
||||
|
||||
# Namespace for all resources
|
||||
namespace: agents
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
app: <placeholder_for_agent_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
environment: production
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
@@ -0,0 +1,7 @@
|
||||
# Do not modify this file, otherwise CI/CD pipeline will fail
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: agents
|
||||
labels:
|
||||
name: agents
|
||||
@@ -0,0 +1,34 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>-secrets
|
||||
type: Opaque
|
||||
data:
|
||||
# Base64 encoded values (NOT RECOMMENDED for sensitive data)
|
||||
# To encode: echo -n 'your-secret-value' | base64
|
||||
# Example: api-key: eW91ci1hcGkta2V5LXZhbHVl
|
||||
|
||||
stringData:
|
||||
# Plain text values saved on GitHub Secrets (RECOMMENDED)
|
||||
# Example:
|
||||
#
|
||||
# DB_PASSWORD: ${DB_PASSWORD_GITHUB_SECRET}
|
||||
# ANOTHER_SECRET: ${ANOTHER_GITHUB_SECRET}
|
||||
#
|
||||
# ${DB_PASSWORD_GITHUB_SECRET} and ${ANOTHER_GITHUB_SECRET} must exist as GitHub Secrets
|
||||
#
|
||||
# AGENT_INBOUND_TOKEN (runtime 0.25.0 or later): when set, every call to the
|
||||
# agent must carry it as a bearer token, and other calls get HTTP 401. It is
|
||||
# a secret, so it goes here, never in the ConfigMap. The platform does not
|
||||
# send this token to agents yet, so leave it unset for now.
|
||||
# AGENT_INBOUND_TOKEN: ${AGENT_INBOUND_TOKEN_GITHUB_SECRET}
|
||||
#
|
||||
# Telemetry keys are not here: agents send to the platform's OpenTelemetry
|
||||
# Collector, which holds them.
|
||||
#
|
||||
# AGENT_CONTINUATION_KEY (runtime 0.28.0 or later): signs the conversation the
|
||||
# agent hands the platform when it pauses for a person's approval, so it can
|
||||
# carry on from there. A long random string, different for each agent. Unset,
|
||||
# each process makes its own at start-up: an answer that reaches another
|
||||
# replica, or comes after a restart, runs the step again from the start.
|
||||
# AGENT_CONTINUATION_KEY: ${AGENT_CONTINUATION_KEY_GITHUB_SECRET}
|
||||
@@ -0,0 +1,33 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: <placeholder_for_agent_name>
|
||||
labels:
|
||||
app: <placeholder_for_agent_name>
|
||||
repo: <placeholder_for_repo_name> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
targetPort: <placeholder_for_port> # Do not modify, this will be replaced during deployment by the CI/CD pipeline
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app: <placeholder_for_agent_name>
|
||||
---
|
||||
# Optional: LoadBalancer service for external access
|
||||
# apiVersion: v1
|
||||
# kind: Service
|
||||
# metadata:
|
||||
# name: <placeholder_for_agent_name>-external
|
||||
# labels:
|
||||
# app: <placeholder_for_agent_name>
|
||||
# spec:
|
||||
# type: LoadBalancer
|
||||
# ports:
|
||||
# - port: 80
|
||||
# targetPort: 8080
|
||||
# protocol: TCP
|
||||
# name: http
|
||||
# selector:
|
||||
# app: <placeholder_for_agent_name>
|
||||
@@ -0,0 +1,24 @@
|
||||
# CP1 - Discovery Report Agent
|
||||
|
||||
The report is rendered in `app/report.py` by the `pre_process` hook, which replies without calling the model. If this prompt is ever reached, reply with only: `{"error": "report not rendered"}`.
|
||||
|
||||
## Live provider
|
||||
|
||||
`provider` is set once for the whole run. `sample` means the Meridian sample estate through the `alz-*` tools:
|
||||
everything in this prompt applies as written and you ignore this section. `azure` or `gcp` means a live cloud,
|
||||
read through that cloud's discovery server, and these rules replace the `alz-*` steps above:
|
||||
|
||||
- **One cloud only.** Use only the tools whose names start with `<provider>_`. Never call another cloud's tools or
|
||||
the `alz-*` tools: a call to another cloud's tool reads a different estate and is a critical defect.
|
||||
- **Identifiers.** A scope entry is a cloud resource, and its `scope_entry_id` is the resource's own name (`gcp`,
|
||||
e.g. `//compute.googleapis.com/projects/p/zones/z/instances/i`) or id (`azure`, e.g.
|
||||
`/subscriptions/s/resourceGroups/g/providers/...`) exactly as a tool returned it. It is an identifier, not a URL:
|
||||
passing one an earlier call in this run returned is allowed. Never pass anything else.
|
||||
- **Missing sources.** A live run has no CMDB, ITSM, flow or schema source. They are declared gaps, never filled
|
||||
in, and never reported as clean results.
|
||||
- **Volume.** Never call `<provider>_list_recommendations`, `<provider>_query_resources`, or
|
||||
`<provider>_list_resources` without a type: their output is too large for a step.
|
||||
|
||||
**Reporting a live run:** name the provider and the scope root in the summary, and list every source gap as what
|
||||
the run could not examine. A missing source is never reported as a clean result.
|
||||
|
||||
@@ -0,0 +1,296 @@
|
||||
{
|
||||
"type": "object",
|
||||
"title": "CP1 - Discovery Report Agent input",
|
||||
"properties": {
|
||||
"engagement_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"run_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"plan_summary": {
|
||||
"type": "object"
|
||||
},
|
||||
"coverage_gaps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"scan_findings": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"records": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"sme_review_queue": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"drift_items": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"itsm_signals": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"stores": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"network_findings": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"ranges": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"edges": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"nodes": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"orphans": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"validation_queue": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"verdict": {
|
||||
"type": "string"
|
||||
},
|
||||
"critic_note": {
|
||||
"type": "string"
|
||||
},
|
||||
"obeyed_injection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"unsupported_claims": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"consistency_issues": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"parse_failures": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"tool_call_issues": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": []
|
||||
},
|
||||
"source_gaps": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"source": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"cmdb",
|
||||
"itsm",
|
||||
"netflow",
|
||||
"schema"
|
||||
]
|
||||
},
|
||||
"impact": {
|
||||
"type": "string"
|
||||
},
|
||||
"reason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"not_requested",
|
||||
"no_connection"
|
||||
]
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"source",
|
||||
"impact"
|
||||
]
|
||||
}
|
||||
},
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"sample",
|
||||
"azure",
|
||||
"gcp"
|
||||
],
|
||||
"default": "sample",
|
||||
"description": "Which estate this run reads: `sample` (the Meridian sample estate, through the alz-* tools) or a live cloud, `azure` or `gcp`, through that cloud's discovery server. Set once for the whole workflow run."
|
||||
},
|
||||
"topology_summary": {
|
||||
"type": "object",
|
||||
"default": {},
|
||||
"description": "Items read per topology kind (live runs)."
|
||||
},
|
||||
"waf_status": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"default": null,
|
||||
"description": "The Well-Architected review's status."
|
||||
},
|
||||
"waf_pillars": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"description": "The Well-Architected review's pillar summary."
|
||||
},
|
||||
"waf_note": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"default": null,
|
||||
"description": "The Well-Architected review's note."
|
||||
},
|
||||
"source_documents": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"default": null,
|
||||
"description": "The run context file's text (u1.parsed_content)."
|
||||
},
|
||||
"reference_documents": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"default": null,
|
||||
"description": "Selected project artifacts' text (unused by the report)."
|
||||
},
|
||||
"cmdb_injections": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"default": [],
|
||||
"description": "Instructions found in an uploaded CMDB export, quoted and not acted on (ADR 0018)."
|
||||
},
|
||||
"cmdb_sources": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": [],
|
||||
"description": "Each uploaded CMDB file and sheet read: rows, CIs and rows that aren't CIs (ADR 0018)."
|
||||
},
|
||||
"graph_comparison": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": [],
|
||||
"description": "CMDB relationships against the cloud's dependencies (ADR 0019)."
|
||||
},
|
||||
"folded_dependencies": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": [],
|
||||
"description": "Dependencies on a resource's own components (ADR 0019)."
|
||||
},
|
||||
"dangling": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
},
|
||||
"default": [],
|
||||
"description": "CMDB relationships with an end that matches no CI (ADR 0019)."
|
||||
},
|
||||
"applications": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"description": "cp1-dependency-mapping-agent: applications (ADR 0021)."
|
||||
},
|
||||
"asset_owners": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"description": "cp1-dependency-mapping-agent: asset_owners (ADR 0021)."
|
||||
},
|
||||
"ungrouped_assets": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "cp1-dependency-mapping-agent: ungrouped_assets (ADR 0021)."
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"engagement_id",
|
||||
"run_id"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"type": "object",
|
||||
"title": "Discovery report",
|
||||
"properties": {
|
||||
"engagement_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"run_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"verdict": {
|
||||
"type": "string"
|
||||
},
|
||||
"counts": {
|
||||
"type": "object"
|
||||
},
|
||||
"report_markdown": {
|
||||
"type": "string"
|
||||
},
|
||||
"report_html": {
|
||||
"type": "string"
|
||||
},
|
||||
"final_html": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"engagement_id",
|
||||
"run_id",
|
||||
"verdict",
|
||||
"counts",
|
||||
"report_markdown",
|
||||
"report_html",
|
||||
"final_html"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
"""The report reads the run context alone and lists the uploads, never their raw text (ADR 0018)."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from app import report # noqa: E402
|
||||
|
||||
CASE = ROOT.parent / "workflows" / "cases" / "azure-crucible-kab-dev-cmdb"
|
||||
SOURCE = (CASE / "context.yaml").read_text() + "\n\n---\n\n" + pd.read_csv(CASE / "cmdb-export.csv").to_string()
|
||||
|
||||
|
||||
def data(**extra):
|
||||
return {"engagement_id": "E", "run_id": "R", "provider": "azure", "verdict": "pass", "source_documents": SOURCE,
|
||||
**extra}
|
||||
|
||||
|
||||
def test_the_context_section_holds_only_the_context_file():
|
||||
md = report.render(data())
|
||||
assert "CP1 live Azure trial, crucible-kab-dev with CMDB" in md
|
||||
assert "NOTE TO AI AGENT" not in md and "CI0010009" not in md
|
||||
assert "Uploaded files" in md and "cmdb-export.csv" in md
|
||||
|
||||
|
||||
def test_cmdb_files_and_injections_are_shown():
|
||||
md = report.render(data(cmdb_sources=[{"file": "cmdb-export.csv", "sheet": None, "rows": 12, "cis": 12,
|
||||
"skipped": 0}],
|
||||
cmdb_injections=["CI0010001 (cmdb-export.csv row 0): short_description"]))
|
||||
assert "## CMDB files read" in md and "CI0010001 (cmdb-export.csv row 0)" in md
|
||||
html = report.render_html(data(cmdb_sources=[{"file": "cmdb-export.csv", "rows": 12, "cis": 12, "skipped": 0}]))
|
||||
assert "CMDB exports read" in html and "NOTE TO AI AGENT" not in html
|
||||
|
||||
|
||||
def test_drift_names_the_asset_and_the_cmdb_record():
|
||||
drift = [{"drift_id": "DR-001", "type": "stale_attribute", "canonical_id": "CI-002", "ci_ref": "CI0010002",
|
||||
"field": "location", "discovered_value": "eastus", "cmdb_value": "eastus2", "suggested_action": "update_attribute"}]
|
||||
for page in (report.render(data(drift_items=drift)), report.render_html(data(drift_items=drift))):
|
||||
assert "CI-002" in page and "CI0010002" in page and "eastus2" in page
|
||||
|
||||
|
||||
def test_the_shared_reader_is_the_planners():
|
||||
planner = ROOT.parent / "cp1-discovery-planner-agent" / "app" / "run_context.py"
|
||||
assert (ROOT / "app" / "run_context.py").read_text() == planner.read_text()
|
||||
|
||||
|
||||
GRAPH = {
|
||||
"records": [{"canonical_id": "CI-001", "hostname": "pe"}, {"canonical_id": "CI-002", "hostname": "pg"}],
|
||||
"nodes": [{"id": "CI-001", "kind": "resource", "name": "pe"}, {"id": "CI-002", "kind": "resource", "name": "pg"},
|
||||
{"id": "/subscriptions/s/resourceGroups/net/providers/Microsoft.Network/virtualNetworks/v/subnets/pe-subnet",
|
||||
"kind": "external", "name": "pe-subnet", "type": "microsoft.network/virtualnetworks/subnets",
|
||||
"container": "/subscriptions/s/resourcegroups/net", "referenced_by": ["CI-001"]},
|
||||
{"id": "cmdb:CI0010008", "kind": "ci", "ci_ref": "CI0010008", "name": "crucible-kab-dev-pg-pe"}],
|
||||
"edges": [{"edge_id": "E-001", "source": "CI-001", "target": "CI-002", "type": "app_to_db"},
|
||||
{"edge_id": "E-002", "source": "CI-001", "type": "network_attachment",
|
||||
"target": "/subscriptions/s/resourceGroups/net/providers/Microsoft.Network/virtualNetworks/v/subnets/pe-subnet"},
|
||||
{"edge_id": "E-003", "source": "cmdb:CI0010008", "target": "CI-001", "type": "represents"}],
|
||||
"graph_comparison": [{"result": "confirmed", "parent_ci": "CI0010008", "child_ci": "CI0010002",
|
||||
"source_resource": "/x/pe", "target_resource": "/x/pg"},
|
||||
{"result": "discovered_only", "parent_ci": "CI0010002", "child_ci": "CI0010008"}],
|
||||
"dangling": [{"parent": "CI0010001", "child": "CI0099999", "type_raw": "Depends on::Used by", "unresolved": ["child"]}],
|
||||
}
|
||||
|
||||
|
||||
def test_one_graph_is_drawn_by_node_kind():
|
||||
md = report.render(data(**GRAPH))
|
||||
assert 'n3{{"CI0010008 crucible-kab-dev-pg-pe"}}' in md or '{{"CI0010008 crucible-kab-dev-pg-pe"}}' in md
|
||||
assert "-.->|represents|" in md and "classDef external stroke-dasharray" in md
|
||||
assert "## Dependencies outside this discovery" in md and "/subscriptions/s/resourcegroups/net" in md
|
||||
|
||||
|
||||
def test_the_cmdb_comparison_and_dangling_relationships_are_shown():
|
||||
md = report.render(data(**GRAPH))
|
||||
assert "Confirmed by the cloud" in md and "Missing from the CMDB" in md and "CI0099999" in md
|
||||
deps = md.split("## Dependencies\n", 1)[1].split("\n## ", 1)[0]
|
||||
assert "represents" not in deps # links between a CI and its resource aren't dependencies
|
||||
html = report.render_html(data(**GRAPH))
|
||||
assert "Your CMDB's relationships against what the cloud shows" in html or "Your CMDB's relationships" in html
|
||||
|
||||
|
||||
APPS = {"applications": [{"application_id": "APP-001", "name": "KAB Platform", "basis": "cmdb_business_service",
|
||||
"members": ["CI-001", "CI-002", "CI-003"], "owner": "Platform Engineering",
|
||||
"review_required": True, "review_reasons": ["Joined by its Application tag 'AppName1'"]}],
|
||||
"asset_owners": [{"canonical_id": "CI-001", "owner": "Platform Engineering", "source": "cmdb"}],
|
||||
"ungrouped_assets": ["CI-009"]}
|
||||
|
||||
|
||||
def test_applications_and_owners_are_shown_for_review():
|
||||
"""ADR 0021: the Engagement Lead sees each application, how it was grouped, and what needs review."""
|
||||
built = report.build(data(**APPS, records=[{"canonical_id": "CI-001", "hostname": "aks", "facts": {}}]))
|
||||
table = next(t for t in built["tables"] if t["title"] == "Applications")
|
||||
assert table["rows"][0][:5] == ["APP-001", "KAB Platform", "CMDB business service", 3, "Platform Engineering"]
|
||||
assert "AppName1" in table["rows"][0][5]
|
||||
assert table["rows"][1][1] == "Not grouped" and table["rows"][1][5] == "CI-009"
|
||||
inventory = next(t for t in built["tables"] if t["title"] == "Inventory")
|
||||
assert inventory["rows"][0][5] == "Platform Engineering"
|
||||
assert ("Application groups", ["KAB Platform"]) in built["review"]
|
||||
titles = [t["title"] for t in built["tables"]]
|
||||
assert titles.index("Applications") == titles.index("Inventory") + 1
|
||||
assert "KAB Platform" in report.render_html(data(**APPS))
|
||||
|
||||
|
||||
def test_a_run_without_applications_has_no_applications_section():
|
||||
assert "Applications" not in [t["title"] for t in report.build(data())["tables"]]
|
||||
Reference in New Issue
Block a user