- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part hashed (sha256 over canonical JSON), with a manifest and a value scorecard. - Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021): CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them. - Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum narrower than its producer's. - Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic agents; the report and Well-Architected review agents; ADRs 0012-0021. - Workflow cases with value bars and run limits; run evaluation and usage collection scripts; the run context template and the field-by-field review for ADR 0016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
223 lines
15 KiB
Python
223 lines
15 KiB
Python
#!/usr/bin/env python3
|
|
"""Add the `provider` run input and the live cloud tools to the nine individual Discovery agents (ADR 0013).
|
|
|
|
`provider` is `sample` (the Meridian estate through the alz-* tools, unchanged) or a live cloud, `azure` or `gcp`,
|
|
read through the existing azure-discovery and gcp-discovery servers with the connection stored in KAB's vault.
|
|
For each agent this adds the input to schemas/input.json, the live tools it needs to agent.yaml `mcp:`, a
|
|
"Live provider" section to prompt.md (before the role), a minor version and a release note. Idempotent: an agent
|
|
that already has the section is left alone. Run scripts/build_workflow.py afterwards. Standard library only.
|
|
|
|
AWS isn't offered yet: its tools take account_id and region, and no AWS connection is stored.
|
|
|
|
Usage (from the Feature folder): python3 scripts/add_live_provider.py
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import re
|
|
from pathlib import Path
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
DATE = "2026-11-03"
|
|
MARK = "## Live provider"
|
|
PROVIDERS = ("azure", "gcp")
|
|
# The MCP gateway offers servers by display name, and agent.yaml `mcp:` must use it.
|
|
GATEWAY_NAMES = {"azure": "Azure Discovery", "gcp": "Google Cloud Discovery"}
|
|
PROVIDER_INPUT = {"type": "string", "enum": ["sample", *PROVIDERS], "default": "sample",
|
|
"description": "Which estate this run reads: `sample` (the Meridian sample estate, through the alz-* "
|
|
"tools) or a live cloud, `azure` or `gcp`, through that cloud's discovery server. Set "
|
|
"once for the whole workflow run."}
|
|
LIST = ("list_servers", "list_workloads", "list_kubernetes_clusters", "list_data_stores")
|
|
|
|
KINDS = """| Tool | `azure` kinds | `gcp` kinds |
|
|
|---|---|---|
|
|
| `list_accounts` | `subscriptions`, `resource_groups` | `projects` |
|
|
| `list_servers` | `virtual_machines`, `scale_sets` | `instances` |
|
|
| `list_workloads` | `app_service`, `container_apps`, `container_instances` | `cloud_run`, `cloud_functions`, `app_engine` |
|
|
| `list_kubernetes_clusters` | (takes no kind) | `clusters` |
|
|
| `list_data_stores` | `sql`, `cosmos_db`, `storage_accounts`, `postgresql`, `mysql`, `redis` | `cloud_sql`, `firestore`, `cloud_storage` |
|
|
| `get_network_topology` | `virtual_networks`, `route_tables`, `network_security_groups`, `gateways`, `load_balancers`, `private_endpoints`, `public_ips` | `networks`, `subnetworks`, `routes`, `firewall_rules`, `load_balancers`, `gateways` |
|
|
|
|
Call a tool once for **every** kind its row lists, with `page_size` 50, and follow `next_page_token` to the end. A
|
|
tool called without `kind` returns its first kind only."""
|
|
|
|
COMMON = """`provider` is set once for the whole run. `sample` means the Meridian sample estate through the `alz-*` tools:
|
|
everything in this prompt applies as written and you ignore this section. `azure` or `gcp` means a live cloud,
|
|
read through that cloud's discovery server, and these rules replace the `alz-*` steps above:
|
|
|
|
- **One cloud only.** Use only the tools whose names start with `{provider}_`. Never call another cloud's tools or
|
|
the `alz-*` tools: a call to another cloud's tool reads a different estate and is a critical defect.
|
|
- **Identifiers.** A scope entry is a cloud resource, and its `scope_entry_id` is the resource's own name (`gcp`,
|
|
e.g. `//compute.googleapis.com/projects/p/zones/z/instances/i`) or id (`azure`, e.g.
|
|
`/subscriptions/s/resourceGroups/g/providers/...`) exactly as a tool returned it. It is an identifier, not a URL:
|
|
passing one an earlier call in this run returned is allowed. Never pass anything else.
|
|
- **Missing sources.** A live run has no CMDB, ITSM, flow or schema source. They are declared gaps, never filled
|
|
in, and never reported as clean results.
|
|
- **Volume.** Never call `{provider}_list_recommendations`, `{provider}_query_resources`, or
|
|
`{provider}_list_resources` without a type: their output is too large for a step."""
|
|
|
|
AGENTS = {
|
|
"cp1-discovery-planner-agent": dict(tools=("list_accounts", *LIST), kinds=True, text="""
|
|
**Planning a live run:**
|
|
|
|
1. Call `{provider}_list_accounts` (`projects` / `subscriptions`), then each of `{provider}_list_servers`,
|
|
`{provider}_list_workloads`, `{provider}_list_kubernetes_clusters` and `{provider}_list_data_stores` for every
|
|
kind. Leave out disks, scale-set instances and node pools: they belong to the server or cluster they're part of.
|
|
2. The items `{provider}_list_accounts` returns (projects, folders, subscriptions, resource groups) are the
|
|
estate's structure and confirm the scope root. They are never scope entries: plan no task for one. Every item the
|
|
four other list tools return is one scope entry. Its site is its region: a `gcp` zone such as `europe-west2-a` is the
|
|
region `europe-west2`; an `azure` location is used as it is.
|
|
3. Tasks: one `scan_target` per scope entry (`assigned_agent` `ALZ-DSC-02`, `mcp_tool`
|
|
`{provider}-discovery.{provider}_get_resource_metadata`, `target_ref` the scope entry id, batches of 10,
|
|
`declared_rps` 1); one `classify_data` per data store (ALZ-DSC-05); one `collect_network` with `target_ref`
|
|
`scope` (ALZ-DSC-07); one `trace_dependencies` (ALZ-DSC-06). No ingest or reconcile task.
|
|
4. `source_gaps`: `cmdb`, `itsm`, `netflow` and `schema`, each with `reason` `no_connection` and its impact
|
|
(no CMDB: owners, cost and criticality are unknown; no ITSM: change and incident history are unknown; no
|
|
flows: traffic, bottlenecks and unknown sources can't be shown; no schemas: data can't be classified beyond
|
|
store metadata).
|
|
5. A resource the list tools return but that can't be scanned (no metadata access) is a coverage gap with
|
|
`no_read_only_credential`. Nothing is dropped silently."""),
|
|
"cp1-estate-scan-executor-agent": dict(tools=("get_resource_metadata",), kinds=False, text="""
|
|
**Scanning a live run:** for every `scan_target` task, call `{provider}_get_resource_metadata` once, with
|
|
`resource_name` (`gcp`) or `resource_id` (`azure`) set to the task's `target_ref`, and `scope_entry_id` set to the
|
|
same value. Fill the finding from that result only:
|
|
|
|
- `hostname`: the resource's display name; `os`: the image or OS fields, if the metadata has them, else `""`;
|
|
`services`: engine, runtime and version fields, if any.
|
|
- `dependencies`: other scope entries this resource references (its network, subnetwork, attached disks' owner,
|
|
the database it connects to), by their scope entry id. Never a service account or a person.
|
|
- `migration_risks`: only what the metadata states, e.g. a public address, an end-of-life version, no backups.
|
|
- `provider_attributes`: the metadata's own values, copied verbatim where present: resource type, region, zone,
|
|
machine type or size, state, OS image, disk names, network and subnetwork, public address (yes or no), labels or
|
|
tags, deletion protection, Secure Boot, encryption, backup and versioning settings. Leave out what isn't there.
|
|
|
|
A refused or failed call is a finding with empty fields and the refusal quoted in `migration_risks`."""),
|
|
"cp1-asset-inventory-normalisation-agent": dict(tools=(), kinds=False, text="""
|
|
**Normalising a live run:** there is no application list and no CMDB, so call no tool. Build one record per
|
|
finding:
|
|
|
|
- `canonical_id` `CI-001`, `CI-002`, … in the findings' order; `scope_entry_id` the finding's.
|
|
- `asset_class`: VMs and instances `server`; databases `database`; buckets and storage accounts `storage_array`;
|
|
everything else, clusters included, `cloud_resource`.
|
|
- `environment`: from a label or tag named `env` or `environment`, if present, else `unknown`.
|
|
- `facts`: only labels or tags that name an owner, cost centre, criticality or business service, copied verbatim.
|
|
Leave every other fact out; never infer one.
|
|
- `provider_attributes`: the finding's, unchanged; `facts.site` its region (a `gcp` zone such as
|
|
`europe-west2-b` is the region `europe-west2`).
|
|
- `sources`: one, `{ source_type: 'provider_api', source_ref: <scope entry id>, evidence_ref: <the finding's> }`;
|
|
`match_basis` `['cloud_resource_id']`; `confidence` 0.85 (one source)."""),
|
|
"cp1-cmdb-itsm-reconciliation-agent": dict(tools=(), kinds=False, text="""
|
|
**A live run** has `cmdb` and `itsm` in `source_gaps`: call no tool, and return empty `drift_items` and
|
|
`itsm_signals` with every summary count 0. Never report an asset as missing from a CMDB that wasn't read."""),
|
|
"cp1-data-classification-agent": dict(tools=("list_data_stores",), kinds=True, text="""
|
|
**Classifying a live run:** call `{provider}_list_data_stores` for every kind in **Kinds**. For each data store
|
|
that is a scope entry in `records`: `store_name` its name; `highest_class` `unclassified`; every class count 0;
|
|
`column_tags` empty; `reason` "Metadata only: no schema source in this run". Set `requires_disposition_controls`
|
|
true only when a label or tag marks the store as holding personal or card data, and say so in `reason`."""),
|
|
"cp1-network-discovery-assessment-agent": dict(tools=("get_network_topology",), kinds=True, text="""
|
|
**Assessing a live run's network:** call `{provider}_get_network_topology` for every kind in **Kinds**.
|
|
|
|
- `ranges`: each subnetwork's CIDR, with `site` its region and `source_ref` its resource name or id; mark overlaps.
|
|
- `findings`: `address_overlap` for overlapping CIDRs; `internet_exposure` (severity `high`) for each firewall or
|
|
NSG rule allowing `0.0.0.0/0`, `::/0` or `*` inbound to 22, 3389, 21, 1433, 3306 or 5432; `single_point_of_failure`
|
|
only where the topology shows one path and nothing else.
|
|
- `flows`: empty. `netflow` is a source gap, so never report a bottleneck or an unknown source.
|
|
- `topology_summary`: how many items of each kind you read, e.g. `{"networks": 1, "subnetworks": 1, "routes": 2,
|
|
"firewall_rules": 0}`, every kind in **Kinds** included, with 0 for none."""),
|
|
"cp1-dependency-mapping-agent": dict(tools=(), kinds=False, text="""
|
|
**Mapping a live run:** call no tool. For every finding, emit one edge per entry in its `dependencies`:
|
|
|
|
- `source`: the `canonical_id` of that finding's record.
|
|
- `target`: the dependency's record `canonical_id` when it is a scope entry; otherwise the referenced resource's
|
|
name or id exactly as the finding gives it. A network, subnetwork or disk the scope doesn't list is still a
|
|
dependency: never drop one because its target isn't in scope.
|
|
- `type`: `network_attachment` for a network or subnetwork, `hosted_on` for a disk, `unclassified` otherwise;
|
|
`evidence` `[{ kind: 'config', ref: <the finding's evidence_ref> }]`; `confidence` 0.6; `validation`
|
|
`sme_review_recommended`.
|
|
|
|
`nodes`: one per record, with `fan_in` and `fan_out` counted from these edges; a record something depends on has
|
|
`retire_constraint` `cannot_retire_without_mitigation`. `orphans`: only records with no edge in either direction.
|
|
Configured links only: say so, because traffic isn't known."""),
|
|
"cp1-discovery-critic-agent": dict(tools=LIST, kinds=True, text="""
|
|
**Validating a live run:**
|
|
|
|
- The authoritative scope is the cloud's own listing. Call `{provider}_list_servers`, `{provider}_list_workloads`,
|
|
`{provider}_list_kubernetes_clusters` and `{provider}_list_data_stores` for every kind, and measure coverage
|
|
against that, never against the findings.
|
|
- `alz-scope` and `alz-evidence` aren't used in a live run: leave `tool_call_issues` empty, and put `call_log` and
|
|
`guard_hits` in `not_checked`.
|
|
- Every entry in a finding's `dependencies` must appear in `edges`, from that finding's record. A missing one is a
|
|
`consistency_issues` entry and fails the run: the scan found a dependency the map dropped.
|
|
- A scope entry that is a project, folder, subscription or resource group is a `consistency_issues` entry and fails
|
|
the run: those are the estate's structure, not assets.
|
|
- A finding, record or edge whose identifier belongs to the other cloud (a `gcp` name begins `//`; an `azure` id
|
|
begins `/subscriptions/`) is a `consistency_issues` entry and fails the run."""),
|
|
"cp1-discovery-report-agent": dict(tools=(), kinds=False, text="""
|
|
**Reporting a live run:** name the provider and the scope root in the summary, and list every source gap as what
|
|
the run could not examine. A missing source is never reported as a clean result."""),
|
|
}
|
|
|
|
|
|
def bump(version: str) -> str:
|
|
major, minor, _ = (int(x) for x in version.split("."))
|
|
return f"{major}.{minor + 1}.0"
|
|
|
|
|
|
def add_tools(yaml_text: str, tools) -> str:
|
|
if not tools:
|
|
return yaml_text
|
|
block = "".join(f' "{GATEWAY_NAMES[p]}":\n' + "".join(f' - "{p}_{t}"\n' for t in tools) for p in PROVIDERS)
|
|
if re.search(r"^mcp:\n", yaml_text, re.M):
|
|
return re.sub(r"^(mcp:\n(?: .*\n| .*\n)+)", lambda m: m.group(1) + block, yaml_text, count=1, flags=re.M)
|
|
return re.sub(r"^skills:", "mcp:\n" + block + "skills:", yaml_text, count=1, flags=re.M)
|
|
|
|
|
|
def main() -> int:
|
|
for agent, spec in AGENTS.items():
|
|
d = ROOT / agent
|
|
prompt = (d / "prompt.md").read_text()
|
|
if MARK in prompt:
|
|
print(f"{agent}: already has the live provider, unchanged")
|
|
continue
|
|
manifest = (d / "agent.yaml").read_text()
|
|
old = re.search(r'^version: "([\d.]+)"', manifest, re.M).group(1)
|
|
new = bump(old)
|
|
manifest = manifest.replace(f'version: "{old}"', f'version: "{new}"', 1)
|
|
(d / "agent.yaml").write_text(add_tools(manifest, spec["tools"]))
|
|
|
|
schema = json.loads((d / "schemas" / "input.json").read_text())
|
|
schema["properties"]["provider"] = PROVIDER_INPUT
|
|
(d / "schemas" / "input.json").write_text(json.dumps(schema, indent=2) + "\n")
|
|
|
|
if agent == "cp1-network-discovery-assessment-agent":
|
|
out = json.loads((d / "schemas" / "output.json").read_text())
|
|
types = out["properties"]["findings"]["items"]["properties"]["type"]["enum"]
|
|
if "internet_exposure" not in types:
|
|
types.append("internet_exposure")
|
|
(d / "schemas" / "output.json").write_text(json.dumps(out, indent=2) + "\n")
|
|
|
|
live_tools = "".join(f"| `{p}-discovery` | `{p}_{t}` | read |\n" for p in PROVIDERS for t in spec["tools"])
|
|
if live_tools:
|
|
prompt = re.sub(r"(## MCP tools\n\n\| Server \| Tool \| Access \|\n\|---\|---\|---\|\n(?:\|.*\n)*)",
|
|
lambda m: m.group(1) + live_tools, prompt, count=1)
|
|
section = f"{MARK}\n\n{COMMON}\n" + (f"\n**Kinds:**\n\n{KINDS}\n" if spec["kinds"] else "") + spec["text"] + "\n\n"
|
|
at = prompt.find("## Role: ")
|
|
prompt = prompt[:at] + section + prompt[at:] if at >= 0 else prompt.rstrip() + "\n\n" + section
|
|
# The runtime reads {name} in a prompt as a session variable: placeholders are written <name>.
|
|
prompt = re.sub(r"\{([A-Za-z_][A-Za-z0-9_]*)\}", r"<\1>", prompt)
|
|
(d / "prompt.md").write_text(prompt)
|
|
|
|
notes = (d / "RELEASE_NOTES.md").read_text()
|
|
entry = (f"## [{new}] - {DATE}\n\n### Added\n\n- The `provider` input: `sample` (unchanged) or a live cloud, "
|
|
f"`azure` or `gcp`, read through the existing discovery servers (ADR 0013)."
|
|
+ (f" Live tools: {', '.join(f'`{{provider}}_{t}`' for t in spec['tools'])}." if spec["tools"] else "")
|
|
+ "\n\n")
|
|
(d / "RELEASE_NOTES.md").write_text(entry + notes)
|
|
print(f"{agent}: {old} -> {new}, {len(spec['tools']) * len(PROVIDERS)} live tools")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|