Files
cp1-agents/individual-agents/ALZ-F01: Discovery Workflow Agent/docs/adr/0018.md
Jonathan Boniface 9e503885cc ALZ-F01 Discovery: package step, value scorecard, application grouping, cases
- New cp1-discovery-package-agent (ADR 0020): one package per reviewed run, every part
  hashed (sha256 over canonical JSON), with a manifest and a value scorecard.
- Applications grouped CMDB -> tag -> resource group with a judged model fallback (ADR 0021):
  CMDB 0.7.0 carries ownership, dependency mapping 0.6.0 groups, report 0.9.0 shows them.
- Critic 0.6.1 accepts the producers' enums; build_workflow.py refuses a consumer enum
  narrower than its producer's.
- Code hooks, sub-agents and tests for the inventory, CMDB, dependency, planner and critic
  agents; the report and Well-Architected review agents; ADRs 0012-0021.
- Workflow cases with value bars and run limits; run evaluation and usage collection scripts;
  the run context template and the field-by-field review for ADR 0016.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:13:25 +00:00

5.3 KiB
Raw Blame History

0018. Uploaded CMDB exports are normalised by the model to our CI schema, and judged and reconciled in code

Status Accepted
Date 2026-11-05
Decided by Product owner, 2026-11-05: "the model should ingest the data and normalise it to the data schema that we own and judge". Plan and decisions D1–D4: engineering_hub_responsiblities/reports/cp1/cmdb-upload-ingestion-plan-2026-11-05.md
Requirements changed ALZ-DSC-04 (ADO 3214454): a live run reads CMDB exports uploaded with the run. ALZ-DSC-01 (ADO 3214451): an uploaded CMDB is a source, not a gap

Context

  • A live run never read a CMDB. cmdb was always a source gap, and the reconciliation step returned nothing.
  • A second uploaded file broke the run. The upload step joins files with \n\n---\n\n. The planner, inventory and report parsed the whole text as one YAML document, failed, and carried on as if no run context had been given.
  • Exports differ in layout. CMDB exports differ by customer and tool, so no column can be fixed in code.

Decision

  • Uploads are listed in the run context. The context's uploads: list names the files uploaded after it, in order, each with a source (e.g. cmdb). kab_run_workflow.py uploads them. The shared app/run_context.py (identical in the planner, inventory, reconciliation and report agents) splits the text into the context and each listed file. It cuts a CSV, or each XLSX sheet, into a header line and one line per row. It knows no column.

  • The model normalises; code judges. The normalise_cmdb_batch sub-agent maps each batch of rows (25 by default, each batch with its header) to schemas/cmdb_ci.json. Every string is copied verbatim; environment and lifecycle are the only judgments; any other column can be kept in other_attributes. Code rejects a reply unless:

    • every row has exactly one result (or a reasoned skip);
    • every value is found in its row;
    • every key is a header column;
    • every judgment is an allowed value.

    A batch gets at most three tries (ADR 0015's rule). One that fails all three, or an upload the platform couldn't parse, fails the step.

  • Injections: a row is flagged when the model or a code pattern sees an instruction. The quoted text goes to the critic (cmdb_injections) and to the report.

  • Reconciliation is code (app/reconcile.py), against the step's own scope listing (app/scope.py, the planner's rules) and the inventory:

    • match: by cloud resource id, otherwise by name when exactly one listed resource has it;
    • stale attributes:
      • environment, compared at enum level (D1);
      • location, compared with the record's site;
      • a retired CI whose resource exists (investigate);
      • owner and business service, when the record carries them;
    • ghost: an unmatched, not-retired CI whose id lies in a scope container the listing shows (D2). It's investigate outside context.scope.regions, in exclude, or with an unknown lifecycle, and otherwise retire_ci. A CI with no cloud id is unplaced (D3), and one in another container is out_of_scope. Neither is drift.
    • orphan: an inventory record no CI matched (add_ci). Folded components are never orphans.
  • The planner adds one ingest_cmdb task per uploaded CMDB file, and drops the cmdb gap when include_cmdb is yes and a file is uploaded.

Consequences

  • New layouts need no code change. A new export layout, whether a ServiceNow CSV or an asset register XLSX with other column names, is handled by the model's mapping. Code checks that mapping on every row.
  • The model never decides drift. A manipulated normaliser can only return values that are in the row.
  • The step makes model calls (one per batch) on a live run with a CMDB. Without one, it makes none.
  • Subscription scope is cautious. At subscription scope, a CI in a resource group with no listed resources is out_of_scope, not a ghost. Returning the connection's scope root from the existing discovery servers would make this exact; that is left until it's needed.
  • Planner and step listings can differ. The reconciliation step lists the scope again, so a resource created between the planner's listing and this one is matched, not a ghost.

Changes after the first live run (2026-11-05)

Run exec_a0a3c486ec9b (case azure-crucible-kab-dev-cmdb) passed 13 of 19 checks. Every planted CMDB result was found except the environment difference.

  • env tags were redacted by the discovery servers' shared _scrub (env is a secret key, for a container's environment variables). The inventory then took env: [redacted] over Environment: dev, and 8 of 10 records had environment unknown, so the planted environment difference couldn't be found. A single value under env is now kept; a list or map under it is still redacted (azure-discovery 0.1.6, gcp-discovery 0.1.4, aws-discovery 0.1.2).
  • Orphans name their resource (field resource, discovered_value the resource id). Before, they named only the inventory's canonical id (reconciliation 0.5.1).
  • The critic no longer raises "drift with no CMDB source" when source_gaps has no cmdb (critic 0.5.2).
  • The report's drift table shows both the asset and the CMDB record (report 0.7.2).