The evaluation (both cases three times: report, results, spread, each run's Deliverables and lesson candidates) and the hand-off through the Orchestration engine (report, the document of links, both agents' kept files and records). Benchmark cases only, all fictional; no machine paths, and the links carry no signatures. Other runs stay ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
362 lines
17 KiB
JSON
362 lines
17 KiB
JSON
{
|
|
"dry_run": {
|
|
"verdict": "pass",
|
|
"evidence": "44 of 44 scenario walk-throughs pass."
|
|
},
|
|
"checks": [
|
|
{
|
|
"evidence": "Capability C-7 handles declarative state and configuration management, covering all required items. Platform Engineer (R-4) is responsible during business hours.",
|
|
"verdict": "pass",
|
|
"check": "OC-1"
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"check": "OC-2",
|
|
"evidence": "Capability C-9 manages service continuity and DR failover rehearsals, covering items S-1, S-7, and S-9. Platform Engineer (R-4) is responsible."
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"check": "OC-3",
|
|
"evidence": "Capability C-7 manages configuration state and audits, ensuring isolation configurations are maintained. Platform Engineer (R-4) is responsible."
|
|
},
|
|
{
|
|
"evidence": "Capability C-7 audits configuration state for workload APIs, and Platform Engineer (R-4) is responsible.",
|
|
"check": "OC-4",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"check": "OC-5",
|
|
"evidence": "Capability C-10 reviews capacity and performance metrics, including the dedicated hybrid network interconnect. Platform Engineer (R-4) is responsible."
|
|
},
|
|
{
|
|
"check": "OC-6",
|
|
"evidence": "Capability C-13 provides security monitoring and response, analyzing egress traffic inspection data 24x7 via Security Analyst (R-3).",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"evidence": "Capability C-9 conducts bi-monthly DR failover rehearsals to ensure the 4-hour RTO is achievable. Platform Engineer (R-4) is responsible.",
|
|
"verdict": "pass",
|
|
"check": "OC-7"
|
|
},
|
|
{
|
|
"evidence": "AF-3: The solution lacks an automated backup service or vault to ensure state and configuration achieve the maximum data loss window (RPO) of 1 hour.",
|
|
"check": "OC-8",
|
|
"verdict": "waits_on_finding",
|
|
"finding": "AF-3"
|
|
},
|
|
{
|
|
"check": "OC-9",
|
|
"evidence": "Capability C-11 covers access management and policy reviews for corporate identity gateways. Platform Engineer (R-4) is responsible.",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"check": "OC-10",
|
|
"evidence": "Capability C-20 conducts supplier management and service reviews, incorporating DORA risk assessments. Platform Reliability Lead (R-1) is responsible.",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"evidence": "Capability C-21 continuously collates regulatory evidence for PCI-DSS v4.0. Compliance Officer (R-6) is responsible.",
|
|
"check": "OC-11",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"evidence": "Capability C-11 governs access management for the cryptographic key management service (S-7). Platform Engineer (R-4) is responsible.",
|
|
"check": "OC-12",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"check": "OC-13",
|
|
"verdict": "pass",
|
|
"evidence": "Capability C-13 provides 24x7 security monitoring and response for audit logs (S-8). Security Analyst (R-3) is responsible."
|
|
},
|
|
{
|
|
"evidence": "Capability C-15 aggregates and reports on service level objectives, including availability targets. Platform SRE (R-2) is responsible.",
|
|
"check": "OC-14",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"evidence": "Capability C-21 collates regulatory and compliance evidence, including carbon-free energy metrics. Compliance Officer (R-6) is responsible.",
|
|
"check": "OC-15",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"evidence": "Capability C-16 manages the platform backlog and adoption roadmap to confirm workload ingestion readiness. Product Manager (R-7) is responsible.",
|
|
"check": "OC-16"
|
|
},
|
|
{
|
|
"evidence": "Capability C-1 handles 24x7 incident management, and Platform SRE (R-2) provides 24x7 coverage to log and respond to incidents.",
|
|
"check": "OC-17",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"evidence": "Capability C-5 manages change enablement and pull request approvals. Platform Engineer (R-4) is responsible.",
|
|
"verdict": "pass",
|
|
"check": "OC-18"
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"check": "OC-19",
|
|
"evidence": "Capability C-10 reviews capacity and performance for network virtual appliances. Platform Engineer (R-4) is responsible."
|
|
},
|
|
{
|
|
"check": "OC-20",
|
|
"evidence": "AF-2: The solution lacks automated budget anomaly alerts out-of-the-box to track the minimal pilot light footprint of the warm standby region.",
|
|
"verdict": "waits_on_finding",
|
|
"finding": "AF-2"
|
|
},
|
|
{
|
|
"check": "OC-21",
|
|
"verdict": "pass",
|
|
"evidence": "Capability C-9 conducts service continuity failovers, managing the DR infrastructure automation. Platform Engineer (R-4) is responsible."
|
|
},
|
|
{
|
|
"evidence": "Role R-1 (Platform Reliability Lead) is part of the Managed Service Provider party, fulfilling the agreed service-provider framework.",
|
|
"verdict": "pass",
|
|
"check": "OC-22"
|
|
},
|
|
{
|
|
"verdict": "pass",
|
|
"check": "OC-23",
|
|
"evidence": "Role R-1 is located in GB, which is an authorized delivery country."
|
|
},
|
|
{
|
|
"check": "OC-24",
|
|
"verdict": "pass",
|
|
"evidence": "Capability C-2 performs continuous monitoring and alerting using the integrated operations suite. Platform SRE (R-2) is responsible 24x7."
|
|
},
|
|
{
|
|
"evidence": "Capability C-21 continuously collates audit evidence across DORA, ISO 27001, and PCI-DSS v4.0. Compliance Officer (R-6) is responsible.",
|
|
"check": "OC-25",
|
|
"verdict": "pass"
|
|
},
|
|
{
|
|
"check": "OC-26",
|
|
"evidence": "AF-1: The solution lacks a secure remote access mechanism (such as a bastion host or Identity-Aware Proxy) as a way in for responders to the Centralised Egress Traffic Inspection Gateway.",
|
|
"verdict": "waits_on_finding",
|
|
"finding": "AF-1"
|
|
},
|
|
{
|
|
"check": "SC-fault-S-1",
|
|
"verdict": "pass",
|
|
"evidence": "Multi-Tier Environment Hierarchy & Isolation Boundary (S-1) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-1",
|
|
"verdict": "pass",
|
|
"evidence": "Multi-Tier Environment Hierarchy & Isolation Boundary (S-1) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-2",
|
|
"verdict": "pass",
|
|
"evidence": "Dedicated Hybrid Network Interconnect Gateway (S-2) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-2",
|
|
"verdict": "pass",
|
|
"evidence": "Dedicated Hybrid Network Interconnect Gateway (S-2) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-3",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Egress Traffic Inspection Gateway (S-3) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-3",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Egress Traffic Inspection Gateway (S-3) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-4",
|
|
"verdict": "pass",
|
|
"evidence": "Transit Network Fabric & Routing Controller (S-4) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-4",
|
|
"verdict": "pass",
|
|
"evidence": "Transit Network Fabric & Routing Controller (S-4) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-5",
|
|
"verdict": "pass",
|
|
"evidence": "Corporate Identity Federation Gateway (S-5) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-5",
|
|
"verdict": "pass",
|
|
"evidence": "Corporate Identity Federation Gateway (S-5) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-6",
|
|
"verdict": "pass",
|
|
"evidence": "Automated Delivery Pipeline Identity Broker (S-6) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-6",
|
|
"verdict": "pass",
|
|
"evidence": "Automated Delivery Pipeline Identity Broker (S-6) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-7",
|
|
"verdict": "pass",
|
|
"evidence": "Cryptographic Key Management & Rotation Service (S-7) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-7",
|
|
"verdict": "pass",
|
|
"evidence": "Cryptographic Key Management & Rotation Service (S-7) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-8",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Immutable Audit Log Repository (S-8) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-8",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Immutable Audit Log Repository (S-8) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-9",
|
|
"verdict": "pass",
|
|
"evidence": "Platform Compliance & Governance Guardrail Controller (S-9) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-9",
|
|
"verdict": "pass",
|
|
"evidence": "Platform Compliance & Governance Guardrail Controller (S-9) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-10",
|
|
"verdict": "pass",
|
|
"evidence": "Declarative Infrastructure Provisioning & State Engine (S-10) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-10",
|
|
"verdict": "pass",
|
|
"evidence": "Declarative Infrastructure Provisioning & State Engine (S-10) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-fault-S-11",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Operational Telemetry & Alerting Engine (S-11) fails at an hour faults are responded to (24x7). Platform SRE responds under C-1, and Platform Reliability Lead is answerable."
|
|
},
|
|
{
|
|
"check": "SC-change-S-11",
|
|
"verdict": "pass",
|
|
"evidence": "Centralised Operational Telemetry & Alerting Engine (S-11) must be changed. Platform Engineer makes the change under C-5."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-1",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-1 must be shown to hold: Customer data and payment processing remain within United Kingdom and European Union legal jurisdictions. Check OC-1 on C-7 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-2",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-2 must be shown to hold: Warm standby environment operates with reduced capacity until activated during a disaster recovery failover. Check OC-2 on C-9 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-3",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-3 must be shown to hold: PCI-regulated production environment is isolated from non-PCI scopes according to PCI-DSS v4.0 network segmentation requirements. Check OC-3 on C-7 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-4",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-4 must be shown to hold: Workload APIs and payment services are accessible exclusively through private corporate network paths. Check OC-4 on C-7 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-5",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-5 must be shown to hold: Hybrid connection maintains low-latency, SLA-backed private data transmission between on-premises and Google Cloud. Check OC-5 on C-10 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-6",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-6 must be shown to hold: Centralised inspection appliances perform next-generation stateful packet inspection, URL filtering, and threat detection on outbound internet sessions. Check OC-6 on C-13 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-7",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-7 must be shown to hold: Platform foundation failover to europe-west1 can be completed within 4 hours during a primary region outage. Check OC-7 on C-9 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-8",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-8 must be shown to hold: Configuration drift and backup state achieve a maximum data loss window (RPO) of 1 hour. Check OC-8 on C-7 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-9",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-9 must be shown to hold: Corporate user lifecycle management and MFA enforcement are governed centrally by Entra ID conditional access policies. Check OC-9 on C-11 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-10",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-10 must be shown to hold: Platform governance and resilience architecture comply with DORA regulatory testing and third-party risk management frameworks. Check OC-10 on C-20 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-11",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-11 must be shown to hold: Production cardholder data environment satisfies PCI-DSS v4.0 isolation and security control requirements. Check OC-11 on C-21 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-12",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-12 must be shown to hold: Confidential payment data is secured against unauthorized platform-level access or data exposure. Check OC-12 on C-11 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-13",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-13 must be shown to hold: Audit logs cannot be modified, deleted, or shortened by any identity or administrator prior to retention expiration. Check OC-13 on C-13 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-14",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-14 must be shown to hold: Monthly availability of the core platform landing zone and network routing meets or exceeds 99.9%. Check OC-14 on C-15 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-15",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-15 must be shown to hold: Primary and secondary regions leverage Google Cloud data centres with published carbon-free energy (CFE) performance metrics. Check OC-15 on C-21 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-16",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-16 must be shown to hold: Platform foundation delivery and workload ingestion readiness are achieved within 10 weeks of project commencement. Check OC-16 on C-16 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-17",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-17 must be shown to hold: Central platform team operates 24/7 support for infrastructure incidents and availability alerts. Check OC-17 on C-1 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-18",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-18 must be shown to hold: Pull request workflows require automated CI/CD linting, validation, and peer approval prior to merge and deployment. Check OC-18 on C-5 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-19",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-19 must be shown to hold: NVA instances use machine families that support high bandwidth networking. Check OC-19 on C-10 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-20",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-20 must be shown to hold: The warm standby environment operates at a minimal pilot light footprint during normal operations. Check OC-20 on C-14 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-condition-K-21",
|
|
"verdict": "pass",
|
|
"evidence": "Condition K-21 must be shown to hold: Automated disaster recovery scripts scale the standby environment to full capacity only during a failover event. Check OC-21 on C-9 keeps it true."
|
|
},
|
|
{
|
|
"check": "SC-party-P-1",
|
|
"verdict": "pass",
|
|
"evidence": "Managed Service Provider cannot be reached. Every Capability it is answerable for has a Role in another Party to turn to."
|
|
}
|
|
],
|
|
"proposed_additions": []
|
|
} |