docs: publish the runs of 2026-10-21
The evaluation (both cases three times: report, results, spread, each run's Deliverables and lesson candidates) and the hand-off through the Orchestration engine (report, the document of links, both agents' kept files and records). Benchmark cases only, all fictional; no machine paths, and the links carry no signatures. Other runs stay ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,282 @@
|
||||
schema_version: 1
|
||||
metadata:
|
||||
customer_case: case-08-landing-zone
|
||||
engagement_reference: ask
|
||||
client_name: Example Payments UK
|
||||
author: Evaluation
|
||||
revision: 1.0.0
|
||||
status: UNDER_REVIEW
|
||||
engagement:
|
||||
kind: platform_foundation
|
||||
intent:
|
||||
statement: 'We are a UK-regulated payment platform expanding into Google Cloud.
|
||||
We need an
|
||||
|
||||
enterprise-grade landing zone ready in under 10 weeks to support our core payments
|
||||
|
||||
engine. All payment processing must comply with DORA and PCI-DSS v4.0. We have
|
||||
|
||||
existing on-premises systems that require private, low-latency connectivity over
|
||||
a
|
||||
|
||||
10Gbps dedicated interconnect. We use Entra ID for corporate single sign-on and
|
||||
|
||||
GitHub Actions for our delivery pipelines. Security requires that all egress
|
||||
|
||||
traffic flows through centralised inspection appliances before touching the public
|
||||
|
||||
internet, and customer data must be encrypted with keys we control.
|
||||
|
||||
'
|
||||
requirements:
|
||||
- id: REQ-01
|
||||
category: Timeline
|
||||
statement: Platform foundation must be ready for workload ingestion within 10
|
||||
weeks.
|
||||
- id: REQ-02
|
||||
category: Compliance
|
||||
statement: Must achieve DORA resiliency alignment and PCI-DSS v4.0 perimeter isolation.
|
||||
- id: REQ-03
|
||||
category: Networking
|
||||
statement: 10Gbps on-premises hybrid connectivity; centralised inspection appliances
|
||||
for egress.
|
||||
- id: REQ-04
|
||||
category: Identity and security
|
||||
statement: Workload identity federation with GitHub Actions; Entra ID single sign-on;
|
||||
customer-managed encryption keys.
|
||||
exclusions:
|
||||
- Application container deployment; workloads are managed by application squads.
|
||||
- An active-active secondary region in phase 1; warm standby is acceptable.
|
||||
success_criteria:
|
||||
- The platform foundation is provisioned within 10 weeks, compliant with DORA and
|
||||
PCI-DSS v4.0, providing 10Gbps dedicated interconnect hybrid connectivity, centralised
|
||||
egress inspection, and customer-managed encryption keys.
|
||||
estate:
|
||||
cloud_provider: gcp
|
||||
mode: greenfield
|
||||
regions:
|
||||
primary: europe-west2
|
||||
secondary: europe-west1
|
||||
resiliency_tier: warm_standby
|
||||
data_residency: United Kingdom and European Union
|
||||
existing_systems: On-premises payment systems that stay in place and connect over
|
||||
the interconnect.
|
||||
workload:
|
||||
users:
|
||||
who: not_applicable
|
||||
count: not_applicable
|
||||
peak_concurrent: not_applicable
|
||||
availability:
|
||||
target: 99.9%
|
||||
hours: always
|
||||
recovery_time: 4 hours
|
||||
recovery_point: 1 hour
|
||||
compliance:
|
||||
industry: financial_services
|
||||
frameworks:
|
||||
- DORA
|
||||
- ISO27001
|
||||
- PCI_DSS_v4
|
||||
data_classification: confidential
|
||||
encryption:
|
||||
type: cmek
|
||||
key_rotation_days: 90
|
||||
audit_logging:
|
||||
retention_days: 365
|
||||
immutable_retention_policy: true
|
||||
networking:
|
||||
exposure: internal_only
|
||||
topology: shared_vpc_hub_and_spoke
|
||||
hybrid_connectivity:
|
||||
enabled: true
|
||||
type: cloud_interconnect
|
||||
bandwidth_gbps: 10
|
||||
egress_control:
|
||||
model: centralized_appliance
|
||||
appliance_vendor: None mandated; to be selected during detailed design
|
||||
cidr_allocations: Private RFC 1918 ranges that do not overlap with on-premises systems,
|
||||
sized for growth across development, staging, and production environments.
|
||||
governance:
|
||||
identity_source_of_truth: Entra ID
|
||||
pipeline_authentication: workload identity federation
|
||||
environment_tiers:
|
||||
- name: development
|
||||
archetype: corp_workload
|
||||
- name: staging
|
||||
archetype: corp_workload
|
||||
- name: production
|
||||
archetype: pci_regulated_workload
|
||||
operations:
|
||||
operating_model: central platform team brokering access for application squads
|
||||
support_hours: always
|
||||
monitoring: Google Cloud Operations Suite (Cloud Monitoring and Cloud Logging) with
|
||||
alerts on availability and errors.
|
||||
change_process: Infrastructure changes managed via pull requests with required approvals
|
||||
and automated GitHub Actions CI/CD pipelines.
|
||||
integrations: []
|
||||
delivery:
|
||||
target_start_date: ask
|
||||
window_weeks: 10
|
||||
commercial_model: fixed_price_milestone
|
||||
iac_tooling: terraform
|
||||
ci_cd: GitHub Actions
|
||||
team_size: 3
|
||||
working_days_per_week: 5
|
||||
cost:
|
||||
monthly_budget: not_applicable
|
||||
currency: GBP
|
||||
constraints:
|
||||
products_required: []
|
||||
products_forbidden: []
|
||||
other: ask
|
||||
run:
|
||||
pre_approved: true
|
||||
allow_cloud_read: false
|
||||
overlay_skills: ask
|
||||
terraform_modules: false
|
||||
inferred_assumptions:
|
||||
- id: ASSUMP-01
|
||||
field: intent.success_criteria
|
||||
statement: Assumed intent.success_criteria is ['The platform foundation is provisioned
|
||||
within 10 weeks, compliant with DORA and PCI-DSS v4.0, providing 10Gbps dedicated
|
||||
interconnect hybrid connectivity, centralised egress inspection, and customer-managed
|
||||
encryption keys.']. Derived from customer statement and requirements.
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-02
|
||||
field: workload.availability.target
|
||||
statement: Assumed workload.availability.target is 99.9%. Catalogue default for
|
||||
platform foundation availability.
|
||||
impact_if_invalid: Changes resilience design and cost.
|
||||
- id: ASSUMP-03
|
||||
field: workload.availability.recovery_time
|
||||
statement: Assumed workload.availability.recovery_time is 4 hours. Catalogue default
|
||||
recovery time objective for warm standby resilience tier.
|
||||
impact_if_invalid: Changes backup and standby design.
|
||||
- id: ASSUMP-04
|
||||
field: workload.availability.recovery_point
|
||||
statement: Assumed workload.availability.recovery_point is 1 hour. Catalogue default
|
||||
recovery point objective.
|
||||
impact_if_invalid: Changes backup frequency.
|
||||
- id: ASSUMP-05
|
||||
field: networking.egress_control.appliance_vendor
|
||||
statement: Assumed networking.egress_control.appliance_vendor is None mandated;
|
||||
to be selected during detailed design. Customer mandates centralised inspection
|
||||
appliances but did not specify a vendor.
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-06
|
||||
field: networking.cidr_allocations
|
||||
statement: 'Assumed networking.cidr_allocations is Private RFC 1918 ranges that
|
||||
do not overlap with on-premises systems, sized for growth across development,
|
||||
staging, and production environments.. Catalogue default: non-overlapping private
|
||||
RFC 1918 ranges sized for growth across environment tiers.'
|
||||
impact_if_invalid: Changes every network resource.
|
||||
- id: ASSUMP-07
|
||||
field: operations.monitoring
|
||||
statement: 'Assumed operations.monitoring is Google Cloud Operations Suite (Cloud
|
||||
Monitoring and Cloud Logging) with alerts on availability and errors.. Catalogue
|
||||
default: Cloud provider native monitoring and alerting.'
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-08
|
||||
field: operations.change_process
|
||||
statement: Assumed operations.change_process is Infrastructure changes managed via
|
||||
pull requests with required approvals and automated GitHub Actions CI/CD pipelines..
|
||||
Aligned with stated GitHub Actions tooling and compliance controls.
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-09
|
||||
field: integrations
|
||||
statement: 'Assumed integrations is []. Catalogue default: no third-party integrations
|
||||
beyond existing on-premises hybrid connectivity.'
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-10
|
||||
field: constraints.products_required
|
||||
statement: 'Assumed constraints.products_required is []. Catalogue default: no mandated
|
||||
products beyond customer stated tools (Entra ID, GitHub Actions).'
|
||||
impact_if_invalid: Local to this field.
|
||||
- id: ASSUMP-11
|
||||
field: constraints.products_forbidden
|
||||
statement: 'Assumed constraints.products_forbidden is []. Catalogue default: no
|
||||
forbidden products identified.'
|
||||
impact_if_invalid: Local to this field.
|
||||
discovery_gaps:
|
||||
- field: metadata.engagement_reference
|
||||
prompt: What is metadata.engagement_reference?
|
||||
severity: LOW
|
||||
- field: delivery.target_start_date
|
||||
prompt: When should work start?
|
||||
severity: LOW
|
||||
- field: constraints.other
|
||||
prompt: What is constraints.other?
|
||||
severity: LOW
|
||||
- field: run.overlay_skills
|
||||
prompt: What is run.overlay_skills?
|
||||
severity: LOW
|
||||
provenance:
|
||||
compliance.audit_logging.immutable_retention_policy: supplied
|
||||
compliance.audit_logging.retention_days: supplied
|
||||
compliance.data_classification: supplied
|
||||
compliance.encryption.key_rotation_days: supplied
|
||||
compliance.encryption.type: supplied
|
||||
compliance.frameworks: supplied
|
||||
compliance.industry: supplied
|
||||
constraints.other: unknown
|
||||
constraints.products_forbidden: assumed
|
||||
constraints.products_required: assumed
|
||||
cost.currency: supplied
|
||||
cost.monthly_budget: not_applicable
|
||||
delivery.ci_cd: supplied
|
||||
delivery.commercial_model: supplied
|
||||
delivery.iac_tooling: supplied
|
||||
delivery.target_start_date: unknown
|
||||
delivery.team_size: supplied
|
||||
delivery.window_weeks: supplied
|
||||
delivery.working_days_per_week: supplied
|
||||
engagement.kind: supplied
|
||||
estate.cloud_provider: supplied
|
||||
estate.data_residency: supplied
|
||||
estate.existing_systems: supplied
|
||||
estate.mode: supplied
|
||||
estate.regions.primary: supplied
|
||||
estate.regions.resiliency_tier: supplied
|
||||
estate.regions.secondary: supplied
|
||||
governance.environment_tiers: supplied
|
||||
governance.identity_source_of_truth: supplied
|
||||
governance.pipeline_authentication: supplied
|
||||
integrations: assumed
|
||||
intent.exclusions: supplied
|
||||
intent.requirements: supplied
|
||||
intent.statement: supplied
|
||||
intent.success_criteria: assumed
|
||||
metadata.author: supplied
|
||||
metadata.client_name: supplied
|
||||
metadata.customer_case: supplied
|
||||
metadata.engagement_reference: unknown
|
||||
metadata.revision: supplied
|
||||
networking.cidr_allocations: assumed
|
||||
networking.egress_control.appliance_vendor: assumed
|
||||
networking.egress_control.model: supplied
|
||||
networking.exposure: supplied
|
||||
networking.hybrid_connectivity.bandwidth_gbps: supplied
|
||||
networking.hybrid_connectivity.enabled: supplied
|
||||
networking.hybrid_connectivity.type: supplied
|
||||
networking.topology: supplied
|
||||
operations.change_process: assumed
|
||||
operations.monitoring: assumed
|
||||
operations.operating_model: supplied
|
||||
operations.support_hours: supplied
|
||||
run.allow_cloud_read: supplied
|
||||
run.overlay_skills: unknown
|
||||
run.pre_approved: supplied
|
||||
run.terraform_modules: supplied
|
||||
workload.availability.hours: supplied
|
||||
workload.availability.recovery_point: assumed
|
||||
workload.availability.recovery_time: assumed
|
||||
workload.availability.target: assumed
|
||||
workload.users.count: not_applicable
|
||||
workload.users.peak_concurrent: not_applicable
|
||||
workload.users.who: not_applicable
|
||||
readiness:
|
||||
supplied: 44
|
||||
answered: 0
|
||||
assumed: 11
|
||||
unknown: 4
|
||||
not_applicable: 4
|
||||
Reference in New Issue
Block a user