Files
solution_architecture_agent/eval/datasets/judge/case-02-design-recommendations.md
Jonathan Boniface c84dbdea20 feat: the solution architecture agent on the shared Execution engine
The reference agent's subject, cloud solution architecture, as an agent in
the operating model agent's pattern: the Execution engine (from
orchestration_engine) and one Workflow in app/solution_architecture,
built for the settings and provider bundle by assembly.workflow_for.

Copied from gcp_solution_architecture_agent at 8b397cd without its own
engine: its role files, Skills, provider bundle, ADRs, vocabulary, scripts
and evaluation datasets. The image takes the engine from a named build
context and installs Terraform. It is called through the Orchestration
engine, and its files are what the operating model agent digests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 19:49:28 +01:00

5.7 KiB

Design Recommendations

This document outlines the architectural configurations and best practices applied to the components of the high-scale data ingestion pipeline. These recommendations integrate the functional and non-functional requirements (NFRs) with guidance from the Google Cloud Well-Architected Framework pillar advisors to ensure a secure, reliable, cost-optimized, and high-performing system.

Security, privacy, and compliance

To protect the ingestion endpoint from volumetric DDoS attacks and Layer 7 exploits, a Global External Application Load Balancer with Google Cloud Armor is deployed. This fulfills the requirement for strict perimeter isolation while serving the 10,000 RPS traffic over TLS 1.2+ (DR-1).

The architecture implements a zero-trust model where all internal pipeline components (Cloud Run, Pub/Sub, Cloud Storage, BigQuery) communicate privately via VPC Service Controls and direct VPC egress (DR-3). All processes run under dedicated, least-privilege IAM service accounts, tightly binding permissions to only the required resources (DR-4). To meet regulatory data protection mandates and ensure cryptographic sovereignty, all persistent data across the Pub/Sub staging buffer, Cloud Storage sink, and BigQuery quarantine store is encrypted with Customer-Managed Encryption Keys (CMEK) (DR-2).

Reliability

Reliability is anchored on minimizing data loss (RPO 0) and avoiding systemic failures under heavy load. The ingestion endpoint mitigates the risk of cold-start latency spikes by maintaining a minimum instance baseline, allowing it to sustainably absorb traffic bursts without violating the 200ms p95 latency target (DR-5).

Within the asynchronous pipeline, downstream processing degradation is isolated from ingestion. The Pub/Sub validation subscription utilizes a dead-letter policy with bounded delivery attempts (5 retries) to redirect unparseable poison-pill messages to BigQuery, preventing queue head-of-line blocking (DR-6). Additionally, the valid record storage sink leverages multi-region replication and object versioning to guarantee data durability even during a regional outage or accidental deletion event (DR-7). Finally, the Cloud Run services utilize explicit startup and liveness probes to ensure traffic is only routed to healthy, fully-initialized containers (DR-8).

Operational excellence

A foundational principle of operational excellence is reproducible, reviewable, and automated changes. Every pipeline component—from IAM bindings to Pub/Sub schemas and serverless runtimes—is defined declaratively as Infrastructure as Code (Terraform) (DR-9). This declarative approach acts as the primary recovery mechanism to satisfy the RTO < 60 minutes disaster recovery objective.

Cost optimization

Cost optimizations prioritize scaling tightly with business demand. Both the Ingestion and Validation Cloud Run services are configured to allocate CPU only during request processing, meaning they incur zero compute costs when idle, perfectly matching the strict scale-to-zero requirement (DR-10).

Storage costs are minimized through aggressive lifecycle management. The Pub/Sub buffer retains messages only for a shortened 48-hour window, as RPO=0 durability is fulfilled by the downstream sinks (DR-11). The Cloud Storage bucket transitions older validated batches to Nearline storage after 30 days (DR-12), and the BigQuery dead-letter quarantine datasets enforce a 30-day automated partition expiration to prevent unbound storage accumulation of malformed payloads (DR-13).

Performance efficiency

To sustain the 10,000 RPS throughput requirement with sub-second latencies, the pipeline leverages advanced serverless execution features. Both Cloud Run services use Execution Environment Gen 2 and Startup CPU Boost to optimize network packet processing and eliminate initialization drag (DR-14). The Ingestion service's concurrency is explicitly tuned to handle high parallel request volumes (e.g., 80 concurrent requests per instance), preventing excessive container scale-out thrashing (DR-15).

Between the buffer and the processing tier, the Validation Engine relies on a Pub/Sub push subscription to immediately invoke the validation containers, minimizing pull polling overhead and reducing end-to-end event latency (DR-16).

Sustainability

To minimize the workload's carbon footprint, all services are provisioned together within a Google Cloud region boasting a high Carbon-Free Energy (CFE) score (e.g., europe-west1 or us-central1) (DR-17).

Energy waste is further curtailed by enforcing schema compliance at the initial Pub/Sub buffer boundary, dropping malformed payloads early and preventing wasted validation compute cycles (DR-19). Lastly, the Quarantine Replay Processor is architected as an on-demand Cloud Run Job rather than an always-on polling consumer, aligning energy expenditure directly to actual remediation activity (DR-18).

Assumptions

As this design was produced in pre-approved mode, the following assumptions were made:

  • Cloud Region Selection: We assume the selection of a specific region (like europe-west1 or us-central1) will balance the multi-zone availability requirements with the high CFE targets for sustainability.
  • Micro-batching: We assume the Validation Engine service includes internal micro-batching logic to aggregate records before writing to Cloud Storage, avoiding excessive single-object API calls that could degrade performance.
  • Trace Sampling & Logging: Observability costs will be governed by sampling Cloud Trace ingestion and filtering out routine Cloud Run health check (/healthz) logs out-of-band via standard project sinks.