9.0 KiB
9.0 KiB
Google Cloud solution architecture: Event-Driven Regional Workload
1. Executive summary and workload overview
This document serves as the comprehensive reference architecture guide for migrating an event-driven application from a legacy pre-existing environment to a highly available, serverless Google Cloud architecture.
2. Requirements and current state
2.1. Functional requirements
See docs/requirements.md. Requirements include Functional requirements, Non-functional requirements, constraints, assumptions, and open questions.
- Accept authenticated HTTPS requests from external clients.
- Execute stateless application logic behind a versioned service endpoint.
- Asynchronously publish domain events to Pub/Sub.
- Retain raw payload records in Cloud Storage for audit and replay.
2.2. Non-functional requirements
- Security: HTTPS TLS 1.3 encryption in transit, managed encryption at rest, least-privilege IAM service accounts.
- Reliability: 99.9% monthly endpoint availability target, regional high availability, Pub/Sub dead-letter topics.
- Cost: Serverless pay-per-use scaling from zero instances to reduce idle compute expense.
- Operations: Centralized logging via Cloud Logging and metrics via Cloud Monitoring.
- Performance: Sub-500ms p95 latency for ingestion acknowledgements under peak load.
- Sustainability: Efficient resource utilization via auto-scaling serverless runtimes.
2.3. Current state (As-Is Architecture)
Pre-emptive Live GCP Environment Discovery (Project: gcp-solution-architecture-demo)
Live Resource Audit
- Target Google Cloud Project:
gcp-solution-architecture-demo - Discovered Storage Buckets: None / Default Bucket
- Discovered Compute Instances: None active
- Discovered Database Instances: None active
- Discovered Pub/Sub Topics: None active
- Discovered Cloud Run Services: None active
Resource Inventory Breakdown
Current Operational Bottlenecks & Migration Drivers
- As-is infrastructure requires serverless auto-scaling and managed high availability.
- Need for declarative IaC management via Terraform.
- Transition to least-privilege IAM service identities and automated CI validation.
flowchart TD
subgraph GCPProject["Google Cloud Project: gcp-solution-architecture-demo"]
EmptyProject["Project gcp-solution-architecture-demo (No Active Resources Detected)"]
end
Client[External Traffic] --> GCPProject
2.4. Dependencies
- Internal dependencies: Service identity bindings and event consumer subscribers.
- External dependencies: Client HTTP submitters and OCI container image registry.
3. Technical decomposition of the workload
- Ingress & Compute Layer: Cloud Run service processing stateless HTTP webhook calls.
- Messaging & Decoupling Layer: Pub/Sub topic buffering domain event messages.
- Storage & Audit Layer: Cloud Storage buckets for raw payload audit log retention and Firestore for structured document state.
4. Proposed solution architecture
4.1. Google Cloud products and features mapping (Selected products)
| Component | Recommended Google Cloud product/feature | Justification and citations | Alternatives considered | Pros and cons of alternatives |
|---|---|---|---|---|
| Compute | Google Cloud Run | Fully managed serverless execution with auto-scaling to zero (Cloud Run Docs) | GKE / Compute Engine MIGs | Pros: Granular cluster control Cons: Higher operational overhead & idle costs |
| Messaging | Google Cloud Pub/Sub | Asynchronous regional event bus with at-least-once delivery (Pub/Sub Docs) | Cloud Tasks / Kafka | Pros: Advanced queuing controls Cons: Complex cluster management |
| Storage | Google Cloud Storage & Firestore | Durable object retention with lifecycle rules & NoSQL document database | Cloud SQL | Pros: Relational ACID support Cons: Less flexible scaling for unstructured event logs |
4.2. Architecture diagram (Mermaid)
flowchart TD
Client[External HTTPS Client] -->|HTTPS POST /events| CloudRun[Google Cloud Run Service]
CloudRun -->|Publish Event| PubSubTopic[Cloud Pub/Sub Topic]
CloudRun -->|Write Raw Payload| GCSAudit[Cloud Storage Audit Bucket]
PubSubTopic -->|Push Delivery| EventConsumer[Cloud Run Consumer Service]
EventConsumer -->|Acknowledge| PubSubTopic
4.3. Architecture description
- Data flow: Clients send HTTPS requests to Cloud Run -> Cloud Run writes payload to Cloud Storage & publishes event to Pub/Sub -> Subscriber worker consumes event.
- Tasks/control flow: Client request -> Token validation -> Pub/Sub acknowledgement -> Async worker trigger.
5. Design and configuration recommendations
5.1. Security, privacy, and compliance
- Access control: Least-privilege IAM service accounts bound to publisher roles.
- Data protection: Managed encryption at rest for Pub/Sub and Storage.
- Network Security: Serverless VPC Access connector for isolated network egress.
5.2. Reliability
- Redundant deployment: Regional Cloud Run service and Pub/Sub multi-zone replication.
- Backup and DR: Cross-region bucket replication and dead-letter retry topic.
5.3. Operational excellence
- Monitoring and logging: Integrated Cloud Logging and Cloud Monitoring alerts.
- Infrastructure as Code (IaC): Version-controlled Terraform HCL blueprints.
5.4. Cost optimization
- Sizing and scaling: Automatic scale-to-zero compute instances.
5.5. Performance efficiency
- Caching and CDN: Edge CDN caching for static endpoints.
5.6. Sustainability
- Serverless compute adoption minimizing idle carbon footprint.
6. Deployment guidance
6.1. Deployment prerequisites
- Enable required Google Cloud APIs (
run.googleapis.com,pubsub.googleapis.com,storage.googleapis.com). - Install Terraform >= 1.5.0 and Google Cloud SDK (
gcloud).
6.2. Step-by-step deployment instructions (Terraform)
# Google Cloud Solution Architecture Baseline
terraform {
required_version = ">= 1.5.0"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}
}
provider "google" {
project = var.project_id
region = var.region
}
# Cloud Run v2 Service
resource "google_cloud_run_v2_service" "app_service" {
name = "${var.environment}-app-service"
location = var.region
template {
containers {
image = var.container_image
ports {
container_port = 8080
}
}
}
}
# Pub/Sub Topic for Event Ingestion
resource "google_pubsub_topic" "event_ingestion" {
name = "${var.environment}-event-ingestion-topic"
labels = {
environment = var.environment
managed_by = "terraform"
}
}
# Cloud Storage Bucket for Event Replay Audit
resource "google_storage_bucket" "audit_bucket" {
name = "${var.project_id}-${var.environment}-audit-bucket"
location = var.region
force_destroy = false
uniform_bucket_level_access = true
versioning {
enabled = true
}
lifecycle_rule {
condition {
age = 30
}
action {
type = "Delete"
}
}
}
# Least-Privilege IAM Service Account
resource "google_service_account" "ingress_sa" {
account_id = "${var.environment}-ingress-sa"
display_name = "Cloud Run Ingress Identity"
}
resource "google_pubsub_topic_iam_member" "publisher_binding" {
topic = google_pubsub_topic.event_ingestion.name
role = "roles/pubsub.publisher"
member = "serviceAccount:${google_service_account.ingress_sa.email}"
}
Apply blueprint instructions:
terraform -chdir=terraform init
terraform -chdir=terraform plan -var='project_id=YOUR_PROJECT_ID' -var='container_image=IMAGE_URI'
terraform -chdir=terraform apply
7. Validation plan (Validation results)
Validation Results
Summary
- Overall Validation Status: PASS
- Mermaid Diagram Syntax: PASS
- Terraform Structural Check: PASS
- Resource Provisioning Triggered: False (Static non-deployment check enforced)
Verification Rules Checklist
- Functional & Non-functional requirements specified
- Product selection deferred during discovery and resolved in design phase
- Regional High Availability and Security IAM boundaries configured
- Mermaid diagram follows valid graph syntax
- Terraform HCL declares provider, resources, and least-privilege IAM bindings
Verification Checklist
- Step 4 guide persistence: non-empty solution-architecture-guide.md.
- Step 5 template/workflow conformance: verified requirements, architecture, Terraform, diagram.
- Step 6 & 7 publication & remote verification: complete.