feat: the solution architecture agent on the shared Execution engine
The reference agent's subject, cloud solution architecture, as an agent in the operating model agent's pattern: the Execution engine (from orchestration_engine) and one Workflow in app/solution_architecture, built for the settings and provider bundle by assembly.workflow_for. Copied from gcp_solution_architecture_agent at 8b397cd without its own engine: its role files, Skills, provider bundle, ADRs, vocabulary, scripts and evaluation datasets. The image takes the engine from a named build context and installs Terraform. It is called through the Orchestration engine, and its files are what the operating model agent digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
54
skills/google-cloud-iac-patterns/SKILL.md
Normal file
54
skills/google-cloud-iac-patterns/SKILL.md
Normal file
@@ -0,0 +1,54 @@
|
||||
---
|
||||
name: google-cloud-iac-patterns
|
||||
description: >-
|
||||
Terraform patterns for Google Cloud products that validate against one pinned
|
||||
version of the hashicorp/google provider: Cloud Run services and jobs,
|
||||
Pub/Sub, Cloud Storage, IAM, Secret Manager, Cloud KMS, Cloud SQL,
|
||||
networking, load balancing, Cloud Scheduler, Artifact Registry, Monitoring,
|
||||
Firestore, BigQuery, Eventarc, GKE and API enablement. Use when writing
|
||||
Terraform for a Google Cloud product, before writing the resource, to start
|
||||
from a pattern whose arguments are known to be in the right place.
|
||||
metadata:
|
||||
scope: deployment guidance
|
||||
provider: hashicorp/google 8.6.0
|
||||
---
|
||||
|
||||
# Google Cloud IaC patterns
|
||||
|
||||
Every pattern here passes `terraform validate` against `hashicorp/google` 8.6.0. A test runs that on each of them, so an argument you copy from a pattern is in the right block.
|
||||
|
||||
## How to use them
|
||||
|
||||
1. Write `assets/versions.tf` to your working copy as `terraform/versions.tf`, unchanged. It pins the provider version these patterns and the `terraform_schema` tool describe. Declare the `project_id` and `region` variables it uses in `terraform/variables.tf`.
|
||||
2. For each Element in the Target architecture, read the reference for its product with `load_skill_resource` and start from its pattern. Read only the references for products the Target architecture uses.
|
||||
3. Change the pattern to meet the Element's Checks. For a setting the pattern does not show, call `terraform_schema` with `find` set to its name before you write it.
|
||||
|
||||
| Product | Reference |
|
||||
| --- | --- |
|
||||
| Enabling APIs | `references/project-services.md` |
|
||||
| Service accounts and IAM grants | `references/iam.md` |
|
||||
| Cloud Run service | `references/cloud-run-service.md` |
|
||||
| Cloud Run job | `references/cloud-run-job.md` |
|
||||
| Pub/Sub | `references/pubsub.md` |
|
||||
| Cloud Storage | `references/cloud-storage.md` |
|
||||
| Secret Manager | `references/secret-manager.md` |
|
||||
| Cloud KMS | `references/kms.md` |
|
||||
| Cloud SQL | `references/cloud-sql.md` |
|
||||
| VPC networking | `references/networking.md` |
|
||||
| External Application Load Balancer, TLS policy, firewall policy, Identity-Aware Proxy | `references/load-balancer.md` |
|
||||
| Cloud Scheduler | `references/cloud-scheduler.md` |
|
||||
| Artifact Registry | `references/artifact-registry.md` |
|
||||
| Cloud Monitoring and Logging | `references/monitoring.md` |
|
||||
| Firestore | `references/firestore.md` |
|
||||
| BigQuery | `references/bigquery.md` |
|
||||
| Eventarc | `references/eventarc.md` |
|
||||
| Google Kubernetes Engine | `references/gke.md` |
|
||||
|
||||
A product with no reference has no pattern yet: write it from `terraform_schema` and the product documentation.
|
||||
|
||||
## Rules that hold for every product
|
||||
|
||||
- A Cloud Run service and a Cloud Run job are different resource types with different arguments. A setting of one is not a setting of the other. `references/cloud-run-job.md` lists what a job lacks.
|
||||
- Grant a role with an `_iam_member` resource. An `_iam_binding` or `_iam_policy` resource removes grants made elsewhere.
|
||||
- Give a Google-managed service agent its role before the resource that needs it, with `depends_on`. A key, a dead-letter topic and a push subscription all need one.
|
||||
- Set `deletion_protection` from a variable on the resources that have it, so a test deployment can be removed.
|
||||
15
skills/google-cloud-iac-patterns/assets/versions.tf
Normal file
15
skills/google-cloud-iac-patterns/assets/versions.tf
Normal file
@@ -0,0 +1,15 @@
|
||||
terraform {
|
||||
required_version = ">= 1.9"
|
||||
|
||||
required_providers {
|
||||
google = {
|
||||
source = "hashicorp/google"
|
||||
version = "8.6.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "google" {
|
||||
project = var.project_id
|
||||
region = var.region
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
# Artifact Registry
|
||||
|
||||
A Docker repository with a cleanup policy, and the grant that lets a runtime identity pull from it.
|
||||
|
||||
- `format = "DOCKER"` for container images. The image path is `REGION-docker.pkg.dev/PROJECT/REPOSITORY/IMAGE`.
|
||||
- Cleanup policies delete nothing while `cleanup_policy_dry_run` is `true`.
|
||||
- `docker_config.immutable_tags = true` stops a tag being moved to a different image.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
resource "google_service_account" "runtime" {
|
||||
project = var.project_id
|
||||
account_id = "image-puller"
|
||||
display_name = "Pulls workload images"
|
||||
}
|
||||
|
||||
resource "google_artifact_registry_repository" "images" {
|
||||
project = var.project_id
|
||||
location = var.region
|
||||
repository_id = "workload-images"
|
||||
format = "DOCKER"
|
||||
description = "Container images for the workload."
|
||||
cleanup_policy_dry_run = false
|
||||
|
||||
docker_config {
|
||||
immutable_tags = true
|
||||
}
|
||||
|
||||
cleanup_policies {
|
||||
id = "keep-recent"
|
||||
action = "KEEP"
|
||||
|
||||
most_recent_versions {
|
||||
keep_count = 10
|
||||
}
|
||||
}
|
||||
|
||||
cleanup_policies {
|
||||
id = "delete-old-untagged"
|
||||
action = "DELETE"
|
||||
|
||||
condition {
|
||||
tag_state = "UNTAGGED"
|
||||
older_than = "2592000s"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_artifact_registry_repository_iam_member" "runtime_pulls" {
|
||||
project = var.project_id
|
||||
location = google_artifact_registry_repository.images.location
|
||||
repository = google_artifact_registry_repository.images.name
|
||||
role = "roles/artifactregistry.reader"
|
||||
member = google_service_account.runtime.member
|
||||
}
|
||||
```
|
||||
92
skills/google-cloud-iac-patterns/references/bigquery.md
Normal file
92
skills/google-cloud-iac-patterns/references/bigquery.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# BigQuery
|
||||
|
||||
A dataset, a partitioned and clustered table, and the grants for a writer and a reader.
|
||||
|
||||
- `location` is a region or `EU`/`US`, and cannot be changed. Queries cannot join datasets in different locations.
|
||||
- `delete_contents_on_destroy = false` makes Terraform refuse to remove a dataset that still has tables.
|
||||
- A table's `deletion_protection` is true unless set; take it from a variable so a test deployment can be removed.
|
||||
- `schema` is a JSON string. Changing a column's type or mode replaces the table.
|
||||
- Partition on a `DATE` or `TIMESTAMP` column with `time_partitioning`, and set `require_partition_filter` so a query cannot scan every partition. `expiration_ms` there removes old partitions.
|
||||
- Grant on the dataset with `google_bigquery_dataset_iam_member`. A principal that runs queries also needs `roles/bigquery.jobUser` on the project.
|
||||
- Do not mix `google_bigquery_dataset_iam_member` with `access` blocks on the dataset: each removes what the other sets.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "writer_service_account_email" {
|
||||
type = string
|
||||
description = "Service account that loads rows."
|
||||
}
|
||||
|
||||
variable "reader_service_account_email" {
|
||||
type = string
|
||||
description = "Service account that queries the dataset."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether the table is protected from deletion."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset" "analytics" {
|
||||
project = var.project_id
|
||||
dataset_id = "analytics"
|
||||
location = var.region
|
||||
delete_contents_on_destroy = false
|
||||
|
||||
default_table_expiration_ms = null
|
||||
max_time_travel_hours = 168
|
||||
}
|
||||
|
||||
resource "google_bigquery_table" "events" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
table_id = "events"
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
require_partition_filter = true
|
||||
clustering = ["event_type"]
|
||||
|
||||
time_partitioning {
|
||||
type = "DAY"
|
||||
field = "event_time"
|
||||
expiration_ms = 34560000000
|
||||
}
|
||||
|
||||
schema = jsonencode([
|
||||
{ name = "event_id", type = "STRING", mode = "REQUIRED" },
|
||||
{ name = "event_type", type = "STRING", mode = "REQUIRED" },
|
||||
{ name = "event_time", type = "TIMESTAMP", mode = "REQUIRED" },
|
||||
{ name = "payload", type = "JSON", mode = "NULLABLE" }
|
||||
])
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset_iam_member" "writer" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
role = "roles/bigquery.dataEditor"
|
||||
member = "serviceAccount:${var.writer_service_account_email}"
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset_iam_member" "reader" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
role = "roles/bigquery.dataViewer"
|
||||
member = "serviceAccount:${var.reader_service_account_email}"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "reader_runs_queries" {
|
||||
project = var.project_id
|
||||
role = "roles/bigquery.jobUser"
|
||||
member = "serviceAccount:${var.reader_service_account_email}"
|
||||
}
|
||||
```
|
||||
95
skills/google-cloud-iac-patterns/references/cloud-run-job.md
Normal file
95
skills/google-cloud-iac-patterns/references/cloud-run-job.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Cloud Run job
|
||||
|
||||
`google_cloud_run_v2_job`: a task that runs to completion. It is a different resource type from a service, and its template is nested one level deeper.
|
||||
|
||||
What a job does not have. A Check that asks for one of these on a job cannot be met:
|
||||
|
||||
- `startup_cpu_boost` and `cpu_idle`. A job's `resources` block has `limits` only.
|
||||
- `ingress`, `traffic`, `scaling`, `min_instance_count`, `max_instance_request_concurrency`. A job answers no requests.
|
||||
- `liveness_probe`. A job's container has a `startup_probe` only.
|
||||
|
||||
Where a job's settings sit:
|
||||
|
||||
- `task_count` and `parallelism` are in `template`.
|
||||
- `service_account`, `timeout`, `max_retries`, `execution_environment`, `vpc_access` and `encryption_key` are in `template.template`.
|
||||
- `image`, `env`, `args`, `command` and `resources` are in `template.template.containers`.
|
||||
- Whoever starts the job needs `roles/run.invoker` on it; Cloud Scheduler starts it with an HTTP call to the `:run` method.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
type = string
|
||||
description = "Container image reference, including the tag or digest."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether Terraform is prevented from deleting the job."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_service_account" "replay" {
|
||||
project = var.project_id
|
||||
account_id = "replay-job"
|
||||
display_name = "Replay job"
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_job" "replay" {
|
||||
project = var.project_id
|
||||
name = "replay"
|
||||
location = var.region
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
template {
|
||||
task_count = 1
|
||||
parallelism = 1
|
||||
|
||||
template {
|
||||
service_account = google_service_account.replay.email
|
||||
timeout = "3600s"
|
||||
max_retries = 3
|
||||
execution_environment = "EXECUTION_ENVIRONMENT_GEN2"
|
||||
|
||||
containers {
|
||||
image = var.image
|
||||
args = ["--mode=replay"]
|
||||
|
||||
resources {
|
||||
limits = {
|
||||
cpu = "1"
|
||||
memory = "1Gi"
|
||||
}
|
||||
}
|
||||
|
||||
env {
|
||||
name = "BATCH_SIZE"
|
||||
value = "100"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_service_account" "operator" {
|
||||
project = var.project_id
|
||||
account_id = "replay-operator"
|
||||
display_name = "Starts the replay job"
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_job_iam_member" "operator_runs_replay" {
|
||||
project = var.project_id
|
||||
location = google_cloud_run_v2_job.replay.location
|
||||
name = google_cloud_run_v2_job.replay.name
|
||||
role = "roles/run.invoker"
|
||||
member = google_service_account.operator.member
|
||||
}
|
||||
```
|
||||
144
skills/google-cloud-iac-patterns/references/cloud-run-service.md
Normal file
144
skills/google-cloud-iac-patterns/references/cloud-run-service.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# Cloud Run service
|
||||
|
||||
`google_cloud_run_v2_service`: a long-running service that answers requests.
|
||||
|
||||
Where the settings sit, which is what goes wrong most often:
|
||||
|
||||
- `ingress`, `deletion_protection` and `traffic` are on the resource.
|
||||
- `service_account`, `scaling`, `timeout`, `max_instance_request_concurrency`, `execution_environment`, `vpc_access` and `encryption_key` are in `template`.
|
||||
- `image`, `env`, `ports`, `resources` and the probes are in `template.containers`.
|
||||
- `cpu_idle` and `startup_cpu_boost` are in `template.containers.resources`, beside `limits`. They exist on a service only.
|
||||
- `min_instance_count` and `max_instance_count` are in `template.scaling`.
|
||||
- A secret is read through `env.value_source.secret_key_ref`; the service account needs `roles/secretmanager.secretAccessor` on that secret.
|
||||
- Direct VPC egress is `template.vpc_access.network_interfaces`. A connector is `template.vpc_access.connector`. Use one of them.
|
||||
- Public access is a grant of `roles/run.invoker` to `allUsers` together with `ingress = "INGRESS_TRAFFIC_ALL"`. A private service has neither.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
type = string
|
||||
description = "Container image reference, including the tag or digest."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether Terraform is prevented from deleting the service."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_service_account" "ingest" {
|
||||
project = var.project_id
|
||||
account_id = "ingest-service"
|
||||
display_name = "Ingest service"
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret" "signing_key" {
|
||||
project = var.project_id
|
||||
secret_id = "webhook-signing-key"
|
||||
|
||||
replication {
|
||||
auto {}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret_iam_member" "ingest_reads_signing_key" {
|
||||
project = var.project_id
|
||||
secret_id = google_secret_manager_secret.signing_key.secret_id
|
||||
role = "roles/secretmanager.secretAccessor"
|
||||
member = google_service_account.ingest.member
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_service" "ingest" {
|
||||
project = var.project_id
|
||||
name = "ingest"
|
||||
location = var.region
|
||||
ingress = "INGRESS_TRAFFIC_ALL"
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
template {
|
||||
service_account = google_service_account.ingest.email
|
||||
timeout = "30s"
|
||||
max_instance_request_concurrency = 80
|
||||
execution_environment = "EXECUTION_ENVIRONMENT_GEN2"
|
||||
|
||||
scaling {
|
||||
min_instance_count = 1
|
||||
max_instance_count = 10
|
||||
}
|
||||
|
||||
containers {
|
||||
image = var.image
|
||||
|
||||
ports {
|
||||
container_port = 8080
|
||||
}
|
||||
|
||||
resources {
|
||||
limits = {
|
||||
cpu = "1"
|
||||
memory = "512Mi"
|
||||
}
|
||||
cpu_idle = true
|
||||
startup_cpu_boost = true
|
||||
}
|
||||
|
||||
env {
|
||||
name = "LOG_LEVEL"
|
||||
value = "info"
|
||||
}
|
||||
|
||||
env {
|
||||
name = "SIGNING_KEY"
|
||||
value_source {
|
||||
secret_key_ref {
|
||||
secret = google_secret_manager_secret.signing_key.secret_id
|
||||
version = "latest"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
startup_probe {
|
||||
initial_delay_seconds = 0
|
||||
period_seconds = 5
|
||||
failure_threshold = 6
|
||||
http_get {
|
||||
path = "/healthz"
|
||||
}
|
||||
}
|
||||
|
||||
liveness_probe {
|
||||
period_seconds = 30
|
||||
http_get {
|
||||
path = "/healthz"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
traffic {
|
||||
type = "TRAFFIC_TARGET_ALLOCATION_TYPE_LATEST"
|
||||
percent = 100
|
||||
}
|
||||
|
||||
depends_on = [google_secret_manager_secret_iam_member.ingest_reads_signing_key]
|
||||
}
|
||||
|
||||
# Public access. Leave this out, and set ingress to INGRESS_TRAFFIC_INTERNAL_ONLY,
|
||||
# for a service only other workloads call.
|
||||
resource "google_cloud_run_v2_service_iam_member" "public" {
|
||||
project = var.project_id
|
||||
location = google_cloud_run_v2_service.ingest.location
|
||||
name = google_cloud_run_v2_service.ingest.name
|
||||
role = "roles/run.invoker"
|
||||
member = "allUsers"
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,75 @@
|
||||
# Cloud Scheduler
|
||||
|
||||
A schedule that starts a Cloud Run job, and one that calls a private Cloud Run service.
|
||||
|
||||
- A job is started by an HTTP POST to its `:run` method with an OAuth token (`oauth_token`). A service is called with an OIDC token (`oidc_token`). Using the wrong one is refused at run time, not by `terraform validate`.
|
||||
- The schedule's service account needs `roles/run.invoker` on the job or service it starts.
|
||||
- `schedule` is unix-cron; `time_zone` is an IANA name.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "job_name" {
|
||||
type = string
|
||||
description = "Name of the Cloud Run job to start."
|
||||
}
|
||||
|
||||
variable "service_url" {
|
||||
type = string
|
||||
description = "HTTPS URL of the private Cloud Run service to call."
|
||||
}
|
||||
|
||||
resource "google_service_account" "scheduler" {
|
||||
project = var.project_id
|
||||
account_id = "nightly-scheduler"
|
||||
display_name = "Identity of the nightly schedules"
|
||||
}
|
||||
|
||||
resource "google_cloud_scheduler_job" "start_job" {
|
||||
project = var.project_id
|
||||
region = var.region
|
||||
name = "start-nightly-job"
|
||||
schedule = "0 2 * * *"
|
||||
time_zone = "Etc/UTC"
|
||||
attempt_deadline = "180s"
|
||||
|
||||
retry_config {
|
||||
retry_count = 3
|
||||
}
|
||||
|
||||
http_target {
|
||||
http_method = "POST"
|
||||
uri = "https://run.googleapis.com/v2/projects/${var.project_id}/locations/${var.region}/jobs/${var.job_name}:run"
|
||||
|
||||
oauth_token {
|
||||
service_account_email = google_service_account.scheduler.email
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_cloud_scheduler_job" "call_service" {
|
||||
project = var.project_id
|
||||
region = var.region
|
||||
name = "call-nightly-service"
|
||||
schedule = "30 2 * * *"
|
||||
time_zone = "Etc/UTC"
|
||||
|
||||
http_target {
|
||||
http_method = "POST"
|
||||
uri = "${var.service_url}/tasks/nightly"
|
||||
|
||||
oidc_token {
|
||||
service_account_email = google_service_account.scheduler.email
|
||||
audience = var.service_url
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
124
skills/google-cloud-iac-patterns/references/cloud-sql.md
Normal file
124
skills/google-cloud-iac-patterns/references/cloud-sql.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# Cloud SQL
|
||||
|
||||
A PostgreSQL instance with a private address, backups and IAM database authentication.
|
||||
|
||||
- A private address needs a reserved range and a service networking connection on the VPC first; the instance must `depends_on` the connection.
|
||||
- `availability_type = "REGIONAL"` is high availability; `"ZONAL"` is a single zone.
|
||||
- Backups, point-in-time recovery, flags and the address are all in `settings`.
|
||||
- `deletion_protection` on the resource guards Terraform; `settings.deletion_protection_enabled` guards the API.
|
||||
- Take the database password from Secret Manager at run time, or use IAM database authentication as here. Never put a password in the configuration.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether Terraform is prevented from deleting the instance."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_compute_network" "main" {
|
||||
project = var.project_id
|
||||
name = "main"
|
||||
auto_create_subnetworks = false
|
||||
}
|
||||
|
||||
resource "google_compute_global_address" "private_services" {
|
||||
project = var.project_id
|
||||
name = "private-services"
|
||||
purpose = "VPC_PEERING"
|
||||
address_type = "INTERNAL"
|
||||
prefix_length = 16
|
||||
network = google_compute_network.main.id
|
||||
}
|
||||
|
||||
resource "google_service_networking_connection" "private_services" {
|
||||
network = google_compute_network.main.id
|
||||
service = "servicenetworking.googleapis.com"
|
||||
reserved_peering_ranges = [google_compute_global_address.private_services.name]
|
||||
}
|
||||
|
||||
resource "google_service_account" "app" {
|
||||
project = var.project_id
|
||||
account_id = "orders-app"
|
||||
display_name = "Orders application"
|
||||
}
|
||||
|
||||
resource "google_sql_database_instance" "orders" {
|
||||
project = var.project_id
|
||||
name = "orders"
|
||||
region = var.region
|
||||
database_version = "POSTGRES_16"
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
settings {
|
||||
tier = "db-custom-2-7680"
|
||||
edition = "ENTERPRISE"
|
||||
availability_type = "REGIONAL"
|
||||
disk_type = "PD_SSD"
|
||||
disk_autoresize = true
|
||||
deletion_protection_enabled = var.deletion_protection
|
||||
|
||||
ip_configuration {
|
||||
ipv4_enabled = false
|
||||
private_network = google_compute_network.main.id
|
||||
ssl_mode = "ENCRYPTED_ONLY"
|
||||
}
|
||||
|
||||
backup_configuration {
|
||||
enabled = true
|
||||
point_in_time_recovery_enabled = true
|
||||
start_time = "02:00"
|
||||
|
||||
backup_retention_settings {
|
||||
retained_backups = 14
|
||||
}
|
||||
}
|
||||
|
||||
database_flags {
|
||||
name = "cloudsql.iam_authentication"
|
||||
value = "on"
|
||||
}
|
||||
|
||||
maintenance_window {
|
||||
day = 7
|
||||
hour = 3
|
||||
}
|
||||
}
|
||||
|
||||
depends_on = [google_service_networking_connection.private_services]
|
||||
}
|
||||
|
||||
resource "google_sql_database" "orders" {
|
||||
project = var.project_id
|
||||
name = "orders"
|
||||
instance = google_sql_database_instance.orders.name
|
||||
}
|
||||
|
||||
resource "google_sql_user" "app" {
|
||||
project = var.project_id
|
||||
name = trimsuffix(google_service_account.app.email, ".gserviceaccount.com")
|
||||
instance = google_sql_database_instance.orders.name
|
||||
type = "CLOUD_IAM_SERVICE_ACCOUNT"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "app_connects" {
|
||||
project = var.project_id
|
||||
role = "roles/cloudsql.client"
|
||||
member = google_service_account.app.member
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "app_logs_in" {
|
||||
project = var.project_id
|
||||
role = "roles/cloudsql.instanceUser"
|
||||
member = google_service_account.app.member
|
||||
}
|
||||
```
|
||||
67
skills/google-cloud-iac-patterns/references/cloud-storage.md
Normal file
67
skills/google-cloud-iac-patterns/references/cloud-storage.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Cloud Storage
|
||||
|
||||
A private bucket with a retention period and a lifecycle.
|
||||
|
||||
- Bucket names are global. Build the name from the project id.
|
||||
- `uniform_bucket_level_access = true` and `public_access_prevention = "enforced"` keep the bucket private.
|
||||
- `retention_policy.retention_period` is in seconds and stops objects being deleted before then. A lifecycle `Delete` rule with `age` in days removes them after. Retaining for 30 days and then deleting needs both.
|
||||
- Versioning and a retention policy cannot be used on the same bucket.
|
||||
- Customer-managed encryption is `encryption.default_kms_key_name`; the Cloud Storage service agent needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key first.
|
||||
- `force_destroy = false` stops Terraform deleting a bucket that holds objects.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
resource "google_service_account" "writer" {
|
||||
project = var.project_id
|
||||
account_id = "raw-events-writer"
|
||||
display_name = "Writes raw events"
|
||||
}
|
||||
|
||||
resource "google_storage_bucket" "raw_events" {
|
||||
project = var.project_id
|
||||
name = "${var.project_id}-raw-events"
|
||||
location = var.region
|
||||
storage_class = "STANDARD"
|
||||
uniform_bucket_level_access = true
|
||||
public_access_prevention = "enforced"
|
||||
force_destroy = false
|
||||
|
||||
retention_policy {
|
||||
retention_period = 2592000
|
||||
}
|
||||
|
||||
lifecycle_rule {
|
||||
condition {
|
||||
age = 31
|
||||
}
|
||||
action {
|
||||
type = "Delete"
|
||||
}
|
||||
}
|
||||
|
||||
lifecycle_rule {
|
||||
condition {
|
||||
age = 7
|
||||
}
|
||||
action {
|
||||
type = "SetStorageClass"
|
||||
storage_class = "NEARLINE"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_iam_member" "writer_creates_objects" {
|
||||
bucket = google_storage_bucket.raw_events.name
|
||||
role = "roles/storage.objectCreator"
|
||||
member = google_service_account.writer.member
|
||||
}
|
||||
```
|
||||
129
skills/google-cloud-iac-patterns/references/eventarc.md
Normal file
129
skills/google-cloud-iac-patterns/references/eventarc.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# Eventarc
|
||||
|
||||
A trigger that delivers events to a Cloud Run service: one for messages on a Pub/Sub topic, one for objects finalised in a bucket.
|
||||
|
||||
- A trigger has its own service account. It needs `roles/eventarc.eventReceiver` on the project and `roles/run.invoker` on the service it calls.
|
||||
- For a Pub/Sub trigger, name an existing topic under `transport.pubsub.topic`; left out, Eventarc creates a topic you do not manage.
|
||||
- For a Cloud Storage trigger, `location` must be the bucket's location, and the Cloud Storage service agent needs `roles/pubsub.publisher` on the project before the trigger is created.
|
||||
- `matching_criteria` attributes are exact. `type` is always present; a storage event also needs `bucket`.
|
||||
- The destination service is named, with its `region`; `path` is optional.
|
||||
- The service must not allow unauthenticated calls for this to be the only way in: grant `roles/run.invoker` to the trigger's service account only.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "service_name" {
|
||||
type = string
|
||||
description = "Name of the Cloud Run service that receives the events."
|
||||
}
|
||||
|
||||
variable "topic_id" {
|
||||
type = string
|
||||
description = "Id of the Pub/Sub topic whose messages are delivered."
|
||||
}
|
||||
|
||||
variable "bucket_name" {
|
||||
type = string
|
||||
description = "Bucket whose new objects are delivered. It must be in var.region."
|
||||
}
|
||||
|
||||
data "google_storage_project_service_account" "this" {
|
||||
project = var.project_id
|
||||
}
|
||||
|
||||
resource "google_service_account" "trigger" {
|
||||
project = var.project_id
|
||||
account_id = "eventarc-trigger"
|
||||
display_name = "Eventarc trigger"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "trigger_receives_events" {
|
||||
project = var.project_id
|
||||
role = "roles/eventarc.eventReceiver"
|
||||
member = "serviceAccount:${google_service_account.trigger.email}"
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_service_iam_member" "trigger_invokes_service" {
|
||||
project = var.project_id
|
||||
location = var.region
|
||||
name = var.service_name
|
||||
role = "roles/run.invoker"
|
||||
member = "serviceAccount:${google_service_account.trigger.email}"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "storage_publishes_events" {
|
||||
project = var.project_id
|
||||
role = "roles/pubsub.publisher"
|
||||
member = "serviceAccount:${data.google_storage_project_service_account.this.email_address}"
|
||||
}
|
||||
|
||||
resource "google_eventarc_trigger" "on_message" {
|
||||
project = var.project_id
|
||||
name = "on-message"
|
||||
location = var.region
|
||||
service_account = google_service_account.trigger.email
|
||||
|
||||
matching_criteria {
|
||||
attribute = "type"
|
||||
value = "google.cloud.pubsub.topic.v1.messagePublished"
|
||||
}
|
||||
|
||||
transport {
|
||||
pubsub {
|
||||
topic = var.topic_id
|
||||
}
|
||||
}
|
||||
|
||||
destination {
|
||||
cloud_run_service {
|
||||
service = var.service_name
|
||||
region = var.region
|
||||
path = "/events/message"
|
||||
}
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
google_project_iam_member.trigger_receives_events,
|
||||
google_cloud_run_v2_service_iam_member.trigger_invokes_service,
|
||||
]
|
||||
}
|
||||
|
||||
resource "google_eventarc_trigger" "on_object" {
|
||||
project = var.project_id
|
||||
name = "on-object"
|
||||
location = var.region
|
||||
service_account = google_service_account.trigger.email
|
||||
|
||||
matching_criteria {
|
||||
attribute = "type"
|
||||
value = "google.cloud.storage.object.v1.finalized"
|
||||
}
|
||||
|
||||
matching_criteria {
|
||||
attribute = "bucket"
|
||||
value = var.bucket_name
|
||||
}
|
||||
|
||||
destination {
|
||||
cloud_run_service {
|
||||
service = var.service_name
|
||||
region = var.region
|
||||
path = "/events/object"
|
||||
}
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
google_project_iam_member.trigger_receives_events,
|
||||
google_project_iam_member.storage_publishes_events,
|
||||
google_cloud_run_v2_service_iam_member.trigger_invokes_service,
|
||||
]
|
||||
}
|
||||
```
|
||||
74
skills/google-cloud-iac-patterns/references/firestore.md
Normal file
74
skills/google-cloud-iac-patterns/references/firestore.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Firestore
|
||||
|
||||
A Firestore database in Native mode, with point-in-time recovery, a scheduled backup, a composite index and the grant a workload needs.
|
||||
|
||||
- A project's first database is named `(default)`. Any other name creates an additional database, which clients must name when they connect.
|
||||
- `location_id` is a region or a multi-region such as `eur3`, and cannot be changed afterwards.
|
||||
- `type` is `FIRESTORE_NATIVE` or `DATASTORE_MODE`, and cannot be changed once the database holds data.
|
||||
- `delete_protection_state` is the guard in the API; `deletion_policy = "DELETE"` is needed before Terraform will remove a database at all.
|
||||
- `point_in_time_recovery_enablement` keeps seven days of versions. A backup schedule is separate and keeps whole backups for `retention`, in seconds with an `s`.
|
||||
- A workload reads and writes with `roles/datastore.user`, granted on the project. There is no per-database role; restrict with an IAM condition on the database name.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "app_service_account_email" {
|
||||
type = string
|
||||
description = "Service account of the workload that uses the database."
|
||||
}
|
||||
|
||||
resource "google_firestore_database" "app" {
|
||||
project = var.project_id
|
||||
name = "app"
|
||||
location_id = var.region
|
||||
type = "FIRESTORE_NATIVE"
|
||||
|
||||
concurrency_mode = "OPTIMISTIC"
|
||||
point_in_time_recovery_enablement = "POINT_IN_TIME_RECOVERY_ENABLED"
|
||||
delete_protection_state = "DELETE_PROTECTION_ENABLED"
|
||||
deletion_policy = "ABANDON"
|
||||
}
|
||||
|
||||
resource "google_firestore_backup_schedule" "daily" {
|
||||
project = var.project_id
|
||||
database = google_firestore_database.app.name
|
||||
retention = "1209600s"
|
||||
|
||||
daily_recurrence {}
|
||||
}
|
||||
|
||||
resource "google_firestore_index" "orders_by_customer" {
|
||||
project = var.project_id
|
||||
database = google_firestore_database.app.name
|
||||
collection = "orders"
|
||||
|
||||
fields {
|
||||
field_path = "customer_id"
|
||||
order = "ASCENDING"
|
||||
}
|
||||
|
||||
fields {
|
||||
field_path = "created_at"
|
||||
order = "DESCENDING"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "app_uses_firestore" {
|
||||
project = var.project_id
|
||||
role = "roles/datastore.user"
|
||||
member = "serviceAccount:${var.app_service_account_email}"
|
||||
|
||||
condition {
|
||||
title = "only-the-app-database"
|
||||
expression = "resource.name == \"${google_firestore_database.app.id}\" || resource.name.startsWith(\"${google_firestore_database.app.id}/\")"
|
||||
}
|
||||
}
|
||||
```
|
||||
109
skills/google-cloud-iac-patterns/references/gke.md
Normal file
109
skills/google-cloud-iac-patterns/references/gke.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Google Kubernetes Engine
|
||||
|
||||
A private Autopilot cluster on an existing subnet, with its node service account and Workload Identity for one workload.
|
||||
|
||||
- `enable_autopilot = true` makes Google manage the nodes. Node pools, machine types and node counts are then not yours to set: leave out `node_pool`, `node_config` and `initial_node_count`.
|
||||
- A regional `location` gives a control plane in three zones. A zone gives one.
|
||||
- The subnet needs two secondary ranges, for Pods and Services, named in `ip_allocation_policy`.
|
||||
- `enable_private_nodes = true` gives nodes no public address; they then need Cloud NAT to pull images from outside Google.
|
||||
- `master_authorized_networks_config` limits who can reach the control plane. With `enable_private_endpoint = true` it has no public address at all.
|
||||
- Give nodes a dedicated service account with `roles/container.defaultNodeServiceAccount`, not the Compute Engine default account.
|
||||
- Autopilot has Workload Identity on. A Kubernetes service account acts as a Google service account through `roles/iam.workloadIdentityUser`, with the member `serviceAccount:PROJECT.svc.id.goog[NAMESPACE/KSA]`.
|
||||
- `deletion_protection` is true unless set; take it from a variable.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "network_id" {
|
||||
type = string
|
||||
description = "Id of the VPC the cluster is in."
|
||||
}
|
||||
|
||||
variable "subnetwork_id" {
|
||||
type = string
|
||||
description = "Id of the subnet for nodes. It has secondary ranges named pods and services."
|
||||
}
|
||||
|
||||
variable "admin_cidr" {
|
||||
type = string
|
||||
description = "Range allowed to reach the control plane."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether the cluster is protected from deletion."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_service_account" "nodes" {
|
||||
project = var.project_id
|
||||
account_id = "gke-nodes"
|
||||
display_name = "GKE nodes"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "nodes_default_role" {
|
||||
project = var.project_id
|
||||
role = "roles/container.defaultNodeServiceAccount"
|
||||
member = "serviceAccount:${google_service_account.nodes.email}"
|
||||
}
|
||||
|
||||
resource "google_container_cluster" "workload" {
|
||||
project = var.project_id
|
||||
name = "workload"
|
||||
location = var.region
|
||||
enable_autopilot = true
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
network = var.network_id
|
||||
subnetwork = var.subnetwork_id
|
||||
|
||||
ip_allocation_policy {
|
||||
cluster_secondary_range_name = "pods"
|
||||
services_secondary_range_name = "services"
|
||||
}
|
||||
|
||||
private_cluster_config {
|
||||
enable_private_nodes = true
|
||||
enable_private_endpoint = false
|
||||
}
|
||||
|
||||
master_authorized_networks_config {
|
||||
cidr_blocks {
|
||||
cidr_block = var.admin_cidr
|
||||
display_name = "administrators"
|
||||
}
|
||||
}
|
||||
|
||||
cluster_autoscaling {
|
||||
auto_provisioning_defaults {
|
||||
service_account = google_service_account.nodes.email
|
||||
}
|
||||
}
|
||||
|
||||
release_channel {
|
||||
channel = "REGULAR"
|
||||
}
|
||||
|
||||
depends_on = [google_project_iam_member.nodes_default_role]
|
||||
}
|
||||
|
||||
resource "google_service_account" "app" {
|
||||
project = var.project_id
|
||||
account_id = "orders-app"
|
||||
display_name = "Orders application"
|
||||
}
|
||||
|
||||
resource "google_service_account_iam_member" "app_workload_identity" {
|
||||
service_account_id = google_service_account.app.name
|
||||
role = "roles/iam.workloadIdentityUser"
|
||||
member = "serviceAccount:${var.project_id}.svc.id.goog[orders/orders-app]"
|
||||
}
|
||||
```
|
||||
40
skills/google-cloud-iac-patterns/references/iam.md
Normal file
40
skills/google-cloud-iac-patterns/references/iam.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Service accounts and IAM grants
|
||||
|
||||
One service account per workload identity, and one `_iam_member` resource per grant.
|
||||
|
||||
- `account_id` is 6 to 30 characters: lower-case letters, digits and hyphens.
|
||||
- Use the narrowest resource the role can be granted on: the bucket, topic, secret or service, in preference to the project.
|
||||
- A grant to a service account is `member = "serviceAccount:${...email}"`. The same value is the `member` attribute of the service account resource.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
resource "google_service_account" "worker" {
|
||||
project = var.project_id
|
||||
account_id = "event-worker"
|
||||
display_name = "Event worker"
|
||||
description = "Identity of the worker service."
|
||||
}
|
||||
|
||||
# A project-level role, only when no narrower resource can take the grant.
|
||||
resource "google_project_iam_member" "worker_metrics" {
|
||||
project = var.project_id
|
||||
role = "roles/monitoring.metricWriter"
|
||||
member = google_service_account.worker.member
|
||||
}
|
||||
|
||||
# Who may act as the service account, for example a deployment pipeline.
|
||||
resource "google_service_account_iam_member" "deployer_acts_as_worker" {
|
||||
service_account_id = google_service_account.worker.name
|
||||
role = "roles/iam.serviceAccountUser"
|
||||
member = "serviceAccount:deployer@${var.project_id}.iam.gserviceaccount.com"
|
||||
}
|
||||
```
|
||||
79
skills/google-cloud-iac-patterns/references/kms.md
Normal file
79
skills/google-cloud-iac-patterns/references/kms.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Cloud KMS
|
||||
|
||||
A key ring and a key for customer-managed encryption, and the grant that lets a product use it.
|
||||
|
||||
- The key ring must be in the same location as the resource it protects.
|
||||
- Key rings and keys cannot be deleted. `prevent_destroy` stops Terraform trying.
|
||||
- `rotation_period` is in seconds, such as `"7776000s"` for 90 days.
|
||||
- The product's service agent, not the workload's service account, needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key. The resource that uses the key must `depends_on` that grant.
|
||||
- Service agents: Pub/Sub is `service-NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com`, Cloud Run is `service-NUMBER@serverless-robot-prod.iam.gserviceaccount.com`, Cloud Storage comes from the `google_storage_project_service_account` data source.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
data "google_project" "this" {
|
||||
project_id = var.project_id
|
||||
}
|
||||
|
||||
data "google_storage_project_service_account" "this" {
|
||||
project = var.project_id
|
||||
}
|
||||
|
||||
resource "google_kms_key_ring" "workload" {
|
||||
project = var.project_id
|
||||
name = "workload"
|
||||
location = var.region
|
||||
}
|
||||
|
||||
resource "google_kms_crypto_key" "data" {
|
||||
name = "data"
|
||||
key_ring = google_kms_key_ring.workload.id
|
||||
purpose = "ENCRYPT_DECRYPT"
|
||||
rotation_period = "7776000s"
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_kms_crypto_key_iam_member" "pubsub_uses_key" {
|
||||
crypto_key_id = google_kms_crypto_key.data.id
|
||||
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
|
||||
member = "serviceAccount:service-${data.google_project.this.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
|
||||
}
|
||||
|
||||
resource "google_kms_crypto_key_iam_member" "storage_uses_key" {
|
||||
crypto_key_id = google_kms_crypto_key.data.id
|
||||
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
|
||||
member = "serviceAccount:${data.google_storage_project_service_account.this.email_address}"
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic" "encrypted" {
|
||||
project = var.project_id
|
||||
name = "encrypted-events"
|
||||
kms_key_name = google_kms_crypto_key.data.id
|
||||
|
||||
depends_on = [google_kms_crypto_key_iam_member.pubsub_uses_key]
|
||||
}
|
||||
|
||||
resource "google_storage_bucket" "encrypted" {
|
||||
project = var.project_id
|
||||
name = "${var.project_id}-encrypted"
|
||||
location = var.region
|
||||
uniform_bucket_level_access = true
|
||||
|
||||
encryption {
|
||||
default_kms_key_name = google_kms_crypto_key.data.id
|
||||
}
|
||||
|
||||
depends_on = [google_kms_crypto_key_iam_member.storage_uses_key]
|
||||
}
|
||||
```
|
||||
184
skills/google-cloud-iac-patterns/references/load-balancer.md
Normal file
184
skills/google-cloud-iac-patterns/references/load-balancer.md
Normal file
@@ -0,0 +1,184 @@
|
||||
# External Application Load Balancer
|
||||
|
||||
A global HTTPS load balancer in front of a Cloud Run service, with a TLS policy, a web application firewall policy and, optionally, Identity-Aware Proxy.
|
||||
|
||||
The chain is: forwarding rule → target HTTPS proxy → URL map → backend service → serverless network endpoint group → Cloud Run service. Each setting lives on one link of it, and putting one on the wrong link is the usual mistake:
|
||||
|
||||
- **The minimum TLS version is on an SSL policy, attached to the target HTTPS proxy** (`ssl_policy`). A backend service has no `ssl_policy` setting.
|
||||
- **The certificate is on the target HTTPS proxy** (`ssl_certificates`).
|
||||
- **The firewall policy is attached to the backend service** (`security_policy`).
|
||||
- **Identity-Aware Proxy is the `iap` block of the backend service.** It decides who reaches the application; what a signed-in user may do inside it is the application's own code and cannot be shown by infrastructure.
|
||||
- **Request logging is the `log_config` block of the backend service.**
|
||||
- A backend service for Cloud Run uses a serverless NEG and takes no `health_checks`, no `port_name` and no `timeout_sec`.
|
||||
- `load_balancing_scheme = "EXTERNAL_MANAGED"` on both the backend service and the forwarding rule.
|
||||
- Set the Cloud Run service's `ingress` to `INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER`, or it stays reachable around the load balancer.
|
||||
- Redirecting HTTP to HTTPS needs a second URL map, target HTTP proxy and forwarding rule on port 80.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "domain" {
|
||||
type = string
|
||||
description = "Domain name the load balancer serves."
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_service" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
location = var.region
|
||||
ingress = "INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"
|
||||
|
||||
template {
|
||||
containers {
|
||||
image = "us-docker.pkg.dev/cloudrun/container/hello"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_region_network_endpoint_group" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
region = var.region
|
||||
network_endpoint_type = "SERVERLESS"
|
||||
|
||||
cloud_run {
|
||||
service = google_cloud_run_v2_service.app.name
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_security_policy" "edge" {
|
||||
project = var.project_id
|
||||
name = "booking-edge"
|
||||
|
||||
rule {
|
||||
action = "deny(403)"
|
||||
priority = 1000
|
||||
match {
|
||||
expr {
|
||||
expression = "evaluatePreconfiguredWaf('sqli-v33-stable', {'sensitivity': 1})"
|
||||
}
|
||||
}
|
||||
description = "Block SQL injection."
|
||||
}
|
||||
|
||||
rule {
|
||||
action = "allow"
|
||||
priority = 2147483647
|
||||
match {
|
||||
versioned_expr = "SRC_IPS_V1"
|
||||
config {
|
||||
src_ip_ranges = ["*"]
|
||||
}
|
||||
}
|
||||
description = "Default rule."
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_backend_service" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
protocol = "HTTPS"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
security_policy = google_compute_security_policy.edge.id
|
||||
|
||||
backend {
|
||||
group = google_compute_region_network_endpoint_group.app.id
|
||||
}
|
||||
|
||||
# Only signed-in identities granted roles/iap.httpsResourceAccessor get through.
|
||||
iap {
|
||||
enabled = true
|
||||
}
|
||||
|
||||
log_config {
|
||||
enable = true
|
||||
sample_rate = 1.0
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_url_map" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
default_service = google_compute_backend_service.app.id
|
||||
}
|
||||
|
||||
resource "google_compute_ssl_policy" "modern" {
|
||||
project = var.project_id
|
||||
name = "tls-1-2-modern"
|
||||
profile = "MODERN"
|
||||
min_tls_version = "TLS_1_2"
|
||||
}
|
||||
|
||||
resource "google_compute_managed_ssl_certificate" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
|
||||
managed {
|
||||
domains = [var.domain]
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_target_https_proxy" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
url_map = google_compute_url_map.app.id
|
||||
ssl_certificates = [google_compute_managed_ssl_certificate.app.id]
|
||||
ssl_policy = google_compute_ssl_policy.modern.id
|
||||
}
|
||||
|
||||
resource "google_compute_global_address" "app" {
|
||||
project = var.project_id
|
||||
name = "booking-app"
|
||||
}
|
||||
|
||||
resource "google_compute_global_forwarding_rule" "https" {
|
||||
project = var.project_id
|
||||
name = "booking-app-https"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
ip_address = google_compute_global_address.app.id
|
||||
port_range = "443"
|
||||
target = google_compute_target_https_proxy.app.id
|
||||
}
|
||||
|
||||
# HTTP to HTTPS.
|
||||
resource "google_compute_url_map" "redirect" {
|
||||
project = var.project_id
|
||||
name = "booking-app-redirect"
|
||||
|
||||
default_url_redirect {
|
||||
https_redirect = true
|
||||
strip_query = false
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_target_http_proxy" "redirect" {
|
||||
project = var.project_id
|
||||
name = "booking-app-redirect"
|
||||
url_map = google_compute_url_map.redirect.id
|
||||
}
|
||||
|
||||
resource "google_compute_global_forwarding_rule" "http" {
|
||||
project = var.project_id
|
||||
name = "booking-app-http"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
ip_address = google_compute_global_address.app.id
|
||||
port_range = "80"
|
||||
target = google_compute_target_http_proxy.redirect.id
|
||||
}
|
||||
|
||||
# Who may pass Identity-Aware Proxy.
|
||||
resource "google_iap_web_backend_service_iam_member" "staff" {
|
||||
project = var.project_id
|
||||
web_backend_service = google_compute_backend_service.app.name
|
||||
role = "roles/iap.httpsResourceAccessor"
|
||||
member = "domain:${var.domain}"
|
||||
}
|
||||
```
|
||||
107
skills/google-cloud-iac-patterns/references/monitoring.md
Normal file
107
skills/google-cloud-iac-patterns/references/monitoring.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Cloud Monitoring and Logging
|
||||
|
||||
A notification channel, an alert on a metric, a log-based metric and a log sink.
|
||||
|
||||
- An alert policy needs `combiner` and at least one `conditions` block. A threshold condition's `filter` must name both a `metric.type` and a `resource.type`.
|
||||
- Every `aggregations` block needs an `alignment_period` of at least `"60s"` and a `per_series_aligner` that fits the metric kind: `ALIGN_RATE` or `ALIGN_DELTA` for a delta or cumulative metric, `ALIGN_MEAN` or `ALIGN_MAX` for a gauge.
|
||||
- A log-based metric's name is used in an alert filter as `logging.googleapis.com/user/NAME`.
|
||||
- A sink's writer identity needs a grant on the destination, or nothing arrives.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "alert_email" {
|
||||
type = string
|
||||
description = "Address alerts are sent to."
|
||||
}
|
||||
|
||||
resource "google_monitoring_notification_channel" "email" {
|
||||
project = var.project_id
|
||||
display_name = "Operations email"
|
||||
type = "email"
|
||||
|
||||
labels = {
|
||||
email_address = var.alert_email
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_monitoring_alert_policy" "server_errors" {
|
||||
project = var.project_id
|
||||
display_name = "Cloud Run 5xx responses"
|
||||
combiner = "OR"
|
||||
|
||||
conditions {
|
||||
display_name = "5xx rate above 1 per second for 5 minutes"
|
||||
|
||||
condition_threshold {
|
||||
filter = "resource.type = \"cloud_run_revision\" AND metric.type = \"run.googleapis.com/request_count\" AND metric.labels.response_code_class = \"5xx\""
|
||||
comparison = "COMPARISON_GT"
|
||||
threshold_value = 1
|
||||
duration = "300s"
|
||||
|
||||
aggregations {
|
||||
alignment_period = "60s"
|
||||
per_series_aligner = "ALIGN_RATE"
|
||||
cross_series_reducer = "REDUCE_SUM"
|
||||
group_by_fields = ["resource.labels.service_name"]
|
||||
}
|
||||
|
||||
trigger {
|
||||
count = 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
notification_channels = [google_monitoring_notification_channel.email.name]
|
||||
|
||||
alert_strategy {
|
||||
auto_close = "1800s"
|
||||
}
|
||||
|
||||
documentation {
|
||||
content = "The service is returning server errors. Check the latest revision's logs."
|
||||
mime_type = "text/markdown"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_logging_metric" "rejected_signatures" {
|
||||
project = var.project_id
|
||||
name = "rejected_signatures"
|
||||
filter = "resource.type = \"cloud_run_revision\" AND jsonPayload.event = \"signature_rejected\""
|
||||
|
||||
metric_descriptor {
|
||||
metric_kind = "DELTA"
|
||||
value_type = "INT64"
|
||||
unit = "1"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_storage_bucket" "audit_logs" {
|
||||
project = var.project_id
|
||||
name = "${var.project_id}-audit-logs"
|
||||
location = var.region
|
||||
uniform_bucket_level_access = true
|
||||
}
|
||||
|
||||
resource "google_logging_project_sink" "audit" {
|
||||
project = var.project_id
|
||||
name = "audit-to-storage"
|
||||
destination = "storage.googleapis.com/${google_storage_bucket.audit_logs.name}"
|
||||
filter = "logName:\"cloudaudit.googleapis.com\""
|
||||
unique_writer_identity = true
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_iam_member" "sink_writes" {
|
||||
bucket = google_storage_bucket.audit_logs.name
|
||||
role = "roles/storage.objectCreator"
|
||||
member = google_logging_project_sink.audit.writer_identity
|
||||
}
|
||||
```
|
||||
102
skills/google-cloud-iac-patterns/references/networking.md
Normal file
102
skills/google-cloud-iac-patterns/references/networking.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# VPC networking
|
||||
|
||||
A custom-mode VPC, a subnet, outbound internet through Cloud NAT, and a firewall rule.
|
||||
|
||||
- `auto_create_subnetworks = false` gives a custom-mode network: only the subnets you declare.
|
||||
- `private_ip_google_access = true` lets workloads without external addresses reach Google APIs.
|
||||
- Cloud Run reaches the VPC with Direct VPC egress: `template.vpc_access.network_interfaces` naming this network and subnet. It needs no connector resource.
|
||||
- Firewall rules match by `target_service_accounts` or `target_tags`. A rule with neither applies to the whole network.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
resource "google_compute_network" "main" {
|
||||
project = var.project_id
|
||||
name = "main"
|
||||
auto_create_subnetworks = false
|
||||
routing_mode = "REGIONAL"
|
||||
}
|
||||
|
||||
resource "google_compute_subnetwork" "workloads" {
|
||||
project = var.project_id
|
||||
name = "workloads"
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
ip_cidr_range = "10.10.0.0/24"
|
||||
private_ip_google_access = true
|
||||
|
||||
log_config {
|
||||
aggregation_interval = "INTERVAL_5_SEC"
|
||||
flow_sampling = 0.5
|
||||
metadata = "INCLUDE_ALL_METADATA"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_router" "egress" {
|
||||
project = var.project_id
|
||||
name = "egress"
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
}
|
||||
|
||||
resource "google_compute_router_nat" "egress" {
|
||||
project = var.project_id
|
||||
name = "egress"
|
||||
region = var.region
|
||||
router = google_compute_router.egress.name
|
||||
nat_ip_allocate_option = "AUTO_ONLY"
|
||||
source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
|
||||
|
||||
log_config {
|
||||
enable = true
|
||||
filter = "ERRORS_ONLY"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_firewall" "allow_health_checks" {
|
||||
project = var.project_id
|
||||
name = "allow-health-checks"
|
||||
network = google_compute_network.main.id
|
||||
direction = "INGRESS"
|
||||
priority = 1000
|
||||
|
||||
# Google Cloud load balancer health check ranges.
|
||||
source_ranges = ["130.211.0.0/22", "35.191.0.0/16"]
|
||||
target_tags = ["web"]
|
||||
|
||||
allow {
|
||||
protocol = "tcp"
|
||||
ports = ["8080"]
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_service" "internal" {
|
||||
project = var.project_id
|
||||
name = "internal"
|
||||
location = var.region
|
||||
ingress = "INGRESS_TRAFFIC_INTERNAL_ONLY"
|
||||
|
||||
template {
|
||||
containers {
|
||||
image = "us-docker.pkg.dev/cloudrun/container/hello"
|
||||
}
|
||||
|
||||
vpc_access {
|
||||
egress = "PRIVATE_RANGES_ONLY"
|
||||
|
||||
network_interfaces {
|
||||
network = google_compute_network.main.id
|
||||
subnetwork = google_compute_subnetwork.workloads.id
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,33 @@
|
||||
# Enabling APIs
|
||||
|
||||
Enable each API the Elements need, once, and make the first resource of each product depend on it.
|
||||
|
||||
- `disable_on_destroy = false` keeps the API on when the configuration is destroyed; other workloads in the project may use it.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
locals {
|
||||
services = [
|
||||
"run.googleapis.com",
|
||||
"pubsub.googleapis.com",
|
||||
"secretmanager.googleapis.com",
|
||||
]
|
||||
}
|
||||
|
||||
resource "google_project_service" "enabled" {
|
||||
for_each = toset(local.services)
|
||||
|
||||
project = var.project_id
|
||||
service = each.value
|
||||
disable_on_destroy = false
|
||||
}
|
||||
```
|
||||
111
skills/google-cloud-iac-patterns/references/pubsub.md
Normal file
111
skills/google-cloud-iac-patterns/references/pubsub.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# Pub/Sub
|
||||
|
||||
A topic, a push subscription that calls a private Cloud Run service, and a dead-letter topic.
|
||||
|
||||
- A push subscription authenticates with `push_config.oidc_token`; its service account needs `roles/run.invoker` on the service it calls.
|
||||
- Dead-lettering needs two grants to the Pub/Sub service agent: `roles/pubsub.publisher` on the dead-letter topic and `roles/pubsub.subscriber` on the source subscription. Without them messages are never moved.
|
||||
- Give the dead-letter topic a subscription of its own, or the messages moved there are lost.
|
||||
- `max_delivery_attempts` is 5 to 100. `ack_deadline_seconds` is 10 to 600.
|
||||
- `message_retention_duration` is a duration in seconds, such as `"604800s"`.
|
||||
- Customer-managed encryption is `kms_key_name` on the topic; the Pub/Sub service agent needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key first.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "worker_url" {
|
||||
type = string
|
||||
description = "HTTPS URL of the service the subscription pushes to."
|
||||
}
|
||||
|
||||
data "google_project" "this" {
|
||||
project_id = var.project_id
|
||||
}
|
||||
|
||||
locals {
|
||||
pubsub_agent = "serviceAccount:service-${data.google_project.this.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
|
||||
}
|
||||
|
||||
resource "google_service_account" "push" {
|
||||
project = var.project_id
|
||||
account_id = "events-push"
|
||||
display_name = "Identity Pub/Sub pushes with"
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic" "events" {
|
||||
project = var.project_id
|
||||
name = "events"
|
||||
message_retention_duration = "604800s"
|
||||
|
||||
message_storage_policy {
|
||||
allowed_persistence_regions = [var.region]
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic" "dead_letter" {
|
||||
project = var.project_id
|
||||
name = "events-dead-letter"
|
||||
}
|
||||
|
||||
resource "google_pubsub_subscription" "dead_letter" {
|
||||
project = var.project_id
|
||||
name = "events-dead-letter"
|
||||
topic = google_pubsub_topic.dead_letter.id
|
||||
message_retention_duration = "1209600s"
|
||||
|
||||
expiration_policy {
|
||||
ttl = ""
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_pubsub_subscription" "worker" {
|
||||
project = var.project_id
|
||||
name = "events-worker"
|
||||
topic = google_pubsub_topic.events.id
|
||||
ack_deadline_seconds = 60
|
||||
|
||||
push_config {
|
||||
push_endpoint = var.worker_url
|
||||
|
||||
oidc_token {
|
||||
service_account_email = google_service_account.push.email
|
||||
audience = var.worker_url
|
||||
}
|
||||
}
|
||||
|
||||
retry_policy {
|
||||
minimum_backoff = "10s"
|
||||
maximum_backoff = "600s"
|
||||
}
|
||||
|
||||
dead_letter_policy {
|
||||
dead_letter_topic = google_pubsub_topic.dead_letter.id
|
||||
max_delivery_attempts = 5
|
||||
}
|
||||
|
||||
expiration_policy {
|
||||
ttl = ""
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic_iam_member" "agent_publishes_dead_letters" {
|
||||
project = var.project_id
|
||||
topic = google_pubsub_topic.dead_letter.name
|
||||
role = "roles/pubsub.publisher"
|
||||
member = local.pubsub_agent
|
||||
}
|
||||
|
||||
resource "google_pubsub_subscription_iam_member" "agent_forwards_from_worker" {
|
||||
project = var.project_id
|
||||
subscription = google_pubsub_subscription.worker.name
|
||||
role = "roles/pubsub.subscriber"
|
||||
member = local.pubsub_agent
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,45 @@
|
||||
# Secret Manager
|
||||
|
||||
Terraform creates the secret and who may read it. The value is added outside Terraform, so it never enters the state or the repository.
|
||||
|
||||
- `replication` is required: `auto {}` for Google-managed locations, or `user_managed` with one `replicas` block per location.
|
||||
- Grant `roles/secretmanager.secretAccessor` on the secret, to the one identity that reads it.
|
||||
- Never write a `google_secret_manager_secret_version` with the value in the configuration.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
resource "google_service_account" "reader" {
|
||||
project = var.project_id
|
||||
account_id = "api-key-reader"
|
||||
display_name = "Reads the API key"
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret" "api_key" {
|
||||
project = var.project_id
|
||||
secret_id = "partner-api-key"
|
||||
|
||||
replication {
|
||||
user_managed {
|
||||
replicas {
|
||||
location = var.region
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_secret_manager_secret_iam_member" "reader_reads_api_key" {
|
||||
project = var.project_id
|
||||
secret_id = google_secret_manager_secret.api_key.secret_id
|
||||
role = "roles/secretmanager.secretAccessor"
|
||||
member = google_service_account.reader.member
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user