fixed multile issues
Some checks failed
validation / verify (push) Failing after 8s

This commit is contained in:
2026-10-07 17:33:36 +01:00
parent 4c4d44ef87
commit 5bb22d984e
76 changed files with 5799 additions and 179 deletions

View File

@@ -0,0 +1,49 @@
---
name: google-cloud-iac-patterns
description: >-
Terraform patterns for Google Cloud products that validate against one pinned
version of the hashicorp/google provider: Cloud Run services and jobs,
Pub/Sub, Cloud Storage, IAM, Secret Manager, Cloud KMS, Cloud SQL,
networking, Cloud Scheduler, Artifact Registry, Monitoring and API
enablement. Use when writing Terraform for a Google Cloud product, before
writing the resource, to start from a pattern whose arguments are known to be
in the right place.
metadata:
scope: deployment guidance
provider: hashicorp/google 8.6.0
---
# Google Cloud IaC patterns
Every pattern here passes `terraform validate` against `hashicorp/google` 8.6.0. A test runs that on each of them, so an argument you copy from a pattern is in the right block.
## How to use them
1. Write `assets/versions.tf` to your working copy as `terraform/versions.tf`, unchanged. It pins the provider version these patterns and the `terraform_schema` tool describe. Declare the `project_id` and `region` variables it uses in `terraform/variables.tf`.
2. For each Element in the Target architecture, read the reference for its product with `load_skill_resource` and start from its pattern. Read only the references for products the Target architecture uses.
3. Change the pattern to meet the Element's Checks. For a setting the pattern does not show, call `terraform_schema` with `find` set to its name before you write it.
| Product | Reference |
| --- | --- |
| Enabling APIs | `references/project-services.md` |
| Service accounts and IAM grants | `references/iam.md` |
| Cloud Run service | `references/cloud-run-service.md` |
| Cloud Run job | `references/cloud-run-job.md` |
| Pub/Sub | `references/pubsub.md` |
| Cloud Storage | `references/cloud-storage.md` |
| Secret Manager | `references/secret-manager.md` |
| Cloud KMS | `references/kms.md` |
| Cloud SQL | `references/cloud-sql.md` |
| VPC networking | `references/networking.md` |
| Cloud Scheduler | `references/cloud-scheduler.md` |
| Artifact Registry | `references/artifact-registry.md` |
| Cloud Monitoring and Logging | `references/monitoring.md` |
A product with no reference has no pattern yet: write it from `terraform_schema` and the product documentation.
## Rules that hold for every product
- A Cloud Run service and a Cloud Run job are different resource types with different arguments. A setting of one is not a setting of the other. `references/cloud-run-job.md` lists what a job lacks.
- Grant a role with an `_iam_member` resource. An `_iam_binding` or `_iam_policy` resource removes grants made elsewhere.
- Give a Google-managed service agent its role before the resource that needs it, with `depends_on`. A key, a dead-letter topic and a push subscription all need one.
- Set `deletion_protection` from a variable on the resources that have it, so a test deployment can be removed.

View File

@@ -0,0 +1,15 @@
terraform {
required_version = ">= 1.9"
required_providers {
google = {
source = "hashicorp/google"
version = "8.6.0"
}
}
}
provider "google" {
project = var.project_id
region = var.region
}

View File

@@ -0,0 +1,65 @@
# Artifact Registry
A Docker repository with a cleanup policy, and the grant that lets a runtime identity pull from it.
- `format = "DOCKER"` for container images. The image path is `REGION-docker.pkg.dev/PROJECT/REPOSITORY/IMAGE`.
- Cleanup policies delete nothing while `cleanup_policy_dry_run` is `true`.
- `docker_config.immutable_tags = true` stops a tag being moved to a different image.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
resource "google_service_account" "runtime" {
project = var.project_id
account_id = "image-puller"
display_name = "Pulls workload images"
}
resource "google_artifact_registry_repository" "images" {
project = var.project_id
location = var.region
repository_id = "workload-images"
format = "DOCKER"
description = "Container images for the workload."
cleanup_policy_dry_run = false
docker_config {
immutable_tags = true
}
cleanup_policies {
id = "keep-recent"
action = "KEEP"
most_recent_versions {
keep_count = 10
}
}
cleanup_policies {
id = "delete-old-untagged"
action = "DELETE"
condition {
tag_state = "UNTAGGED"
older_than = "2592000s"
}
}
}
resource "google_artifact_registry_repository_iam_member" "runtime_pulls" {
project = var.project_id
location = google_artifact_registry_repository.images.location
repository = google_artifact_registry_repository.images.name
role = "roles/artifactregistry.reader"
member = google_service_account.runtime.member
}
```

View File

@@ -0,0 +1,95 @@
# Cloud Run job
`google_cloud_run_v2_job`: a task that runs to completion. It is a different resource type from a service, and its template is nested one level deeper.
What a job does not have. A Check that asks for one of these on a job cannot be met:
- `startup_cpu_boost` and `cpu_idle`. A job's `resources` block has `limits` only.
- `ingress`, `traffic`, `scaling`, `min_instance_count`, `max_instance_request_concurrency`. A job answers no requests.
- `liveness_probe`. A job's container has a `startup_probe` only.
Where a job's settings sit:
- `task_count` and `parallelism` are in `template`.
- `service_account`, `timeout`, `max_retries`, `execution_environment`, `vpc_access` and `encryption_key` are in `template.template`.
- `image`, `env`, `args`, `command` and `resources` are in `template.template.containers`.
- Whoever starts the job needs `roles/run.invoker` on it; Cloud Scheduler starts it with an HTTP call to the `:run` method.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "image" {
type = string
description = "Container image reference, including the tag or digest."
}
variable "deletion_protection" {
type = bool
description = "Whether Terraform is prevented from deleting the job."
default = true
}
resource "google_service_account" "replay" {
project = var.project_id
account_id = "replay-job"
display_name = "Replay job"
}
resource "google_cloud_run_v2_job" "replay" {
project = var.project_id
name = "replay"
location = var.region
deletion_protection = var.deletion_protection
template {
task_count = 1
parallelism = 1
template {
service_account = google_service_account.replay.email
timeout = "3600s"
max_retries = 3
execution_environment = "EXECUTION_ENVIRONMENT_GEN2"
containers {
image = var.image
args = ["--mode=replay"]
resources {
limits = {
cpu = "1"
memory = "1Gi"
}
}
env {
name = "BATCH_SIZE"
value = "100"
}
}
}
}
}
resource "google_service_account" "operator" {
project = var.project_id
account_id = "replay-operator"
display_name = "Starts the replay job"
}
resource "google_cloud_run_v2_job_iam_member" "operator_runs_replay" {
project = var.project_id
location = google_cloud_run_v2_job.replay.location
name = google_cloud_run_v2_job.replay.name
role = "roles/run.invoker"
member = google_service_account.operator.member
}
```

View File

@@ -0,0 +1,144 @@
# Cloud Run service
`google_cloud_run_v2_service`: a long-running service that answers requests.
Where the settings sit, which is what goes wrong most often:
- `ingress`, `deletion_protection` and `traffic` are on the resource.
- `service_account`, `scaling`, `timeout`, `max_instance_request_concurrency`, `execution_environment`, `vpc_access` and `encryption_key` are in `template`.
- `image`, `env`, `ports`, `resources` and the probes are in `template.containers`.
- `cpu_idle` and `startup_cpu_boost` are in `template.containers.resources`, beside `limits`. They exist on a service only.
- `min_instance_count` and `max_instance_count` are in `template.scaling`.
- A secret is read through `env.value_source.secret_key_ref`; the service account needs `roles/secretmanager.secretAccessor` on that secret.
- Direct VPC egress is `template.vpc_access.network_interfaces`. A connector is `template.vpc_access.connector`. Use one of them.
- Public access is a grant of `roles/run.invoker` to `allUsers` together with `ingress = "INGRESS_TRAFFIC_ALL"`. A private service has neither.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "image" {
type = string
description = "Container image reference, including the tag or digest."
}
variable "deletion_protection" {
type = bool
description = "Whether Terraform is prevented from deleting the service."
default = true
}
resource "google_service_account" "ingest" {
project = var.project_id
account_id = "ingest-service"
display_name = "Ingest service"
}
resource "google_secret_manager_secret" "signing_key" {
project = var.project_id
secret_id = "webhook-signing-key"
replication {
auto {}
}
}
resource "google_secret_manager_secret_iam_member" "ingest_reads_signing_key" {
project = var.project_id
secret_id = google_secret_manager_secret.signing_key.secret_id
role = "roles/secretmanager.secretAccessor"
member = google_service_account.ingest.member
}
resource "google_cloud_run_v2_service" "ingest" {
project = var.project_id
name = "ingest"
location = var.region
ingress = "INGRESS_TRAFFIC_ALL"
deletion_protection = var.deletion_protection
template {
service_account = google_service_account.ingest.email
timeout = "30s"
max_instance_request_concurrency = 80
execution_environment = "EXECUTION_ENVIRONMENT_GEN2"
scaling {
min_instance_count = 1
max_instance_count = 10
}
containers {
image = var.image
ports {
container_port = 8080
}
resources {
limits = {
cpu = "1"
memory = "512Mi"
}
cpu_idle = true
startup_cpu_boost = true
}
env {
name = "LOG_LEVEL"
value = "info"
}
env {
name = "SIGNING_KEY"
value_source {
secret_key_ref {
secret = google_secret_manager_secret.signing_key.secret_id
version = "latest"
}
}
}
startup_probe {
initial_delay_seconds = 0
period_seconds = 5
failure_threshold = 6
http_get {
path = "/healthz"
}
}
liveness_probe {
period_seconds = 30
http_get {
path = "/healthz"
}
}
}
}
traffic {
type = "TRAFFIC_TARGET_ALLOCATION_TYPE_LATEST"
percent = 100
}
depends_on = [google_secret_manager_secret_iam_member.ingest_reads_signing_key]
}
# Public access. Leave this out, and set ingress to INGRESS_TRAFFIC_INTERNAL_ONLY,
# for a service only other workloads call.
resource "google_cloud_run_v2_service_iam_member" "public" {
project = var.project_id
location = google_cloud_run_v2_service.ingest.location
name = google_cloud_run_v2_service.ingest.name
role = "roles/run.invoker"
member = "allUsers"
}
```

View File

@@ -0,0 +1,75 @@
# Cloud Scheduler
A schedule that starts a Cloud Run job, and one that calls a private Cloud Run service.
- A job is started by an HTTP POST to its `:run` method with an OAuth token (`oauth_token`). A service is called with an OIDC token (`oidc_token`). Using the wrong one is refused at run time, not by `terraform validate`.
- The schedule's service account needs `roles/run.invoker` on the job or service it starts.
- `schedule` is unix-cron; `time_zone` is an IANA name.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "job_name" {
type = string
description = "Name of the Cloud Run job to start."
}
variable "service_url" {
type = string
description = "HTTPS URL of the private Cloud Run service to call."
}
resource "google_service_account" "scheduler" {
project = var.project_id
account_id = "nightly-scheduler"
display_name = "Identity of the nightly schedules"
}
resource "google_cloud_scheduler_job" "start_job" {
project = var.project_id
region = var.region
name = "start-nightly-job"
schedule = "0 2 * * *"
time_zone = "Etc/UTC"
attempt_deadline = "180s"
retry_config {
retry_count = 3
}
http_target {
http_method = "POST"
uri = "https://run.googleapis.com/v2/projects/${var.project_id}/locations/${var.region}/jobs/${var.job_name}:run"
oauth_token {
service_account_email = google_service_account.scheduler.email
}
}
}
resource "google_cloud_scheduler_job" "call_service" {
project = var.project_id
region = var.region
name = "call-nightly-service"
schedule = "30 2 * * *"
time_zone = "Etc/UTC"
http_target {
http_method = "POST"
uri = "${var.service_url}/tasks/nightly"
oidc_token {
service_account_email = google_service_account.scheduler.email
audience = var.service_url
}
}
}
```

View File

@@ -0,0 +1,124 @@
# Cloud SQL
A PostgreSQL instance with a private address, backups and IAM database authentication.
- A private address needs a reserved range and a service networking connection on the VPC first; the instance must `depends_on` the connection.
- `availability_type = "REGIONAL"` is high availability; `"ZONAL"` is a single zone.
- Backups, point-in-time recovery, flags and the address are all in `settings`.
- `deletion_protection` on the resource guards Terraform; `settings.deletion_protection_enabled` guards the API.
- Take the database password from Secret Manager at run time, or use IAM database authentication as here. Never put a password in the configuration.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "deletion_protection" {
type = bool
description = "Whether Terraform is prevented from deleting the instance."
default = true
}
resource "google_compute_network" "main" {
project = var.project_id
name = "main"
auto_create_subnetworks = false
}
resource "google_compute_global_address" "private_services" {
project = var.project_id
name = "private-services"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 16
network = google_compute_network.main.id
}
resource "google_service_networking_connection" "private_services" {
network = google_compute_network.main.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_services.name]
}
resource "google_service_account" "app" {
project = var.project_id
account_id = "orders-app"
display_name = "Orders application"
}
resource "google_sql_database_instance" "orders" {
project = var.project_id
name = "orders"
region = var.region
database_version = "POSTGRES_16"
deletion_protection = var.deletion_protection
settings {
tier = "db-custom-2-7680"
edition = "ENTERPRISE"
availability_type = "REGIONAL"
disk_type = "PD_SSD"
disk_autoresize = true
deletion_protection_enabled = var.deletion_protection
ip_configuration {
ipv4_enabled = false
private_network = google_compute_network.main.id
ssl_mode = "ENCRYPTED_ONLY"
}
backup_configuration {
enabled = true
point_in_time_recovery_enabled = true
start_time = "02:00"
backup_retention_settings {
retained_backups = 14
}
}
database_flags {
name = "cloudsql.iam_authentication"
value = "on"
}
maintenance_window {
day = 7
hour = 3
}
}
depends_on = [google_service_networking_connection.private_services]
}
resource "google_sql_database" "orders" {
project = var.project_id
name = "orders"
instance = google_sql_database_instance.orders.name
}
resource "google_sql_user" "app" {
project = var.project_id
name = trimsuffix(google_service_account.app.email, ".gserviceaccount.com")
instance = google_sql_database_instance.orders.name
type = "CLOUD_IAM_SERVICE_ACCOUNT"
}
resource "google_project_iam_member" "app_connects" {
project = var.project_id
role = "roles/cloudsql.client"
member = google_service_account.app.member
}
resource "google_project_iam_member" "app_logs_in" {
project = var.project_id
role = "roles/cloudsql.instanceUser"
member = google_service_account.app.member
}
```

View File

@@ -0,0 +1,67 @@
# Cloud Storage
A private bucket with a retention period and a lifecycle.
- Bucket names are global. Build the name from the project id.
- `uniform_bucket_level_access = true` and `public_access_prevention = "enforced"` keep the bucket private.
- `retention_policy.retention_period` is in seconds and stops objects being deleted before then. A lifecycle `Delete` rule with `age` in days removes them after. Retaining for 30 days and then deleting needs both.
- Versioning and a retention policy cannot be used on the same bucket.
- Customer-managed encryption is `encryption.default_kms_key_name`; the Cloud Storage service agent needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key first.
- `force_destroy = false` stops Terraform deleting a bucket that holds objects.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
resource "google_service_account" "writer" {
project = var.project_id
account_id = "raw-events-writer"
display_name = "Writes raw events"
}
resource "google_storage_bucket" "raw_events" {
project = var.project_id
name = "${var.project_id}-raw-events"
location = var.region
storage_class = "STANDARD"
uniform_bucket_level_access = true
public_access_prevention = "enforced"
force_destroy = false
retention_policy {
retention_period = 2592000
}
lifecycle_rule {
condition {
age = 31
}
action {
type = "Delete"
}
}
lifecycle_rule {
condition {
age = 7
}
action {
type = "SetStorageClass"
storage_class = "NEARLINE"
}
}
}
resource "google_storage_bucket_iam_member" "writer_creates_objects" {
bucket = google_storage_bucket.raw_events.name
role = "roles/storage.objectCreator"
member = google_service_account.writer.member
}
```

View File

@@ -0,0 +1,40 @@
# Service accounts and IAM grants
One service account per workload identity, and one `_iam_member` resource per grant.
- `account_id` is 6 to 30 characters: lower-case letters, digits and hyphens.
- Use the narrowest resource the role can be granted on: the bucket, topic, secret or service, in preference to the project.
- A grant to a service account is `member = "serviceAccount:${...email}"`. The same value is the `member` attribute of the service account resource.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
resource "google_service_account" "worker" {
project = var.project_id
account_id = "event-worker"
display_name = "Event worker"
description = "Identity of the worker service."
}
# A project-level role, only when no narrower resource can take the grant.
resource "google_project_iam_member" "worker_metrics" {
project = var.project_id
role = "roles/monitoring.metricWriter"
member = google_service_account.worker.member
}
# Who may act as the service account, for example a deployment pipeline.
resource "google_service_account_iam_member" "deployer_acts_as_worker" {
service_account_id = google_service_account.worker.name
role = "roles/iam.serviceAccountUser"
member = "serviceAccount:deployer@${var.project_id}.iam.gserviceaccount.com"
}
```

View File

@@ -0,0 +1,79 @@
# Cloud KMS
A key ring and a key for customer-managed encryption, and the grant that lets a product use it.
- The key ring must be in the same location as the resource it protects.
- Key rings and keys cannot be deleted. `prevent_destroy` stops Terraform trying.
- `rotation_period` is in seconds, such as `"7776000s"` for 90 days.
- The product's service agent, not the workload's service account, needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key. The resource that uses the key must `depends_on` that grant.
- Service agents: Pub/Sub is `service-NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com`, Cloud Run is `service-NUMBER@serverless-robot-prod.iam.gserviceaccount.com`, Cloud Storage comes from the `google_storage_project_service_account` data source.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
data "google_project" "this" {
project_id = var.project_id
}
data "google_storage_project_service_account" "this" {
project = var.project_id
}
resource "google_kms_key_ring" "workload" {
project = var.project_id
name = "workload"
location = var.region
}
resource "google_kms_crypto_key" "data" {
name = "data"
key_ring = google_kms_key_ring.workload.id
purpose = "ENCRYPT_DECRYPT"
rotation_period = "7776000s"
lifecycle {
prevent_destroy = true
}
}
resource "google_kms_crypto_key_iam_member" "pubsub_uses_key" {
crypto_key_id = google_kms_crypto_key.data.id
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
member = "serviceAccount:service-${data.google_project.this.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}
resource "google_kms_crypto_key_iam_member" "storage_uses_key" {
crypto_key_id = google_kms_crypto_key.data.id
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
member = "serviceAccount:${data.google_storage_project_service_account.this.email_address}"
}
resource "google_pubsub_topic" "encrypted" {
project = var.project_id
name = "encrypted-events"
kms_key_name = google_kms_crypto_key.data.id
depends_on = [google_kms_crypto_key_iam_member.pubsub_uses_key]
}
resource "google_storage_bucket" "encrypted" {
project = var.project_id
name = "${var.project_id}-encrypted"
location = var.region
uniform_bucket_level_access = true
encryption {
default_kms_key_name = google_kms_crypto_key.data.id
}
depends_on = [google_kms_crypto_key_iam_member.storage_uses_key]
}
```

View File

@@ -0,0 +1,107 @@
# Cloud Monitoring and Logging
A notification channel, an alert on a metric, a log-based metric and a log sink.
- An alert policy needs `combiner` and at least one `conditions` block. A threshold condition's `filter` must name both a `metric.type` and a `resource.type`.
- Every `aggregations` block needs an `alignment_period` of at least `"60s"` and a `per_series_aligner` that fits the metric kind: `ALIGN_RATE` or `ALIGN_DELTA` for a delta or cumulative metric, `ALIGN_MEAN` or `ALIGN_MAX` for a gauge.
- A log-based metric's name is used in an alert filter as `logging.googleapis.com/user/NAME`.
- A sink's writer identity needs a grant on the destination, or nothing arrives.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "alert_email" {
type = string
description = "Address alerts are sent to."
}
resource "google_monitoring_notification_channel" "email" {
project = var.project_id
display_name = "Operations email"
type = "email"
labels = {
email_address = var.alert_email
}
}
resource "google_monitoring_alert_policy" "server_errors" {
project = var.project_id
display_name = "Cloud Run 5xx responses"
combiner = "OR"
conditions {
display_name = "5xx rate above 1 per second for 5 minutes"
condition_threshold {
filter = "resource.type = \"cloud_run_revision\" AND metric.type = \"run.googleapis.com/request_count\" AND metric.labels.response_code_class = \"5xx\""
comparison = "COMPARISON_GT"
threshold_value = 1
duration = "300s"
aggregations {
alignment_period = "60s"
per_series_aligner = "ALIGN_RATE"
cross_series_reducer = "REDUCE_SUM"
group_by_fields = ["resource.labels.service_name"]
}
trigger {
count = 1
}
}
}
notification_channels = [google_monitoring_notification_channel.email.name]
alert_strategy {
auto_close = "1800s"
}
documentation {
content = "The service is returning server errors. Check the latest revision's logs."
mime_type = "text/markdown"
}
}
resource "google_logging_metric" "rejected_signatures" {
project = var.project_id
name = "rejected_signatures"
filter = "resource.type = \"cloud_run_revision\" AND jsonPayload.event = \"signature_rejected\""
metric_descriptor {
metric_kind = "DELTA"
value_type = "INT64"
unit = "1"
}
}
resource "google_storage_bucket" "audit_logs" {
project = var.project_id
name = "${var.project_id}-audit-logs"
location = var.region
uniform_bucket_level_access = true
}
resource "google_logging_project_sink" "audit" {
project = var.project_id
name = "audit-to-storage"
destination = "storage.googleapis.com/${google_storage_bucket.audit_logs.name}"
filter = "logName:\"cloudaudit.googleapis.com\""
unique_writer_identity = true
}
resource "google_storage_bucket_iam_member" "sink_writes" {
bucket = google_storage_bucket.audit_logs.name
role = "roles/storage.objectCreator"
member = google_logging_project_sink.audit.writer_identity
}
```

View File

@@ -0,0 +1,102 @@
# VPC networking
A custom-mode VPC, a subnet, outbound internet through Cloud NAT, and a firewall rule.
- `auto_create_subnetworks = false` gives a custom-mode network: only the subnets you declare.
- `private_ip_google_access = true` lets workloads without external addresses reach Google APIs.
- Cloud Run reaches the VPC with Direct VPC egress: `template.vpc_access.network_interfaces` naming this network and subnet. It needs no connector resource.
- Firewall rules match by `target_service_accounts` or `target_tags`. A rule with neither applies to the whole network.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
resource "google_compute_network" "main" {
project = var.project_id
name = "main"
auto_create_subnetworks = false
routing_mode = "REGIONAL"
}
resource "google_compute_subnetwork" "workloads" {
project = var.project_id
name = "workloads"
region = var.region
network = google_compute_network.main.id
ip_cidr_range = "10.10.0.0/24"
private_ip_google_access = true
log_config {
aggregation_interval = "INTERVAL_5_SEC"
flow_sampling = 0.5
metadata = "INCLUDE_ALL_METADATA"
}
}
resource "google_compute_router" "egress" {
project = var.project_id
name = "egress"
region = var.region
network = google_compute_network.main.id
}
resource "google_compute_router_nat" "egress" {
project = var.project_id
name = "egress"
region = var.region
router = google_compute_router.egress.name
nat_ip_allocate_option = "AUTO_ONLY"
source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
log_config {
enable = true
filter = "ERRORS_ONLY"
}
}
resource "google_compute_firewall" "allow_health_checks" {
project = var.project_id
name = "allow-health-checks"
network = google_compute_network.main.id
direction = "INGRESS"
priority = 1000
# Google Cloud load balancer health check ranges.
source_ranges = ["130.211.0.0/22", "35.191.0.0/16"]
target_tags = ["web"]
allow {
protocol = "tcp"
ports = ["8080"]
}
}
resource "google_cloud_run_v2_service" "internal" {
project = var.project_id
name = "internal"
location = var.region
ingress = "INGRESS_TRAFFIC_INTERNAL_ONLY"
template {
containers {
image = "us-docker.pkg.dev/cloudrun/container/hello"
}
vpc_access {
egress = "PRIVATE_RANGES_ONLY"
network_interfaces {
network = google_compute_network.main.id
subnetwork = google_compute_subnetwork.workloads.id
}
}
}
}
```

View File

@@ -0,0 +1,33 @@
# Enabling APIs
Enable each API the Elements need, once, and make the first resource of each product depend on it.
- `disable_on_destroy = false` keeps the API on when the configuration is destroyed; other workloads in the project may use it.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
locals {
services = [
"run.googleapis.com",
"pubsub.googleapis.com",
"secretmanager.googleapis.com",
]
}
resource "google_project_service" "enabled" {
for_each = toset(local.services)
project = var.project_id
service = each.value
disable_on_destroy = false
}
```

View File

@@ -0,0 +1,111 @@
# Pub/Sub
A topic, a push subscription that calls a private Cloud Run service, and a dead-letter topic.
- A push subscription authenticates with `push_config.oidc_token`; its service account needs `roles/run.invoker` on the service it calls.
- Dead-lettering needs two grants to the Pub/Sub service agent: `roles/pubsub.publisher` on the dead-letter topic and `roles/pubsub.subscriber` on the source subscription. Without them messages are never moved.
- Give the dead-letter topic a subscription of its own, or the messages moved there are lost.
- `max_delivery_attempts` is 5 to 100. `ack_deadline_seconds` is 10 to 600.
- `message_retention_duration` is a duration in seconds, such as `"604800s"`.
- Customer-managed encryption is `kms_key_name` on the topic; the Pub/Sub service agent needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key first.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "worker_url" {
type = string
description = "HTTPS URL of the service the subscription pushes to."
}
data "google_project" "this" {
project_id = var.project_id
}
locals {
pubsub_agent = "serviceAccount:service-${data.google_project.this.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}
resource "google_service_account" "push" {
project = var.project_id
account_id = "events-push"
display_name = "Identity Pub/Sub pushes with"
}
resource "google_pubsub_topic" "events" {
project = var.project_id
name = "events"
message_retention_duration = "604800s"
message_storage_policy {
allowed_persistence_regions = [var.region]
}
}
resource "google_pubsub_topic" "dead_letter" {
project = var.project_id
name = "events-dead-letter"
}
resource "google_pubsub_subscription" "dead_letter" {
project = var.project_id
name = "events-dead-letter"
topic = google_pubsub_topic.dead_letter.id
message_retention_duration = "1209600s"
expiration_policy {
ttl = ""
}
}
resource "google_pubsub_subscription" "worker" {
project = var.project_id
name = "events-worker"
topic = google_pubsub_topic.events.id
ack_deadline_seconds = 60
push_config {
push_endpoint = var.worker_url
oidc_token {
service_account_email = google_service_account.push.email
audience = var.worker_url
}
}
retry_policy {
minimum_backoff = "10s"
maximum_backoff = "600s"
}
dead_letter_policy {
dead_letter_topic = google_pubsub_topic.dead_letter.id
max_delivery_attempts = 5
}
expiration_policy {
ttl = ""
}
}
resource "google_pubsub_topic_iam_member" "agent_publishes_dead_letters" {
project = var.project_id
topic = google_pubsub_topic.dead_letter.name
role = "roles/pubsub.publisher"
member = local.pubsub_agent
}
resource "google_pubsub_subscription_iam_member" "agent_forwards_from_worker" {
project = var.project_id
subscription = google_pubsub_subscription.worker.name
role = "roles/pubsub.subscriber"
member = local.pubsub_agent
}
```

View File

@@ -0,0 +1,45 @@
# Secret Manager
Terraform creates the secret and who may read it. The value is added outside Terraform, so it never enters the state or the repository.
- `replication` is required: `auto {}` for Google-managed locations, or `user_managed` with one `replicas` block per location.
- Grant `roles/secretmanager.secretAccessor` on the secret, to the one identity that reads it.
- Never write a `google_secret_manager_secret_version` with the value in the configuration.
```hcl
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
resource "google_service_account" "reader" {
project = var.project_id
account_id = "api-key-reader"
display_name = "Reads the API key"
}
resource "google_secret_manager_secret" "api_key" {
project = var.project_id
secret_id = "partner-api-key"
replication {
user_managed {
replicas {
location = var.region
}
}
}
}
resource "google_secret_manager_secret_iam_member" "reader_reads_api_key" {
project = var.project_id
secret_id = google_secret_manager_secret.api_key.secret_id
role = "roles/secretmanager.secretAccessor"
member = google_service_account.reader.member
}
```

View File

@@ -2,31 +2,35 @@
name: iac-validation
description: >-
Checks Terraform infrastructure-as-code without provisioning anything, using
the local terraform_fmt, terraform_validate and terraform_plan tools. Use when
writing deployment guidance or validating it before deployment.
the local terraform_fmt, terraform_validate, terraform_plan and
terraform_schema tools. Use when writing deployment guidance or validating it
before deployment.
metadata:
scope: deployment-guidance validation
---
# IaC validation
Every check runs on a scratch copy of the files you pass. Nothing is applied, and no state is written.
Every check runs on a scratch copy of your Terraform files. Nothing is applied, and no state is written.
The files are your working copy: the files you wrote with `write_file`, or, when you are validating, the files that were submitted. Pass the tools no files.
## Tools
- `terraform_fmt(files)`: returns the files formatting changed. Replace your copies with the returned content.
- `terraform_validate(files)`: runs `init -backend=false` and `validate` offline. It reports syntax errors, unknown references, missing variables and invalid arguments.
- `terraform_plan(files, variables)`: previews the changes against the live cloud environment. It reads live state, so the first call may pause for the user's permission. Pass every variable the configuration requires.
- `terraform_fmt()`: formats your working copy in place and names the files it changed.
- `terraform_validate()`: runs `init -backend=false` and `validate` offline. It reports syntax errors, unknown references, missing variables and invalid arguments.
- `terraform_plan(variables)`: previews the changes against the live cloud environment. It reads live state, so the first call may pause for the user's permission. Pass every variable the configuration requires.
- `terraform_schema(resource_type, block, find)`: what a resource type accepts, from the provider's own schema. With `resource_type` alone it lists the top-level arguments and outlines every block. With `block`, such as `template.containers`, it gives that block's arguments in full. With `find`, it says where an argument or block of that name sits, or that the resource type has none.
## When writing deployment guidance
1. Run `terraform_fmt`, then `terraform_validate`, on the complete set of files you will attach.
2. Fix every error and run both again, until `terraform_validate` passes.
3. Attach the formatted files.
1. Run `terraform_fmt`, then `terraform_validate`.
2. Fix each error with `edit_file`, changing only the lines concerned, and run `terraform_validate` again until it passes.
3. On "An argument named X is not expected here" or "Unsupported block type", call `terraform_schema` with `find` set to the name. It is either in a different block, which the reply names, or it does not exist on that resource type. Never move it around by trial.
## When validating deployment guidance
1. Run `terraform_validate` on the attached files, and report its verdict and output.
1. Run `terraform_validate`, and report its verdict and output.
2. When permission to read the live environment is granted, run `terraform_plan` with the variable values from the requirements, or clearly marked placeholders. Summarise the resources to add, change and destroy, and flag any destroy or replace.
3. When a check can't run (a missing CLI, a refused permission or a provider download failure), give the user the exact commands to run themselves:
@@ -40,4 +44,6 @@ Every check runs on a scratch copy of the files you pass. Nothing is applied, an
Give a verdict on every Check in the Target architecture and attach them as `verdicts.json`, in the format the requirements-chain Skill gives. The dry-run's result goes in its `dry_run` entry: `pass` when `terraform_validate` passes, `fail` with the error when it does not, and `not_run` when the tool could not run.
A Check fails when the code could meet it and does not. When `terraform_schema` shows the resource type has no such setting, no code could meet it: the verdict is `not_verifiable`, with the schema's answer as evidence.
Never suggest `terraform apply` as part of validation.

View File

@@ -92,14 +92,23 @@ A Check is a statement that is true or false of the infrastructure code. Name th
Split a broad recommendation into several Checks, one per setting. A Check that the validator cannot verify from the code is sent back to you.
A Check must name a setting its Element's resource type really has. Two similar resource types often differ: a setting of a long-running service may not exist on a batch job. A Check on a setting the resource type lacks is sent back to you.
The tables of Elements, Relationships and Checks, and the diagram, are written from this file when you submit. Write the graph once, here.
### `implementation.json` (deployment guidance)
One entry per Check in the Target architecture, saying where the code satisfies it. `file` is an attached file; `address` is the resource address and argument.
```json
[{"check": "K-1", "file": "terraform/main.tf", "address": "google_cloud_run_v2_service.ingest.ingress"}]
[
{"check": "K-1", "file": "terraform/main.tf", "address": "google_cloud_run_v2_service.ingest.ingress"},
{"check": "K-7", "unimplementable": "google_cloud_run_v2_job has no startup_cpu_boost argument; it exists only on a service."}
]
```
When a Check names a setting its resource type does not have, give it an `unimplementable` entry in place of `file` and `address`, saying what the resource type lacks. Write no code for it. The Target architecture is revised before anything is validated.
Implement every Element, Relationship and Check, and nothing the Target architecture does not ask for. When you believe the Target architecture is wrong or incomplete, say so with `needs_input`; do not quietly build something else.
### `verdicts.json` (validation results)