chore: housekeeping from the runs of 2026-10-09
Some checks failed
validation / verify (push) Failing after 12s
Some checks failed
validation / verify (push) Failing after 12s
- An Execution's files are written as each Deliverable is submitted, marked in_progress, so a killed process or one waiting for its user leaves them on disk. A write removes files an earlier write left that it no longer has. - A question about delivered work is answered from that Execution's solution guide by the reply writer; a question about the service is answered as before. Nothing is started for either. - Lessons: prove writes only for lessons the Skill holds; the Skill records the vendored Skill versions the lessons were read against, status names any that changed, and a test fails until they are reviewed. - Four more code patterns: Firestore, BigQuery, Eventarc and GKE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
92
skills/google-cloud-iac-patterns/references/bigquery.md
Normal file
92
skills/google-cloud-iac-patterns/references/bigquery.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# BigQuery
|
||||
|
||||
A dataset, a partitioned and clustered table, and the grants for a writer and a reader.
|
||||
|
||||
- `location` is a region or `EU`/`US`, and cannot be changed. Queries cannot join datasets in different locations.
|
||||
- `delete_contents_on_destroy = false` makes Terraform refuse to remove a dataset that still has tables.
|
||||
- A table's `deletion_protection` is true unless set; take it from a variable so a test deployment can be removed.
|
||||
- `schema` is a JSON string. Changing a column's type or mode replaces the table.
|
||||
- Partition on a `DATE` or `TIMESTAMP` column with `time_partitioning`, and set `require_partition_filter` so a query cannot scan every partition. `expiration_ms` there removes old partitions.
|
||||
- Grant on the dataset with `google_bigquery_dataset_iam_member`. A principal that runs queries also needs `roles/bigquery.jobUser` on the project.
|
||||
- Do not mix `google_bigquery_dataset_iam_member` with `access` blocks on the dataset: each removes what the other sets.
|
||||
|
||||
```hcl
|
||||
variable "project_id" {
|
||||
type = string
|
||||
description = "Project to deploy into."
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
type = string
|
||||
description = "Region for regional resources."
|
||||
}
|
||||
|
||||
variable "writer_service_account_email" {
|
||||
type = string
|
||||
description = "Service account that loads rows."
|
||||
}
|
||||
|
||||
variable "reader_service_account_email" {
|
||||
type = string
|
||||
description = "Service account that queries the dataset."
|
||||
}
|
||||
|
||||
variable "deletion_protection" {
|
||||
type = bool
|
||||
description = "Whether the table is protected from deletion."
|
||||
default = true
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset" "analytics" {
|
||||
project = var.project_id
|
||||
dataset_id = "analytics"
|
||||
location = var.region
|
||||
delete_contents_on_destroy = false
|
||||
|
||||
default_table_expiration_ms = null
|
||||
max_time_travel_hours = 168
|
||||
}
|
||||
|
||||
resource "google_bigquery_table" "events" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
table_id = "events"
|
||||
deletion_protection = var.deletion_protection
|
||||
|
||||
require_partition_filter = true
|
||||
clustering = ["event_type"]
|
||||
|
||||
time_partitioning {
|
||||
type = "DAY"
|
||||
field = "event_time"
|
||||
expiration_ms = 34560000000
|
||||
}
|
||||
|
||||
schema = jsonencode([
|
||||
{ name = "event_id", type = "STRING", mode = "REQUIRED" },
|
||||
{ name = "event_type", type = "STRING", mode = "REQUIRED" },
|
||||
{ name = "event_time", type = "TIMESTAMP", mode = "REQUIRED" },
|
||||
{ name = "payload", type = "JSON", mode = "NULLABLE" }
|
||||
])
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset_iam_member" "writer" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
role = "roles/bigquery.dataEditor"
|
||||
member = "serviceAccount:${var.writer_service_account_email}"
|
||||
}
|
||||
|
||||
resource "google_bigquery_dataset_iam_member" "reader" {
|
||||
project = var.project_id
|
||||
dataset_id = google_bigquery_dataset.analytics.dataset_id
|
||||
role = "roles/bigquery.dataViewer"
|
||||
member = "serviceAccount:${var.reader_service_account_email}"
|
||||
}
|
||||
|
||||
resource "google_project_iam_member" "reader_runs_queries" {
|
||||
project = var.project_id
|
||||
role = "roles/bigquery.jobUser"
|
||||
member = "serviceAccount:${var.reader_service_account_email}"
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user