The reference agent's subject, cloud solution architecture, as an agent in the operating model agent's pattern: the Execution engine (from orchestration_engine) and one Workflow in app/solution_architecture, built for the settings and provider bundle by assembly.workflow_for. Copied from gcp_solution_architecture_agent at 8b397cd without its own engine: its role files, Skills, provider bundle, ADRs, vocabulary, scripts and evaluation datasets. The image takes the engine from a named build context and installs Terraform. It is called through the Orchestration engine, and its files are what the operating model agent digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.2 KiB
3.2 KiB
BigQuery
A dataset, a partitioned and clustered table, and the grants for a writer and a reader.
locationis a region orEU/US, and cannot be changed. Queries cannot join datasets in different locations.delete_contents_on_destroy = falsemakes Terraform refuse to remove a dataset that still has tables.- A table's
deletion_protectionis true unless set; take it from a variable so a test deployment can be removed. schemais a JSON string. Changing a column's type or mode replaces the table.- Partition on a
DATEorTIMESTAMPcolumn withtime_partitioning, and setrequire_partition_filterso a query cannot scan every partition.expiration_msthere removes old partitions. - Grant on the dataset with
google_bigquery_dataset_iam_member. A principal that runs queries also needsroles/bigquery.jobUseron the project. - Do not mix
google_bigquery_dataset_iam_memberwithaccessblocks on the dataset: each removes what the other sets.
variable "project_id" {
type = string
description = "Project to deploy into."
}
variable "region" {
type = string
description = "Region for regional resources."
}
variable "writer_service_account_email" {
type = string
description = "Service account that loads rows."
}
variable "reader_service_account_email" {
type = string
description = "Service account that queries the dataset."
}
variable "deletion_protection" {
type = bool
description = "Whether the table is protected from deletion."
default = true
}
resource "google_bigquery_dataset" "analytics" {
project = var.project_id
dataset_id = "analytics"
location = var.region
delete_contents_on_destroy = false
default_table_expiration_ms = null
max_time_travel_hours = 168
}
resource "google_bigquery_table" "events" {
project = var.project_id
dataset_id = google_bigquery_dataset.analytics.dataset_id
table_id = "events"
deletion_protection = var.deletion_protection
require_partition_filter = true
clustering = ["event_type"]
time_partitioning {
type = "DAY"
field = "event_time"
expiration_ms = 34560000000
}
schema = jsonencode([
{ name = "event_id", type = "STRING", mode = "REQUIRED" },
{ name = "event_type", type = "STRING", mode = "REQUIRED" },
{ name = "event_time", type = "TIMESTAMP", mode = "REQUIRED" },
{ name = "payload", type = "JSON", mode = "NULLABLE" }
])
}
resource "google_bigquery_dataset_iam_member" "writer" {
project = var.project_id
dataset_id = google_bigquery_dataset.analytics.dataset_id
role = "roles/bigquery.dataEditor"
member = "serviceAccount:${var.writer_service_account_email}"
}
resource "google_bigquery_dataset_iam_member" "reader" {
project = var.project_id
dataset_id = google_bigquery_dataset.analytics.dataset_id
role = "roles/bigquery.dataViewer"
member = "serviceAccount:${var.reader_service_account_email}"
}
resource "google_project_iam_member" "reader_runs_queries" {
project = var.project_id
role = "roles/bigquery.jobUser"
member = "serviceAccount:${var.reader_service_account_email}"
}