Skip to content

Latest commit

 

History

History
194 lines (142 loc) · 8.05 KB

README.md

File metadata and controls

194 lines (142 loc) · 8.05 KB
Raw

terraform-gcp-bigquery

Build Status Latest Release GitHub tag (latest SemVer) Terraform Version Infrastructure Tests pre-commit checkov Infrastructure Tests

The beginnings of a bigquery module for GCP, now with compute firewall options. The bigquery is designed to work primarily with a private Kubernetes Cluster and is enabled for OS Logins. You'll need to add the service role an OS role to your users they will be able to SSH into it. Basic Kubernetes tools are also installed into the bigquery by default.

2 examples are included, one with and one without a static IP. To find the image family and project:

gcloud compute images list

gcloud kms locations list
gcloud kms keyrings create --location=europe-west1 examplea

Usage

Add module.bigquery.tf to your code:-

module "bigquery" {
  source             = "JamesWoolfenden/bigquery/gcp"
  version            = "0.1.13"
  image              = var.image
  name               = var.name
  network_interface  = var.network_interface
  project            = var.project
  service_email      = var.service_email
  source_cidrs       = var.source_cidrs
  zone               = var.zone
}

Requirements

No requirements.

Providers

Name Version
google n/a

Modules

No modules.

Resources

Name Type
google_bigquery_dataset.example resource
google_bigquery_job.example resource
google_bigquery_table.example resource
google_kms_crypto_key.example resource
google_kms_crypto_key_iam_binding.crypto_key resource

Inputs

Name Description Type Default Required
dataset n/a 
object({
    dataset_id                  = string
    friendly_name               = string
    description                 = string
    default_table_expiration_ms = number
  })
 n/a yes
deletion_protection n/a bool true no
job n/a 
object({
    job_id              = string
    query               = string
    allow_large_results = bool
    flatten_results     = bool


    key_result_statement = string

  })
 n/a yes
key_name n/a string "crypto-key-example" no
key_users (optional) describe your variable list(string) n/a yes
keyring n/a any n/a yes
labels (optional) describe your variable map(string) n/a yes
location n/a string n/a yes
table n/a 
object({
    table_id = string
    external_data_configuration = object({
      autodetect    = bool
      source_format = string
      google_sheets_options = object({
        skip_leading_rows = number
      })
      source_uris = list(string)
    })
  })
 n/a yes

Outputs

No outputs.

Role and Permissions

The Terraform resource required is:

resource "google_project_iam_custom_role" "terraform_pike" {
  project     = "pike"
  role_id     = "terraform_pike"
  title       = "terraform_pike"
  description = "A user with least privileges"
  permissions = [
    "bigquery.datasets.create",
    "bigquery.jobs.create",
    "cloudkms.cryptoKeyVersions.destroy",
    "cloudkms.cryptoKeyVersions.list",
    "cloudkms.cryptoKeys.create",
    "cloudkms.cryptoKeys.get",
    "cloudkms.cryptoKeys.update"
  ]
}

Information

The Terraform resource required is:

resource "google_project_iam_custom_role" "terraform_pike" {
  project     = "pike"
  role_id     = "terraform_pike"
  title       = "terraform_pike"
  description = "A user with least privileges"
  permissions = [
    "bigquery.datasets.create",
    "bigquery.jobs.create",
    "cloudkms.cryptoKeyVersions.destroy",
    "cloudkms.cryptoKeyVersions.list",
    "cloudkms.cryptoKeys.create",
    "cloudkms.cryptoKeys.get",
    "cloudkms.cryptoKeys.update"
  ]
}

Related Projects

Check out these related projects.

Help

Got a question?

File a GitHub issue.

Contributing

Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

Copyrights

Copyright � 2019-2023 James Woolfenden

License

License

See LICENSE for full details.

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Contributors

James Woolfenden
James Woolfenden