https://github.com/ackeecz/terraform-gcp-dataflow_pubsub_to_bq

Dataflow job subscriber to PubSub subscription. It takes message from subscription and push it into BigQuery table.
https://github.com/ackeecz/terraform-gcp-dataflow_pubsub_to_bq

bigquery dataflow pubsub terraform-module

Last synced: 29 days ago
JSON representation

Dataflow job subscriber to PubSub subscription. It takes message from subscription and push it into BigQuery table.

• Host: GitHub
• URL: https://github.com/ackeecz/terraform-gcp-dataflow_pubsub_to_bq
• Owner: AckeeCZ
• Created: 2021-10-23T17:18:09.000Z (over 3 years ago)
• Default Branch: feat/63119-init
• Last Pushed: 2021-10-23T17:20:02.000Z (over 3 years ago)
• Last Synced: 2024-11-10T15:49:59.393Z (3 months ago)
• Topics: bigquery, dataflow, pubsub, terraform-module
• Language: HCL
• Homepage:
• Size: 9.77 KB
• Stars: 1
• Watchers: 4
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md

Awesome Lists containing this project

README

        
# Dataflow PubSub to BigQuery

Dataflow job subscriber to PubSub subscription. It takes message from subscription and push it into BigQuery table.

## Before you do anything in this module

Install pre-commit hooks by running following commands:

```shell script

brew install pre-commit terraform-docs

pre-commit install

```

## Requirements

| Name | Version |

|------|---------|

|  [terraform](#requirement\_terraform) | >= 0.13 |

## Providers

| Name | Version |

|------|---------|

|  [google](#provider\_google) | n/a |

|  [google-beta](#provider\_google-beta) | n/a |

|  [random](#provider\_random) | n/a |

## Modules

No modules.

## Resources

| Name | Type |

|------|------|

| [google-beta_google_dataflow_flex_template_job.pubsub_to_bq_job](https://registry.terraform.io/providers/hashicorp/google-beta/latest/docs/resources/google_dataflow_flex_template_job) | resource |

| [google_storage_bucket.provisioning_bucket](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket) | resource |

| [google_storage_bucket_object.pubsub_subscription_to_bq](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_object) | resource |

| [random_string.random](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) | resource |

## Inputs

| Name | Description | Type | Default | Required |

|------|-------------|------|---------|:--------:|

|  [bigquery\_schema](#input\_bigquery\_schema) | Schema of target bigquery table | `string` | n/a | yes |

|  [bigquery\_table](#input\_bigquery\_table) | table\_id of target table | `string` | n/a | yes |

|  [input\_subscription](#input\_input\_subscription) | Name of input subscription | `string` | n/a | yes |

|  [region](#input\_region) | Region to deploy dataflow job to | `string` | n/a | yes |

## Outputs

No outputs.