{"id":25841650,"url":"https://github.com/somaz94/terraform-bigquery-googlesheet","last_synced_at":"2026-02-10T19:33:47.533Z","repository":{"id":209967047,"uuid":"725015840","full_name":"somaz94/terraform-bigquery-googlesheet","owner":"somaz94","description":"terraform-bigquery-googlesheet","archived":false,"fork":false,"pushed_at":"2025-02-21T09:04:32.000Z","size":166,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-31T21:12:54.443Z","etag":null,"topics":["bigquery","data-structures","googlesheetapi","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/somaz94.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-11-29T09:08:41.000Z","updated_at":"2025-02-21T09:04:36.000Z","dependencies_parsed_at":"2024-02-19T04:48:54.771Z","dependency_job_id":null,"html_url":"https://github.com/somaz94/terraform-bigquery-googlesheet","commit_stats":null,"previous_names":["somaz94/terraform-bigquery-googlesheet"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/somaz94/terraform-bigquery-googlesheet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somaz94%2Fterraform-bigquery-googlesheet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somaz94%2Fterraform-bigquery-googlesheet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somaz94%2Fterraform-bigquery-googlesheet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somaz94%2Fterraform-bigquery-googlesheet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/somaz94","download_url":"https://codeload.github.com/somaz94/terraform-bigquery-googlesheet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/somaz94%2Fterraform-bigquery-googlesheet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29313138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-10T17:48:59.043Z","status":"ssl_error","status_checked_at":"2026-02-10T17:45:37.240Z","response_time":65,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","data-structures","googlesheetapi","terraform"],"created_at":"2025-03-01T05:29:55.400Z","updated_at":"2026-02-10T19:33:47.516Z","avatar_url":"https://github.com/somaz94.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚀 Project Overview\n\nThis project integrates various data sources with Google Sheets and BigQuery using Cloud Functions. It is structured to handle different data processing and transfer needs efficiently.\n\n\u003cbr/\u003e\n\n## Data Architecture\n```mermaid\ngraph TD\n    A[MongoDB] --\u003e|Cloud Function: Dataflow Invocation| B[Dataflow: MongoDB Data Transfer]\n    B --\u003e|Import to DataSet| I[BigQuery: DataSet]\n    I --\u003e|Deduplication| P[Cloud Function: Deduplication]\n    P --\u003e|Update DataSet| I\n    I --\u003e|Direct to Sheets| K[Google Sheets]\n    C[CloudSQL] --\u003e|Cloud Function: SQL Query| D[Direct to Sheets]\n    D --\u003e K\n    E[Google Analytics] --\u003e|API Data Fetch| F[Cloud Function: GA Query]\n    F --\u003e K\n    G[Dune] --\u003e|API Data Fetch| H[Cloud Function: Dune Query]\n    H --\u003e K\n    L[Cloud Scheduler: Daily Data Transfer] --\u003e|Trigger| A\n    L --\u003e|Trigger| P\n    L --\u003e|Schedule| D\n    L --\u003e|Schedule| F\n    L --\u003e|Schedule| H\n\n    style A fill:#f9f,stroke:#333,stroke-width:2px\n    style B fill:#f9f,stroke:#333,stroke-width:2px\n    style C fill:#f9f,stroke:#333,stroke-width:2px\n    style E fill:#f9f,stroke:#333,stroke-width:2px\n    style G fill:#f9f,stroke:#333,stroke-width:2px\n    style I fill:#9cf,stroke:#333,stroke-width:2px\n    style K fill:#9cf,stroke:#333,stroke-width:2px\n    style L fill:#ff9,stroke:#333,stroke-width:4px\n    style P fill:#f66,stroke:#333,stroke-width:2px\n```\n- https://mermaid.live/edit\n\n\u003cbr/\u003e\n\n## 📁 File Structure\n\n- **📜 LICENSE**: Contains the MIT License details for this project.\n- **📄 README.md**: This file.\n- **🔑 key**\n  - `terraform.json` - Stores the Terraform configuration key.\n- **📦 modules**: Terraform modules for different infrastructure components.\n  - **🌐 gcs_buckets**\n    - Management of Google Cloud Storage buckets.\n  - **🗄️ mysql**\n    - Manages MySQL configurations, including read replicas.\n  - **🌍 network**\n    - Network configurations and related components like firewall rules and subnets.\n  - **🔐 service_accounts**\n    - Manages service accounts configurations.\n- **🔧 project/somaz-bigquery**: Contains Cloud Functions for data processing and transfer.\n  - **📈 analytics-to-sheet-new-web-visitors**\n    - Retrieves new web visitors count from Google Analytics and inputs it into Google Sheets.\n  - **🌏 analytics-to-sheet-new-web-visitors-country**\n    - Fetches country-wise new web visitor counts from Google Analytics for input into Google Sheets.\n  - **🔄 bigquery-deduplication**\n    - Removes duplicate data entries in BigQuery imported from MongoDB.\n  - **📊 bigquery-to-sheet-multiple/simple/retention/wallet**\n    - Functions for querying data from BigQuery and exporting to Google Sheets.\n  - **📋 copy-formula-to-sheet**\n    - Copies cells with formulas in Google Sheets.\n  - **📋 copy-formula-monthly-to-sheet**\n    - Copies cells with formulas in Google Sheets.\n  - **💱 matic-value-to-sheet**\n    - Fetches the daily price of Matic from [CoinGecko](https://www.coingecko.com/ko/%EC%BD%94%EC%9D%B8/polygon/historical_data#panel) and inputs it into Google Sheets.\n  - **🔄 mongodb-to-bigquery**\n    - Transforms data from MongoDB for export to BigQuery.\n  - **🌐 onchain-agent-common-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-agent-epic-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-agent-legend-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-agent-rare-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-agent-uncommon-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-materials-dp-chip-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-materials-skill-exchange-ticket-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-pack-basic-epic1-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  - **🌐 onchain-quest2-daily-global-to-sheet**\n    - Imports Onchain data from Dune API to Google Sheets.\n  ...\n\n\u003cbr/\u003e\n\n## ⚠️ Cautionary Notes\n\n- **🔐 Service Accounts**: Only Service Accounts from your project can be used with Cloud Functions and Cloud Scheduler.\n- **🔗 API Usage**: \n  - For GA4, the [Analytics Reporting API](https://console.cloud.google.com/apis/api/analyticsreporting.googleapis.com/overview?hl=ko\u0026project=mgmt-2023\u0026supportedpurview=project) is not available.\n  - However, the [Google Analytics Data API](https://console.cloud.google.com/apis/api/analyticsdata.googleapis.com/overview?hl=ko\u0026project=mgmt-2023\u0026supportedpurview=project) can be used.\n  - To use the [Dune API](https://dune.com/docs/api/) for Onchain data, you will need to create an API Key and choose an appropriate billing plan.\n\n\u003cbr/\u003e\n\n## 📜 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsomaz94%2Fterraform-bigquery-googlesheet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsomaz94%2Fterraform-bigquery-googlesheet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsomaz94%2Fterraform-bigquery-googlesheet/lists"}