BigQuery
Google BigQuery enables companies to handle large amounts of data without having to manage infrastructure. Google’s documentation describes it as a « serverless architecture (that) lets you use SQL queries to answer your organization’s biggest questions with zero infrastructure management. BigQuery’s scalable, distributed analysis engine lets you query terabytes in seconds and petabytes in minutes. » Its client libraries allow the use of widely known languages such as Python, Java, JavaScript, and Go. Federated queries are also supported, making it flexible to read data from external sources.
📖 A highly rated canonical book on it is « Google BigQuery: The Definitive Guide », a comprehensive reference. Another enriching read on the subject is the inside story told in the article by the founding product manager of BigQuery celebrating its 10th anniversary.
- GitHub: https://github.com/topics/bigquery
- Wikipedia: https://en.wikipedia.org/wiki/BigQuery/
- Repo: https://github.com/GoogleCloudPlatform/bigquery-utils/
- Released: May 19, 2010
- Related Topics: cloud-computing,
- Aliases: bq,
- Last updated: 2026-06-15 00:03:38 UTC
- JSON Representation
https://github.com/snithish/tpc-ds_big-query
Scripts to execute TPC - DS on Big Query
benchmark bigquery tpc-ds-benchmark tpc-ds-queries
Last synced: 07 May 2025
https://github.com/sigpwned/litecene
A simple cross-data store full-text search language for Java 8+
bigquery full-text-search java query-language search
Last synced: 12 Apr 2025
https://github.com/mchmarny/pubsub-to-bigquery-pump
Simple utility combining Cloud Run and Stackdriver metrics to drain JSON messages from PubSub topic into BigQuery table
bigquery cloudrun events golang metrics pubsub stackdriver
Last synced: 21 Apr 2025
https://github.com/memsjava/bigquery-helper
A helper package for Google BigQuery operations
bigquery google pandas-dataframe
Last synced: 01 Aug 2025
https://github.com/dmytrovoytko/data-engineering-amazon-reviews
Data Engineering project for ZoomCamp`24: JSONL -> PostgreSQL/BigQuery + Metabase + Mage.AI
bash-script bigquery codespaces data-analysis data-visualization etl metabase pipeline python-script
Last synced: 31 Jul 2025
https://github.com/dp6/raft-suite-hub
O Hub é a solução responsável por centralizar a consolidação dos dados no BigQuery, ferramenta escolhida para servir de data warehouse do raft-suite.
bigquery data data-quality google-cloud google-cloud-functions hacktoberfest
Last synced: 28 Jun 2025
https://github.com/fairscript/interact
A database interaction library for node.js/JavaScript/TypeScript that uses code reflection to maximize type safety and minimize friction. Supports PostgreSQL, Google BigQuery and SQLite.
bigquery data database linq orm postgresql reflection sql sqlite typesafe
Last synced: 19 Apr 2025
https://github.com/tamanobi/bq-query-unittest
BigQueryのクエリのロジックをデータ走査量を最小限してテストできるツール
Last synced: 22 Jun 2025
https://github.com/doitintl/terraform-bq-scheduled-queries
This is a demo project to use Terraform to manage BigQuery scheduled queries with Cloud Build CI/CD
bigquery cicd cloudbuild terraform
Last synced: 30 Apr 2025
https://github.com/ya7010/turu-py
Simple Database API for Typed Python
async bigquery pep249 postgres postgresql postgresql-database python snowflake snowflakedb sqlite sqlite-database sqlite3 sqlite3-database typed-python
Last synced: 19 Apr 2026
https://github.com/fivetran/zetasql-npm-examples
This repo contains examples of usage of zetasql-npm library
Last synced: 16 Apr 2025
https://github.com/corneliusweig/krew-index-tracker
Saves download statistics of `krew.dev` plugins to BigQuery
bigquery history krew krew-index statistics
Last synced: 21 Apr 2025
https://github.com/kellyjadams/run-sql-in-python
Scripts to connect python to BigQuery or a PostgreSQL database.
Last synced: 29 Jul 2025
https://github.com/wayfair-incubator/gbq
Python wrapper for interacting with Google BigQuery.
bigquery gbq google google-bigquery google-cloud-platform hacktoberfest python
Last synced: 03 Feb 2026
https://github.com/lixx21/airflow-dbt-gcp
A comprehensive data pipeline leveraging Airflow, DBT, Google Cloud Platform (GCP), and Docker to extract, transform, and load data seamlessly from a staging layer to a data warehouse and data mart.
airflow bigquery data-engineer dbt gcp
Last synced: 10 Apr 2025
https://github.com/tobked/fetch-apache-ga-stats
Repository to make "snapshots" of GitHub Action queue for later analysis
bigquery gcp github github-actions
Last synced: 26 Feb 2025
https://github.com/gr8distance/blanton
BigQuery API wrapped by Elixir
bigquery bigquery-schema elixir
Last synced: 25 Mar 2025
https://github.com/k1low/tbls-meta
tbls-meta is an external subcommand of tbls for applying metadata managed by tbls to the datasource.
bigquery data-catalog-management
Last synced: 01 Apr 2026
https://github.com/dataform-co/bigquery-ml-pipeline
An example of machine pipeline on Bigquery ML using Dataform
bigquery bigquery-ml dataform machine-learning-pip sql
Last synced: 19 Sep 2025
https://github.com/trocco-io/embulk-output-bigquery_java
Java flavor faster Embulk output plugin to load/insert data into Google BigQuery
Last synced: 27 Feb 2026
https://github.com/kitta65/bq2cst
Parse GoogleSQL, which is a dialect of BigQuery, into a concrete syntax tree
Last synced: 22 Feb 2026
https://github.com/urbanclimatefr/de-zoomcamp-2025-project
end to end data pipeline
batch-processing bigquery dbt etl-pipeline gcs hko kestra temperature
Last synced: 01 Feb 2026
https://github.com/blockchain-etl/iotex-etl
ETL (extract, transform and load) tools for ingesting IoTeX blockchain data to Google BigQuery and Pub/Sub
bigquery blockchain-data iotex sql
Last synced: 14 Apr 2025
https://github.com/trk54ylmz/spark-bigquery
Google BigQuery support for Spark SQL
Last synced: 12 May 2025
https://github.com/hsbc/bqtools
The code repo for bqtools-json a package for managing bigquery using json exemplar data structure and home of the bqsync utility.
Last synced: 14 Jul 2025
https://github.com/jordicenzano/brighcove-live-ssai-ccu
POC (proof of concept) to show a possible way to calculate the real time CCU (concurrent viewers) for any Brightcove live stream with SSAI
analytics bigquery brightcove gcp-ap gcp-appengine-flex gcp-cloud-functions hls live streaming video
Last synced: 21 Aug 2025
https://github.com/johannaojeling/go-beam-pipeline
Data pipeline built with the Apache Beam Go SDK
apache-beam batch-processing bigquery cloud-sql cloud-storage dataflow elasticsearch firestore go google-cloud memorystore mongodb mysql postgresql redis
Last synced: 16 Jun 2025
https://github.com/ygor-j/sql-guard
A small package for data quality rules using Standard SQL
assertions bigquery data data-cleaning data-quality data-quality-checks duckdb ducklake gcp in-memory pure-sql sql testing-tools
Last synced: 26 Oct 2025
https://github.com/dav009/dbtmock
end to end unit tests for dbt ( Data build tool ) pipelines
bigquery data-build-tool dbt mock pipelines test testing unittest unittesting
Last synced: 27 Apr 2026
https://github.com/webocs/mining-github-microservices
Gihub mining replication package for the article "Microservices in the Wild: the Github Landscape". It's A short node program that takes a prefiltered set of github repositories (Filtered with Google BigQuery) and uses GitHub API to find the ones that have a X nubmer of stars
bigdata bigquery microservices node
Last synced: 25 Sep 2025
https://github.com/pierrec1024/airflow-provider-bigquery-reservation
Airflow provider for bigquery reservation operators.
Last synced: 30 Oct 2025
https://github.com/glific/recipes
Sample projects to integrate Glific with third party services.
bigquery google-sheets nestjs nodejs postgresql
Last synced: 30 Jul 2025
https://github.com/shnewto/bqjson
bqjson - Serialize/Deserialzie BigQuery TableResults to/from JSON
bigquery java json maven serde serde-json serialization serializer tableresult testing tests
Last synced: 17 Mar 2025
https://github.com/cata-network/cadence-docs
cadence document, Chinese version
Last synced: 11 Apr 2025
https://github.com/staffbase/bigquery-github-action
GitHub Action for writing data into BigQuery from within a Github workflow
actions bigquery github-actions
Last synced: 19 Jul 2025
https://github.com/wintermi/movielens-dataform
An example Dataform project which will use the publicly available Movielens dataset to demonstrate how to upload your product catalog and user events into either the Google Cloud Retail API or Google Cloud Discovery Engine and train a personalised product recommendation model.
bigquery dataform google-cloud google-cloud-platform vertex-ai
Last synced: 05 May 2025
https://github.com/sukanyabag/gcp-ai-notebooks
This repository contains all practice notebooks with which I performed hands-on labs in Google Cloud Training Program's "Cloud ML-AI Track"
bigquery cloudml-samples data-science dataprep tensorflow-tutorials
Last synced: 08 Feb 2026
https://github.com/lin-jun-xiang/pyga4
📊Python Google Analytics 4 (GA4) Data Extraction and Analysis Toolkit
bigquery free ga ga4 google-analytics google-analytics-python-api python
Last synced: 19 Mar 2025
https://github.com/realkinetic/codelab
Codelab for Google Cloud (BigQuery, NLP, GAE)
bigquery cloud google google-cloud natural-language-processing python
Last synced: 13 Aug 2025
https://github.com/rittmananalytics/ra_dbt_to_dataform
An open-source tool that partially automates the migration of dbt packages to Dataform
bigquery dataform dbt dbt-core migration-tool
Last synced: 20 Jun 2025
https://github.com/nodeart/koatuu-to-bigquery
load koatuu from https://data.gov.ua/dataset/dc081fb0-f504-4696-916c-a5b24312ab6e to Google BigQuey in denormalized form
bigquery google-bigquey koatuu
Last synced: 13 May 2025
https://github.com/stkchan/googleanalytics4-publicdataset-ecommerce-dashboard-powerbi
This dashboard uses Power BI Desktop as a visualization tool by extracting data from Google BigQuery.
analytics bigquery dashboard portfolio portfolio-project powerbi sql
Last synced: 12 Aug 2025
https://github.com/moh-ayman/mongodb-to-bigquery---cloud-func-etl
Google Cloud Function built to perform an ETL Job to Collect MongoDB Data and Transform it to be able to Import it to Bigquery.
bigquery etl-pipeline gcp-cloud-functions mongodb pandas-python
Last synced: 12 Apr 2025
https://github.com/adam-cowley/neo4j-bigquery
Yo dawg, I heard you like queries so we put some BigQuery in your query so you can query BigQuery from your query
bigquery cypher neo4j neo4j-procedures
Last synced: 08 May 2026
https://github.com/wintermi/bq2csv
A command line application designed to provide a simple method to execute a BigQuery SQL script from "stdin", outputting all results to "stdout" in CSV format. A detailed log is output to the console "stderr" providing you with the available execution statistics.
bigquery google-cloud google-cloud-platform
Last synced: 30 Oct 2025
https://github.com/jroakes/npath
Exploring path sequences in GA4 BigQuery data
analytics bigquery pathfinding-algorithm
Last synced: 28 Apr 2026
https://github.com/bzzt/alchemy_table
Opinionated framework for working with Bigtable and BigQuery
bigquery bigtable database elixir gcp googlecloud googlecloudplatform
Last synced: 28 Jul 2025
https://github.com/cloudymoma/rustmq
Cloud-native distributed message queue with storage-compute separation architecture. Licensed under BDL 1.0 (academic use permitted, commercial use requires license).
bigquery cloud-native distributed-systems google-cloud message-queue object-storage quic rust streaming webassembly
Last synced: 19 Sep 2025
https://github.com/badal-io/dataflow-timeseries-iot-gas-demo
Dataflow code for integration with GCP Core IoT and FogLamp
Last synced: 15 Jun 2025
https://github.com/jolares/example-gcp-dataform
Example end-to-end ELT data pipeline using GCP Dataform.
bigquery dataform etl-pipeline
Last synced: 15 Apr 2025
https://github.com/tirendazacademy/hands-on-data-science-with-gcp
Google BigQuery Tutorial
big-data big-data-analytics bigdata bigquery bigquery-ml bigqueryml cloud-computing data-analysis data-analytics data-engineering data-science dataanalysis dataengineering google-bigquery google-cloud-platform machienlearning machine-learning
Last synced: 06 Oct 2025
https://github.com/airscholar/dbt-bigquery-crash-course
A deep dive into the powerful combination of DBT and BigQuery, the game-changers in modern data engineering.
bigquery data-engineering dbt google-cloud
Last synced: 10 Apr 2025
https://github.com/phstudy/postgresql-zetasketch
ZetaSketch HLL++ functions for PostgreSQL
bigquery hll java postgresql postgresql-extension zetasketch
Last synced: 09 May 2026
https://github.com/googlecloudplatform/dcm2bq
A service for creating a JSON metadata representation for DICOM from multiple input sources and storing into Google Cloud Big Query (BQ).
bigquery dicom gcs googlecloud googlecloudplatform googlecloudstorage json
Last synced: 28 Feb 2026
https://github.com/poogles/pytest-bq
pytest fixtures for a local bigquery suitable for local development.
bigquery bigquery-emulator pytest
Last synced: 11 Feb 2026
https://github.com/42digital/bqtools
Python Tools for BigQuery
bigquery bigquery-schema migrations python
Last synced: 09 Apr 2025
https://github.com/wintermi/bqrunner
A command line application designed to provide a simple method to execute one or more SQL queries against a given dataset in BigQuery. A detailed log is output to the console providing you with the available execution statistics.
bigquery google-cloud google-cloud-platform
Last synced: 12 Jan 2026
https://github.com/yashika-malhotra/strategic-analysis-of-retail-brand-in-south-america-using-sql
Leveraged Big Query and MySQL to analyze 100K records for sales optimization, trend identification, and enhancing customer satisfaction for a retail brand in South America and to provide insights and recommendations to improve their userbase and improve their services
bigquery data-analysis data-science database database-schema google-bigquery mysql-server sql
Last synced: 18 Apr 2026
https://github.com/itkq/incidentio-to-bq
Load incident data from incident.io to BigQuery to analyze the MTTR etc.
Last synced: 13 May 2026
https://github.com/sungchun12/image-labeling-and-translation-data-analysis-google-cloud
:label: Invokes cloud vision and translation APIs with Python
bigquery cloud-translation-api cloud-vision-api colaboratory google-cloud python sql
Last synced: 09 May 2026
https://github.com/pualien/py-gcloud-connectors
Utilities to simplify connection with Google APIs
bigquery data-analysis google-analytics google-analytics-4 google-cloud google-cloud-platform google-cloud-storage pandas python
Last synced: 28 Apr 2026
https://github.com/googlecloudplatform/aicoe
This repository contains an end-to-end walkthrough to leverage Google Cloud services to demonstrate Solution Accelerators for few business domains
aimlops bigquery dataflow googlecloudplatform mlops security-analytics solution-accelerators vertex-ai
Last synced: 20 Oct 2025
https://github.com/greenpeace/gpes-bigquery-recipes
Google Big Query recipes to Analyse our data.
bigquery database-management sql
Last synced: 11 May 2025
https://github.com/sigpwned/jdbq
JDBI-inspired Database Access Framework for Java + BigQuery
bigquery data-access-framework data-access-layer data-access-library data-lake java persistence persistence-framework persistence-layer
Last synced: 15 May 2025
https://github.com/neo4j-field/bigquery-connector
Bi-directional connectivity between Google BigQuery and Neo4j AuraDS
arrow-flight bigquery neo4j protobuf python spark
Last synced: 13 Apr 2025
https://github.com/indatawetrust/bigquery-api
A quick and easy to use package for Google Cloud BigQuery
Last synced: 16 Jul 2025
https://github.com/olajideolagunju/gcp_mage_data_pipeline
An end-to-end data engineering pipeline that processes and analyzes Maintenance Work Orders using Mage, Docker, Google BigQuery, MariaDB, and Looker Studio. It features a seamless integration of cloud and open-source tools for scalable data storage, transformation, and visualization.
automation bigquery cloud compute-engine data data-engineering database database-schema docker-compose excel gcp mage-ai maintenance mariadb orchestration python sql virtual-machine visualization-dashboard work-orders
Last synced: 07 Mar 2025
https://github.com/42DIGITAL/bqtools
Python Tools for BigQuery
bigquery bigquery-schema migrations python
Last synced: 02 Apr 2025
https://github.com/gjbae1212/go-bqworker
go-esworker is an async worker that data can bulk insert, update to the BigQuery.
async bigquery bigquery-bulk gcp go golang parallel worker
Last synced: 26 Apr 2026
https://github.com/wintermi/fashion-dataform
An example Dataform project to load and transform the publicly available dataset from H&M Group into a format which could be imported into Discovery AI for Retail or Vertex AI Search and Conversation, , allowing you to train a retail recommendations model.
bigquery dataform google-cloud google-cloud-platform vertex-ai
Last synced: 05 May 2025
https://github.com/galois1915/google-ml-engineer
This program provides the skills you need to advance your career and provides training to support your preparation for the industry-recognized Google Cloud Professional Machine Learning Engineer certification.
api automl bigquery keras mlops-workflow tensorflow2 vertex-ai
Last synced: 16 Feb 2026
https://github.com/huuvuong0912/rag-llm-based-recommender
Explore a smarter way to shop online with this full-stack project built on the infrastructure of Google Cloud Platform (GCP) for RAG based e-commerce with LLM.
bigquery fastapi langchain llm-inference llmops pyspark rag react recommender-system vertex-ai
Last synced: 13 Oct 2025
https://github.com/wintermi/bqwrite-test
A command line application designed to provide a method to test the BigQuery Streaming API or BigQuery Storage Write API, allowing you to get a view of the potential throughput available via a given host.
bigquery google-cloud google-cloud-platform
Last synced: 12 Mar 2026
https://github.com/vigneshss-07/google-cloud-professional-data-engineer-acompleteguide
This Repo contains all study, lab and supportive materials for Udemy course on "Google Cloud Professional Data Engineer - A Complete Guide".
big-data bigquery cloud-computing dataengineering elt-pipeline etl-framework gcp-services gcp-storage google-cloud machine-learning
Last synced: 06 Mar 2026
https://github.com/d8a-tech/d8a
A data collection service fully compatible with GA4 tracking protocols. Ingest into ClickHouse or BigQuery database while maintaining complete control over your data.
bigquery clickhouse data ga4 tracker
Last synced: 10 Apr 2026
https://github.com/takegue/bigquery-porter
BigQuery Deployment and Metadata Management tool
Last synced: 12 Feb 2026
https://github.com/phenrickson/bgg-data-warehouse
ETL process for BGG cloud data warehouse
bigquery data-engineering elt-pipeline python
Last synced: 15 Apr 2026
https://github.com/wintermi/tmdb-dataform
An example Dataform project to load and transform the publicly available dataset from The Movie Database into a format which could be imported into Vertex AI Search for Media, allowing you to build a search engine for movies.
bigquery dataform google-cloud google-cloud-platform
Last synced: 07 Feb 2026
https://github.com/rezuankassim/bqanalytic
Laravel package to use analytic data imported to Big Query from Firebase Analytic
bigquery firebase-analytics laravel
Last synced: 01 Feb 2026
https://github.com/ovotech/bigquery-metrics-exporter
A Golang application to export table level metrics from BigQuery into Datadog.
Last synced: 06 Feb 2026
https://github.com/oguzgn/firebase-ab-test-analysis-for-a-mobile-race-game
This repository showcases an infrastructure designed for analyzing A/B tests in mobile games. It leverages BigQuery to process Firebase and GA4-based event data and uses Looker Studio for dynamic visualization. The project simplifies A/B test comparisons, enabling stakeholders to view results directly through interactive dashboards.
ab-testing ab-testing-analysis bigquery event-based-tracking firebase looker-studio mobile-game-analytics race-game sql
Last synced: 19 May 2026
https://github.com/tuancamtbtx/gcp-udfs-example
Google BigQuery Javascript UDF Function Examples
bigquery gcp javascript nodejs npm udf
Last synced: 04 Jul 2025
https://github.com/t3n/gtmetrix-bq
A script running browser test of specified urls through GTmetrix and saving metrics in BigQuery.
Last synced: 15 May 2026
https://github.com/benitomartin/benitomartin
Personal profile 😎
anaconda artificial-intelligence aws bash-script bigquery data-science gcp lambda-functions large-language-models linux machine-learning python pytorch retrieval-augmented-generation sagemaker scikit-learn tensorflow terraform
Last synced: 12 Apr 2025
https://github.com/taquynhnga2001/proptech-dagster
Build an ELT pipeline with dagster and dbt to schedule loading HDB resale transactions in Singapore into Google BigQuery data warehouse, then create Power BI dashboard to enhance insight exploration.
bigquery dagster data-integration data-orchestration data-warehouse dbt elt etl powerbi python
Last synced: 14 Feb 2026
https://github.com/tuanai-vireox/gcp-udfs-example
Google BigQuery Javascript UDF Function Examples
bigquery gcp javascript nodejs npm udf
Last synced: 19 Apr 2026
https://github.com/toddbirchard/bigquery-to-sql
:bar_chart: :arrow_right: :floppy_disk: Lightweight ETL script to migrate data from BigQuery to SQL.
bigquery etl google-cloud python sql sqlalchemy
Last synced: 09 Feb 2026
https://github.com/prodriguezdefino/apache-beam-streaming-tests
A testing suite for Dataflow streaming pipelines
aggregation bigquery bigtable dataflow gcp kafka pubsub pubsublite streaming
Last synced: 27 Oct 2025
https://github.com/ryanmcdowell/dataflow-bigquery-dynamic-destinations
An example pipeline for dynamically routing events from Pub/Sub to different BigQuery tables based on a message attribute.
apache-beam bigquery google-cloud-dataflow google-cloud-platform
Last synced: 09 Sep 2025
