Projects in Awesome Lists tagged with cloud-composer
A curated list of projects in awesome lists tagged with cloud-composer .
https://github.com/GoogleCloudPlatform/public-datasets-pipelines
Cloud-native, data onboarding architecture for Google Cloud Datasets
airflow bigquery cloud-composer cloud-native cloud-storage data-architecture data-engineering data-pipelines datasets google-cloud open-data
Last synced: 23 Apr 2025
https://github.com/googlecloudplatform/public-datasets-pipelines
Cloud-native, data onboarding architecture for Google Cloud Datasets
airflow bigquery cloud-composer cloud-native cloud-storage data-architecture data-engineering data-pipelines datasets google-cloud open-data
Last synced: 12 Apr 2025
https://github.com/sungchun12/airflow-toolkit
Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) :desktop_computer: >> [ :rocket:, :ship: ]
actions airflow airflow-environments airflow-toolkit cloud cloud-composer composer dbt docker gcp google-cloud hcl kubernetes kubernetes-deployment python python3 shell-script terraform terragrunt terragrunt-deployment
Last synced: 16 Mar 2025
https://github.com/doitintl/doit-composer-airflow-training
Getting started with Apache Airflow on Cloud Composer
apache-airflow book cloud-composer google-cloud-platform mini-book
Last synced: 30 Apr 2025
https://github.com/janaom/gcp-data-engineering-etl-with-composer-dataflow
This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering solution for processing, storing, and reporting daily transaction data in the online food delivery industry.
airflow apache-beam cloud-composer cloud-storage data-engineering dataflow de-project gcp gcs looker
Last synced: 12 Apr 2025
https://github.com/googlecloudplatform/cloud-composer-mssql-dataflow-bigquery
This repository contains an example of how to leverage Cloud Composer and Cloud Dataflow to move data from a Microsoft SQL Server to BigQuery. The diagrams below demonstrate the workflow pipeline.
airflow bigquery cloud-composer dataflow microsoft-sql-server
Last synced: 05 Feb 2025
https://github.com/wintermi/cloud-composer-trigger
Examples on how to Trigger a Google Cloud Composer DAG
airflow cloud-composer cloud-function google-cloud google-cloud-platform
Last synced: 22 Feb 2025
https://github.com/sofiasawczenko/etl_employee_info_pipeline
This repository contains code and configuration files for an Extract, Transform, Load (ETL) project using Google Cloud Data Fusion for data extraction, Apache Airflow/Composer for orchestration, and Google BigQuery for data loading.
airflow cloud-composer gcp-bigquery gcp-cloud-storage
Last synced: 01 Apr 2025
