Projects in Awesome Lists tagged with gcs-bucket

https://github.com/danengelbrecht/golongtail

Command line front end for longtail synchronization tool

archive chunking delivery download gcs gcs-bucket s3 s3-storage synchronization upload

Last synced: 07 May 2025

https://github.com/ducktors/storagebus

A storage abstraction layer for Node.js that removes any difference among multiple public cloud storage services and local filesystems

gcs gcs-bucket s3 s3-bucket storage

Last synced: 09 Apr 2025

https://github.com/gbotemib/gharchive_de_project

An end-to-end data engineering project on github activities data

bigquery dbtcloud docker gcp gcs-bucket looker-studio prefect spark terraform

Last synced: 27 Feb 2025

https://github.com/mesmacosta/datacatalog-fileset-enricher

A Python package to enrich Google Cloud Data Catalog Fileset Entries with tags.

bigdata datacatalog docker fileset-enricher fileset-entries gcp-datacatalog gcp-storage gcs-bucket python statistics

Last synced: 30 Apr 2025

https://github.com/mongoexpuser/object-storage-interaction

Interacting with public cloud object storage

aws aws-comprehend backblaze backblaze-b2 boto3 dask dask-sql gcp gcs-bucket linode linode-objs object-storage pandas pandasql pyspark python s3-bucket s3-select s3fs storage

Last synced: 22 Mar 2025

https://github.com/george-nyamao/gcp_etl_project

An ETL pipeline to move an uploaded flat file ffrom GCS, mask PII, store Big Query, and Create a report in Looker.

airflow bigquery cloudcomposer data-fusion gcs-bucket looker python3 wrangler

Last synced: 15 Mar 2025

https://github.com/hitthecodelabs/gcsbucketbridge

Migrate data between Google Cloud Storage (GCS) buckets

gcloud-sdk gcs-bucket gsutil python script

Last synced: 08 Apr 2025

https://github.com/bervproject/gcs-blob-store

Blob store to store in Google Cloud Storage

gcs-bucket

Last synced: 19 Feb 2025

https://github.com/victorcezeh/data-engineering-final-semester-portfolio

This GitHub repository serves as a comprehensive platform for managing and showcasing my data engineering projects and assessments throughout my final semester at Alt School Africa. Designed to foster collaboration, organization, and continuous improvement, this repository is the backbone of my academic journey in data engineering.

bigquery docker gcs-bucket postgresql python

Last synced: 11 May 2025

https://github.com/kaushik-puttaswamy/walmart-sales-data-ingestion-and-transformation-in-bigquery-using-airflow

An ETL pipeline that ingests Walmart sales data from Google Cloud Storage into BigQuery, automates table creation, and performs data transformation using SQL MERGE with Apache Airflow.

airflow-dags bigquery etl-pipeline gcs-bucket google-cloud-platform merge python sql transformation