Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with gcs

A curated list of projects in awesome lists tagged with gcs .

https://github.com/apache/opendal

Apache OpenDAL: One Layer, All Storage.

azblob gcs hacktoberfest redis rust s3 storage

Last synced: 12 May 2025

https://github.com/rilldata/rill-developer

Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code.

bi business-analytics csv data data-analysis data-visualization dataviz duckdb gcs golang parquet parquet-tools parquet-viewer s3 sql sql-editor svelte sveltejs sveltekit

Last synced: 08 Mar 2025

https://github.com/altinity/clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.

azblob backup clickhouse clickhouse-backup clickhousedump dump ftp gcs s3 sftp

Last synced: 29 Jan 2026

https://github.com/Altinity/clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.

azblob backup clickhouse clickhouse-backup clickhousedump dump ftp gcs s3 sftp

Last synced: 04 Apr 2025

https://github.com/fsouza/fake-gcs-server

Google Cloud Storage emulator & testing library.

emulator gcp gcs google-cloud-storage hacktoberfest storage

Last synced: 04 Feb 2026

https://github.com/ysde/grafana-backup-tool

A Python-based application to backup Grafana settings by using the Grafana API

backup docker gcs grafana s3

Last synced: 14 May 2025

https://github.com/beyondstorage/go-storage

A vendor-neutral storage library for Golang: Write once, run on every storage service.

azblob cloud-storage dropbox files fs gcs golang qingstor s3 storage

Last synced: 15 Dec 2025

https://github.com/astronomer/astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

airflow apache-airflow bigquery dags data-analysis data-science elt etl gcs pandas postgres python s3 snowflake sql sqlite workflows

Last synced: 13 Apr 2025

https://github.com/hairyhenderson/go-fsimpl

Go io/fs.FS filesystem implementations for various URL schemes

aws aws-s3 consul distributed-tracing filesystem gcs git go golang iofs opentelemetry s3 url vault

Last synced: 05 Feb 2026

https://github.com/yoshidan/google-cloud-rust

Google Cloud Client Libraries for Rust.

bigquery gcp gcs google-cloud-platform pubsub rust spanner

Last synced: 14 May 2025

https://github.com/hayorov/helm-gcs

Manage Helm 3 repositories on Google Cloud Storage 🔐 **privately**

bucket chart gcs google-cloud-storage helm helm-gcs helm-plugin kubernetes

Last synced: 25 Dec 2025

https://github.com/google-github-actions/upload-cloud-storage

A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket.

actions gcp gcs github-actions google-cloud google-cloud-platform google-cloud-storage

Last synced: 14 May 2025

https://github.com/lichuang/replited

Replicate SQLite to every where(S3\ftp\webdav\google drive\dropbox,etc)

azure-storage backup database ftp gcs replication restore rust s3 sql sqlite sqlite3 storage

Last synced: 04 Apr 2025

https://github.com/boring-registry/boring-registry

A Simple Terraform Provider and Module Registry

aws azure gcs minio registry s3 terraform

Last synced: 30 Mar 2025

https://github.com/justindujardin/pathy

simple, flexible, offline capable, cloud storage with a Python path-like interface

azure blobs bucket-apis bucket-storage-providers gcs pathy s3

Last synced: 20 Oct 2025

https://github.com/thanos-io/objstore

Go module providing unified interface and efficient clients to work with various object storage providers until like GCS, S3, Azure, SWIFT, COS and more.

azure-storage gcs minio object-storage s3 swift

Last synced: 14 May 2025

https://github.com/benjamin-maynard/kubernetes-cloud-mysql-backup

kubernetes-cloud-mysql-backup is a Docker Image based on Alpine Linux that automatically performs backups of MySQL databases, before uploading them to Amazon S3 or Google Cloud Storage. It is designed to be run as a Cronjob in Kubernetes for scheduled database backups. It also features Slack integration.

aws backup batch cronjob database docker gcp gcs google googlecloud iam-users k8s kubernetes mariadb mysql mysql-database s3-bucket schedule slack

Last synced: 01 Oct 2025

https://github.com/firebolt-db/firebolt-core

Firebolt Core is a free, self-hosted edition of Firebolt's distributed query engine (https://www.firebolt.io/); it provides high-performance data warehousing capabilities that can be deployed anywhere from a single laptop to enterprise datacenters.

ai analytics big-data cloud-native database gcs iceberg jdbc parquet postgresql query-engine s3 self-hosted sql

Last synced: 02 Jul 2025

https://github.com/googlecloudplatform/documentai-notebooks

A centralized repository for AI Platform notebooks using the Google Cloud Document AI API.

ai-platform-notebooks cloud gcs notebook

Last synced: 20 Oct 2025

https://github.com/monax/hoard

Hoard is a stateless, deterministically encrypted, content-addressed object store.

aes-gcm azure convergent cryptography daemon encryption filesystem gcs grpc hoard ipfs s3 storage-backend

Last synced: 28 Aug 2025

https://github.com/spotify/gcs-tools

GCS support for avro-tools, parquet-tools and protobuf

avro gcp gcs gcs-connector google-storage parquet protobuf

Last synced: 09 Apr 2025

https://github.com/atlanhq/flyio

Input Output Files in R from Cloud or Local

amazon aws-s3 cloud gcs google r-package rstats

Last synced: 13 Jul 2025

https://github.com/src-d/jgscm

Jupyter support for Google Cloud Storage

gcs google-cloud-storage jupyter notebook

Last synced: 05 May 2025

https://github.com/tharwaninitin/etlflow

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.

aws bigquery dataproc etl etl-framework etl-pipeline gcp gcs redis s3 scala spark zio

Last synced: 28 Feb 2025

https://github.com/goodrobots/mavcesium

An experimental web based map display for MAVProxy based on Cesium

ardupilot cesium gcs ground-station hud mavproxy python tornado websockets

Last synced: 23 Apr 2025

https://github.com/data-integrations/google-cloud

A collection of Google Cloud Platform (GCP) plugins

bigquery cdap cdap-plugin gcs google pubsub

Last synced: 04 Oct 2025

https://github.com/andrcuns/allure-report-publisher

Upload allure reports to cloud storage

allure gcs s3 test-reporting

Last synced: 18 Apr 2026

https://github.com/embarkstudios/tame-gcs

📂 A small library with a limited set of Google Cloud Storage operations 🦀

gcs hacktoberfest rust

Last synced: 02 Jul 2025

https://github.com/googlecloudplatform/spark-on-k8s-gcp-examples

Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub

bigquery cloud-pubsub gcs gcs-connector kubernetes spark

Last synced: 03 Oct 2025

https://github.com/danengelbrecht/golongtail

Command line front end for longtail synchronization tool

archive chunking delivery download gcs gcs-bucket s3 s3-storage synchronization upload

Last synced: 06 Jul 2025

https://github.com/tosh2230/stairlight

A data lineage tool detects table dependencies from rendered SQL statements.

bigquery data-catalog data-discovery data-engineering data-governance data-lineage data-management data-ops dbt gcs lineage redash s3 sql

Last synced: 16 May 2025

https://github.com/guzba/depot

Depot is a helpful companion for working with S3-compatible storage provider APIs.

aws b2 backblaze cloudflare gcs nim r2 s3

Last synced: 12 Apr 2025

https://github.com/hauntsaninja/boostedblob

Command line tool and async library to perform basic file operations on local paths, Google Cloud Storage paths and Azure Blob Storage paths.

asyncio azure cli cloud-storage gcs

Last synced: 11 Sep 2025

https://github.com/janaom/gcp-data-engineering-etl-with-composer-dataflow

This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering solution for processing, storing, and reporting daily transaction data in the online food delivery industry.

airflow apache-beam cloud-composer cloud-storage data-engineering dataflow de-project gcp gcs looker

Last synced: 12 Apr 2025

https://github.com/artefactory/sftp-gateway

This repository contains a docker image configured to use the SSH File Transfer Protocol (SFTP) to transfer all its files to Cloud Blob Storage Services. This image can be deployed on a Kubernetes cluster with Helm.

docker gcs helm inotify kubernetes python3 s3 sftp

Last synced: 24 Oct 2025

https://github.com/hortonworks-spark/cloud-integration

Spark cloud integration: tests, cloud committers and more

apache-spark aws-s3 azure gcs spark

Last synced: 14 Oct 2025

https://github.com/hpc-gridware/clusterscheduler

A drop-in replacement of #SunGridEngine (SGE) or #UnivaGridEngine (UGE) with modern features, stable, and ultra-fast.

gcs gridengine gridwareclusterscheduler hpc jobscheduler ocs openclusterscheduler sge uge

Last synced: 03 Sep 2025

https://github.com/zombiezen/aptblob

CLI for creating an APT repository on S3 or GCS

apt debian gcs packaging s3 ubuntu

Last synced: 25 Oct 2025

https://github.com/orsinium-labs/dimutex

asyncio Python library for distributed mutex with GCS as a backend

distributed-systems gcp gcs google-cloud google-cloud-platform google-cloud-storage mutex python

Last synced: 07 Jul 2025

https://github.com/transitive-bullshit/abstract-object-storage

Collection of useful utilities for working with Google Cloud Storage.

bucket gcp gcs google-cloud google-cloud-storage object-storage storage

Last synced: 14 Oct 2025

https://github.com/mansagroup/gcs-cache-action

A GitHub Action to cache your workload to a Google Cloud Storage bucket.

cache gcs github-action github-actions google-cloud google-cloud-platform google-cloud-storage

Last synced: 05 Oct 2025

https://github.com/ducktors/storagebus

A storage abstraction layer for Node.js that removes any difference among multiple public cloud storage services and local filesystems

gcs gcs-bucket s3 s3-bucket storage

Last synced: 04 Jul 2025

https://github.com/mansagroup/nx-gcs-remote-cache

A Google Cloud Storage distributed cache plugin for Nx.

cache distributed gcp gcs google-cloud-platform google-cloud-storage monorepo nrwl nx

Last synced: 13 Jul 2025

https://github.com/mauri870/gcsfs

Golang io/fs implementation for Google Cloud Storage

gcs golang google-storage iofs

Last synced: 13 Apr 2025

https://github.com/orisano/bctx

bctx is a tool for packing and uploading docker build context.

cloudbuild docker gcs s3

Last synced: 07 May 2025

https://github.com/judahpaul16/canarygc

A web-based ground control station (GCS) for remote autopilot management via the MAVLink protocol.

ardupilot drone gcs iot mavlink plane px4 quadcopter raspberry-pi rc rover sitl uav

Last synced: 11 Jul 2026

https://github.com/paparazzi/pprzgcs

GCS for PaparazziUAV

drone drones gcs uav uavs

Last synced: 07 Apr 2025

https://github.com/raniellyferreira/rotate-files

Rotate files locally, in S3 (api-compatible), Google Storage or Azure Blob Storage based on custom backup rotation scheme

aws azure backup blob-storage cli database devops files gcs golang google lightweight logs rotate rotate-backups rotate-files rotate-logs s3 s3-api storage

Last synced: 17 Oct 2025

https://github.com/cboudereau/gcs-rsync

Lightweight Google Cloud Storage sync Rust Client with better performance than gsutil rsync

gcs gcs-rsync gsutil rsync rust rust-lang

Last synced: 11 Apr 2025

https://github.com/affordablemobiles/gcs-sftp-server

SFTP Server with backing storage via Google Cloud Storage (GCS)

docker gcs golang google-cloud-storage kubernetes sftp

Last synced: 11 Jul 2025

https://github.com/jpalumickas/uplo

Handle file uploads to different storage services like Amazon S3, Google Cloud or etc. It also supports different type of ORM adapters, like Prisma.

aws aws-s3 digitalocean-spaces file file-upload file-upload-server file-uploader files gcs hacktoberfest javascript node nodejs prisma react react-native s3 upload uploader

Last synced: 26 Oct 2025

https://github.com/hackersandslackers/bigquery-python-tutorial

:bar_chart: :snake: Create tables in Google BigQuery, auto-generate their schemas, and retrieve said schemas.

bigquery data-warehouse gcs google-bigquery google-cloud google-cloud-sdk google-cloud-storage python tutorial

Last synced: 28 Apr 2025

https://github.com/relaycorp/object-storage-js

Node.js client for object stores like AWS S3, GCP GCS or Minio

gcs minio object-storage object-store s3

Last synced: 08 Apr 2025

https://github.com/exasol/cloud-storage-extension

Exasol Cloud Storage Extension for accessing formatted data Avro, Orc and Parquet, on public cloud storage systems

avro azure-blob-storage azure-storage cloud-storage exasol exasol-integration gcs orc parquet s3

Last synced: 12 Feb 2026

https://github.com/apache/opendal-go-services

Apache OpenDAL Go Binding Services Releases

azblob gcs redis rust s3 storage

Last synced: 19 Oct 2025

https://github.com/c123r/helm-blob

Helm Plugin that allows you to manage private helm repositories on blob storage(Azure Blob, GCS, S3)

azure-blob-storage cncf gcs helm helm-chart helm-plugins kubernetes s3

Last synced: 08 Oct 2025

https://github.com/russfellows/s3dlio

Part of the sai3 project that delivers multi-protocol storage access for AI/ML workflows, supporting Pytorch, Tensorflow and Jax. This project provides a CLI, along with Rust and Python libraries for AI/ML storage workflows. Supporting S3, File, Azure Blob and GCS using the latest Rust SDKs.

ai-ml ai-storage aws-s3 azure-blob file-io gcs jax python-library pytorch rust rust-crate s3 sai3 tensorflow

Last synced: 13 May 2026

https://github.com/dustinrouillard/screenshot

Custom screenshot tools because of terrible selection of screenshot tools out there.

gcs macos screenshot screenshot-utility

Last synced: 14 Apr 2025

https://github.com/kkdai/linebot-video-gcp

A LINE Bot with Google Cloud Storage and Speech-To-Text features written in Go. Full article https://www.evanlin.com/til-gcp-speech-video-flex/

gcp gcs golang linebot stt

Last synced: 07 May 2025

https://github.com/danmasta/ghost-gcs-adapter

Ghost GCS Storage Adapter

adapter cms gcs ghost storage

Last synced: 30 Jul 2025

https://github.com/mchmarny/long-running-job

Demo showcasing the use container execution feature in GCE to effectively run long running jobs

api container gce gcp gcs golang metadata vm

Last synced: 31 Jul 2025

https://github.com/onsails/cloud-storage-sync

Library for syncing local files with Google Cloud Storage buckets

gcs google google-cloud google-cloud-storage rsync sync

Last synced: 30 Oct 2025

https://github.com/glytching/terraform-gcp-storage-bucket

A Terraform module for creating and managing GCS buckets

gcp gcp-storage gcs hcl inspec terraform terraform-module terraform-modules

Last synced: 27 Oct 2025

https://github.com/soumik12345/dataset-scripts

GCS specific notebooks and scripts to download VOC 2007, COCO 2017 and DIV2K Datasets and create TFRecords.

coco div2k gcs gsutil tensorflow tfrecords voc

Last synced: 12 Jul 2025

https://github.com/jeroenvisser101/gcs_sign

A tiny package for signing Google Cloud Storage URLs and forms

cloud-storage elixir gcs google-cloud google-cloud-storage

Last synced: 12 May 2025

https://github.com/Stono/ghost-static

Ghost blog, in a container, with scripts to upload static content to gcs

blogging docker gcs ghost

Last synced: 03 Apr 2025

https://github.com/ateliware/uploader

The uploader service you desperately need!

gcs google-cloud-storage upload uploader

Last synced: 08 May 2025

https://github.com/fillol/chronicle-sniffer

Plug into extended SecOps: Bring Google Cloud's analytics to your local network. tshark captures on-prem, GCP transforms to UDM. Scalable, event-driven, via Terraform.

cloud-run cybersecurity data-pipeline docker gcp gcs iac network-monitoring network-traffic-analysis pcap pubsub secops security-analytics serverless terraform unified-data-model wireshark

Last synced: 26 Apr 2026

https://github.com/arshtiwari2004/sahyog

Centralized Disaster Response and Inventory Management System that leverages AI and Google Cloud Technologies to predict disasters, optimize resource management, and provide real-time coordination.

adam-optimizer apache-kafka apache-spark cnn-classification docker efficientnetb4 gan-models gcp-storage gcs gemini-api hyperledger-fabric keras-classification-models l2-regularization lstm-model numpy opencv tensorflow twilio vertex-ai yolov8-detection

Last synced: 13 May 2025

https://github.com/ecmwf/open-ecpds

OpenECPDS boosts the efficiency and productivity of data services by using proven and innovative technologies. It offers a portable, adaptable application for diverse environments, with a user-friendly tool for managing data acquisition, dissemination with push/pull mechanisms, and a notification system, all using standard protocols.

acquisition azure dissemination ftp ftp-server gcs https https-server mqtt mqtt-broker notification-service rest-api s3 sftp

Last synced: 12 Apr 2025

https://github.com/userbradley/gcs-web-server

Infrastructure to deploy and host a website on GKE using NGINX and GCS

cloud gcs google google-cloud google-cloud-storage kubernetes nginx

Last synced: 13 Feb 2026

https://github.com/shareed2k/storage

Cloud storage abstraction package for Go build on top of Rclone inspired by Laravel File Storage

file gcs golang rclone s3 storage

Last synced: 12 Aug 2025

https://github.com/googlecloudplatform/dcm2bq

A service for creating a JSON metadata representation for DICOM from multiple input sources and storing into Google Cloud Big Query (BQ).

bigquery dicom gcs googlecloud googlecloudplatform googlecloudstorage json

Last synced: 28 Feb 2026

https://github.com/secjs/storage

🗃️ Handle your application files in Node.js

cloud disk filesystem gcs nodejs s3 storage

Last synced: 18 Apr 2026

https://github.com/arnaud-dezandee/packer-vagrant-box-gcs

Packer plugin to upload Vagrant boxes to Google GCS.

gcs google-cloud-storage packer packer-plugin vagrant

Last synced: 04 Apr 2025

https://github.com/bemusic/bmsurge-rebooted

Infrastructure for Be-Music Surge, a nonstop BMS radio station. BMS packages are renderered into MP3 files at large scale using Google Cloud Run.

cloud-run docker express firebase gcs mongodb mongodb-atlas sonarcloud

Last synced: 11 Apr 2026