Ecosyste.ms: Awesome

https://github.com/scratchdata/scratchdata

Scratch is a swiss army knife for big data.

bigquery clickhouse data-warehouse duckdb hacktoberfest motherduck olap redshift snowflake

Last synced: 26 Jun 2024

https://github.com/Snowflake-Labs/sfguide-recommender-pipeline

Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker

amazon-sagemaker machine-learning sagemaker sagemaker-example snowflake snowflake-guide snowflakedb

Last synced: 21 Jun 2024

https://github.com/Snowflake-Labs/sfguide-twitter-auto-ingest

Learn how to auto-ingest streaming data into Snowflake using Snowpipe.

demo snowflake snowpipe twitter

Last synced: 21 Jun 2024

https://github.com/CartoDB/analytics-toolbox-core

A set of UDFs and Procedures to extend BigQuery, Snowflake, Redshift, Postgres and Databricks with Spatial Analytics capabilities

analytics-toolbox bigquery carto databricks geospatial gis postgres redshift snowflake sql

Last synced: 21 Jun 2024

https://github.com/Snowflake-Labs/Excelerator

This is an Excel Addin for Windows that reads and writes data to Snowflake

excel odbc-driver snowflake windows

Last synced: 16 Jun 2024

https://github.com/Snowflake-Labs/awesome-snowflake

A curated list of resources about Snowflake

awesome-list snowflake snowflakedb

Last synced: 16 Jun 2024

https://github.com/bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

bigquery copy-database data-ingestion data-integration data-pipeline duckdb ingestion-pipeline mssql postgresql snowflake

Last synced: 15 Jun 2024

https://github.com/starlake-ai/jsqltranspiler

Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL (with deep transformation of functions, data types and format characters) using Java.

bigquery databricks duckdb java query redshift rewrite snowflake transpiler

Last synced: 13 Jun 2024

https://github.com/gp2112/snowflake-cli

A simple python script for hosting a Snowflake Proxy in your python program or with it's standalone cli

onion privacy proxy python snowflake tor

Last synced: 07 Jun 2024

https://github.com/justwatchcom/sql_exporter

Flexible SQL Exporter for Prometheus.

athena clickhouse kubernetes mysql postgresql prometheus prometheus-exporter snowflake sql

Last synced: 07 Jun 2024

https://github.com/artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

apache-kafka bigquery cdc change-data-capture data-integration data-pipelines database debezium elt golang kafka redshift snowflake

Last synced: 07 Jun 2024

https://github.com/PeerDB-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

bigquery cdc clickhouse cloud-native distributed-systems etl eventhubs kafka postgres postgresql realtime rust s3 snowflake sql stream-processing

Last synced: 07 Jun 2024

https://github.com/ibis-project/ibis

the portable Python dataframe library

bigquery clickhouse dask database datafusion duckdb impala mssql mysql pandas polars postgresql pyarrow pyspark python snowflake sql sqlalchemy sqlite trino

Last synced: 05 Jun 2024

https://github.com/Multiwoven/multiwoven

🔥🔥🔥 Open Source Alternative to Hightouch, Census, and RudderStack. Leading Reverse ETL and Customer Data Platform (CDP) for Data Teams.

bigquery cdp customer-data-platform data-activation data-engineering data-pipeline data-warehouse databricks dbt etl hacktoberfest open-source postresql react redshift reverse-etl ruby self-hosted snowflake typescript

Last synced: 05 Jun 2024

https://github.com/sodadata/soda-core

:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

data-contracts data-engineering data-governance data-monitoring data-observability data-profiling data-quality data-quality-checks data-quality-monitoring data-quality-testing data-reliability data-testing data-unit-tests data-validation dataquality datatesting dbt pipeline-testing python snowflake

Last synced: 02 Jun 2024

https://github.com/ploomber/jupysql

Better SQL in Jupyter. 📊

bigquery clickhouse data-engineering data-science duckdb hive jupyter mysql polars postgres presto python redshift snowflake spark-sql sql sqlite trino tsql

Last synced: 02 Jun 2024

https://github.com/grai-io/grai-core

data data-lineage data-science dataengineering datalineage dbt django fivetran hacktoberfest mssql mysql open-source parquet postgresql python redshift snowflake

Last synced: 02 Jun 2024

https://github.com/apache/doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

bigquery database dbt delta-lake elt etl hadoop hive hudi iceberg lakehouse olap query-engine real-time redshift snowflake spark sql

Last synced: 31 May 2024

https://github.com/discordlookup/discordlookup

DiscordLookup | Get more out of Discord with Discord Lookup! Snowflake Decoder, Guild List with Stats, Invite Info and more...

discord discord-bot discord-tools discordlookup experiments hacktoberfest lookup snowflake tools

Last synced: 31 May 2024

https://github.com/RobThree/IdGen

Twitter Snowflake-alike ID generator for .Net

c-sharp dotnet idgenerator snowflake twitter-snowflake unique-id

Last synced: 31 May 2024

https://github.com/zsvoboda/dbd

dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

bigquery csv database database-schemas elt etl excel json mysql parquet postgresql python python3 redshift snowflake sql sqlite xls xlsx

Last synced: 27 May 2024

https://github.com/anvilogic-forge/armory

Anvilogic Forge

detection detection-engineering snowflake splunk threat-hunting

Last synced: 26 May 2024

https://github.com/vd2org/snowflake

The Snowflake generator done right.

id id-generator ids snowflake snowflake-id

Last synced: 21 May 2024

https://github.com/kaarthik108/snowChat

Chat snowflake database - Text to SQL

aws bedrock chatgpt gpt-4 langchain llama mistral python snowflake snowpark streamlit supabase vector

Last synced: 19 May 2024

https://github.com/mlcraft-io/mlcraft

Synmetrix – open source semantic layer / Boost your LLM precision

big-data bigquery business-intelligence clickhouse cube cubejs data-engineering databricks dremio druid firebolt llm prestodb redshift semantic-layer snowflake vertica

Last synced: 19 May 2024

https://github.com/sqlchat/sqlchat

Chat-based SQL Client and Editor for the next decade

chatgpt clickhouse cockroachdb gpt mongodb mssql mysql openai oracle postgres redis snowflake sql sqlclient sqleditor sqlite sqlserver text-to-sql text2sql tidb

Last synced: 18 May 2024

https://github.com/yitter/IdGenerator

💎多语言实现，高性能生成唯一数字ID。 💎优化的雪花算法（SnowFlake）——雪花漂移算法，在缩短ID长度的同时，具备极高瞬时并发处理能力（50W/0.1s）。 💎原生支持 C#/Java/Go/Rust/C/JavaScript/TypeScript/Python/Pascal 多语言，提供其它适用于其它语言的多线程安全调用动态库（FFI）。💎支持容器环境自动扩容（自动注册 WorkerId ），单机或分布式唯一IdGenerator。💎顶尖优化，超强效能。

c c-sharp delphi dotnet go idgenerator java javascript pascal php rust snowflake sql twritter-snowflake typescript unique-id

Last synced: 18 May 2024

https://github.com/rudderlabs/rudder-server

Privacy and Security focused Segment-alternative, in Golang and React

bigquery customer-data customer-data-lake customer-data-pipeline customer-data-platform data-integration data-pipeline data-synchronization data-warehouse etl golang hybrid-cloud privacy redshift rudderstack security segment-alternative snowflake warehouse-first warehouse-management

Last synced: 16 May 2024

https://github.com/snowflakedb/snowflake-cli

Snowflake CLI is an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations.

cli data-ops devops-tools snowflake sql

Last synced: 15 May 2024

https://github.com/tobymao/sqlglot

Python SQL Parser and Transpiler

bigquery clickhouse databricks duckdb hive mysql optimizer parser postgres presto python redshift snowflake spark sql sqlite sqlparser transpiler trino tsql

Last synced: 15 May 2024

https://github.com/HotPotatoC/snowflake

❄️ Simple twitter's snowflake uniquely identifiable descriptors (IDs) format generator for Go

go golang snowflake snowflake-twitter unique-id unique-identifier

Last synced: 14 May 2024

https://github.com/omnata-labs/dbt-ml-preprocessing

A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.

bigquery dbt redshift scikit-learn snowflake

Last synced: 13 May 2024

https://github.com/datafold/data-diff

Compare tables within or across databases

data data-diffing data-engineering data-quality data-quality-monitoring data-science database databricks-sql dataengineering dataquality dbt mysql oracle-database postgres postgresql python rdbms snowflake sql trino

Last synced: 13 May 2024

https://github.com/ScalefreeCOM/datavault4dbt

Scalefree's dbt package for a Data Vault 2.0 implementation congruent to the original Data Vault 2.0 definition by Dan Linstedt including the Staging Area, DV2.0 main entities, PITs and Snapshot Tables.

azure-synapse bigquery datavault dbt dbt-packages exasol google-bigquery hubs links pits postgresql redshift satellites scalefree snapshots snowflake sourcemarts stagingarea

Last synced: 13 May 2024

https://github.com/get-select/dbt-snowflake-monitoring

A dbt package from SELECT to help you monitor Snowflake performance and costs

dbt snowflake

Last synced: 13 May 2024

https://github.com/datnguye/dbt-translate

✨ Let's translate Numbers to Words ✨

dbt money2word num2word package snowflake sql sqlserver udf

Last synced: 13 May 2024

https://github.com/bwmarrin/snowflake

A simple to use Go (golang) package to generate or parse Twitter snowflake IDs

go golang snowflake

Last synced: 13 May 2024

https://github.com/dbt-checkpoint/dbt-checkpoint

:fishing_pole_and_fish: List of `pre-commit` hooks to ensure the quality of your `dbt` projects.

bigquery business-intelligence dbt pre-commit pre-commit-hook quality-assurance snowflake sql

Last synced: 13 May 2024

https://github.com/elementary-data/elementary

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

analytics-engineer bigquery data-analysis data-governance data-lineage data-observability data-pipeline data-pipelines data-reliability data-warehouse dataops dbt dbt-artifacts dbt-packages lineage redshift snowflake

Last synced: 13 May 2024

https://github.com/ByConity/ByConity

ByConity is an open source cloud data warehouse

bytedance clickhouse clickhouse-database cloud kubernets lakehouse olap s3 snowflake sql tiktok

Last synced: 13 May 2024

https://github.com/DataKitchen/data-observability-installer

Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.

data data-engineering data-observability data-profiling data-quality data-reliability data-science datachecker datacleaner datacleaning dataops dataquality datatesting datavalidation mssql pipeline-tests postgresql redshift self-hosted snowflake

Last synced: 12 May 2024

https://github.com/airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

bigquery change-data-capture data data-analysis data-collection data-engineering data-integration data-pipeline elt etl java mssql mysql pipeline postgresql python redshift s3 self-hosted snowflake

Last synced: 09 May 2024

https://github.com/growthbook/growthbook

Open Source Feature Flagging and A/B Testing Platform

ab-testing abtest abtesting analytics bigquery clickhouse continuous-delivery data-analysis data-engineering data-science experimentation feature-flagging feature-flags mixpanel redshift remote-config snowflake split-testing statistics

Last synced: 09 May 2024

https://github.com/cuebook/CueObserve

Timeseries Anomaly detection and Root Cause Analysis on data in SQL data warehouses and databases

anomaly anomaly-detection bigquery datawarehouse prophet-facebook redshift root-cause-analysis snowflake sql timeseries-analysis timeseries-forecasting

Last synced: 09 May 2024

https://github.com/snowflakedb/snowflake-kafka-connector

Snowflake Kafka Connector (Sink Connector)

java kafka kafka-connect snowflake

Last synced: 07 May 2024

https://github.com/fluentmigrator/fluentmigrator

Fluent migrations framework for .NET

database database-schema db2 dotnet fluentmigrator migration mysql oracle postgres redshift snowflake sql sql-server

Last synced: 05 May 2024

https://github.com/rdagumampan/yuniql

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

amazon-rds azure-sql-database data-engineering database-migrations datawarehouse dotnet-core dotnet-tool mariadb mysql oracle postgresql redshift snowflake sql sqlserver yuniql

Last synced: 05 May 2024

https://github.com/snowflake-so/move-cron-parser

Cron parser library written for Move language

cron cron-parser snowflake

Last synced: 03 May 2024

https://github.com/Ahoo-Wang/CosId

Universal, flexible, high-performance distributed ID generator. | 通用、灵活、高性能的分布式 ID 生成器

clock clock-synchronization cloud-native distributed generator gradle id id-generator idgenerator java k8s kubernetes microservice redis sharding snowflake spring spring-boot spring-cloud zookeeper

Last synced: 02 May 2024

https://github.com/muyo/sno

Compact, sortable and fast unique IDs with embedded metadata.

id-generator snowflake unique-id uuid

Last synced: 30 Apr 2024

https://github.com/edwingeng/wuid

An extremely fast globally unique number generator.

guid snowflake uuid

Last synced: 30 Apr 2024

https://github.com/osamingo/indigo

A distributed unique ID generator of using Sonyflake and encoded by Base58

base58 go id identity snowflake sonyflake

Last synced: 29 Apr 2024

https://github.com/bytebase/bytebase

The GitLab/GitHub for database DevOps. World's most advanced database DevOps and CI/CD for Developer, DBA and Platform Engineering teams.

cicd clickhouse dbeaver devops github gitlab gitops mongodb mysql oracle postgres postgresql redis schema-changes schema-migrations snowflake sql sql-client sqlserver tidb

Last synced: 29 Apr 2024

https://github.com/build-trust/ockam

Orchestrate end-to-end encryption, cryptographic identities, mutual authentication, and authorization policies between distributed applications – at massive scale.

authentication authorization credentials distributed-systems e2ee encrypted-connections encrypted-messages encryption end-to-end-encryption identity kafka key-management messaging rust security snowflake trust zero-trust

Last synced: 29 Apr 2024

https://github.com/datafuselabs/databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com

ai bigdata database rust serverless snowflake

Last synced: 29 Apr 2024

https://github.com/bartonhammond/snowflake

:snowflake: A React-Native Android iOS Starter App/ BoilerPlate / Example with Redux, RN Router, & Jest with the Snowflake Hapi Server running locally or on RedHat OpenShift for the backend, or a Parse Server running locally or remotely on Heroku

async-storage bitrise eslint fastlane hapi-server i18n immutablejs jest key-mirror parse-server react-native react-native-router-flux react-native-vector-icons redux snowflake standard-js tcomb-form-native validate

Last synced: 25 Apr 2024

https://github.com/dragonfruitnetwork/onionfruit

OnionFruit™ Connect - Tor access client with country selection, bridge configuration, pluggable transports and experimental DNS support

bridge dns meek obfs4 obfs4proxy onion onionfruit proxy snowflake socks5 tor tor-bridge tor-browser tor-network vpn

Last synced: 24 Apr 2024

https://github.com/open-metadata/OpenMetadata

Open Standard for Metadata. A Single place to Discover, Collaborate and Get your data right.

data-catalog data-collaboration data-contracts data-discovery data-governance data-lineage data-observability data-profiling data-quality data-quality-checks data-science data-validation datacatalog datadiscovery dataengineering dataquality dbt metadata metadata-management snowflake

Last synced: 22 Apr 2024

https://github.com/marco-roy/DDO

A DBT package to perform DataOps & administrative CI/CD on your data warehouse.

data dataops datawarehouse datawarehouseautomation dbt snowflake

Last synced: 22 Apr 2024

https://github.com/rupurt/odbc-scanner-duckdb-extension

A DuckDB extension to read data directly from databases supporting the ODBC interface

analytics bigquery columnar-database cpp data-engineering db2 duckdb mariadb mssql mysql nix odbc olap oracle postgres snowflake vector-engine

Last synced: 20 Apr 2024

https://github.com/jitsucom/jitsu

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

bigquery clickhouse data-collection data-connectors data-integration golang postgres redshift snowflake

Last synced: 18 Apr 2024

https://github.com/Canner/vulcan-sql

Data API Framework for AI Agents and Data Apps

ai ai-agent analytics api-builder bigquery clickhouse data-lake data-warehouse database duckdb ksqldb postgresql reporting restful-api snowflake spreadsheet sql typescript vulcan-sql vulcansql

Last synced: 13 Apr 2024

https://github.com/k1LoW/tbls

tbls is a CI-Friendly tool for document a database, written in Go.

bigquery continuous-integration database-document database-schema documentation-tool dynamodb er-diagram excel hacktoberfest mariadb markdown mermaid mysql plantuml postgresql redshift snowflake spanner sqlite sqlserver

Last synced: 13 Apr 2024

https://github.com/sqitchers/sqitch

Sensible database change management

cockroachdb database exasol firebird mysql oracle postgresql snowflake sqitch sqlite vertica yugabytedb

Last synced: 13 Apr 2024

https://github.com/Tencent/APIJSON

🏆 零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码，前端(客户端) 定制返回 JSON 的数据和结构。 🏆 A JSON Transmission Protocol and an ORM Library 🚀 provides APIs and Docs without writing any code.

baas clickhouse crud databricks elasticsearch hadoop hive influxdb low-code lowcode milvus nocode oracle postgresql postgresql-database serverless snowflake sqlserver tdengine tidb

Last synced: 13 Apr 2024

https://github.com/getstrm/pace

Data policy IN, dynamic view OUT: PACE is the Policy As Code Engine. It helps you to programatically create and apply a data policy to a processing platform like Databricks, Snowflake or BigQuery, with definitions imported from Collibra, Datahub, ODD and the like.

bigquery data-catalog data-contracts data-governance data-processing databricks policy-enforcement snowflake

Last synced: 11 Apr 2024

https://github.com/Datavault-UK/automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

data-vault dataengineering datalake datavault datavault20 datawarehouse datawarehousing dbt elt etl metadata snowflake sql