Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with duckdb

A curated list of projects in awesome lists tagged with duckdb .

https://github.com/canner/wrenai

πŸ€– Open-source GenBI AI Agent that empowers data-driven teams to chat with their data to generate Text-to-SQL, charts, spreadsheets, reports, dashboards and BI. πŸ“ˆπŸ“ŠπŸ“‹πŸ§‘β€πŸ’»

agent anthropic bedrock bigquery business-intelligence charts duckdb genbi llm openai postgresql rag spreadsheets sql sqlai text-to-sql text2sql vertex

Last synced: 12 May 2025

https://github.com/evidence-dev/evidence

Business intelligence as code: build fast, interactive data visualizations in SQL and markdown

analytics business-intelligence dashboard data-engineering data-science data-visualization dbt duckdb exploratory-data-analysis self-hosted sql svelte tailwindcss webassembly

Last synced: 13 May 2025

https://github.com/lancedb/lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

apache-arrow computer-vision data-analysis data-analytics data-centric data-format data-science dataops deep-learning duckdb embeddings llms machine-learning mlops python rust

Last synced: 05 May 2025

https://github.com/dotnetcore/freesql

.NET aot orm, VB.NET/C# orm, Mysql/PostgreSQL/SqlServer/Oracle orm, Sqlite/Firebird/Clickhouse/DuckDB orm, θΎΎζ’¦/金仓/θ™šθ°·/翰高/ι«˜ζ–― orm, η₯žι€š orm, ε—ε€§ι€šη”¨ orm, ε›½δΊ§ orm, TDengine orm, QuestDB orm, MsAccess orm.

access clickhouse codefirst dameng duckdb firebird gbase hacktoberfest highgo kingbase mysql odbc oracle orm postgresql questdb shentongdata sqlite sqlserver xugu

Last synced: 14 May 2025

https://github.com/dotnetcore/FreeSql

πŸ¦„ .NET aot orm, C# orm, VB.NET orm, Mysql orm, Postgresql orm, SqlServer orm, Oracle orm, Sqlite orm, Firebird orm, θΎΎζ’¦ orm, 人倧金仓 orm, η₯žι€š orm, 翰高 orm, ε—ε€§ι€šη”¨ orm, θ™šθ°· orm, ε›½δΊ§ orm, Clickhouse orm, DuckDB orm, TDengine orm, QuestDB orm, MsAccess orm.

access clickhouse codefirst dameng duckdb firebird gbase hacktoberfest highgo kingbase mysql odbc oracle orm postgresql questdb shentongdata sqlite sqlserver xugu

Last synced: 21 Mar 2025

https://github.com/antonycourtney/tad

A desktop application for viewing and analyzing tabular data

csv data-analysis data-science database desktop-application duckdb parquet-viewer pivot-tables pivots tabular-data

Last synced: 22 Mar 2025

https://github.com/bruin-data/ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

bigquery copy-database data-ingestion data-integration data-pipeline duckdb ingestion-pipeline mssql postgresql snowflake

Last synced: 13 May 2025

https://github.com/rilldata/rill

Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code.

bi business-analytics csv data data-analysis data-visualization dataviz duckdb gcs golang parquet parquet-tools parquet-viewer s3 sql sql-editor svelte sveltejs sveltekit

Last synced: 13 May 2025

https://github.com/Canner/WrenAI

πŸš€ Open-source SQL AI Agent for Text-to-SQL. Supporting PostgreSQL, DuckDB, MySQL, MS SQL, ClickHouse, Trino, JSON, CSV, Parquet data sources, and more! πŸš€

agent ai bigquery duckdb fastapi gpt hacktoberfest llm nextjs nlp openai postgresql python rag sql sqlai text-to-sql text2sql typescript

Last synced: 25 Nov 2024

https://github.com/rilldata/rill-developer

Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code.

bi business-analytics csv data data-analysis data-visualization dataviz duckdb gcs golang parquet parquet-tools parquet-viewer s3 sql sql-editor svelte sveltejs sveltekit

Last synced: 08 Mar 2025

https://github.com/moj-analytical-services/splink

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

data-matching data-science deduplicate-data deduplication duckdb em-algorithm entity-resolution fuzzy-matching record-linkage spark uk-gov-data-science

Last synced: 13 May 2025

https://github.com/BemiHQ/BemiDB

Single-binary Postgres read replica optimized for analytics

analytics data-lakehouse data-movement data-warehouse duckdb iceberg olap parquet postgresql replication zero-etl

Last synced: 01 May 2025

https://github.com/duckdb/dbt-duckdb

dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)

dbt duckdb

Last synced: 14 May 2025

https://github.com/uwdata/mosaic

An extensible framework for linking databases and interactive views.

data duckdb interaction scalability visualization

Last synced: 14 May 2025

https://github.com/arkflow-rs/arkflow

High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.

arkflow datafusion duckdb flow kafka mysql postgresql rust rust-lang sql sqlite stream tokio tokio-rs

Last synced: 16 May 2025

https://github.com/deepseek-ai/smallpond

A lightweight data processing framework built on DuckDB and 3FS.

data-processing duckdb

Last synced: 09 Mar 2025

https://github.com/litestar-org/advanced-alchemy

A carefully crafted, thoroughly tested, optimized companion library for SQLAlchemy

alembic cockroachdb duckdb fastapi flask litestar litestar-org mssql mysql oracle-db postgresql repository-pattern sanic spanner sqlalchemy sqlite starlette

Last synced: 14 May 2025

https://github.com/giorgi/duckdb.net

Bindings and ADO.NET Provider for DuckDB

ado-net data-science duckdb duckdb-database hacktoberfest

Last synced: 14 May 2025

https://github.com/incentius-foss/WhatTheDuck

WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data.

csv duckdb sql

Last synced: 31 Dec 2024

https://github.com/Giorgi/DuckDB.NET

Bindings and ADO.NET Provider for DuckDB

ado-net data-science duckdb duckdb-database hacktoberfest

Last synced: 24 Mar 2025

https://github.com/ark-flow/arkflow

High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.

datafusion duckdb flow kafka mysql postgresql rust rust-lang sql sqlite stream tokio tokio-rs

Last synced: 03 Apr 2025

https://github.com/ekzhang/inline-sql

πŸͺ„ Inline SQL in any Python program

dsl duckdb olap python query-language sql

Last synced: 13 Apr 2025

https://github.com/mause/duckdb_engine

SQLAlchemy driver for DuckDB

duckdb duckdb-engine python sql sqlalchemy

Last synced: 14 May 2025

https://github.com/Mause/duckdb_engine

SQLAlchemy driver for DuckDB

duckdb duckdb-engine python sql sqlalchemy

Last synced: 24 Mar 2025

https://github.com/turbot/powerpipe

Powerpipe: Dashboards for DevOps. Visualize cloud configurations. Assess security posture against a massive library of benchmarks. Build custom dashboards with code.

aws azure cis cloud cnapp cspm dashboards devops devsecops duckdb gcp hacktoberfest hcl kubernetes mysql postgresql security sql sqlite terraform

Last synced: 16 May 2025

https://github.com/rpbouman/huey

Light-weight, browser-based ROLAP pivot tables on top of DuckDB-WASM

data duckdb excel pivot-tables rolap small-data sql

Last synced: 24 Mar 2025

https://github.com/duckdb/duckdb-rs

Ergonomic bindings to duckdb for Rust

arrow database duckdb ffi ffi-bindings olap rust

Last synced: 07 Apr 2025

https://github.com/alitrack/duckdb_fdw

DuckDB Foreign Data Wrapper for PostgreSQL

duckdb duckdb-fdw fdw foreign-data-wrapper postgresql

Last synced: 24 Mar 2025

https://github.com/l1xnan/duckling

A fast viewer for CSV/Parquet files and databases such as DuckDB, SQLite, PostgreSQL, MySQL, Clickhouse, etc., base on Tauri

clickhouse duckdb mysql parquet postgresql rust sqlite tauri

Last synced: 16 May 2025

https://github.com/sutoiku/puffin

Serverless HTAP cloud data platform powered by Arrow Γ— DuckDB Γ— Iceberg

arrow duckdb iceberg serverless

Last synced: 27 Apr 2025

https://github.com/datonic/datadex

πŸ“¦ Serverless and local-first Open Data Platform

dbt duckdb open-data quarto sql

Last synced: 24 Mar 2025

https://github.com/nshiab/simple-data-analysis

Easy-to-use and high-performance JavaScript library for data analysis. Works with tabular and geospatial data.

analysis bun data data-analysis data-science duckdb geospatial javascript node node-js nodejs spatial spatial-analysis sql typescript

Last synced: 10 Apr 2025

https://github.com/kraina-ai/quackosm

QuackOSM: an open-source Python and CLI tool for reading OpenStreetMap PBF files using DuckDB

duckdb geo geospatial openstreetmap osm pbf python

Last synced: 15 May 2025

https://github.com/evidence-dev/duckdb_gsheets

DuckDB extension to read and write Google Sheets using SQL

duckdb duckdb-extension google-sheets

Last synced: 23 Apr 2025

https://github.com/theseus-rs/rsql

Command line SQL interface for relational databases and common data file formats

cockroachdb command-line csv data database duckdb json mariadb mysql parquet postgres postgresql redshift snowflake sql sqlite sqlite3 sqlserver

Last synced: 16 May 2025

https://github.com/duckdb/duckdb-sqlite

DuckDB extension to read and write to SQLite databases

duckdb sql sqlite

Last synced: 12 Apr 2025

https://github.com/duckdb/sqlite_scanner

DuckDB extension to read and write to SQLite databases

duckdb sql sqlite

Last synced: 13 Dec 2024

https://github.com/metrico/quackpipe

QuackPipe is an OLAP API built on top of DuckDB with ClickHouse compatibility bits

api clickhouse clickhouse-server csv database duckdb duckdb-api duckdb-engine gigapipe golang lambda lambda-functions olap parquet qryn rest-api s3 server sql

Last synced: 04 Apr 2025

https://github.com/cnstlungu/portable-data-stack-dagster

A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB and Superset

business-intelligence dagster data-engineering data-visualization dbt duckdb python superset

Last synced: 01 May 2025

https://github.com/isaacbrodsky/h3-duckdb

Bindings for H3 to DuckDB

duckdb geospatial h3 hexagon spatial-indexing

Last synced: 10 Apr 2025

https://github.com/quackscience/duckdb-extension-httpserver

DuckDB HTTP API Server and Query Interface in a Community Extension

api api-server community-extension duckdb extension http-server olap user-interface

Last synced: 05 Apr 2025

https://github.com/duckdb/duckdb-r

The duckdb R package

database duckdb olap r

Last synced: 13 Apr 2025

https://github.com/dagster-io/mdsfest-opensource-mds

Demo Project for Open Source MDS

dagster data duckdb mds modern stack

Last synced: 05 May 2025

https://github.com/buremba/universql

The bridge to effortless multi-engine data applications, currently supports Snowflake ❄️ and DuckDB πŸ¦†

databricks dbt duckdb proxy-server snowflake sql sql-proxy sqlglot

Last synced: 12 Apr 2025

https://github.com/valmi-io/valmi-activation

⚑ valmi.io reverse ETL (data activation) is the open source ( OSS ) data activation platform to load data from warehouses into Webhooks and SaaS tools like Klaviyo, Facebook Ads, Salesforce, Braze etc. Valmi.io Customer Data Platform (CDP) helps track and ingest user activity events from websites, shopify, serverside events. https://cloud.valmi.io

airbyte cdp composable-cdp dagster dbt duckdb ecommerce email-marketing etl event-ingestion event-tracking marketing-automation modern-data-stack open-source push-notifications reverse-etl shopify shopify-app shopify-events user-behavior

Last synced: 01 May 2025

https://github.com/mattf96s/quackdb

Open-source in-browser DuckDB SQL editor

apache-arrow comlink duckdb duckdb-wasm parquet remix remix-run shadcn sql sst

Last synced: 16 Apr 2025

https://github.com/dbt-labs/jaffle_shop_duckdb

Get started with dbt in less than 1 minute from `git clone` to `dbt docs serve` for free!

data dbt duckdb sql

Last synced: 07 Apr 2025

https://github.com/clickhouse/jsonbench

JSONBench: a Benchmark For Data Analytics On JSON

analytics benchmark clickhouse database duckdb elasticsearch json mongodb postgresql sql

Last synced: 06 Apr 2025

https://github.com/tobilg/duckerd

CLI to create an ER Diagram from DuckDB database files

cli database-diagram duckdb erd erdiagram

Last synced: 06 Apr 2025

https://github.com/BauplanLabs/quack-reduce

A playground for running duckdb as a stateless query engine over a data lake

duckdb lambda serverless sql streamlit-dashboard

Last synced: 24 Mar 2025

https://github.com/rayokota/kwack

In-Memory Analytics for Kafka using DuckDB

analytics duckdb kafka

Last synced: 10 Apr 2025

https://github.com/maxcountryman/warc-parquet

πŸ—„οΈ A simple CLI for converting WARC to Parquet.

crawling duckdb parquet warc web-archiving

Last synced: 16 May 2025

https://github.com/evanwashere/duckdb

πŸ¦† fast duckdb bindings for bun runtime

bun duckdb

Last synced: 12 Apr 2025

https://github.com/igor-suhorukov/openstreetmap_h3

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps

apach-sedona apache-arrow apache-spark arrow citusdb column-store converter duckdb geometry-processing geospatial java openstreetmap parquet parquet-files pbf pbf-format postgis postgresql world

Last synced: 07 Apr 2025

https://github.com/ktanaka101/mcp-server-duckdb

A Model Context Protocol (MCP) server implementation for DuckDB, providing database interaction capabilities

duckdb mcp mcp-server

Last synced: 09 Apr 2025

https://github.com/caioricciuti/duck-ui

Duck-UI is a web-based interface for interacting with DuckDB, a high-performance analytical database system. It features a SQL editor, data import/export, data explorer, query history, theme toggle, and keyboard shortcuts, all running seamlessly in the browser using DuckDB's WebAssembly (WASM) capabilities.

data-science data-visualization dataanalysis datanalytics duckdb local

Last synced: 04 Apr 2025

https://github.com/ClickHouse/JSONBench

JSONBench: a Benchmark For Data Analytics On JSON

analytics benchmark clickhouse database duckdb elasticsearch json mongodb postgresql sql

Last synced: 07 Feb 2025

https://github.com/rupurt/odbc-scanner-duckdb-extension

A DuckDB extension to read data directly from databases supporting the ODBC interface

analytics bigquery columnar-database cpp data-engineering db2 duckdb mariadb mssql mysql nix odbc olap oracle postgres snowflake vector-engine

Last synced: 15 Feb 2025

https://github.com/elsbrock/hetzner-radar

πŸ•΅οΈ Track prices of the Hetzner dedicated server auction

cloudflare cloudflare-d1 cloudflare-workers duckdb hetzner sqlite sveltekit tailwind wasm

Last synced: 11 Apr 2025

https://github.com/josephmachado/python_essentials_for_data_engineers

Code for blog at https://www.startdataengineering.com/post/python-for-de/

data-engineering data-quality-checks duckdb polars python transformations

Last synced: 15 Apr 2025

https://github.com/marcua/ayb

ayb makes it easy to create databases, share them with collaborators, and query them from a web application or the command line

duckdb sqlite

Last synced: 04 Apr 2025

https://github.com/rustyconover/duckdb-shellfs-extension

DuckDB extension allowing shell commands to be used for input and output.

duckdb duckdb-extension popen shell

Last synced: 12 May 2025

https://github.com/techascent/tmducken

tech.ml.dataset integration with duckdb

clojure dataanalytics duckdb

Last synced: 16 Mar 2025

https://github.com/quackscience/quackflight

DuckDB API Server with Arrow Flight SQL Airport support and concurrent writes/reads (quackpipe)

airport api arrow clickhouse duckdb duckdb-server flight flight-sql fuck-trump server

Last synced: 15 Apr 2025

https://github.com/tjanif/airflow-elt-blueprint

A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.

airflow astro-cli astro-python-sdk astronomer codespaces duckdb minio streamlit

Last synced: 15 May 2025

https://github.com/gwenwindflower/octocatalog

Nicely modeled data built on the Github Archive.

dbt duckdb github-archive public-datasets

Last synced: 01 May 2025

https://github.com/duckdb/duckdb-java

DuckDB JDBC Driver

database duckdb java jdbc sql

Last synced: 05 Apr 2025

https://github.com/quackscience/duckdb-extension-httpclient

DuckDB HTTP GET/POST Client in a Community Extension

duckdb duckdb-extension extension http-client

Last synced: 07 May 2025

https://github.com/mattf96s/QuackDB

Open-source in-browser DuckDB SQL editor

apache-arrow comlink duckdb duckdb-wasm parquet remix remix-run shadcn sql sst

Last synced: 14 May 2025

https://github.com/datawan-labs/schools

2nd Place Solid Hack 2024. Visualizing School Distribution and Population Density in Indonesia, By mapping where people live alongside school locations, this study shows how spatial data can reveal patterns of educational access across the country.

duckdb map mapbox maplibre population-density solidjs visualization

Last synced: 19 Apr 2025

https://github.com/cldellow/datasette-parquet

Add DuckDB, Parquet, CSV and JSON lines support to Datasette

datasette datasette-plugin duckdb parquet

Last synced: 13 Apr 2025

https://github.com/josephmachado/cost_effective_data_pipelines

Cost Efficient Data Pipelines with DuckDB

duckdb python sql

Last synced: 15 Apr 2025

https://github.com/tsffarias/liftoff_data

Arquitetura CRM de Baixo Custo com Gen AI, projetada para startups que precisam processar e analisar dados de vendas de forma eficiente.

ai airflow api briefer crm dbt docker duckdb etl faker fastapi minio n8n pgduckdb postgresql pydantic python sqlalchemy startup streamlit

Last synced: 02 Feb 2025

https://github.com/squashql/squashql

Official repository of SquashQL, the SQL query engine for multi-dimensional and hierarchical analysis that empowers your SQL database

bigquery clickhouse database duckdb java jdbc query querybuilder snowflake spark sql typescript

Last synced: 14 Dec 2024

https://github.com/starlake-ai/jsqltranspiler

Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB compatible SQL (with deep transformation of functions, data types and format characters) using Java.

abstract-syntax-tree bigquery column databricks duckdb java lineage query redshift resolver rewrite snowflake transpiler

Last synced: 16 May 2025

https://github.com/quackscience/duckdb-extension-openprompt

DuckDB Community Extension to prompt LLMs from SQL

duckdb duckdb-database extension llm llms ollama prompt sql

Last synced: 05 May 2025

https://github.com/dfinke/psduckdb

PSDuckDB is a PowerShell module that provides seamless integration with DuckDB, enabling efficient execution of analytical SQL queries directly from the PowerShell environment.

data-analysis data-science duckdb powershell sql

Last synced: 16 Mar 2025

https://github.com/dfinke/PSDuckDB

PSDuckDB is a PowerShell module that provides seamless integration with DuckDB, enabling efficient execution of analytical SQL queries directly from the PowerShell environment.

data-analysis data-science duckdb powershell sql

Last synced: 16 Dec 2024

https://github.com/quackscience/duckdb-extension-clickhouse-sql

DuckDB Community Extension implementing ClickHouse SQL Dialect macros and Custom functions for DuckDB

chdb chsql clickhouse clickhouse-database clickhouse-server duckdb duckdb-community duckdb-extension duckdb-server quackpipe sql

Last synced: 19 Dec 2024

https://github.com/phenobarbital/asyncdb

Collection of Asynchronous (Asyncio-based) generic Database connectors

aioinflux asyncio asyncio-redis asyncmy asyncpg cassandra duckdb hazelcast mysql rethinkdb

Last synced: 05 Apr 2025

https://github.com/uwdata/mosaic-framework-example

Using Mosaic and DuckDB within Observable Framework

dashboard duckdb mosaic publishing visualization

Last synced: 08 Apr 2025

https://github.com/randomfractals/chicago-crimes

Exploring Chicago crimes dataset with Jupyter notebooks, DuckDB, Malloy and new Panel/PyScript data and dashboard tools.

chicago crimes duckdb julia jupyter-notebooks large-csv malloy malloydata parquet polars pyarrow

Last synced: 22 Mar 2025