awesome-duckdb
  
  
    🦆 A curated list of awesome DuckDB resources 
    https://github.com/davidgasquez/awesome-duckdb
  
        Last synced: about 23 hours ago 
        JSON representation
    
- 
            
Resources
- DuckDB setup - GitHub Action to install DuckDB in CI.
 - DuckDB snippets - Collection of snippets curated by MotherDuck.
 - DuckDB tldr page - DuckDB's entry in [tldr pages](https://tldr.sh/), available in CLI via the `tldr duckdb` command.
 - Observable notebooks - Notebooks using DuckDB on the Observable data visualization platform.
 - Official documentation - Official DuckDB documentation.
 - Serverless DuckDB over S3 - Running DuckDB over a data lake on S3 using lambda.
 - docker-duckdb - Docker image for DuckDB CLI.
 - DuckDB AWS Lambda layer - Run DuckDB in AWS Lambda functions.
 - Serverless DuckDB as API - Use DuckDB as API with Amazon API Gateway and AWS Lambda.
 - Serverless Parquet Repartitioner - Use DuckDB to repartition data in S3-based Data Lakes.
 - duckdb-nf - Example uses of DuckDB with Nextflow.
 - DuckLake - A lakehouse format from the team behind DuckDB.
 - DuckERD CLI
 - DuckDB clients - Client APIs for DuckDB.
 - Official blog - Official DuckDB blog.
 - DuckDB RSS feed - Feed for the official DuckDB blog.
 - DuckDB clients - Client APIs for DuckDB.
 - DuckDB documentation PDF - The DuckDB documentation as a single PDF file.
 - Compatible DuckDB extensions for AWS Lambda - Extensions specifically compiled for the AWS Lambda runtime (GLIBC 2.26).
 - duckdb-docker - Official Docker image for the DuckDB CLI.
 - SQLNotebook - SQL notebooks by TimerStored powered by DuckDB.
 - Awesome DuckLake - A curated list of awesome DuckLake tools and resources.
 - DuckDB in Science - A collection of scientific papers building on DuckDB.
 - DuckDB documentation MD - The DuckDB documentation as a single Markdown file.
 - DuckDB execution plan visualizer - Visualizing and understanding DuckDB EXPLAIN plans made easy.
 - DuckDB documentation PDF - The DuckDB documentation as a single PDF file.
 - DuckDB documentation MD - The DuckDB documentation as a single Markdown file.
 
 - 
            
Client APIs
 - 
            
Tools Powered by DuckDB
- 
                    
                    
- MotherDuck - Serverless data warehouse powered by DuckDB.
 - Boiling Data - Serverless data analytics overlay on top of S3 Data Lakes.
 - Hex Dataframe SQL - Hex's Dataframe SQL cells are powered by DuckDB.
 - Mode - Mode uses DuckDB for their in-memory data engine.
 - Honeycomb Maps - A browser-based geospatial analysis tool leveraging DuckDB Wasm.
 - VulcanSQL - DuckDB can be used as a caching layer or a data connector in VulcanSQL, a Data API framework for data folks to create REST APIs by writing SQL templates.
 - Bauplan - A serverless data transformation platform for data lakes.
 - Excalichart.com - A fast, free dashboard for exploring your data.
 - Malloy - Malloy is an experimental language for describing data relationships and transformations. Malloy connects to BigQuery, Snowflake, Trino, and Postgres, and natively supports DuckDB.
 - Shaper - Open Source, SQL-driven Data Dashboards powered by DuckDB.
 - ParadeDB - Postgres for Search and Analytics, powered by DuckDB-embedded-in-Postgres.
 - DatalakeStudio - Load, explore, transform your datasets and expose them via API. Integration with external APIs, S3, PostgreSQL and ChatGPT.
 - Spice.ai - A unified SQL query interface and portable runtime to locally materialize (using an embedded DuckDB), accelerate, and query datasets from any database, data warehouse, or data lake.
 - Quackpipe - Serverless OLAP API/UI built on top of DuckDB with basic ClickHouse API compatibility and MotherDuck support.
 - Census - Census's dataset diffing for incremental syncs is powered by DuckDB.
 - Crunchy Bridge for Analytics - Fully managed DBaaS based in Postgres integrated with DuckDB.
 - UniverSQL - An implementation of Snowflake API, enables running queries on Snowflake tables locally with DuckDB without a running warehouse.
 - Honeycomb Maps - A browser-based geospatial analysis tool leveraging DuckDB-Wasm.
 - Iceburst - The real-time data lake for monitoring & security.
 - SQLGlot - Python transpiler that translates between 24 different SQL dialects including DuckDB.
 - Latitude - Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light.
 - Parquet Explorer - Visual Studio Code extension for exploring Parquet files with SQL, powered by DuckDB.
 - DQOps - Data quality platform for data engineers, data quality teams and data operations.
 - Ibis Project - A DataFrame API for interacting with DuckDB (and other compute engines).
 - yato - The smallest DuckDB SQL orchestrator on Earth.
 - Whereabouts - Fast, accurate, open-source geocoding in Python, using DuckDB.
 - Phoenix Analytics - Plug and play analytics for Phoenix applications, powered by DuckDB.
 - Excalichart.com - A fast, free dashboard for exploring your data.
 - Iceburst - The real-time data lake for monitoring & security.
 - ADPivot - No code tool built on top of DuckDB-Wasm and Pyodide that helps build pivot tables from databases of any size with a few clicks.
 - SQLMesh - A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally.
 - Huey - Blazing-fast & intuitive pivot tables on Parquet, CSV, JSON files and DuckDB tables in the browser based on DuckDB-Wasm. open-source (MIT). Zero install!
 - Kepler.gl - Kepler.gl is a powerful open-source geospatial analysis tool for large-scale data sets, now embeds duckdb wasm to create geospatial layers.
 - DuckDB OPFS Todo List App - A fully-functional todo list application that demonstrates DuckDB WASM OPFS (Origin Private File System) persistence using a pure functional programming approach.
 - Apple Embedding Atlas - A tool that provides interactive visualizations for large embeddings. Uses DuckDB.
 - Definite - Definite pulls all your data into a single place for analytics and dashboards. No engineering or SQL required. Get a managed data warehouse (DuckDB), ELT, data modeling / transformations and BI in a single platform.
 - Amphi ETL - Low-code data pipelines for structured and unstructured data. SQL transformations are powered by DuckDB.
 - Sloggo - Minimal RFC 5424 syslog collector and viewer based on DuckDB. Runs as a single, resource-friendly process.
 - Rill Data - Tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL.
 - Duck-UI - Duck-UI is a web-based interface for interacting with DuckDB with a SQL editor, data import/export, data explorer, query history, theme toggle and keyboard shortcuts.
 - Tailpipe - An open-source SIEM for instant log insights.
 - Latitude - Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light.
 - ETLX - DuckDB-powered ETL tool written in Go, inspired by evidence.dev’s syntax. It uses a structured Markdown config where heading levels define nested blocks, yaml code blocks specify metadata, and sql code blocks handle data interactions. Enables clean, code-light orchestration with minimal setup.
 - Datakit - The privacy-first data analysis toolkit.
 - Hugr - An data mesh platform and high-performance GraphQL backend powered by DuckDB.
 - duckdb.yazi - Preview csv/tsv, json, and Parquet files in the yazi file manager using duckdb. View the raw data, or a "summarized" view with data-types, min, max, avg etc. for all columns.
 - Greybeam - Routes your Snowflake queries to a DuckDB powered warehouse to reduce costs and speed up queries.
 - Boilstream - Manage with SQL, like for creating topics (tables) and derived topics (materialised views) - all landing on object storage in DuckLake as optimised Parquet files.
 - Arc - Time-series data warehouse built on DuckDB.
 
 - 
                    
Web Clients
- QuackDB - Open-source online DuckDB SQL playground and editor.
 - SQL Playground - DuckDB-WASM based SQL playground for running queries on local or remote data, being able to show data as tables or visually as graphs.
 - Sidequery - Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data.
 
 - 
                    
Web Clients (WebAssembly)
- DuckDB UI - A local web-based notebook interface for DuckDB.
 - WhatTheDuck - WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data.
 - Online DuckDB Shell - Online DuckDB shell powered by DuckDB-Wasm.
 - Sekuel Playground - Query your local Parquet, CSV, JSON. Your data will not be sent out of the device you are using.
 - CSVFiddle - Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone.
 - Codapi - Embed executable code snippets directly into your product documentation, online course or blog post.
 - PondPilot - Open-source, 100% client-side data exploration tool that enables users to analyze local and remote data using SQL. Zero-copy direct access to local datasets sets PondPilot apart from similar tools. It runs entirely in the browser—no servers, no cloud uploads, and no setup required.
 
 
 - 
                    
                    
 - 
            
Integrations
- 
                    
Web Clients
- Kestra DuckDB plugin - Run queries with DuckDB to schedule data transformations and process automations, and run event-driven anomaly detection pipelines.
 - data load tool - DuckDB destination - Extract and load data from APIs to DuckDB using dlt.
 - target-duckdb - Load data to DuckDB based on Singer spec.
 - nodbi - NoSQL Database Connector for R, providing a common API across Elasticsearch, CouchDB, MongoDB, SQLite, PostgreSQL, and DuckDB.
 
 - 
                    
                    
- Airbyte DuckDB destination - Load data to DuckDB with Airbyte.
 
 
 - 
                    
 - 
            
SQL Clients and IDE that Support DuckDB
- 
                    
                    
- Harlequin - The DuckDB IDE for your terminal. ([GitHub](https://github.com/tconbeer/harlequin)).
 
 - 
                    
Web Clients
- VSCode SQLTools (Free) - Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support.
 - jOOQ - Type safe querying of DuckDB (and many other RDBMS) from Java. A [transpiler](https://www.jooq.org/translate/) from and to DuckDB is also available.
 - SQL DATA LENS - A lightweight, commercial SQL IDE that supports different DBMS, including DuckDB. The focus on performance and special DBMS features.
 - qStudio - A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results.
 - DuckDB SQL Tools - Free DuckDB SQL Tools for VS Code IDE. [Premium version available](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) with advanced features.
 - DataGrip - Paid SQL IDE by Jetbrains that supports many different database technologies, including DuckDB.
 - manifold-sql (DuckDB for Java) - Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support.
 
 
 - 
                    
                    
 - 
            
Extensions
- 
                    
Web Clients
- `postgres_scanner` - Allows DuckDB to directly read data from a running Postgres instance.
 - `spatial` - Enables geospatial processing.
 - `sqlite_scanner` - To read and write to SQLite databases.
 - ERPL - DuckDB SAP connector using RFC, ODP, or BICS.
 - Official Extensions - Official DuckDB extensions.
 - `fts` - To support full text search.
 - Official Extensions - Official DuckDB extensions.
 - `sqlite` - To read from and write to SQLite databases.
 - `mysql` - To read from and write to MySQL databases.
 - `postgres` - To read from and write to PostgreSQL databases.
 - `vss` - Add support for vector similarity search.
 - Kùzu - Scan DuckDB tables in Kùzu, an embeddable property graph database management system.
 
 - 
                    
                    
- Lance - Integrate Lance (modern columnar data format for ML implemented in Rust) with DuckDB.
 
 
 - 
                    
 - 
            
Media
- 
                    
Talks
- In-Process Analytical Data Management with DuckDB @ PyData Amsterdam - Hannes Mühleisen.
 - DuckDB: Bringing analytical SQL directly to your Python shell @ EuroPython - Pedro Holanda.
 - DuckCon #3 playlist
 - DuckDB: Bringing Analytical SQL Directly To Your Python Shell @ FOSDEM - Pedro Holanda.
 - State Of The Duck @ DuckCon - Hannes Mühleisen & Mark Raasveldt.
 - DuckDB Extensions @ DuckCon - Pedro Holanda & Sam Ansmink.
 - Developing Systems in Academia: The Good, the Bad, and the not-so-Ugly Duckling @ CIDR - Hannes Mühleisen.
 - DuckDB An Embeddable Analytical Database @ FOSDEM - Hannes Mühleisen.
 - DuckDB tutorials playlist by Learn Data with Mark - Mark Needham.
 - DuckDB tutorials playlist by MotherDuck - Mehdi Ouazza (MotherDuck).
 - Nextflow and database uses: powering data engineering, exploring DuckDB, and beyond - Edmund Miller.
 - State of the Duck @ DuckCon #4 - Hannes Mühleisen and Mark Raasveldt.
 - State of the Duck @ DuckCon #3 - Hannes Mühleisen and Mark Raasveldt.
 - State of the Duck @ DuckCon #2 - Hannes Mühleisen & Mark Raasveldt.
 - DuckCon #4 playlist
 - DuckDB: Bringing Analytical SQL Directly To Your Python Shell @ FOSDEM - Pedro Holanda.
 - Why should you care about DuckDB? @ Dublin DuckDB meetup - Mihai Bojin.
 - Exploring Monte Carlo Simulations With DuckDB @ Dublin DuckDB meetup - James McNeill.
 - DuckDB Extensions @ DuckCon - Pedro Holanda & Sam Ansmink.
 - Developing Systems in Academia: The Good, the Bad, and the not-so-Ugly Duckling @ CIDR - Hannes Mühleisen.
 - DuckDB An Embeddable Analytical Database @ FOSDEM - Hannes Mühleisen.
 - DuckDB and recommenders : a lightning fast synergy @ Dublin DuckDB meetup - Khalil Muhammad.
 - DuckDB: The Power of a Data Warehouse in your Python Process @ PyData Yerevan - Gábor Szárnyas.
 - DuckDB keynote @ Data + AI Summit 2023 - Hannes Mühleisen.
 
 - 
                    
Podcasts
- The Personal Data Warehouse - Jordan Tigani.
 - Software Engineering Daily: DuckDB - Hannes Mühleisen.
 - Move Your Database To The Data And Speed Up Your Analytics With DuckDB - Hannes Mühleisen.
 - Developer Voices: Implementing Hardware-Friendly Databases - Hannes Mühleisen.
 - The Geek Narrator: DuckDB Internals - Mark Raasveldt.
 - The Analytics Engineering Podcast: The Personal Data Warehouse - Jordan Tigani.
 - The Analytics Engineering Podcast: The Personal Data Warehouse - Jordan Tigani.
 
 - 
                    
Blog Posts
- Modern Data Stack in a Box - Fast, free, and open-source Modern Data Stack deployed on a laptop using the combination of DuckDB, Meltano, dbt, and Apache Superset.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - How to use DuckDB, Motherduck and Kestra for ETL - How DuckDB can transform data, mask sensitive PII information, detect anomalies in event-driven workflows, and streamline reporting use cases.
 - DuckDB vs. MotherDuck — how do they compare - What are key differences between them, and when to choose each of these options.
 - Building DuckDB Extensions with Zig and Nix - For Nix users and Zig developers familiar with DuckDB looking to extend it’s capabilities with custom extensions.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Putting DuckDB in Postgres to Query Iceberg - How ParadeDB embedded DuckDB in Postgres to achieve fast analytics and Apache Iceberg compatibility from Postgres.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - DuckDB: The Rising Star in the Big Data Landscape
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - How to Make a DuckDB Extension for a Table Function? - How to make a DuckDB extension to fetch data from external sources.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 - Exploring StarCraft 2 data with Airflow, DuckDB and Streamlit - Example project using DuckDB to persist API data, but also explains how to use DuckDB as a versatile data manipulation tool in data wrangling scripts.
 
 
 - 
                    
 - 
            
Installers
- DuckDB version manager (`duckman`) - platform installer and version manager for DuckDB.
 - Snap package - Snap package of DuckDB, e.g., for Ubuntu Linux.
 - Chocolatey packages - Chocolatey package for Windows.
 - Official installation page
 - Brew formula
 
 - 
            
Libraries Powered by DuckDB
- 
                    
Web Clients
- pg_lakehouse - PostgreSQL extension embedding DuckDB-in-Postgres for fast on-disk and remote object storage analytics from Postgres. Built as a Foreign Data Wrapper with full query pushdown to DuckDB.
 - PyGWalker - A Pyhon library that turns your dataframe into an interactive UI for data visualization.
 
 
 - 
            
Projects Powered by DuckDB
- 
                    
Web Clients
- `endoflife.date` database - Daily dumps of endoflife.date data.
 
 
 - 
                    
 - 
            
Chat with this page
 - 
            
Backends
- duckdb-server - A Rust-based server that runs a local DuckDB instance and supports queries over Web Sockets or HTTP/HTTPS, returning data in either Apache Arrow or JSON format.
 - DuckDB API - a TypeScript-based Docker image containing DuckDB, and a Hono framework REST API with JSON or streaming Arrow responses.
 - Mosaic DuckDB Server - A Python-based server that runs a local DuckDB instance and supports queries over Web Sockets or HTTP, returning data in either Apache Arrow or JSON format.
 
 - 
            
Newsletters
- DuckDB Newsletter - Monthly newsletter by MotherDuck.
 - Learn DuckDB by example - Newsletter by Tobias Müller.
 
 - 
            
Logos and Icons
 
            Programming Languages
          
          
        
            Categories
          
          
        
            Sub Categories
          
          
        
            Keywords
          
          
              
                duckdb
                17
              
              
                sql
                10
              
              
                database
                5
              
              
                parquet
                5
              
              
                data-science
                4
              
              
                olap
                4
              
              
                csv
                3
              
              
                analytics
                3
              
              
                data
                3
              
              
                s3
                3
              
              
                excel
                3
              
              
                golang
                2
              
              
                object-storage
                2
              
              
                parquet-viewer
                2
              
              
                data-analysis
                2
              
              
                clickhouse
                2
              
              
                databricks
                2
              
              
                dbt
                2
              
              
                snowflake
                2
              
              
                etl
                2
              
              
                python
                2
              
              
                pivot-tables
                2
              
              
                ffi-bindings
                2
              
              
                ffi
                2
              
              
                api
                1
              
              
                clickhouse-server
                1
              
              
                machine-learning
                1
              
              
                infrastructure
                1
              
              
                duckdb-api
                1
              
              
                duckdb-engine
                1
              
              
                gigapipe
                1
              
              
                lambda
                1
              
              
                lambda-functions
                1
              
              
                qryn
                1
              
              
                rest-api
                1
              
              
                server
                1
              
              
                proxy-server
                1
              
              
                sql-proxy
                1
              
              
                sqlglot
                1
              
              
                bigquery
                1
              
              
                hive
                1
              
              
                mysql
                1
              
              
                optimizer
                1
              
              
                ffi-wrapper
                1
              
              
                performance
                1
              
              
                php
                1
              
              
                php-library
                1
              
              
                php8
                1
              
              
                docker
                1
              
              
                version-manager
                1