https://github.com/isaacbrodsky/duckdb-zipfs
DuckDB extension to read files within zip archives.
https://github.com/isaacbrodsky/duckdb-zipfs
duckdb duckdb-extension zip
Last synced: about 1 year ago
JSON representation
DuckDB extension to read files within zip archives.
- Host: GitHub
- URL: https://github.com/isaacbrodsky/duckdb-zipfs
- Owner: isaacbrodsky
- License: mit
- Created: 2025-01-17T16:33:12.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-06T22:01:07.000Z (about 1 year ago)
- Last Synced: 2025-06-06T23:17:55.186Z (about 1 year ago)
- Topics: duckdb, duckdb-extension, zip
- Language: C++
- Homepage:
- Size: 41 KB
- Stars: 34
- Watchers: 2
- Forks: 4
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/isaacbrodsky/duckdb-zipfs/actions/workflows/MainDistributionPipeline.yml)
[](https://github.com/duckdb/duckdb/releases/tag/v1.3.0)
[](LICENSE)
This is a [DuckDB](https://duckdb.org) extension that adds support for reading files from within [zip archives](https://en.wikipedia.org/wiki/ZIP_(file_format)).
# Get started
Load from the [community extensions repository](https://community-extensions.duckdb.org/extensions/zipfs.html):
```SQL
INSTALL zipfs FROM community;
LOAD zipfs;
```
To read a file:
```SQL
SELECT * FROM 'zip://examples/a.zip/a.csv';
```
To read a file from azure blob storage (or other file system):
```SQL
SELECT * FROM 'zip://az://yourstorageaccount.blob.core.windows.net/yourcontainer/examples/a.zip/a.csv';
```
## File names
File names passed into the `zip://` URL scheme are expected to end with `.zip`, which indicates the end of the zip file name. The path after
that is taken to be the file path within the zip archive.
Globbing within the zip archive is supported, but see below for performance limitations. A glob query looks like:
```SQL
SELECT * FROM 'zip://examples/a.zip/*.csv';
```
Globbing for multiple zip files:
```SQL
SELECT * FROM 'zip://examples/*.zip/*.csv';
```
You may use options to turn this behavior off and instead choose some string to split on:
```SQL
SET zipfs_split = "!!";
SELECT * FROM 'zip://examples/a.zip!!b.csv'
```
## Performance considerations
This extension is intended more for convience than high performance. It does not implement a file metadata cache as `tarfs` (on which this
extension is based) does. As such, operations which require the central directory (index) of the zip file, such as globbing files, must
reread the central directory multiple times, once for the glob and once for each file to open.
# License
duckdb-zipfs Copyright 2025 Isaac Brodsky. Licensed under the [MIT License](./LICENSE).
[DuckDB](https://github.com/duckdb/duckdb) Copyright 2018-2022 Stichting DuckDB Foundation (MIT License)
[miniz](https://github.com/richgel999/miniz)
Copyright 2013-2014 RAD Game Tools and Valve Software
Copyright 2010-2014 Rich Geldreich and Tenacious Software LLC
(MIT License)
[DuckDB extension-template](https://github.com/duckdb/extension-template) Copyright 2018-2022 DuckDB Labs BV (MIT License)
[duckdb_tarfs](https://github.com/Maxxen/duckdb_tarfs) (MIT license)