https://github.com/dentiny/duckdb-curl-filesystem
Filesystem built upon libcurl.
https://github.com/dentiny/duckdb-curl-filesystem
Last synced: 2 days ago
JSON representation
Filesystem built upon libcurl.
- Host: GitHub
- URL: https://github.com/dentiny/duckdb-curl-filesystem
- Owner: dentiny
- License: mit
- Created: 2025-09-21T22:46:56.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-11-13T11:22:58.000Z (about 1 month ago)
- Last Synced: 2025-11-13T13:20:27.739Z (about 1 month ago)
- Language: C++
- Size: 145 KB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-duckdb - `curl_httpfs` - Enhanced HTTP file system with connection pooling, HTTP/2 support, and asynchronous I/O operations. (Extensions / [Community Extensions](https://duckdb.org/community_extensions/))
README
# curl_httpfs - DuckDB extension for connection pool, HTTP/2 and asynchronous network IO
## What is curl_httpfs?
It's a DuckDB extension that provides additional IO features upon [httpfs extension](https://duckdb.org/docs/stable/core_extensions/httpfs/overview.html), including connection pool, HTTP/2, and async network IO.
It's 100% compatible with `httpfs` extension, all advanced features are implemented based on uses curl-based solution, meanwhile it also allows users to fallback to httplib.
## Usage
```sql
-- Install and load the curl_httpfs extension
FORCE INSTALL curl_httpfs FROM community;
LOAD curl_httpfs;
-- Users could access file as usual.
D SELECT length(content) AS char_count FROM read_text('https://raw.githubusercontent.com/dentiny/duck-read-cache-fs/main/test/data/stock-exchanges.csv');
┌────────────┐
│ char_count │
│ int64 │
├────────────┤
│ 16205 │
└────────────┘
-- Switch back to httplib.
D SET httpfs_client_implementation='httplib';
-- Switch back to curl.
D SET httpfs_client_implementation='curl';
```