{"id":31392371,"url":"https://github.com/Query-farm/httpserver","last_synced_at":"2025-09-29T04:02:38.823Z","repository":{"id":258236371,"uuid":"869750358","full_name":"Query-farm/httpserver","owner":"Query-farm","description":"DuckDB HTTP API Server and Query Interface in a  Community Extension","archived":false,"fork":false,"pushed_at":"2025-09-23T19:24:38.000Z","size":168,"stargazers_count":231,"open_issues_count":10,"forks_count":10,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-23T21:21:06.157Z","etag":null,"topics":["api","api-server","community-extension","duckdb","extension","http-server","olap","user-interface"],"latest_commit_sha":null,"homepage":"https://duckdb.org/community_extensions/extensions/httpserver.html","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Query-farm.png","metadata":{"files":{"readme":"docs/README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["lmangani"]}},"created_at":"2024-10-08T20:25:46.000Z","updated_at":"2025-09-23T19:24:42.000Z","dependencies_parsed_at":"2024-11-14T20:23:57.152Z","dependency_job_id":"cac80fbb-d2c6-47de-9eec-cc7c562415ef","html_url":"https://github.com/Query-farm/httpserver","commit_stats":null,"previous_names":["lmangani/duckdb-extension-httpserver","quackscience/duckdb-extension-httpserver","query-farm/duckdb-extension-httpserver","query-farm/httpserver"],"tags_count":8,"template":false,"template_full_name":"duckdb/extension-template","purl":"pkg:github/Query-farm/httpserver","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Query-farm%2Fhttpserver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Query-farm%2Fhttpserver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Query-farm%2Fhttpserver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Query-farm%2Fhttpserver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Query-farm","download_url":"https://codeload.github.com/Query-farm/httpserver/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Query-farm%2Fhttpserver/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277462598,"owners_count":25822036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-29T02:00:09.175Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","api-server","community-extension","duckdb","extension","http-server","olap","user-interface"],"created_at":"2025-09-29T04:01:20.650Z","updated_at":"2025-09-29T04:02:38.810Z","avatar_url":"https://github.com/Query-farm.png","language":"HTML","readme":"\u003cimg width=\"200\" height=\"200\" alt=\"image\" src=\"https://github.com/user-attachments/assets/a0a844c8-f8b2-40ae-8d6a-78722574cacf\" /\u003e\n\n\n# DuckDB HTTP Server Extension\nThis extension transforms **DuckDB** instances into tiny multi-player **HTTP OLAP API** services.\u003cbr\u003e\nSupports Authentication _(Basic Auth or X-Token)_ and includes the _play_ SQL user interface.\n\n\u003e This DuckDB extension was created by [Query.Farm](https://query.farm/), where we develop and maintain many extensions that expand DuckDB’s capabilities by connecting it to new data sources, formats, and features. You can explore our full collection at [Query.Farm](https://query.farm/)\n\n### Features\n\n- Turn any [DuckDB](https://duckdb.org) instance into an **HTTP OLAP API** Server\n- Use the embedded **Play User Interface** to query and visualize data \n- Pair with [chsql](https://community-extensions.duckdb.org/extensions/chsql.html) extension for **ClickHouse flavoured SQL**\n- Work with local and remote datasets including [MotherDuck](https://motherduck.com) 🐤\n- _100% Opensource, ready to use and extend by the Community!_\n\n\u003cbr\u003e\n\n![image](https://github.com/user-attachments/assets/e930a8d2-b3e4-454e-ba12-e5e91b30bfbe)\n\n#### Extension Functions\n- `httpserve_start(host, port, auth)`: starts the server using provided parameters\n- `httpserve_stop()`: stops the server thread\n\n#### Notes\n\n🛑 Run DuckDB in `-readonly` mode for enhanced security\n\n\u003cbr\u003e\n\n### 📦 [Installation](https://community-extensions.duckdb.org/extensions/httpserver.html)\n```sql\nINSTALL httpserver FROM community;\nLOAD httpserver;\n```\n\n### 🔌 Usage\nStart the HTTP server providing the `host`, `port` and `auth` parameters.\u003cbr\u003e\n\u003e * If you want no authentication, just pass an empty string as parameter.\u003cbr\u003e\n\u003e * If you want the API run in foreground set `DUCKDB_HTTPSERVER_FOREGROUND=1`\n\u003e * If you want logs set `DUCKDB_HTTPSERVER_DEBUG` or `DUCKDB_HTTPSERVER_SYSLOG`\n\n#### Basic Auth\n```sql\nD SELECT httpserve_start('localhost', 9999, 'user:pass');\n\n┌───────────────────────────────────────────────┐\n│ httpserve_start('0.0.0.0', 9999, 'user:pass') │\n│                    varchar                    │\n├───────────────────────────────────────────────┤\n│ HTTP server started on 0.0.0.0:9999           │\n└───────────────────────────────────────────────┘\n```\n```bash\ncurl -X POST -d \"SELECT 'hello', version()\" \"http://user:pass@localhost:9999/\"\n```\n\n#### Token Auth\n```sql\nSELECT httpserve_start('localhost', 9999, 'supersecretkey');\n\n┌───────────────────────────────────────────────┐\n│ httpserve_start('0.0.0.0', 9999, 'secretkey') │\n│                    varchar                    │\n├───────────────────────────────────────────────┤\n│ HTTP server started on 0.0.0.0:9999           │\n└───────────────────────────────────────────────┘\n```\n\nQuery your endpoint using the `X-API-Key` token:\n\n```bash\ncurl -X POST --header \"X-API-Key: secretkey\" -d \"SELECT 'hello', version()\" \"http://localhost:9999/\"\n```\n\nYou can perform the same action from DuckDB using HTTP `extra_http_headers`:\n\n```sql\nD CREATE SECRET extra_http_headers (\n      TYPE HTTP,\n      EXTRA_HTTP_HEADERS MAP{\n          'X-API-Key': 'secret'\n      }\n  );\n\nD SELECT * FROM duck_flock('SELECT version()', ['http://localhost:9999']);\n┌─────────────┐\n│ \"version\"() │\n│   varchar   │\n├─────────────┤\n│ v1.1.3      │\n└─────────────┘\n```\n\n\n\n#### 👉 QUERY UI\nBrowse to your endpoint and use the built-in quackplay interface _(experimental)_\n\n![image](https://github.com/user-attachments/assets/0ee751d0-7360-4d3d-949d-3fb930634ebd)\n\n#### 👉 QUERY API\nQuery your API endpoint using curl GET/POST requests\n\n```bash\ncurl -X POST -d \"SELECT 'hello', version()\" \"http://localhost:9999/?default_format=JSONCompact\"\n```\n```json\n{\n  \"meta\": [\n    {\n      \"name\": \"'hello'\",\n      \"type\": \"String\"\n    },\n    {\n      \"name\": \"\\\"version\\\"()\",\n      \"type\": \"String\"\n    }\n  ],\n  \"data\": [\n    [\n      \"hello\",\n      \"v1.1.3\"\n    ]\n  ],\n  \"rows\": 1,\n  \"statistics\": {\n    \"elapsed\": 0.01,\n    \"rows_read\": 1,\n    \"bytes_read\": 0\n  }\n}\n```\n\n#### 👉 CROSS-OVER EXAMPLES\n\nYou can now have DuckDB instances query each other and... _themselves!_\n\n```sql\nD LOAD json;\nD LOAD httpfs;\nD SELECT httpserve_start('0.0.0.0', 9999, '');\n┌─────────────────────────────────────┐\n│  httpserve_start('0.0.0.0', 9999)   │\n│               varchar               │\n├─────────────────────────────────────┤\n│ HTTP server started on 0.0.0.0:9999 │\n└─────────────────────────────────────┘\nD SELECT * FROM read_json_auto('http://localhost:9999/?q=SELECT version()');\n┌─────────────┐\n│ \"version\"() │\n│   varchar   │\n├─────────────┤\n│ v1.1.3      │\n└─────────────┘\n```\n\n#### Flock Macro by @carlopi\nCheck out this flocking macro from fellow _Italo-Amsterdammer_ @carlopi @ DuckDB Labs\n\n![image](https://github.com/user-attachments/assets/b409ec0e-86e0-4a8d-822c-377ddbae524d)\n\n* a DuckDB CLI, running httpserver extension\n* a DuckDB from Python, running httpserver extension\n* a DuckDB from the Web, querying all 3 DuckDB at the same time\n\n\u003cbr\u003e\n\n\u003chr\u003e\n\n\u003cbr\u003e\n\n### API Documentation\n\n#### Endpoints Overview\n\n| Endpoint | Methods | Description |\n|----------|---------|-------------|\n| `/`      | GET, POST | Query API endpoint |\n| `/ping`  | GET       | Health check endpoint |\n\n#### Detailed Endpoint Specifications\n\n##### Query API\n\n**Methods:** `GET`, `POST`\n\n**Parameters:**\n\n| Parameter | Description | Supported Values |\n|-----------|-------------|-------------------|\n| `default_format` | Specifies the output format | `JSONEachRow`, `JSONCompact` |\n| `query` | The DuckDB SQL query to execute | Any valid DuckDB SQL query |\n\n##### Notes\n\n- Ensure that your queries are properly formatted and escaped when sending them as part of the request.\n- The root endpoint (`/`) supports both GET and POST methods, but POST is recommended for complex queries or when the query length exceeds URL length limitations.\n- Always specify the `default_format` parameter to ensure consistent output formatting.\n\n\u003cbr\u003e\n\n## Development\n\n### Cloning the Repository\n\nClone the repository and all its submodules\n\n```bash\ngit clone \u003cyour-fork-url\u003e\ngit submodule update --init --recursive\n```\n\n### Setting up CLion\n**Opening project:**\nConfiguring CLion with the extension template requires a little work. Firstly, make sure that the DuckDB submodule is available.\nThen make sure to open `./duckdb/CMakeLists.txt` (so not the top level `CMakeLists.txt` file from this repo) as a project in CLion.\nNow to fix your project path go to `tools-\u003eCMake-\u003eChange Project Root`([docs](https://www.jetbrains.com/help/clion/change-project-root-directory.html)) to set the project root to the root dir of this repo.\n\n**Debugging:**\nTo set up debugging in CLion, there are two simple steps required. Firstly, in `CLion -\u003e Settings / Preferences -\u003e Build, Execution, Deploy -\u003e CMake` you will need to add the desired builds (e.g. Debug, Release, RelDebug, etc). There's different ways to configure this, but the easiest is to leave all empty, except the `build path`, which needs to be set to `../build/{build type}`. Now on a clean repository you will first need to run `make {build type}` to initialize the CMake build directory. After running make, you will be able to (re)build from CLion by using the build target we just created. If you use the CLion editor, you can create a CLion CMake profiles matching the CMake variables that are described in the makefile, and then you don't need to invoke the Makefile.\n\nThe second step is to configure the unittest runner as a run/debug configuration. To do this, go to `Run -\u003e Edit Configurations` and click `+ -\u003e Cmake Application`. The target and executable should be `unittest`. This will run all the DuckDB tests. To specify only running the extension specific tests, add `--test-dir ../../.. [sql]` to the `Program Arguments`. Note that it is recommended to use the `unittest` executable for testing/development within CLion. The actual DuckDB CLI currently does not reliably work as a run target in CLion.\n\n\n### Testing\n\nTo run the E2E test install all packages necessary:\n\n```bash\npip install -r requirements.txt\n```\n\nThen run the test suite:\n\n```bash\npytest pytest test_http_api\n```\n\n##### :black_joker: Disclaimers \n\n[^1]: DuckDB ® is a trademark of DuckDB Foundation. All rights reserved by their respective owners. [^1]\n[^2]: ClickHouse ® is a trademark of ClickHouse Inc. No direct affiliation or endorsement. [^2]\n[^3]: Released under the MIT license. See LICENSE for details. All rights reserved by their respective owners. [^3]\n","funding_links":["https://github.com/sponsors/lmangani"],"categories":["HTML"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQuery-farm%2Fhttpserver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FQuery-farm%2Fhttpserver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQuery-farm%2Fhttpserver/lists"}