{"id":26091481,"url":"https://github.com/quackscience/quackflight","last_synced_at":"2025-09-06T17:40:24.234Z","repository":{"id":258447300,"uuid":"875015917","full_name":"quackscience/quackflight","owner":"quackscience","description":"DuckDB API Server with Arrow Flight SQL Airport support and concurrent writes/reads (quackpipe)","archived":false,"fork":false,"pushed_at":"2025-03-05T09:32:05.000Z","size":198,"stargazers_count":66,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T18:09:35.896Z","etag":null,"topics":["airport","api","arrow","clickhouse","duckdb","duckdb-server","flight","flight-sql","fuck-trump","server"],"latest_commit_sha":null,"homepage":"https://quackpy.fly.dev","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quackscience.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-18T22:51:26.000Z","updated_at":"2025-03-26T21:51:42.000Z","dependencies_parsed_at":"2025-02-28T21:51:16.242Z","dependency_job_id":"b4d01851-38a5-43a6-97ae-b6f84b7fb624","html_url":"https://github.com/quackscience/quackflight","commit_stats":null,"previous_names":["duckheads/quackpy","quackscience/quackpy","quackscience/quackflight"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fquackflight","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fquackflight/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fquackflight/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quackscience%2Fquackflight/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quackscience","download_url":"https://codeload.github.com/quackscience/quackflight/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249032584,"owners_count":21201500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airport","api","arrow","clickhouse","duckdb","duckdb-server","flight","flight-sql","fuck-trump","server"],"created_at":"2025-03-09T10:01:50.287Z","updated_at":"2025-04-15T08:22:58.684Z","avatar_url":"https://github.com/quackscience.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003c!-- \u003cimg src=\"https://github.com/user-attachments/assets/c204da32-71bc-45d7-a83d-e0e99ad169ab\" width=250 /\u003e --\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/48107bf8-c1f9-4a4e-a5fa-20882e2aba9d\" width=250 /\u003e\n\n\n# :baby_chick: [QuackFlight](https://quackpy.fly.dev/?user=default#U0VMRUNUCiAgICB0b3duLAogICAgZGlzdHJpY3QsCiAgICBjb3VudCgpIEFTIGMsCkZST00gcmVhZF9wYXJxdWV0KCdodHRwczovL2RhdGFzZXRzLWRvY3VtZW50YXRpb24uczMuZXUtd2VzdC0zLmFtYXpvbmF3cy5jb20vaG91c2VfcGFycXVldC9ob3VzZV8wLnBhcnF1ZXQnKQpXSEVSRSByZWFkX3BhcnF1ZXQudG93biA9PSAnTE9ORE9OJwpHUk9VUCBCWQogICAgdG93biwKICAgIGRpc3RyaWN0Ck9SREVSIEJZIGMgREVTQwpMSU1JVCAxMA==)\n\n\n_Serverless OLAP API built on top of DuckDB exposing HTTP/S and Arrow Flight SQL interfaces_\n\n\n\u003e [!IMPORTANT]\n\u003e - Arrow Flight API for modern data clients _(DuckDB Airport)_\n\u003e - Easy HTTP API with multiple formats _(JSON,CSV,Parquet)_\n\u003e - Unlocked Concurrent inserts and querying on DuckDB\n\u003e - Persistent storage using w/ multiuser authentication\n\u003e - Native access to any DuckDB Extension \u0026 Format\n\u003e - Embedded SQL Query Interface for instant usage\n\n\n\u003cbr\u003e\n\n![quackflight_banner_yellow](https://github.com/user-attachments/assets/3f55d787-7888-4647-a856-ba78111ed657)\n\n### :seedling: Get Started\nRun using [docker](https://github.com/quackscience/quackflight/pkgs/container/quackflight) or build from source\n```bash\ndocker pull ghcr.io/quackscience/quackflight:latest\ndocker run -ti --rm -p 8123:8123 -p 8815:8815 ghcr.io/quackscience/quackflight:latest\n```\n\n### 👉 Usage\n\n\u003e See the [Examples](https://github.com/quackscience/quackflight/tree/main/examples) directory for quick starters\n\n\u003e [!NOTE]\n\u003e Quackpipe executes queries in `:memory:` unless _authentication_ details are provided for data persistence\n\n\u003cbr\u003e\n\n#### 🕸️  HTTP API\nExecute DuckDB queries using the HTTP POST/GET API _(compatible with the ClickHouse HTTP API)_\n```bash\ncurl -X POST \"http://user:persistence@localhost:8123\" \\\n   -H \"Content-Type: application/json\" \\\n   -d 'SELECT version()'  \n```\n\n\u003cbr\u003e\n\n\u003cimg src=\"https://github.com/user-attachments/assets/deddab90-e409-4b1b-881f-d53f02597a1d\" width=80 /\u003e\n\n#### ✈️ FLIGHT API\nExecute DuckDB queries using the _experimental_ Flight GRPC API and [Airport](https://github.com/Query-farm/duckdb-airport-extension)\n\n\u003e [!NOTE]\n\u003e Quackpipe executes queries in `:memory:` unless an `authorization` header is provided for data persistence\n\n##### 🎫 Pass Airport Security\n```sql\nCREATE PERSISTENT SECRET airport_flight (\n·       type airport,\n‣       auth_token 'user:persistence',\n·       scope 'grpc://localhost:8815'\n· );\n```\n\n##### 🎫 Take Airport Flights\n```sql\nD select flight_descriptor, endpoint from airport_list_flights('grpc://127.0.0.1:8815', null);\n┌─────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐\n│        flight_descriptor        │                                                    endpoint                                                    │\n│ union(cmd blob, path varchar[]) │           struct(ticket blob, \"location\" varchar[], expiration_time timestamp, app_metadata blob)[]            │\n├─────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤\n│ show_databases                  │ [{'ticket': SHOW DATABASES, 'location': [grpc://localhost:8815], 'expiration_time': NULL, 'app_metadata': }]   │\n│ show_tables                     │ [{'ticket': SHOW TABLES, 'location': [grpc://localhost:8815], 'expiration_time': NULL, 'app_metadata': }]      │\n│ show_version                    │ [{'ticket': SELECT version(), 'location': [grpc://localhost:8815], 'expiration_time': NULL, 'app_metadata': }] │\n└─────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘\n\nD select * from airport_take_flight('grpc://localhost:8815/', ['show_version']);\n┌─────────────┐\n│ \"version\"() │\n│   varchar   │\n├─────────────┤\n│ v1.2.0      │\n└─────────────┘\n```\n\n##### 🎫 ATTACH Flights Tables\n```sql\nD --- Attach to Flight Server\nD ATTACH 'deltalake' (TYPE AIRPORT, location 'grpc://localhost:8815/'); \n\nD --- Create Schema + Table\nD CREATE SCHEMA deltalake.test1; \nD CREATE TABLE deltalake.test1.people (\n     name VARCHAR,\n     love_of_duckdb INT,\n     tags VARCHAR[]\n    );\n\nD --- Insert into Flight Table\nD INSERT INTO deltalake.test1.people values\n  ('rusty', 5, ['airport', 'datasketches']);\n\nD --- Select from Flight Table\nD SELECT * FROM deltalake.test1.people;\n┌─────────┬────────────────┬─────────────────────────┐\n│  name   │ love_of_duckdb │          tags           │\n│ varchar │     int32      │        varchar[]        │\n├─────────┼────────────────┼─────────────────────────┤\n│ rusty   │              5 │ [airport, datasketches] │\n├─────────┴────────────────┴─────────────────────────┤\n│ 1 row.                                   3 columns │\n└────────────────────────────────────────────────────┘\n```\n\n\u003e Flight Tables can be accessed via HTTP API using the schema name\n```sql\nUSE test1; SELECT * FROM people;\n```\n![image](https://github.com/user-attachments/assets/82d9c7bf-cbf2-49d3-b4dc-a57a0ddaf46a)\n\n##### 🎫 Take Custom Flights w/ Custom Headers + Ticket\n```sql\nD SELECT * FROM airport_take_flight('grpc://localhost:8815', 'SELECT 1', headers := MAP{'authorization':'user:persistence'} );\n┌───────┐\n│   1   │\n│ int32 │\n├───────┤\n│   1   │\n└───────┘\n```\n\n##### 🎫 Take Python Flights\n```python\nfrom pyarrow.flight import FlightClient, Ticket, FlightCallOptions \nimport json\nimport pandas\nimport tabulate\n\nsql=\"\"\"SELECT version()\"\"\"\n  \nflight_ticket = Ticket(sql)\n\ntoken = (b\"authorization\", bytes(f\"user:persistence\".encode('utf-8')))\noptions = FlightCallOptions(headers=[token])\nclient = FlightClient(f\"grpc://localhost:8815\")\n\nreader = client.do_get(flight_ticket, options)\narrow_table = reader.read_all()\n# Use pyarrow and pandas to view and analyze data\ndata_frame = arrow_table.to_pandas()\nprint(data_frame.to_markdown())\n```\n```sql\n|    | \"version\"()   |\n|---:|:--------------|\n|  0 | v1.2.0        |\n```\n\n\u003cbr\u003e\n\n### 📺 SQL User-Interface\nquackflight ships with the DuckDB SQL quack user-interface based on [duck-ui](https://github.com/caioricciuti/duck-ui)\n\n\u003ca href=\"https://quackpy.fly.dev\"\u003e\n  \u003cimg src=\"https://github.com/user-attachments/assets/902a6336-c4f4-4a4e-85d5-78dd62cb7602\"\u003e\n\u003c/a\u003e\n\n```mermaid\n\nsequenceDiagram\n    participant Client\n    participant QuackFlight\n    participant DuckDB\n\n\n    Client -\u003e\u003e QuackFlight: ListFlights\n    QuackFlight -\u003e\u003e Client: Return Flights Table\n    Client -\u003e\u003e QuackFlight: GetFlightInfo\n    QuackFlight -\u003e\u003e DuckDB: DuckDB Execute\n    DuckDB -\u003e\u003e QuackFlight: Arrow Results Stream\n    QuackFlight -\u003e\u003e Client: FlightInfo(ticket)\n    Client -\u003e\u003e QuackFlight: do_get(ticket)\n    QuackFlight -\u003e\u003e Client: Stream of Results\n\n```\n\n\u003cbr\u003e\n\n\n###### :black_joker: Disclaimers \n\n[^1]: DuckDB ® is a trademark of DuckDB Foundation. All rights reserved by their respective owners. [^1]\n[^2]: ClickHouse ® is a trademark of ClickHouse Inc. No direct affiliation or endorsement. [^2]\n[^3]: Released under the MIT license. See LICENSE for details. All rights reserved by their respective owners. [^3]\n[^4]: Flight implementation inspired by [Duck Takes Flight](https://www.definite.app/blog/duck-takes-flight) [^4]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackscience%2Fquackflight","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquackscience%2Fquackflight","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquackscience%2Fquackflight/lists"}