{"id":49818928,"url":"https://github.com/tableau/hyper-api-rust","last_synced_at":"2026-06-17T01:01:45.456Z","repository":{"id":357502737,"uuid":"1236019051","full_name":"tableau/hyper-api-rust","owner":"tableau","description":"HyperDB API for Rust","archived":false,"fork":false,"pushed_at":"2026-06-09T23:39:52.000Z","size":2114,"stargazers_count":1,"open_issues_count":18,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-06-09T23:45:14.653Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tableau.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-11T21:54:32.000Z","updated_at":"2026-06-09T23:22:25.000Z","dependencies_parsed_at":"2026-05-16T12:01:43.120Z","dependency_job_id":null,"html_url":"https://github.com/tableau/hyper-api-rust","commit_stats":null,"previous_names":["tableau/hyper-api-rust"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/tableau/hyper-api-rust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tableau%2Fhyper-api-rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tableau%2Fhyper-api-rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tableau%2Fhyper-api-rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tableau%2Fhyper-api-rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tableau","download_url":"https://codeload.github.com/tableau/hyper-api-rust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tableau%2Fhyper-api-rust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34428197,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-13T09:09:32.362Z","updated_at":"2026-06-17T01:01:45.366Z","avatar_url":"https://github.com/tableau.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hyper API for Rust\n\n[![CI](https://github.com/tableau/hyper-api-rust/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/tableau/hyper-api-rust/actions/workflows/ci.yml)\n[![crates.io](https://img.shields.io/crates/v/hyperdb-api.svg)](https://crates.io/crates/hyperdb-api)\n[![docs.rs](https://img.shields.io/docsrs/hyperdb-api)](https://docs.rs/hyperdb-api)\n[![Downloads](https://img.shields.io/crates/d/hyperdb-api.svg)](https://crates.io/crates/hyperdb-api)\n[![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue)](#license)\n\nA **pure-Rust** implementation of the Hyper database API, using the PostgreSQL\nwire protocol with Hyper-specific extensions. Create, read, and manipulate Hyper\ndatabase files (`.hyper`) without any C library dependencies.\n\n\u003e **Project Status — 0.4.x, AI-Assisted**\n\u003e\n\u003e This crate is **AI-assisted but human-directed**: much of the code was written\n\u003e by AI coding assistants under close review, with the design, architecture, and\n\u003e engineering trade-offs decided by an experienced developer. The pre-1.0\n\u003e (**0.x**) line will probably undergo more large breaking changes; the public\n\u003e API won't settle until the 1.0.0 release.\n\u003e\n\u003e Contributors and reviewers should, at a minimum, run an **AI code reviewer**\n\u003e over any changes, following the conventions, layering rules, and patterns\n\u003e captured in [AGENTS.md](AGENTS.md) (and the subdirectory\n\u003e [`hyperdb-api-node/AGENTS.md`](hyperdb-api-node/AGENTS.md)). Those files are\n\u003e the authoritative guidance for AI assistants working in this repository.\n\n## Key Features\n\n- **Pure Rust** — no C library dependencies, standard `cargo build`\n- **High Performance** — 22-24M rows/sec inserts, 18M rows/sec queries (100M row benchmark)\n- **Memory Safe** — streaming by default, constant memory for billion-row results\n- **Dual Architecture** — sync (`Connection`) and async (`AsyncConnection`) APIs\n- **Typed Row Mapping** — `#[derive(FromRow)]` structs, including streaming `stream_as` for constant-memory typed queries\n- **Compile-time SQL Validation** — opt-in `query_as!` macro checks SQL against your schema at build time (red squigglies in VS Code)\n- **Connection Pooling** — async pooling via `deadpool` for high-concurrency applications\n- **Arrow Integration** — insert and read data in Arrow IPC stream format\n- **gRPC Transport** — read-only access with Arrow IPC and load balancing support\n- **Full Type Support** — all Hyper types including Numeric, Geography, Intervals\n- **Salesforce Auth** — OAuth 2.0 and JWT Bearer Token flows for Data Cloud\n- **TLS** — via rustls (always-on, pure Rust)\n- **Formal Verification** — Kani proof harnesses for model-checked correctness\n\n## Quick Start\n\n### Build from Source\n\nInstall Rust via [rustup.rs](https://rustup.rs/), install `protoc` for your\nplatform, download the `hyperd` executable with `make download-hyperd`\n(bundled helper — see [hyperdb-bootstrap](hyperdb-bootstrap/README.md)), then\nbuild:\n\n| Platform | Install `protoc` | Build |\n|----------|------------------|-------|\n| **macOS** | `brew install protobuf` | `make build` |\n| **Linux** (Debian/Ubuntu) | `sudo apt-get install -y protobuf-compiler build-essential` | `make build` |\n| **Linux** (Fedora/RHEL) | `sudo dnf install protobuf-compiler` | `make build` |\n| **Windows** | `choco install protoc` (also install [VS Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) with the \"Desktop development with C++\" workload for the MSVC linker) | `.\\build.ps1 build` |\n\n```bash\n# Linux / macOS\nmake download-hyperd  # downloads hyperd into .hyperd/current/ (first time only)\nmake build            # or `make build-release` for optimized builds\nmake test             # runs unit + integration tests, test-release for release build\nmake doc              # builds the Hyper Rust documentation\n\n# Windows (PowerShell)\n.\\build.ps1 download-hyperd\n.\\build.ps1 build     # or `.\\build.ps1 build-release`\n.\\build.ps1 test      # or test-release for release build\n.\\build.ps1 doc       # builds the Hyper Rust documentation\n```\n\nThe `Makefile` and `build.ps1` wrappers auto-discover the downloaded\n`hyperd` at `.hyperd/current/hyperd`, and — if nothing is found on disk\n— auto-run `download-hyperd` the first time you invoke a target that\nactually needs `hyperd` (build, test, examples, doc). So `make test`\nfrom a clean checkout Just Works; subsequent runs are cache hits. If\nyou already have a `hyperd` elsewhere, set `HYPERD_PATH=/path/to/hyperd`\nand the downloader stays inert — nothing is fetched, and no build step\ntouches the network. Plain `cargo build` / `cargo test` also work as\nlong as one of those is true.\n\nSee [DEVELOPMENT.md](DEVELOPMENT.md#building--development) for the full\nbuild guide including WSL, cross-compilation, benchmarks, and\nper-platform troubleshooting.\n\n### Installation\n\nAdd to your `Cargo.toml`:\n\n```toml\n[dependencies]\nhyperdb-api = { path = \"hyperdb-api\" }\n```\n\n#### Installing the CLIs\n\n`hyperdb-mcp` and `hyperdb-bootstrap` ship two ways:\n\n**Via npm** (recommended for `hyperdb-mcp`; bundles a matching `hyperd`):\n\n```bash\nnpm install -g hyperdb-mcp\n```\n\nSupported platforms: macOS ARM64 (Apple Silicon), Linux x64 (glibc),\nWindows x64. Intel macOS is built-from-source only at the moment — see\nthe platform table in [`hyperdb-mcp/README.md`](hyperdb-mcp/README.md#installation).\n\n**Via crates.io** (compiles from source; no bundled `hyperd`):\n\n```bash\ncargo install hyperdb-mcp\ncargo install hyperdb-bootstrap\n```\n\n`hyperdb-bootstrap` will then download a compatible `hyperd` for you:\n\n```bash\nhyperdb-bootstrap download\n```\n\n### Environment Setup\n\nThe `hyperd` executable (Hyper database server) must be available. The\nsimplest path is:\n\n```bash\nmake download-hyperd   # or `.\\build.ps1 download-hyperd` on Windows\n```\n\nThis installs `hyperd` under `.hyperd/current/` in the repo and is\nauto-discovered by the Makefile / `build.ps1`. If you already have a\n`hyperd` elsewhere, export `HYPERD_PATH` instead:\n\n```bash\nexport HYPERD_PATH=/path/to/hyperd\n```\n\n### Sync Example\n\n```rust\nuse hyperdb_api::{\n    Catalog, Connection, CreateMode, HyperProcess, Inserter,\n    Result, SqlType, TableDefinition,\n};\n\nfn main() -\u003e Result\u003c()\u003e {\n    let hyper = HyperProcess::new(None, None)?;\n    let conn = Connection::new(\u0026hyper, \"example.hyper\", CreateMode::CreateIfNotExists)?;\n\n    // Create a table\n    let table_def = TableDefinition::from(\"users\")\n        .add_required_column(\"id\", SqlType::int())\n        .add_required_column(\"name\", SqlType::text());\n    Catalog::new(\u0026conn).create_table(\u0026table_def)?;\n\n    // Insert data (COPY protocol, 22M+ rows/sec)\n    {\n        let mut inserter = Inserter::new(\u0026conn, \u0026table_def)?;\n        inserter.add_row(\u0026[\u00261i32, \u0026\"Alice\"])?;\n        inserter.add_row(\u0026[\u00262i32, \u0026\"Bob\"])?;\n        inserter.execute()?;\n    }\n\n    // Query data\n    let result = conn.execute_query(\"SELECT * FROM users\")?;\n    for row in result.rows() {\n        let row = row?;\n        let id: Option\u003ci32\u003e = row.get(0);\n        let name: Option\u003cString\u003e = row.get(1);\n        println!(\"{:?} - {:?}\", id, name);\n    }\n\n    Ok(())\n}\n```\n\n### Async Example\n\n```rust\nuse hyperdb_api::{AsyncConnection, CreateMode, HyperProcess, Result};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let hyper = HyperProcess::new(None, None)?;\n    let endpoint = hyper.require_endpoint()?;\n\n    let conn = AsyncConnection::connect(\n        endpoint,\n        \"example_async.hyper\",\n        CreateMode::CreateIfNotExists,\n    ).await?;\n\n    conn.execute_command(\"CREATE TABLE users (id INT, name TEXT)\").await?;\n    conn.execute_command(\"INSERT INTO users VALUES (1, 'Alice')\").await?;\n    conn.close().await?;\n    Ok(())\n}\n```\n\n## Crate Overview\n\n| Crate | Purpose | Published |\n|-------|---------|-----------|\n| **[hyperdb-api](hyperdb-api/README.md)** | High-level API — connections, inserters, catalog, Arrow, pooling | crates.io |\n| **[hyperdb-api-core](hyperdb-api-core/README.md)** | Internal implementation details (types, protocol, client). Not a public API — depend on `hyperdb-api` instead. | crates.io |\n| **[hyperdb-api-salesforce](hyperdb-api-salesforce/README.md)** | Salesforce Data Cloud OAuth authentication | crates.io |\n| **[hyperdb-mcp](hyperdb-mcp/README.md)** | MCP server for LLM-driven SQL analytics on `.hyper` files | crates.io |\n| **[sea-query-hyperdb](sea-query-hyperdb/README.md)** | HyperDB dialect backend for sea-query | crates.io |\n| **[hyperdb-api-node](hyperdb-api-node/README.md)** | Node.js/TypeScript bindings via napi-rs | npm |\n| **[hyperdb-bootstrap](hyperdb-bootstrap/README.md)** | Download the `hyperd` executable from Tableau's release packages | crates.io |\n\n## Examples\n\nThe API ships 14 examples in `hyperdb-api/examples/` plus 2 companion crate examples.\n\n### Core Examples\n\n| Example | Description |\n|---------|-------------|\n| `insert_data_into_single_table` | Create a table and insert data using `Inserter` |\n| `insert_data_into_multiple_tables` | Multiple related tables |\n| `create_hyper_file_from_csv` | Load CSV data into a Hyper table |\n| `delete_data_in_existing_hyper_file` | Delete data with SQL `DELETE` |\n| `update_data_in_existing_hyper_file` | Update data with SQL `UPDATE` |\n| `read_and_print_data_from_existing_hyper_file` | Read table definitions and query data |\n| `insert_data_with_expressions` | Column mappings with `MappedInserter` |\n| `insert_geospatial_data_to_a_hyper_file` | Insert geospatial data |\n\n### Rust-Specific Examples\n\n| Example | Description |\n|---------|-------------|\n| `arrow` | Read/write Arrow `RecordBatch` data |\n| `async_usage` | `AsyncConnection` and Tokio patterns |\n| `threaded_inserter` | Multi-threaded bulk insertion with `InsertChunk`/`ChunkSender` |\n| `grpc_query` | gRPC transport, Arrow IPC results |\n| `connection_pool` | Async connection pooling with deadpool |\n| `transactions` | RAII guards, multi-table rollback, DDL, reconnect semantics |\n\n### Running Examples\n\n```bash\nexport HYPERD_PATH=/path/to/hyperd\n\n# Run individual examples\ncargo run -p hyperdb-api --example insert_data_into_single_table\ncargo run -p hyperdb-api --example arrow\ncargo run -p hyperdb-api --example connection_pool\n\n# Companion crate examples\ncargo run -p sea-query-hyperdb --example basic_usage\ncargo run -p hyperdb-api-salesforce --example salesforce_auth_example\n\n# Run all examples\n./run_all_examples.sh\n```\n\n## Companion Crates\n\n### sea-query-hyperdb\n\nHyperDB dialect backend for [sea-query](https://crates.io/crates/sea-query) — use for\nwindow functions, CTEs, complex JOINs, and type-safe query composition:\n\n```toml\n[dependencies]\nsea-query = \"0.32\"\nsea-query-hyperdb = { path = \"sea-query-hyperdb\" }\n```\n\n```rust\nuse sea_query::{Query, Expr, Iden};\nuse sea_query_hyperdb::HyperQueryBuilder;\n\nlet sql = Query::select()\n    .column(Users::Name)\n    .from(Users::Table)\n    .and_where(Expr::col(Users::Age).gt(18))\n    .to_string(HyperQueryBuilder);\n\nlet result = conn.fetch_all(\u0026sql)?;\n```\n\n### hyperdb-api-salesforce\n\nSalesforce Data Cloud OAuth authentication — JWT Bearer Token, Username-Password,\nand Refresh Token flows:\n\n```toml\n[dependencies]\nhyperdb-api-salesforce = { path = \"hyperdb-api-salesforce\" }\n```\n\n```rust\nuse hyperdb_api_salesforce::{SalesforceAuthConfig, AuthMode, SharedTokenProvider};\n\nlet auth_config = SalesforceAuthConfig::new(\n    \"https://login.salesforce.com\",\n    \"your-connected-app-consumer-key\",\n)?\n.auth_mode(AuthMode::private_key(\"user@example.com\", \u0026private_key_pem)?);\n\nlet token_provider = SharedTokenProvider::new(auth_config)?;\n```\n\nSee [hyperdb-api-salesforce/README.md](hyperdb-api-salesforce/README.md) for full setup guide.\n\n## Node.js Bindings\n\nThe `hyperdb-api-node` package provides Node.js and TypeScript bindings built with\n[napi-rs](https://napi.rs/):\n\n```typescript\nconst { HyperProcess, Connection, CreateMode } = require('hyperdb-api-node');\n\nconst hyper = new HyperProcess();\nconst conn = await Connection.connect(hyper.endpoint, 'my.hyper', CreateMode.CreateAndReplace);\n\n// Tagged template literals — SQL injection safe\nconst rows = await conn.sql`SELECT * FROM users WHERE age \u003e ${18}`;\nawait conn.close();\nhyper.close();\n```\n\nSee [hyperdb-api-node/README.md](hyperdb-api-node/README.md) for full documentation.\n\n## Platform Support\n\n| Platform | Status | Build Tool |\n|----------|--------|------------|\n| Linux (x86_64) | Supported | `make build` |\n| macOS (ARM \u0026 x64) | Supported | `make build` |\n| Windows | Supported | `.\\build.ps1 build` |\n| WSL | Supported | `make build` |\n\n**MSRV:** Check `rust-version` in `Cargo.toml`.\n\n## Documentation\n\n| Resource | Description |\n|----------|-------------|\n| **[hyperdb-api/README.md](hyperdb-api/README.md)** | Full user guide for the `hyperdb-api` crate |\n| **[docs/WHATS_NEW_0.4.md](docs/WHATS_NEW_0.4.md)** | Highlights of the 0.4.0 release |\n| **[docs/ROW_MAPPING.md](docs/ROW_MAPPING.md)** | The five ways to map result rows into Rust values |\n| **[hyperdb-api-derive/README.md](hyperdb-api-derive/README.md)** | `#[derive(FromRow)]`, `#[derive(Table)]`, and compile-time SQL validation |\n| **[DEVELOPMENT.md](DEVELOPMENT.md)** | Architecture, building, testing, benchmarks — for contributors |\n| **[CONTRIBUTING.md](CONTRIBUTING.md)** | How to contribute |\n| **[docs/TRANSACTIONS.md](docs/TRANSACTIONS.md)** | Transaction API design |\n| **[docs/BENCHMARK_GUIDE.md](docs/BENCHMARK_GUIDE.md)** | How to run benchmarks |\n\nPer-crate documentation: each crate has its own `README.md` (see [Crate Overview](#crate-overview)).\n\nGenerate API docs locally:\n\n```bash\nmake doc    # or: cargo doc --no-deps --open\n```\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for the governance model, contribution checklist,\ncommit message format, and pull request process.\n\n## Acknowledgments\n\nThis project includes code adapted from\n[sfackler/rust-postgres](https://github.com/sfackler/rust-postgres) (the\n`postgres-protocol`, `tokio-postgres`, and `postgres-types` crates by Steven\nFackler, MIT or Apache-2.0). See [`NOTICE`](NOTICE) for the full third-party\nattribution list and the upstream license text.\n\n## License\n\nLicensed under either of [MIT](LICENSE-MIT.txt) or [Apache-2.0](LICENSE-APACHE.txt) at your option.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftableau%2Fhyper-api-rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftableau%2Fhyper-api-rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftableau%2Fhyper-api-rust/lists"}