{"id":49748086,"url":"https://github.com/zvndev/powdb","last_synced_at":"2026-06-05T04:00:56.535Z","repository":{"id":351220051,"uuid":"1203449401","full_name":"zvndev/powdb","owner":"zvndev","description":"PowDB — Rust native database with PowQL query language","archived":false,"fork":false,"pushed_at":"2026-05-27T05:03:59.000Z","size":23532,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-27T05:27:51.826Z","etag":null,"topics":["btree","database","embedded-database","nosql","powql","query-language","rust","wal"],"latest_commit_sha":null,"homepage":"https://zvndev.github.io/powdb/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zvndev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-07T03:43:16.000Z","updated_at":"2026-05-27T05:03:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zvndev/powdb","commit_stats":null,"previous_names":["zvndev/powdb"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/zvndev/powdb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zvndev%2Fpowdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zvndev%2Fpowdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zvndev%2Fpowdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zvndev%2Fpowdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zvndev","download_url":"https://codeload.github.com/zvndev/powdb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zvndev%2Fpowdb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33928631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["btree","database","embedded-database","nosql","powql","query-language","rust","wal"],"created_at":"2026-05-10T07:03:14.172Z","updated_at":"2026-06-05T04:00:56.497Z","avatar_url":"https://github.com/zvndev.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PowDB\n\n[![CI](https://github.com/zvndev/powdb/actions/workflows/ci.yml/badge.svg)](https://github.com/zvndev/powdb/actions/workflows/ci.yml)\n[![bench](https://github.com/zvndev/powdb/actions/workflows/bench.yml/badge.svg)](https://github.com/zvndev/powdb/actions/workflows/bench.yml)\n[![crates.io](https://img.shields.io/crates/v/powdb-cli.svg)](https://crates.io/crates/powdb-cli)\n[![docs.rs](https://img.shields.io/docsrs/powdb-query)](https://docs.rs/powdb-query)\n[![MSRV](https://img.shields.io/badge/MSRV-1.93-blue)](https://github.com/zvndev/powdb/blob/main/Cargo.toml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\n**PowDB is a pure-Rust embedded database with a compiled query execution engine that delivers 3-10x SQLite on aggregate and scan workloads.**\n\n- **Performance** -- compiled byte-level predicates, zero-copy mmap scans, and a plan cache with literal substitution. Filter and aggregate paths skip full row decoding.\n- **Platform** -- 100% pure-Rust core, no C dependencies, embeddable and server modes, single `cargo install` on every platform Rust supports.\n- **DX** -- PowQL is the front door: a left-to-right pipeline syntax that reads like an iterator chain.\n\nEvaluating PowDB? Start with the honest comparison: [PowDB vs SQLite -- when to use which](docs/powdb-vs-sqlite.md).\n\n## How it works\n\n**Compiled predicate engine.** Filter expressions on integer columns are compiled into branch-free, byte-level operations that run directly against the encoded row bytes. The executor pattern-matches on `Filter(SeqScan)` plan shapes and dispatches to fast paths that never decode columns they don't need. On scan + filter + aggregate workloads, this is where the 3-10x SQLite wins come from.\n\n**Plan cache + tight planner-executor contract.** The planner is pure (no catalog access) and produces a canonical `PlanNode` tree; the cache hashes the canonical shape with FNV-1a and substitutes literals at lookup time, so repeat queries skip lex/parse/plan entirely. Range scans without a matching index are lowered to `Filter(SeqScan)` at execution time, keeping the planner stateless and the executor's fast paths fireable.\n\n**Zero-copy mmap scans.** Heap files are memory-mapped and scanned via `try_for_each_row_raw`, a zero-syscall, zero-allocation iterator over raw row bytes. Combined with the compiled predicates, a full scan + filter + count never copies a row.\n\n**PowQL -- the front door.** PowQL replaces SQL's inside-out clause structure with a left-to-right pipeline. You name the table, chain operations, and project fields, all in reading order.\n\n| Task | SQL | PowQL |\n|---|---|---|\n| Filter + project | `SELECT name, age FROM User WHERE age \u003e 25` | `User filter .age \u003e 25 { .name, .age }` |\n| Sort + limit | `SELECT * FROM User ORDER BY age DESC LIMIT 10` | `User order .age desc limit 10` |\n| Aggregate with filter | `SELECT AVG(age) FROM User WHERE city = 'NYC'` | `avg(User filter .city = \"NYC\" { .age })` |\n| Group + having | `SELECT status, COUNT(*) FROM User GROUP BY status HAVING COUNT(*) \u003e 5` | `User group .status having count(*) \u003e 5 { .status, count(*) }` |\n\nPowQL uses `.field` dot syntax for column references, `:=` for assignments, and `\"double quotes\"` for strings. The pipeline reads like a sentence: *\"User, filter age greater than 25, order by name, limit 10, give me name and age.\"*\n\nFull language reference: [docs/POWQL.md](https://github.com/zvndev/powdb/blob/main/docs/POWQL.md) | Getting started: [docs/getting-started.md](https://github.com/zvndev/powdb/blob/main/docs/getting-started.md)\n\n## Install\n\n```bash\n# From crates.io (Rust 1.93+)\ncargo install powdb-cli\ncargo install powdb-server\n\n# TypeScript client (Node 18+)\nnpm install @zvndev/powdb-client\n\n# Prebuilt binaries (linux x86_64, macos aarch64)\n# https://github.com/zvndev/powdb/releases/latest\n\n# Docker\ndocker pull ghcr.io/zvndev/powdb:latest\n\n# Or build from source\ngit clone https://github.com/zvndev/powdb\ncd powdb\ncargo build --release\n```\n\nRequires Rust 1.93+. This builds all crates: the storage engine, query engine, TCP server, CLI, and benchmarks. TLS support in `powdb-server` pulls `aws-lc-sys`, which requires a C toolchain (`cmake`); disable the default `tls` feature for a fully-Rust build.\n\n## Benchmark: PowDB vs SQLite (100K rows, M1)\n\nPowDB's compiled predicate engine excels at read-heavy aggregate and scan workloads. Write performance is an active area of improvement.\n\n| Workload | PowDB | SQLite | Result |\n|---|---|---|---|\n| Aggregate MIN | 236 us | 2,340 us | **9.9x faster** |\n| Aggregate MAX | 236 us | 2,100 us | **8.9x faster** |\n| Aggregate SUM | 231 us | 1,870 us | **8.1x faster** |\n| Update by primary key | 55 ns | 412 ns | **7.5x faster** |\n| Aggregate AVG | 401 us | 2,300 us | **5.7x faster** |\n| Scan + filter + count | 381 us | 1,950 us | **5.1x faster** |\n| Scan + filter + sort + limit 10 | 2.66 ms | 9.77 ms | **3.7x faster** |\n| Update by filter (10K rows) | 2.16 ms | 6.77 ms | **3.1x faster** |\n| Indexed point lookup | 93 ns | 282 ns | **3.0x faster** |\n| Multi-column AND filter | 2.22 ms | 4.70 ms | **2.1x faster** |\n| Insert batch (1K rows) | 238 ns | 320 ns | **1.3x faster** |\n| Delete by filter (10K rows) | 1.76 ms | 2.35 ms | **1.3x faster** |\n| Scan + filter + project top 100 | 9.6 us | 12.7 us | **1.3x faster** |\n| Non-indexed point lookup | 350 us | 432 us | **1.2x faster** |\n\nPowDB is fastest where it matters most: the compiled predicate engine avoids full row decoding during scans and aggregates, delivering 3-10x gains on analytical queries. Point lookups benefit from a minimal parse-plan-execute pipeline. Write performance is competitive with SQLite across the board.\n\nBoth engines use in-memory mode (PowDB: `WalSyncMode::Off`, SQLite: `:memory:`). Full results in `crates/compare/`.\n\n## PowQL\n\nPowQL reads left to right. You name the table, apply operations, and project fields -- all in one pipeline.\n\n```\n-- Define a schema\ntype User {\n  required name: str,\n  required email: str,\n  age: int\n}\n\n-- Insert\ninsert User { name := \"Alice\", email := \"alice@example.com\", age := 30 }\n\n-- Query pipeline: source -\u003e filter -\u003e order -\u003e limit -\u003e projection\nUser filter .age \u003e 25 order .age desc limit 10 { .name, .age }\n\n-- Aggregates\ncount(User filter .age \u003e 25)\nsum(User { .age })\navg(User filter .city = \"NYC\" { .age })\n\n-- Joins\nUser as u inner join Team as t on u.team_id = t.id { u.name, team_name: t.name }\n\n-- GROUP BY + HAVING\nUser group .city having avg(.age) \u003e 30 { .city, avg_age: avg(.age) }\n\n-- Subqueries\nUser filter .id in (Order filter .total \u003e 100 { .user_id })\n\n-- Set operations\n(User filter .age \u003e 30) union (User filter .city = \"NYC\")\n\n-- Mutations\nUser filter .age \u003c 18 delete\nUser filter .id = 1 update { age := 31 }\n\n-- DDL\nalter User add column score: int\nalter User drop column score\nalter User add index .email\ndrop User\n```\n\n## Run\n\n### Embedded (CLI / REPL)\n\n```bash\npowdb-cli\n# or from source:\ncargo run --release -p powdb-cli\n```\n\nOpens an interactive REPL with tab completion, command history, and meta-commands (`.tables`, `.schema`, `.timing`, `.help`). Data is stored in `./powdb_data/` by default.\n\n### Server mode\n\n```bash\npowdb-server --port 5433 --data-dir ./powdb_data\n# or from source:\ncargo run --release -p powdb-server -- --port 5433 --data-dir ./powdb_data\n```\n\nListens on TCP with a binary wire protocol. Connect via the CLI:\n\n```bash\npowdb-cli --remote localhost:5433\n```\n\nOr the TypeScript client:\n\n```typescript\nimport { Client } from \"@zvndev/powdb-client\";\n\nconst client = await Client.connect({ host: \"localhost\", port: 5433 });\nconst result = await client.query(\"User filter .age \u003e 25 { .name, .age }\");\nif (result.kind === \"rows\") console.table(result.rows);\n```\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `POWDB_PORT` | `5433` | TCP port for the server |\n| `POWDB_DATA` | `./powdb_data` | Data directory (heap files, WAL, catalog, indexes) |\n| `POWDB_PASSWORD` | *(none)* | Require this password on connect (set as env var) |\n| `POWDB_REQUIRE_TLS` | *(off)* | When set (`1`/`true`), refuse to start if a password is configured without TLS |\n| `POWDB_QUERY_MEMORY_LIMIT` | `268435456` | Per-query memory budget in bytes (256 MiB); over-budget queries error instead of OOM-killing the server |\n| `RUST_LOG` | `info` | Log level (`debug`, `trace` for per-query timings) |\n\n### Production checklist\n\nBefore exposing `powdb-server` beyond `127.0.0.1`:\n\n- [ ] Set `POWDB_PASSWORD` to a strong secret. The server logs a `WARN` on startup when unset and will accept any connection.\n- [ ] Enable TLS via `POWDB_TLS_CERT` and `POWDB_TLS_KEY` (or run behind a TLS-terminating proxy). Set `POWDB_REQUIRE_TLS=1` to make the server refuse to start with a password but no TLS, so credentials can never transit in cleartext by misconfiguration.\n- [ ] Bind to a specific interface with `--bind` rather than `0.0.0.0` if you can.\n- [ ] Mount `POWDB_DATA` on a persistent, durable volume. WAL replay assumes the directory is not wiped between restarts.\n- [ ] Pin the version (`cargo install powdb-server --version 0.4.2 --locked` or the matching ghcr tag). PowDB is pre-1.0; minor bumps may change on-disk formats.\n- [ ] Size `POWDB_QUERY_MEMORY_LIMIT` for your host's RAM: it bounds a **single** query's materialization, not aggregate concurrent usage, so the 256 MiB default times many simultaneous connections can still exceed the process ceiling and get OOM-killed on memory-capped hosts (Railway/Fly/small AWS). Lower it accordingly.\n\nFor a self-hostable starting point, see [`examples/deploy/fly.toml`](https://github.com/zvndev/powdb/blob/main/examples/deploy/fly.toml).\n\n## Features\n\n**Storage engine**\n- Slotted-page heap with 4KB pages\n- B+tree indexes with crash-safe persistence (BIDX binary format)\n- Write-ahead log with statement-boundary group commit\n- Crash recovery (WAL replay + page-zero recovery + index rebuild)\n- Memory-mapped reads (zero-syscall scan path)\n- Compiled integer predicates (branch-free filter at the byte level)\n- Thread-safe concurrent reads via pread(2)/pwrite(2)\n\n**Query engine**\n- PowQL parser + planner + executor with plan cache (FNV-1a hashing, literal substitution)\n- Joins (nested-loop + hash join for equi-joins)\n- GROUP BY, HAVING, DISTINCT\n- UNION / UNION ALL\n- Subqueries (IN, EXISTS)\n- Expressions in projections and filters (arithmetic, string ops, BETWEEN, LIKE, IN-list)\n- COUNT, SUM, AVG, MIN, MAX, COUNT DISTINCT\n- ORDER BY (multi-column), LIMIT, OFFSET\n- Window functions (ROW_NUMBER, RANK, DENSE_RANK, SUM/AVG/MIN/MAX OVER)\n- CAST, CASE/WHEN, COALESCE (`??`)\n- Scalar functions: UPPER, LOWER, LENGTH, TRIM, SUBSTRING, CONCAT, ABS, ROUND, CEIL, FLOOR, SQRT, POW, NOW, EXTRACT, DATE_ADD, DATE_DIFF\n- Materialized views with automatic dirty tracking\n- UPSERT with ON CONFLICT\n- Prepared queries with literal substitution\n- EXPLAIN for query plan inspection\n\n**DDL**\n- `type` (create table), `drop` (drop table)\n- `alter \u003cT\u003e add column`, `alter \u003cT\u003e drop column` (with full heap rewrite)\n- `alter \u003cT\u003e add index` (B+tree, persisted)\n\n**Server**\n- Tokio async TCP with `Arc\u003cRwLock\u003cEngine\u003e\u003e` for parallel readers\n- Binary wire protocol (length-prefixed framing)\n- TLS support for encrypted connections\n- Password authentication via `POWDB_PASSWORD` env var\n\n**Pure Rust core**\n- No SQL parsing layer, no `libsqlite3-sys`, no bindgen\n- Storage, query, and CLI are 100% Rust\n- TLS (`powdb-server` only) pulls `aws-lc-sys`; disable the `tls` feature for a C-free build\n- Single `cargo install` on any platform Rust supports\n\n## Architecture\n\n```\ncrates/\n  storage/   Heap files, B+tree, WAL, catalog, page cache, row encoding\n  query/     Lexer, parser, planner, executor (Engine), plan cache\n  server/    Tokio TCP server + binary wire protocol\n  cli/       Interactive REPL (embedded + remote modes)\n  bench/     Criterion benchmarks + regression gate\n  compare/   PowDB vs SQLite wide-bench harness\n```\n\nThe engine is `powdb_query::executor::Engine`. It owns a `Catalog` (which owns `Table`s, each backed by a `HeapFile` + optional `BTree` indexes) and a `Wal`. The server wraps it in `Arc\u003cRwLock\u003cEngine\u003e\u003e` for concurrent access.\n\n## Benchmarks\n\nPowDB has a CI-enforced regression gate that blocks PRs to `main` if any workload regresses beyond its threshold. Run locally:\n\n```bash\ncargo bench -p powdb-bench              # criterion suite (~60s)\ncargo run --release -p powdb-bench --bin compare   # regression gate\n```\n\nRun the PowDB vs SQLite comparison bench:\n\n```bash\ncargo run --release -p powdb-compare    # prints table + writes results.csv\n```\n\n## Tests\n\n```bash\ncargo test --workspace\n```\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzvndev%2Fpowdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzvndev%2Fpowdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzvndev%2Fpowdb/lists"}