{"id":47915503,"url":"https://github.com/vmvarela/sql-pipe","last_synced_at":"2026-05-01T11:01:09.581Z","repository":{"id":341509619,"uuid":"1170380112","full_name":"vmvarela/sql-pipe","owner":"vmvarela","description":"A blazing-fast, zero-dependency CLI that pipes CSV data from stdin into an in-memory SQLite database for instant SQL querying. Built with Zig.","archived":false,"fork":false,"pushed_at":"2026-05-01T09:25:01.000Z","size":241,"stargazers_count":3,"open_issues_count":14,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-05-01T10:25:52.751Z","etag":null,"topics":["csv","sql","sqlite","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmvarela.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-02T03:54:15.000Z","updated_at":"2026-05-01T09:16:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vmvarela/sql-pipe","commit_stats":null,"previous_names":["vmvarela/sql-pipe"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/vmvarela/sql-pipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmvarela%2Fsql-pipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmvarela%2Fsql-pipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmvarela%2Fsql-pipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmvarela%2Fsql-pipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmvarela","download_url":"https://codeload.github.com/vmvarela/sql-pipe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmvarela%2Fsql-pipe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32494275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","sql","sqlite","zig"],"created_at":"2026-04-04T05:33:56.280Z","updated_at":"2026-05-01T11:01:09.567Z","avatar_url":"https://github.com/vmvarela.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sql-pipe\n\n[![CI](https://github.com/vmvarela/sql-pipe/actions/workflows/ci.yml/badge.svg)](https://github.com/vmvarela/sql-pipe/actions/workflows/ci.yml)\n[![Release](https://img.shields.io/github/v/release/vmvarela/sql-pipe)](https://github.com/vmvarela/sql-pipe/releases/latest)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n\n`sql-pipe` reads CSV from stdin, loads it into an in-memory SQLite database, runs a SQL query, and prints the results as CSV. No server, no schema files, no setup.\n\nIt exists because `awk` is cryptic, spinning up a Python interpreter for a one-liner feels wrong, and `sqlite3 :memory:` takes four commands before you can query anything. If you know SQL and work with CSV in the terminal, this is the tool you've been reaching for.\n\n```sh\n$ curl -s https://example.com/data.csv | sql-pipe 'SELECT region, SUM(revenue) FROM t GROUP BY region ORDER BY 2 DESC'\n```\n\n## Quick Start\n\n**macOS / Linux via Homebrew:**\n\n```sh\nbrew tap vmvarela/homebrew-tap\nbrew install sql-pipe\n```\n\n**Pre-built binaries** for Linux, macOS (Intel + Apple Silicon), and Windows are also available on the [Releases page](https://github.com/vmvarela/sql-pipe/releases).\n\n**Shell installer (Linux/macOS):**\n\n```sh\ncurl -sSL https://raw.githubusercontent.com/vmvarela/sql-pipe/master/install.sh | sh\n```\n\nBy default it installs to `/usr/local/bin`. Override with `INSTALL_DIR`:\n\n```sh\ncurl -sSL https://raw.githubusercontent.com/vmvarela/sql-pipe/master/install.sh | INSTALL_DIR=\"$HOME/.local/bin\" sh\n```\n\n**Debian / Ubuntu (.deb package):**\n\n```sh\nwget https://github.com/vmvarela/sql-pipe/releases/latest/download/sql-pipe_VERSION_amd64.deb\nsudo dpkg -i sql-pipe_VERSION_amd64.deb\n```\n\nReplace `VERSION` with the release version (e.g. `0.2.0`) and `amd64` with your architecture (`arm64`, `armhf`, or `386`).\n\n**Fedora / RHEL / openSUSE (.rpm package):**\n\n```sh\nsudo rpm -i https://github.com/vmvarela/sql-pipe/releases/latest/download/sql-pipe-VERSION-1.x86_64.rpm\n```\n\nReplace `VERSION` with the release version (e.g. `0.2.0`) and `x86_64` with your architecture (`aarch64`).\n\n**Alpine Linux (.apk package):**\n\n```sh\nwget https://github.com/vmvarela/sql-pipe/releases/latest/download/sql-pipe_VERSION_x86_64.apk\nsudo apk add --allow-untrusted sql-pipe_VERSION_x86_64.apk\n```\n\nReplace `VERSION` with the release version (e.g. `0.2.0`) and `x86_64` with your architecture (`aarch64`).\n\n**Arch Linux (AUR):** install with your preferred AUR helper:\n\n```sh\nyay -S sql-pipe\n# or\nparu -S sql-pipe\n```\n\n**Nix / NixOS:**\n\n```sh\n# Run without installing\nnix run github:vmvarela/sql-pipe -- 'SELECT * FROM t'\n\n# Install to profile\nnix profile install github:vmvarela/sql-pipe\n\n# Non-flake\nnix-env -if https://github.com/vmvarela/sql-pipe/archive/master.tar.gz\n```\n\n**Windows (Chocolatey):**\n\n```powershell\nchoco install sql-pipe\n```\n\n**Windows (WinGet):**\n\n```powershell\nwinget install vmvarela.sql-pipe\n```\n\n**Windows (Scoop):**\n\n```powershell\nscoop bucket add sql-pipe https://github.com/vmvarela/scoop-sql-pipe\nscoop install sql-pipe\n```\n\nTo build from source (requires [Zig 0.15+](https://ziglang.org/download/)):\n\n```sh\ngit clone https://github.com/vmvarela/sql-pipe\ncd sql-pipe\nmkdir -p lib\ncurl -fsSL https://www.sqlite.org/2025/sqlite-amalgamation-3490100.zip -o sqlite.zip\nunzip -j sqlite.zip '*/sqlite3.c' '*/sqlite3.h' -d lib/\nzig build -Dbundle-sqlite=true -Doptimize=ReleaseSafe\n```\n\nBinary lands at `./zig-out/bin/sql-pipe`. SQLite is compiled from the official amalgamation — no system dependencies.\n\n## Usage\n\nThe CSV comes from stdin. The first row must be a header — those column names become the schema for a table called `t`. Results go to stdout as comma-separated values.\n\n```sh\n$ printf 'name,age\\nAlice,30\\nBob,25\\nCarol,35' | sql-pipe 'SELECT * FROM t'\nAlice,30\nBob,25\nCarol,35\n```\n\nColumns are auto-detected as `INTEGER`, `REAL`, or `TEXT` based on the first 100 rows. Use `--no-type-inference` to force all columns to `TEXT`:\n\n```sh\n$ cat orders.csv | sql-pipe 'SELECT COUNT(*), AVG(amount) FROM t WHERE status = \"paid\"'\n142,87.35\n```\n\nColumn names with spaces work — quote them in SQL:\n\n```sh\n$ cat report.csv | sql-pipe 'SELECT \"first name\", \"last name\" FROM t WHERE \"dept id\" = \"42\"'\n```\n\nUse a custom input delimiter with `-d` / `--delimiter` (single character), or `--tsv` for tab-separated files:\n\n```sh\n$ cat data.psv | sql-pipe -d '|' 'SELECT * FROM t'\n$ cat data.tsv | sql-pipe --tsv 'SELECT * FROM t'\n# equivalent:\n$ cat data.tsv | sql-pipe --delimiter '\\t' 'SELECT * FROM t'\n```\n\nOutput results as a JSON array of objects with `--json`:\n\n```sh\n$ printf 'name,age\\nAlice,30\\nBob,25' | sql-pipe --json 'SELECT * FROM t'\n[{\"name\":\"Alice\",\"age\":30},{\"name\":\"Bob\",\"age\":25}]\n```\n\n`--json` is mutually exclusive with `-H`/`--header`. It can be combined with `-d`/`--delimiter` and `--tsv` to read non-comma-separated input.\n\nChain queries by piping back in — useful for two-pass aggregations:\n\n```sh\n$ cat events.csv \\\n  | sql-pipe 'SELECT user_id, COUNT(*) as n FROM t GROUP BY user_id' \\\n  | sql-pipe 'SELECT * FROM t WHERE n \u003e 100'\n```\n\n### Flags\n\n| Flag | Description |\n|------|-------------|\n| `-d`, `--delimiter \u003cchar\u003e` | Input field delimiter (single character, default `,`) |\n| `--tsv` | Alias for `--delimiter '\\t'` |\n| `--no-type-inference` | Treat all columns as TEXT (skip auto-detection) |\n| `-H`, `--header` | Print column names as the first output row |\n| `--json` | Output results as a JSON array of objects (mutually exclusive with `-H`) |\n| `--max-rows \u003cn\u003e` | Stop if more than `n` data rows are read (exit 1) |\n| `--columns` | Read the CSV header row, print each column name on its own line, and exit 0. With `-v`/`--verbose`, also shows the inferred type per column (`name INTEGER`). Respects `--delimiter` and `--tsv`. Mutually exclusive with a query argument. |\n| `--output \u003cfile\u003e` | Write results to the given file instead of stdout. Creates or overwrites the file. Exits 1 if the file cannot be created. |\n| `-v`, `--verbose` | Print `Loaded \u003cn\u003e rows in \u003ct\u003es` to stderr after loading (always on TTY; forced with flag) |\n| `-h`, `--help` | Show usage help and exit |\n| `-V`, `--version` | Print version and exit |\n\nAfter loading, `sql-pipe` prints `Loaded \u003cn\u003e rows in \u003ct\u003es` to stderr whenever stderr is a TTY (interactive terminal). The message is suppressed in scripts and pipes to keep them noise-free. Use `-v` / `--verbose` to force it regardless of TTY:\n\n```sh\n$ cat sales.csv | sql-pipe --verbose 'SELECT region, SUM(revenue) FROM t GROUP BY region'\n# stderr: Loaded 42,317 rows in 1.2s\n```\n\nWhen stderr is a TTY and the input exceeds 10,000 rows, a running counter updates in place on stderr during loading:\n\n```\nLoading... 10,000 rows\nLoading... 20,000 rows\n...\nLoaded 42,317 rows in 1.2s\n```\n\nWhen `--max-rows` is set, the total limit is shown alongside the current count:\n\n```\nLoading... 10,000 / 100,000 rows\n```\n\nThe counter is suppressed in pipes and scripts (zero overhead when stderr is not a TTY). The count uses thousands separators (`42,317` not `42317`). It is always written to stderr so stdout remains clean for piping.\n\n### Exit Codes\n\n| Code | Meaning |\n|------|----------|\n| `0` | Success |\n| `1` | Usage error (missing query, bad arguments) |\n| `2` | CSV parse error (with 1-based row number) |\n| `3` | SQL error (with sqlite3 error message, available columns, and a \"did you mean?\" hint when applicable) |\n\nAll error messages are prefixed with `error:` and written to stderr.\n\nOn SQL error, `sql-pipe` also prints the list of columns available in table `t` and,\nwhen the unknown identifier closely matches a column name (edit distance ≤ 2), a hint:\n\n```\nerror: no such column: amout\n  table \"t\" has columns: id, amount, region\n  hint: did you mean \"amount\"?\n```\n\n## Recipes\n\n**Top N rows by a column:**\n\n```sh\n$ cat sales.csv | sql-pipe 'SELECT product, revenue FROM t ORDER BY revenue DESC LIMIT 10'\n```\n\n**Deduplicate rows:**\n\n```sh\n$ cat contacts.csv | sql-pipe 'SELECT DISTINCT email FROM t'\n```\n\n**Find rows with missing values:**\n\n```sh\n$ cat users.csv | sql-pipe 'SELECT * FROM t WHERE email = \"\" OR email IS NULL'\n```\n\n**Date range filter (dates stored as text):**\n\n```sh\n$ cat logs.csv | sql-pipe 'SELECT * FROM t WHERE ts \u003e= \"2024-01-01\" AND ts \u003c \"2024-02-01\"'\n```\n\n**Compute a derived column:**\n\n```sh\n$ cat products.csv | sql-pipe 'SELECT name, price, ROUND(price * 0.9, 2) as discounted FROM t'\n```\n\n**Pivot-like aggregation with conditional sums:**\n\n```sh\n$ cat orders.csv | sql-pipe 'SELECT region, SUM(CASE WHEN status=\"paid\" THEN amount ELSE 0 END) as paid, SUM(CASE WHEN status=\"refunded\" THEN amount ELSE 0 END) as refunded FROM t GROUP BY region'\n```\n\n## How it works\n\nEach run opens a fresh `:memory:` SQLite database. The header row drives a `CREATE TABLE t (...)` with all columns as `TEXT`. Rows are loaded in a single transaction via a prepared `INSERT` statement, then `sqlite3_exec` runs your query and prints rows one by one.\n\nThe database never touches disk and vanishes when the process exits. No state, no cleanup.\n\n## Limitations\n\n- **Single table per invocation.** For joins, use chained `sql-pipe` calls or a `WITH` CTE.\n\n## Related\n\n- **[q](https://harelba.github.io/q/)** — similar concept in Python; handles quoted CSV fields and more formats. Better if you're already in a Python environment.\n- **[trdsql](https://github.com/noborus/trdsql)** — Go alternative with multi-format support (JSON, LTSV) and output formatting. Better if you need non-CSV inputs.\n- **[sqlite-utils](https://sqlite-utils.datasette.io/)** — better if you need persistent databases, schema management, or Python scripting.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmvarela%2Fsql-pipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmvarela%2Fsql-pipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmvarela%2Fsql-pipe/lists"}