{"id":51305593,"url":"https://github.com/melihbirim/csvql","last_synced_at":"2026-07-01T00:00:18.503Z","repository":{"id":351010578,"uuid":"1164657235","full_name":"melihbirim/csvql","owner":"melihbirim","description":"SQL queries for CSV files. Ultra-fast CSV query engine in Zig with SIMD parsing and parallel execution. CLI: csvql.","archived":false,"fork":false,"pushed_at":"2026-06-30T16:17:33.000Z","size":10925,"stargazers_count":17,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-30T16:27:47.925Z","etag":null,"topics":["cli","command-line","csv","csv-parser","data-processing","duckdb","fast","query-engine","sql","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/melihbirim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-23T10:37:50.000Z","updated_at":"2026-06-30T16:17:38.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/melihbirim/csvql","commit_stats":null,"previous_names":["melihbirim/csvql"],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/melihbirim/csvql","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melihbirim%2Fcsvql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melihbirim%2Fcsvql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melihbirim%2Fcsvql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melihbirim%2Fcsvql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/melihbirim","download_url":"https://codeload.github.com/melihbirim/csvql/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melihbirim%2Fcsvql/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34987610,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-30T02:00:05.919Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line","csv","csv-parser","data-processing","duckdb","fast","query-engine","sql","zig"],"created_at":"2026-07-01T00:00:17.830Z","updated_at":"2026-07-01T00:00:18.496Z","avatar_url":"https://github.com/melihbirim.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"logo.svg\" alt=\"csvql\" width=\"420\"/\u003e\n\u003c/p\u003e\n\n[![CI](https://github.com/melihbirim/csvql/actions/workflows/ci.yml/badge.svg)](https://github.com/melihbirim/csvql/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE.md)\n[![Release](https://img.shields.io/github/v/release/melihbirim/csvql)](https://github.com/melihbirim/csvql/releases)\n\n**Query CSV files with SQL. Faster than DuckDB.**\n\n```bash\n$ csvql \"SELECT name, city, salary FROM 'employees.csv' WHERE salary \u003e 100000 ORDER BY salary DESC LIMIT 5\"\n\nname,city,salary\nAlice,San Francisco,185000\nBob,New York,172000\nCarol,Seattle,168000\nDave,Austin,155000\nEve,Boston,142000\n\n  0.020s — 9x faster than DuckDB on 1M rows\n```\n\n[Quick Start](#quick-start) · [Installation](#installation) · [Performance](#performance) · [SQL Reference](#sql-reference) · [Docs](#documentation)\n\n---\n\n## Quick Start\n\ncsvql auto-detects SQL or simple mode from your input:\n\n```bash\n# SQL mode\ncsvql \"SELECT name, salary FROM 'data.csv' WHERE age \u003e 30 ORDER BY salary DESC LIMIT 10\"\n\n# Simple mode — same query, shorter syntax\ncsvql data.csv \"name,salary\" \"age\u003e30\" 10 \"salary:desc\"\n\n# Just browse a file\ncsvql data.csv\n```\n\n### Unix Pipes\n\n```bash\ncat data.csv | csvql \"SELECT name, age FROM '-' WHERE age \u003e 25\"\ncsvql \"SELECT * FROM 'data.csv' WHERE status = 'active'\" \u003e output.csv\ncsvql \"SELECT email FROM 'users.csv'\" | wc -l\n```\n\n### Flags\n\n| Flag                 | Short | Description                                         |\n| -------------------- | ----- | --------------------------------------------------- |\n| `--no-header`        |       | Suppress header row in output                       |\n| `--delimiter \u003cchar\u003e` | `-d`  | Field delimiter (default `,`). Use `\\t` for TSV     |\n| `--json`             |       | Output as a JSON array (`[{...}, ...]`)             |\n| `--jsonl`            |       | Output as JSONL / NDJSON (one JSON object per line) |\n| `--version`          | `-v`  | Show version                                        |\n| `--help`             | `-h`  | Show help                                           |\n| `--mcp`              |       | Start as an MCP server (stdio JSON-RPC transport)   |\n\n```bash\n# TSV file\ncsvql \"SELECT name, salary FROM 'data.tsv'\" -d $'\\t'\n\n# Pipe into another tool that expects no header\ncsvql \"SELECT name, age FROM 'data.csv'\" --no-header | awk -F, '{print $2}'\n\n# TSV input, no header in output\ncat data.tsv | csvql \"SELECT * FROM '-'\" -d $'\\t' --no-header\n```\n\n## Installation\n\n### Homebrew (macOS / Linux)\n\n```bash\nbrew install melihbirim/csvql/csvql\n```\n\nOr in two steps if you plan to install multiple tools from this tap:\n\n```bash\nbrew tap melihbirim/csvql\nbrew install csvql\n```\n\n\u003e `melihbirim/csvql` is the tap (the formula repository), and the trailing `/csvql` is the formula name inside it.\n\n### Prebuilt Binaries\n\nDownload from [GitHub Releases](https://github.com/melihbirim/csvql/releases):\n\n```bash\n# macOS (Apple Silicon)\ncurl -L https://github.com/melihbirim/csvql/releases/latest/download/csvql-macos-aarch64.tar.gz | tar xz\nsudo mv csvql-macos-aarch64 /usr/local/bin/csvql\n\n# macOS (Intel)\ncurl -L https://github.com/melihbirim/csvql/releases/latest/download/csvql-macos-x86_64.tar.gz | tar xz\nsudo mv csvql-macos-x86_64 /usr/local/bin/csvql\n\n# Linux (x86_64)\ncurl -L https://github.com/melihbirim/csvql/releases/latest/download/csvql-linux-x86_64.tar.gz | tar xz\nsudo mv csvql-linux-x86_64 /usr/local/bin/csvql\n```\n\n### Build from Source\n\nRequires [Zig](https://ziglang.org/) 0.13.0+ (tested with 0.15.2):\n\n```bash\ngit clone https://github.com/melihbirim/csvql.git\ncd csvql\nzig build -Doptimize=ReleaseFast\nsudo cp zig-out/bin/csvql /usr/local/bin/\n```\n\n## Performance\n\n**1M rows, 35MB CSV, Apple M2** — all tools forced to output all rows (no display tricks):\n\n| Query                             | csvql      | DuckDB | Speedup  |\n| --------------------------------- | ---------- | ------ | -------- |\n| WHERE + ORDER BY LIMIT 10         | **0.020s** | 0.179s | **9x**   |\n| ORDER BY LIMIT 10                 | **0.041s** | 0.165s | **4x**   |\n| ORDER BY (all 1M rows)            | **0.156s** | 1.221s | **7.8x** |\n| WHERE (full output)               | **0.141s** | 0.739s | **5.2x** |\n| Full scan (all 1M rows)           | **0.196s** | 1.163s | **5.9x** |\n| `COUNT(*) GROUP BY` (6 groups)    | **0.060s** | 0.110s | **1.8x** |\n| `SUM + AVG GROUP BY` (6 groups)   | **0.070s** | 0.110s | **1.6x** |\n| `SUM(CASE WHEN) GROUP BY`         | **0.016s** | 0.114s | **7.1x** |\n| `SELECT DISTINCT city` (8 values) | **0.060s** | 0.110s | **1.8x** |\n| `SELECT COUNT(*)` scalar          | **0.050s** | 0.100s | **2x**   |\n| `SELECT SUM(salary)` scalar       | **0.050s** | 0.110s | **2.2x** |\n\n**35x less memory** than DuckDB (1.8MB vs 63.5MB).\n\n**5M rows, 173MB CSV, Apple M2** — output-format benchmark (full output, all rows matched, `\u003e /dev/null`):\n\n| Output format              | csvql      | DuckDB | Speedup  |\n| -------------------------- | ---------- | ------ | -------- |\n| CSV                        | **0.100s** | 0.354s | **3.5x** |\n| JSON array (`--json`)      | **0.164s** | 0.434s | **2.6x** |\n| JSONL / NDJSON (`--jsonl`) | **0.172s** | 0.422s | **2.5x** |\n\nOutputs are semantically/byte-identical to DuckDB (verified: CSV byte-for-byte diff; JSONL byte-for-byte diff; JSON array Python-parsed row comparison).\n\nRun the benchmark yourself: [`bench/bench_all.sh --section formats`](bench/bench_all.sh)\n\n**5M rows, 173MB CSV, Apple M2** — LIKE operator benchmark (CSV output, `\u003e /dev/null`):\n\n| Pattern                        | Description              | csvql     | DuckDB | Speedup  |\n| ------------------------------ | ------------------------ | --------- | ------ | -------- |\n| `WHERE name LIKE 'A%'`         | Prefix wildcard          | **0.06s** | 2.17s  | **~36x** |\n| `WHERE city LIKE '%on'`        | Suffix wildcard          | **0.06s** | 1.12s  | **~19x** |\n| `WHERE department LIKE '%ing'` | Suffix, high selectivity | **0.07s** | 2.54s  | **~36x** |\n\nRow counts verified identical to DuckDB.\n\nRun the benchmark yourself: [`bench/bench_all.sh --section like`](bench/bench_all.sh)\n\n**1M rows, 35MB CSV, Apple M2** — JOIN benchmark (hash-join, CSV output, `\u003e /dev/null`):\n\n| Query                                          | csvql      | DuckDB | Speedup   |\n| ---------------------------------------------- | ---------- | ------ | --------- |\n| `JOIN departments` (1M × 6 rows)               | **0.140s** | 1.492s | **10.7x** |\n| `JOIN + WHERE d.region = 'West'` (1M × 6)      | **0.102s** | 0.600s | **5.9x**  |\n| `JOIN SELECT *` (1M × 6, all cols)             | **0.220s** | 4.130s | **18.8x** |\n| `JOIN cities` (1M × 8 rows)                    | **0.146s** | 1.464s | **10.0x** |\n| `JOIN bonus_50k` (1M × 50K rows, numeric key)  | **0.104s** | 0.276s | **2.7x**  |\n\nRun the benchmark yourself: [`bench/bench_all.sh --section join`](bench/bench_all.sh)\n\nRun the full suite (all sections): [`bench/bench_all.sh`](bench/bench_all.sh)\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eHow is csvql so fast?\u003c/b\u003e\u003c/summary\u003e\n\n- **Memory-mapped I/O** — zero-copy reading at 1.4 GB/sec\n- **7-core parallel execution** — lock-free architecture, 669% CPU utilization\n- **SIMD field parsing** — vectorized comma detection\n- **Radix sort** — O(8N) with IEEE 754 f64→u64 bit trick and pass-skipping\n- **Top-K heap** — O(N log K) for LIMIT queries, avoids sorting entire dataset\n- **Hardware-aware thresholds** — ARM vs x86 tuned for L1 cache\n- **Zero per-row allocations** — arena buffers, zero-copy slices\n- **Adaptive GROUP BY pre-sizing** — hash table capacity tuned to chunk size, eliminates rehash cycles\n- **Zero-copy worker scans** — each thread iterates a direct mmap slice, no `pread` syscalls or seam buffers\n\nSee [ARCHITECTURE.md](ARCHITECTURE.md) for the full optimization story.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eBenchmark methodology\u003c/b\u003e\u003c/summary\u003e\n\nDuckDB and DataFusion CLIs default to displaying only 40 rows, making them appear faster than they are. Our benchmarks use `-csv` mode (DuckDB) and `FORMAT CSV` (ClickHouse) to force full output materialization. DataFusion CLI caps output at ~8K rows regardless of settings, so full-output numbers are unavailable.\n\nSee [BENCHMARKS.md](BENCHMARKS.md) for the complete analysis.\n\n\u003c/details\u003e\n\n## SQL Reference\n\n### Supported\n\n| Feature       | Syntax                                                                  |\n| ------------- | ----------------------------------------------------------------------- |\n| **SELECT**    | `SELECT col1, col2` or `SELECT *`                                       |\n| **AS alias**  | `SELECT expr AS alias` — rename any column or expression in output      |\n| **DISTINCT**  | `SELECT DISTINCT col1, col2` — deduplicates output rows                 |\n| **FROM**      | `FROM 'file.csv'` or `FROM -` (stdin)                                   |\n| **WHERE**     | `=`, `!=`, `\u003e`, `\u003e=`, `\u003c`, `\u003c=` with auto numeric coercion              |\n| **LIKE**      | `WHERE col LIKE 'pattern'` — `%` any sequence, `_` any single char      |\n| **ILIKE**     | `WHERE col ILIKE 'pattern'` — same as LIKE but case-insensitive         |\n| **BETWEEN**   | `WHERE col BETWEEN low AND high` — inclusive numeric or string range    |\n| **IN**        | `WHERE col IN ('a', 'b', 'c')` — membership test                        |\n| **IS NULL**   | `WHERE col IS NULL` / `WHERE col IS NOT NULL` — empty-field test        |\n| **NOT**       | `WHERE NOT expr` — logical negation of any condition                    |\n| **AND / OR**  | `WHERE cond1 AND cond2` / `WHERE cond1 OR cond2` — compound conditions  |\n| **JOIN**      | `FROM 'a.csv' a [INNER] JOIN 'b.csv' b ON a.key = b.key`               |\n| **GROUP BY**  | `GROUP BY col1` or `GROUP BY alias` — groups rows; accepts SELECT aliases |\n| **COUNT**     | `COUNT(*)` or `COUNT(col)` — with or without `GROUP BY`                 |\n| **SUM**       | `SUM(col)` or `SUM(CASE WHEN cond THEN n ELSE m END)` — conditional sum |\n| **AVG**       | `AVG(col)` — full precision; with or without `GROUP BY`                 |\n| **CASE WHEN** | `CASE WHEN col OP val THEN n ELSE m END` inside any aggregate function  |\n| **MIN / MAX** | `MIN(col)`, `MAX(col)` — with or without `GROUP BY`                     |\n| **HAVING**    | `HAVING expr` — filter groups after aggregation (e.g. `HAVING COUNT(*) \u003e 5`) |\n| **STRFTIME**  | `STRFTIME('%Y-%m', col)` — date bucketing in `SELECT` and `GROUP BY`    |\n| **UPPER / LOWER** | `SELECT UPPER(col), LOWER(col)` — case conversion                  |\n| **TRIM**      | `SELECT TRIM(col)` — strip leading and trailing whitespace              |\n| **LENGTH**    | `SELECT LENGTH(col)` — byte length of the value                         |\n| **SUBSTR**    | `SELECT SUBSTR(col, start, len)` — substring (1-based, `len` optional)  |\n| **ABS / CEIL / FLOOR** | `SELECT ABS(col), CEIL(col), FLOOR(col)` — numeric functions  |\n| **MOD**       | `SELECT MOD(col, n)` — modulo by a numeric literal                      |\n| **ROUND**     | `SELECT ROUND(col)` — round to integer; `ROUND(col, n)` — round to `n` decimal places |\n| **COALESCE**  | `SELECT COALESCE(col, 'default')` — replace empty/null with fallback    |\n| **CAST**      | `SELECT CAST(col AS INTEGER/FLOAT/TEXT)` — type conversion              |\n| **DATEDIFF**  | `DATEDIFF('unit', start_col, end_col)` — duration between two datetime columns. Units: `second`, `minute`, `hour`, `day`, `week`, `month` (≈30 days), `year` (≈365 days). Auto-detects ISO-8601, US (MM/DD/YYYY), EU (DD.MM.YYYY) and mixed formats in the same file |\n| **DATEADD**   | `DATEADD('unit', amount, date_col)` — add/subtract interval from a datetime column. `amount` may be negative. Units: `second`, `minute`, `hour`, `day`, `week`, `month` (≈30 days), `year` (≈365 days). Returns `YYYY-MM-DD HH:MM:SS` |\n| **ORDER BY**  | `ORDER BY col [ASC\\|DESC]`, multi-column `ORDER BY col1 ASC, col2 DESC`, alias, or positional (`ORDER BY 1`) |\n| **LIMIT**     | `LIMIT n`                                                               |\n\n### Aggregate Examples\n\n```bash\n# Scalar aggregates (whole table)\ncsvql \"SELECT COUNT(*), SUM(salary), AVG(salary), MIN(age), MAX(age) FROM 'data.csv'\"\n\n# Grouped aggregates\ncsvql \"SELECT department, COUNT(*), AVG(salary) FROM 'data.csv' GROUP BY department ORDER BY department\"\n\n# HAVING — filter groups after aggregation\ncsvql \"SELECT department, SUM(salary) FROM 'data.csv' GROUP BY department HAVING SUM(salary) \u003e 500000\"\ncsvql \"SELECT category, COUNT(*) FROM 'orders.csv' GROUP BY category HAVING COUNT(*) \u003e 1000\"\n\n# CASE WHEN inside aggregates — conditional counting and summing\ncsvql \"SELECT department, COUNT(*) AS total, SUM(CASE WHEN city = 'London' THEN 1 ELSE 0 END) AS london_count FROM 'data.csv' GROUP BY department\"\ncsvql \"SELECT SUM(CASE WHEN status = 'active' THEN 1 ELSE 0 END) AS active, SUM(CASE WHEN status = 'inactive' THEN 1 ELSE 0 END) AS inactive FROM 'data.csv'\"\ncsvql \"SELECT product, COUNT(*) AS total, SUM(CASE WHEN status = 'returned' THEN 1 ELSE 0 END) AS returns FROM 'orders.csv' GROUP BY product ORDER BY returns DESC\"\n\n# DISTINCT\ncsvql \"SELECT DISTINCT city FROM 'data.csv' ORDER BY city\"\ncsvql \"SELECT DISTINCT city, department FROM 'data.csv'\"\n\n# DISTINCT with WHERE\ncsvql \"SELECT DISTINCT department FROM 'data.csv' WHERE salary \u003e 100000\"\n```\n\n### Scalar Function Examples\n\nScalar functions transform column values row-by-row in `SELECT`. They can also be used in `GROUP BY` projections.\n\n```bash\n# String functions\ncsvql \"SELECT UPPER(name), LOWER(city), TRIM(notes) FROM 'data.csv'\"\ncsvql \"SELECT name, LENGTH(name), SUBSTR(name, 1, 3) FROM 'data.csv'\"\n\n# Numeric functions\ncsvql \"SELECT name, ABS(balance), CEIL(score), FLOOR(score) FROM 'data.csv'\"\ncsvql \"SELECT name, MOD(age, 10) AS age_decade FROM 'data.csv'\"\ncsvql \"SELECT name, ROUND(price) AS rounded, ROUND(price, 2) AS price_2dp FROM 'data.csv'\"\n\n# COALESCE — replace empty values with a fallback\ncsvql \"SELECT name, COALESCE(email, 'unknown') FROM 'data.csv'\"\ncsvql \"SELECT COALESCE(phone, COALESCE(email, 'no contact')) FROM 'contacts.csv'\"\n\n# CAST — explicit type conversion\ncsvql \"SELECT name, CAST(price AS INTEGER), CAST(id AS TEXT) FROM 'products.csv'\"\ncsvql \"SELECT CAST(year AS INTEGER) AS yr, SUM(revenue) FROM 'data.csv' GROUP BY yr\"\n\n# ILIKE — case-insensitive LIKE\ncsvql \"SELECT * FROM 'data.csv' WHERE name ILIKE '%smith%'\"\ncsvql \"SELECT * FROM 'data.csv' WHERE email ILIKE '%@gmail.com'\"\n\n# Scalar functions work with GROUP BY\ncsvql \"SELECT UPPER(city), COUNT(*) FROM 'data.csv' GROUP BY city\"\ncsvql \"SELECT LOWER(department) AS dept, AVG(salary) FROM 'data.csv' GROUP BY department\"\n\n# Mix scalars with AS aliases\ncsvql \"SELECT UPPER(name) AS Name, CAST(salary AS INTEGER) AS Salary FROM 'data.csv' ORDER BY Salary DESC\"\n```\n\n### WHERE Filter Examples\n\n```bash\n# Comparison operators\ncsvql \"SELECT name, salary FROM 'data.csv' WHERE salary \u003e 80000\"\n\n# BETWEEN — inclusive range (numeric or string)\ncsvql \"SELECT name, salary FROM 'data.csv' WHERE salary BETWEEN 50000 AND 80000\"\ncsvql \"SELECT * FROM 'orders.csv' WHERE order_date BETWEEN '2025-01-01' AND '2025-12-31'\"\n\n# IN — membership test\ncsvql \"SELECT name FROM 'data.csv' WHERE city IN ('London', 'Paris', 'Berlin')\"\n\n# IS NULL / IS NOT NULL — test for missing (empty) fields\ncsvql \"SELECT * FROM 'data.csv' WHERE email IS NULL\"\ncsvql \"SELECT * FROM 'data.csv' WHERE email IS NOT NULL\"\n\n# NOT — negate any condition\ncsvql \"SELECT * FROM 'data.csv' WHERE NOT city IN ('London', 'Paris')\"\ncsvql \"SELECT * FROM 'data.csv' WHERE NOT salary BETWEEN 40000 AND 60000\"\n\n# AND / OR — compound conditions\ncsvql \"SELECT * FROM 'data.csv' WHERE age \u003e 30 AND department = 'Engineering'\"\ncsvql \"SELECT * FROM 'data.csv' WHERE city = 'London' OR city = 'Berlin'\"\ncsvql \"SELECT * FROM 'data.csv' WHERE status LIKE 'active%' AND salary \u003e 50000\"\n\n# AS alias + ORDER BY alias or positional\ncsvql \"SELECT name AS employee, salary AS pay FROM 'data.csv' ORDER BY pay DESC LIMIT 10\"\ncsvql \"SELECT city, COUNT(*) AS cnt FROM 'data.csv' GROUP BY city ORDER BY cnt DESC\"\ncsvql \"SELECT name, salary FROM 'data.csv' ORDER BY 2 DESC LIMIT 5\"  # ORDER BY positional\n```\n\n### ORDER BY Examples\n\n```bash\n# Single-column ORDER BY\ncsvql \"SELECT name, salary FROM 'data.csv' ORDER BY salary DESC LIMIT 10\"\ncsvql \"SELECT * FROM 'data.csv' ORDER BY name ASC\"\n\n# Multi-column ORDER BY — sort by primary key, then break ties with secondary key(s)\ncsvql \"SELECT name, department, salary FROM 'data.csv' ORDER BY department ASC, salary DESC\"\ncsvql \"SELECT * FROM 'data.csv' ORDER BY city, age, name\"\ncsvql \"SELECT name, city, salary FROM 'employees.csv' WHERE salary \u003e 80000 ORDER BY city ASC, salary DESC\"\n\n# Multi-column ORDER BY with GROUP BY results\ncsvql \"SELECT department, AVG(salary) AS avg_sal FROM 'data.csv' GROUP BY department ORDER BY avg_sal DESC, department ASC\"\n\n# ORDER BY alias (resolved from SELECT clause)\ncsvql \"SELECT name, salary AS pay FROM 'data.csv' ORDER BY pay DESC LIMIT 5\"\n\n# ORDER BY positional (1-based column index)\ncsvql \"SELECT name, city, salary FROM 'data.csv' ORDER BY 3 DESC LIMIT 10\"\n```\n\n### Time-Series and Date Bucketing\n\n`STRFTIME('%fmt', column)` extracts or truncates date components for time-series aggregation.\n\nSupported format specifiers: `%Y` (year), `%m` (month), `%d` (day), `%H` (hour), `%M` (minute), `%S` (second).\n\nInput dates can be ISO-8601 date (`YYYY-MM-DD`) or datetime (`YYYY-MM-DD HH:MM:SS`).\n\n```bash\n# Monthly revenue trend — GROUP BY the full STRFTIME expression\ncsvql \"SELECT STRFTIME('%Y-%m', order_date), COUNT(*), SUM(price) FROM 'orders.csv' GROUP BY STRFTIME('%Y-%m', order_date)\"\n\n# Same query using AS alias — GROUP BY the alias name\ncsvql \"SELECT STRFTIME('%Y-%m', order_date) AS month, COUNT(*) AS orders, SUM(price) AS revenue FROM 'orders.csv' GROUP BY month ORDER BY month\"\n\n# Year-over-year breakdown by category\ncsvql \"SELECT category, STRFTIME('%Y', order_date) AS yr, SUM(price) FROM 'orders.csv' GROUP BY category, yr\"\n\n# Date range filter + monthly bucketing + HAVING\ncsvql \"SELECT STRFTIME('%Y-%m', order_date) AS month, COUNT(*), SUM(price) FROM 'orders.csv' WHERE order_date \u003e= '2026-01-01' GROUP BY month HAVING COUNT(*) \u003e 1000000\"\n\n# Daily active users\ncsvql \"SELECT STRFTIME('%Y-%m-%d', event_date) AS day, COUNT(DISTINCT user_id) FROM 'events.csv' GROUP BY day ORDER BY day\"\n```\n\n### DateTime and Duration Functions\n\n`DATEDIFF` and `DATEADD` work with **four datetime formats** in the same CSV — no pre-processing needed:\n\n| Format | Example |\n|--------|---------|\n| ISO-8601 (space) | `2026-01-15 09:30:00` |\n| ISO-8601 (T) | `2026-01-16T10:00:00` |\n| US (MM/DD/YYYY) | `01/15/2026 08:00:00` |\n| EU (DD.MM.YYYY) | `15.01.2026 07:30:00` |\n\n```bash\n# Order workflow: time from order to pick (in minutes)\ncsvql \"SELECT order_id, DATEDIFF('minute', ordered_at, picked_at) AS pick_min FROM 'orders.csv' WHERE picked_at != ''\"\n\n# Delivery time in days\ncsvql \"SELECT order_id, DATEDIFF('day', shipped_at, delivered_at) AS ship_days FROM 'orders.csv' WHERE shipped_at != '' AND delivered_at != '' ORDER BY ship_days DESC\"\n\n# SLA check — select orders with pick time, then filter in your shell (DATEDIFF in WHERE not yet supported)\ncsvql \"SELECT order_id, customer_name, DATEDIFF('hour', ordered_at, picked_at) AS hrs FROM 'orders.csv' WHERE picked_at != ''\"\n\n# Average processing time by status\ncsvql \"SELECT status, AVG(DATEDIFF('minute', ordered_at, packaged_at)) AS avg_proc_min FROM 'orders.csv' WHERE packaged_at != '' GROUP BY status ORDER BY avg_proc_min\"\n\n# DATEADD — compute SLA deadlines\ncsvql \"SELECT order_id, ordered_at, DATEADD('hour', 2, ordered_at) AS pick_deadline FROM 'orders.csv'\"\n\n# Estimated delivery date (ship date + 2 days)\ncsvql \"SELECT order_id, shipped_at, DATEADD('day', 2, shipped_at) AS est_delivery FROM 'orders.csv' WHERE shipped_at != ''\"\n\n# Supported units for both functions: second, minute, hour, day, week, month (approx 30 days), year (approx 365 days)\ncsvql \"SELECT order_id, DATEDIFF('second', ordered_at, picked_at) AS pick_secs FROM 'orders.csv'\"\ncsvql \"SELECT order_id, DATEADD('week', -1, delivered_at) AS sent_reminder FROM 'orders.csv'\"\n```\n\n**Mixed formats work automatically** — a single CSV can have some dates as `2026-01-15 09:30:00`, others as `01/15/2026 08:00:00`, and `DATEDIFF` handles them all.\n\n### JOIN Examples\n\n```bash\n# Basic INNER JOIN — select columns from both tables using aliases\ncsvql \"SELECT e.name, d.dept_name FROM 'employees.csv' e INNER JOIN 'departments.csv' d ON e.dept_id = d.id\"\n\n# Bare JOIN (INNER is optional)\ncsvql \"SELECT e.name, d.dept_name FROM 'employees.csv' e JOIN 'departments.csv' d ON e.dept_id = d.id\"\n\n# JOIN with WHERE — filter on joined columns\ncsvql \"SELECT e.name, d.dept_name FROM 'employees.csv' e JOIN 'departments.csv' d ON e.dept_id = d.id WHERE d.dept_name = 'Engineering'\"\n\n# SELECT * on join returns all columns from both tables\ncsvql \"SELECT * FROM 'orders.csv' o JOIN 'customers.csv' c ON o.customer_id = c.id\"\n\n# JOIN with LIMIT\ncsvql \"SELECT e.name, d.dept_name FROM 'employees.csv' e JOIN 'departments.csv' d ON e.dept_id = d.id LIMIT 10\"\n```\n\n**Notes:**\n- Table aliases are required when using qualified column references (`alias.col`)\n- Unqualified column names are resolved from the left table first, then the right\n- The right table is fully loaded into memory (build side); the left table is streamed (probe side)\n\n### Simple Mode\n\nPositional args: `csvql \u003cfile\u003e [columns] [filter] [limit] [sort]`\n\n```bash\ncsvql data.csv \"name,salary\" \"age\u003e30\" 10 \"salary:desc\"\n```\n\nSee [SIMPLE_QUERY_LANGUAGE.md](SIMPLE_QUERY_LANGUAGE.md) for the full reference.\n\n## MCP Server\n\ncsvql ships as a [Model Context Protocol](https://modelcontextprotocol.io/) server, letting AI assistants (Claude, Copilot, etc.) query your CSV files directly.\n\n```bash\ncsvql --mcp\n```\n\n### Exposed Tools\n\n| Tool | Description |\n|------|-------------|\n| `csv_query(sql)` | Execute any supported SQL query, returns results as JSON |\n| `csv_schema(file)` | Column names and sample rows for a CSV file |\n| `csv_list(directory?)` | List CSV files in a directory |\n\n### Supported Queries via MCP\n\n`csv_query` accepts the full SQL dialect supported by csvql. You can ask your AI assistant things like:\n\n| Natural language prompt | SQL sent to `csv_query` |\n|---|---|\n| \"Show me the top 10 customers by revenue\" | `SELECT customer, SUM(revenue) AS total FROM 'sales.csv' GROUP BY customer ORDER BY total DESC LIMIT 10` |\n| \"How many orders per month in 2025?\" | `SELECT STRFTIME('%Y-%m', order_date) AS month, COUNT(*) AS orders FROM 'orders.csv' WHERE order_date BETWEEN '2025-01-01' AND '2025-12-31' GROUP BY month ORDER BY 1` |\n| \"How long does delivery take on average?\" | `SELECT AVG(DATEDIFF('hour', shipped_at, delivered_at)) AS avg_hours FROM 'orders.csv' WHERE delivered_at != ''` |\n| \"Flag orders where picking exceeded SLA\" | `SELECT order_id, DATEDIFF('minute', ordered_at, picked_at) AS mins FROM 'orders.csv' WHERE picked_at != ''` (scalar functions in WHERE not yet supported — filter by `mins \u003e 90` in your shell) |\n| \"Add 2-day estimated delivery to shipments\" | `SELECT order_id, DATEADD('day', 2, shipped_at) AS est_delivery FROM 'orders.csv' WHERE shipped_at != ''` |\n| \"Which employees have no department?\" | `SELECT name FROM 'employees.csv' WHERE department IS NULL` |\n| \"List all cities, deduplicated, sorted\" | `SELECT DISTINCT city FROM 'data.csv' ORDER BY city` |\n| \"Average salary by department, only \u003e 80k avg\" | `SELECT department, AVG(salary) AS avg_sal FROM 'data.csv' GROUP BY department HAVING AVG(salary) \u003e 80000 ORDER BY avg_sal DESC` |\n| \"Join orders with customers, filter by region\" | `SELECT o.id, c.name FROM 'orders.csv' o JOIN 'customers.csv' c ON o.customer_id = c.id WHERE c.region = 'West'` |\n| \"Salaries in range 50k–70k\" | `SELECT name, salary FROM 'data.csv' WHERE salary BETWEEN 50000 AND 70000 ORDER BY salary` |\n| \"Employees not in London or Paris\" | `SELECT name, city FROM 'data.csv' WHERE NOT city IN ('London', 'Paris')` |\n\n**Full WHERE clause support:** `=`, `!=`, `\u003e`, `\u003e=`, `\u003c`, `\u003c=`, `LIKE`, `BETWEEN`, `IN`, `IS NULL`, `IS NOT NULL`, `NOT`, `AND`, `OR`\n\n**Full SELECT support:** column projections, `AS` aliases, `DISTINCT`, `COUNT`/`SUM`/`AVG`/`MIN`/`MAX`, `GROUP BY`, `HAVING`, `ORDER BY` (by name, alias, or position), `LIMIT`, `STRFTIME()`, `JOIN`, `UPPER`/`LOWER`/`TRIM`/`LENGTH`/`SUBSTR`, `ABS`/`CEIL`/`FLOOR`/`MOD`/`ROUND`, `COALESCE`, `CAST`, `DATEDIFF`, `DATEADD`, `EXTRACT`\n\n### Setup\n\n**VS Code (Copilot)** — create `.vscode/mcp.json` in your workspace:\n\n```json\n{\n  \"servers\": {\n    \"csvql\": {\n      \"type\": \"stdio\",\n      \"command\": \"/usr/local/bin/csvql\",\n      \"args\": [\"--mcp\"]\n    }\n  }\n}\n```\n\n**Claude Desktop** — add to `~/Library/Application Support/Claude/claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"csvql\": {\n      \"command\": \"/usr/local/bin/csvql\",\n      \"args\": [\"--mcp\"]\n    }\n  }\n}\n```\n\nOnce connected, you can ask your AI assistant to query CSV files directly:\n\u003e *\"What are the top 5 product categories by revenue this year?\"*\n\n## Language Libraries\n\ncsvql ships as a native library for Python and Node.js — same SIMD engine, same performance, no subprocess.\n\n### Node.js\n\n```bash\n# 1. Build the native addon\nzig build node -Doptimize=ReleaseFast\n# → zig-out/lib/csvql.node\n\n# 2. Run\nnode nodejs/bench.js\n```\n\n```js\nconst csvql = require('csvql-query');\n```\n\n#### `find()` — no SQL required\n\nFor users who don't want to write SQL. Pass a file path and a plain options object:\n\n```js\n// All rows\ncsvql.find('employees.csv')\n\n// Pick columns + filter\ncsvql.find('employees.csv', {\n    columns: ['name', 'city', 'salary'],\n    where:   'salary\u003e100000',\n})\n\n// AND / OR conditions, sort, limit\ncsvql.find('employees.csv', {\n    where:   'department=Engineering AND salary\u003e80000',\n    orderBy: 'salary:desc',\n    limit:   10,\n})\n\n// OR condition\ncsvql.find('employees.csv', {\n    columns: 'name,city',\n    where:   'city=Austin OR city=Boston',\n})\n```\n\n`where` operators: `=` `!=` `\u003e` `\u003e=` `\u003c` `\u003c=` — string values are quoted automatically, numbers stay numeric. Combine with `AND` / `OR`. For aggregates (`COUNT`, `SUM`, `AVG`, `GROUP BY`) use `query()`.\n\n#### `query()` — full SQL\n\n```js\n// Returns an array of objects (numbers are typed, not strings)\nconst rows = csvql.query(\"SELECT city, COUNT(*) as n, AVG(salary) as avg FROM 'employees.csv' GROUP BY city ORDER BY avg DESC\");\n// [{ city: 'Austin', n: 3, avg: 126666.67 }, ...]\n\n// Return raw CSV text instead\nconst csv = csvql.queryCsv(\"SELECT name, salary FROM 'employees.csv' WHERE salary \u003e 100000\");\n```\n\n#### Options\n\nBoth `query()` and `queryCsv()` accept an optional second argument:\n\n| Option           | Type      | Description                                                         |\n| ---------------- | --------- | ------------------------------------------------------------------- |\n| `delimiter`      | `string`  | Source field separator when not a comma — `'\\t'` for TSV, `'\\|'` for pipe-delimited, etc. |\n| `comment`        | `string`  | Skip lines that start with this string — e.g. `'#'` for shell-style comments. |\n| `skipEmptyLines` | `boolean` | Strip blank / whitespace-only lines before querying. Default `false`. |\n\n```js\n// TSV file\ncsvql.query(\"SELECT * FROM 'scores.tsv' WHERE score \u003e 90\", { delimiter: '\\t' })\n\n// File with comment rows and blank lines\ncsvql.query(\"SELECT * FROM 'data.csv'\", { comment: '#', skipEmptyLines: true })\n\n// JOIN two TSV files\ncsvql.query(\n  \"SELECT e.name, d.name as dept FROM 'employees.tsv' e JOIN 'departments.tsv' d ON e.dept_id = d.id\",\n  { delimiter: '\\t' }\n)\n```\n\n**Memory:** the engine streams through the file internally — RAM stays ~5 MB regardless of file size. Compare with `csv-parse` or `papaparse` sync APIs, which materialise all rows as JS objects (200–650 MB for a 42 MB / 1M row file).\n\n#### ETL — CSV → filter → database\n\nThe most common real-world pattern is: read a CSV, keep only the rows you want, insert them into a database. How each library handles this reveals a fundamental difference.\n\n**csv-parse** has no query language — it is a parser only. To filter you must load the entire file into memory as JS objects first, then call `.filter()`:\n\n```js\n// csv-parse: loads ALL 100k rows into heap, then filters in JS\nconst all  = parse(fs.readFileSync('employees.csv'), { columns: true, cast: true });\nconst rows = all.filter(r =\u003e\n    r.department === 'Engineering' \u0026\u0026 r.active === 'true' \u0026\u0026 r.salary \u003e 80000\n);\n// ⚠ cast:true converts numbers but NOT boolean strings — 'true' stays a string,\n//   so r.active === true silently returns 0 rows. You must compare against 'true'.\ndb.insert(rows);\n```\n\n**csvql** pushes the WHERE clause into the Zig engine. Only matching rows are ever returned to JS — the heap cost scales with the result set, not the source file:\n\n```js\n// csvql: engine filters first, only 9,991 rows reach JS\nconst rows = csvql.query(`\n    SELECT id, name, city, salary, department\n    FROM 'employees.csv'\n    WHERE department = 'Engineering'\n      AND active     = 'true'\n      AND salary     \u003e 80000\n`);\ndb.insert(rows);\n```\n\nResults on 100k rows (42 MB CSV, filter matches ~10%):\n\n|                       | csv-parse        | csvql                        |\n| --------------------- | ---------------- | ---------------------------- |\n| Rows read into JS     | 100,000 (all)    | 9,991 (matching only)        |\n| Time                  | ~200 ms          | ~22 ms                       |\n| Heap allocated        | +31 MB           | +3 MB                        |\n| On a 10 GB CSV        | OOM or very slow | same ~22 ms, same ~3 MB heap |\n\nOn a 10 GB file csv-parse loads the entire dataset into memory before a single row reaches the database. csvql's heap cost stays constant because the engine never materialises rows that don't match the filter.\n\nFull runnable example: `node --experimental-sqlite nodejs/example_etl.js`\n\n**Benchmarks:** `node --expose-gc nodejs/bench.js` — comparison against `csv-parse` and `papaparse` on 1M rows.  \n**Feature comparison:** `node nodejs/compare.js` — side-by-side code and output for 10 common tasks.\n\n---\n\n### Python\n\n```bash\npip install csvql\n```\n\n```python\nimport csvql\n\n# List of dicts\nrows = csvql.query(\"SELECT city, COUNT(*) as n FROM 'employees.csv' GROUP BY city\")\n# [{'city': 'Austin', 'n': '3'}, ...]\n\n# Raw CSV string\ncsv_text = csvql.query_csv(\"SELECT * FROM 'employees.csv' WHERE salary \u003e 100000\")\n\n# pandas DataFrame (requires pandas)\ndf = csvql.query_df(\"SELECT region, SUM(revenue) FROM 'sales.csv' GROUP BY region\")\n\n# Plain tuples — lowest overhead\nheaders, rows = csvql.query_tuples(\"SELECT name, age FROM 'employees.csv'\")\n```\n\n---\n\n## Documentation\n\n| Document                                             | Description                                         |\n| ---------------------------------------------------- | --------------------------------------------------- |\n| [ARCHITECTURE.md](ARCHITECTURE.md)                   | Engine design, optimization techniques              |\n| [BENCHMARKS.md](BENCHMARKS.md)                       | Detailed performance analysis vs DuckDB, ClickHouse |\n| [SIMPLE_QUERY_LANGUAGE.md](SIMPLE_QUERY_LANGUAGE.md) | Simple mode syntax reference                        |\n| [docs/LIBRARY.md](docs/LIBRARY.md)                   | Using the CSV parser as a Zig library               |\n| [CONTRIBUTING.md](CONTRIBUTING.md)                   | Contribution guidelines                             |\n\n## Roadmap\n\n| Feature                             | Issue                                                | Status              |\n| ----------------------------------- | ---------------------------------------------------- | ------------------- |\n| `--no-header` / `--delimiter` flags | [#12](https://github.com/melihbirim/csvql/issues/12) | ✅ shipped (v0.5.0) |\n| `LIKE` operator in WHERE            | [#13](https://github.com/melihbirim/csvql/issues/13) | ✅ shipped          |\n| `--json` / `--jsonl` output format  | [#14](https://github.com/melihbirim/csvql/issues/14) | ✅ shipped          |\n| `HAVING` clause                     |                                                      | ✅ shipped          |\n| `STRFTIME()` date bucketing         |                                                      | ✅ shipped          |\n| MCP server (`--mcp`)                |                                                      | ✅ shipped          |\n| `AS` alias in SELECT \u0026 ORDER BY     |                                                      | ✅ shipped          |\n| `BETWEEN low AND high`              |                                                      | ✅ shipped          |\n| `IS NULL` / `IS NOT NULL`           |                                                      | ✅ shipped          |\n| `NOT` prefix for conditions         |                                                      | ✅ shipped          |\n| `ORDER BY` positional (`ORDER BY 1`)|                                                      | ✅ shipped          |\n| `GROUP BY` alias (`GROUP BY month`) |                                                      | ✅ shipped          |\n| `CASE WHEN` inside aggregates       |                                                      | ✅ shipped          |\n| `ILIKE` in WHERE                    |                                                      | ✅ shipped          |\n| `UPPER`, `LOWER`, `TRIM`, `LENGTH`, `SUBSTR` in SELECT |                             | ✅ shipped          |\n| `ABS`, `CEIL`, `FLOOR`, `MOD` in SELECT |                                                 | ✅ shipped          |\n| `ROUND(col)` / `ROUND(col, n)` in SELECT |                                               | ✅ shipped          |\n| `COALESCE` in SELECT                |                                                      | ✅ shipped          |\n| `CAST` in SELECT                    |                                                      | ✅ shipped          |\n\n## Contributing\n\nContributions welcome — bug reports, performance improvements, features, docs. See [CONTRIBUTING.md](CONTRIBUTING.md).\n\n## License\n\nMIT — see [LICENSE.md](LICENSE.md).\n\n---\n\n**Built with Zig** · **9x faster than DuckDB** · **MCP Server** · [GitHub](https://github.com/melihbirim/csvql)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelihbirim%2Fcsvql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmelihbirim%2Fcsvql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelihbirim%2Fcsvql/lists"}