{"id":51686891,"url":"https://github.com/moderately-ai/squonk","last_synced_at":"2026-07-19T01:01:40.347Z","repository":{"id":371169445,"uuid":"1298701684","full_name":"moderately-ai/squonk","owner":"moderately-ai","description":"Fast, typed, multi-dialect SQL parser for Rust, Python, and TypeScript","archived":false,"fork":false,"pushed_at":"2026-07-15T21:59:22.000Z","size":13297,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-15T22:28:42.574Z","etag":null,"topics":["ast","parser","python","rust","sql","sql-parser","typescript","webassembly"],"latest_commit_sha":null,"homepage":"https://github.com/moderately-ai/squonk/tree/main/docs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moderately-ai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-12T23:51:32.000Z","updated_at":"2026-07-15T21:59:26.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/moderately-ai/squonk","commit_stats":null,"previous_names":["moderately-ai/squonk"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/moderately-ai/squonk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moderately-ai%2Fsquonk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moderately-ai%2Fsquonk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moderately-ai%2Fsquonk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moderately-ai%2Fsquonk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moderately-ai","download_url":"https://codeload.github.com/moderately-ai/squonk/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moderately-ai%2Fsquonk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35560452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-16T02:00:06.687Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","parser","python","rust","sql","sql-parser","typescript","webassembly"],"created_at":"2026-07-15T22:03:55.332Z","updated_at":"2026-07-16T23:01:19.954Z","avatar_url":"https://github.com/moderately-ai.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# squonk\n\nA fast, compact, and standards-minded library for parsing, rendering, transforming, and analysing SQL.\n\n\u003cimg\n  src=\"./docs/assets/squonk-comic.png\"\n  alt=\"A Squonk dissolves into tears after being shown cross-dialect SQL.\"\n  width=\"900\"\n/\u003e\n\n## Why another SQL parser?\n\nThe [`sqlparser`](https://crates.io/crates/sqlparser) crate, maintained in the [`datafusion-sqlparser-rs`](https://github.com/apache/datafusion-sqlparser-rs) repository, is a mature and widely used SQL parser for Rust. `squonk` addresses the same problem with a different architecture: declarative dialects, a compact owned AST, and conformance testing against real database engines.\n\nDialect behaviour is defined through a trait with 184 overridable methods. As a result, each dialect becomes a patchwork of code overrides that is difficult to inspect, compare, and extend. Its allocation-heavy AST also keeps parsing among DataFusion’s identified performance opportunities.\n\n`squonk` approaches the problem differently:\n\n* Dialects are data, not subclasses.\n* The AST is owned, compact, and designed for long-lived tooling.\n* Engine-backed dialects are tested against the engines themselves, not just against our expectations\n\n## What makes it useful?\n\n* **Dialects are data.** An attempt to create a common feature based spec has been made, users and contributors can inspect a dialect, compare two presets field by field, or define a custom dialect as a small delta instead of forking parser code.\n* **The AST is built for tooling.** Every node is owned, `'static`, and includes a byte span and stable node ID. Trees can be stored, sent across threads, and mapped back to the exact source bytes.\n* **Errors are structured, and recovery is built in.** A recovering parse preserves valid statements while reporting invalid ones with locations and diagnostics. One malformed query does not get to ruin everyone else’s day.\n* **Rendering understands context.** Produce canonical, fully parenthesized, or PII-redacted SQL; render for a target dialect; or transpile between dialects in one call.\n* **The default dependency tree is tiny.** The Rust crate depends on one micro-dependency, `thin-vec`. Serde support is optional, keeping compile times and supply-chain surface pleasantly boring.\n* **Rust, Python, and TypeScript share one core.** Every language surface parses SQL using the same implementation, avoiding subtle differences between bindings.\n\nThe full subsystem and statement inventory lives in [docs/architecture.md](./docs/architecture.md). Design decisions are recorded in [docs/adr/](./docs/adr/).\n\n## Quick start\n\n### Rust\n\nAdd `squonk` with Cargo:\n\n```sh\ncargo add squonk\n```\n\n```rust\nuse squonk::parse;\n\nlet parsed = parse(\"select 1 +  2\").expect(\"well-formed SQL parses\");\n\nassert_eq!(parsed.to_string(), \"SELECT 1 + 2\");\n```\n\nThe default build includes only the ANSI dialect. Other native dialects are enabled through Cargo features; see [Feature flags](#feature-flags).\n\n### Python\n\n```sh\npip install squonk\n```\n\n```python\nimport squonk as sql\n\ndoc = sql.parse(\n    \"select salary from employees\",\n    dialect=\"ansi\",\n)\n\nassert doc.to_sql() == \"SELECT salary FROM employees\"\n\nidents = [ident.text for ident in doc.find_all(sql.Ident)]\n```\n\nThe Python API also supports recovering parses, structured diagnostics, tokenization with optional trivia, rendering, redaction, and transpilation.\n\nSee [crates/squonk-python](./crates/squonk-python/README.md).\n\n### TypeScript\n\n```sh\nnpm install @squonk-sql/postgres\n```\n\n```ts\nimport { parse } from \"@squonk-sql/postgres\";\n\nconst doc = parse(\"select id, name from users where id = $1\");\n\nconsole.log(doc.toSQL());\n```\n\nNode and Bun expose a synchronous API backed by a prebuilt Node-API engine, with automatic WASM fallback when addons are unavailable. Deno and Workers receive permissionless runtime-specific WASM entrypoints. Browsers use the explicit async factory from `@squonk-sql/postgres/browser`. Focused `@squonk-sql/*` packages retain dialect-gated APIs and dialect-sized WASM artifacts; the unscoped `squonk` package includes every built-in dialect.\n\nThe parser is pure Rust, so building it does not require a C toolchain.\n\nSee [crates/squonk-wasm](./crates/squonk-wasm/README.md).\n\n## Correctness\n\nParsers should not grade their own homework.\n\nEngine-backed dialects are tested against the engines they claim to support:\n\n* PostgreSQL through [`libpg_query`](https://github.com/pganalyze/libpg_query), running in process\n* SQLite and DuckDB through their embedded engines\n* MySQL through a running server\n\nThe test corpus includes regression suites vendored from those projects. CI pins exact conformance counts and follows one hard rule: `squonk` must not accept SQL that the reference engine rejects. Pins move only after fresh measurements.\n\nAdditional coverage includes:\n\n* property-based tests\n* cross-dialect fuzzing\n* round-trip checks requiring parse, render, and reparse operations to produce the same structure\n\nDialects without a full engine oracle—such as Snowflake, Databricks, and Redshift—use deliberately conservative presets. QuiltDB is held to its frozen parser-verdict corpus and curated reject fixture. Every enabled feature must cite an engine, standard, or first-party contract rather than rely on optimistic guesswork.\n\nPartial or comparison-based oracles are labelled clearly. ClickHouse is checked with `clickhouse local`, while BigQuery is cross-checked against sqlglot.\n\nEach dialect’s support tier and source of truth are listed in [docs/support-tiers.md](./docs/support-tiers.md).\n\n## Performance\n\nThe publication benchmark measures one operation: parsing the same frozen SQL into a\ncomplete AST through each tool's public API. Results are compared within an ecosystem so\nlanguage-runtime costs are not mistaken for parser-core differences.\n\n\u003cimg\n  src=\"./docs/assets/full-ast-throughput.png\"\n  alt=\"Horizontal bars show actual full-AST throughput in MiB per second for Squonk and its direct Rust, Python, and Node peers. Squonk reaches 35.36, 27.70, and 20.96 MiB per second through those APIs.\"\n  width=\"1200\"\n/\u003e\n\nOn the controlled Linux snapshot, Squonk delivered **2.33×**, **43.43×**, and **13.63×**\nthe full-AST throughput of its direct Rust, Python, and Node peers, respectively. The binding\nmeasurements exercise the public document APIs, including their native object construction;\nthey are product measurements rather than Rust-core-only results. The chart uses actual MiB/s\non its shared axis; multiplier annotations are comparisons only within each ecosystem pair.\n\nWhy this metric is useful, which comparisons are valid, cold-start and retained-memory\nresults, uncertainty, limitations, and the raw observations are documented in\n[docs/performance.md](./docs/performance.md).\n\n## Feature flags\n\nThe default build includes only ANSI and depends on `thin-vec`. Everything else is opt-in:\n\n* `postgres`, `mysql`, `sqlite`, `duckdb`, `quiltdb` — stable, conformance-gated dialects\n* `bigquery`, `clickhouse`, `databricks`, `hive`, `mssql`, `redshift`, `snowflake` — documentation-backed or partially verified presets; see [support tiers](./docs/support-tiers.md)\n* `lenient` — a permissive union dialect for parse-almost-anything workloads\n* `full` — all dialect features listed above\n* `serde` — AST serialization and deserialization\n* `serde-serialize` — serialization only\n* `serde-deserialize` — deserialization only\n\n## Platform support\n\nThe `cargo add`, `pip install`, and `npm install` examples are intentionally operating-system neutral.\n\nActual support levels—fully tested, compile-verified, or best effort—are defined in [docs/platform-support.md](./docs/platform-support.md). Every promised tier has a corresponding CI check.\n\nThe squonk may be elusive. Platform guarantees should not be.\n\n## Safety and MSRV\n\nThe AST crate, `squonk-ast`, declares:\n\n```rust\n#![forbid(unsafe_code)]\n```\n\nThis is a compiler-enforced rule that cannot be overridden within the crate.\n\nThe parser crate and the rest of the workspace use:\n\n```rust\n#![deny(unsafe_code)]\n```\n\nThe workspace contains no unsafe code today. Using `deny` outside the AST crate leaves room for a narrow, reviewed `#[allow(unsafe_code)]` block if a future performance-critical path genuinely requires one.\n\nThe default build’s sole dependency, `thin-vec`, is the only vetted crate containing internal `unsafe`.\n\nThe minimum supported Rust version is **1.85**. It is declared through `rust-version` in the workspace manifest and verified in CI, preventing accidental version drift.\n\n## Contributing\n\nWe welcome focused issues and pull requests. Please read [CONTRIBUTING.md](CONTRIBUTING.md)\nbefore starting a substantial change, or contact us at\n[opensource@moderately.ai](mailto:opensource@moderately.ai).\n\nSend security-related reports directly to [security@moderately.ai](mailto:security@moderately.ai).\n\nAll project interaction is governed by our [Code of Conduct](CODE_OF_CONDUCT.md). Conduct concerns may be reported to [opensource@moderately.ai](mailto:opensource@moderately.ai).\n\nBefore requesting a new dialect or syntax feature, read the [dialect and syntax request policy](docs/dialect-requests.md). It explains the initial support tier for new dialects and the engine or documentation evidence required for a request.\n\nFor everything else:\n\n* [SUPPORT.md](SUPPORT.md) explains where questions, bugs, and reports belong.\n* [SECURITY.md](SECURITY.md) explains how to report vulnerabilities.\n\n## Sponsors\n\n`squonk` is maintained and sponsored by Moderately AI Inc.\n\nTo discuss sponsorship, contact [opensource@moderately.ai](mailto:opensource@moderately.ai).\n\n## Acknowledgements\n\nNo parser appears from nowhere. This project builds on prior work and draws much of its test surface from the engines themselves:\n\n* [`datafusion-sqlparser-rs`](https://github.com/apache/datafusion-sqlparser-rs) — the primary prior art, a source of lessons, and the benchmark reference\n* [`libpg_query`](https://github.com/pganalyze/libpg_query), through [`pg_query.rs`](https://github.com/pganalyze/pg_query.rs) — PostgreSQL’s extracted parser, our PostgreSQL differential oracle, and our first-party performance comparison\n* [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/), [SQLite](https://sqlite.org/), and [DuckDB](https://duckdb.org/) — their engines act as conformance oracles, and their regression suites provide the corpora we vendor and test against\n\n## License\n\nLicensed under the [MIT License](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoderately-ai%2Fsquonk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoderately-ai%2Fsquonk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoderately-ai%2Fsquonk/lists"}