{"id":29346042,"url":"https://github.com/sqlparser-rs/sqlparser-rs","last_synced_at":"2025-07-08T16:02:40.508Z","repository":{"id":37042766,"uuid":"120538653","full_name":"apache/datafusion-sqlparser-rs","owner":"apache","description":"Extensible SQL Lexer and Parser for Rust","archived":false,"fork":false,"pushed_at":"2025-06-30T15:51:55.000Z","size":7602,"stargazers_count":3113,"open_issues_count":203,"forks_count":618,"subscribers_count":42,"default_branch":"main","last_synced_at":"2025-06-30T16:48:39.739Z","etag":null,"topics":["big-data","rust","sql"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.TXT","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-06T23:58:43.000Z","updated_at":"2025-06-30T15:52:00.000Z","dependencies_parsed_at":"2022-07-13T15:59:46.589Z","dependency_job_id":"915af2b2-9c1c-4f39-92dc-ef7ab26e5340","html_url":"https://github.com/apache/datafusion-sqlparser-rs","commit_stats":{"total_commits":1313,"total_committers":262,"mean_commits":5.011450381679389,"dds":0.8423457730388424,"last_synced_commit":"c761f0babbeefdc7b2e8fff5bf0e7bb02988ad03"},"previous_names":["apache/datafusion-sqlparser-rs","sqlparser-rs/sqlparser-rs"],"tags_count":86,"template":false,"template_full_name":null,"purl":"pkg:github/apache/datafusion-sqlparser-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fdatafusion-sqlparser-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fdatafusion-sqlparser-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fdatafusion-sqlparser-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fdatafusion-sqlparser-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/datafusion-sqlparser-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fdatafusion-sqlparser-rs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263557250,"owners_count":23480118,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","rust","sql"],"created_at":"2025-07-08T16:01:50.340Z","updated_at":"2025-07-08T16:02:40.501Z","avatar_url":"https://github.com/apache.png","language":"Rust","readme":"\u003c!---\n  Licensed to the Apache Software Foundation (ASF) under one\n  or more contributor license agreements.  See the NOTICE file\n  distributed with this work for additional information\n  regarding copyright ownership.  The ASF licenses this file\n  to you under the Apache License, Version 2.0 (the\n  \"License\"); you may not use this file except in compliance\n  with the License.  You may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing,\n  software distributed under the License is distributed on an\n  \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n  KIND, either express or implied.  See the License for the\n  specific language governing permissions and limitations\n  under the License.\n--\u003e\n\n# Extensible SQL Lexer and Parser for Rust\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Version](https://img.shields.io/crates/v/sqlparser.svg)](https://crates.io/crates/sqlparser)\n[![Build Status](https://github.com/sqlparser-rs/sqlparser-rs/workflows/Rust/badge.svg?branch=main)](https://github.com/sqlparser-rs/sqlparser-rs/actions?query=workflow%3ARust+branch%3Amain)\n[![Coverage Status](https://coveralls.io/repos/github/sqlparser-rs/sqlparser-rs/badge.svg?branch=main)](https://coveralls.io/github/sqlparser-rs/sqlparser-rs?branch=main)\n[![Gitter Chat](https://badges.gitter.im/sqlparser-rs/community.svg)](https://gitter.im/sqlparser-rs/community?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\nThis crate contains a lexer and parser for SQL that conforms with the\n[ANSI/ISO SQL standard][sql-standard] and other dialects. This crate\nis used as a foundation for SQL query engines, vendor-specific\nparsers, and various SQL analysis.\n\n## Example\n\nTo parse a simple `SELECT` statement:\n\n```rust\nuse sqlparser::dialect::GenericDialect;\nuse sqlparser::parser::Parser;\n\nlet sql = \"SELECT a, b, 123, myfunc(b) \\\n           FROM table_1 \\\n           WHERE a \u003e b AND b \u003c 100 \\\n           ORDER BY a DESC, b\";\n\nlet dialect = GenericDialect {}; // or AnsiDialect, or your own dialect ...\n\nlet ast = Parser::parse_sql(\u0026dialect, sql).unwrap();\n\nprintln!(\"AST: {:?}\", ast);\n```\n\nThis outputs\n\n```rust\nAST: [Query(Query { ctes: [], body: Select(Select { distinct: false, projection: [UnnamedExpr(Identifier(\"a\")), UnnamedExpr(Identifier(\"b\")), UnnamedExpr(Value(Long(123))), UnnamedExpr(Function(Function { name:ObjectName([Identifier(Ident { value: \"myfunc\", quote_style: None })]), args: [Identifier(\"b\")], filter: None, over: None, distinct: false }))], from: [TableWithJoins { relation: Table { name: ObjectName([Identifier(Ident { value: \"table_1\", quote_style: None })]), alias: None, args: [], with_hints: [] }, joins: [] }], selection: Some(BinaryOp { left: BinaryOp { left: Identifier(\"a\"), op: Gt, right: Identifier(\"b\") }, op: And, right: BinaryOp { left: Identifier(\"b\"), op: Lt, right: Value(Long(100)) } }), group_by: [], having: None }), order_by: [OrderByExpr { expr: Identifier(\"a\"), asc: Some(false) }, OrderByExpr { expr: Identifier(\"b\"), asc: None }], limit: None, offset: None, fetch: None })]\n```\n\n\n## Features\n\nThe following optional [crate  features](https://doc.rust-lang.org/cargo/reference/features.html) are available:\n\n* `serde`: Adds [Serde](https://serde.rs/) support by implementing  `Serialize` and `Deserialize` for all AST nodes.\n* `visitor`: Adds a `Visitor` capable of recursively walking the AST tree.\n* `recursive-protection` (enabled by default), uses [recursive](https://docs.rs/recursive/latest/recursive/) for stack overflow protection. \n\n## Syntax vs Semantics\n\nThis crate provides only a syntax parser, and tries to avoid applying\nany SQL semantics, and accepts queries that specific databases would\nreject, even when using that Database's specific `Dialect`. For\nexample, `CREATE TABLE(x int, x int)` is accepted by this crate, even\nthough most SQL engines will reject this statement due to the repeated\ncolumn name `x`.\n\nThis crate avoids semantic analysis because it varies drastically\nbetween dialects and implementations. If you want to do semantic\nanalysis, feel free to use this project as a base.\n\n## Preserves Syntax Round Trip \n\nThis crate allows users to recover the original SQL text (with comments removed,\nnormalized whitespace and keyword capitalization), which is useful for tools\nthat analyze and manipulate SQL.\n\nThis means that other than comments, whitespace and the capitalization of\nkeywords, the following should hold true for all SQL:\n\n```rust\n// Parse SQL\nlet sql = \"SELECT 'hello'\";\nlet ast = Parser::parse_sql(\u0026GenericDialect, sql).unwrap();\n\n// The original SQL text can be generated from the AST\nassert_eq!(ast[0].to_string(), sql);\n\n// The SQL can also be pretty-printed with newlines and indentation\nassert_eq!(format!(\"{:#}\", ast[0]), \"SELECT\\n  'hello'\");\n```\n\nThere are still some cases in this crate where different SQL with seemingly\nsimilar semantics are represented with the same AST. We welcome PRs to fix such\nissues and distinguish different syntaxes in the AST.\n\n\n## Source Locations (Work in Progress)\n\nThis crate allows recovering source locations from AST nodes via the [Spanned]\ntrait, which can be used for advanced diagnostics tooling. Note that this\nfeature is a work in progress and many nodes report missing or inaccurate spans.\nPlease see [this ticket] for information on how to contribute missing\nimprovements.\n\n[Spanned]: https://docs.rs/sqlparser/latest/sqlparser/ast/trait.Spanned.html\n[this ticket]: https://github.com/apache/datafusion-sqlparser-rs/issues/1548\n\n```rust\n// Parse SQL\nlet ast = Parser::parse_sql(\u0026GenericDialect, \"SELECT A FROM B\").unwrap();\n\n// The source span can be retrieved with start and end locations\nassert_eq!(ast[0].span(), Span {\n  start: Location::of(1, 1),\n  end: Location::of(1, 16),\n});\n```\n\n## SQL compliance\n\nSQL was first standardized in 1987, and revisions of the standard have been\npublished regularly since. Most revisions have added significant new features to\nthe language, and as a result no database claims to support the full breadth of\nfeatures. This parser currently supports most of the SQL-92 syntax, plus some\nsyntax from newer versions that have been explicitly requested, plus various\nother dialect-specific syntax. Whenever possible, the [online SQL:2016\ngrammar][sql-2016-grammar] is used to guide what syntax to accept.\n\nUnfortunately, stating anything more specific about compliance is difficult.\nThere is no publicly available test suite that can assess compliance\nautomatically, and doing so manually would strain the project's limited\nresources. Still, we are interested in eventually supporting the full SQL\ndialect, and we are slowly building out our own test suite.\n\nIf you are assessing whether this project will be suitable for your needs,\nyou'll likely need to experimentally verify whether it supports the subset of\nSQL that you need. Please file issues about any unsupported queries that you\ndiscover. Doing so helps us prioritize support for the portions of the standard\nthat are actually used. Note that if you urgently need support for a feature,\nyou will likely need to write the implementation yourself. See the\n[Contributing](#Contributing) section for details.\n\n## Command line\n\nThis crate contains a CLI program that can parse a file and dump the results as JSON:\n```\n$ cargo run --features json_example --example cli FILENAME.sql [--dialectname]\n```\n\n## Users\n\nThis parser is currently being used by the [DataFusion] query engine, [LocustDB],\n[Ballista], [GlueSQL], [Opteryx], [Polars], [PRQL], [Qrlew], [JumpWire], [ParadeDB], [CipherStash Proxy],\nand [GreptimeDB].\n\nIf your project is using sqlparser-rs feel free to make a PR to add it\nto this list.\n\n## Design\n\nThe core expression parser uses the [Pratt Parser] design, which is a top-down\noperator-precedence (TDOP) parser, while the surrounding SQL statement parser is\na traditional, hand-written recursive descent parser. Eli Bendersky has a good\n[tutorial on TDOP parsers][tdop-tutorial], if you are interested in learning\nmore about the technique.\n\nWe are a fan of this design pattern over parser generators for the following\nreasons:\n\n- Code is simple to write and can be concise and elegant\n- Performance is generally better than code generated by parser generators\n- Debugging is much easier with hand-written code\n- It is far easier to extend and make dialect-specific extensions\n  compared to using a parser generator\n\n### Supporting custom SQL dialects\n\nThis is a work in progress, but we have some notes on [writing a custom SQL\nparser](docs/custom_sql_parser.md).\n\n## Contributing\n\nContributions are highly encouraged! However, the bandwidth we have to\nmaintain this crate is limited. Please read the following sections carefully.\n\n### New Syntax\n\nThe most commonly accepted PRs add support for or fix a bug in a feature in the\nSQL standard, or a popular RDBMS, such as Microsoft SQL\nServer or PostgreSQL, will likely be accepted after a brief\nreview.  Any SQL feature that is dialect specific should be parsed by *both* the relevant [`Dialect`] \nas well as [`GenericDialect`].\n\n### Major API Changes\n\nThe current maintainers do not plan for any substantial changes to\nthis crate's API. PRs proposing major refactors\nare not likely to be accepted.\n\n### Testing\n\nWhile we hope to review PRs in a reasonably\ntimely fashion, it may take a week or more. In order to speed the process,\nplease make sure the PR passes all CI checks, and includes tests\ndemonstrating your code works as intended (and to avoid\nregressions). Remember to also test error paths.\n\nPRs without tests will not be reviewed or merged.  Since the CI\nensures that `cargo test`, `cargo fmt`, and `cargo clippy`, pass you\nshould likely to run all three commands locally before submitting\nyour PR.\n\n### Filing Issues\n\nIf you are unable to submit a patch, feel free to file an issue instead. Please\ntry to include:\n\n  * some representative examples of the syntax you wish to support or fix;\n  * the relevant bits of the [SQL grammar][sql-2016-grammar], if the syntax is\n    part of SQL:2016; and\n  * links to documentation for the feature for a few of the most popular\n    databases that support it.\n\nUnfortunately, if you need support for a feature, you will likely need to implement\nit yourself, or file a well enough described ticket that another member of the community can do so.\nOur goal as maintainers is to facilitate the integration\nof various features from various contributors, but not to provide the\nimplementations ourselves, as we simply don't have the resources.\n\n### Benchmarking\n\nThere are several micro benchmarks in the `sqlparser_bench` directory. \nYou can run them with:\n\n```\ngit checkout main\ncd sqlparser_bench\ncargo bench -- --save-baseline main\ngit checkout \u003cyour branch\u003e\ncargo bench -- --baseline main\n```\n\nBy adding the `--save-baseline main` and `--baseline main` you can track the\nprogress of your improvements as you continue working on the feature branch.\n\n## Licensing\n\nAll code in this repository is licensed under the [Apache Software License 2.0](LICENSE.txt).\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\nlicensed as above, without any additional terms or conditions.\n\n\n[tdop-tutorial]: https://eli.thegreenplace.net/2010/01/02/top-down-operator-precedence-parsing\n[`cargo fmt`]: https://github.com/rust-lang/rustfmt#on-the-stable-toolchain\n[current issues]: https://github.com/sqlparser-rs/sqlparser-rs/issues\n[DataFusion]: https://github.com/apache/arrow-datafusion\n[LocustDB]: https://github.com/cswinter/LocustDB\n[Ballista]: https://github.com/apache/arrow-ballista\n[GlueSQL]: https://github.com/gluesql/gluesql\n[Opteryx]: https://github.com/mabel-dev/opteryx\n[Polars]: https://pola.rs/\n[PRQL]: https://github.com/PRQL/prql\n[Qrlew]: https://github.com/Qrlew/qrlew\n[JumpWire]: https://github.com/extragoodlabs/jumpwire\n[ParadeDB]: https://github.com/paradedb/paradedb\n[Pratt Parser]: https://tdop.github.io/\n[sql-2016-grammar]: https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html\n[sql-standard]: https://en.wikipedia.org/wiki/ISO/IEC_9075\n[`Dialect`]: https://docs.rs/sqlparser/latest/sqlparser/dialect/trait.Dialect.html\n[`GenericDialect`]: https://docs.rs/sqlparser/latest/sqlparser/dialect/struct.GenericDialect.html\n[CipherStash Proxy]: https://github.com/cipherstash/proxy\n[GreptimeDB]: https://github.com/GreptimeTeam/greptimedb\n","funding_links":[],"categories":["Rust","SQL"],"sub_categories":["PHP - Docblock Parser"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsqlparser-rs%2Fsqlparser-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsqlparser-rs%2Fsqlparser-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsqlparser-rs%2Fsqlparser-rs/lists"}