{"id":20313060,"url":"https://github.com/zombodb/postgres-parser","last_synced_at":"2025-07-24T02:34:06.615Z","repository":{"id":38350104,"uuid":"265639079","full_name":"zombodb/postgres-parser","owner":"zombodb","description":"Postgres' query parser, as a Rust crate!","archived":false,"fork":false,"pushed_at":"2023-01-17T12:01:03.000Z","size":314,"stargazers_count":93,"open_issues_count":10,"forks_count":13,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-06-20T15:18:45.999Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"postgresql","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zombodb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":["eeeebbbbrrrr"]}},"created_at":"2020-05-20T17:29:55.000Z","updated_at":"2025-04-14T03:46:42.000Z","dependencies_parsed_at":"2023-01-30T03:31:09.078Z","dependency_job_id":null,"html_url":"https://github.com/zombodb/postgres-parser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zombodb/postgres-parser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zombodb%2Fpostgres-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zombodb%2Fpostgres-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zombodb%2Fpostgres-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zombodb%2Fpostgres-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zombodb","download_url":"https://codeload.github.com/zombodb/postgres-parser/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zombodb%2Fpostgres-parser/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266785529,"owners_count":23983827,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T18:09:06.502Z","updated_at":"2025-07-24T02:34:06.586Z","avatar_url":"https://github.com/zombodb.png","language":"Rust","funding_links":["https://github.com/sponsors/eeeebbbbrrrr"],"categories":[],"sub_categories":[],"readme":"[![Actions Status](https://github.com/zombodb/postgres-parser/workflows/cargo%20test%20--all/badge.svg)](https://github.com/zombodb/postgres-parser/actions)\n[![crates.io badge](https://img.shields.io/crates/v/postgres-parser.svg)](https://crates.io/crates/postgres-parser)\n[![docs.rs badge](https://docs.rs/postgres-parser/badge.svg)](https://docs.rs/postgres-parser)\n[![Twitter Follow](https://img.shields.io/twitter/follow/zombodb.svg?style=flat)](https://twitter.com/zombodb)  \n\n## postgres-parser\n\n\nThis project is the beginnings of using Postgres v13.0's SQL Parser\n(effectively `gram.y` and the `List *raw_parser(const char *str)` function)\nfrom Rust.\n\nThe way this works is by downloading the Postgres source code, patching\na few of its Makefiles (see [`patches/makefiles-13.0.patch`](patches/makefiles-12.3.patch)),\ncompiling it to LLVM IR, optimizing/assembling that to LLVM bitcode, performing\nlink-time optimization (LTO) to generate a static library containing only the symbols/code\nnecessary to properly use Postgres' `raw_parser()` function, and finally, \nlinking against that library with Rust.\n\nThis is accomplished via a custom [`build.rs`](build.rs) script, which \nshells out to [`build.sh`](build.sh) to perform all the hard work.\n\nAt the end of the process we're left with a `libpostgres_parser.a` archive, which\n`build.rs` instructs cargo to link against.\n\n### Using this Crate\n\nUsing this create is just like any other.  Add it as a dependency to your `Cargo.toml`:\n\n```toml\n[dependencies]\npostgres-parser = \"0.2.2\"\n```\n\nAdditionally, see the [System Requirements](#System+Requirements) section below.\n\nHere's a simple example that outputs a SQL parse tree as JSON.\n\n```rust\nuse postgres_parser::*;\n\nfn main() {\n    let args: Vec\u003cString\u003e = std::env::args().collect();\n    let query_string = args.get(1).expect(\"no arguments\");\n    let parse_list = match parse_query(query_string) {\n        Ok(query) =\u003e query,\n        Err(e) =\u003e {\n            eprintln!(\"{:?}\", e);\n            std::process::exit(1);\n        }\n    };\n\n    let as_json = serde_json::to_string_pretty(\u0026parse_list).expect(\"failed to convert to json\");\n    println!(\"{}\", as_json);\n}\n```\n\n### System Requirements\n\nFor an Ubuntu-based Linux system you'll need:\n\n```shell script\n$ sudo apt-get install clang llvm make curl\n$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n```\n\nFor MacOS you'll need:\n\n```shell script\n$ brew install wget\n$ brew install llvm\n$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n```\n\nAs Linux goes, so far this has been tested on Ubuntu 18.04 with LLVM 6.0.0, and\nUbuntu 20.04 with LLVM 10.0.0.\n\nYou'll also want to make sure the LLVM and clang tools are on your `$PATH`.\nEspecially the `clang`, `opt`, and `llvm-ar` tools.\n\n### Building\n\nBuild this just like any other Rust binary crate:\n\n```shell script\n$ cargo build [--release]\n```\n\nThis will take awhile as again, the build process:\n\n - Downloads Postgres source code\n - Configures Postgres\n - Compiles Postgres (in parallel up to # of your CPUS)\n - Optimizes the resulting LLVM ir into LLVM bitcode\n\nOn my relatively new MacBook Pro 16\", this process takes about 2.5 minutes the first\ntime.  \n\nOn my incredibly old Mac Mini, running Ubuntu 16.04 (yikes!), this process takes about\n25 minutes.  So be patient if you have an older computer.\n\nSubsequent builds (assuming no `cargo clean`) are able to elide all of the above \nsteps as the final `libpostgres_parser.a` archive artifact is cached in the `target/`\ndirectory.\n\n\n### Known Issues\n\n- While the postgres parser supports `UTF8` input, enabling this somehow causes the resulting compiled binary to bloat to nearly \n10 megabytes.  Seemingly, including Postgres' `SetDatabaseEncoding` function causes LLVM's `opt` to fail to\ndo proper global dead code elimination.  This is being investigated.  As such, input must match Postgres' `SQL_ASCII` encoding for now\n\n- Building on MacOS with XCode `\u003e=11.4.0` doesn't work.  This appears to be a problem with these versions\nof XCode.  This is the bug: https://openradar.appspot.com/FB7647406.  This happens while building Postgres.\nAny suggestions for a work around would be greatly appreciated.\n\n- Single-threaded query parsing... Postgres isn't thread safe, so we're required to lock on a Mutex while\nparsing queries.  Which means one-at-a-time.  There may be some things we can do in the future to improve\nthis situation.  The underlying dilemma is around how Postgres allocates memory, and this approach to\nembedding Postgres' parser necessitates it use that system\n\n### Please Help!\n\nWe'd sincerely appreciate the time and effort you spend cloning this repo and at\nleast trying to `cargo test --all`.  If it doesn't work, or if these instructions are bad, \nwe definitely want to know.  We'd like this to be as easy as possible for everyone.\n\nFurthermore, this is v0.0.1.  Please feel free to submit bug reports, feature requests, and\nmost especially Pull Requests.\n\n### Thanks\n\nThanks for checking this out.  [Here's the obligatory GitHub Sponsors link](https://github.com/sponsors/eeeebbbbrrrr).\n  \nIf you like what we're doing and where this is going, your sponsorship will keep us \nmotivated.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzombodb%2Fpostgres-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzombodb%2Fpostgres-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzombodb%2Fpostgres-parser/lists"}