{"id":31768324,"url":"https://github.com/wseaton/sqloxide","last_synced_at":"2025-10-21T20:39:12.812Z","repository":{"id":38099578,"uuid":"320693648","full_name":"wseaton/sqloxide","owner":"wseaton","description":"Python bindings for sqlparser-rs","archived":false,"fork":false,"pushed_at":"2025-05-17T17:57:16.000Z","size":245,"stargazers_count":196,"open_issues_count":7,"forks_count":23,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-09-27T20:01:59.674Z","etag":null,"topics":["benchmarks","parser","python","rust","sql"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wseaton.png","metadata":{"files":{"readme":"readme.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-11T22:03:01.000Z","updated_at":"2025-09-17T10:32:32.000Z","dependencies_parsed_at":"2023-02-17T04:16:13.255Z","dependency_job_id":"8769bb37-8f5b-42e8-b9b6-d8b6a57c72ec","html_url":"https://github.com/wseaton/sqloxide","commit_stats":{"total_commits":123,"total_committers":8,"mean_commits":15.375,"dds":0.7154471544715447,"last_synced_commit":"1c2362446b792e944180ff8052fa1c0f4e58d375"},"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/wseaton/sqloxide","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wseaton%2Fsqloxide","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wseaton%2Fsqloxide/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wseaton%2Fsqloxide/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wseaton%2Fsqloxide/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wseaton","download_url":"https://codeload.github.com/wseaton/sqloxide/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wseaton%2Fsqloxide/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002431,"owners_count":26083381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarks","parser","python","rust","sql"],"created_at":"2025-10-10T02:00:40.463Z","updated_at":"2025-10-10T02:02:03.712Z","avatar_url":"https://github.com/wseaton.png","language":"Python","readme":"# sqloxide\n\n[![GitHub Workflow Status (with event)](https://img.shields.io/github/actions/workflow/status/wseaton/sqloxide/ci.yml)](https://github.com/wseaton/sqloxide/actions/workflows/ci.yml)[![Downloads](https://static.pepy.tech/badge/sqloxide)](https://pepy.tech/project/sqloxide)\n\n---\n\n- [sqloxide](#sqloxide)\n  - [Installation](#installation)\n  - [Usage](#usage)\n    - [Parsing](#parsing)\n    - [AST Rewrites](#ast-rewrites)\n  - [Benchmarks](#benchmarks)\n  - [Example](#example)\n  - [Develop](#develop)\n\n---\n\n`sqloxide` wraps rust bindings for [sqlparser-rs](https://github.com/ballista-compute/sqlparser-rs) into a python package using `pyO3`.\n\nThe original goal of this project was to have a very fast, efficient, and accurate SQL parser I could use for building data lineage graphs across large code bases (think hundreds of auto-generated .sql files). Most existing sql parsing approaches for python are either very slow or not accurate (especially in regards to deeply nested queries, sub-selects and/or table aliases). Looking to the rust community for support, I found the excellent `sqlparser-rs` crate which is quite easy to wrap in python code.\n\n## Installation\n\nThe project provides `manylinux2014` wheels on pypi so it should be compatible with most linux distributions. Native wheels are also now available for OSX and Windows.\n\nTo install from pypi:\n\n```sh\npip install sqloxide\n```\n\n## Usage\n\n### Parsing\n\nParsing a SQL query is relatively straight forward:\n\n```python\nfrom sqloxide import parse_sql\n\nsql = \"\"\"\nSELECT employee.first_name, employee.last_name,\n       call.start_time, call.end_time, call_outcome.outcome_text\nFROM employee\nINNER JOIN call ON call.employee_id = employee.id\nINNER JOIN call_outcome ON call.call_outcome_id = call_outcome.id\nORDER BY call.start_time ASC;\n\"\"\"\n\noutput = parse_sql(sql=sql, dialect='ansi')\n\nprint(output)\n\n\u003e\u003e\u003e [\n  {\n    \"Query\": {\n      \"ctes\": [],\n      \"body\": {\n        \"Select\": {\n          \"distinct\": false,\n          \"top\": null,\n          \"projection\": [\n            {\n              \"UnnamedExpr\": {\n                \"CompoundIdentifier\": [\n                  {\n                    \"value\": \"employee\",\n                    \"quote_style\": null\n                  },\n                  {\n                    \"value\": \"first_name\",\n                    \"quote_style\": null\n                  }\n                ]\n              }\n            },\n            {\n              \"UnnamedExpr\": {\n                \"CompoundIdentifier\": [\n                  {\n                    \"value\": \"employee\",\n                    \"quote_style\": null\n                  },\n                  {\n                    \"value\": \"last_name\",\n                    \"quote_style\": null\n                  }\n                ]\n              }\n            },\n            {\n              \"UnnamedExpr\": {\n                \"CompoundIdentifier\": [\n                  {\n                    \"value\": \"call\",\n                    \"quote_style\": null\n                  },\n                  {\n                    \"value\": \"start_time\",\n                    \"quote_style\": null\n                  }\n                ]\n              }\n            },\n            { # OUTPUT TRUNCATED\n```\n\nNote that you get back what looks like a JSON document but in actual python types, this is a typed AST that matches the sqlparser-rs AST schema.\n\nWe can convert this AST back into a SQL query by running:\n\n```python\nfrom sqloxide import restore_ast\n\nquery = restore_ast(ast=output)\nprint(query)\n```\n\nThis reconstruction is helpful if you want to make manual edits to the AST in python.\n\n### AST Rewrites\n\nIf you want a more structured approach to AST edits, we also expose APIs that allow you to use the visitor pattern to make query modifications.\n\nHere is an example for mutating a subset of the expressions in the query to be SHOUTING UPPERCASE:\n\n```python\nfrom sqloxide import parse_sql, mutate_expressions\n\nsql = \"SELECT something from somewhere where something = 1 and something_else = 2\"\n\ndef func(x):\n    if \"CompoundIdentifier\" in x.keys():\n        for y in x[\"CompoundIdentifier\"]:\n            y[\"value\"] = y[\"value\"].upper()\n    return x\n\nast = parse_sql(sql=sql, dialect=\"ansi\")\nresult = mutate_expressions(parsed_query=ast, func=func)\nprint(result)\n---\n\u003e\u003e\u003e ['SELECT something FROM somewhere WHERE something = 1 AND something_else = 2']\n```\n\nWhat if you needed to make a structured edit to the table name in the above query? There is also an API for that as well:\n\n```python\nfrom sqloxide import parse_sql, mutate_relations\n\ndef func(x):\n    return x.replace(\"somewhere\", \"anywhere\")\nresult = mutate_relations(parsed_query=ast, func=func)\nprint(result)\n---\n\u003e\u003e\u003e ['SELECT something FROM anywhere WHERE something = 1 AND something_else = 2']\n\n```\n\nThese features combined allow for powerful semantic rewrites of queries, if you have any examples you'd like to share please contribue back to the `examples/` folder!\n\n## Benchmarks\n\nWe run 4 benchmarks, comparing to some python native sql parsing libraries:\n\n- `test_sqloxide` - parse query and get a python object back from rust\n- `test_sqlparser` - testing [sqlparse](https://pypi.org/project/sqlparse/), query -\u003e AST\n- `test_mozsqlparser` - testing [moz-sql-parser](https://pypi.org/project/moz-sql-parser/), full roundtrip as in the docs, query -\u003e JSON\n- `test_sqlglot` - testing [sqlglot](https://github.com/tobymao/sqlglot/), query -\u003e AST\n\nTo run them on your machine:\n\n```\npoetry run pytest tests/benchmark.py\n```\n\n```\n------------------------------------------------------------------------------------------- benchmark: 4 tests -------------------------------------------------------------------------------------------\nName (time in us)            Min                    Max                  Mean              StdDev                Median                 IQR            Outliers          OPS            Rounds  Iterations\n----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\ntest_sqloxide            29.6800 (1.0)          50.4300 (1.0)         30.6219 (1.0)        0.7367 (1.0)         30.4900 (1.0)        0.2390 (1.0)       527;716  32,656.3811 (1.0)        9099           1\ntest_sqlglot            365.8420 (12.33)       692.8950 (13.74)      377.2422 (12.32)     11.7692 (15.98)      375.7825 (12.32)      4.3145 (18.05)       62;97   2,650.8168 (0.08)       2260           1\ntest_sqlparser        1,577.7720 (53.16)     9,751.9699 (193.38)   1,651.5547 (53.93)    355.5511 (482.64)   1,620.7315 (53.16)     30.9200 (129.37)       3;60     605.4901 (0.02)        538           1\ntest_mozsqlparser     2,793.8400 (94.13)    12,358.7790 (245.07)   3,091.8519 (100.97)   960.4173 (\u003e1000.0)  2,937.6310 (96.35)    243.3220 (\u003e1000.0)       4;4     323.4308 (0.01)        316           1\n----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n```\n\n## Example\n\nThe `depgraph` example reads a bunch of `.sql` files from disk using glob, and builds a dependency graph of all of the objects using graphviz.\n\n```\npoetry run python ./examples/depgraph.py --path {path/to/folder/with/queries}\n```\n\n## Develop\n\n1) Install `rustup`\n\n2) `poetry install` will automatically create the venv, compile the package and install it into the venv via the build script.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwseaton%2Fsqloxide","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwseaton%2Fsqloxide","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwseaton%2Fsqloxide/lists"}