{"id":24698589,"url":"https://github.com/faradayio/joinery","last_synced_at":"2025-10-09T06:30:55.323Z","repository":{"id":208435250,"uuid":"701094312","full_name":"faradayio/joinery","owner":"faradayio","description":"Safe SQL transpiler, written in Rust","archived":false,"fork":false,"pushed_at":"2024-12-09T00:43:21.000Z","size":778,"stargazers_count":10,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-28T11:21:48.683Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/faradayio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-05T22:50:50.000Z","updated_at":"2025-02-01T19:00:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"bfc47c80-2770-4f9c-801b-431d402a35bc","html_url":"https://github.com/faradayio/joinery","commit_stats":null,"previous_names":["faradayio/joinery"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/faradayio/joinery","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faradayio%2Fjoinery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faradayio%2Fjoinery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faradayio%2Fjoinery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faradayio%2Fjoinery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/faradayio","download_url":"https://codeload.github.com/faradayio/joinery/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faradayio%2Fjoinery/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000825,"owners_count":26082950,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-27T04:29:24.803Z","updated_at":"2025-10-09T06:30:54.653Z","avatar_url":"https://github.com/faradayio.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `joinery`: Safe SQL transpiler, written in Rust\n\n## What is this?\n\nThis is an experimental tool to transpile (some) SQL code written in BigQuery's \"Standard SQL\" dialect into other dialects. For example, it can transform:\n\n```sql\nSELECT ARRAY(\n    SELECT DISTINCT val / 2\n    FROM UNNEST(arr) AS val\n    WHERE MOD(val, 2) = 0\n) AS arr\nFROM array_select_data;\n```\n\n...into the Trino-compatible SQL:\n\n```sql\nSELECT ARRAY_DISTINCT(\n    TRANSFORM(\n        FILTER(arr, val -\u003e MOD(val, 2) = 0),\n        val -\u003e val / 2\n    )\n) AS arr\nFROM array_select_data\n```\n\nIt even does type inference, which is needed for certain complex transformations like handling `SELECT * EXCEPT c1 FROM t1`! The transformation process makes some effort to preserve whitespace and comments, so the output SQL is still mostly readable.\n\nIf you want to run _your_ production workloads, **you will almost certainly need to contribute code.** In particular, our API coverage is limited. See [`tests/sql/`](./tests/sql/) for examples of what we support.\n\n## Usage\n\n```\n$ joinery --help\nUsage: joinery \u003cCOMMAND\u003e\n\nCommands:\n  parse      Parse SQL from a CSV file containing `id` and `query` columns\n  sql-test   Run SQL tests from a directory\n  transpile  Transpile BigQuery SQL to another dialect\n  help       Print this message or the help of the given subcommand(s)\n\nOptions:\n  -h, --help  Print help\n```\n\n## Status\n\n- Trino has passing unit tests for all our use cases, but probably not yours. Also, there's a difference between \"works with the SQL test suite that _theoretically_ covers the features we support\" and \"works with gnarly production queries that do tricky things with correlated subqueries.\"\n- AWS Athena 3 is basically Trino, except UDFs are different and we don't support them yet. There may also be dialect differences. Not currently tested.\n\n## Design philosophy\n\nIn an _ideal_ world, `joinery` would do one of two things:\n\n1. Translate your SQL into something that runs identically on your target database, or\n2. Report a clear error explaining why it can't.\n\nIn the real world, neither BigQuery's Standard SQL nor any of our target dialects have any kind of formal semantics, and there has been way too much empiricism and guesswork involved. But `joinery` has been designed to approach the ideal over time.\n\n## Installing\n\n```bash\ngit clone https://github.com/faradayio/joinery.git\ncd joinery\ncargo install --path .\n```\n\n## Running\n\n```bash\njoinery --help\njoinery parse example_queries.csv\njoinery sql-test tests/sql/\n```\n\nFor the `sql-test` test format, see the [test format docs][tests].\n\n[tests]: ./tests/sql/README.md\n\n## Supported databases\n\n### SQLite3\n\nThis is the default. You don't need to do anything. It's not very useful, and will either be removed or replaced with DuckDB.\n\n### Trino\n\nTo run under Trino, you may want to load the plugin, as described in [`java/trino-plugin/README.md`](./java/trino-plugin/README.md). For local testing, make sure you `curl` and [`asdf`](https://asdf-vm.com/) installed, and then install `just`:\n\n```bash\ncargo install just\n```\n\nThen you can run:\n\n```bash\njust docker-run-trino\n```\n\nFrom here, you can run the unit tests:\n\n```bash\njust check-trino\n```\n\nThen you need to start a Trino shell:\n\n```bash\njust trino-shell\n```\n\nFor more details on all these commands, see the [`Justfile`](./Justfile).\n\n## Developing\n\nSee [ARCHITECTURE.md](./ARCHITECTURE.md) for an overview of the codebase.\n\n## Other projects of interest\n\nIf you're interested in running analytic SQL queries across multiple databases, you may also be interested in:\n\n- [PRQL](https://prql-lang.org/). There are a lot of languages out there that compile to SQL, but this one is my favorite. It reads as a pipeline from top to bottom, the features are clean and orthogonal, and it handles window functions. If you're starting a greenfield project, definitely take a look.\n- [Logica](https://logica.dev/). This is probably the most mature Datalog-to-SQL compiler, with a particular focus on BigQuery.\n- [`sqlglot`](https://github.com/tobymao/sqlglot). Transform between many different SQL dialects. Much better feature coverage than we have, though it may generate incorrect SQL in tricky cases. If you're planning on adjusting your translated queries by hand, or if you need to support a wide variety of dialects, this is probably a better choice than `joinery`.\n- [`dbt-core`](https://github.com/dbt-labs/dbt-core).\n- [BigQuery Emulator](https://github.com/goccy/bigquery-emulator). A local emulator for BigQuery. This supports a larger fraction of BigQuery features than we do.\n\n## Corporate support\n\njoinery is open-sourced by [Faraday](https://faraday.ai)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaradayio%2Fjoinery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffaradayio%2Fjoinery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaradayio%2Fjoinery/lists"}