{"id":18646763,"url":"https://github.com/metabase/macaw","last_synced_at":"2025-04-11T12:32:01.359Z","repository":{"id":222138815,"uuid":"756345198","full_name":"metabase/macaw","owner":"metabase","description":"A Clojure wrapper for JSqlParser 🦜","archived":false,"fork":false,"pushed_at":"2024-12-11T10:59:34.000Z","size":1526,"stargazers_count":13,"open_issues_count":11,"forks_count":1,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-10T05:49:37.014Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metabase.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-12T13:41:03.000Z","updated_at":"2024-12-11T10:59:39.000Z","dependencies_parsed_at":"2024-02-12T16:15:42.660Z","dependency_job_id":"64fdff97-6862-43cc-9b43-26c83b3f81f7","html_url":"https://github.com/metabase/macaw","commit_stats":null,"previous_names":["metabase/macaw"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabase%2Fmacaw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabase%2Fmacaw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabase%2Fmacaw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabase%2Fmacaw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metabase","download_url":"https://codeload.github.com/metabase/macaw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248402009,"owners_count":21097328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T06:22:29.426Z","updated_at":"2025-04-11T12:32:01.011Z","avatar_url":"https://github.com/metabase.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![License](https://img.shields.io/badge/license-Eclipse%20Public%20License-blue.svg?style=for-the-badge)](https://raw.githubusercontent.com/metabase/macaw/master/LICENSE)\n[![GitHub last commit](https://img.shields.io/github/last-commit/metabase/second-date?style=for-the-badge)](https://github.com/metabase/macaw/commits/)\n\n[![Clojars Project](https://clojars.org/io.github.metabase/macaw/latest-version.svg)](https://clojars.org/io.github.metabase/macaw)\n\n\u003cimg src=\"./assets/logo.png\" width=\"400px\" alt=\"Macaw logo\" /\u003e\n\n# Macaw\n\nMacaw is a limited Clojure wrapper for [JSqlParser](https://github.com/JSQLParser/JSqlParser). Similar to its parrot\nnamesake, it's intelligent, can be taught to speak SQL, and has many colors (supports many dialects).\n\n## Rationale\n\nJSqlParser does great work actually parsing SQL, but its use of the visitor pattern makes Clojure inter-op very\nawkward. Macaw exists to make working with JSqlParser from within Clojure feel more idiomatic and pleasant, letting you\nwalk over a query with custom callbacks and returning persistent data structures.\n\nCurrently it's especially useful for extracting the columns, tables, and side-effecting commands from a SQL string (see\n[Query Parsing](#query-parsing)) and also intelligently renaming the columns and tables used in it (see [Query\nRewriting](#query-rewriting)).\n\n# Preliminaries\n\n## Building\n\nTo build a local JAR, use\n\n```\n./bin/build-jar\n```\n\nThis will create a JAR in the `target` directory.\n\n## Working with the Java files\n\nTo compile the Java files, use\n\n```\n./bin/java-compile\n```\n\nIf you're working on Macaw and make changes to a Java file, you must:\n\n1. Recompile\n2. Restart your Clojure REPL\n\nfor the changes to take effect.\n\n# Usage\n\nFor detailed documentation, refer to [the Marginalia documentation here](https://metabase-dev-docs.github.io/macaw/).\n\n## Query Parsing\n\nFor extracting information from a query, use `parsed-query` to get a parse object and `query-\u003ecomponents` to turn it\ninto something useful. For example:\n\n```clojure\n;; macaw.core\n(-\u003e \"SELECT total FROM orders\"\n    parsed-query\n    query-\u003ecomponents)\n;; =\u003e {:columns           #{{:component {:column \"total\"}, :context [\"SELECT\"]}},\n;;     :has-wildcard?     #{},\n;;     :mutation-commands #{},\n;;     :tables            #{{:component {:table \"orders\"}, :context [\"FROM\" \"SELECT\"]}},\n;;     :table-wildcards   #{}}\n```\n\nThe returned map will always have that general shape as of Macaw 0.1.30. Each of the main keys will always refer to a\nset, and each set will always consist of maps with a `:component` key (described below) and a `:context` key (described\n[farther below](#context)).\n\n### Column and table `:component`s\n\nColumns will have a `:column` key with the name as it appears in the query, and may also have a `:table` or `:schema`\nkey if available.\n\n```clojure\n(-\u003e \"SELECT id, orders.total, public.orders.tax FROM public.orders\"\n    parsed-query\n    query-\u003ecomponents\n    :columns)\n;; =\u003e #{{:component {:column \"id\"},                                       :context [\"SELECT\"]}\n;;      {:component {:column \"total\", :table \"orders\", :schema \"public\"}, :context [\"SELECT\"]}\n;;      {:component {:column \"tax\",   :table \"orders\", :schema \"public\"}, :context [\"SELECT\"]}}\n```\nNote that the schema for `total` was inferred, but neither the table nor the schema for `id` was inferred since there\nare similar queries where it could be ambiguous (e.g., with a JOIN).\n\nMacaw will also resolve aliases sensibly:\n\n```clojure\n(-\u003e \"SELECT o.id, o.total, u.name FROM public.orders o JOIN users u ON u.id = o.user_id\"\n    parsed-query\n    query-\u003ecomponents\n    (select-keys [:columns :tables]))\n;; =\u003e {:columns\n;;     #{{:component {:column \"id\", :table \"orders\", :schema \"public\"}, :context [\"SELECT\"]}\n;;       {:component {:column \"total\", :table \"orders\", :schema \"public\"}, :context [\"SELECT\"]}\n;;       {:component {:column \"name\", :table \"users\"}, :context [\"SELECT\"]}\n;;       {:component {:column \"id\", :table \"users\"}, :context [\"JOIN\" \"SELECT\"]}\n;;       {:component {:column \"user_id\", :table \"orders\", :schema \"public\"}, :context [\"JOIN\" \"SELECT\"]}},\n;;     :tables\n;;     #{{:component {:table \"orders\", :schema \"public\"}, :context [\"FROM\" \"SELECT\"]}\n;;       {:component {:table \"users\"}, :context [\"FROM\" \"JOIN\" \"SELECT\"]}}}\n```\nNote that `:tables` is similar to `:columns`, but only contains the `:table` and (if available anywhere in the query)\n`:schema` keys.\n\n### Wildcard `:component`s\n\nThe `:has-wildcard?` and `:table-wildcards` keys refer to `*`s:\n\n```clojure\n(-\u003e \"SELECT * from orders\"\n    parsed-query\n    query-\u003ecomponents\n    :has-wildcard?)\n;; =\u003e #{{:component true, :context [\"SELECT\"]}}\n```\n\n```clojure\n(-\u003e \"SELECT o.*, u.* FROM public.orders o JOIN users u ON u.id = o.user_id\"\n    parsed-query\n    query-\u003ecomponents\n    :table-wildcards)\n;; =\u003e #{{:component {:table \"users\"}, :context [\"SELECT\"]}\n;;      {:component {:table \"orders\", :schema \"public\"}, :context [\"SELECT\"]}}\n```\n\nThe shape of `:table-wildcards` will be the same as the shape of `:tables`. The `:component` for `has-wildcard?` will\nnever be false.\n\n### Mutation commands\n\nAny commands that could change the state of the database are returned in `:mutation-commands`:\n\n```clojure\n(-\u003e \"DROP TABLE orders\"\n    parsed-query\n    query-\u003ecomponents\n    :mutation-commands)\n;; =\u003e #{{:component \"drop\", :context []}}\n```\n\nThe list of recognized commands as of 0.1.30 is:\n\n```bash\n$ grep MUTATION_COMMAND java/com/metabase/macaw/AstWalker.java | grep -oEi '\".+\"' | sort\n\"alter-sequence\"\n\"alter-session\"\n\"alter-system\"\n\"alter-table\"\n\"alter-view\"\n\"create-function\"\n\"create-index\"\n\"create-schema\"\n\"create-sequence\"\n\"create-synonym\"\n\"create-table\"\n\"create-view\"\n\"delete\"\n\"drop\"\n\"grant\"\n\"insert\"\n\"purge\"\n\"rename-table\"\n\"truncate\"\n\"update\"\n```\n\n### Context\n\nAs can be seen above, every item additional has a `:context` key containing a stack (most-specific first) of the query\ncomponents in which the item was found. The definitive list of components if found in\n`com.metabase.macaw.AstWalker.QueryScopeLabel`, but as of 0.1.30 the list is as follows (and is unlikely to change):\n\n```java\nDELETE,\nELSE,\nFROM,\nGROUP_BY,\nHAVING,\nIF,\nINSERT,\nJOIN,\nSELECT,\nSUB_SELECT,\nUPDATE,\nWHERE;\n```\n\n## Query Rewriting\n\nEditing queries can be done with `replace-names`. It takes two arguments, the query itself (as a string) and a map of\nmaps. The outer keys of the map are `:schemas`, `:tables`, and `:columns`. The inner maps take the form `old-name -\u003e new-name`. For\nexample:\n\n```clojure\n(replace-names \"SELECT p.id, orders.total FROM people p, public.orders;\"\n               {:schemas  {\"public\" \"private\"}\n                :tables   {\"people\" \"users\"}\n                :columns  {{:table \"orders\" :column \"total\"} \"amount\"}})\n;; =\u003e \"SELECT p.id, orders.amount FROM users p, private.orders;\"\n```\n## Error Handling\n\nMacaw makes no effort to recover from errors. Malformed SQL strings will probably raise a\n[JSqlParserException](https://javadoc.io/static/com.github.jsqlparser/jsqlparser/4.9/net/sf/jsqlparser/JSQLParserException.html),\nwhich Macaw will forward on to your code.\n\n# Contributing\n\nExternal contributions are welcome! We require all third-party contributors to sign [our license\nagreement](https://docs.google.com/a/metabase.com/forms/d/e/1FAIpQLSfc9GWyJ3F9U_4NzHLeTblgtog1FKtG3CjLshE4FAAKSdvNoQ/viewform).\nMacaw's philosophy towards third-party contributions is largely similar to [that of\nMetabase's](https://github.com/metabase/metabase/blob/master/docs/developers-guide/contributing.md). We'd especially\nlike to call out that document's YOLO method of PR submission:\n\n\u003e If you come up with something really cool, and want to share it with us, just submit a PR. If it hasn't gone through\n\u003e the above process, we probably won't merge it as is, but if it's compelling, we're more than willing to help you via\n\u003e code review, design review and generally OCD nitpicking so that it fits into the rest of our codebase.\n\nSince Macaw is a *much* simpler project than Metabase (and also doesn't involve a graphical user interface) we should\nbe able to respond to product proposals fairly quickly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetabase%2Fmacaw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetabase%2Fmacaw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetabase%2Fmacaw/lists"}