{"id":13710875,"url":"https://github.com/uber/queryparser","last_synced_at":"2025-05-16T14:05:42.387Z","repository":{"id":37444240,"uuid":"88126172","full_name":"uber/queryparser","owner":"uber","description":"Parsing and analysis of Vertica, Hive, and Presto SQL.","archived":false,"fork":false,"pushed_at":"2022-02-16T11:16:33.000Z","size":410,"stargazers_count":1080,"open_issues_count":23,"forks_count":145,"subscribers_count":56,"default_branch":"master","last_synced_at":"2025-04-19T15:58:13.570Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uber.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-13T04:51:29.000Z","updated_at":"2025-04-09T20:35:12.000Z","dependencies_parsed_at":"2022-08-03T03:46:11.730Z","dependency_job_id":null,"html_url":"https://github.com/uber/queryparser","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fqueryparser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fqueryparser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fqueryparser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fqueryparser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uber","download_url":"https://codeload.github.com/uber/queryparser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254544146,"owners_count":22088807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T23:01:01.703Z","updated_at":"2025-05-16T14:05:42.344Z","avatar_url":"https://github.com/uber.png","language":"Haskell","readme":"# queryparser\n\nParsing and analysis of Vertica, Hive, and Presto SQL.\n\n\n## Gentle introduction\n\nQueryparser supports parsing for three sql-dialects (Vertica, Hive, and\nPresto).\n\nEach dialect implements its own tokenization and parsing logic. There is a\nsingle abstract syntax tree (AST) for representing queries of any dialect. This\nAST is defined in Database/Sql/Type.hs and Database/Sql/Type/Query.hs.\n\nThe parsing logic produces an AST with table and column identifiers that are\n\"raw\" or optionally qualified. Frequently, it is desirable to convert the AST\nover raw names to an AST over resolved names, where identifiers are fully\nqualified. This transformation is called \"name resolution\" or simply\n\"resolution\". It requires as input the full list of columns in every table and\nthe full list of tables in every schema, otherwise known as \"catalog\ninformation\".\n\nAn example of resolution:\n\n * `SELECT                  column_c FROM          table_t` is a query with raw names\n * `SELECT schema_s.table_t.column_c FROM schema_s.table_t` is the same query with resolved names\n\nVarious utility functions (\"analyses\") have been defined for ASTs over resolved\nnames, such as:\n\n * what tables appear in the query?\n * what columns appear in the query, per clause?\n * what is the table/column lineage of a query?\n * what sets of columns does a query compare for equality?\n\nThe analyses are the main value-add of this library.\n\n\n## Demo\n\nEnough talk, let's show what it can do!\n\n    $ stack ghci\n    ...\n    ...\u003e :set prompt \"\u003e \"\n    \u003e import Demo\n    \u003e -- for the purposes of our demo, we have two tables: foo with columns a,b,c and bar with columns x,y,z\n    \u003e demoAllAnalyses \"SELECT * FROM foo\" -- note that the SELECT * expands to a,b,c\n    Tables accessed:\n        public.foo\n    Columns accessed by clause:\n        public.foo.a\tSELECT\n        public.foo.b\tSELECT\n        public.foo.c\tSELECT\n    Joins:\n        no joins\n    Table lineage:\n        no tables modified\n    \u003e demoAllAnalyses \"SELECT * FROM bar\" -- and here the SELECT * expands to x,y,z\n    Tables accessed:\n        public.bar\n    Columns accessed by clause:\n        public.bar.x\tSELECT\n        public.bar.y\tSELECT\n        public.bar.z\tSELECT\n    Joins:\n        no joins\n    Table lineage:\n        no tables modified\n    \u003e demoAllAnalyses \"SELECT x, count(1) FROM foo JOIN bar ON foo.a = bar.y WHERE z IS NOT NULL GROUP BY 1 ORDER BY 2 DESC, b\"\n    Tables accessed:\n        public.bar\n        public.foo\n    Columns accessed by clause:\n        public.bar.x\tGROUPBY\n        public.bar.x\tSELECT\n        public.bar.y\tJOIN\n        public.bar.z\tWHERE\n        public.foo.a\tJOIN\n        public.foo.b\tORDER\n    Joins:\n        public.bar.y \u003c-\u003e public.foo.a\n    Table lineage:\n        no tables modified\n    \u003e -- let's play with some queries that modify table-data!\n    \u003e demoTableLineage \"INSERT INTO foo SELECT * FROM bar\"\n    public.foo after the query depends on public.bar, public.foo before the query\n    \u003e demoTableLineage \"TRUNCATE TABLE foo\"\n    public.foo no longer has data\n    \u003e demoTableLineage \"ALTER TABLE bar, foo RENAME TO baz, bar\"\n    public.bar after the query depends on public.foo before the query\n    public.baz after the query depends on public.bar before the query\n    public.foo no longer has data\n    \u003e -- let's explore a few subtler behaviors of the \"joins\" analysis (admittedly, something of a misnomer)\n    \u003e demoJoins \"SELECT * FROM foo JOIN bar ON a=x AND b+c = y+z\"\n    public.bar.x \u003c-\u003e public.foo.a\n    public.bar.y \u003c-\u003e public.foo.b\n    public.bar.y \u003c-\u003e public.foo.c\n    public.bar.z \u003c-\u003e public.foo.b\n    public.bar.z \u003c-\u003e public.foo.c\n    \u003e demoJoins \"SELECT a FROM foo UNION SELECT x FROM bar\"\n    public.bar.x \u003c-\u003e public.foo.a\n\nSpin up your own ghci and paste in your own queries!\n\n\n## Requirements\n\nTo build, you need:\n\n- stack\n- docker (recommended)\n\n### OS X\n\nYou can install stack on OS X with brew:\n\n    brew install ghc haskell-stack\n\nTo install docker, use the default installer found on https://docs.docker.com/mac/\n\nTo allow stack to see the docker daemon, add `eval \"$(docker-machine\nenv default)\"` to your bashrc or equivalent. (This is following https://docs.docker.com/machine/reference/env/)\n\n### Linux \u0026 Other\n\nFollow the directions at https://github.com/commercialhaskell/stack#how-to-install\n\n\n## Installation\n\nOnce you've got what you need, check out the repo and change to the directory.\n\nStack will download other dependencies for you:\n\n    stack setup\n\n\nNow you can build the package with:\n\n    stack build\n\n\nOr run the tests with:\n\n    stack test\n\n\nOr run the benchmarks with:\n\n    stack bench\n\n\nOr pull things up in ghci with:\n\n    stack ghci\n\n## Contributing\n\nIf you'd like to contribute to the repo, use the above installation instructions to get started.\n\nWhen you're ready, make sure that the code compiles with the `Development` flag, i.e.:\n\n    stack build --flag queryparser:development\n\n## Use\n\nMostly it boils down to this function:\n\n    parse :: Text -\u003e Either ParseError SqlQuery\n\nTo parse some sql from the repl,\n\n    parse \"SELECT 1;\"\n\n## Areas of future interest\n\nThere is substantial room for future work in Queryparser. For more details, see\n[Areas of future interest](FUTURE.md).\n","funding_links":[],"categories":["Tools"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber%2Fqueryparser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuber%2Fqueryparser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber%2Fqueryparser/lists"}