{"id":13569132,"url":"https://github.com/melin/superior-sql-parser","last_synced_at":"2025-08-12T23:02:00.888Z","repository":{"id":43028770,"uuid":"140644913","full_name":"melin/superior-sql-parser","owner":"melin","description":"基于 antlr4 的多种数据库SQL解析器，获取SQL中元数据，可用于数据平台产品中的多个场景：ddl语句提取元数据、sql 权限校验、表级血缘、sql语法校验等场景。支持spark、flink、gauss、starrocks、Oracle、MYSQL、Postgresql，sqlserver,、db2等","archived":false,"fork":false,"pushed_at":"2024-11-04T07:02:08.000Z","size":10512,"stargazers_count":292,"open_issues_count":5,"forks_count":98,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-11-05T01:34:45.555Z","etag":null,"topics":["flink","gauss","lineage","metadata","mysql","parser","postgres","spark","sql","starrocks"],"latest_commit_sha":null,"homepage":"","language":"ANTLR","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/melin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-12T01:28:59.000Z","updated_at":"2024-11-04T08:56:11.000Z","dependencies_parsed_at":"2024-01-14T03:47:54.269Z","dependency_job_id":"5d321609-5692-4caf-b833-e2e9dbe26985","html_url":"https://github.com/melin/superior-sql-parser","commit_stats":null,"previous_names":["melin/dataworker-sql-parser"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melin%2Fsuperior-sql-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melin%2Fsuperior-sql-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melin%2Fsuperior-sql-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/melin%2Fsuperior-sql-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/melin","download_url":"https://codeload.github.com/melin/superior-sql-parser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247128702,"owners_count":20888232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flink","gauss","lineage","metadata","mysql","parser","postgres","spark","sql","starrocks"],"created_at":"2024-08-01T14:00:36.277Z","updated_at":"2025-08-12T23:02:00.809Z","avatar_url":"https://github.com/melin.png","language":"ANTLR","readme":"## 介绍\n\n基于 antlr4 的多种数据库SQL解析器，获取SQL中元数据，可用于数据平台产品中的多个场景：ddl语句提取元数据、sql 权限校验、表级血缘、sql语法校验等场景。支持spark、flink、gauss、starrocks、Oracle、MYSQL、Postgresql，sqlserver,、db2等\n```\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.github.melin.superior\u003c/groupId\u003e\n    \u003cartifactId\u003esuperior-[spark|presto|mysql|oracle|...]-parser\u003c/artifactId\u003e\n    \u003cversion\u003e4.1.0-SNAPSHOT\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n\u003e 4.0.x 支持jdk8, antlr 4.9.3, 4.1.x 支持 jdk11, antlr 4.13.1\n\n## Build\n```\nmvn clean deploy\n```\n\n### API\n\n每个数据库SQL 提供 Helper 类，Helper 方法提供四个方法：\n```agsl\n1. parseStatement(String sql) // 解析单个完整sql\n    a. ddl: 获取ddl 详细信息。例如：数据库执行完ddl以后，解析ddl，获取到相关信息，同步到元数据信息。\n    b. dml: 获取sql中使用到的表，用于构建表级血缘，或者校验表权限。\n2. parseMultiStatement(String sql) // 解析多个完整sql，支持空格、换行、分号分隔\n3. splitSql(String sql) // sql 文本包含多个完整sql，方法用于分隔sql语句，支持空格、换行、分号分隔\n4. checkSqlSyntax(String sql) // 验证单个完整sql语法是否正确\n5. sqlKeywords() // 获取sql 关键字，主要用于sql editor 关键字提示\n```\n\n## Example\n\n```kotlin\n// Spark SQL\nval sql = \"select bzdys, bzhyyh, bzdy, week, round((bzdy-bzdys)*100/bzdys, 2) \" +\n        \"from (select lag(bzdy) over (order by week) bzdys, bzhyyh, bzdy, week \" +\n        \"from (select count(distinct partner_code) bzhyyh, count(1) bzdy, week from tdl_dt2x_table)) limit 111\"\n\nval statement = SparkSQLHelper.parseStatement(sql)\nif (statement is QueryStmt) {\n    Assert.assertEquals(StatementType.SELECT, statement.statementType)\n    Assert.assertEquals(1, statement.inputTables.size)\n    Assert.assertEquals(\"tdl_dt2x_table\", statement.inputTables.get(0).tableName)\n    Assert.assertEquals(111, statement.limit)\n} else {\n    Assert.fail()\n}\n\n// Spark Jar\nval sql = \"\"\"\n    set spark.shuffle.compress=true;set spark.rdd.compress=true;\n    set spark.driver.maxResultSize=3g;\n    set spark.serializer=org.apache.spark.serializer.KryoSerializer;\n    set spark.kryoserializer.buffer.max=1024m;\n    set spark.kryoserializer.buffer=256m;\n    set spark.network.timeout=300s;\n    examples-jar-with-dependencies.jar imei_test.euSaveHBase gaea_offline:account_mobile sh md shda.interest_radar_mobile_score_dt 20180318 /xiaoyong.fu/sh/mobile/loan 400 '%7B%22job_type%22=' --jar\n    \"\"\";\n\nval statementDatas = JobTaskHelper.parseStatement(sql)\nAssert.assertEquals(8, statementDatas.size)\nvar statementData = statementDatas.get(7)\nvar statement = statementData.statement\nif (statement is JobData) {\n    Assert.assertEquals(StatementType.JOB, statement.statementType)\n    Assert.assertEquals(\"createHfile-1.2-SNAPSHOT-jar-with-dependencies.jar\", statement.resourceName)\n    Assert.assertEquals(\"imei_test.euSaveHBase\", statement.className)\n    Assert.assertEquals(\"/xiaoyong.fu/sh/mobile/loan\", statement.params?.get(5))\n    Assert.assertEquals(\"400\", statement.params?.get(6))\n    Assert.assertEquals(\"%7B%22job_type%22=\", statement.params?.get(7))\n    Assert.assertEquals(\"--jar\", statement.params?.get(8))\n} else {\n    Assert.fail()\n}\n\n// MySQL\nval sql = \"insert into bigdata.user select * from users a left outer join address b on a.address_id = b.id\"\nval statement = MySQLHelper.parseStatement(sql)\nif(statement is QueryStmt) {\n    Assert.assertEquals(StatementType.INSERT_SELECT, statement.statementType)\n    Assert.assertEquals(\"bigdata\", statement.outpuTables.get(0).databaseName)\n    Assert.assertEquals(\"user\", statement.outpuTables.get(0).tableName)\n    Assert.assertEquals(2, statement.inputTables.size)\n} else {\n    Assert.fail()\n}\n\n// Postgres\nval sql = \"\"\"\n    select a.* from datacompute1.datacompute.dc_job a left join datacompute1.datacompute.dc_job_scheduler b on a.id=b.job_id\n\"\"\".trimIndent()\n\nval statement = PostgreSQLHelper.parseStatement(sql)\nif (statement is QueryStmt) {\n    Assert.assertEquals(StatementType.SELECT, statement.statementType)\n    Assert.assertEquals(2, statement.inputTables.size)\n} else {\n    Assert.fail()\n}\n```\n\n## 支持数据库\n1. [MySQL](https://github.com/antlr/grammars-v4/tree/master/sql/mysql)\n2. [PrestoSQL](https://github.com/prestosql/presto/tree/master/presto-parser/src/main/antlr4/io/prestosql/sql/parser)\n3. [PostgreSQL](https://github.com/pgcodekeeper/pgcodekeeper/tree/master/apgdiff/antlr-src)\n4. [Spark 3.x](https://github.com/apache/spark/tree/master/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser)\n5. [Sql Server](https://github.com/antlr/grammars-v4/tree/master/sql/tsql) \n6. [StarRocks](https://github.com/StarRocks/starrocks/tree/main/fe/fe-core/src/main/java/com/starrocks/sql/parser)\n7. [Oracle](https://github.com/antlr/grammars-v4/tree/master/sql/plsql)\n8. [OceanBase](https://github.com/oceanbase/odc/tree/main/libs/ob-sql-parser)\n9. [Flink SQL / Flink CDC SQL](https://github.com/DTStack/dt-sql-parser/tree/main/src/grammar/flinksql)\n\n## 相关项目\n1. https://gitee.com/melin/bee\n2. https://github.com/melin/sqlflow/ 字段血缘解析\n3. https://github.com/melin/superior-sql-formatter spark sql 代码格式化\n4. https://github.com/melin/datatunnel spark 数据同步工具\n5. https://github.com/melin/flink-jobserver \n6. https://github.com/melin/spark-jobserver\n\n","funding_links":[],"categories":["ANTLR"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelin%2Fsuperior-sql-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmelin%2Fsuperior-sql-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelin%2Fsuperior-sql-parser/lists"}