{"id":25487931,"url":"https://github.com/anonymity94/spl2sql","last_synced_at":"2025-10-11T18:10:13.677Z","repository":{"id":44378068,"uuid":"341480836","full_name":"Anonymity94/spl2sql","owner":"Anonymity94","description":"Convert Splunk SPL to ClickHouse SQL with pegjs","archived":false,"fork":false,"pushed_at":"2022-07-19T04:37:48.000Z","size":527,"stargazers_count":13,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-08-30T03:52:08.914Z","etag":null,"topics":["clickhouse","dsl","pegjs","sql"],"latest_commit_sha":null,"homepage":"https://anonymity94.github.io/spl2sql/examples/","language":"PEG.js","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Anonymity94.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-23T08:23:48.000Z","updated_at":"2024-12-06T09:45:50.000Z","dependencies_parsed_at":"2022-07-12T18:20:33.514Z","dependency_job_id":null,"html_url":"https://github.com/Anonymity94/spl2sql","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Anonymity94/spl2sql","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonymity94%2Fspl2sql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonymity94%2Fspl2sql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonymity94%2Fspl2sql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonymity94%2Fspl2sql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Anonymity94","download_url":"https://codeload.github.com/Anonymity94/spl2sql/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonymity94%2Fspl2sql/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279008298,"owners_count":26084428,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","dsl","pegjs","sql"],"created_at":"2025-02-18T20:37:58.748Z","updated_at":"2025-10-11T18:10:13.661Z","avatar_url":"https://github.com/Anonymity94.png","language":"PEG.js","readme":"# Splunk-SPL-to-ClickHouse-SQL\n\n一个将 `Splunk SPL` 转化成 `ClickHouse SQL` 的转换器，并做了一些拓展。\n\n可以用来做自定义过滤查询。\n\n## 使用\n\n```ts\nimport converter from \"spl2sql\";\n\ntry {\n  const result = converter.parse(`| search a=1 and b=2`, {\n    json: true, // 结果以json形式返回\n  });\n  console.log(result);\n} catch (error) {\n  console.log(error.message);\n}\n```\n\n```html\n\u003cscript src=\"../dist/spl2sql.js\"\u003e\u003c/script\u003e\n\n\u003cscript\u003e\n  try {\n    var result = splToSqlConverter.parse(value, {\n      json: true,\n    });\n  } catch (error) {\n    console.log(error);\n  }\n\u003c/script\u003e\n```\n\n## 开发\n\n```sh\n# install package\nyarn\n\n# build\nyarn build\n\n# test\nyarn test\n```\n\n\n## 一个完整的搜索\n\n```\n# 源IPv4等于1.1.1.1 并且 (省份=北京 或者 北京=山东)\nipv4_initiator\u003cIPv4\u003e = '1.1.1.1' AND (province = '北京' OR province = '山东')\n# 时间范围7天内\n| gentimes start_time start=now-7d end=now\n# 按照 start_time 倒序\n| sort -start_time\n# 返回前1000条数据\n| head 1000\n```\n\n## 语法说明\n\n```\n# 搜索表名，可以省略\n[source \u003ctableName\u003e]\n# 搜索字段\n[[| search] \u003cfield-name\u003e \u003coperate\u003e \u003cfield-value\u003e] [\u003clogical-connector\u003e \u003cfield-name\u003e \u003coperate\u003e \u003cfield-value\u003e]]\n\n# 限制时间\n[| gentimes \u003ctime-field\u003e start \u003ctime-value\u003e [end \u003ctime-value\u003e]]\n\n# 字段排序\n# +: ASC\n# -: DESC\n[| sort [+/-]\u003cfield-name\u003e[,[+/-]\u003cfield-name\u003e]]\n\n# 返回命中条件的前 N 条\n[| head \u003cnumber\u003e]\n\n# 查询结果中包含（排除）的字段\n# +: 包含字段\n# -: 排除字段\n[| fields [+/-]\u003cfield-name\u003e[,[+/-]\u003cfield-name\u003e]\n```\n\n## 参数说明\n\n|         参数          |    名称    | 描述                                                                                                                     |\n| :-------------------: | :--------: | :----------------------------------------------------------------------------------------------------------------------- |\n|    `\u003cfield-name\u003e`     |   字段名   | 允许输入大小字母、数字、下划线`_`、英文的点`.`\u003cbr /\u003e例如：`start_time`、`cup.usage`                                      |\n|      `\u003coperate\u003e`      |   操作符   | `=`、`!=`、`\u003e`、`\u003e=`、`\u003c`、`\u003c=`、`IN`、`NOT IN`、`LIKE`、`NOT LIKE`、`EXISTS`、`NOT_EXISTS`\u003cbr /\u003e注意：不区分大小写      |\n|    `\u003cfield-value\u003e`    |   字段值   | 无特殊限制，允许内容被单引号`''`或双引号`\"\"`包裹，但是引号内不允许出现引号。\u003cbr /\u003e例如：`12`、`\"1.2\"`、`\"中国\"`、`\"a_b\"` |\n| `\u003clogical-connector\u003e` | 逻辑关系符 | `AND`、`OR`、`\u0026\u0026`\u003cbr /\u003e注意：不区分大小写                                                                                |\n|    `\u003ctime-field\u003e`     | 时间字段名 | 同`\u003cfield-name\u003e`                                                                                                         |\n|    `\u003ctime-value\u003e`     | 时间内容值 | [时间范围](#时间范围)\u003cbr/\u003e绝对时间值请用单引号或双引号包裹                                                               |\n\n## 字段带类型时的转换逻辑\n\n### IPv4\n\n\u003e [toIPv4(string)](https://clickhouse.tech/docs/en/sql-reference/functions/ip-address-functions/#toipv4string)\n\n```\nipv4_field\u003cIPv4\u003e = 1.1.1.1\n```\n\n支持 `CIDR` 格式\n\n\u003e [ipv4cidrtorangeipv4-cidr](https://clickhouse.tech/docs/en/sql-reference/functions/ip-address-functions/#ipv4cidrtorangeipv4-cidr)\n\n```\nipv4_cidr_field\u003cIPv4\u003e = 1.1.1.1/10\n```\n\n### IPv6\n\n\u003e [toIPv6(string)](https://clickhouse.tech/docs/en/sql-reference/functions/ip-address-functions/#toipv6string)\n\n```\nipv6_field\u003cIPv6\u003e = 8090::4\n```\n\n支持 `CIDR` 格式\n\n\u003e [ipv4cidrtorangeipv4-cidr](https://clickhouse.tech/docs/en/sql-reference/functions/ip-address-functions/#ipv4cidrtorangeipv4-cidr)\n\n```\nipv6_cidr_field\u003cIPv6\u003e = 8090::4/10\n```\n\n### Array\n\n\u003e [has(arr, elem)](https://clickhouse.tech/docs/en/sql-reference/functions/array-functions/#hasarr-elem)\n\n```\narray_fieid\u003cArray\u003e = '测试'\n```\n\n### Array\u003cIPv4\u003e\n\n```\nipv4_array_fieid\u003cArray\u003cIPv4\u003e\u003e = '1.1.1.1'\n```\n\n### Array\u003cIPv6\u003e\n\n```\nipv6_array_fieid\u003cArray\u003cIPv6\u003e\u003e = '8090::4'\n```\n\n## Demo\n\n### 字段条件\n\n⚠️ 开头的 `| search` 可省略\n\n#### 操作符 `=`\n\n```\n# 含义：字段a 等于 1.2.3.4\n| search a = 1.2.3.4\n# 等价于\n a = 1.2.3.4\n\n```\n\n#### 操作符 `!=`\n\n```\n# 含义：字段 name 不等于 张三\nname != '张三'\n```\n\n#### 操作符 `\u003e`\n\n```\n# 含义：字段 age 大于 18\nage \u003e 18\n```\n\n#### 操作符 `\u003e=`\n\n```\n# 含义：字段 age 大于等于 18\nc \u003e= 18\n```\n\n#### 操作符 `\u003c`\n\n```\n# 含义：字段 age 小于 18\nage \u003c 18\n```\n\n#### 操作符 `\u003c=`\n\n```\n# 含义：字段 age 小于等于 18\nage \u003c= 18\n```\n\n#### 操作符 `IN`\n\n可用于搜索多个值\n\n```\n# 含义：字段name=张三 或者 name=李四\nname IN (\"张三\", \"李四\")\n\n# 等价于\nname = \"张三\" OR name = \"李四\"\n```\n\n#### 操作符 `NOT IN`\n\n可用于排除多个值\n\n```\n# 含义：字段name!=张三 并且 name!=李四\nname NOT IN (\"张三\", \"李四\")\n\n# 等价于\nname != \"张三\" AND name != \"李四\"\n```\n\n#### 操作符 `LIKE`\n\n可用于模糊查询，条件可以分为四种匹配模式\n\n① `％` 表示零个或任意多个字符\n\n```\n# 以\"山\"开头的省份，例如：山东、山西\nprovince LIKE \"山%\"\n\n# 以\"东\"结尾的省份，例如：山东、广东\nprovince LIKE \"%东\"\n\n# 包含\"马\"名字，例如：马云、马化腾、司马光、\nname LIKE \"%马%\"\n```\n\n② `_` 任意单个字符、匹配单个任意字符\n\n```\n# 以 \"C\" 开头，然后是一个任意字符，然后是 \"r，然后是任意字符，然后是 \"er\"：\nname LIKE \"C_r_er\"\n```\n\n#### 操作符 `NOT LIKE`\n\n可用于排除字段。用法同操作符 `LIKE`\n\n#### 操作符 `EXISTS`\n\n**⚠️ 因 `empty()` 仅适用于 `String` 和`Array` 类型，所以需要格外注意使用场景**\n\n\u003e [string-functions/#empty](https://clickhouse.tech/docs/en/sql-reference/functions/string-functions/#empty)\n\n排除 `''` 或 `NULL` 或 `[]` ，其他的都会被命中。\n\n```\n# name 字段不为空\nname EXISTS\n```\n\n#### 操作符 `NOT_EXISTS`\n\n**⚠️ 因 `NOT empty()` 仅适用于 `String` 和`Array` 类型，所以需要格外注意使用场景**\n\n\u003e [string-functions/#notempty](https://clickhouse.tech/docs/en/sql-reference/functions/string-functions/#notempty)\n\n搜索不存在值的字段，字段值为 `''` 或 `NULL` 或 `[]` 时会被命中。\n\n```\n# name 不存在值\nname NOT_EXISTS\n```\n\n#### 使用逻辑关系表达式查询多个字段\n\n```\na=1 AND b\u003e4\na=1 \u0026\u0026 (b=1 AND (c=\"2\" OR c='3')) OR d!='2'\na=1 and b IN ('2','3','4') and c LIKE \"%a_b%\"\na=1 or b in ('2','3','4')\n```\n\n#### 字段排序\n\n```\n# 按 create_time 倒序，按 age 正序\n| sort -create_time, +age\n```\n\n#### 返回前 N 条记录\n\n```\n| head 100\n```\n\n#### 自定义结果中返回的字段\n\n```\n| fields +name,+age\n```\n\n针对时间格式做处理一些调整，这里的时间格式和`Splunk`中标准的时间格式不同。\n\n#### splunk 标准格式\n\n`Splunk` 中的时间格式为：`| gentimes start=\u003ctimestamp\u003e [end=\u003ctimestamp\u003e] [increment=\u003cincrement\u003e]` [Gentimes 文档](https://docs.splunk.com/Documentation/Splunk/8.0.5/SearchReference/Gentimes)\n\n其中 `timestamp` 的格式为：`MM/DD/YYYY[:HH:MM:SS] | \u003cint\u003e`\n\n---\n\n#### 修改后的时间内容值\n\n`| gentimes \u003ctime-field\u003e start=\u003ctime-value\u003e [end=\u003ctime-value\u003e]`\n\n时间的内容值可以分为**相对时间**和**绝对时间**：\n\n- 相对时间\n\n  - `now` 当前时间\n\n  - `now-\u003cint\u003e(y | M | w | d | H | h | m | s)`\n\n    | 单位       | 说明      |\n    | ---------- | --------- |\n    | `y`        | `Year`    |\n    | `M`        | `Months`  |\n    | `w`        | `Weeks`   |\n    | `d`        | `Days`    |\n    | `h` or `H` | `Hours`   |\n    | `m`        | `Minutes` |\n    | `s`        | `Seconds` |\n\n    例如：`now-7d`，7 天前\n\n- 绝对时间\n\n  - `2017-10-14T12:34:56+08`\n  - `2017-10-14T12:34:56+0800`\n  - `2017-10-14T12:34:56+08:00`\n  - `2020-10-14 12:34:56` 没有时区时以当前时区为准\n\n  - 时间戳（毫秒）\n\n#### 使用 Demo\n\n- `| gentimes \u003ctime-field\u003e start=\"2020-07-13T00:00:00+08\" end=\"2020-07-13T23:59:59+08\"`\n- `| gentimes \u003ctime-field\u003e start=now-7d end=now`\n- `| gentimes \u003ctime-field\u003e start=1594569600000 end=1594624363506`\n\n## Links\n\n- [SPL: Search Commands](https://docs.splunk.com/Documentation/Splunk/8.1.2/SearchReference/Abstract)\n- [SELECT Query](https://clickhouse.tech/docs/en/sql-reference/statements/select/)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanonymity94%2Fspl2sql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanonymity94%2Fspl2sql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanonymity94%2Fspl2sql/lists"}