{"id":19993308,"url":"https://github.com/gitchennan/elasticsearch-analysis-lc-pinyin","last_synced_at":"2025-05-04T12:31:14.073Z","repository":{"id":56685315,"uuid":"73531058","full_name":"gitchennan/elasticsearch-analysis-lc-pinyin","owner":"gitchennan","description":"一款运行于Elasticsearch之上的中文拼音智能分词插件，支持全拼、首字母、中文混合搜索","archived":false,"fork":false,"pushed_at":"2023-12-16T20:26:45.000Z","size":271,"stargazers_count":155,"open_issues_count":15,"forks_count":57,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-11-13T04:56:17.737Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"artistic-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gitchennan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-11-12T04:18:25.000Z","updated_at":"2024-09-14T09:21:57.000Z","dependencies_parsed_at":"2022-08-15T23:20:16.115Z","dependency_job_id":null,"html_url":"https://github.com/gitchennan/elasticsearch-analysis-lc-pinyin","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gitchennan%2Felasticsearch-analysis-lc-pinyin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gitchennan%2Felasticsearch-analysis-lc-pinyin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gitchennan%2Felasticsearch-analysis-lc-pinyin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gitchennan%2Felasticsearch-analysis-lc-pinyin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gitchennan","download_url":"https://codeload.github.com/gitchennan/elasticsearch-analysis-lc-pinyin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252334322,"owners_count":21731385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T04:52:35.007Z","updated_at":"2025-05-04T12:31:09.065Z","avatar_url":"https://github.com/gitchennan.png","language":"Java","funding_links":[],"categories":["Java","人工智能"],"sub_categories":["自然语言处理"],"readme":"LC Pinyin Analysis for Elasticsearch\r\n====================================\r\n\r\nLc Pinyin版本\r\n-------------\r\n\r\nLC version | ES version\r\n-----------|-----------\r\nmaster | 5.3.0 -\u003e master\r\n5.3.0.1 | 5.3.0\r\n5.2.2.1 | 5.2.2\r\n5.2.0.1 | 5.2.0\r\n5.1.2.1 | 5.1.2\r\n5.1.1.1 | 5.1.1\r\n5.0.2.1 | 5.0.2\r\n5.0.1.2 | 5.0.1\r\n5.0.1.1 | 5.0.1\r\n2.4.2.1 | 2.4.2\r\n2.2.2.1 | 2.2.2\r\n1.4.5.2 | 1.4.5\r\n1.4.5.1 | 1.4.5\r\n\r\nLc Pinyin介绍\r\n-------------\r\n\u003e `elasticsearch-analysis-lc-pinyin`是一款`elasticsearch`拼音分词插件，可以支持按照全拼、首字母，中文混合搜索。\r\n例如我们在某宝搜索框中输入“jianpan” 可以搜索到关键字包含“键盘”的商品。不仅仅输入全拼，有时候我们输入首字母、拼音和首字母、中文和首字母的混合输入，比如：“键pan”、“j盘”、“jianp”、“jpan”、“jianp”、“jp”\r\n等等，都应该匹配到键盘。通过elasticsearch-analysis-lc-pinyin这个插件就能做到类似的搜索\r\n\r\n\u003e * 此拼音插件主要用在`短文档`的搜索上，如文章的标题、作者，商品的品牌等，不建议用在`长文档`中\r\n\r\n分析器 - Analyzer\r\n------\r\n\u003e * `lc_index` : 该分词器用于索引数据时指定，将中文转换为全拼和首字，同时保留中文\r\n\u003e * `lc_search`: 该分词器用于拼音搜索时指定，按最小拼音分词个数拆分拼音，优先拆分全拼\r\n\r\n------\r\n\r\n分词器 - Tokenizer\r\n------\r\n\u003e * `lc_index`：参数 mode: *full_pinyin*，*first_letter*，*chinese_char*\r\n\u003e * `lc_search`：参数 mode: *smart_pinyin*，*single_letter*\r\n\r\n------\r\n\r\n过滤器 - TokenFilter\r\n------\r\n\u003e * `lc_full_pinyin`：将中文Token转全拼(支持多音字)\r\n\u003e * `lc_first_letter`：将中文Token转首字母(支持多音字)\r\n\r\n------\r\n## 过滤器使用示例\r\n以下是使用ik分词结合拼音过滤器使用例子：\r\n1. 创建一个索引，并定义分析器\r\n```bash\r\nPUT /index\r\n{\r\n  \"settings\": {\r\n    \"analysis\": {\r\n      \"analyzer\": {\r\n        \"ik_letter_smart\": {\r\n          \"type\": \"custom\",\r\n          \"tokenizer\": \"ik_max_word\",\r\n          \"filter\": [\r\n            \"lc_first_letter\"\r\n          ]\r\n        },\r\n        \"ik_py_smart\": {\r\n          \"type\": \"custom\",\r\n          \"tokenizer\": \"ik_max_word\",\r\n          \"filter\": [\r\n            \"lc_full_pinyin\"\r\n          ]\r\n        }\r\n      }\r\n    }\r\n  }\r\n}\r\n```\r\n2. 测试分词\r\n```bash\r\ncurl -XGET 'http://192.168.0.109:9200/index/_analyze?analyzer=ik_py_smart\u0026pretty\u0026text=英雄难过美人关'\r\n\r\n#返回词条：\r\n{\r\n  \"tokens\" : [\r\n    {\r\n      \"token\" : \"yingxiongnanguomeirenguan\",\r\n      \"start_offset\" : 0,\r\n      \"end_offset\" : 7,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 0\r\n    },\r\n    {\r\n      \"token\" : \"yingxiongnanguo\",\r\n      \"start_offset\" : 0,\r\n      \"end_offset\" : 4,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 1\r\n    },\r\n    {\r\n      \"token\" : \"yingxiong\",\r\n      \"start_offset\" : 0,\r\n      \"end_offset\" : 2,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 2\r\n    },\r\n    {\r\n      \"token\" : \"nanguo\",\r\n      \"start_offset\" : 2,\r\n      \"end_offset\" : 4,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 3\r\n    },\r\n    {\r\n      \"token\" : \"meirenguan\",\r\n      \"start_offset\" : 4,\r\n      \"end_offset\" : 7,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 4\r\n    },\r\n    {\r\n      \"token\" : \"meiren\",\r\n      \"start_offset\" : 4,\r\n      \"end_offset\" : 6,\r\n      \"type\" : \"CN_WORD\",\r\n      \"position\" : 5\r\n    },\r\n    {\r\n      \"token\" : \"guan\",\r\n      \"start_offset\" : 6,\r\n      \"end_offset\" : 7,\r\n      \"type\" : \"CN_CHAR\",\r\n      \"position\" : 6\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\n## 分析器使用示例\r\n\r\n1.创建一个索引`index`\r\n\r\n```bash\r\ncurl -XPUT http://localhost:9200/index\r\n```\r\n\r\n2.创建类型`brand`的mapping\r\n\r\n```bash\r\ncurl -XPOST http://localhost:9200/index/_mapping/brand -d'\r\n{\r\n  \"brand\": {\r\n    \"properties\": {\r\n      \"name\": {\r\n        \"type\": \"text\",\r\n        \"analyzer\": \"lc_index\",\r\n        \"search_analyzer\": \"lc_search\",\r\n        \"term_vector\": \"with_positions_offsets\"\r\n      }\r\n    }\r\n  }\r\n}'\r\n```\r\n\r\n3.索引一些互联网品牌\r\n\r\n```bash\r\ncurl -XPOST http://localhost:9200/index/brand/1 -d'{\"name\":\"百度\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/8 -d'{\"name\":\"百度糯米\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/2 -d'{\"name\":\"阿里巴巴\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/3 -d'{\"name\":\"腾讯科技\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/4 -d'{\"name\":\"网易游戏\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/9 -d'{\"name\":\"大众点评\"}'\r\ncurl -XPOST http://localhost:9200/index/brand/10 -d'{\"name\":\"携程旅行网\"}'\r\n```\r\n\r\n4.编写高亮查询DSL\r\n\r\n此示例通过`lc_search`分词器配合`match_phrase`查询实现品牌的`全拼`搜索\r\n搜索全拼关键字`baidu`，请求DSL如下：\r\n```bash\r\ncurl -XPOST http://localhost:9200/index/brand/_search  -d'\r\n{\r\n    \"query\": {\r\n        \"match\": {\r\n          \"name\": {\r\n            \"query\": \"baidu\",\r\n            \"analyzer\": \"lc_search\",\r\n            \"type\": \"phrase\"\r\n          }\r\n        }\r\n    },\r\n    \"highlight\" : {\r\n        \"pre_tags\" : [\"\u003ctag1\u003e\"],\r\n        \"post_tags\" : [\"\u003c/tag1\u003e\"],\r\n        \"fields\" : {\r\n            \"name\" : {}\r\n        }\r\n    }\r\n}'\r\n```\r\n\r\n匹配到`百度`、`百度糯米`两个品牌\r\ntip：`百度`排在`百度糯米`的前面，因为name字段长度更短\r\n查询结果：\r\n```bash\r\n{\r\n    \"took\": 18,\r\n    \"timed_out\": false,\r\n    \"_shards\": {\r\n        \"total\": 1,\r\n        \"successful\": 1,\r\n        \"failed\": 0\r\n    },\r\n    \"hits\": {\r\n        \"total\": 2,\r\n        \"max_score\": 2.5751648,\r\n        \"hits\": [\r\n            {\r\n                \"_index\": \"index\",\r\n                \"_type\": \"brand\",\r\n                \"_id\": \"1\",\r\n                \"_score\": 2.5751648,\r\n                \"_source\": {\r\n                    \"name\": \"百度\"\r\n                },\r\n                \"highlight\": {\r\n                    \"name\": [\r\n                        \"\u003ctag1\u003e百度\u003c/tag1\u003e\"\r\n                    ]\r\n                }\r\n            }\r\n            ,\r\n            {\r\n                \"_index\": \"index\",\r\n                \"_type\": \"brand\",\r\n                \"_id\": \"8\",\r\n                \"_score\": 2.0601318,\r\n                \"_source\": {\r\n                    \"name\": \"百度糯米\"\r\n                },\r\n                \"highlight\": {\r\n                    \"name\": [\r\n                        \"\u003ctag1\u003e百度\u003c/tag1\u003e糯米\"\r\n                    ]\r\n                }\r\n            }\r\n        ]\r\n    }\r\n}\r\n```\r\n\r\n此示例通过`lc_search`分词器配合`match_phrase`查询实现品牌的`中文\u0026全拼`搜索\r\n搜索全拼关键字`xie程lu行wang`，请求DSL如下：\r\n```bash\r\ncurl -XPOST http://localhost:9200/index/brand/_search  -d'\r\n{\r\n    \"query\": {\r\n        \"match\": {\r\n          \"name\": {\r\n            \"query\": \"xie程lu行\",\r\n            \"analyzer\": \"lc_search\",\r\n            \"type\": \"phrase\"\r\n          }\r\n        }\r\n    },\r\n    \"highlight\" : {\r\n        \"pre_tags\" : [\"\u003ctag1\u003e\"],\r\n        \"post_tags\" : [\"\u003c/tag1\u003e\"],\r\n        \"fields\" : {\r\n            \"name\" : {}\r\n        }\r\n    }\r\n}'\r\n```\r\n\r\n匹配到`携程旅行网` 结果如下：\r\n```bash\r\n#匹配到`携程旅行网` 结果如下：\r\n{\r\n    \"took\": 4,\r\n    \"timed_out\": false,\r\n    \"_shards\": {\r\n        \"total\": 1,\r\n        \"successful\": 1,\r\n        \"failed\": 0\r\n    },\r\n    \"hits\": {\r\n        \"total\": 1,\r\n        \"max_score\": 4.5665164,\r\n        \"hits\": [\r\n            {\r\n                \"_index\": \"index\",\r\n                \"_type\": \"brand\",\r\n                \"_id\": \"10\",\r\n                \"_score\": 4.5665164,\r\n                \"_source\": {\r\n                    \"name\": \"携程旅行网\"\r\n                },\r\n                \"highlight\": {\r\n                    \"name\": [\r\n                        \"\u003ctag1\u003e携程旅行\u003c/tag1\u003e网\"\r\n                    ]\r\n                }\r\n            }\r\n        ]\r\n    }\r\n}\r\n\r\n```\r\n\r\n```bash\r\n# 此示例通过`lc_search`分词器配合`match_phrase`查询实现品牌的`首字母`搜索\r\n# 此示例中也可以通过`lc_first_letter`分词器搜索，结果和`lc_search`一样\r\n#\r\n# 这两个分词器的主要区别:\r\n#     lc_first_letterl 会把所有输入的字母拆分成单字母用于首字母匹配\r\n#     lc_search        会优先把输入的字母串拆分成全拼,并找到一个最优拆分结果\r\n#\r\n# 搜索全拼关键字`albb`，请求DSL如下：\r\ncurl -XPOST http://localhost:9200/index/brand/_search  -d'\r\n{\r\n    \"query\": {\r\n        \"match\": {\r\n          \"name\": {\r\n            \"query\": \"albb\",\r\n            \"analyzer\": \"lc_search\",\r\n            \"type\": \"phrase\"\r\n          }\r\n        }\r\n    },\r\n    \"highlight\" : {\r\n        \"pre_tags\" : [\"\u003ctag1\u003e\"],\r\n        \"post_tags\" : [\"\u003c/tag1\u003e\"],\r\n        \"fields\" : {\r\n            \"name\" : {}\r\n        }\r\n    }\r\n}'\r\n```\r\n\r\n匹配到`阿里巴巴`，结果如下：\r\n```bash\r\n{\r\n    \"took\": 4,\r\n    \"timed_out\": false,\r\n    \"_shards\": {\r\n        \"total\": 1,\r\n        \"successful\": 1,\r\n        \"failed\": 0\r\n    },\r\n    \"hits\": {\r\n        \"total\": 1,\r\n        \"max_score\": 3.9560113,\r\n        \"hits\": [\r\n            {\r\n                \"_index\": \"index\",\r\n                \"_type\": \"brand\",\r\n                \"_id\": \"2\",\r\n                \"_score\": 3.9560113,\r\n                \"_source\": {\r\n                    \"name\": \"阿里巴巴\"\r\n                },\r\n                \"highlight\": {\r\n                    \"name\": [\r\n                        \"\u003ctag1\u003e阿里巴巴\u003c/tag1\u003e\"\r\n                    ]\r\n                }\r\n            }\r\n        ]\r\n    }\r\n}\r\n```\r\n\r\n```java\r\n//java api实现\r\n//该查询会匹配`阿里巴巴`这条数据\r\nQueryBuilder pinyinQueryBuilder =  QueryBuilders.matchPhraseQuery(\"name\", \"ali巴b\").analyzer(\"lc_search\");\r\n        SearchRequestBuilder requestBuilder = client.prepareSearch(\"index\").setTypes(\"brand\");\r\n        requestBuilder.setQuery(pinyinQueryBuilder)\r\n                .setHighlighterPreTags(\"\u003ctag1\u003e\")\r\n                .setHighlighterPostTags(\"\u003c/tag1\u003e\")\r\n                .addHighlightedField(\"name\")\r\n                .execute().actionGet();\r\n                \r\n                \r\n//该查询会匹配`大众点评`这条数据\r\nQueryBuilder pinyinQueryBuilder =  QueryBuilders.matchPhraseQuery(\"name\", \"dzdp\").analyzer(\"lc_first_letter\");\r\n        SearchRequestBuilder requestBuilder = client.prepareSearch(\"index\").setTypes(\"brand\");\r\n        requestBuilder.setQuery(pinyinQueryBuilder)\r\n                .setHighlighterPreTags(\"\u003ctag1\u003e\")\r\n                .setHighlighterPostTags(\"\u003c/tag1\u003e\")\r\n                .addHighlightedField(\"name\")\r\n                .execute().actionGet();\r\n                \r\n//该查询也会匹配`大众点评`这条数据\r\nQueryBuilder pinyinQueryBuilder =  QueryBuilders.matchPhraseQuery(\"name\", \"dzdp\").analyzer(\"lc_search\");\r\n        SearchRequestBuilder requestBuilder = client.prepareSearch(\"index\").setTypes(\"brand\");\r\n        requestBuilder.setQuery(pinyinQueryBuilder)\r\n                .setHighlighterPreTags(\"\u003ctag1\u003e\")\r\n                .setHighlighterPostTags(\"\u003c/tag1\u003e\")\r\n                .addHighlightedField(\"name\")\r\n                .execute().actionGet();\r\n```\r\n作者:  [@陈楠][1]\r\nEmail: 465360798@qq.com\r\n\r\n\u003c完\u003e\r\n\r\n[1]: http://blog.csdn.net/chennanymy?viewmode=contents","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgitchennan%2Felasticsearch-analysis-lc-pinyin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgitchennan%2Felasticsearch-analysis-lc-pinyin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgitchennan%2Felasticsearch-analysis-lc-pinyin/lists"}