{"id":17994071,"url":"https://github.com/anylisten/elasticsearch-analysis-hanlp","last_synced_at":"2025-03-26T02:30:47.468Z","repository":{"id":94641415,"uuid":"152359028","full_name":"AnyListen/elasticsearch-analysis-hanlp","owner":"AnyListen","description":"HanLP Analysis for Elasticsearch","archived":false,"fork":false,"pushed_at":"2019-04-21T16:57:46.000Z","size":83,"stargazers_count":89,"open_issues_count":3,"forks_count":30,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-21T01:41:29.703Z","etag":null,"topics":["analysis","elasticsearch","elasticsearch-plugin","es","hanlp"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnyListen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-10T03:40:54.000Z","updated_at":"2024-10-06T11:56:16.000Z","dependencies_parsed_at":"2023-07-28T21:30:59.686Z","dependency_job_id":null,"html_url":"https://github.com/AnyListen/elasticsearch-analysis-hanlp","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnyListen%2Felasticsearch-analysis-hanlp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnyListen%2Felasticsearch-analysis-hanlp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnyListen%2Felasticsearch-analysis-hanlp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnyListen%2Felasticsearch-analysis-hanlp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnyListen","download_url":"https://codeload.github.com/AnyListen/elasticsearch-analysis-hanlp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245576522,"owners_count":20638123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analysis","elasticsearch","elasticsearch-plugin","es","hanlp"],"created_at":"2024-10-29T20:13:57.840Z","updated_at":"2025-03-26T02:30:47.445Z","avatar_url":"https://github.com/AnyListen.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"HanLP Analysis for Elasticsearch\n=====\n\n基于 [HanLP](https://github.com/hankcs/HanLP) 的 Elasticsearch 中文分词插件，核心功能：\n\n1. 兼容 ES 5.x-7.x；\n2. 内置词典，无需额外配置即可使用；\n3. 支持用户自定义词典；\n4. 支持远程词典热更新（待开发）；\n5. 内置多种分词模式，适合不同场景；\n6. 拼音过滤器（待开发）；\n7. 简繁体转换过滤器（待开发）。\n\n## 版本\n插件版本和 ES 版本一致，直接下载对应版本的插件进行安装即可。\n\n- 插件开发完成时，最新版本已经为 6.5.2 了，所以个人只对典型的版本进行了测试；\n- 5.X 在 5.0.0、5.5.0 版本进行了测试；\n- 6.X 在 6.0.0、6.3.0、6.4.1、6.5.1 版本进行了测试；\n- 7.X 在 7.0.0 版本进行了测试。\n\n## 安装使用\n### 下载编译\ngit clone 对应版本的代码，打开 `pom.xml` 文件，修改 `\u003celasticsearch.version\u003e6.5.1\u003c/elasticsearch.version\u003e` 为需要的 ES 版本；然后使用 `mvn package` 生产打包文件，最终文件在 `target/release` 文件夹下。\n\n打包完成后，使用离线方式安装即可。\n\n### 使用默认词典\n- 在线安装：`.\\elasticsearch-plugin install https://github.com/AnyListen/elasticsearch-analysis-hanlp/releases/download/vA.B.C/elasticsearch-analysis-hanlp-A.B.C.zip`\n- 离线安装：`.\\elasticsearch-plugin install file:///FILE_PATH/elasticsearch-analysis-hanlp-A.B.C.zip`\n\n\u003e 离线安装请把 `FILE_PATH` 更改为 zip 文件路径；A、B、C 对应的是 ES 版本号。\n\n### 使用自定义词典\n默认词典是精简版的词典，能够满足基本需求，但是无法使用感知机和 CRF 等基于模型的分词器。\n\nHanLP 提供了更加[完整的词典](http://nlp.hankcs.com/download.php?file=data)，请按需下载。\n\n词典下载后，解压到任意目录，然后修改**插件安装目录下**的 `hanlp.properties` 文件，只需修改第一行\n```\nroot=D:/JavaProjects/HanLP/\n```\n为 `data` 的父目录即可，比如 `data` 目录是 `/Users/hankcs/Documents/data`，那么 `root=/Users/hankcs/Documents/`。\n\n### 使用自定义配置文件\n如果你在其他地方使用了 HanLP，希望能够复用 `hanlp.properties` 文件，你只需要修改**插件安装目录下**的 `plugin.properties` 文件，将 `configPath` 配置为已有的 `hanlp.properties` 文件地址即可。\n\n## 内置分词器\n### 分析器(Analysis)\n- hanlp_index：细粒度切分\n- hanlp_smart：常规切分\n- hanlp_nlp：命名实体识别\n- hanlp_per：感知机分词\n- hanlp_crf：CRF分词\n- hanlp：自定义\n\n### 分词器(Tokenizer)\n- hanlp_index：细粒度切分\n- hanlp_smart：常规切分\n- hanlp_nlp：命名实体识别\n- hanlp_per：感知机分词\n- hanlp_crf：CRF分词\n- hanlp：自定义\n\n### 自定义分词器\n插件有较为丰富的选项允许用户自定义分词器，下面是可用的配置项：\n\n| 配置项名称       | 功能   |  默认值  |\n| --------   | -----:  | :----:  |\n| algorithm   | 可选项有：\u003cbr/\u003e viterbi：维特比分词 \u003cbr/\u003e |   viterbi     |\n| enableIndexMode    | 设为索引模式（细粒度切分） |   false     |\n| enableCustomDictionary    | 是否启用用户词典 |   true     |\n| customDictionaryPath    | 用户词典路径(绝对路径,多个词典用`;`隔开) |   null     |\n| enableCustomDictionaryForcing    | [用户词典高优先级](https://github.com/hankcs/HanLP/wiki/FAQ#%E4%B8%BA%E4%BB%80%E4%B9%88%E4%BF%AE%E6%94%B9%E4%BA%86%E8%AF%8D%E5%85%B8%E8%BF%98%E6%98%AF%E6%B2%A1%E6%9C%89%E6%95%88%E6%9E%9C) |   false     |\n| enableStopWord    | 是否启用停用词过滤 |   false     |\n| stopWordDictionaryPath    | 停用词词典路径 |   null     |\n| enableNumberQuantifierRecognize    | 是否启用数词和数量词识别 |   true     |\n| enableNameRecognize    | 开启人名识别 |   true     |\n| enableTranslatedNameRecognize    | 是否启用音译人名识别 |   false     |\n| enableJapaneseNameRecognize    | 是否启用日本人名识别 |   false     |\n| enableOrganizationRecognize    | 开启机构名识别 |   false     |\n| enablePlaceRecognize    | 开启地名识别 |   false     |\n| enableTraditionalChineseMode    | 开启精准繁体中文分词 |   false     |\n\n**案例展示：**\n```\n# 创建自定义分词器\nPUT my_index\n{\n  \"settings\": {\n    \"analysis\": {\n      \"analyzer\": {\n        \"my_analyzer\": {\n          \"type\": \"hanlp\",\n          \"algorithm\": \"viterbi\",\n          \"enableIndexMode\": \"true\",\n          \"enableCustomDictionary\": \"true\",\n          \"customDictionaryPath\": \"\",\n          \"enableCustomDictionaryForcing\": \"false\",\n          \"enableStopWord\": \"true\",\n          \"stopWordDictionaryPath\": \"\",\n          \"enableNumberQuantifierRecognize\": \"true\",\n          \"enableNameRecognize\": \"true\",\n          \"enableTranslatedNameRecognize\": \"true\",\n          \"enableJapaneseNameRecognize\": \"true\",\n          \"enableOrganizationRecognize\": \"true\",\n          \"enablePlaceRecognize\": \"true\",\n          \"enableTraditionalChineseMode\": \"false\"\n        }\n      }\n    }\n  }\n}\n\n# 测试分词器\nPOST my_index/_analyze\n{\n  \"analyzer\": \"my_analyzer\",\n  \"text\": \"张惠妹在上海市举办演唱会啦\"\n}\n```\n\n## 分词速度（仅供参考）\n\u003e 借助 `_analyze` API（**1核1G单线程**），通过改变分词器类型，对 2W 字的文本进行分词，以下为从请求到返回的耗时：\n\n分词器 | 耗时（ms）\n--- | ---\n`hanlp_smart` | 148\n`hanlp_nlp`  | 182\n`hanlp_per`  | 286\n`hanlp_crf` | 357\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanylisten%2Felasticsearch-analysis-hanlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanylisten%2Felasticsearch-analysis-hanlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanylisten%2Felasticsearch-analysis-hanlp/lists"}