{"id":13413556,"url":"https://github.com/yanyiwu/gojieba","last_synced_at":"2025-05-14T10:06:55.003Z","repository":{"id":37431346,"uuid":"42340004","full_name":"yanyiwu/gojieba","owner":"yanyiwu","description":"\"结巴\"中文分词的Golang版本","archived":false,"fork":false,"pushed_at":"2024-09-14T15:49:34.000Z","size":8366,"stargazers_count":2423,"open_issues_count":17,"forks_count":303,"subscribers_count":67,"default_branch":"master","last_synced_at":"2024-10-29T15:35:14.069Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yanyiwu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-09-12T01:30:44.000Z","updated_at":"2024-10-29T15:04:55.000Z","dependencies_parsed_at":"2024-11-08T03:33:15.674Z","dependency_job_id":"ca12b273-80b0-4379-9ee2-c56bce431b66","html_url":"https://github.com/yanyiwu/gojieba","commit_stats":{"total_commits":174,"total_committers":18,"mean_commits":9.666666666666666,"dds":0.5459770114942528,"last_synced_commit":"a53d40fa14c54bc09834cb526baca79322da5b0a"},"previous_names":[],"tags_count":29,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanyiwu%2Fgojieba","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanyiwu%2Fgojieba/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanyiwu%2Fgojieba/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanyiwu%2Fgojieba/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yanyiwu","download_url":"https://codeload.github.com/yanyiwu/gojieba/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245556980,"owners_count":20634896,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:43.102Z","updated_at":"2025-03-25T23:18:11.324Z","avatar_url":"https://github.com/yanyiwu.png","language":"Go","funding_links":[],"categories":["开源类库","自然语言处理","Go","Open source library","Natural Language Processing","Microsoft Office","\u003cspan id=\"自然语言处理-natural-language-processing\"\u003e自然语言处理 Natural Language Processing\u003c/span\u003e","Bot Building","Relational Databases","Libraries"],"sub_categories":["搜索推荐","分词器","Search Recommendations","Tokenizers","Uncategorized","\u003cspan id=\"高级控制台用户界面-advanced-console-uis\"\u003e高级控制台用户界面 Advanced Console UIs\u003c/span\u003e","Advanced Console UIs","Strings","暂未分类","交流","Books","暂未分类这些库被放在这里是因为其他类别似乎都不适合。"],"readme":"# GoJieba\n\n[![Test](https://github.com/yanyiwu/gojieba/actions/workflows/test.yml/badge.svg)](https://github.com/yanyiwu/gojieba/actions/workflows/test.yml)\n[![Author](https://img.shields.io/badge/author-@yanyiwu-blue.svg?style=flat)](http://yanyiwu.com/) \n[![Tag](https://img.shields.io/github/v/tag/yanyiwu/gojieba.svg)](https://github.com/yanyiwu/gojieba/releases)\n[![Performance](https://img.shields.io/badge/performance-excellent-brightgreen.svg?style=flat)](http://yanyiwu.com/work/2015/06/14/jieba-series-performance-test.html) \n[![License](https://img.shields.io/badge/license-MIT-yellow.svg?style=flat)](http://yanyiwu.mit-license.org)\n[![GoDoc](https://godoc.org/github.com/yanyiwu/gojieba?status.svg)](https://godoc.org/github.com/yanyiwu/gojieba)\n[![Coverage Status](https://coveralls.io/repos/yanyiwu/gojieba/badge.svg?branch=master\u0026service=github)](https://coveralls.io/github/yanyiwu/gojieba?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/yanyiwu/gojieba)](https://goreportcard.com/report/yanyiwu/gojieba)\n[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/avelino/awesome-go) \n\n[GoJieba]是\"结巴\"中文分词的Golang语言版本。\n\n## 简介\n\n+ 支持多种分词方式，包括: 最大概率模式, HMM新词发现模式, 搜索引擎模式, 全模式\n+ 核心算法底层由C++实现，性能高效。\n+ 字典路径可配置，NewJieba(...string), NewExtractor(...string) 可变形参，当参数为空时使用默认词典(推荐方式)\n\n## 用法\n\n```bash\ngo get github.com/yanyiwu/gojieba\n```\n\n分词示例\n\n```golang\npackage main\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/yanyiwu/gojieba\"\n)\n\nfunc main() {\n\tvar s string\n\tvar words []string\n\tuse_hmm := true\n\tx := gojieba.NewJieba()\n\tdefer x.Free()\n\n\ts = \"我来到北京清华大学\"\n\twords = x.CutAll(s)\n\tfmt.Println(s)\n\tfmt.Println(\"全模式:\", strings.Join(words, \"/\"))\n\n\twords = x.Cut(s, use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"精确模式:\", strings.Join(words, \"/\"))\n\ts = \"比特币\"\n\twords = x.Cut(s, use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"精确模式:\", strings.Join(words, \"/\"))\n\n\tx.AddWord(\"比特币\")\n\t// `AddWordEx` 支持指定词语的权重，作为 `AddWord` 权重太低加词失败的补充。\n\t// `tag` 参数可以为空字符串，也可以指定词性。\n\t// x.AddWordEx(\"比特币\", 100000, \"\")\n\ts = \"比特币\"\n\twords = x.Cut(s, use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"添加词典后,精确模式:\", strings.Join(words, \"/\"))\n\n\ts = \"他来到了网易杭研大厦\"\n\twords = x.Cut(s, use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"新词识别:\", strings.Join(words, \"/\"))\n\n\ts = \"小明硕士毕业于中国科学院计算所，后在日本京都大学深造\"\n\twords = x.CutForSearch(s, use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"搜索引擎模式:\", strings.Join(words, \"/\"))\n\n\ts = \"长春市长春药店\"\n\twords = x.Tag(s)\n\tfmt.Println(s)\n\tfmt.Println(\"词性标注:\", strings.Join(words, \",\"))\n\n\ts = \"区块链\"\n\twords = x.Tag(s)\n\tfmt.Println(s)\n\tfmt.Println(\"词性标注:\", strings.Join(words, \",\"))\n\n\ts = \"长江大桥\"\n\twords = x.CutForSearch(s, !use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"搜索引擎模式:\", strings.Join(words, \"/\"))\n\n\twordinfos := x.Tokenize(s, gojieba.SearchMode, !use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"Tokenize:(搜索引擎模式)\", wordinfos)\n\n\twordinfos = x.Tokenize(s, gojieba.DefaultMode, !use_hmm)\n\tfmt.Println(s)\n\tfmt.Println(\"Tokenize:(默认模式)\", wordinfos)\n\n\tkeywords := x.ExtractWithWeight(s, 5)\n\tfmt.Println(\"Extract:\", keywords)\n}\n```\n\n```\n我来到北京清华大学\n全模式: 我/来到/北京/清华/清华大学/华大/大学\n我来到北京清华大学\n精确模式: 我/来到/北京/清华大学\n比特币\n精确模式: 比特/币\n比特币\n添加词典后,精确模式: 比特币\n他来到了网易杭研大厦\n新词识别: 他/来到/了/网易/杭研/大厦\n小明硕士毕业于中国科学院计算所，后在日本京都大学深造\n搜索引擎模式: 小明/硕士/毕业/于/中国/科学/学院/科学院/中国科学院/计算/计算所/，/后/在/日本/京都/大学/日本京都大学/深造\n长春市长春药店\n词性标注: 长春市/ns,长春/ns,药店/n\n区块链\n词性标注: 区块链/nz\n长江大桥\n搜索引擎模式: 长江/大桥/长江大桥\n长江大桥\nTokenize: [{长江 0 6} {大桥 6 12} {长江大桥 0 12}]\n```\n\nSee Details in [gojieba-demo](http://github.com/yanyiwu/gojieba-demo)\nSee example in [jieba_test](jieba_test.go), [extractor_test](extractor_test.go)\n\n## Benchmark\n\n[Jieba中文分词系列性能评测]\n\nUnittest\n\n```bash\ngo test ./...\n```\n\nBenchmark\n\n```bash\ngo test -bench \"Jieba\" -test.benchtime 10s\ngo test -bench \"Extractor\" -test.benchtime 10s\n```\n\n## Contributors\n\n### Code Contributors\n\nThis project exists thanks to all the people who contribute.\n\u003ca href=\"https://github.com/yanyiwu/gojieba/graphs/contributors\"\u003e\u003cimg src=\"https://opencollective.com/gojieba/contributors.svg?width=890\u0026button=false\" /\u003e\u003c/a\u003e\n\n[CppJieba]:http://github.com/yanyiwu/cppjieba\n[GoJieba]:http://github.com/yanyiwu/gojieba\n[Jieba]:https://github.com/fxsjy/jieba\n[Jieba中文分词系列性能评测]:http://yanyiwu.com/work/2015/06/14/jieba-series-performance-test.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanyiwu%2Fgojieba","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyanyiwu%2Fgojieba","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanyiwu%2Fgojieba/lists"}