{"id":13484042,"url":"https://github.com/Droidtown/ArticutAPI","last_synced_at":"2025-03-27T15:31:04.027Z","repository":{"id":41390565,"uuid":"183543778","full_name":"Droidtown/ArticutAPI","owner":"Droidtown","description":"API of Articut 中文斷詞 (兼具語意詞性標記)：「斷詞」又稱「分詞」，是中文資訊處理的基礎。Articut 不用機器學習，不需資料模型，只用現代白話中文語法規則，即能達到 SIGHAN 2005 F1-measure 94% 以上，Recall 96% 以上的成績。","archived":false,"fork":false,"pushed_at":"2024-08-23T02:40:56.000Z","size":51686,"stargazers_count":407,"open_issues_count":0,"forks_count":36,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-10-30T04:43:48.888Z","etag":null,"topics":["artificial-intelligence","cws","natural-language-processing","natural-language-understanding","nlp","nlu","part-of-speech-embdding","part-of-speech-tagger","pos-tagger","pos-tagging"],"latest_commit_sha":null,"homepage":"https://api.droidtown.co","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Droidtown.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-26T02:38:33.000Z","updated_at":"2024-10-16T02:01:40.000Z","dependencies_parsed_at":"2024-11-13T15:35:14.821Z","dependency_job_id":null,"html_url":"https://github.com/Droidtown/ArticutAPI","commit_stats":{"total_commits":405,"total_committers":8,"mean_commits":50.625,"dds":0.6987654320987654,"last_synced_commit":"b2a86ab2a21be2cf855749413a207d3015c70b7b"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Droidtown%2FArticutAPI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Droidtown%2FArticutAPI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Droidtown%2FArticutAPI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Droidtown%2FArticutAPI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Droidtown","download_url":"https://codeload.github.com/Droidtown/ArticutAPI/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245871763,"owners_count":20686255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","cws","natural-language-processing","natural-language-understanding","nlp","nlu","part-of-speech-embdding","part-of-speech-tagger","pos-tagger","pos-tagging"],"created_at":"2024-07-31T17:01:18.519Z","updated_at":"2025-03-27T15:30:59.019Z","avatar_url":"https://github.com/Droidtown.png","language":"Python","funding_links":[],"categories":["python"],"sub_categories":["collection"],"readme":"# Articut 中文斷詞暨詞性標記服務\n## [依語法結構計算，而非統計方法的中文斷詞。]\n\n### [Articut API Website](https://api.droidtown.co/)\n### [Document](https://api.droidtown.co/ArticutAPI/document/)\n### [![Articut Demo](https://img.youtube.com/vi/AnvdKmVLlcA/0.jpg)](https://youtu.be/AnvdKmVLlcA \"Articut Demo\")\n\n## Benchmark\n### 設計目標\n名稱        | ArticutAPI          | MP_ArticutAPI   | WS_ArticutAPI | \n:----------:|:-------------------:|:-------------------:|:------------------:|\n產品        | Online / Docker  | Docker                | Docker              |\n技術        | HTTP Request    | MultiProcessing | WebSocket        |\n特色        | 簡單易用             | 批次處理             | 即時處理            |\n適用情景 | 任何                    | 文本分析             | 聊天機器人         |\n\n### 處理速度\n名稱         | ArticutAPI | MP_ArticutAPI | WS_ArticutAPI | \n:----------:|-------------:|-------------------:|-------------------:|\n時間         |  0.1252 秒 |         0.1206 秒 |          0.0677 秒 |\n\n### 大量文本\n句數         | ArticutAPI  | MP_ArticutAPI | WS_ArticutAPI |\n:----------:|--------------:|-------------------:|------------------:|\n方法        |        parse() |  bulk_parse(20) |              parse() |\n1K           |        155 秒 |                   8 秒 |                18 秒 |\n2K           |        306 秒 |                 14 秒 |                35 秒 |\n3K           |        455 秒 |                 17 秒 |                43 秒 |\n\n- 測試平台爲 4 核心 CPU 並使用 4 個 Process。\n- `MP_ArticutAPI`使用 bulk_parse(bulkSize=20) 方法。\n- `WS_ArticutAPI`使用 parse() 方法。\n\n----------------------\n\n# ArticutAPI\n\n## 安裝方法\n```sh\npip3 install ArticutAPI\n```\n\n## 說明文件\n函數說明請參閱 Docs/index.html\n\n## 使用方法\n### Articut CWS (Chinese word segmentation)\n```\nfrom ArticutAPI import Articut\nfrom pprint import pprint\nusername = \"\" #這裡填入您在 https://api.droidtown.co 使用的帳號 email。若使用空字串，則預設使用每小時 2000 字的公用額度。\napikey   = \"\" #這裡填入您在 https://api.droidtown.co 登入後取得的 api Key。若使用空字串，則預設使用每小時 2000 字的公用額度。\narticut = Articut(username, apikey)\ninputSTR = \"會被大家盯上，才證明你有實力。\"\nresultDICT = articut.parse(inputSTR)\npprint(resultDICT)\n```\n### 回傳結果\n```\n{\"exec_time\": 0.06723856925964355,\n \"level\": \"lv2\",\n \"msg\": \"Success!\",\n \n \"result_pos\": [\"\u003cMODAL\u003e會\u003c/MODAL\u003e\u003cACTION_lightVerb\u003e被\u003c/ACTION_lightVerb\u003e\u003cENTITY_nouny\u003e大家\u003c/ENTITY_nouny\u003e\u003cACTION_verb\u003e盯上\u003c/ACTION_verb\u003e\",\n                \"，\",\n                \"\u003cMODAL\u003e才\u003c/MODAL\u003e\u003cACTION_verb\u003e證明\u003c/ACTION_verb\u003e\u003cENTITY_pronoun\u003e你\u003c/ENTITY_pronoun\u003e\u003cACTION_verb\u003e有\u003c/ACTION_verb\u003e\u003cENTITY_noun\u003e實力\u003c/ENTITY_noun\u003e\",\n                \"。\"],\n \"result_segmentation\": \"會/被/大家/盯上/，/才/證明/你/有/實力/。/\",\n \"status\": True,\n \"version\": \"v118\",\n \"word_count_balance\": 9985,\n \"product\": \"https://api.droidtown.co/product/\",\n \"document\": \"https://api.droidtown.co/document/\"\n}\n```\n\n### 列出斷詞結果所有詞性標記的內容詞 ###\n可以依需求找出「名詞」、「動詞」或是「形容詞」…等詞彙語意本身已經完整的詞彙。\n```\ninputSTR = \"你計劃過地球人類補完計劃\"\nresultDICT = articut.parse(inputSTR, level=\"lv1\")\npprint(resultDICT[\"result_pos\"])\n\n#列出所有的 content word.\ncontentWordLIST = articut.getContentWordLIST(resultDICT)\npprint(contentWordLIST)\n\n#列出所有的 verb word. (動詞)\nverbStemLIST = articut.getVerbStemLIST(resultDICT)\npprint(verbStemLIST)\n\n#列出所有的 noun word. (名詞)\nnounStemLIST = articut.getNounStemLIST(resultDICT)\npprint(nounStemLIST)\n\n#列出所有的 location word. (地方名稱)\nlocationStemLIST = articut.getLocationStemLIST(resultDICT)\npprint(locationStemLIST)\n```\n\n### 回傳結果 ###\n```\n#resultDICT[\"result_pos\"]\n[\"\u003cENTITY_pronoun\u003e你\u003c/ENTITY_pronoun\u003e\u003cACTION_verb\u003e計劃\u003c/ACTION_verb\u003e\u003cASPECT\u003e過\u003c/ASPECT\u003e\u003cLOCATION\u003e地球\u003c/LOCATION\u003e\u003cENTITY_oov\u003e人類\u003c/ENTITY_oov\u003e\u003cACTION_verb\u003e補完\u003c/ACTION_verb\u003e\u003cENTITY_nounHead\u003e計劃\u003c/ENTITY_nounHead\u003e\"]\n\n#列出所有的 content word.\n[[(47, 49, '計劃'), (117, 119, '人類'), (146, 147, '補'), (196, 198, '計劃')]]\n\n#列出所有的 verb word. (動詞)\n[[(47, 49, '計劃'), (146, 147, '補')]]\n\n#列出所有的 noun word. (名詞)\n[[(117, 119, '人類'), (196, 198, '計劃')]]\n\n#列出所有的 location word. (地方名稱)\n[[(91, 93, '地球')]]\n```\n\n### 取得 Articut 版本列表\n```\nresultDICT = articut.versions()\npprint(resultDICT)\n```\n### 回傳結果\n```\n{\"msg\": \"Success!\",\n \"status\": True,\n \"versions\": [{\"level\": [\"lv1\", \"lv2\"],\n               \"release_date\": \"2019-04-25\",\n               \"version\": \"latest\"},\n              {\"level\": [\"lv1\", \"lv2\"],\n               \"release_date\": \"2019-04-25\",\n               \"version\": \"v118\"},\n              {\"level\": [\"lv1\", \"lv2\"],\n               \"release_date\": \"2019-04-24\",\n               \"version\": \"v117\"},...\n}\n```\n\n----------------------\n\n## 進階用法\n### 進階用法01 \u003e\u003e Articut Level :斷詞的深度。數字愈小，切得愈細 (預設: lv2)。\n```\ninputSTR = \"小紅帽\"\nresultDICT = articut.parse(inputSTR, level=\"lv1\")\npprint(resultDICT)\n```\n### 回傳結果 lv1 \n極致斷詞，適合 NLU 或機器自動翻譯使用。呈現結果將句子中的每個元素都儘量細分出來。\n```\n{\"exec_time\": 0.04814624786376953,\n \"level\": \"lv1\",\n \"msg\": \"Success!\",\n \"result_pos\": [\"\u003cMODIFIER\u003e小\u003c/MODIFIER\u003e\u003cMODIFIER_color\u003e紅\u003c/MODIFIER_color\u003e\u003cENTITY_nounHead\u003e帽\u003c/ENTITY_nounHead\u003e\"],\n \"result_segmentation\": \"小/紅/帽/\",\n \"status\": True,\n \"version\": \"v118\",\n \"word_count_balance\": 9997,...}\n```\n\n### 回傳結果 lv2 \n詞組斷詞，適合文本分析、特徵值計算、關鍵字擷取…等應用。呈現結果將以具意義的最小單位呈現。\n```\n{\"exec_time\": 0.04195523262023926,\n \"level\": \"lv2\",\n \"msg\": \"Success!\",\n \"result_pos\": [\"\u003cENTITY_nouny\u003e小紅帽\u003c/ENTITY_nouny\u003e\"],\n \"result_segmentation\": \"小紅帽/\",\n \"status\": True,\n \"version\": \"v118\",\n \"word_count_balance\": 9997,...}\n```\n\n----------------------\n### 進階用法 02 \u003e\u003e UserDefinedDictFile :使用者自定詞典。\n[![Articut UserDefined Demo](http://i3.ytimg.com/vi/fOyyQyVkZ2k/hqdefault.jpg)](https://youtu.be/fOyyQyVkZ2k \"Articut UserDefined Demo\")\n\n因為 Articut 只處理「語言知識」而不處理「百科知識」。\n我們提供「使用者自定義」詞彙表的功能，使用 Dictionary 格式，請自行編寫。\n\nUserDefinedFile.json\n```\n{\"雷姆\":[\"小老婆\"],\n \"艾蜜莉亞\":[\"大老婆\"],\n \"初音未來\": [\"初音\", \"只是個軟體\"],\n \"李敏鎬\": [\"全民歐巴\", \"歐巴\"]}\n```\n\nrunArticut.py\n```\nfrom ArticutAPI import Articut\nfrom pprint import pprint\n\narticut = Articut()\nuserDefined = \"./UserDefinedFile.json\"\ninputSTR = \"我的最愛是小老婆，不是初音未來。\"\n\n# 使用自定義詞典\nresultDICT = articut.parse(inputSTR, userDefinedDictFILE=userDefined)\npprint(resultDICT)\n\n# 未使用自定義詞典\nresultDICT = articut.parse(inputSTR)\npprint(resultDICT)\n```\n\n### 回傳結果\n```\n# 使用自定義詞典\n{\"result_pos\": [\"\u003cENTITY_pronoun\u003e我\u003c/ENTITY_pronoun\u003e\u003cFUNC_inner\u003e的\u003c/FUNC_inner\u003e\u003cACTION_verb\u003e最愛\u003c/ACTION_verb\u003e\u003cAUX\u003e是\u003c/AUX\u003e\u003cUserDefined\u003e小老婆\u003c/UserDefined\u003e\",\n                \"，\",\n                \"\u003cFUNC_negation\u003e不\u003c/FUNC_negation\u003e\u003cAUX\u003e是\u003c/AUX\u003e\u003cUserDefined\u003e初音未來\u003c/UserDefined\u003e\",\n                \"。\"],\n \"result_segmentation\": \"我/的/最愛/是/小老婆/，/不/是/初音未來/。/\",...}\n\n# 未使用自定義詞典\n{\"result_pos\": [\"\u003cENTITY_pronoun\u003e我\u003c/ENTITY_pronoun\u003e\u003cFUNC_inner\u003e的\u003c/FUNC_inner\u003e\u003cACTION_verb\u003e最愛\u003c/ACTION_verb\u003e\u003cAUX\u003e是\u003c/AUX\u003e\u003cENTITY_nouny\u003e小老婆\u003c/ENTITY_nouny\u003e\",\n                \"，\",\n                \"\u003cFUNC_negation\u003e不\u003c/FUNC_negation\u003e\u003cAUX\u003e是\u003c/AUX\u003e\u003cENTITY_nouny\u003e初音\u003c/ENTITY_nouny\u003e\u003cTIME_justtime\u003e未來\u003c/TIME_justtime\u003e\",\n                \"。\"],\n \"result_segmentation\": \"我/的/最愛/是/小老婆/，/不/是/初音/未來/。/\",...}\n```\n----------------------\n### 進階用法 03 - 調用資料觀光資訊資料庫\n政府開放平台中存有「交通部觀光局蒐集各政府機關所發佈空間化觀光資訊」。Articut 可取用其中的資訊，並標記為 \\\u003cKNOWLEDGE_place\u003e\n\n**上傳內容 (JSON 格式)**\n```\n{\n\t\"username\": \"test@email.com\",\n\t\"api_key\": \"anapikeyfordocthatdoesnwork@all\",\n\t\"input_str\": \"花蓮的原野牧場有一間餐廳\",\n\t\"version\": \"v137\",\n\t\"level\": \"lv1\",\n\t\"opendata_place\": true\n}\n```\n\n**回傳內容 (JSON 格式)**\n```\n{\n\t\"exec_time\": 0.013453006744384766,\n\t\"level\": \"lv1\",\n\t\"msg\": \"Success!\",\n\t\"result_pos\": [\"\u003cLOCATION\u003e花蓮\u003c/LOCATION\u003e\u003cFUNC_inner\u003e的\u003c/FUNC_inner\u003e\u003cKNOWLEDGE_place\u003e原野牧場\u003c/KNOWLEDGE_place\u003e\u003cACTION_verb\u003e有\u003c/ACTION_verb\u003e\u003cENTITY_classifier\u003e一間\u003c/ENTITY_classifier\u003e\u003cENTITY_noun\u003e餐廳\u003c/ENTITY_noun\u003e\"],\n\t\"result_segmentation\": \"花蓮/的/原野牧場/有/一間/餐廳/\",\n\t\"status\": True,\n\t\"version\": \"v137\",\n\t\"word_count_balance\": 99987\n}\n```\n----------------------\n\n### 進階用法 04 - 基於 TF-IDF 算法的關鍵詞抽取\n\n* articut.analyse.extract_tags(sentence, topK=20, withWeight=False, allowPOS=())\n\t* sentence 為要提取關鍵詞的文本\n\t* topK 為提取幾個 TF-IDF 的關鍵詞，預設值為 20\n\t* withWeight 為是否返回關鍵詞權重值，預設值為 False\n\t* allowPOS 僅抽取指定詞性的詞，預設值為空，亦即全部抽取\n* articut.analyse.TFIDF(idf\\_path=None) 新建 TFIDF 物件，idf_path 為 IDF 語料庫路徑\n\n使用範例：\n\u003chttps://github.com/Droidtown/ArticutAPI/blob/master/ArticutAPI.py#L624\u003e\n\n---\n\n### 進階用法 05 - 基於 TextRank 算法的關鍵詞抽取\n\n* articut.analyse.textrank(sentence, topK=20, withWeight=False, allowPOS=())\n\t* sentence 為要提取關鍵詞的文本\n\t* topK 為提取幾個 TF-IDF 的關鍵詞，預設值為 20\n\t* withWeight 為是否返回關鍵詞權重值，預設值為 False\n\t* allowPOS 僅抽取指定詞性的詞，預設值為空，亦即全部抽取\n* articut.analyse.TextRank() 新建 TextRank 物件\n\n算法論文：[TextRank: Bringing Order into Texts](http://web.eecs.umich.edu/~mihalcea/papers/mihalcea.emnlp04.pdf)\n\n### 基本思想：\n\n1. 將待抽取關鍵詞的文本斷詞\n2. 以固定的窗格大小 (預設值為 5，通過 span 屬性調整)，詞之間的共現關係，建構出不帶權圖\n3. 計算途中節點的 PageRank\n\n使用範例：\n\u003chttps://github.com/Droidtown/ArticutAPI/blob/master/ArticutAPI.py#L629\u003e\n\n---\n\n### 進階用法 06 - 使用 GraphQL 查詢斷詞結果\n[![Watch the video](Screenshots/Articut-GraphQL.png)](https://youtu.be/0Ubx08WJ7rU)\n\n### 使用 GraphiQL 工具\n\n**環境需求**\n\n```\nPython 3.6.1\n$ pip install graphene\n$ pip install starlette\n$ pip install jinja2\n$ pip install uvicorn\n```\n\n執行 ArticutGraphQL.py 帶入 Articut 斷詞結果檔案路徑，並開啟瀏覽器輸入網址 http://0.0.0.0:8000/\n\n```\n$ python ArticutGraphQL.py articutResult.json\n```\n\n### 使用範例 01\n![GraphiQL Example 01](Screenshots/GraphiQL_Example01.png)\n\n### 使用範例 02\n![GraphiQL Example 02](Screenshots/GraphiQL_Example02.png)\n\n### 使用 Articut-GraphQL\n\n安裝 graphene 模組\n\n```\n$ pip install graphene\n```\n### 使用範例 01\n```\ninputSTR = \"地址：宜蘭縣宜蘭市縣政北七路六段55巷1號2樓\"\nresult = articut.parse(inputSTR)\nwith open(\"articutResult.json\", \"w\", encoding=\"utf-8\") as resultFile:\n    json.dump(result, resultFile, ensure_ascii=False)\n\t\ngraphQLResult = articut.graphQL.query(\n    filePath=\"articutResult.json\",\n    query=\"\"\"\n\t{\n\t  meta {\n\t    lang\n\t    description\n\t  }\n\t  doc {\n\t    text\n\t    tokens {\n\t      text\n\t      pos_\n\t      tag_\n\t      isStop\n\t      isEntity\n\t      isVerb\n\t      isTime\n\t      isClause\n\t      isKnowledge\n\t    }\n\t  }\n\t}\"\"\")\npprint(graphQLResult)\n```\n\n### 回傳結果\n![Articut-GraphQL Example 01](Screenshots/Articut-GraphQL_Example01.png)\n\n### 使用範例 02\n```\ninputSTR = \"劉克襄在本次活動當中，分享了台北中山北路一日遊路線。他表示當初自己領著柯文哲一同探索了雙連市場與中山捷運站的小吃與商圈，還有商圈內的文創商店與日系雜物店鋪，都令柯文哲留下深刻的印象。劉克襄也認為，雙連市場內的魯肉飯、圓仔湯與切仔麵，還有九條通的日式店家、居酒屋等特色，也能讓人感受到台北舊城區不一樣的魅力。\"\nresult = articut.parse(inputSTR)\nwith open(\"articutResult.json\", \"w\", encoding=\"utf-8\") as resultFile:\n    json.dump(result, resultFile, ensure_ascii=False)\n\t\ngraphQLResult = articut.graphQL.query(\n    filePath=\"articutResult.json\",\n    query=\"\"\"\n\t{\n\t  meta {\n\t    lang\n\t    description\n\t  }\n\t  doc {\n\t    text\n\t    ents {\n\t      persons {\n\t        text\n\t        pos_\n\t        tag_\n\t      }\n\t    }\n\t  }\n\t}\"\"\")\npprint(graphQLResult)\n```\n\t\n### 回傳結果\n![Articut-GraphQL 回傳結果2](Screenshots/Articut-GraphQL_Example02.png)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDroidtown%2FArticutAPI","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDroidtown%2FArticutAPI","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDroidtown%2FArticutAPI/lists"}