{"id":20630377,"url":"https://github.com/koichiyasuoka/esupar","last_synced_at":"2025-04-06T12:07:52.226Z","repository":{"id":40407984,"uuid":"407694835","full_name":"KoichiYasuoka/esupar","owner":"KoichiYasuoka","description":"Tokenizer POS-Tagger and Dependency-parser with BERT/RoBERTa/DeBERTa/GPT models for Japanese and other languages","archived":false,"fork":false,"pushed_at":"2025-03-28T08:10:50.000Z","size":216,"stargazers_count":50,"open_issues_count":1,"forks_count":6,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-30T11:07:20.830Z","etag":null,"topics":["ainu","chinese","classical-chinese","coptic","english","german","japanese","natural-language-processing","nlp","serbian","thai","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KoichiYasuoka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-17T22:28:30.000Z","updated_at":"2025-03-28T08:10:53.000Z","dependencies_parsed_at":"2023-11-08T11:25:24.214Z","dependency_job_id":"0aaee949-b5a2-4565-a3f4-d61afa80e4c1","html_url":"https://github.com/KoichiYasuoka/esupar","commit_stats":{"total_commits":210,"total_committers":1,"mean_commits":210.0,"dds":0.0,"last_synced_commit":"7e0c30edd1f7a370814e84f81ab67668ea4075bb"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KoichiYasuoka%2Fesupar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KoichiYasuoka%2Fesupar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KoichiYasuoka%2Fesupar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KoichiYasuoka%2Fesupar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KoichiYasuoka","download_url":"https://codeload.github.com/KoichiYasuoka/esupar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247478320,"owners_count":20945266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ainu","chinese","classical-chinese","coptic","english","german","japanese","natural-language-processing","nlp","serbian","thai","transformers"],"created_at":"2024-11-16T14:07:42.351Z","updated_at":"2025-04-06T12:07:52.205Z","avatar_url":"https://github.com/KoichiYasuoka.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Current PyPI packages](https://badge.fury.io/py/esupar.svg)](https://pypi.org/project/esupar/)\n\n# esupar\n\nTokenizer, POS-tagger, and dependency-parser with [Transformers](https://huggingface.co/transformers/) and [SuPar](https://pypi.org/project/supar/).\n\n## Basic usage\n\n```py\n\u003e\u003e\u003e import esupar\n\u003e\u003e\u003e nlp=esupar.load(\"ja\")\n\u003e\u003e\u003e doc=nlp(\"太郎は花子が読んでいる本を次郎に渡した\")\n\u003e\u003e\u003e print(doc)\n1\t太郎\t_\tPROPN\t_\t_\t12\tnsubj\t_\tSpaceAfter=No\n2\tは\t_\tADP\t_\t_\t1\tcase\t_\tSpaceAfter=No\n3\t花子\t_\tPROPN\t_\t_\t5\tnsubj\t_\tSpaceAfter=No\n4\tが\t_\tADP\t_\t_\t3\tcase\t_\tSpaceAfter=No\n5\t読ん\t_\tVERB\t_\t_\t8\tacl\t_\tSpaceAfter=No\n6\tで\t_\tSCONJ\t_\t_\t5\tmark\t_\tSpaceAfter=No\n7\tいる\t_\tAUX\t_\t_\t5\taux\t_\tSpaceAfter=No\n8\t本\t_\tNOUN\t_\t_\t12\tobj\t_\tSpaceAfter=No\n9\tを\t_\tADP\t_\t_\t8\tcase\t_\tSpaceAfter=No\n10\t次郎\t_\tPROPN\t_\t_\t12\tobl\t_\tSpaceAfter=No\n11\tに\t_\tADP\t_\t_\t10\tcase\t_\tSpaceAfter=No\n12\t渡し\t_\tVERB\t_\t_\t0\troot\t_\tSpaceAfter=No\n13\tた\t_\tAUX\t_\t_\t12\taux\t_\t_\n\n\u003e\u003e\u003e import deplacy\n\u003e\u003e\u003e deplacy.render(doc,Japanese=True)\n太郎 PROPN ═╗\u003c════════╗ nsubj(主語)\nは   ADP   \u003c╝         ║ case(格表示)\n花子 PROPN ═╗\u003c══╗     ║ nsubj(主語)\nが   ADP   \u003c╝   ║     ║ case(格表示)\n読ん VERB  ═╗═╗═╝\u003c╗   ║ acl(連体修飾節)\nで   SCONJ \u003c╝ ║   ║   ║ mark(標識)\nいる AUX   \u003c══╝   ║   ║ aux(動詞補助成分)\n本   NOUN  ═╗═════╝\u003c╗ ║ obj(目的語)\nを   ADP   \u003c╝       ║ ║ case(格表示)\n次郎 PROPN ═╗\u003c╗     ║ ║ obl(斜格補語)\nに   ADP   \u003c╝ ║     ║ ║ case(格表示)\n渡し VERB  ═╗═╝═════╝═╝ root(親)\nた   AUX   \u003c╝           aux(動詞補助成分)\n```\n\n`esupar.load(model)` loads a natural language processor pipeline, working on [Universal Dependencies](https://universaldependencies.org/format.html). Available `model` options are:\n\n* `model=\"ja\"` Japanese model [bert-base-japanese-upos](https://huggingface.co/KoichiYasuoka/bert-base-japanese-upos) (default)\n* `model=\"ja_large\"` Japanese model [bert-large-japanese-upos](https://huggingface.co/KoichiYasuoka/bert-large-japanese-upos)\n* `model=\"ja_luw_small\"` Japanese long-unit-word model [roberta-small-japanese-char-luw-upos](https://huggingface.co/KoichiYasuoka/roberta-small-japanese-char-luw-upos)\n* `model=\"ja_luw_base\"` Japanese long-unit-word model [bert-base-japanese-luw-upos](https://huggingface.co/KoichiYasuoka/bert-base-japanese-luw-upos)\n* `model=\"ja_luw_large\"` Japanese long-unit-word model [bert-large-japanese-luw-upos](https://huggingface.co/KoichiYasuoka/bert-large-japanese-luw-upos)\n* `model=\"ko\"` Korean model [roberta-base-korean-upos](https://huggingface.co/KoichiYasuoka/roberta-base-korean-upos)\n* `model=\"ko_large\"` Korean model [roberta-large-korean-upos](https://huggingface.co/KoichiYasuoka/roberta-large-korean-upos)\n* `model=\"ko_morph_base\"` Korean morpheme model [roberta-base-korean-morph-upos](https://huggingface.co/KoichiYasuoka/roberta-base-korean-morph-upos)\n* `model=\"ko_morph_large\"` Korean morpheme model [roberta-large-korean-morph-upos](https://huggingface.co/KoichiYasuoka/roberta-large-korean-morph-upos)\n* `model=\"zh\"` Chinese model [chinese-bert-wwm-ext-upos](https://huggingface.co/KoichiYasuoka/chinese-bert-wwm-ext-upos)\n* `model=\"zh_base\"` Chinese model [chinese-roberta-base-upos](https://huggingface.co/KoichiYasuoka/chinese-roberta-base-upos)\n* `model=\"zh_large\"` Chinese model [chinese-roberta-large-upos](https://huggingface.co/KoichiYasuoka/chinese-roberta-large-upos)\n* `model=\"lzh\"` Classical Chinese model [roberta-classical-chinese-base-upos](https://huggingface.co/KoichiYasuoka/roberta-classical-chinese-base-upos)\n* `model=\"lzh_large\"` Classical Chinese model [roberta-classical-chinese-large-upos](https://huggingface.co/KoichiYasuoka/roberta-classical-chinese-large-upos)\n* `model=\"th\"` Thai model [roberta-base-thai-spm-upos](https://huggingface.co/KoichiYasuoka/roberta-base-thai-spm-upos)\n* `model=\"vi\"` Vietnamese model [bert-base-vietnamese-upos](https://huggingface.co/KoichiYasuoka/bert-base-vietnamese-upos)\n* `model=\"en\"` English model [roberta-base-english-upos](https://huggingface.co/KoichiYasuoka/roberta-base-english-upos)\n* `model=\"en_large\"` English model [roberta-large-english-upos](https://huggingface.co/KoichiYasuoka/roberta-large-english-upos)\n* `model=\"de\"` German model [bert-base-german-upos](https://huggingface.co/KoichiYasuoka/bert-base-german-upos)\n* `model=\"de_large\"` German model [bert-large-german-upos](https://huggingface.co/KoichiYasuoka/bert-large-german-upos)\n* `model=\"sr\"` Serbian (Cyrillic and Latin) model [gpt2-small-serbian-upos](https://huggingface.co/KoichiYasuoka/gpt2-small-serbian-upos)\n* `model=\"sr_large\"` Serbian (Cyrillic and Latin) model [gpt2-large-serbian-upos](https://huggingface.co/KoichiYasuoka/gpt2-large-serbian-upos)\n* `model=\"cop\"` Coptic model [roberta-base-coptic-upos](https://huggingface.co/KoichiYasuoka/roberta-base-coptic-upos)\n* `model=\"ain\"` Ainu model [roberta-base-ainu-upos](https://huggingface.co/KoichiYasuoka/roberta-base-ainu-upos)\n\n## Installation for Linux\n\n```sh\npip3 install esupar --user\n```\n\n## Installation for Cygwin64\n\nMake sure to get `python37-devel` `python37-pip` `python37-cython` `python37-numpy` `python37-wheel` `gcc-g++` `mingw64-x86_64-gcc-g++` `git` `curl` `make` `cmake`, and then:\n\n```sh\ncurl -L https://raw.githubusercontent.com/KoichiYasuoka/CygTorch/master/installer/supar.sh | sh\npip3.7 install esupar\n```\n\n## Installation for Google Colaboratory\n\n```py\n!pip install esupar\n```\n\nTry [notebook](https://colab.research.google.com/github/KoichiYasuoka/esupar/blob/master/esupar.ipynb).\n\n## Author\n\nKoichi Yasuoka (安岡孝一)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoichiyasuoka%2Fesupar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkoichiyasuoka%2Fesupar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoichiyasuoka%2Fesupar/lists"}