{"id":15009813,"url":"https://github.com/ikegami-yukino/rakutenma-python","last_synced_at":"2025-10-25T08:34:25.764Z","repository":{"id":25270904,"uuid":"28696366","full_name":"ikegami-yukino/rakutenma-python","owner":"ikegami-yukino","description":"Rakuten MA (Python version)","archived":false,"fork":false,"pushed_at":"2017-05-22T07:45:36.000Z","size":25223,"stargazers_count":22,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-23T19:51:33.270Z","etag":null,"topics":["chinese","japanese-language","nlp","part-of-speech-tagger","pos-tagging","python","word-segmentation"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ikegami-yukino.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-01T21:40:43.000Z","updated_at":"2024-05-25T04:22:49.000Z","dependencies_parsed_at":"2022-09-06T02:00:45.877Z","dependency_job_id":null,"html_url":"https://github.com/ikegami-yukino/rakutenma-python","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikegami-yukino%2Frakutenma-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikegami-yukino%2Frakutenma-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikegami-yukino%2Frakutenma-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikegami-yukino%2Frakutenma-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ikegami-yukino","download_url":"https://codeload.github.com/ikegami-yukino/rakutenma-python/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248083391,"owners_count":21045089,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chinese","japanese-language","nlp","part-of-speech-tagger","pos-tagging","python","word-segmentation"],"created_at":"2024-09-24T19:28:45.699Z","updated_at":"2025-10-25T08:34:20.733Z","avatar_url":"https://github.com/ikegami-yukino.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Rakuten MA Python\n===================\n\n|travis| |coveralls| |pyversion| |version|  |landscape|  |license|\n\nRakuten MA Python (morphological analyzer) is a Python version of Rakuten MA (word segmentor + PoS Tagger) for Chinese and Japanese.\n\nFor details about Rakuten MA, See https://github.com/rakuten-nlp/rakutenma\n\nSee also http://qiita.com/yukinoi/items/925bc238185aa2fad8a7 (In Japanese)\n\nContributions are welcome!\n\n\nInstallation\n==============\n\n::\n\n pip install rakutenma\n\nExample\n===========\n\n.. code:: python\n\n from rakutenma import RakutenMA\n\n # Initialize a RakutenMA instance with an empty model\n # the default ja feature set is set already\n rma = RakutenMA()\n\n # Let's analyze a sample sentence (from http://tatoeba.org/jpn/sentences/show/103809)\n # With a disastrous result, since the model is empty!\n print(rma.tokenize(\"彼は新しい仕事できっと成功するだろう。\"))\n\n # Feed the model with ten sample sentences from tatoeba.com\n # \"tatoeba.json\" is available at https://github.com/rakuten-nlp/rakutenma\n import json\n tatoeba = json.load(open(\"tatoeba.json\"))\n for i in tatoeba:\n     rma.train_one(i)\n\n # Now what does the result look like?\n print(rma.tokenize(\"彼は新しい仕事できっと成功するだろう。\"))\n\n # Initialize a RakutenMA instance with a pre-trained model\n rma = RakutenMA(phi=1024, c=0.007812)  # Specify hyperparameter for SCW (for demonstration purpose)\n rma.load(\"model_ja.json\")\n\n # Set the feature hash function (15bit)\n rma.hash_func = rma.create_hash_func(15)\n\n # Tokenize one sample sentence\n print(rma.tokenize(\"うらにわにはにわにわとりがいる\"));\n\n # Re-train the model feeding the right answer (pairs of [token, PoS tag])\n res = rma.train_one(\n        [[\"うらにわ\",\"N-nc\"],\n         [\"に\",\"P-k\"],\n         [\"は\",\"P-rj\"],\n         [\"にわ\",\"N-n\"],\n         [\"にわとり\",\"N-nc\"],\n         [\"が\",\"P-k\"],\n         [\"いる\",\"V-c\"]])\n # The result of train_one contains:\n #   sys: the system output (using the current model)\n #   ans: answer fed by the user\n #   update: whether the model was updated\n print(res)\n\n # Now what does the result look like?\n print(rma.tokenize(\"うらにわにはにわにわとりがいる\"))\n\n\nNOTE\n===========\n\nAdded API\n--------------\nAs compared to original RakutenMA, following methods are added:\n\n- RakutenMA::load(model_path)\n  - Load model from JSON file\n\n- RakutenMA::save(model_path)\n  - Save model to path\n\nmisc\n--------------\nAs initial setting, following values are set:\n\n- rma.featset = CTYPE_JA_PATTERNS  # RakutenMA.default_featset_ja\n- rma.hash_func = rma.create_hash_func(15)\n- rma.tag_scheme = \"SBIEO\"  # if using Chinese, set \"IOB2\"\n\nLICENSE\n=========\n\nApache License version 2.0\n\n\nCopyright\n=============\n\nRakuten MA Python\n(c) 2015- Yukino Ikegami. All Rights Reserved.\n\nRakuten MA (original)\n(c) 2014 Rakuten NLP Project. All Rights Reserved.\n\n.. |travis| image:: https://travis-ci.org/ikegami-yukino/rakutenma-python.svg?branch=master\n    :target: https://travis-ci.org/ikegami-yukino/rakutenma-python\n    :alt: travis-ci.org\n.. |coveralls| image:: https://coveralls.io/repos/ikegami-yukino/rakutenma-python/badge.png\n    :target: https://coveralls.io/r/ikegami-yukino/rakutenma-python\n    :alt: coveralls.io\n\n.. |pyversion| image:: https://img.shields.io/pypi/pyversions/rakutenma.svg\n\n.. |version| image:: https://img.shields.io/pypi/v/rakutenma.svg\n    :target: http://pypi.python.org/pypi/rakutenma/\n    :alt: latest version\n\n.. |landscape| image:: https://landscape.io/github/ikegami-yukino/rakutenma-python/master/landscape.svg?style=flat\n   :target: https://landscape.io/github/ikegami-yukino/rakutenma-python/master\n   :alt: Code Health\n\n.. |license| image:: https://img.shields.io/pypi/l/rakutenma.svg\n    :target: http://pypi.python.org/pypi/rakutenma/\n    :alt: license\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikegami-yukino%2Frakutenma-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fikegami-yukino%2Frakutenma-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikegami-yukino%2Frakutenma-python/lists"}