{"id":13935266,"url":"https://github.com/ai-forever/model-zoo","last_synced_at":"2025-04-19T15:12:55.617Z","repository":{"id":72676804,"uuid":"385233179","full_name":"ai-forever/model-zoo","owner":"ai-forever","description":"NLP model zoo for Russian","archived":false,"fork":false,"pushed_at":"2021-07-15T06:31:59.000Z","size":23584,"stargazers_count":45,"open_issues_count":3,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T09:12:07.202Z","etag":null,"topics":["bert","nlp","pytorch","roberta","roberta-model","russian","russian-language","t5","t5-model","transformers"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ai-forever.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-07-12T12:03:23.000Z","updated_at":"2024-06-21T14:48:15.000Z","dependencies_parsed_at":"2024-04-22T23:59:26.956Z","dependency_job_id":null,"html_url":"https://github.com/ai-forever/model-zoo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fmodel-zoo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fmodel-zoo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fmodel-zoo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fmodel-zoo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ai-forever","download_url":"https://codeload.github.com/ai-forever/model-zoo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249723809,"owners_count":21316114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","nlp","pytorch","roberta","roberta-model","russian","russian-language","t5","t5-model","transformers"],"created_at":"2024-08-07T23:01:31.953Z","updated_at":"2025-04-19T15:12:55.563Z","avatar_url":"https://github.com/ai-forever.png","language":null,"funding_links":[],"categories":["Others"],"sub_categories":[],"readme":"# Welcome to the Model Zoo!\n\n### Here you can find NLP models for Russian, implemented in HF [transformers](https://huggingface.co/sberbank-ai/)🤗\n\n[![See Examples In Colab!](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sberbank-ai/model-zoo/blob/master/examples/Sber_ai_examples.ipynb) \n\n\n## Models:\n\n| Model           | Task                 | Type            | Tokenizer | Dict size | Num Parameters  | Training Data Volume |\n|-----------------|----------------------|-----------------|-----------|-----------|-----------------|----------------------|\n| ruBERT-base     | mask filling         | encoder         | bpe       | 120 138   | 178 M         | 30 GB                |\n| ruBERT-large    | mask filling         | encoder         | bpe       | 120 138   | 427 M         | 30 GB                |\n| ruRoBERTa-large | mask filling         | encoder         | bbpe      | 50 257    | 355 M         | 250 GB               |\n| ruT5-base       | text2text generation | encoder-decoder | bpe       | 32101     | 222 M         | 300 GB               |\n| ruT5-large      | text2text generation | encoder-decoder | bpe       | 32101     | 737 M         | 300 GB               |\n\n\n### ruT5\nText2Text Generation task\n[T5 paper](https://arxiv.org/abs/1910.10683)\n - Large: [HF Model](https://huggingface.co/sberbank-ai/ruT5-large)\n - Base: [HF Model](https://huggingface.co/sberbank-ai/ruT5-base)\n\n [Model parameters](https://huggingface.co/transformers/model_doc/t5.html)\n  \n###  ruRoBerta\nfill-mask task\n[Roberta paper](https://arxiv.org/abs/1907.11692)\n- Large: [HF Model](https://huggingface.co/sberbank-ai/ruRoberta-large)\n\n  \n###  ruBert\nfill-mask task\n[Bert paper](https://arxiv.org/abs/1810.04805)\n - Large: [HF Model](https://huggingface.co/sberbank-ai/ruBert-large)\n - Base: [HF Model](https://huggingface.co/sberbank-ai/ruBert-base)\n  \n## How to:\n\nUse this [![Colab!](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sberbank-ai/model-zoo/blob/master/examples/Sber_ai_examples.ipynb) to explore the models or run them on your machine.\n### Model set up:\n```pip install -r requirements.txt```\n\n### Pipeline usage\n```\nfrom transformers import pipeline\n\nunmasker = pipeline(\"fill-mask\", model=\"sberbank-ai/ruRoberta-large\")\nunmasker(\"Евгений Понасенков назвал \u003cmask\u003e величайшим маэстро.\", top_k=1)\n```\n![](/examples/Screenshot%20from%202021-07-07%2002-27-07.png)\n\n### Classical usage\n\n```\n# ruRoberta-large example \nfrom transformers import RobertaForMaskedLM,RobertaTokenizer\n\nmodel=RobertaForMaskedLM.from_pretrained('sberbank-ai/ruRoberta-large')\n\ntokenizer=RobertaTokenizer.from_pretrained('sberbank-ai/ruRoberta-large')\n\nunmasker = pipeline('fill-mask', model=model,tokenizer=tokenizer)\nunmasker(\"Стоит чаще писать на Хабр про \u003cmask\u003e.\")\n```\n\n  \n### Use BertViz to obtain model visualizations \n \n Roberta model_view:\n\n  ![](/examples/roberta_small.gif) / ! [](https://github.com/sberbank-ai/model-zoo/examples/roberta_small.gif)\n\n```\nfrom transformers import RobertaModel, RobertaTokenizer\nfrom bertviz import model_view\n\nmodel_version = 'sberbank-ai/ruRoberta-large'\nmodel = RobertaModel.from_pretrained(model_version, output_attentions=True)\ntokenizer = RobertaTokenizer.from_pretrained(model_version)\n\nsentence_a = \"The cat sat on the mat\"\nsentence_b = \"The cat lay on the rug\"\ninputs = tokenizer.encode_plus(sentence_a, sentence_b, return_tensors='pt', add_special_tokens=True)\ninput_ids = inputs['input_ids']\nattention = model(input_ids)[-1]\ninput_id_list = input_ids[0].tolist() # Batch index 0\ntokens = tokenizer.convert_ids_to_tokens(input_id_list)\nmodel_view(attention, tokens)\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fmodel-zoo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fai-forever%2Fmodel-zoo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fmodel-zoo/lists"}