{"id":13496997,"url":"https://github.com/guillaume-be/rust-bert","last_synced_at":"2025-05-14T08:05:19.223Z","repository":{"id":39652054,"uuid":"236150741","full_name":"guillaume-be/rust-bert","owner":"guillaume-be","description":"Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)","archived":false,"fork":false,"pushed_at":"2025-03-06T03:47:39.000Z","size":4264,"stargazers_count":2835,"open_issues_count":65,"forks_count":223,"subscribers_count":40,"default_branch":"main","last_synced_at":"2025-05-07T08:00:05.927Z","etag":null,"topics":["bart","bert","deep-learning","electra","gpt","gpt-2","language-generation","machine-learning","ner","nlp","question-answering","roberta","rust","rust-lang","sentiment-analysis","transformer","translation"],"latest_commit_sha":null,"homepage":"https://docs.rs/crate/rust-bert","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guillaume-be.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-01-25T09:40:07.000Z","updated_at":"2025-05-05T16:17:58.000Z","dependencies_parsed_at":"2023-09-26T21:41:11.504Z","dependency_job_id":"a5ea17c1-95ac-4654-a74a-e210f5c97fb0","html_url":"https://github.com/guillaume-be/rust-bert","commit_stats":{"total_commits":955,"total_committers":40,"mean_commits":23.875,"dds":0.0596858638743456,"last_synced_commit":"c3a3f394685e0fda1566beee5cdb9ceaf363a99b"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guillaume-be%2Frust-bert","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guillaume-be%2Frust-bert/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guillaume-be%2Frust-bert/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guillaume-be%2Frust-bert/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guillaume-be","download_url":"https://codeload.github.com/guillaume-be/rust-bert/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254101588,"owners_count":22014907,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bart","bert","deep-learning","electra","gpt","gpt-2","language-generation","machine-learning","ner","nlp","question-answering","roberta","rust","rust-lang","sentiment-analysis","transformer","translation"],"created_at":"2024-07-31T20:00:20.518Z","updated_at":"2025-05-14T08:05:19.200Z","avatar_url":"https://github.com/guillaume-be.png","language":"Rust","funding_links":[],"categories":["Models \u0026 Inference","Rust","NLP","BERT优化","Libraries","sentiment-analysis"],"sub_categories":["Artificial Intelligence","General-Purpose Machine Learning","Books"],"readme":"# rust-bert\n\n[![Build Status](https://github.com/guillaume-be/rust-bert/workflows/Build/badge.svg?event=push)](https://github.com/guillaume-be/rust-bert/actions)\n[![Latest version](https://img.shields.io/crates/v/rust_bert.svg)](https://crates.io/crates/rust_bert)\n[![Documentation](https://docs.rs/rust-bert/badge.svg)](https://docs.rs/rust-bert)\n![License](https://img.shields.io/crates/l/rust_bert.svg)\n\nRust-native state-of-the-art Natural Language Processing models and pipelines.\nPort of Hugging Face's\n[Transformers library](https://github.com/huggingface/transformers), using\n[tch-rs](https://github.com/LaurentMazare/tch-rs) or\n[onnxruntime bindings](https://github.com/pykeio/ort) and pre-processing from\n[rust-tokenizers](https://github.com/guillaume-be/rust-tokenizers). Supports\nmulti-threaded tokenization and GPU inference. This repository exposes the model\nbase architecture, task-specific heads (see below) and\n[ready-to-use pipelines](#ready-to-use-pipelines). [Benchmarks](#benchmarks) are\navailable at the end of this document.\n\nGet started with tasks including question answering, named entity recognition,\ntranslation, summarization, text generation, conversational agents and more in\njust a few lines of code:\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Where does Amy live ?\");\nlet context = String::from(\"Amy lives in Amsterdam\");\n\nlet answers = qa_model.predict( \u0026 [QaInput { question, context }], 1, 32);\n```\n\nOutput:\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\nThe tasks currently supported include:\n\n- Translation\n- Summarization\n- Multi-turn dialogue\n- Zero-shot classification\n- Sentiment Analysis\n- Named Entity Recognition\n- Part of Speech tagging\n- Question-Answering\n- Language Generation\n- Masked Language Model\n- Sentence Embeddings\n- Keywords extraction\n\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003eExpand to display the supported models/tasks matrix \u003c/b\u003e \u003c/summary\u003e\n\n|              | **Sequence classification** | **Token classification** | **Question answering** | **Text Generation** | **Summarization** | **Translation** | **Masked LM** | **Sentence Embeddings** |\n|:------------:|:---------------------------:|:------------------------:|:----------------------:|:-------------------:|:-----------------:|:---------------:|:-------------:|:-----------------------:|\n|  DistilBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|  MobileBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   DeBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n| DeBERTa (v2) |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     FNet     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     BERT     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|   RoBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|     GPT      |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     GPT2     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   GPT-Neo    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|    GPT-J     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     BART     |              ✅              |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    Marian    |                             |                          |                        |                     |                   |        ✅        |               |                         |\n|    MBart     |              ✅              |                          |                        |          ✅          |                   |                 |               |                         |\n|    M2M100    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     NLLB     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   Electra    |                             |            ✅             |                        |                     |                   |                 |       ✅       |                         |\n|    ALBERT    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|      T5      |                             |                          |                        |          ✅          |         ✅         |        ✅        |               |            ✅            |\n|    LongT5    |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    XLNet     |              ✅              |            ✅             |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|   Reformer   |              ✅              |                          |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|  ProphetNet  |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|  Longformer  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   Pegasus    |                             |                          |                        |                     |         ✅         |                 |               |                         |\n\n\u003c/details\u003e\n\n## Getting started\n\nThis library relies on the [tch](https://github.com/LaurentMazare/tch-rs) crate\nfor bindings to the C++ Libtorch API. The libtorch library is required can be\ndownloaded either automatically or manually. The following provides a reference\non how to set-up your environment to use these bindings, please refer to the\n[tch](https://github.com/LaurentMazare/tch-rs) for detailed information or\nsupport.\n\nFurthermore, this library relies on a cache folder for downloading pre-trained\nmodels. This cache location defaults to `~/.cache/.rustbert`, but can be changed\nby setting the `RUSTBERT_CACHE` environment variable. Note that the language\nmodels used by this library are in the order of the 100s of MBs to GBs.\n\n### Manual installation (recommended)\n\n1. Download `libtorch` from https://pytorch.org/get-started/locally/. This\n   package requires `v2.4`: if this version is no longer available on the \"get\n   started\" page, the file should be accessible by modifying the target link,\n   for example\n   `https://download.pytorch.org/libtorch/cu124/libtorch-cxx11-abi-shared-with-deps-2.4.0%2Bcu124.zip`\n   for a Linux version with CUDA12. **NOTE:** When using `rust-bert` as\n   dependency from [crates.io](https://crates.io), please check the required\n   `LIBTORCH` on the published package\n   [readme](https://crates.io/crates/rust-bert) as it may differ from the\n   version documented here (applying to the current repository version).\n2. Extract the library to a location of your choice\n3. Set the following environment variables\n\n##### Linux:\n\n```bash\nexport LIBTORCH=/path/to/libtorch\nexport LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH\n```\n\n##### Windows\n\n```powershell\n$Env:LIBTORCH = \"X:\\path\\to\\libtorch\"\n$Env:Path += \";X:\\path\\to\\libtorch\\lib\"\n```\n\n#### macOS + Homebrew\n\n```bash\nbrew install pytorch jq\nexport LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version')\nexport LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH\n```\n\n### Automatic installation\n\nAlternatively, you can let the `build` script automatically download the\n`libtorch` library for you. The `download-libtorch` feature flag needs to be\nenabled. The CPU version of libtorch will be downloaded by default. To download\na CUDA version, please set the environment variable `TORCH_CUDA_VERSION` to\n`cu124`. Note that the libtorch library is large (order of several GBs for the\nCUDA-enabled version) and the first build may therefore take several minutes to\ncomplete.\n\n### Verifying installation\n\nVerify your installation (and linking with libtorch) by adding the `rust-bert`\ndependency to your `Cargo.toml` or by cloning the rust-bert source and running\nan example:\n\n```bash\ngit clone git@github.com:guillaume-be/rust-bert.git\ncd rust-bert\ncargo run --example sentence_embeddings\n```\n\n## ONNX Support (Optional)\n\nONNX support can be enabled via the optional `onnx` feature. This crate then\nleverages the [ort](https://github.com/pykeio/ort) crate with bindings to the\nonnxruntime C++ library. We refer the user to this page project for further\ninstallation instructions/support.\n\n1. Enable the optional `onnx` feature. The `rust-bert` crate does not include\n   any optional dependencies for `ort`, the end user should select the set of\n   features that would be adequate for pulling the required `onnxruntime` C++\n   library.\n2. The current recommended installation is to use dynamic linking by pointing to\n   an existing library location. Use the `load-dynamic` cargo feature for `ort`.\n3. set the `ORT_DYLIB_PATH` to point to the location of downloaded onnxruntime\n   library (`onnxruntime.dll`/`libonnxruntime.so`/`libonnxruntime.dylib`\n   depending on the operating system). These can be downloaded from the\n   [release page](https://github.com/microsoft/onnxruntime/releases) of the\n   onnxruntime project\n\nMost architectures (including encoders, decoders and encoder-decoders) are\nsupported. the library aims at keeping compatibility with models exported using\nthe [Optimum](https://github.com/huggingface/optimum) library. A detailed guide\non how to export a Transformer model to ONNX using Optimum is available at\nhttps://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model\nThe resources used to create ONNX models are similar to those based on Pytorch,\nreplacing the pytorch by the ONNX model. Since ONNX models are less flexible\nthan their Pytorch counterparts in the handling of optional arguments, exporting\na decoder or encoder-decoder model to ONNX will usually result in multiple\nfiles. These files are expected (but not all are necessary) for use in this\nlibrary as per the table below:\n\n| Architecture                | Encoder file | Decoder without past file | Decoder with past file |\n|-----------------------------|--------------|---------------------------|------------------------|\n| Encoder (e.g. BERT)         | required     | not used                  | not used               |\n| Decoder (e.g. GPT2)         | not used     | required                  | optional               |\n| Encoder-decoder (e.g. BART) | required     | required                  | optional               |\n\nNote that the computational efficiency will drop when the `decoder with past`\nfile is optional but not provided since the model will not used cached past keys\nand values for the attention mechanism, leading to a high number of redundant\ncomputations. The Optimum library offers export options to ensure such a\n`decoder with past` model file is created. The base encoder and decoder model\narchitecture are available (and exposed for convenience) in the `encoder` and\n`decoder` modules, respectively.\n\nGeneration models (pure decoder or encoder/decoder architectures) are available\nin the `models` module. ost pipelines are available for ONNX model checkpoints,\nincluding sequence classification, zero-shot classification, token\nclassification (including named entity recognition and part-of-speech tagging),\nquestion answering, text generation, summarization and translation. These models\nuse the same configuration and tokenizer files as their Pytorch counterparts\nwhen used in a pipeline. Examples leveraging ONNX models are given in the\n`./examples` directory\n\n## Ready-to-use pipelines\n\nBased on Hugging Face's pipelines, ready to use end-to-end NLP pipelines are\navailable as part of this crate. The following capabilities are currently\navailable:\n\n**Disclaimer** The contributors of this repository are not responsible for any\ngeneration from the 3rd party utilization of the pretrained systems proposed\nherein.\n\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e1. Question Answering\u003c/b\u003e \u003c/summary\u003e\n\nExtractive question answering from a given question and context. DistilBERT\nmodel fine-tuned on SQuAD (Stanford Question Answering Dataset)\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Where does Amy live ?\");\nlet context = String::from(\"Amy lives in Amsterdam\");\n\nlet answers = qa_model.predict( \u0026 [QaInput { question, context }], 1, 32);\n```\n\nOutput:\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e2. Translation \u003c/b\u003e \u003c/summary\u003e\n\nTranslation pipeline supporting a broad range of source and target languages.\nLeverages two main architectures for translation tasks:\n\n- Marian-based models, for specific source/target combinations\n- M2M100 models allowing for direct translation between 100 languages (at a\n  higher computational cost and lower performance for some selected languages)\n\nMarian-based pretrained models for the following language pairs are readily\navailable in the library - but the user can import any Pytorch-based model for\npredictions\n\n- English \u003c-\u003e French\n- English \u003c-\u003e Spanish\n- English \u003c-\u003e Portuguese\n- English \u003c-\u003e Italian\n- English \u003c-\u003e Catalan\n- English \u003c-\u003e German\n- English \u003c-\u003e Russian\n- English \u003c-\u003e Chinese\n- English \u003c-\u003e Dutch\n- English \u003c-\u003e Swedish\n- English \u003c-\u003e Arabic\n- English \u003c-\u003e Hebrew\n- English \u003c-\u003e Hindi\n- French \u003c-\u003e German\n\nFor languages not supported by the proposed pretrained Marian models, the user\ncan leverage a M2M100 model supporting direct translation between 100 languages\n(without intermediate English translation) The full list of supported languages\nis available in the\n[crate documentation](https://docs.rs/rust-bert/latest/rust_bert/pipelines/translation/enum.Language.html)\n\n```rust\nuse rust_bert::pipelines::translation::{Language, TranslationModelBuilder};\nfn main() -\u003e anyhow::Result\u003c()\u003e {\n    let model = TranslationModelBuilder::new()\n        .with_source_languages(vec![Language::English])\n        .with_target_languages(vec![Language::Spanish, Language::French, Language::Italian])\n        .create_model()?;\n    let input_text = \"This is a sentence to be translated\";\n    let output = model.translate(\u0026[input_text], None, Language::French)?;\n    for sentence in output {\n        println!(\"{}\", sentence);\n    }\n    Ok(())\n}\n```\n\nOutput:\n\n```\nIl s'agit d'une phrase à traduire\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e3. Summarization \u003c/b\u003e \u003c/summary\u003e\n\nAbstractive summarization using a pretrained BART model.\n\n```rust\n    let summarization_model = SummarizationModel::new(Default::default ()) ?;\n\nlet input = [\"In findings published Tuesday in Cornell University's arXiv by a team of scientists \\\nfrom the University of Montreal and a separate report published Wednesday in Nature Astronomy by a team \\\nfrom University College London (UCL), the presence of water vapour was confirmed in the atmosphere of K2-18b, \\\na planet circling a star in the constellation Leo. This is the first such discovery in a planet in its star's \\\nhabitable zone — not too hot and not too cold for liquid water to exist. The Montreal team, led by Björn Benneke, \\\nused data from the NASA's Hubble telescope to assess changes in the light coming from K2-18b's star as the planet \\\npassed between it and Earth. They found that certain wavelengths of light, which are usually absorbed by water, \\\nweakened when the planet was in the way, indicating not only does K2-18b have an atmosphere, but the atmosphere \\\ncontains water in vapour form. The team from UCL then analyzed the Montreal team's data using their own software \\\nand confirmed their conclusion. This was not the first time scientists have found signs of water on an exoplanet, \\\nbut previous discoveries were made on planets with high temperatures or other pronounced differences from Earth. \\\n\\\"This is the first potentially habitable planet where the temperature is right and where we now know there is water,\\\" \\\nsaid UCL astronomer Angelos Tsiaras. \\\"It's the best candidate for habitability right now.\\\" \\\"It's a good sign\\\", \\\nsaid Ryan Cloutier of the Harvard–Smithsonian Center for Astrophysics, who was not one of either study's authors. \\\n\\\"Overall,\\\" he continued, \\\"the presence of water in its atmosphere certainly improves the prospect of K2-18b being \\\na potentially habitable planet, but further observations will be required to say for sure. \\\"\nK2-18b was first identified in 2015 by the Kepler space telescope. It is about 110 light-years from Earth and larger \\\nbut less dense. Its star, a red dwarf, is cooler than the Sun, but the planet's orbit is much closer, such that a year \\\non K2-18b lasts 33 Earth days. According to The Guardian, astronomers were optimistic that NASA's James Webb space \\\ntelescope — scheduled for launch in 2021 — and the European Space Agency's 2028 ARIEL program, could reveal more \\\nabout exoplanets like K2-18b.\"];\n\nlet output = summarization_model.summarize( \u0026 input);\n```\n\n(example from:\n[WikiNews](https://en.wikinews.org/wiki/Astronomers_find_water_vapour_in_atmosphere_of_exoplanet_K2-18b))\n\nOutput:\n\n```\n\"Scientists have found water vapour on K2-18b, a planet 110 light-years from Earth. \nThis is the first such discovery in a planet in its star's habitable zone. \nThe planet is not too hot and not too cold for liquid water to exist.\"\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e4. Dialogue Model \u003c/b\u003e \u003c/summary\u003e\n\nConversation model based on Microsoft's\n[DialoGPT](https://github.com/microsoft/DialoGPT). This pipeline allows the\ngeneration of single or multi-turn conversations between a human and a model.\nThe DialoGPT's page states that\n\n\u003e The human evaluation results indicate that the response generated from\n\u003e DialoGPT is comparable to human response quality under a single-turn\n\u003e conversation Turing test.\n\u003e ([DialoGPT repository](https://github.com/microsoft/DialoGPT))\n\nThe model uses a `ConversationManager` to keep track of active conversations and\ngenerate responses to them.\n\n```rust\nuse rust_bert::pipelines::conversation::{ConversationModel, ConversationManager};\n\nlet conversation_model = ConversationModel::new(Default::default ());\nlet mut conversation_manager = ConversationManager::new();\n\nlet conversation_id = conversation_manager.create(\"Going to the movies tonight - any suggestions?\");\nlet output = conversation_model.generate_responses( \u0026 mut conversation_manager);\n```\n\nExample output:\n\n```\n\"The Big Lebowski.\"\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e5. Natural Language Generation \u003c/b\u003e \u003c/summary\u003e\n\nGenerate language based on a prompt. GPT2 and GPT available as base models.\nInclude techniques such as beam search, top-k and nucleus sampling, temperature\nsetting and repetition penalty. Supports batch generation of sentences from\nseveral prompts. Sequences will be left-padded with the model's padding token if\npresent, the unknown token otherwise. This may impact the results, it is\nrecommended to submit prompts of similar length for best results\n\n```rust\n    let model = GPT2Generator::new(Default::default ()) ?;\n\nlet input_context_1 = \"The dog\";\nlet input_context_2 = \"The cat was\";\n\nlet generate_options = GenerateOptions {\nmax_length: 30,\n..Default::default ()\n};\n\nlet output = model.generate(Some( \u0026 [input_context_1, input_context_2]), generate_options);\n```\n\nExample output:\n\n```\n[\n    \"The dog's owners, however, did not want to be named. According to the lawsuit, the animal's owner, a 29-year\"\n    \"The dog has always been part of the family. \\\"He was always going to be my dog and he was always looking out for me\"\n    \"The dog has been able to stay in the home for more than three months now. \\\"It's a very good dog. She's\"\n    \"The cat was discovered earlier this month in the home of a relative of the deceased. The cat\\'s owner, who wished to remain anonymous,\"\n    \"The cat was pulled from the street by two-year-old Jazmine.\\\"I didn't know what to do,\\\" she said\"\n    \"The cat was attacked by two stray dogs and was taken to a hospital. Two other cats were also injured in the attack and are being treated.\"\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e6. Zero-shot classification \u003c/b\u003e \u003c/summary\u003e\n\nPerforms zero-shot classification on input sentences with provided labels using\na model fine-tuned for Natural Language Inference.\n\n```rust\n    let sequence_classification_model = ZeroShotClassificationModel::new(Default::default ()) ?;\n\nlet input_sentence = \"Who are you voting for in 2020?\";\nlet input_sequence_2 = \"The prime minister has announced a stimulus package which was widely criticized by the opposition.\";\nlet candidate_labels = \u0026 [\"politics\", \"public health\", \"economics\", \"sports\"];\n\nlet output = sequence_classification_model.predict_multilabel(\n\u0026 [input_sentence, input_sequence_2],\ncandidate_labels,\nNone,\n128,\n);\n```\n\nOutput:\n\n```\n[\n  [ Label { \"politics\", score: 0.972 }, Label { \"public health\", score: 0.032 }, Label {\"economics\", score: 0.006 }, Label {\"sports\", score: 0.004 } ],\n  [ Label { \"politics\", score: 0.975 }, Label { \"public health\", score: 0.0818 }, Label {\"economics\", score: 0.852 }, Label {\"sports\", score: 0.001 } ],\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e7. Sentiment analysis \u003c/b\u003e \u003c/summary\u003e\n\nPredicts the binary sentiment for a sentence. DistilBERT model fine-tuned on\nSST-2.\n\n```rust\n    let sentiment_classifier = SentimentModel::new(Default::default ()) ?;\n\nlet input = [\n\"Probably my all-time favorite movie, a story of selflessness, sacrifice and dedication to a noble cause, but it's not preachy or boring.\",\n\"This film tried to be too many things all at once: stinging political satire, Hollywood blockbuster, sappy romantic comedy, family values promo...\",\n\"If you like original gut wrenching laughter you will like this movie. If you are young or old then you will love this movie, hell even my mom liked it.\",\n];\n\nlet output = sentiment_classifier.predict( \u0026 input);\n```\n\n(Example courtesy of [IMDb](http://www.imdb.com))\n\nOutput:\n\n```\n[\n    Sentiment { polarity: Positive, score: 0.9981985493795946 },\n    Sentiment { polarity: Negative, score: 0.9927982091903687 },\n    Sentiment { polarity: Positive, score: 0.9997248985164333 }\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e8. Named Entity Recognition \u003c/b\u003e \u003c/summary\u003e\n\nExtracts entities (Person, Location, Organization, Miscellaneous) from text.\nBERT cased large model fine-tuned on CoNNL03, contributed by the\n[MDZ Digital Library team at the Bavarian State Library](https://github.com/dbmdz).\nModels are currently available for English, German, Spanish and Dutch.\n\n```rust\n    let ner_model = NERModel::new( default::default ()) ?;\n\nlet input = [\n\"My name is Amy. I live in Paris.\",\n\"Paris is a city in France.\"\n];\n\nlet output = ner_model.predict( \u0026 input);\n```\n\nOutput:\n\n```\n[\n  [\n    Entity { word: \"Amy\", score: 0.9986, label: \"I-PER\" }\n    Entity { word: \"Paris\", score: 0.9985, label: \"I-LOC\" }\n  ],\n  [\n    Entity { word: \"Paris\", score: 0.9988, label: \"I-LOC\" }\n    Entity { word: \"France\", score: 0.9993, label: \"I-LOC\" }\n  ]\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e9. Keywords/keyphrases extraction\u003c/b\u003e \u003c/summary\u003e\n\nExtract keywords and keyphrases extractions from input documents\n\n```rust\nfn main() -\u003e anyhow::Result\u003c()\u003e {\n    let keyword_extraction_model = KeywordExtractionModel::new(Default::default())?;\n\n    let input = \"Rust is a multi-paradigm, general-purpose programming language. \\\n       Rust emphasizes performance, type safety, and concurrency. Rust enforces memory safety—that is, \\\n       that all references point to valid memory—without requiring the use of a garbage collector or \\\n       reference counting present in other memory-safe languages. To simultaneously enforce \\\n       memory safety and prevent concurrent data races, Rust's borrow checker tracks the object lifetime \\\n       and variable scope of all references in a program during compilation. Rust is popular for \\\n       systems programming but also offers high-level features including functional programming constructs.\";\n\n    let output = keyword_extraction_model.predict(\u0026[input])?;\n}\n```\n\nOutput:\n\n```\n\"rust\" - 0.50910604\n\"programming\" - 0.35731024\n\"concurrency\" - 0.33825397\n\"concurrent\" - 0.31229728\n\"program\" - 0.29115444\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e10. Part of Speech tagging \u003c/b\u003e \u003c/summary\u003e\n\nExtracts Part of Speech tags (Noun, Verb, Adjective...) from text.\n\n```rust\n    let pos_model = POSModel::new( default::default ()) ?;\n\nlet input = [\"My name is Bob\"];\n\nlet output = pos_model.predict( \u0026 input);\n```\n\nOutput:\n\n```\n[\n    Entity { word: \"My\", score: 0.1560, label: \"PRP\" }\n    Entity { word: \"name\", score: 0.6565, label: \"NN\" }\n    Entity { word: \"is\", score: 0.3697, label: \"VBZ\" }\n    Entity { word: \"Bob\", score: 0.7460, label: \"NNP\" }\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e11. Sentence embeddings \u003c/b\u003e \u003c/summary\u003e\n\nGenerate sentence embeddings (vector representation). These can be used for\napplications including dense information retrieval.\n\n```rust\n    let model = SentenceEmbeddingsBuilder::remote(\nSentenceEmbeddingsModelType::AllMiniLmL12V2\n).create_model() ?;\n\nlet sentences = [\n\"this is an example sentence\",\n\"each sentence is converted\"\n];\n\nlet output = model.encode( \u0026 sentences) ?;\n```\n\nOutput:\n\n```\n[\n    [-0.000202666, 0.08148022, 0.03136178, 0.002920636 ...],\n    [0.064757116, 0.048519745, -0.01786038, -0.0479775 ...]\n]\n```\n\n\u003c/details\u003e\n\u0026nbsp;\n\u003cdetails\u003e\n\u003csummary\u003e \u003cb\u003e12. Masked Language Model \u003c/b\u003e \u003c/summary\u003e\n\nPredict masked words in input sentences.\n\n```rust\n    let model = MaskedLanguageModel::new(Default::default ()) ?;\n\nlet sentences = [\n\"Hello I am a \u003cmask\u003e student\",\n\"Paris is the \u003cmask\u003e of France. It is \u003cmask\u003e in Europe.\",\n];\n\nlet output = model.predict( \u0026 sentences);\n```\n\nOutput:\n\n```\n[\n    [MaskedToken { text: \"college\", id: 2267, score: 8.091}],\n    [\n        MaskedToken { text: \"capital\", id: 3007, score: 16.7249}, \n        MaskedToken { text: \"located\", id: 2284, score: 9.0452}\n    ]\n]\n```\n\n\u003c/details\u003e\n\n## Benchmarks\n\nFor simple pipelines (sequence classification, tokens classification, question\nanswering) the performance between Python and Rust is expected to be comparable.\nThis is because the most expensive part of these pipeline is the language model\nitself, sharing a common implementation in the Torch backend. The\n[End-to-end NLP Pipelines in Rust](https://www.aclweb.org/anthology/2020.nlposs-1.4/)\nprovides a benchmarks section covering all pipelines.\n\nFor text generation tasks (summarization, translation, conversation, free text\ngeneration), significant benefits can be expected (up to 2 to 4 times faster\nprocessing depending on the input and application). The article\n[Accelerating text generation with Rust](https://guillaume-be.github.io/2020-11-21/generation_benchmarks)\nfocuses on these text generation applications and provides more details on the\nperformance comparison to Python.\n\n## Loading pretrained and custom model weights\n\nThe base model and task-specific heads are also available for users looking to\nexpose their own transformer based models. Examples on how to prepare the date\nusing a native tokenizers Rust library are available in `./examples` for BERT,\nDistilBERT, RoBERTa, GPT, GPT2 and BART. Note that when importing models from\nPytorch, the convention for parameters naming needs to be aligned with the Rust\nschema. Loading of the pre-trained weights will fail if any of the model\nparameters weights cannot be found in the weight files. If this quality check is\nto be skipped, an alternative method `load_partial` can be invoked from the\nvariables store.\n\nPretrained models are available on Hugging face's\n[model hub](https://huggingface.co/models?filter=rust) and can be loaded using\n`RemoteResources` defined in this library.\n\nA conversion utility script is included in `./utils` to convert Pytorch weights\nto a set of weights compatible with this library. This script requires Python\nand `torch` to be set-up, and can be used as follows:\n`python ./utils/convert_model.py path/to/pytorch_model.bin` where\n`path/to/pytorch_model.bin` is the location of the original Pytorch weights.\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\n\npip install -r requirements.txt\n\npython utils/convert_model.py path/to/pytorch_model.bin\n```\n\n## Citation\n\nIf you use `rust-bert` for your work, please cite\n[End-to-end NLP Pipelines in Rust](https://www.aclweb.org/anthology/2020.nlposs-1.4/):\n\n```bibtex\n@inproceedings{becquin-2020-end,\n    title = \"End-to-end {NLP} Pipelines in Rust\",\n    author = \"Becquin, Guillaume\",\n    booktitle = \"Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)\",\n    year = \"2020\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://www.aclweb.org/anthology/2020.nlposs-1.4\",\n    pages = \"20--25\",\n}\n```\n\n## Acknowledgements\n\nThank you to [Hugging Face](https://huggingface.co) for hosting a set of weights\ncompatible with this Rust library. The list of ready-to-use pretrained models is\nlisted at\n[https://huggingface.co/models?filter=rust](https://huggingface.co/models?filter=rust).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguillaume-be%2Frust-bert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguillaume-be%2Frust-bert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguillaume-be%2Frust-bert/lists"}