{"id":20543190,"url":"https://github.com/opennmt/opennmt-py","last_synced_at":"2025-10-08T17:12:07.161Z","repository":{"id":37848259,"uuid":"82841862","full_name":"OpenNMT/OpenNMT-py","owner":"OpenNMT","description":"Open Source Neural Machine Translation and (Large) Language Models in PyTorch","archived":false,"fork":false,"pushed_at":"2025-03-06T03:47:21.000Z","size":321653,"stargazers_count":6941,"open_issues_count":33,"forks_count":2257,"subscribers_count":177,"default_branch":"master","last_synced_at":"2025-09-22T18:37:11.159Z","etag":null,"topics":["deep-learning","language-model","llms","machine-translation","neural-machine-translation","pytorch"],"latest_commit_sha":null,"homepage":"https://opennmt.net/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenNMT.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-02-22T19:01:50.000Z","updated_at":"2025-09-22T00:34:44.000Z","dependencies_parsed_at":"2022-07-14T23:30:47.033Z","dependency_job_id":"e2a5c9d7-74f6-4993-996d-446ac53788c4","html_url":"https://github.com/OpenNMT/OpenNMT-py","commit_stats":{"total_commits":2276,"total_committers":204,"mean_commits":"11.156862745098039","dds":0.8536906854130053,"last_synced_commit":"aa06c4c202acb653a977c09772d3fb9069824c1b"},"previous_names":[],"tags_count":50,"template":false,"template_full_name":null,"purl":"pkg:github/OpenNMT/OpenNMT-py","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenNMT%2FOpenNMT-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenNMT%2FOpenNMT-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenNMT%2FOpenNMT-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenNMT%2FOpenNMT-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenNMT","download_url":"https://codeload.github.com/OpenNMT/OpenNMT-py/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenNMT%2FOpenNMT-py/sbom","scorecard":{"id":105447,"data":{"date":"2025-08-11","repo":{"name":"github.com/OpenNMT/OpenNMT-py","commit":"97111d97551c24857076a4102eabdb468b35cff4"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.4,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":3,"reason":"Found 10/28 approved changesets -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/build.yml:1","Warn: no topLevel permission defined: .github/workflows/deploy_docs.yml:1","Warn: no topLevel permission defined: .github/workflows/push.yml:1","Warn: no topLevel permission defined: .github/workflows/release.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.md:0","Info: FSF or OSI recognized license: MIT License: LICENSE.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: third-party GitHubAction not pinned by hash: .github/workflows/build.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/build.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/build.yml:39: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/build.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/build.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/build.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/deploy_docs.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/deploy_docs.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/deploy_docs.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/deploy_docs.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/deploy_docs.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/deploy_docs.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/push.yml:576: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/push.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/push.yml:578: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/push.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/push.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/push.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/push.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/push.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/release.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/release.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/release.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:37: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/release.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:39: update your workflow using https://app.stepsecurity.io/secureworkflow/OpenNMT/OpenNMT-py/release.yml/master?enable=pin","Warn: containerImage not pinned by hash: docker/Dockerfile:2","Warn: pipCommand not pinned by hash: docker/Dockerfile:12","Warn: pipCommand not pinned by hash: docker/Dockerfile:13","Warn: pipCommand not pinned by hash: docker/Dockerfile:16","Warn: pipCommand not pinned by hash: docker/Dockerfile:23","Warn: pipCommand not pinned by hash: docker/Dockerfile:25-31","Warn: pipCommand not pinned by hash: docker/Dockerfile:34","Warn: pipCommand not pinned by hash: docker/Dockerfile:37","Warn: pipCommand not pinned by hash: docker/Dockerfile:41","Warn: pipCommand not pinned by hash: docker/Dockerfile:42","Warn: pipCommand not pinned by hash: .github/workflows/deploy_docs.yml:33","Warn: pipCommand not pinned by hash: .github/workflows/deploy_docs.yml:34","Warn: pipCommand not pinned by hash: .github/workflows/deploy_docs.yml:35","Warn: pipCommand not pinned by hash: .github/workflows/deploy_docs.yml:36","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:21","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:22","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:23","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:24","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:25","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:26","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:583","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:584","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:585","Warn: pipCommand not pinned by hash: .github/workflows/push.yml:586","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:18","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:19","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:20","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:21","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:44","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:45","Info:   0 out of  11 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of   5 third-party GitHubAction dependencies pinned","Info:   0 out of   1 containerImage dependencies pinned","Info:   0 out of  31 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":0,"reason":"45 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-cpwx-vrp4-4pq7","Warn: Project is vulnerable to: GHSA-gmj6-6f8f-6699","Warn: Project is vulnerable to: GHSA-q2x7-8rv6-6q7h","Warn: Project is vulnerable to: GHSA-26jh-r8g2-6fpr","Warn: Project is vulnerable to: PYSEC-2024-219 / GHSA-279j-x4gx-hfrh","Warn: Project is vulnerable to: GHSA-34rf-p3r3-58x2","Warn: Project is vulnerable to: PYSEC-2024-197 / GHSA-37qc-qgx6-9xjv","Warn: Project is vulnerable to: PYSEC-2024-196 / GHSA-3c67-5hwx-f6wx","Warn: Project is vulnerable to: GHSA-3f95-mxq2-2f63","Warn: Project is vulnerable to: GHSA-3gf9-wv65-gwh9","Warn: Project is vulnerable to: PYSEC-2023-90 / GHSA-3qqg-pgqq-3695","Warn: Project is vulnerable to: PYSEC-2023-16 / GHSA-3x5j-9vwr-8rr5","Warn: Project is vulnerable to: GHSA-48cq-79qq-6f7x","Warn: Project is vulnerable to: PYSEC-2024-217 / GHSA-4q3c-cj7g-jcwf","Warn: Project is vulnerable to: PYSEC-2024-215 / GHSA-576c-3j53-r9jj","Warn: Project is vulnerable to: GHSA-5cpq-9538-jm2j","Warn: Project is vulnerable to: PYSEC-2023-249 / GHSA-6qm2-wpxq-7qh2","Warn: Project is vulnerable to: PYSEC-2024-184 / GHSA-6v6g-j5fq-hpvw","Warn: Project is vulnerable to: PYSEC-2024-213 / GHSA-77xq-6g77-h274","Warn: Project is vulnerable to: GHSA-7v2w-h4gh-w5cv","Warn: Project is vulnerable to: PYSEC-2024-214 / GHSA-89v2-pqfv-c5r9","Warn: Project is vulnerable to: PYSEC-2024-216 / GHSA-8c87-gvhj-xm8m","Warn: Project is vulnerable to: GHSA-8jw3-6x8j-v96g","Warn: Project is vulnerable to: GHSA-973g-55hp-3frw","Warn: Project is vulnerable to: GHSA-f3h9-8phc-6gvh","Warn: Project is vulnerable to: PYSEC-2022-229 / GHSA-f8xq-q7px-wg8c","Warn: Project is vulnerable to: GHSA-g6c9-f4xm-9j4x","Warn: Project is vulnerable to: GHSA-g9cj-cfpp-4g2x","Warn: Project is vulnerable to: PYSEC-2023-255 / GHSA-gqvf-3hgp-5hxv","Warn: Project is vulnerable to: PYSEC-2024-220 / GHSA-gvv6-33j7-884g","Warn: Project is vulnerable to: PYSEC-2024-198 / GHSA-hm3c-93pg-4cxw","Warn: Project is vulnerable to: GHSA-hmx6-r76c-85g9","Warn: Project is vulnerable to: GHSA-j2jg-fq62-7c3h","Warn: Project is vulnerable to: PYSEC-2024-199 / GHSA-j757-pf57-f8r4","Warn: Project is vulnerable to: GHSA-m842-4qm8-7gpq","Warn: Project is vulnerable to: GHSA-prpg-p95c-32fv","Warn: Project is vulnerable to: GHSA-qh6x-j82h-vpf9","Warn: Project is vulnerable to: GHSA-r364-m2j9-mf4h","Warn: Project is vulnerable to: PYSEC-2021-873 / GHSA-rhq2-3vr9-6mcr","Warn: Project is vulnerable to: PYSEC-2024-255 / GHSA-rvfh-h6c7-fc3c","Warn: Project is vulnerable to: GHSA-v4q9-qgqf-7jwp","Warn: Project is vulnerable to: PYSEC-2024-218 / GHSA-xh2x-3mrm-fwqm","Warn: Project is vulnerable to: PYSEC-2019-156 / GHSA-xp76-357g-9wqq","Warn: Project is vulnerable to: PYSEC-2023-102","Warn: Project is vulnerable to: PYSEC-2023-114"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 25 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-15T11:00:19.535Z","repository_id":37848259,"created_at":"2025-08-15T11:00:19.535Z","updated_at":"2025-08-15T11:00:19.535Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278981518,"owners_count":26079640,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","language-model","llms","machine-translation","neural-machine-translation","pytorch"],"created_at":"2024-11-16T01:36:13.544Z","updated_at":"2025-10-08T17:12:07.145Z","avatar_url":"https://github.com/OpenNMT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Announcement: OpenNMT-py is no longer actively supported.\n\nWe started a new project [Eole](https://eole-nlp.github.io/eole/) available on [Github](https://github.com/eole-nlp/eole)\n\nIt is a spin-off of OpenNMT-py in terms of features but we revamped a lot of stuff.\n\nEole handles NMT, LLM, Encoders as well as a new concept of Estimator within a NMT Model See this [post](https://medium.com/p/05b00b271a47) and this [news](https://www.linkedin.com/posts/vincentnguyenngoc_embarrassingly-small-english-to-german-model-activity-7203400634727841792-FCre?utm_source=share\u0026utm_medium=member_desktop)\n\nIf you are a developer, switch now. If you are a user only, then we will publish the first py-pi versions shortly.\n\n\n# OpenNMT-py: Open-Source Neural Machine Translation and (Large) Language Models\n\n[![Build Status](https://github.com/OpenNMT/OpenNMT-py/workflows/Lint%20\u0026%20Tests/badge.svg)](https://github.com/OpenNMT/OpenNMT-py/actions)\n[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://opennmt.net/OpenNMT-py/)\n[![Gitter](https://badges.gitter.im/OpenNMT/OpenNMT-py.svg)](https://gitter.im/OpenNMT/OpenNMT-py?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge)\n[![Forum](https://img.shields.io/discourse/status?server=https%3A%2F%2Fforum.opennmt.net%2F)](https://forum.opennmt.net/)\n\nOpenNMT-py is the [PyTorch](https://github.com/pytorch/pytorch) version of the [OpenNMT](https://opennmt.net) project, an open-source (MIT) neural machine translation (and beyond!) framework. It is designed to be research friendly to try out new ideas in translation, language modeling, summarization, and many other NLP tasks. Some companies have proven the code to be production ready.\n\nWe love contributions! Please look at issues marked with the [contributions welcome](https://github.com/OpenNMT/OpenNMT-py/issues?q=is%3Aissue+is%3Aopen+label%3A%22contributions+welcome%22) tag.\n\nBefore raising an issue, make sure you read the requirements and the [Full Documentation](https://opennmt.net/OpenNMT-py/) examples.\n\nUnless there is a bug, please use the [Forum](https://forum.opennmt.net) or [Gitter](https://gitter.im/OpenNMT/OpenNMT-py) to ask questions.\n\n----\n## For beginners:\n\nThere is a step-by-step and explained tuto (Thanks to Yasmin Moslem): [Tutorial](https://github.com/ymoslem/OpenNMT-Tutorial)\n\nPlease try to read and/or follow before raising newbies issues.\n\nOtherwise you can just have a look at the [Quickstart](https://opennmt.net/OpenNMT-py/quickstart.html) steps\n\n----\n## New:\n\n* You will need Pytorch v2 preferably v2.2 which fixes some `scaled_dot_product_attention` issues\n* LLM support with converters for: Llama (+ Mistral), OpenLlama, Redpajama, MPT-7B, Falcon.\n* Support for 8bit and 4bit quantization along with LoRA adapters, with or without checkpointing.\n* You can finetune 7B and 13B models on a single RTX 24GB with 4-bit quantization.\n* Inference can be forced in 4/8bit using the same layer quantization as in finetuning.\n* Tensor parallelism when the model does not fit on one GPU's memory (both training and inference)\n* Once your model is finetuned you can run inference either with OpenNMT-py or faster with CTranslate2.\n* MMLU evaluation script, see results [here](https://github.com/OpenNMT/OpenNMT-py/blob/master/eval_llm/MMLU/readme.md)\n\nFor all usecases including NMT, you can now use Multiquery instead of Multihead attention (faster at training and inference) and remove biases from all Linear (QKV as well as FeedForward modules).\n\n\nIf you used previous versions of OpenNMT-py, you can check the [Changelog](https://github.com/OpenNMT/OpenNMT-py/blob/master/CHANGELOG.md) or the [Breaking Changes](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/changes.md)\n\n----\n\n## Tutorials:\n\n* How to replicate Vicuna with a 7B or 13B llama (or Open llama, MPT-7B, Redpajama)  Language Model: [Tuto Vicuna](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/replicate_vicuna/ReplicateVicuna.md)\n* How to finetune NLLB-200 with your dataset: [Tuto Finetune NLLB-200](https://forum.opennmt.net/t/finetuning-and-curating-nllb-200-with-opennmt-py/5238)\n* How to create a simple OpenNMT-py REST Server: [Tuto REST](https://forum.opennmt.net/t/simple-opennmt-py-rest-server/1392)\n* How to create a simple Web Interface: [Tuto Streamlit](https://forum.opennmt.net/t/simple-web-interface/4527)\n* Replicate the WMT17 en-de experiment: [WMT17 ENDE](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/wmt17/Translation.md)\n\n----\n\n## Setup\n\n### Using docker\n\nTo facilitate setup and reproducibility, some docker images are made available via the Github Container Registry:\nhttps://github.com/OpenNMT/OpenNMT-py/pkgs/container/opennmt-py\n\nYou can adapt the workflow and build your own image(s) depending on specific needs by using `build.sh` and `Dockerfile` in the `docker` directory of the repo.\n\n```\ndocker pull ghcr.io/opennmt/opennmt-py:3.4.3-ubuntu22.04-cuda12.1\n```\n\nExample oneliner to run a container and open a bash shell within it\n```\ndocker run --rm -it --runtime=nvidia ghcr.io/opennmt/opennmt-py:test-ubuntu22.04-cuda12.1\n```\nNote: you need to have the [Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (formerly nvidia-docker) installed to properly take advantage of the CUDA/GPU features.\n\nDepending on your needs you can add various flags:\n- `-p 5000:5000` to forward some exposed port from your container to your host;\n- `-v /some/local/directory:/some/container/directory` to mount some local directory to some container directory;\n- `--entrypoint some_command` to directly run some specific command as the container entry point (instead of the default bash shell);\n\n### Installing locally\n\nOpenNMT-py requires:\n\n- Python \u003e= 3.8\n- PyTorch \u003e= 2.0 \u003c2.2\n\nInstall `OpenNMT-py` from `pip`:\n```bash\npip install OpenNMT-py\n```\n\nor from the source:\n```bash\ngit clone https://github.com/OpenNMT/OpenNMT-py.git\ncd OpenNMT-py\npip install -e .\n```\n\nNote: if you encounter a `MemoryError` during installation, try to use `pip` with `--no-cache-dir`.\n\n*(Optional)* Some advanced features (e.g. working pretrained models or specific transforms) require extra packages, you can install them with:\n\n```bash\npip install -r requirements.opt.txt\n```\n\n### Manual installation of some dependencies\n\nApex is highly recommended to have fast performance (especially the legacy fusedadam optimizer and FusedRMSNorm)\n\n```shell\ngit clone https://github.com/NVIDIA/apex\ncd apex\npip3 install -v --no-build-isolation --config-settings --build-option=\"--cpp_ext --cuda_ext --deprecated_fused_adam --xentropy --fast_multihead_attn\" ./\ncd ..\n```\n\nFlash attention:\n\nAs of Oct. 2023 flash attention 1 has been upstreamed to pytorch v2 but it is recommended to use flash attention 2 with v2.3.1 for sliding window attention support.\n\nWhen using regular `position_encoding=True` or Rotary with `max_relative_positions=-1` OpenNMT-py will try to use an optimized dot-product path.\n\nif you want to use [flash attention](https://github.com/Dao-AILab/flash-attention#installation-and-features) then you need to manually install it first:\n\n```bash\npip install flash-attn --no-build-isolation\n```\n\nif flash attention 2 is not installed, then we will use `F.scaled_dot_product_attention` from pytorch 2.x\n\nWhen using `max_relative_positions \u003e 0` or Alibi `max_relative_positions=-2` OpenNMT-py will use its legacy code for matrix multiplications.\n\nflash attention and `F.scaled_dot_product_attention` are a bit faster and saves some GPU memory.\n\n\nAWQ:\n\nIf you want to run inference or quantize an AWQ model you will need AutoAWQ.\n\nFor [AutoAWQ](https://github.com/casper-hansen/AutoAWQ):\n    pip install autoawq\n\n\n## Documentation \u0026 FAQs\n\n[Full HTML Documentation](https://opennmt.net/OpenNMT-py/quickstart.html)\n\n[FAQs](https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/FAQ.md)\n\n## Acknowledgements\n\nOpenNMT-py is run as a collaborative open-source project.\nProject was incubated by Systran and Harvard NLP in 2016 in Lua and ported to Pytorch in 2017.\n\nCurrent maintainers (since 2018):\n\n[François Hernandez](https://github.com/francoishernandez)\n[Vincent Nguyen](https://github.com/vince62s) (Seedfall)\n\n## Citation\n\nIf you are using OpenNMT-py for academic work, please cite the initial [system demonstration paper](https://www.aclweb.org/anthology/P17-4012) published in ACL 2017:\n\n```\n@misc{klein2018opennmt,\n      title={OpenNMT: Neural Machine Translation Toolkit}, \n      author={Guillaume Klein and Yoon Kim and Yuntian Deng and Vincent Nguyen and Jean Senellart and Alexander M. Rush},\n      year={2018},\n      eprint={1805.11462},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopennmt%2Fopennmt-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopennmt%2Fopennmt-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopennmt%2Fopennmt-py/lists"}