{"id":17191668,"url":"https://github.com/thammegowda/nllb-serve","last_synced_at":"2025-10-04T10:38:14.614Z","repository":{"id":50259465,"uuid":"518591954","full_name":"thammegowda/nllb-serve","owner":"thammegowda","description":"Meta's \"No Language Left Behind\" models served as web app and REST API","archived":false,"fork":false,"pushed_at":"2025-05-27T03:35:14.000Z","size":1561,"stargazers_count":239,"open_issues_count":12,"forks_count":38,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-09-23T09:51:39.812Z","etag":null,"topics":["machine-translation","multilingual","nlp","transformers","translation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thammegowda.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-07-27T19:46:14.000Z","updated_at":"2025-09-22T20:26:27.000Z","dependencies_parsed_at":"2025-06-08T15:39:55.529Z","dependency_job_id":"933ac2e9-ec90-4e9a-97fa-88afd80905fd","html_url":"https://github.com/thammegowda/nllb-serve","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/thammegowda/nllb-serve","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thammegowda%2Fnllb-serve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thammegowda%2Fnllb-serve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thammegowda%2Fnllb-serve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thammegowda%2Fnllb-serve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thammegowda","download_url":"https://codeload.github.com/thammegowda/nllb-serve/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thammegowda%2Fnllb-serve/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278302556,"owners_count":25964519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-translation","multilingual","nlp","transformers","translation"],"created_at":"2024-10-15T01:27:02.046Z","updated_at":"2025-10-04T10:38:14.596Z","avatar_url":"https://github.com/thammegowda.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NLLB Serve\n[![image](http://img.shields.io/pypi/v/nllb-serve.svg)](https://pypi.python.org/pypi/nllb-serve/)\n\nThis project offers a web interface and REST API to Meta's No Language Left Behind (NLLB) models that can translate across 200 languages.\n\n\n## Setup\n\n```bash\n# Option 1: install as read only; recommended to use as is\npip install git+https://github.com/thammegowda/nllb-serve\n\n# Option 2: install for editable mode; recommended if you'd like to modify code\ngit clone  https://github.com/thammegowda/nllb-serve\ncd nllb-serve\npip install -e .\n\n\n# Entry point: either one of these options should work\nnllb-serve -h\npython -m nllb_serve -h\n```\n\n## Start server\n\n```bash\n# Either one of these should work\nnllb-serve\n# or\npython -m nllb_serve\n\n# Use CPU, ignore GPUs even if they exist\nCUDA_VISIBLE_DEVICES= nllb-serve\n\n# Use GPU device 0\nCUDA_VISIBLE_DEVICES=0 nllb-serve\n```\n\nThis starts a service on http://localhost:6060 by default.\n\n\u003cimg src=\"docs/webui-demo.png\" width=600px/\u003e\n\n**Cuda and Torch Compatibility Issues**\n```bash\n# check 1:\n$ python -c 'import torch; print(torch.cuda.is_available())'\nTrue\n\n# Check 2: match the version of installed cudatookit with the version for which torch bins were compiled\n# if exactly matching the versions is not possible/difficult, try getting the versions as close as possible\n$ python -c 'import torch; print(torch.version.cuda)'\n11.7\n$  nvidia-smi  | grep -o 'CUDA.*'\nCUDA Version: 12.0\n```\n\n\n**CLI options:**\n\n```\n$ nllb-serve -h\nusage: nllb-serve [-h] [-d] [-p PORT] [-ho HOST] [-b BASE] [-mi MODEL_ID] [-msl MAX_SRC_LEN]\n\nDeploy NLLB model to a RESTful server\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -d, --debug           Run Flask server in debug mode (default: False)\n  -p PORT, --port PORT  port to run server on (default: 6060)\n  -ho HOST, --host HOST\n                        Host address to bind. (default: 0.0.0.0)\n  -b BASE, --base BASE  Base prefix path for all the URLs. E.g., /v1 (default: None)\n  -mi MODEL_ID, --model_id MODEL_ID\n                        model ID; see https://huggingface.co/models?other=nllb (default: facebook/nllb-200-distilled-600M)\n  -msl MAX_SRC_LEN, --max-src-len MAX_SRC_LEN\n                        max source len; longer seqs will be truncated (default: 250)\n```\n\n## REST API\n\n* `/translate` end point accepts GET and POST requests with the following args:\n  * `source` -- source text. Can be a single string or a batch (i.e., list of strings)\n  * `src_lang` -- source language ID, e.g., `eng_Latn`\n  * `tgt_lang` -- target language ID, e.g., `eng_Latn`\n\nHTTP Clients may send these parameters in three ways:\n1. Query parameters (GET)\\\n   For example:\n   * http://0.0.0.0:6060/translate?source=I%20am%20testing\u0026src_lang=eng_Latn\u0026tgt_lang=fra_Latn\n   * http://0.0.0.0:6060/translate?source=I%20am%20testing\u0026src_lang=eng_Latn\u0026tgt_lang=fra_Latn\u0026source=another%20sentence\n\n2. URL encoded form (POST)\n  ```bash\n   curl --data \"source=Comment allez-vous?\" --data \"source=Bonne journée\" \\\n   --data \"src_lang=fra_Latn\" --data \"tgt_lang=eng_Latn\" \\\n    http://localhost:6060/translate\n  ```\n3. JSON body (POST)\n```bash\n$ curl -H \"Content-Type: application/json\" -X POST \\\n    http://localhost:6060/translate \\\n   --data '{\"source\": [\"Comment allez-vous?\"], \"src_lang\": \"fra_Latn\", \"tgt_lang\": \"kan_Knda\"}'\n```\n\nList of language codes: https://huggingface.co/facebook/nllb-200-distilled-600M/blob/main/special_tokens_map.json \n\n## NLLB-Batch\n\nThis CLI tool is for decoding batches of data. While the REST API is optimized for translating single translation at once, `nllb-batch` is optmized for decoding large files.\n\n```\n$ nllb-batch  --help\nINFO:root:torch device=cuda\nusage: nllb-batch [-h] [-mi MODEL_ID] -sl SRC_LANG -tl TGT_LANG [-i INP] [-o OUT] [-msl MAX_SRC_CHARS] [-b BATCH_SIZE]\n\nServe NLLB model via command line\n\noptions:\n  -h, --help            show this help message and exit\n  -mi MODEL_ID, --model-id MODEL_ID\n                        model ID; see https://huggingface.co/models?other=nllb (default: facebook/nllb-200-distilled-600M)\n  -sl SRC_LANG, --src-lang SRC_LANG\n                        source language identifier; eg: eng_Latn (default: None)\n  -tl TGT_LANG, --tgt-lang TGT_LANG\n                        Target language identifier; eg: eng_Latn (default: None)\n  -i INP, --inp INP     Input file (default: \u003c_io.TextIOWrapper name='\u003cstdin\u003e' mode='r' encoding='utf-8'\u003e)\n  -o OUT, --out OUT     Output file (default: \u003c_io.TextIOWrapper name='\u003cstdout\u003e' mode='w' encoding='utf-8'\u003e)\n  -msl MAX_SRC_CHARS, --max-src-chars MAX_SRC_CHARS\n                        max source chars len; longer seqs will be truncated (default: 512)\n  -b BATCH_SIZE, --batch-size BATCH_SIZE\n                        Batch size; number of sentences (default: 10)\n```\n\n## License\n\nThe code and model weights carry different licenses. \nThe code in this repository is distributed via [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html).  \nBut the model weights are accessed from Huggingface Hub and the original license of model weigts are applicable.\nAt the time of writing, authors of NLLB model distributed weights via CC-BY-NC-4.0 license. Read more at [LICENSE.model.md](https://github.com/facebookresearch/fairseq/blob/nllb/LICENSE.model.md) and [Creative Commons License](https://en.wikipedia.org/wiki/Creative_Commons_license)\n\n\n\n## References\n* https://research.facebook.com/publications/no-language-left-behind/\n* https://huggingface.co/docs/transformers/main/en/model_doc/nllb\n* https://ai.facebook.com/research/no-language-left-behind/\n* https://github.com/facebookresearch/fairseq/tree/nllb/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthammegowda%2Fnllb-serve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthammegowda%2Fnllb-serve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthammegowda%2Fnllb-serve/lists"}