{"id":15647339,"url":"https://github.com/winstxnhdw/nllb-api","last_synced_at":"2025-04-30T11:46:04.943Z","repository":{"id":170076911,"uuid":"646179293","full_name":"winstxnhdw/nllb-api","owner":"winstxnhdw","description":"A low-memory high-performance CPU-based API for Meta's No Language Left Behind (NLLB) using CTranslate2, hosted on Hugging Face Spaces.","archived":false,"fork":false,"pushed_at":"2024-04-19T10:48:50.000Z","size":753,"stargazers_count":45,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-04-20T00:47:18.042Z","etag":null,"topics":["caddy","ctranslate2","docker","fastapi","huggingface","huggingface-spaces","machine-translation","nllb","nllb200","supervisord","transformers","translation","uvicorn-gunicorn"],"latest_commit_sha":null,"homepage":"https://huggingface.co/spaces/winstxnhdw/nllb-api","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/winstxnhdw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-05-27T14:40:56.000Z","updated_at":"2024-04-22T11:56:58.207Z","dependencies_parsed_at":"2024-04-22T11:56:49.649Z","dependency_job_id":null,"html_url":"https://github.com/winstxnhdw/nllb-api","commit_stats":null,"previous_names":["winstxnhdw/nllb-api"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fnllb-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fnllb-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fnllb-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/winstxnhdw%2Fnllb-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/winstxnhdw","download_url":"https://codeload.github.com/winstxnhdw/nllb-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249500246,"owners_count":21282371,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caddy","ctranslate2","docker","fastapi","huggingface","huggingface-spaces","machine-translation","nllb","nllb200","supervisord","transformers","translation","uvicorn-gunicorn"],"created_at":"2024-10-03T12:18:38.176Z","updated_at":"2025-04-30T11:46:04.921Z","avatar_url":"https://github.com/winstxnhdw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nllb-api\n\n[![deploy.yml](https://github.com/winstxnhdw/nllb-api/actions/workflows/deploy.yml/badge.svg)](https://github.com/winstxnhdw/nllb-api/actions/workflows/deploy.yml)\n[![build.yml](https://github.com/winstxnhdw/nllb-api/actions/workflows/build.yml/badge.svg)](https://github.com/winstxnhdw/nllb-api/actions/workflows/build.yml)\n[![cuda.yml](https://github.com/winstxnhdw/nllb-api/actions/workflows/cuda.yml/badge.svg)](https://github.com/winstxnhdw/nllb-api/actions/workflows/cuda.yml)\n[![formatter.yml](https://github.com/winstxnhdw/nllb-api/actions/workflows/formatter.yml/badge.svg)](https://github.com/winstxnhdw/nllb-api/actions/workflows/formatter.yml)\n[![warmer.yml](https://github.com/winstxnhdw/nllb-api/actions/workflows/warmer.yml/badge.svg)](https://github.com/winstxnhdw/nllb-api/actions/workflows/warmer.yml)\n\n[![Open in Spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-md-dark.svg)](https://huggingface.co/spaces/winstxnhdw/nllb-api)\n[![Open a Pull Request](https://huggingface.co/datasets/huggingface/badges/raw/main/open-a-pr-md-dark.svg)](https://github.com/winstxnhdw/nllb-api/compare)\n\nA fast CPU-based API for Meta's [No Language Left Behind](https://huggingface.co/docs/transformers/model_doc/nllb) distilled 1.3B 8-bit quantised variant, hosted on Hugging Face Spaces. To achieve faster executions, we are using [CTranslate2](https://github.com/OpenNMT/CTranslate2) as our inference engine.\n\n\u003e [!WARNING]\\\n\u003e NLLB was trained with input lengths not exceeding 512 tokens. Translating longer sequences might result in quality degradation. Consider splitting your input into smaller chunks if you begin observing artefacts.\n\n## Usage\n\nSimply cURL the endpoint like in the following. The `source` and `target` languages must be specified using FLORES-200 codes.\n\n\u003cdetails\u003e\n\n\u003csummary\u003e List of FLORES-200 Codes \u003c/summary\u003e\n\n\u003cbr\u003e\n\nLanguage                           | FLORES-200 Code\n-----------------------------------|----------------\nAcehnese (Arabic script)           | ace_Arab\nAcehnese (Latin script)            | ace_Latn\nMesopotamian Arabic                | acm_Arab\nTa’izzi-Adeni Arabic               | acq_Arab\nTunisian Arabic                    | aeb_Arab\nAfrikaans                          | afr_Latn\nSouth Levantine Arabic             | ajp_Arab\nAkan                               | aka_Latn\nAmharic                            | amh_Ethi\nNorth Levantine Arabic             | apc_Arab\nModern Standard Arabic             | arb_Arab\nModern Standard Arabic (Romanized) | arb_Latn\nNajdi Arabic                       | ars_Arab\nMoroccan Arabic                    | ary_Arab\nEgyptian Arabic                    | arz_Arab\nAssamese                           | asm_Beng\nAsturian                           | ast_Latn\nAwadhi                             | awa_Deva\nCentral Aymara                     | ayr_Latn\nSouth Azerbaijani                  | azb_Arab\nNorth Azerbaijani                  | azj_Latn\nBashkir                            | bak_Cyrl\nBambara                            | bam_Latn\nBalinese                           | ban_Latn\nBelarusian                         | bel_Cyrl\nBemba                              | bem_Latn\nBengali                            | ben_Beng\nBhojpuri                           | bho_Deva\nBanjar (Arabic script)             | bjn_Arab\nBanjar (Latin script)              | bjn_Latn\nStandard Tibetan                   | bod_Tibt\nBosnian                            | bos_Latn\nBuginese                           | bug_Latn\nBulgarian                          | bul_Cyrl\nCatalan                            | cat_Latn\nCebuano                            | ceb_Latn\nCzech                              | ces_Latn\nChokwe                             | cjk_Latn\nCentral Kurdish                    | ckb_Arab\nCrimean Tatar                      | crh_Latn\nWelsh                              | cym_Latn\nDanish                             | dan_Latn\nGerman                             | deu_Latn\nSouthwestern Dinka                 | dik_Latn\nDyula                              | dyu_Latn\nDzongkha                           | dzo_Tibt\nGreek                              | ell_Grek\nEnglish                            | eng_Latn\nEsperanto                          | epo_Latn\nEstonian                           | est_Latn\nBasque                             | eus_Latn\nEwe                                | ewe_Latn\nFaroese                            | fao_Latn\nFijian                             | fij_Latn\nFinnish                            | fin_Latn\nFon                                | fon_Latn\nFrench                             | fra_Latn\nFriulian                           | fur_Latn\nNigerian Fulfulde                  | fuv_Latn\nScottish Gaelic                    | gla_Latn\nIrish                              | gle_Latn\nGalician                           | glg_Latn\nGuarani                            | grn_Latn\nGujarati                           | guj_Gujr\nHaitian Creole                     | hat_Latn\nHausa                              | hau_Latn\nHebrew                             | heb_Hebr\nHindi                              | hin_Deva\nChhattisgarhi                      | hne_Deva\nCroatian                           | hrv_Latn\nHungarian                          | hun_Latn\nArmenian                           | hye_Armn\nIgbo                               | ibo_Latn\nIlocano                            | ilo_Latn\nIndonesian                         | ind_Latn\nIcelandic                          | isl_Latn\nItalian                            | ita_Latn\nJavanese                           | jav_Latn\nJapanese                           | jpn_Jpan\nKabyle                             | kab_Latn\nJingpho                            | kac_Latn\nKamba                              | kam_Latn\nKannada                            | kan_Knda\nKashmiri (Arabic script)           | kas_Arab\nKashmiri (Devanagari script)       | kas_Deva\nGeorgian                           | kat_Geor\nCentral Kanuri (Arabic script)     | knc_Arab\nCentral Kanuri (Latin script)      | knc_Latn\nKazakh                             | kaz_Cyrl\nKabiyè                             | kbp_Latn\nKabuverdianu                       | kea_Latn\nKhmer                              | khm_Khmr\nKikuyu                             | kik_Latn\nKinyarwanda                        | kin_Latn\nKyrgyz                             | kir_Cyrl\nKimbundu                           | kmb_Latn\nNorthern Kurdish                   | kmr_Latn\nKikongo                            | kon_Latn\nKorean                             | kor_Hang\nLao                                | lao_Laoo\nLigurian                           | lij_Latn\nLimburgish                         | lim_Latn\nLingala                            | lin_Latn\nLithuanian                         | lit_Latn\nLombard                            | lmo_Latn\nLatgalian                          | ltg_Latn\nLuxembourgish                      | ltz_Latn\nLuba-Kasai                         | lua_Latn\nGanda                              | lug_Latn\nLuo                                | luo_Latn\nMizo                               | lus_Latn\nStandard Latvian                   | lvs_Latn\nMagahi                             | mag_Deva\nMaithili                           | mai_Deva\nMalayalam                          | mal_Mlym\nMarathi                            | mar_Deva\nMinangkabau (Arabic script)        | min_Arab\nMinangkabau (Latin script)         | min_Latn\nMacedonian                         | mkd_Cyrl\nPlateau Malagasy                   | plt_Latn\nMaltese                            | mlt_Latn\nMeitei (Bengali script)            | mni_Beng\nHalh Mongolian                     | khk_Cyrl\nMossi                              | mos_Latn\nMaori                              | mri_Latn\nBurmese                            | mya_Mymr\nDutch                              | nld_Latn\nNorwegian Nynorsk                  | nno_Latn\nNorwegian Bokmål                   | nob_Latn\nNepali                             | npi_Deva\nNorthern Sotho                     | nso_Latn\nNuer                               | nus_Latn\nNyanja                             | nya_Latn\nOccitan                            | oci_Latn\nWest Central Oromo                 | gaz_Latn\nOdia                               | ory_Orya\nPangasinan                         | pag_Latn\nEastern Panjabi                    | pan_Guru\nPapiamento                         | pap_Latn\nWestern Persian                    | pes_Arab\nPolish                             | pol_Latn\nPortuguese                         | por_Latn\nDari                               | prs_Arab\nSouthern Pashto                    | pbt_Arab\nAyacucho Quechua                   | quy_Latn\nRomanian                           | ron_Latn\nRundi                              | run_Latn\nRussian                            | rus_Cyrl\nSango                              | sag_Latn\nSanskrit                           | san_Deva\nSantali                            | sat_Olck\nSicilian                           | scn_Latn\nShan                               | shn_Mymr\nSinhala                            | sin_Sinh\nSlovak                             | slk_Latn\nSlovenian                          | slv_Latn\nSamoan                             | smo_Latn\nShona                              | sna_Latn\nSindhi                             | snd_Arab\nSomali                             | som_Latn\nSouthern Sotho                     | sot_Latn\nSpanish                            | spa_Latn\nTosk Albanian                      | als_Latn\nSardinian                          | srd_Latn\nSerbian                            | srp_Cyrl\nSwati                              | ssw_Latn\nSundanese                          | sun_Latn\nSwedish                            | swe_Latn\nSwahili                            | swh_Latn\nSilesian                           | szl_Latn\nTamil                              | tam_Taml\nTatar                              | tat_Cyrl\nTelugu                             | tel_Telu\nTajik                              | tgk_Cyrl\nTagalog                            | tgl_Latn\nThai                               | tha_Thai\nTigrinya                           | tir_Ethi\nTamasheq (Latin script)            | taq_Latn\nTamasheq (Tifinagh script)         | taq_Tfng\nTok Pisin                          | tpi_Latn\nTswana                             | tsn_Latn\nTsonga                             | tso_Latn\nTurkmen                            | tuk_Latn\nTumbuka                            | tum_Latn\nTurkish                            | tur_Latn\nTwi                                | twi_Latn\nCentral Atlas Tamazight            | tzm_Tfng\nUyghur                             | uig_Arab\nUkrainian                          | ukr_Cyrl\nUmbundu                            | umb_Latn\nUrdu                               | urd_Arab\nNorthern Uzbek                     | uzn_Latn\nVenetian                           | vec_Latn\nVietnamese                         | vie_Latn\nWaray                              | war_Latn\nWolof                              | wol_Latn\nXhosa                              | xho_Latn\nEastern Yiddish                    | ydd_Hebr\nYoruba                             | yor_Latn\nYue Chinese                        | yue_Hant\nChinese (Simplified)               | zho_Hans\nChinese (Traditional)              | zho_Hant\nStandard Malay                     | zsm_Latn\nZulu                               | zul_Latn\n\n\u003c/details\u003e\n\n```bash\ncurl 'https://winstxnhdw-nllb-api.hf.space/api/v4/translator?text=Hello\u0026source=eng_Latn\u0026target=spa_Latn'\n```\n\nTo stream translations as Server-Sent Events, you may query the `/translator/stream` endpoint instead.\n\n```bash\ncurl -N 'https://winstxnhdw-nllb-api.hf.space/api/v4/translator/stream?text=Hello\u0026source=eng_Latn\u0026target=spa_Latn'\n```\n\nYou can also determine the source language by querying the following API.\n\n```bash\ncurl 'https://winstxnhdw-nllb-api.hf.space/api/v4/language?text=Hello'\n```\n\n## Self-Hosting\n\nYou can self-host the API and access the Swagger UI at [localhost:7860/api/schema/swagger](http://localhost:7860/api/schema/swagger) with the following minimal configuration\n\n```bash\ndocker run --rm \\\n  -e SERVER_PORT=7860 \\\n  -p 7860:7860 \\\n  ghcr.io/winstxnhdw/nllb-api:main\n```\n\n### Cross-Origin Resource Sharing\n\nYou can configure CORS by passing the following environment variables.\n\n```bash\ndocker run --rm \\\n  -e SERVER_PORT=7860 \\\n  -e ACCESS_CONTROL_ALLOW_ORIGIN=localhost,example.com \\\n  -e ACCESS_CONTROL_ALLOW_CREDENTIALS=true \\\n  -e ACCESS_CONTROL_ALLOW_HEADERS=X-Custom-Header,Upgrade-Insecure-Requests \\\n  -e ACCESS_CONTROL_EXPOSE_HEADERS=Content-Encoding,Kuma-Revision \\\n  -e ACCESS_CONTROL_MAX_AGE=3600 \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_GET=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_POST=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_OPTIONS=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_PUT=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_DELETE=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_PATCH=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_HEAD=true \\\n  -e ACCESS_CONTROL_ALLOW_METHOD_TRACE=true \\\n  -p 7860:7860 \\\n  ghcr.io/winstxnhdw/nllb-api:main\n```\n\n### Optimisation\n\nYou can pass the following environment variables to optimise the API for your own uses. The value of `OMP_NUM_THREADS` increases the number of threads used to translate a given batch of inputs, while `TRANSLATOR_THREADS` increases the number of threads used to handle translate requests in parallel. It is recommended to not modify `WORKER_COUNT` as spawning multiple workers can lead to increased memory usage and poorer performance.\n\n\u003e [!IMPORTANT]\\\n\u003e `OMP_NUM_THREADS` $\\times$ `TRANSLATOR_THREADS` should not exceed the physical number of cores on your machine.\n\n```bash\ndocker run --rm \\\n  -e SERVER_PORT=7860 \\\n  -e OMP_NUM_THREADS=6 \\\n  -e TRANSLATOR_THREADS=2 \\\n  -e WORKER_COUNT=1 \\\n  -p 7860:7860 \\\n  ghcr.io/winstxnhdw/nllb-api:main\n```\n\n### CUDA Support\n\nYou can accelerate your inference with CUDA by building and using `Dockerfile.cuda-build` instead.\n\n```bash\ndocker build -f Dockerfile.cuda-build -t nllb-api .\n```\n\nAfter building the image, you can run the image with the following.\n\n```bash\ndocker run --rm --gpus all \\\n  -e SERVER_PORT=7860 \\\n  -p 7860:7860 \\\n  nllb-api\n```\n\n## Development\n\nFirst, install the required dependencies for your editor with the following.\n\n```bash\nuv sync\n```\n\nNow, you can access the Swagger UI at [localhost:7860/api/schema/swagger](http://localhost:7860/api/schema/swagger) after spinning the server up locally with the following.\n\n```bash\nmake\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwinstxnhdw%2Fnllb-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwinstxnhdw%2Fnllb-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwinstxnhdw%2Fnllb-api/lists"}