{"id":15589705,"url":"https://github.com/jkawamoto/ctranslate2-rs","last_synced_at":"2026-04-05T11:01:17.941Z","repository":{"id":174083718,"uuid":"651317079","full_name":"jkawamoto/ctranslate2-rs","owner":"jkawamoto","description":"Rust bindings for OpenNMT/CTranslate2","archived":false,"fork":false,"pushed_at":"2026-02-06T10:17:48.000Z","size":502,"stargazers_count":49,"open_issues_count":8,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-02-06T18:16:53.043Z","etag":null,"topics":["ai","bindings","ctranslate2","llm","ml","rust"],"latest_commit_sha":null,"homepage":"https://docs.rs/ct2rs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jkawamoto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"ko_fi":"jkawamoto","thanks_dev":"u/gh/jkawamoto"}},"created_at":"2023-06-09T01:43:31.000Z","updated_at":"2026-02-06T10:01:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"b21b72dd-2cda-4365-9fc0-ac2d999d9403","html_url":"https://github.com/jkawamoto/ctranslate2-rs","commit_stats":null,"previous_names":["jkawamoto/ctranslate2-rs"],"tags_count":41,"template":false,"template_full_name":null,"purl":"pkg:github/jkawamoto/ctranslate2-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkawamoto%2Fctranslate2-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkawamoto%2Fctranslate2-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkawamoto%2Fctranslate2-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkawamoto%2Fctranslate2-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jkawamoto","download_url":"https://codeload.github.com/jkawamoto/ctranslate2-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkawamoto%2Fctranslate2-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31433044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T08:13:15.228Z","status":"ssl_error","status_checked_at":"2026-04-05T08:13:11.839Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","bindings","ctranslate2","llm","ml","rust"],"created_at":"2024-10-02T23:04:23.026Z","updated_at":"2026-04-05T11:01:17.935Z","avatar_url":"https://github.com/jkawamoto.png","language":"Rust","funding_links":["https://ko-fi.com/jkawamoto","https://thanks.dev/u/gh/jkawamoto"],"categories":[],"sub_categories":[],"readme":"# ctranslate2-rs\n\n[![Latest version](https://img.shields.io/crates/v/ct2rs.svg)](https://crates.io/crates/ct2rs)\n[![Build](https://github.com/jkawamoto/ctranslate2-rs/actions/workflows/build.yaml/badge.svg)](https://github.com/jkawamoto/ctranslate2-rs/actions/workflows/build.yaml)\n[![docs.rs](https://img.shields.io/docsrs/ct2rs)](https://docs.rs/ct2rs)\n[![GitHub License](https://img.shields.io/github/license/jkawamoto/ctranslate2-rs)](https://github.com/jkawamoto/ctranslate2-rs/blob/main/LICENSE)\n\nThis library provides Rust bindings for [OpenNMT/CTranslate2](https://github.com/OpenNMT/CTranslate2).\n\n## Usage\n\nAdd this crate to your `Cargo.toml` with selecting the backends you want to use as the features:\n\n```toml\n[dependencies]\nct2rs = { version = \"0.9.18\", features = [\"cuda\", \"dnnl\", \"mkl\"] }\n```\n\nOr you can use platform-specific default features by using the `ct2rs-platform` crate:\n\n```toml\n[dependencies]\nct2rs = { version = \"0.9.18\", package = \"ct2rs-platform\" }\n```\n\nIf you want [Whisper](https://huggingface.co/docs/transformers/model_doc/whisper) model support,\nyou need to enable the `whisper` feature.\n\nSee below for more details about the available features.\n\n### Prerequisites\n\nThe installation of [CMake](https://cmake.org/) is required to compile the library.\n\n### Additional notes for Windows\n\nSetting the environment variable `RUSTFLAGS=-C target-feature=+crt-static` might be required.\n\n## Features\n\n### Backend Futures\n\n- `cuda`: Enables CUDA support\n- `cudnn`: Enables cuDNN support\n\nThe above features require setting the `CUDA_TOOLKIT_ROOT_DIR` environment variable appropriately.\n\n- `mkl`: Enables [Intel MKL](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html) support\n- `system-mkl`: Enables [Intel MKL](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html) support\n  using the system-installed MKL library instead of downloading and building it\n- `openblas`: Enables [OpenBLAS](https://www.openblas.net/) support (OpenBLAS needs to be installed manually\n  via [vcpkg](https://vcpkg.io) on Windows)\n- `dnnl`: Enables [oneDNN](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onednn.html) support\n- `ruy`: Enables [Ruy](https://github.com/google/ruy) support\n- `accelerate`: Enables [Apple Accelerate](https://developer.apple.com/documentation/accelerate) support (macOS only)\n- `openmp-runtime-comp`: Enables OpenMP runtime support\n- `openmp-runtime-intel`: Enables OpenMP runtime support for Intel compilers\n- `disable-cpu-dispatch`: Disables CPU kernel dispatching at runtime (should be set when explicitly targeting an\n  architecture)\n\nMultiple features can be enabled at the same time.\n\nTo enable Streaming SIMD Extensions 4.1 (SSE4.1), add the `-C target-feature=+sse4.1` flag to `RUSTFLAGS` environment\nvariable.\n\nBy default, the `ruy` feature is enabled.\n\nIf you want to use platform-specific default features, use the `ct2rs-platform` crate.\n\n### GPU-Specific Features\n\n- `flash-attention`:\n  Enables [Flash Attention](https://huggingface.co/docs/text-generation-inference/conceptual/flash_attention)\n- `tensor-parallel`:\n  Enables [Tensor Parallelism](https://huggingface.co/docs/text-generation-inference/conceptual/tensor_parallelism)\n- `cuda-dynamic-loading`: Enables dynamic loading of CUDA libraries at runtime instead of static linking (requires\n  CUDA \u003e= 11)\n- `cuda-small-binary`: Reduces binary size by compressing device code\n\n### Tokenizer Features\n\n- `sentencepiece`: Enables [SentencePiece](https://github.com/google/sentencepiece) tokenizer support\n- `tokenizers`: Enables HuggingFace's [Tokenizers](https://github.com/huggingface/tokenizers) library support\n- `all-tokenizers`: Enables both `sentencepiece` and `tokenizers` support\n\n### Additional Features\n\n- `whisper`: Enables [Whisper](https://huggingface.co/docs/transformers/model_doc/whisper) model support\n- `hub`: Enables [HuggingFace Hub](https://huggingface.co/docs/hub) integration\n- `system`: Skip compiling CTranslate2 and use the system's pre-installed shared library instead (requires setting the\n  appropriate environment variables to locate the CTranslate2 shared library)\n\n### Platform Specific Features\n\nWhen `ct2rs-platform` is used, the following features are automatically selected based on the platform:\n\n- Windows: `openmp-runtime-intel`, `dnnl`, `cuda`, `cudnn`, `cuda-dynamic-loading`, `mkl`\n- Intel MacOS: `dnnl`, `mkl`\n- Apple Silicon MacOS: `accelerate`, `ruy`\n- Linux (non-ARM): `dnnl`, `openmp-runtime-comp`, `cuda`, `cudnn`, `cuda-dynamic-loading`, `mkl`, `tensor-parallel`\n- Linux (ARM): `openmp-runtime-comp`, `openblas`, `ruy`\n\n## Supported Models\n\nThe ct2rs crate has been tested and confirmed to work with the following models:\n\n- BART\n- BLOOM\n- FALCON\n- Marian-MT\n- MPT\n- NLLB\n- GPT-2\n- GPT-J\n- OPT\n- T5\n- Whisper\n\nPlease see the respective\n[examples](ct2rs/examples)\nfor each model.\n\n## Stream API\n\nThis crate also offers a streaming API that utilizes callback closures.\nPlease refer to the [example code](ct2rs/examples/stream.rs)\nfor more information.\n\n## Model Conversion for CTranslate2\n\nTo use model files with CTranslate2, they must first be converted.\nBelow is an example of how to convert the `nllb-200-distilled-600M` model:\n\n```shell-session\npip install ctranslate2 huggingface_hub torch transformers\nct2-transformers-converter --model facebook/nllb-200-distilled-600M --output_dir nllb-200-distilled-600M \\\n    --copy_files tokenizer.json\n```\n\nFor more details, please refer to\nthe [CTranslate2's docs](https://opennmt.net/CTranslate2/guides/transformers.html#nllb).\n\n## License\n\nThis application is released under the MIT License. For details, see the [LICENSE](LICENSE) file.\n\n[CTranslate2](https://github.com/OpenNMT/CTranslate2) (which this crate builds and links to)\nis licensed under MIT License.\nIf you redistribute binaries or source that include CTranslate2,\nyou must comply with the MIT terms, including preserving license and notice files.\nFor details, see CTranslate2’s upstream LICENSE/NOTICE.\n\nThe published crate may build and/or redistribute artifacts originating from the following third-party projects.\nEach component remains under its own license, which applies in addition to this crate’s MIT license.\nWhen redistributing your software,\nensure that you include the required attributions and license texts for any components\nthat were used by your build configuration and target platform.\nThe exact set of components used can depend on enabled features, target architecture, and toolchain.\n\nThe published crate also includes the following libraries:\n\nBSD-2-Clause:\n\n- [cpuinfo](https://github.com/pytorch/cpuinfo)\n\nBSD-3-Clause:\n\n- [CUTLASS](https://github.com/NVIDIA/cutlass)\n- [GoogleTest](https://github.com/google/googletest)\n- [cub](https://github.com/NVIDIA/cub)\n\nApache License 2.0:\n\n- [cpu_features](https://github.com/google/cpu_features)\n- [The ruy matrix multiplication library](https://github.com/google/ruy)\n- [Thrust: Code at the speed of light](https://github.com/NVIDIA/thrust)\n\nMIT License:\n\n- [cxxopts](https://github.com/jarro2783/cxxopts)\n- [spdlog](https://github.com/gabime/spdlog)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjkawamoto%2Fctranslate2-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjkawamoto%2Fctranslate2-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjkawamoto%2Fctranslate2-rs/lists"}