https://github.com/oxidized-transformers/oxidized-transformers
Modular Rust transformer/LLM library using Candle
https://github.com/oxidized-transformers/oxidized-transformers
bert llama llama2 llm roberta rust transformers
Last synced: 8 months ago
JSON representation
Modular Rust transformer/LLM library using Candle
- Host: GitHub
- URL: https://github.com/oxidized-transformers/oxidized-transformers
- Owner: oxidized-transformers
- License: other
- Created: 2023-08-08T17:41:02.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-05-05T15:41:32.000Z (about 2 years ago)
- Last Synced: 2025-10-21T20:44:01.926Z (8 months ago)
- Topics: bert, llama, llama2, llm, roberta, rust, transformers
- Language: Rust
- Homepage:
- Size: 142 KB
- Stars: 36
- Watchers: 5
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
- Authors: AUTHORS
Awesome Lists containing this project
- awesome-rust-ai-libraries - Oxidized-Transformers - Pure Rust implementation of Hugging Face transformers with experimental support for GPT and BERT models. Provides a fully native Rust alternative to Python-based transformer libraries without FFI bindings. ([Read more](/details/oxidized-transformers.md)) `Transformers` `Pure Rust` `Experimental` (Model Inference)
README
# Oxidized Transformers
Oxidized Transformers is a Rust transformers library that started out as
a port of [Curated Transformers](https://github.com/explosion/curated-transformers/tree/main).
The foundations are in place and some popular models are implemented, but
Oxidized Transformers is still too volatile to use in projects. Keep an eye
on the repo, since progress is currently fast.
## 🧰 Supported Model Architectures
Supported encoder-only models:
- ALBERT
- BERT
- RoBERTa
- XLM-RoBERTa
Supported decoder-only models:
- GPT-NeoX
- Llama 1/2
All types of models can be loaded from Huggingface Hub. Float16/bfloat16
models can use flash attention v2 on recent CUDA GPUs.