Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/auralshin/minbpe_rs
Rust implementation for Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
https://github.com/auralshin/minbpe_rs
gpt gpt-4 openai
Last synced: 6 days ago
JSON representation
Rust implementation for Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
- Host: GitHub
- URL: https://github.com/auralshin/minbpe_rs
- Owner: auralshin
- Created: 2024-02-19T02:41:48.000Z (11 months ago)
- Default Branch: master
- Last Pushed: 2024-02-19T15:11:35.000Z (11 months ago)
- Last Synced: 2024-05-02T05:11:21.221Z (8 months ago)
- Topics: gpt, gpt-4, openai
- Language: Rust
- Homepage:
- Size: 3.91 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0