https://github.com/mithril-security/tokenizers-wasm

wasm bindings for huggingface tokenizers library
https://github.com/mithril-security/tokenizers-wasm

Last synced: about 1 year ago
JSON representation

wasm bindings for huggingface tokenizers library

• Host: GitHub
• URL: https://github.com/mithril-security/tokenizers-wasm
• Owner: mithril-security
• License: apache-2.0
• Created: 2022-06-02T10:32:11.000Z (about 4 years ago)
• Default Branch: main
• Last Pushed: 2022-06-30T12:27:56.000Z (almost 4 years ago)
• Last Synced: 2025-04-01T04:32:45.767Z (about 1 year ago)
• Language: Rust
• Size: 304 KB
• Stars: 34
• Watchers: 2
• Forks: 5
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE_APACHE

Awesome Lists containing this project

README

          
# tokenizers-wasm

Bindings of huggingface tokenizers for web assembly.

- in development

- unstable

- potentially non-compliant with huggingface/tokenizers

try the example out !