https://github.com/daemon/supercuts

High performance wordpiece tokenization with SentencePiece interoperability.
https://github.com/daemon/supercuts

Last synced: 4 months ago
JSON representation

High performance wordpiece tokenization with SentencePiece interoperability.

• Host: GitHub
• URL: https://github.com/daemon/supercuts
• Owner: daemon
• License: mit
• Created: 2019-09-10T14:51:49.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2019-09-10T14:51:50.000Z (almost 7 years ago)
• Last Synced: 2025-10-23T11:41:12.939Z (8 months ago)
• Homepage:
• Size: 1000 Bytes
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# supercuts

High performance tokenization for transformers.