https://github.com/jonsafari/tok-tok

A fast, simple, multilingual tokenizer
https://github.com/jonsafari/tok-tok

multilingual nlp tokeniser tokenizer

Last synced: 8 months ago
JSON representation

A fast, simple, multilingual tokenizer

• Host: GitHub
• URL: https://github.com/jonsafari/tok-tok
• Owner: jonsafari
• License: apache-2.0
• Created: 2015-02-28T20:29:11.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2017-05-24T18:48:00.000Z (about 9 years ago)
• Last Synced: 2024-07-08T15:40:47.378Z (almost 2 years ago)
• Topics: multilingual, nlp, tokeniser, tokenizer
• Language: Python
• Size: 13.7 KB
• Stars: 28
• Watchers: 5
• Forks: 3
• Open Issues: 1

Awesome Lists containing this project

• awesome-community-curated-nlp - Toktok