https://github.com/jonsafari/tok-tok

A fast, simple, multilingual tokenizer
https://github.com/jonsafari/tok-tok

multilingual nlp tokeniser tokenizer

Last synced: 4 days ago
JSON representation

A fast, simple, multilingual tokenizer

• Host: GitHub
• URL: https://github.com/jonsafari/tok-tok
• Owner: jonsafari
• License: apache-2.0
• Created: 2015-02-28T20:29:11.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2017-05-24T18:48:00.000Z (about 7 years ago)
• Last Synced: 2024-03-16T15:34:34.579Z (4 months ago)
• Topics: multilingual, nlp, tokeniser, tokenizer
• Language: Python
• Size: 13.7 KB
• Stars: 28
• Watchers: 5
• Forks: 3
• Open Issues: 1

Lists

• awesome-community-curated-nlp - Toktok