Projects in Awesome Lists by NLPOptimize
A curated list of projects in awesome lists by NLPOptimize .
https://github.com/nlpoptimize/flash-tokenizer
EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
bert berttokenizer cpp cpp17 deep-learning flash huggingface nlp pybind11 python tokenizer trie wordpiece wordpiece-tokenization
Last synced: 15 May 2025
https://github.com/NLPOptimize/flash-tokenizer
EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
bert berttokenizer cpp cpp17 deep-learning flash huggingface nlp pybind11 python tokenizer trie wordpiece wordpiece-tokenization
Last synced: 10 Apr 2025
https://github.com/nlpoptimize/awesome-tokenizers
A curated list of tokenizer libraries for blazing-fast NLP processing.
awesome collections python python-library pythonframework sentence tokenizer wordpiece
Last synced: 10 Apr 2025
https://github.com/nlpoptimize/formatflex
🚀 A flexible Python library for easy handling and conversion of Hierarchical, Tabular, and Serialized data formats.
bson cbor cloudpickle csv dill excel feather hdf5 joblib json messagepack orc parquet pickle python toml ubjson xml yaml
Last synced: 05 Apr 2025