An open API service indexing awesome lists of open source software.

"pythonframework" Awesome Lists

awesome-tokenizers

A curated list of tokenizer libraries for blazing-fast NLP processing.

awesome collections python python-library pythonframework sentence tokenizer wordpiece

5 stars
0 forks
33 projects

Last updated: 10 Apr 2025

Search
Keywords
awesome-list 3,592 awesome 3,265 machine-learning 422 awesome-lists 419 list 401 deep-learning 360 resources 311 hacktoberfest 238 python 205 ai 200 lists 199 javascript 187 programming 168 security 164 llm 155 artificial-intelligence 140 computer-vision 132 blockchain 130 nlp 109 tools 107 data-science 106 react 102 open-source 101 large-language-models 96 chatgpt 92 android 86 natural-language-processing 84 linux 81 learning 80 css 76 awesome-readme 75 papers 74 curated-list 74 ios 73 devops 73 ethereum 72 awesome-resources 70 free 63 cybersecurity 63 nodejs 63 reinforcement-learning 62 computer-science 61 tutorials 61 rust 60 playground 60 game-development 58 free-resources 58 macos 57 getvm 57 robotics 57 kubernetes 57 labex 56 hands-on 56 collection 54 java 54 privacy 54 golang 53 tutorial 52 design 52 exercises 51 web 50 paper 48 hacking 48 frontend 48 go 48 bitcoin 48 openai 47 swift 45 survey 45 datasets 45 php 45 education 44 books 44 dataset 43 web3 43 html 43 deep-neural-networks 43 cryptocurrency 43 development 42 security-tools 42 cloud 42 data 42 gpt 41 data-visualization 40 vue 39 llms 39 opensource 38 gamedev 38 github 38 developer-tools 37 typescript 37 docker 37 documentation 36 collections 36 courses 35 database 35 automation 33 reactjs 33 ml 33 research 33 paper-list 33 neural-network 33 iot 32 aws 32 software 32 framework 32 game 31 generative-ai 30 graph-neural-networks 30 testing 30 projects 30 cpp 29 free-tutorials 29 ruby 29 hardware 29 community 29 flutter 28 architecture 28 video 28 solidity 28 transformer 28 web-development 28 games 28 smart-contracts 28 algorithms 28 libraries 28 tensorflow 27 links 26 dotnet 26 csharp 26 api 26 react-native 26 bioinformatics 26 windows 26 django 25 git 25 defi 25 software-engineering 25 crypto 25 agent 25 music 25 pytorch 24 angular 24 diffusion-models 24 computer-graphics 24 productivity 24 self-supervised-learning 24 science 24 penetration-testing 24 cryptography 24 pentesting 24 object-detection 24 r 23 embedded 23 library 23 osint 23 unity 23 infosec 22 awsome-list 22 gpt-3 22 microsoft 22 engineering 22 gpt-4 22 game-engine 22 neural-networks 22 multimodal 22 serverless 22 best-practices 22 mobile 22 slam 22 markdown 22 visualization 21 ui 21 mathematics 21 python3 21 roadmap 21 apple 21 mcp 21 knowledge-graph 21 data-analysis 21 self-hosted 21 segmentation 21 awesomeness 21 coding 21 kotlin 21 hacktoberfest-accepted 21 deeplearning 20 website 20 jobs 20 interview 20 opendata 20 programming-language 20 cloud-computing 20 guidelines 20 youtube 20 audio 20 apps 20 data-mining 20 startups 20 mlops 19 azure 19 time-series 19 3d 19 sql 19 graph 19 federated-learning 19 raspberry-pi 19 awsome 19 prompt-engineering 19 c 19 saas 19 chinese 19 blogs 18 blog 18 graphql 18 bash 18 nextjs 18 chatbot 18 marketing 18 js 18 llama 18 front-end 18 bugbounty 18 cli 18 machinelearning 18 code 17 algorithm 17 generative-art 17 curated 17 stable-diffusion 17 autonomous-driving 17 decentralized 17 statistics 17 learning-resources 17 optimization 17 startup 17 containers 17 microservices 17 knowledge 17 image-processing 17 networking 17 representation-learning 17 aigc 17 foundation-models 17 language 17 data-structures 16 articles 16 cheatsheet 16 bert 16 ros 16 reverse-engineering 16 analytics 16 agents 16 webgl 16 ai-agents 16 nerf 15 graphics 15 big-data 15 seo 15 resource 15 cloud-native 15 search 15 pentest 15 autonomous-vehicles 15 software-development 15 technology 15 distributed-systems 15 transformers 15 readme 15 backend 15 detection 15 devsecops 15 monitoring 15 data-engineering 14 performance 14 icons 14 mac 14 prompt 14 telegram 14 malware-analysis 14 terminal 14 mysql 14 android-development 14 reasoning 14 multimodal-deep-learning 14 animation 14 node 14 p2p 14 art 14 recommender-system 14 dart 14 open-data 14 selfhosted 14 webassembly 14 android-library 14 large-language-model 14 vuejs 14 react-components 14 swiftui 13 finance 13 evm 13 pose-estimation 13 plugins 13 foss 13 deep-reinforcement-learning 13 search-engine 13 dotnet-core 13 laravel 13 gis 13 generative-adversarial-network 13 unicorns 13 podcast 13 speech-recognition 13 frameworks 13 guide 13 elasticsearch 13 remote-sensing 13 command-line 13 dapp 13 microservice 13 postgresql 12 gaming 12 es6 12 static-analysis 12 gan 12 deepseek 12 videos 12 text-to-image 12 quantum-computing 12 anomaly-detection 12 rails 12 graphics-programming 12 font 12 storage 12 svelte 12 transfer-learning 12 point-cloud 12 oss 12 discord 12 sustainability 12 ai-tools 12 fuzzing 12 leetcode 12 pwa 12 ecommerce 12 language-model 12 wordpress 12 image 12 wasm 12 cms 12 ctf 11 cross-platform 11 cvpr 11 ui-design 11 benchmark 11 automl 11 rag 11 databases 11 webapp 11 objective-c 11 sysadmin 11 creative-coding 11 hacktoberfest2020 11 3d-graphics 11 network 11 beginner-friendly 11 npm 11 vision-and-language 11 rest-api 11 climate-change 11 interview-questions 11 front-end-development 11 question-answering 11 open-science 11 programming-languages 11 awesome-ai 11 yolo 11 webpack 11 multimodal-learning 11 planning 11 threat-intelligence 11 bookmarks 11 mcp-server 11 course 11 unity3d 11 application 11 vscode 11 design-systems 11 cuda 11 malware 11 code-generation 11 free-software 11 movies 10 hacktoberfest2021 10 solana 10 operating-system 10 golang-library 10 authentication 10 cv 10 developer 10 machine-learning-algorithms 10 management 10 reading-list 10 time-series-analysis 10 ansible 10 arduino 10 3d-reconstruction 10 bot 10 unsupervised-learning 10 docs 10 privacy-tools 10 gpts 10 diffusion 10 hacking-tools 10 chatgpt-api 10 edge-computing 10 semantic-web 10 fpga 10 hosting 10 students 10 speech-processing 10 multimodal-large-language-models 10 text-mining 10 bug-bounty 10 image-generation 10 writing 10 android-application 10 android-app 10 flutter-apps 10 medical-imaging 10 webdevelopment 10 interpretability 10 google 10 guides 10 indonesia 10 blockchain-technology 10 datascience 10 haskell 10 video-generation 10 vr 10 geospatial 10 vlm 10 model-compression 10 reddit 9 news 9 cyber-security 9 packages 9 anime 9 social-network 9 websites 9 microservices-architecture 9 book 9 simulation 9 explainable-ai 9 ros2 9 infrastructure-as-code 9 few-shot-learning 9 knowledge-base 9 classification 9 material-design 9 sentiment-analysis 9 webcomponents 9 ionic 9 augmented-reality 9 shell 9 ux 9 neuroscience 9 arkit 9 utilities 9 mobile-development 9 ai-agent 9 trading 9 article 9 exploit 9 erlang 9 cicd 9 linux-desktop 9 conference 9 quantization 9 math 9 flutter-examples 9 ide 9 generative-model 9 vulnerabilities 9 fairness 9 reference 9 embodied-ai 9 unix 9 politics 9 profile 9 email 9 chrome 9 podcasts 9 applications 9 functional-programming 9 elixir 9 llmops 9 gpu 9 in-context-learning 9 interview-preparation 9