An open API service indexing awesome lists of open source software.

"http-requests" Awesome Lists

awesome-web-scraping

A list of libraries, tools, and APIs for web scraping and data processing. Find everything you need for extracting, managing, and processing data from the web, from HTTP libraries to browser automation tools and proxy services.

crawling data data-collection frameworks golang guides http-requests java javascript perl

7 stars
2 forks
46 projects

Last updated: 04 Oct 2025

Search
Keywords
awesome-list 3,712 awesome 3,366 awesome-lists 477 machine-learning 430 list 408 deep-learning 365 resources 313 hacktoberfest 235 ai 221 python 209 lists 201 javascript 187 llm 172 security 163 programming 159 artificial-intelligence 146 computer-vision 136 blockchain 129 nlp 114 tools 111 open-source 110 data-science 108 large-language-models 106 react 103 chatgpt 96 android 89 natural-language-processing 85 linux 83 learning 81 curated-list 79 css 78 papers 76 awesome-readme 76 devops 75 ios 74 awesome-resources 73 ethereum 73 reinforcement-learning 70 cybersecurity 63 nodejs 62 computer-science 62 robotics 60 playground 60 rust 59 game-development 58 free-resources 58 design 57 macos 57 collection 57 getvm 57 kubernetes 57 tutorials 54 free 54 java 54 privacy 53 golang 52 tutorial 52 web 50 survey 50 openai 49 hacking 49 frontend 49 bitcoin 49 paper 48 datasets 48 go 47 cloud 45 labex 45 swift 45 hands-on 45 education 44 deep-neural-networks 44 books 44 llms 44 php 44 web3 43 dataset 43 development 43 html 43 github 42 developer-tools 42 gpt 42 data 42 cryptocurrency 41 security-tools 41 exercises 41 typescript 40 vue 39 data-visualization 39 opensource 38 gamedev 38 documentation 37 docker 36 collections 36 ml 35 database 35 software 35 generative-ai 34 research 34 paper-list 33 reactjs 33 courses 33 neural-network 33 web-development 33 aws 33 iot 33 testing 32 automation 32 framework 32 game 31 diffusion-models 31 projects 30 graph-neural-networks 30 productivity 29 hardware 29 solidity 29 cpp 29 video 29 community 29 ruby 29 algorithms 29 libraries 28 transformer 28 flutter 28 games 28 architecture 28 api 27 agent 27 bioinformatics 27 dotnet 27 tensorflow 27 smart-contracts 27 links 27 windows 26 crypto 26 react-native 26 csharp 26 multimodal 26 software-engineering 26 defi 25 computer-graphics 25 angular 25 mcp 25 music 25 microsoft 24 science 24 git 24 penetration-testing 24 slam 24 pentesting 24 osint 24 pytorch 24 self-supervised-learning 24 django 24 object-detection 24 engineering 23 awsome-list 23 serverless 23 embedded 23 r 23 markdown 23 hacktoberfest-accepted 23 coding 23 cryptography 23 unity 23 library 23 python3 22 mobile 22 infosec 22 gpt-4 22 neural-networks 22 game-engine 22 best-practices 22 roadmap 22 self-hosted 22 sql 22 data-analysis 22 ui 22 ai-agents 22 prompt-engineering 22 mathematics 22 apple 21 audio 21 startups 21 interview 21 blog 21 cli 21 segmentation 21 3d 21 awesomeness 21 opendata 21 free-tutorials 21 visualization 21 kotlin 21 gpt-3 21 knowledge-graph 21 youtube 20 data-mining 20 mlops 20 learning-resources 20 blogs 20 cloud-computing 20 deeplearning 20 jobs 20 website 20 awsome 20 azure 20 federated-learning 20 apps 20 autonomous-driving 20 c 20 programming-language 20 guidelines 20 chinese 20 chatbot 19 ai-tools 19 raspberry-pi 19 saas 19 reasoning 19 time-series 19 graph 19 agents 19 marketing 19 js 19 foundation-models 19 statistics 19 bugbounty 18 aigc 18 graphql 18 curated 18 front-end 18 llama 18 machinelearning 18 representation-learning 18 startup 18 nextjs 18 bash 18 reverse-engineering 18 decentralized 17 transformers 17 code 17 language 17 knowledge 17 generative-art 17 cloud-native 17 optimization 17 backend 17 software-development 17 cheatsheet 17 containers 17 image-processing 17 stable-diffusion 17 algorithm 17 cms 16 analytics 16 nerf 16 big-data 16 bert 16 data-structures 16 networking 16 microservices 16 seo 16 articles 16 webgl 16 resource 15 pentest 15 monitoring 15 readme 15 icons 15 autonomous-vehicles 15 animation 15 mysql 15 detection 15 ros 15 technology 15 dart 15 open-data 15 graphics 15 android-development 14 mac 14 distributed-systems 14 prompt 14 videos 14 finance 14 webassembly 14 telegram 14 react-components 14 diffusion 14 unicorns 14 deepseek 14 selfhosted 14 awesome-ai 14 p2p 14 recommender-system 14 terminal 14 node 14 large-language-model 14 art 14 language-model 14 performance 14 code-generation 14 data-engineering 14 vuejs 14 oss 14 devsecops 14 android-library 14 multimodal-deep-learning 14 foss 14 embodied-ai 13 deep-reinforcement-learning 13 swiftui 13 wordpress 13 evm 13 speech-recognition 13 webdevelopment 13 command-line 13 text-to-image 13 pose-estimation 13 agentic-ai 13 generative-adversarial-network 13 gis 13 transfer-learning 13 dotnet-core 13 quantum-computing 13 remote-sensing 13 frameworks 13 podcast 13 microservice 13 malware-analysis 13 fuzzing 13 postgresql 13 vscode 13 laravel 13 plugins 13 search 13 multimodal-large-language-models 13 npm 12 interview-questions 12 font 12 mcp-server 12 guide 12 vlm 12 gaming 12 svelte 12 multimodal-learning 12 discord 12 elasticsearch 12 es6 12 pwa 12 graphics-programming 12 rails 12 chain-of-thought 12 image-generation 12 anomaly-detection 12 gan 12 generative-model 12 video-generation 12 webapp 12 static-site-generator 12 cross-platform 12 image 12 dapp 12 search-engine 12 sustainability 12 wasm 12 storage 12 leetcode 12 static-analysis 12 rag 12 creative-coding 11 objective-c 11 vibe-coding 11 infrastructure-as-code 11 cvpr 11 databases 11 sysadmin 11 yolo 11 point-cloud 11 rest-api 11 beginner-friendly 11 ux 11 article 11 ui-design 11 developer 11 ecommerce 11 climate-change 11 rl 11 few-shot-learning 11 claude 11 ctf 11 unity3d 11 open-science 11 threat-intelligence 11 hacktoberfest2020 11 vision-and-language 11 cuda 11 automl 11 google 11 free-software 11 design-systems 11 front-end-development 11 programming-languages 11 haskell 11 application 11 benchmark 11 planning 11 course 11 webpack 11 3d-graphics 11 mllm 11 question-answering 11 book 11 hosting 11 management 10 podcasts 10 geospatial 10 model-compression 10 3d-reconstruction 10 vr 10 social-network 10 cv 10 solana 10 arduino 10 interpretability 10 text-mining 10 bot 10 fpga 10 functional-programming 10 reading-list 10 chatgpt-api 10 speech-processing 10 cyber-security 10 neuroscience 10 time-series-analysis 10 datascience 10 network 10 vision-language-model 10 malware 10 operating-system 10 webdesign 10 templates 10 flutter-apps 10 augmented-reality 10 medical-imaging 10 deepseek-r1 10 docs 10 machine-learning-algorithms 10 golang-library 10 hacktoberfest2021 10 semantic-segmentation 10 hacking-tools 10 movies 10 semantic-web 10 authentication 10 langchain 10 android-application 10 android-app 10 writing 10 ide 10 in-context-learning 10 utilities 10 prompts 10 students 10 explainable-ai 10 ansible 10 system-design 10 bug-bounty 10 gpts 10 workflow 10 unsupervised-learning 10 bookmarks 10 erlang 9 blockchain-technology 9 conference 9 reference 9 json 9 webcomponents 9 test-automation 9 compiler 9 xai 9 microservices-architecture 9 jupyter-notebook 9 chrome 9 best-of 9 mobile-development 9 reddit 9 sentiment-analysis 9 news 9 websites 9 anthropic 9 anime 9 interview-preparation 9 unix 9 fairness 9 redis 9 privacy-tools 9 simulation 9 linux-desktop 9 classification 9 indonesia 9