Projects in Awesome Lists by scrapy
A curated list of projects in awesome lists by scrapy .
https://github.com/scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
crawler crawling framework hacktoberfest python scraping web-scraping web-scraping-python
Last synced: 05 Jan 2026
https://github.com/scrapy/scrapely
A pure-python HTML screen-scraping library
Last synced: 13 Apr 2025
https://github.com/scrapy/dirbot
Scrapy project to scrape public web directories (educational) [DEPRECATED]
Last synced: 15 May 2025
https://github.com/scrapy/quotesbot
This is a sample Scrapy project for educational purposes
Last synced: 15 May 2025
https://github.com/scrapy/parsel
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
css hacktoberfest lxml python scraping selectors xml xpath
Last synced: 14 May 2025
https://github.com/scrapy/scrapyd-client
Command line client for Scrapyd server
Last synced: 14 May 2025
https://github.com/scrapy/cssselect
CSS Selectors for Python
css hacktoberfest python selectors
Last synced: 15 May 2025
https://github.com/scrapy/queuelib
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python
hacktoberfest non-persistent persistent python python3 queues
Last synced: 14 May 2025
https://github.com/scrapy/itemadapter
Common interface for data container classes
hacktoberfest metadata python python-attrs python-dataclasses python3 scrapy
Last synced: 16 May 2025
https://github.com/scrapy/protego
A pure-Python robots.txt parser with support for modern conventions.
hacktoberfest python robots-parser robots-txt
Last synced: 16 May 2025
https://github.com/scrapy/itemloaders
Library to populate items using XPath and CSS with a convenient API
Last synced: 16 May 2025
https://github.com/scrapy/scrapy-bench
A CLI for benchmarking Scrapy.
benchmark-suite command-line-tool python scrapy scrapy-bench web-crawler
Last synced: 14 Apr 2025
https://github.com/scrapy/flake8-scrapy
A Flake8 plugin to catch common issues on Scrapy spiders
flake8 flake8-plugin linter linter-plugin scrapy
Last synced: 22 Jun 2025
https://github.com/scrapy/pypydispatcher
A fork of http://pydispatcher.sourceforge.net/ with PyPy support
Last synced: 14 Apr 2025
https://github.com/scrapy/xtractmime
https://mimesniff.spec.whatwg.org/ implementation for Python
Last synced: 14 Sep 2025
https://github.com/scrapy/base-chromium
base component forked from Chromium source https://chromium.googlesource.com/chromium/src/base/
Last synced: 14 Apr 2025
https://github.com/scrapy/form2request
Python 3.8+ library to build HTTP requests out of HTML forms
Last synced: 29 Aug 2025
https://github.com/scrapy/gsoc2014-integration-tests
GSoC2014 - Scrapy Integration tests project
Last synced: 27 Jul 2025
https://github.com/scrapy/url-chromium
url component from Chromium source code, forked from https://chromium.googlesource.com/chromium/src/url
Last synced: 14 Jun 2025
https://github.com/scrapy/sphinx-scrapy
Sphinx extension for documentation in the Scrapy ecosystem
Last synced: 22 Jun 2025