Projects in Awesome Lists by scrapy-plugins
A curated list of projects in awesome lists by scrapy-plugins .
https://github.com/scrapy-plugins/scrapy-splash
Scrapy+Splash for JavaScript integration
Last synced: 17 Dec 2025
https://github.com/scrapy-plugins/scrapy-playwright
🎠Playwright integration for Scrapy
chrome-headless firefox-headless hacktoberfest headless-browser javascript-renderer playwright playwright-python python python-asyncio python3 scrapy webkit-headless
Last synced: 14 May 2025
https://github.com/scrapy-plugins/scrapy-djangoitem
Scrapy extension to write scraped items using Django models
Last synced: 16 May 2025
https://github.com/scrapy-plugins/scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
crawler crawler-detection plugin proxy scraping scrapy
Last synced: 16 May 2025
https://github.com/scrapy-plugins/scrapy-jsonrpc
Scrapy extension to control spiders using JSON-RPC
Last synced: 17 Dec 2025
https://github.com/scrapy-plugins/scrapy-deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
hacktoberfest hacktoberfest2021
Last synced: 04 Apr 2025
https://github.com/scrapy-plugins/scrapy-magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-monkeylearn
A Scrapy pipeline to categorize items using MonkeyLearn
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-pagestorage
A scrapy extension to store requests and responses information in storage service
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-querycleaner
Scrapy spider middleware to clean up query parameters in request URLs
Last synced: 29 Oct 2025
https://github.com/scrapy-plugins/scrapy-splitvariants
Scrapy spider middleware to split an item into multiple items using a multi-valued key
Last synced: 15 Jul 2025
https://github.com/scrapy-plugins/scrapy-dotpersistence
A scrapy extension to sync `.scrapy` folder to an S3 bucket
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-streamitem
Scrapy support for working with streamcorpus Stream Items.
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-snowflake-stage-exporter
Snowflake database loading utility with Scrapy integration
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-bigml
Scrapy pipeline for writing items to BigML datasets
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-hcf
Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs
Last synced: 23 Jun 2025
https://github.com/scrapy-plugins/scrapy-download-handlers-incubator
Additional download handlers for Scrapy
Last synced: 29 May 2026
https://github.com/scrapy-plugins/scrapy-feedexporter-onedrive
Export to OneDrive
Last synced: 13 May 2026
https://github.com/scrapy-plugins/scrapy-feedexporter-dropbox
Scrapy feed exporter for Dropbox
Last synced: 02 May 2025
https://github.com/scrapy-plugins/scrapy-crawl-maps
Scrapy plugin that allows defining the logic of a spider using a directed graph defined in JSON format.
Last synced: 31 Jan 2026