https://github.com/betacatpro/joint-spiders

Scrapy分布式、去重增量爬虫
https://github.com/betacatpro/joint-spiders

redis scrapy

Last synced: about 1 year ago
JSON representation

Scrapy分布式、去重增量爬虫

• Host: GitHub
• URL: https://github.com/betacatpro/joint-spiders
• Owner: BetaCatPro
• Created: 2020-02-21T07:46:44.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2020-04-26T08:05:14.000Z (about 6 years ago)
• Last Synced: 2025-05-08T17:02:32.544Z (about 1 year ago)
• Topics: redis, scrapy
• Language: Python
• Homepage:
• Size: 1.14 MB
• Stars: 20
• Watchers: 2
• Forks: 7
• Open Issues: 0

Awesome Lists containing this project