https://github.com/scrapy/dirbot
Scrapy project to scrape public web directories (educational) [DEPRECATED]
https://github.com/scrapy/dirbot
Last synced: 8 months ago
JSON representation
Scrapy project to scrape public web directories (educational) [DEPRECATED]
- Host: GitHub
- URL: https://github.com/scrapy/dirbot
- Owner: scrapy
- Created: 2011-04-28T04:34:27.000Z (over 14 years ago)
- Default Branch: master
- Last Pushed: 2017-10-27T08:08:05.000Z (about 8 years ago)
- Last Synced: 2025-04-07T21:11:46.602Z (9 months ago)
- Language: Python
- Homepage:
- Size: 15.6 KB
- Stars: 1,635
- Watchers: 167
- Forks: 1,061
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
Awesome Lists containing this project
README
======
dirbot
======
Deprecation notice (March 2017)
===============================
**This project is now deprecated.**
http://dmoz.org is no more and Scrapy's tutorial has been re-written
against http://quotes.toscrape.com/.
Please refer to https://github.com/scrapy/quotesbot for a more relevant
and up-to-date educational project on how to get started with Scrapy.