https://github.com/xinlc/scrapyjiandan
福利 [手动滑稽]
https://github.com/xinlc/scrapyjiandan
scrapy-crawler scrapy-spider
Last synced: 2 months ago
JSON representation
福利 [手动滑稽]
- Host: GitHub
- URL: https://github.com/xinlc/scrapyjiandan
- Owner: xinlc
- License: mit
- Created: 2018-05-12T13:06:38.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2018-05-18T02:15:49.000Z (about 7 years ago)
- Last Synced: 2025-01-21T01:11:33.063Z (4 months ago)
- Topics: scrapy-crawler, scrapy-spider
- Language: Python
- Homepage:
- Size: 870 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Jiandan crawler
## Installation
1. Install dependencies
```bash
$ pip3 install -r requirements.txt
```
2. [Download ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/downloads)
3. Make sure it’s in your PATH, e. g., place it in /usr/bin or /usr/local/bin.
## Usage
```bash
$ scrapy crawl meizitu
# Support multiple pages.
# Modify maxPage in spiders/meizituList/MeizituListSpider to control the number of pages.
$ scrapy crawl meizituList
$ scrapy crawl news
```
## Documents
* [Scrapy](https://scrapy.org/)
* [Selenium](https://seleniumhq.github.io/selenium/docs/api/py/index.html)