https://github.com/igaozp/jobwitcher

JobWitcher 招聘网站爬虫合集
https://github.com/igaozp/jobwitcher

crawler python3 redis scrapy spider

Last synced: 8 days ago
JSON representation

JobWitcher 招聘网站爬虫合集

• Host: GitHub
• URL: https://github.com/igaozp/jobwitcher
• Owner: igaozp
• License: mit
• Created: 2018-09-30T04:36:16.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2019-07-07T12:19:36.000Z (over 5 years ago)
• Last Synced: 2024-11-07T04:24:03.360Z (about 2 months ago)
• Topics: crawler, python3, redis, scrapy, spider
• Language: Python
• Homepage:
• Size: 43 KB
• Stars: 3
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
## JobWitcher - 招聘网站爬虫合集

![PyPI - Python Version](https://img.shields.io/pypi/pyversions/Django.svg) ![GitHub](https://img.shields.io/github/license/mashape/apistatus.svg)

使用 Scrapy 框架实现的招聘网站爬虫，通过 Redis 完成数据的增量抓取，并持久化到 MySQL 数据库中

支持的网站列表：

- 智联招聘

- 拉勾网

- 前程无忧

- BOSS 直聘

- 猎聘网

## 运行

在爬虫根目录下执行命令行命令：

```bash

$python main.py

```

## LICENSE

[MIT](https://github.com/igaozp/JobWitcher/blob/master/LICENSE)