https://github.com/weihanli/proxycrawler

代理爬虫服务，爬取代理IP并保存到 Redis 中, topshelf+Quartz.Net+redis
https://github.com/weihanli/proxycrawler

crawler proxy proxy-ip redis

Last synced: about 1 year ago
JSON representation

代理爬虫服务，爬取代理IP并保存到 Redis 中, topshelf+Quartz.Net+redis

• Host: GitHub
• URL: https://github.com/weihanli/proxycrawler
• Owner: WeihanLi
• License: apache-2.0
• Created: 2018-05-09T03:39:37.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2019-06-26T10:00:36.000Z (almost 7 years ago)
• Last Synced: 2025-04-14T01:37:21.431Z (about 1 year ago)
• Topics: crawler, proxy, proxy-ip, redis
• Language: C#
• Homepage:
• Size: 27.3 KB
• Stars: 25
• Watchers: 3
• Forks: 11
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# ProxyCrawler

## Intro

代理爬虫，爬取网络上的代理并验证可用性，将可用的代理存放到 Redis 中的 windows 服务。

## Deploy

1. 安装 `Redis` 环境

    redis 下载

    - linux : 

    - windows : 

1. 在 Release 模式下编译项目

1. 在生成目录下执行命令 `ProxyCrawler install --sudo` 进行安装服务

## Contact

Contact me: weihanli@outlook.com