https://github.com/xpleaf/ispider

A Distributed Crawler System Designed By Java.
https://github.com/xpleaf/ispider

Last synced: 3 months ago
JSON representation

A Distributed Crawler System Designed By Java.

• Host: GitHub
• URL: https://github.com/xpleaf/ispider
• Owner: xpleaf
• License: apache-2.0
• Created: 2018-04-02T12:00:22.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2018-04-02T13:11:07.000Z (about 6 years ago)
• Last Synced: 2024-01-16T11:46:43.765Z (5 months ago)
• Language: Java
• Homepage:
• Size: 41 KB
• Stars: 205
• Watchers: 34
• Forks: 110
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Lists

• awesome-stars - ispider

README

        
分布式爬虫系统，其整体系统架构如下：

![](http://i2.51cto.com/images/blog/201804/02/6b349d970e418716572470efe2947e46.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)

系统的整体概述与分析可以查看我在51cto上发布的文章：http://blog.51cto.com/xpleaf/2093952