https://github.com/bytebuff/aioscrapy

基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star
https://github.com/bytebuff/aioscrapy

aiohttp asyncio queue scrapy twisted

Last synced: 8 months ago
JSON representation

基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star

• Host: GitHub
• URL: https://github.com/bytebuff/aioscrapy
• Owner: bytebuff
• Created: 2019-08-16T07:26:12.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2019-10-25T14:34:38.000Z (about 6 years ago)
• Last Synced: 2024-03-04T22:35:39.857Z (almost 2 years ago)
• Topics: aiohttp, asyncio, queue, scrapy, twisted
• Language: Python
• Size: 103 KB
• Stars: 35
• Watchers: 3
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# aioScrapy

## Overview

aioScrapy是一款基于asyncio与aiohttp的异步协程爬虫框架。

![Scrapy-Dataflow.png](https://i.loli.net/2019/08/16/blNEjKVvZzagfd8.png)

## Requirements

- Python3.6

- aiohttp3.4.4

## Install

直接下载源码即可