https://github.com/yyhsong/ipyspider

Python网络爬虫与信息提取
https://github.com/yyhsong/ipyspider

beautifulsoup lxml pyquery python-spider requests scrapy urllib

Last synced: 12 days ago
JSON representation

Python网络爬虫与信息提取

• Host: GitHub
• URL: https://github.com/yyhsong/ipyspider
• Owner: yyhsong
• Created: 2018-09-27T02:43:44.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2020-12-27T05:43:38.000Z (over 4 years ago)
• Last Synced: 2025-04-11T20:51:59.084Z (3 months ago)
• Topics: beautifulsoup, lxml, pyquery, python-spider, requests, scrapy, urllib
• Language: Python
• Homepage:
• Size: 172 KB
• Stars: 6
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# iPySpider

基于Python 3.x的网络爬虫与信息提取

网页即接口 The website is the API.

## 网络请求库

- urllib Python内置标准库

- requests 基于urllib的再次封装

## 文档解析及信息提取库

- lxml

- pyquery

- beautifulsoup

- re

## 网络爬虫框架

- scrapy

## 实现定时爬虫任务

- APScheduler