https://github.com/sty945/newspythonspider
learn how to get 1W+ news data from news website
https://github.com/sty945/newspythonspider
Last synced: 4 months ago
JSON representation
learn how to get 1W+ news data from news website
- Host: GitHub
- URL: https://github.com/sty945/newspythonspider
- Owner: sty945
- Created: 2017-10-17T02:23:03.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-10-17T02:48:06.000Z (over 8 years ago)
- Last Synced: 2024-12-30T20:15:34.270Z (over 1 year ago)
- Language: Python
- Size: 3.91 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.txt
Awesome Lists containing this project
README
this is a spider program, i hope this will help you a lot.
this program is use python, and use Requests and BeautifulSoup library to parse the html, and get some useful information.
And i also used a package called multiprocessing,which supports spawining processed using an API.By this package, you can run your spider quickly.
you can run this program in the command line:
python3 main.py