https://github.com/haaroon/bleepingcomputerscraper

Scrapes the forum from bleeping computer
https://github.com/haaroon/bleepingcomputerscraper

Last synced: 2 months ago
JSON representation

Scrapes the forum from bleeping computer

• Host: GitHub
• URL: https://github.com/haaroon/bleepingcomputerscraper
• Owner: Haaroon
• Created: 2021-12-05T20:18:47.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2021-12-05T20:21:41.000Z (over 3 years ago)
• Last Synced: 2025-02-10T09:43:56.116Z (4 months ago)
• Language: Python
• Size: 6.84 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# bleepingComputerScraper

Scrapes the forum from bleeping computer

1. Download this repo

2. pip install scrapy scrapy_useragents pymongo

3. download docker image for mongo 

docker pull mongo

4. run mongo 

docker run --name some-mongo -v /data/data1/bleeping/mongodb:/data/db  -p 27017 -d mongo

5. run scraper 

scrapy crawl forum_post_scraper