https://github.com/goyalyatin/web-crawler

Web Crawler, fetching data from wikipedia and putting to mongoDB
https://github.com/goyalyatin/web-crawler

beautifulsoup mongodb python3 webcrawler

Last synced: 17 days ago
JSON representation

Web Crawler, fetching data from wikipedia and putting to mongoDB

• Host: GitHub
• URL: https://github.com/goyalyatin/web-crawler
• Owner: GoyalYatin
• License: mit
• Created: 2018-06-10T18:57:57.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2021-04-25T16:06:27.000Z (over 3 years ago)
• Last Synced: 2024-11-09T08:38:09.386Z (2 months ago)
• Topics: beautifulsoup, mongodb, python3, webcrawler
• Language: Python
• Homepage:
• Size: 36.1 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Web Crawler

![HighLevelFlow](/doc/Architecture.png)

https://en.wikipedia.org/wiki/Main_Page

https://www.crummy.com/software/BeautifulSoup/bs4/doc/