https://github.com/rudissaar/web-spider

A basic sample of Web Spider written in Python.
https://github.com/rudissaar/web-spider

bot urllib urllib3 web-crawler web-scraping web-spider

Last synced: 21 days ago
JSON representation

A basic sample of Web Spider written in Python.

• Host: GitHub
• URL: https://github.com/rudissaar/web-spider
• Owner: rudissaar
• Created: 2017-08-11T20:05:56.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2019-06-12T21:19:08.000Z (about 7 years ago)
• Last Synced: 2025-02-21T23:15:16.354Z (over 1 year ago)
• Topics: bot, urllib, urllib3, web-crawler, web-scraping, web-spider
• Language: Python
• Homepage:
• Size: 22.5 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Web Spider

A basic sample of Web Spider that takes base URLs and then tries to find all other URLs on that page,

feel free to fork this repository and build your own functionality on it.