https://github.com/prasannavenkadesh/crawl-engine

A Web crawler written in Python that finds and hits URL(s) in a page and goes on.
https://github.com/prasannavenkadesh/crawl-engine

Last synced: 3 months ago
JSON representation

A Web crawler written in Python that finds and hits URL(s) in a page and goes on.

• Host: GitHub
• URL: https://github.com/prasannavenkadesh/crawl-engine
• Owner: PrasannaVenkadesh
• Created: 2013-03-11T18:51:12.000Z (almost 12 years ago)
• Default Branch: master
• Last Pushed: 2013-03-14T08:27:30.000Z (almost 12 years ago)
• Last Synced: 2023-03-12T00:22:45.122Z (almost 2 years ago)
• Language: Python
• Size: 117 KB
• Stars: 3
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
An Internet web-crawler that finds URL(s) in a given root or seed URL and continues to do the same in those spotted URL.

Usage

=====

		python crawl.py