https://github.com/dimo414/pycrawl

Simple Python web crawler, primarily designed for inspecting and diagnosing your own website
https://github.com/dimo414/pycrawl

crawler python

Last synced: 21 days ago
JSON representation

Simple Python web crawler, primarily designed for inspecting and diagnosing your own website

• Host: GitHub
• URL: https://github.com/dimo414/pycrawl
• Owner: dimo414
• License: gpl-3.0
• Created: 2020-01-17T07:59:51.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2020-01-17T08:00:24.000Z (almost 5 years ago)
• Last Synced: 2024-10-30T13:05:44.208Z (2 months ago)
• Topics: crawler, python
• Language: Python
• Homepage:
• Size: 39.1 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 7
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# PyCrawl

An extensible web crawler written in Python.

Designed primarily for use by website developers to test and

inspect their website.

## Copyright

Copyright 2013-2014 Michael Diamond

This program is free software: you can redistribute it and/or modify

it under the terms of the GNU General Public License as published by

the Free Software Foundation, either version 3 of the License, or

(at your option) any later version.

This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

GNU General Public License for more details.

You should have received a copy of the GNU General Public License

along with this program.  If not, see .