https://github.com/natshah/natshah-crawler

Natshah Crawler works to crawl a selected domain with all it's internal links and internal pages.
https://github.com/natshah/natshah-crawler

crawler database filter natshah-crawler

Last synced: 21 days ago
JSON representation

Natshah Crawler works to crawl a selected domain with all it's internal links and internal pages.

• Host: GitHub
• URL: https://github.com/natshah/natshah-crawler
• Owner: Natshah
• License: mit
• Created: 2015-08-24T20:18:15.000Z (over 9 years ago)
• Default Branch: 1.x
• Last Pushed: 2015-08-25T21:36:32.000Z (over 9 years ago)
• Last Synced: 2023-09-17T01:09:00.317Z (over 1 year ago)
• Topics: crawler, database, filter, natshah-crawler
• Language: C++
• Homepage: http://natshah.com
• Size: 152 KB
• Stars: 2
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Natshah Crawler [ Version 1.10.12 ]

Copyright (c) 2015 Natshah.com .  All rights reserved.

The Natshah Crawler 1.x works with the Natshah Archiver 1.x

https://github.com/Natshah/Natshah-Archiver

List of all Crawler commands:

?                  : Synonym for 'help'.

clear              : Clear the current input statement.

delete             : Delete data from the database.

exit               : Exit Crawler. Same as quit.

help               : Display this help.

load               : Load Crawler Settings.

filters            : Show List of Filtered URLS.

extensions         : Show List of File extensions.

configurations     : Show Database configurations.

info,total         : Display Total Information.