https://github.com/jakesidsmith/krawala

Simple javascript web-crawler with command line interface
https://github.com/jakesidsmith/krawala

Last synced: about 2 months ago
JSON representation

Simple javascript web-crawler with command line interface

• Host: GitHub
• URL: https://github.com/jakesidsmith/krawala
• Owner: JakeSidSmith
• License: mit
• Created: 2016-12-11T14:56:26.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2024-02-08T18:22:36.000Z (over 1 year ago)
• Last Synced: 2024-04-14T12:31:40.397Z (about 1 year ago)
• Language: JavaScript
• Size: 181 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 23
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# Krawala

**Simple javascript web-crawler with command line interface**

## Install

Run the following to install as a global CLI

```shell

npm i krawala -g

```

## Usage

```shell

  Usage: krawala crawl [options]

  Options:

    --help, -h      Display help and usage info

    --depth, -d     Depth to crawl                                      [number]

    --format, -f    Format to return                                    [string]

    --interval, -i  Interval between requests (millis)                  [number]

    --timeout, -t   Maximum time to wait for a request (millis)         [number]

    --url, -u       URL to crawl                                        [string]

  Examples:

    krawala crawl -u http://domain.com -d 100

```