https://github.com/shaunlwm/dwix

😈 A parallel open directory mapper.
https://github.com/shaunlwm/dwix

indexer opendirectory parser webscraper webscraping

Last synced: about 2 months ago
JSON representation

😈 A parallel open directory mapper.

• Host: GitHub
• URL: https://github.com/shaunlwm/dwix
• Owner: ShaunLWM
• License: mit
• Created: 2020-05-09T11:48:02.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2022-06-25T13:13:08.000Z (about 4 years ago)
• Last Synced: 2024-04-26T11:47:17.126Z (about 2 years ago)
• Topics: indexer, opendirectory, parser, webscraper, webscraping
• Language: JavaScript
• Homepage:
• Size: 91.8 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# dwix

[!["Monthly Download"](https://img.shields.io/npm/dm/dwix.svg)](https://npmjs.org/package/dwix)

[!["Latest Release"](https://img.shields.io/npm/v/dwix.svg)](https://github.com/ShaunLWM/dwix/releases/latest)

[![MIT license](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/ShaunLWM/dwix/blob/master/LICENSE)

A parallel open directory mapper.

![output](/img/output.png)

## Requirement

- NodeJS v8 and above

## Usage

```

Usage: dwix --verbose --output [dir] 

Options:

  --version             Show version number                            [boolean]

  -v, --verbose         Enable verbose logging

  -p, --parallel        Number of urls to scan at once

  -o, --output          Output scanned urls to file

  -w, --whitelist       List of whitelisted extensions [affects output only]

  -i, --ignore-unknown  Ignore files with no extension [affects output only]

  -h, --help            Show help                                      [boolean]

Examples:

  dwix http://127.0.0.1

  dwix --verbose http://127.0.0.1

  dwix --verbose --output urls.txt http://127.0.0.1

  dwix --verbose --output urls.txt -i http://127.0.0.1

  dwix --verbose --output urls.txt --whitelist mp3 http://127.0.0.1

```

## Instructions

1. `npm install -g dwix` or `yarn global add dwix`

2. Follow the usage stated above

## TODO

- None for now

## License

MIT License - Copyright (c) 2020 Shaun