https://github.com/yerkopalma/bash-crawler
:computer: Get a site links with bash
https://github.com/yerkopalma/bash-crawler
bash crawler
Last synced: 11 months ago
JSON representation
:computer: Get a site links with bash
- Host: GitHub
- URL: https://github.com/yerkopalma/bash-crawler
- Owner: YerkoPalma
- License: mit
- Created: 2017-05-17T14:53:33.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2017-05-22T21:21:08.000Z (about 9 years ago)
- Last Synced: 2025-04-11T12:52:40.901Z (about 1 year ago)
- Topics: bash, crawler
- Language: Shell
- Size: 9.77 KB
- Stars: 6
- Watchers: 1
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# bash-crawler
> Get a site links with bash
## Requirements
1. Get all the links from a website
2. For each link, get all links of the page
## Usage
```bash
$ crawler
Usage
$ crawler [options]
Options
--depth, -d Depth of the links searched 5
--help, -h Prints this helps false
--list, -l Print a list formatted output false
--verbose, -v Verbose output false
Examples
$ crawler -d 5 www.github.com
```
## Tests
Tests were made using [bats][bats] framework, to run tests type:
```bash
$ bats --tap crawler.bats
```
## License
[MIT](/LICENSE)
[bats]: https://github.com/sstephenson/bats