Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hakluke/hakcheckurl
Takes a list of URLs and returns their HTTP response codes
https://github.com/hakluke/hakcheckurl
Last synced: 10 days ago
JSON representation
Takes a list of URLs and returns their HTTP response codes
- Host: GitHub
- URL: https://github.com/hakluke/hakcheckurl
- Owner: hakluke
- License: mit
- Created: 2020-01-07T14:02:57.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-10-17T09:31:32.000Z (about 1 year ago)
- Last Synced: 2024-08-02T15:47:50.859Z (3 months ago)
- Language: Go
- Size: 17.6 KB
- Stars: 384
- Watchers: 8
- Forks: 44
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- WebHackersWeapons - hakcheckurl
README
# hakcheckurl
Takes a list of URLs and returns their HTTP response codes. This tool is perfect for quickly determining the status of multiple web pages, especially when combined with other tools.
This tool was written to be chained with [hakrawler](https://github.com/hakluke/hakrawler) to easily check the response codes of discovered URLs.
## Features
- **Concurrent Processing**: Utilize multiple threads to speed up the checking process.
- **Configurable Timeout**: Define how long each request should wait before timing out.
- **Retry Mechanism**: Automatically retry failed requests to handle temporary network glitches.
This tool was written to be chained with [hakrawler](https://github.com/hakluke/hakrawler) to easily check the response codes of discovered URLs.
# Installation
```
go install github.com/hakluke/hakcheckurl@latest
```
# Usage
```
-t : Specify the number of threads to use.
-retry : Define how many times to retry failed requests.
-timeout : Set a timeout for each request.
-retry-sleep : Set the duration to sleep between retries.
```
# Sample Usage
```
hakluke~$ assetfinder google.com | hakrawler -plain | hakcheckurl | grep -v 404
200 http://mw1.google.com/transit?
200 http://mw1.google.com/places/
200 http://mw1.google.com/notebook/search?
200 http://mw1.google.com/reader/
200 http://mw1.google.com/views?
200 http://mw1.google.com/sprint_xhtml
200 http://mw1.google.com/sprint_wml
200 http://mw1.google.com/scholar
200 https://area120.google.com/
200 https://area120.google.com/static/js/main.min.js
...
```