Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pete911/blackhole
simple webserver generating random links
https://github.com/pete911/blackhole
bot-testing crawler-testing go golang spider-test spider-trap
Last synced: 6 days ago
JSON representation
simple webserver generating random links
- Host: GitHub
- URL: https://github.com/pete911/blackhole
- Owner: pete911
- Created: 2020-02-12T23:04:23.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2024-12-06T09:02:16.000Z (21 days ago)
- Last Synced: 2024-12-06T10:18:55.739Z (21 days ago)
- Topics: bot-testing, crawler-testing, go, golang, spider-test, spider-trap
- Language: Go
- Homepage:
- Size: 65.4 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Blackhole
Simple web server generating random links. The aim of this project is to keep bots/web crawlers occupied within
'blackhole'. Project can be run as docker image either built locally or downloaded from
[dockeruhb](https://hub.docker.com/r/pete911/blackhole).
```shell script
docker run --rm -it -p 8080:8080 pete911/blackhole
```
## local build and run
- go `make build && ./blackhole` ([go](https://golang.org/dl/) required)
- docker - `docker build -t blackhole . && docker run -d -p 8080:8080 --name blackhole blackhole` ([docker](https://www.docker.com/) required)
creates local server on port `8080`, let bots/crawlers/spiders run against it.
### optional flags
```
-max-link-depth int
max. link depth (number of path segments) (default 10)
-max-links int
max. number of links to generate (default 50)
-min-link-depth int
min. link depth (number of path segments) (default 1)
-min-links int
min. number of links to generate (default 10)
-port int
blackhole server port (default 8080)
-profile-port int
blackhole pprofile server port, 0 means pprof server is disabled (default 0)
```
Each option can be by env. variable as well. Env. variables have the same name as flags but are in uppercase, prefixed
with `BH_` and hyphens are replaced with underscores. e.g. flag `-max-links` can be set with `BH_MAX_LINKS` env. variable.