{"id":13645744,"url":"https://github.com/adobe/blackhole","last_synced_at":"2026-01-14T15:54:37.202Z","repository":{"id":40355798,"uuid":"249750588","full_name":"adobe/blackhole","owner":"adobe","description":"An HTTP sink (for testing) with optional recording and playback ability","archived":false,"fork":false,"pushed_at":"2025-01-02T16:51:50.000Z","size":194,"stargazers_count":68,"open_issues_count":1,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-08-15T14:30:43.208Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adobe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-24T15:51:02.000Z","updated_at":"2024-10-16T16:29:42.000Z","dependencies_parsed_at":"2024-06-19T05:16:40.000Z","dependency_job_id":"2adedff5-96a2-4517-90b9-6396225f3ce9","html_url":"https://github.com/adobe/blackhole","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/adobe/blackhole","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fblackhole","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fblackhole/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fblackhole/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fblackhole/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adobe","download_url":"https://codeload.github.com/adobe/blackhole/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fblackhole/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28425560,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T15:24:48.085Z","status":"ssl_error","status_checked_at":"2026-01-14T15:23:41.940Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:02:40.937Z","updated_at":"2026-01-14T15:54:37.184Z","avatar_url":"https://github.com/adobe.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# blackhole\n\n### Introduction\n * Backhole is an HTTP sink, to be used for testing \u0026 protoyping. Good for testing your outgoing http senders or collect sample request data for your real webservice.\n   - Pretends to be your real HTTP API/server\n   - Consumes everything thrown at it (any path, any \"host\")\n   - Reponds with a 200 and empty body\n   - No pre-canned reponse yet\n   - You can also collect \u0026 sample incoming requests to your real webservice by pairing this tool with something like [tcpcopy](https://github.com/session-replay-tools/tcpcopy)\n\n      * Real server responds to your client\n      * tcpcopy ignores fake response from blackhole\n      * blackhole records requests.\n    \n * Record \u0026 Replay\n \n    - `blackhole` lets you record and `replay` will replay, aka send the traffic to yet another site. \n   See next section for instructions on `replay`\n   \n ### Usage\n \n`$ blackhole`\n\nStarts a service to receive, and acknowledge with 200-OK, all http requests to `:80` on any interface.\nDefault port, tsl (https) settings can be configured via a yaml config file.\nIt should be named `bhconfig.yaml`. Place in current working directory or under `$HOME/.blackhole`\n\n```\n$ cat bhconfig_sample.yaml\nserve:\n  - \"http://:80\"\ntls:\n  cert: /path/to/certs/www.foobar.com.pem\n  privkey: /path/to/certs/www.foobar.com.pem\n```\n\nData, payload of your request, is still ignored and dropped on the floor\n\nWhen data is not saved, blackhole is nothing but a tiny wrapper around the excellent http library `fasthttp` \n\n`$ blackhole -o /path/to/save/files/ -c`\n\nRequests will be saved in a compressed format.\nMany files will be created depending on the nunber of threads\nThis *recording* and subsequent *replay* is the main \nadditional value provided on top of fasthttp\n\n# replay\n\n`$ replay -H host.domain.com:8080 -q /tmp/requests/requests_*.lz4`\n\nSend all data to a different destination.\n\nNOTE: without `-q`, all communication back and forth is printed to stdout.\nThis will be very verbose.\n\nblackhole - benchmarks\n======\n\nPlease note that **all benchmarks assume connection reuse**.\nWithout connection reuse performance will be horrible in any server solution.\n\nI use wrk to generate traffic as shown below\n\n`wrk -t12 -c200 -s post-random.lua -d1m http://target.domain.com:8080/index.html`\n\n* Test 1: **Server \u0026 Client**: Both running on a Macbook Pro\n  * 110,000+ request/sec accepted, read, and then discarded. Run with `--skip-stats`. Request *counting* has a slight overhead, but that is the only thing on top of vanilla fast-http hello-world at this point.\n  * 100,000 request/sec saved to disk (each with a 2k payload). Roughly 13 GB on disk for 6 million requests sent during a 1 minute test. Almost no overhead for disk i/o. `wrk` and other things running on the Mac is taking up some of the CPU, leaving only 4 cores for the app in either case.\n  * `post-random.lua` makes payload random to trigger the pathological case for compression. For truly random input, you will not get much compression. A previous version of this script incorrectly sent same data for all requests.\n  * 95,000 req/sec with 4:1 compression ratio (on compressible repeated content) with LZ4 compression is enabled. Ratio depends on payload. [LZ4](https://github.com/lz4/lz4) is truly awesome and gives us excellent compression without slowing us down.\n\n* Test 2: **Server**: One L8s vm on Azure-US-West, **Client:** One DS2 v3 in Azure-US-East as client\n  * 6,000 to 7,000 request/sec average with\n     * 400 concurrent connections\n     * A random payload of 2,000 characters.\n     * Server uses only 10% cpu resources. We could accomodate more clients.\n     \n* Test 3: **Server**: One L8s vm on Azure-US-West and **Client:** One L4s in Aure-US-West (same subnet)\n  * 138,000 to 140,000 requests/sec average with same payload example as above.\n  * Server uses 60% cpu resources.\n  \n* Test 4: **Server \u0026 Client**: One L8s vm on Azure-US-West (same host)\n  * 260,000 requests/sec average with non-randomized 2k payload. Roughly similar number if payload is not saved.\n  * 140,000 requests/sec with randomized payload (more to write to disk when compressed)\n  * Server uses 95% cpu resources.\n \n ### Go specific benchmarks\nSecond column is iteration \n``` \n BenchmarkBHSave-8     \t  521817\t     19763 ns/op\t    2752 B/op\t      36 allocs/op\n BenchmarkBHNoSave-8   \t  683246\t     17491 ns/op\t    2701 B/op\t      34 allocs/op\n ```\n\nINSTALLING\n======\n\n* Steps\n  * Download and Install Go from https://golang.org/dl/\n  * Option 1 : run `go install ./...` from the cloned directory.\n  Please note the three ellipses `...`, not two, a directory reference.\n  By default, binaries will be installed in `${HOME}/go/bin`, unless you have\n  either `${GOPATH}` or `${GOBIN}` set in which case binaries\n  go to those directories.\n  * Option 2: run `make`. `make` will build to the current directory / project root.\n  If you don't usually develop applications in Go, this is probably the easiest method.\n  \n* To build for another machine (different architecture/os)\n  * Run `./build-all-archs.sh` (builds for Windows, Linux, as well as MacOS) \n\nDesign\n=======\n![Design](design.png)\n\nCode Documentation (For contributors)\n=======\nThis is a binary package, not a library, but some of the components are still written as reusable libraries and the\ndocumentation is available here. Internal APIs may not stable and would change. Please import with caution.\n\n[github.com/adobe/blackhole/lib/archive](https://pkg.go.dev/github.com/adobe/blackhole/lib/archive)\n\n[github.com/adobe/blackhole/lib/request](https://pkg.go.dev/github.com/adobe/blackhole/lib/request)\n\n[github.com/adobe/blackhole/lib/fbr](https://pkg.go.dev/github.com/adobe/blackhole/lib/fbr)\n\n[github.com/adobe/blackhole/lib/sender](https://pkg.go.dev/github.com/adobe/blackhole/lib/sender)\n\n[github.com/adobe/blackhole/lib/slicehacks](https://pkg.go.dev/github.com/adobe/blackhole/lib/slicehacks)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fblackhole","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadobe%2Fblackhole","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fblackhole/lists"}