{"id":17679370,"url":"https://github.com/bbkane/shovel","last_synced_at":"2025-09-09T23:13:33.070Z","repository":{"id":159110396,"uuid":"616286778","full_name":"bbkane/shovel","owner":"bbkane","description":"Explore complex DNS records by making and aggregating DNS requests","archived":false,"fork":false,"pushed_at":"2025-06-16T01:13:48.000Z","size":1900,"stargazers_count":7,"open_issues_count":11,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-02T11:56:20.706Z","etag":null,"topics":["dns","go"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbkane.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-20T04:40:29.000Z","updated_at":"2025-06-16T01:13:52.000Z","dependencies_parsed_at":"2023-12-30T23:19:32.902Z","dependency_job_id":"b9d8ca99-5061-40e4-9a3e-902f710b4ca6","html_url":"https://github.com/bbkane/shovel","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/bbkane/shovel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbkane%2Fshovel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbkane%2Fshovel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbkane%2Fshovel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbkane%2Fshovel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbkane","download_url":"https://codeload.github.com/bbkane/shovel/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbkane%2Fshovel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274379761,"owners_count":25274227,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-09T02:00:10.223Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dns","go"],"created_at":"2024-10-24T08:22:13.460Z","updated_at":"2025-09-09T23:13:33.057Z","avatar_url":"https://github.com/bbkane.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# shovel\n\nMake a lot of DNS requests and count the results! Useful for testing complex dynamic DNS records.\n\nPass multiple qnames, nameservers, record types, and client subnets, either via command line flags, a config, or a combo of both. shovel will dig all combinations of those and show you the results.\n\n## Project Status (2025-06-14)\n\n`shovel` works well, but I don't use it anymore. I'm watching issues; please open one for any questions and especially BEFORE submitting a Pull Request.\n\n## Use\n\nAlso see [examples.md](./examples.md)\n\n![./demo.gif](./demo.gif)\n\nYou can save these flags to a config:\n\n```yaml\n# tmp.yaml\n# see config paths with `shovel dig combine -h`\ndig:\n  combine:\n    count: 10\n    qnames:\n      - example.com\n      - www.example.com\n    nameservers:\n      - 1.1.1.1:53\n    rtypes:\n      - A\n```\n\nRun with:\n\n```bash\nshovel dig combine --config ./tmp.yaml\n```\n\n### Proxy DNS Traffic through a separate server with [`sshuttle`](https://sshuttle.readthedocs.io/en/stable/usage.html)\n\nIn one tab:\n\n```bash\nsshuttle --dns -r username@host --ns-hosts=8.8.8.8 0/0:53 ::/0:53\n```\n\nIn another tab:\n\n```bash\nshovel dig combine -q example.com -r TXT -p tcp4 -n 8.8.8.8:53\n```\n\n## Install\n\n- [Homebrew](https://brew.sh/): `brew install bbkane/tap/shovel`\n- [Scoop](https://scoop.sh/):\n\n```\nscoop bucket add bbkane https://github.com/bbkane/scoop-bucket\nscoop install bbkane/shovel\n```\n\n- Download Mac/Linux/Windows executable: [GitHub releases](https://github.com/bbkane/shovel/releases)\n- Go: `go install go.bbkane.com/shovel@latest`\n- Build with [goreleaser](https://goreleaser.com/) after cloning: `goreleaser --snapshot --skip-publish --clean`\n\n## Notes\n\nSee [Go Developer Tooling](https://www.bbkane.com/blog/go-developer-tooling/) for notes on development tooling.\n\n## Run the webapp locally with [OpenObserve](https://openobserve.ai/)\n\nExport env vars:\n\n```bash\nexport SHOVEL_SERVE_OPENOBSERVE_PASS='...';\nexport SHOVEL_SERVE_OPENOBSERVE_USER='...';\nexport ZO_ROOT_USER_EMAIL='...';\nexport ZO_ROOT_USER_PASSWORD='...';\n```\n\nRun OpenObserve (in another terminal) after downloading:\n\n```bash\n./openobserve\n```\n\nOpen OpenObserve at: http://localhost:5080/web/traces?period=15m\u0026query=\u0026org_identifier=default\n\nRun shovel. Check `go run . serve --help` to see all flags available. Also see [format_jsonl.py]https://github.com/bbkane/dotfiles/blob/master/bin_common/bin_common/format_jsonl.py)\n\n```bash\ngo run . serve | format_jsonl.py fmt\n```\n\nOpen shovel at: http://127.0.0.1:8080/?count=1\u0026nameservers=dns3.p09.nsone.net%3A53\u0026protocol=udp\u0026qnames=linkedin.com+www.linkedin.com\u0026rtypes=A\u0026subnetMap=\u0026subnets=\n\nInstall shovel + OpenObserve as systemd services, on a local dev VM or production VM with [shovel_ansible](https://github.com/bbkane/shovel_ansible/)\n\n## Dev Notes\n\n- Go doc: https://pkg.go.dev/github.com/miekg/dns\n- `miekg/dns` exmple: https://github.com/miekg/exdns/blob/master/q/q.go\n- Look up IP subnets for a country: http://www.nirsoft.net/countryip/ or https://ipinfo.io/countries\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbkane%2Fshovel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbkane%2Fshovel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbkane%2Fshovel/lists"}