{"id":37177491,"url":"https://github.com/zrquan/gatherer","last_synced_at":"2026-01-14T20:42:15.893Z","repository":{"id":233783917,"uuid":"776648844","full_name":"zrquan/gatherer","owner":"zrquan","description":"Gatherer 是一个简易的爬虫工具","archived":false,"fork":false,"pushed_at":"2024-05-12T08:32:46.000Z","size":67,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-05-12T09:32:28.688Z","etag":null,"topics":["crawler","infosec","pentest","security"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zrquan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-24T04:40:27.000Z","updated_at":"2024-06-19T08:57:18.128Z","dependencies_parsed_at":"2024-04-18T15:29:58.283Z","dependency_job_id":"ff6217a8-978b-4e40-8d16-cb416e92ab68","html_url":"https://github.com/zrquan/gatherer","commit_stats":null,"previous_names":["zrquan/gatherer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zrquan/gatherer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zrquan%2Fgatherer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zrquan%2Fgatherer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zrquan%2Fgatherer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zrquan%2Fgatherer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zrquan","download_url":"https://codeload.github.com/zrquan/gatherer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zrquan%2Fgatherer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28434492,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T18:57:19.464Z","status":"ssl_error","status_checked_at":"2026-01-14T18:52:48.501Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","infosec","pentest","security"],"created_at":"2026-01-14T20:42:15.165Z","updated_at":"2026-01-14T20:42:15.874Z","avatar_url":"https://github.com/zrquan.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gatherer\n\nGatherer 是一个简易的爬虫工具，它可以从各种内容中收集资源链接和 API 然后进行访问\n\n[![asciicast](https://asciinema.org/a/lv1EaQyBFkeOtI74DBjP7vFRs.svg)](https://asciinema.org/a/lv1EaQyBFkeOtI74DBjP7vFRs)\n\n```\nGatherer v0.1.0\n\nUsage of ./gatherer:\n  -H value\n        HTTP request headers (eg. -H 'Header1:value' -H 'Header2:value')\n  -ch\n        Run Javascript in headless Chrome\n  -debug\n        Debug mode\n  -dep int\n        Maximum path depth (default 1)\n  -ef string\n        Filter by extensions (separated by commas)\n  -igq\n        Ignore the query portion on the URL from a[href]\n  -json\n        Log as JSON format\n  -lf string\n        Filter by response length (separated by commas)\n  -limit int\n        Maximum number of concurrent requests (default 100)\n  -nr\n        Disallow auto redirect\n  -proxy string\n        Proxy URL\n  -rod string\n        Set the default value of options used by rod.\n  -sf string\n        Filter by status codes (separated by commas)\n  -sub\n        Allow to visit sub-domains\n  -t int\n        Request timeout (second) (default 10)\n  -tt int\n        Total timeout (second)\n  -u string\n        Target URL\n  -ua\n        Use random User-Agent\n  -w string\n        Wordlist file path\n```\n\n## Features\n\n- 从 JS 代码中收集资源链接\n- 从 Webpack 打包的代码中收集动态生成的 JS 资源链接\n- 从 Swagger 文档中解析 API 的完整路径、方法、参数\n- 从 robots.txt 中收集资源链接\n- 从 XML sitemap 中收集资源链接\n- 执行 JS 完成页面渲染，比如 SPA\n\n## Thanks\n\n- [colly](https://github.com/gocolly/colly)\n- [hakrawler](https://github.com/hakluke/hakrawler)\n- [LinkFinder](https://github.com/GerbenJavado/LinkFinder)\n- [Packer-Fuzzer](https://github.com/rtcatc/Packer-Fuzzer)\n- more...","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzrquan%2Fgatherer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzrquan%2Fgatherer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzrquan%2Fgatherer/lists"}