{"id":34118566,"url":"https://github.com/dsecuredcom/archive-finder","last_synced_at":"2026-03-09T23:01:53.794Z","repository":{"id":275054621,"uuid":"924922477","full_name":"dsecuredcom/archive-finder","owner":"dsecuredcom","description":"Find archives for a given hostlist. ","archived":false,"fork":false,"pushed_at":"2025-06-23T10:55:53.000Z","size":47,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-17T06:01:31.289Z","etag":null,"topics":["bugbounty","it-security","pentest","pentesting","redteam"],"latest_commit_sha":null,"homepage":"https://www.dsecured.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dsecuredcom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-30T22:02:00.000Z","updated_at":"2025-06-23T10:55:57.000Z","dependencies_parsed_at":"2025-01-30T23:23:24.499Z","dependency_job_id":"29aeed3a-6199-45ca-86c2-5d1acb381828","html_url":"https://github.com/dsecuredcom/archive-finder","commit_stats":null,"previous_names":["dsecuredcom/archive-finder"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dsecuredcom/archive-finder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsecuredcom%2Farchive-finder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsecuredcom%2Farchive-finder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsecuredcom%2Farchive-finder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsecuredcom%2Farchive-finder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dsecuredcom","download_url":"https://codeload.github.com/dsecuredcom/archive-finder/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsecuredcom%2Farchive-finder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30315957,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T20:05:46.299Z","status":"ssl_error","status_checked_at":"2026-03-09T19:57:04.425Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bugbounty","it-security","pentest","pentesting","redteam"],"created_at":"2025-12-14T20:57:17.396Z","updated_at":"2026-03-09T23:01:53.787Z","avatar_url":"https://github.com/dsecuredcom.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Archive Finder\n\nThis project scans a list of hosts and attempts to locate potential archive files (e.g., `.zip`, `.tar`, `.rar`, etc.) on those hosts. It generates likely archive URLs based on known paths, domain name parts, and date-based patterns, then checks if those URLs lead to real archives.\n\n## Requirements\n\n- Go 1.20 (or higher)\n- A file containing a list of hosts (one per line)\n\n## Installation\n\n1. Clone this repository:\n\n   ```bash\n   git clone https://github.com/dsecuredcom/archive-finder\n   cd archive-finder\n   ```\n\n2. Initialize or update the Go modules (if needed):\n\n   ```bash\n   go mod tidy\n   ```\n\n3. Build the binary:\n\n   ```bash\n   go build -o archive-finder\n   ```\n\n## Usage\n\n```bash\n./archive-finder -hosts /path/to/hosts_file.txt [options]\n```\n\n### CLI Flags\n\n#### Required\n- `-hosts string`  \n  Path to the hosts list file. **(Required)**\n\n#### General Settings\n- `-timeout duration`  \n  Timeout for HTTP requests (default 60s).\n- `-concurrency int`  \n  Maximum number of concurrent requests (default 2500).\n- `-chunksize int`  \n  Chunksize for internal processing (default 500).\n- `-verbose`  \n  Enable verbose output (default false).\n\n#### HTTP Client Options\n- `-fasthttp`  \n  Use fasthttp instead of net/http for potentially faster requests (default false).\n\n#### Dictionary Control\n- `-intensity string`  \n  Choose scanning intensity: \"small\", \"medium\", or \"big\" (default \"medium\").\n  Controls the built-in wordlists, extensions, and backup folder names.\n- `-words string`  \n  Comma-separated list of words (overwrites intensity-based words).\n- `-extensions string`  \n  Comma-separated list of extensions (overwrites intensity-based extensions).\n- `-backup-folders string`  \n  Comma-separated list of backup folders (overwrites intensity-based folders).\n\n#### Entry Generation Modules\n- `-disable-dynamic-entries`  \n  Disable generation of entries based on host (default false).\n- `-only-dynamic-entries`  \n  Use only dynamically generated entries (default false).\n- `-with-host-parts`  \n  Generate based on host parts (default false).\n- `-with-first-chars`  \n  Generate based on first 3-4 chars of first subdomain part (default false).\n- `-with-year`  \n  Generate based on current year (default false).\n- `-with-date`  \n  Generate based on current date (default false).\n\n### Notes\n\n- When using dynamic entries (default behavior or with `-only-dynamic-entries`), you must activate at least one module using the `-with-*` flags.\n- You cannot use both `-disable-dynamic-entries` and `-only-dynamic-entries` together.\n\n### Examples\n\n```bash\n# Basic scan with default settings\n./archive-finder -hosts myhosts.txt\n\n# Verbose output with limited concurrency\n./archive-finder -hosts myhosts.txt -verbose -concurrency 1000\n\n# Use only domain-based dynamic entries\n./archive-finder -hosts myhosts.txt -only-dynamic-entries -with-host-parts\n\n# Comprehensive scan with all dynamic modules\n./archive-finder -hosts myhosts.txt -with-host-parts -with-first-chars -with-year -with-date\n\n# High intensity scan with fasthttp\n./archive-finder -hosts myhosts.txt -intensity big -fasthttp -with-host-parts -with-year\n```\n\n## How It Works\n\n1. Reads host entries from the provided file\n2. Generates potential archive URLs based on:\n    - Static wordlists (controlled by `-intensity`)\n    - Dynamic patterns from domain parts (when `-with-host-parts` is enabled)\n    - First characters of subdomain (when `-with-first-chars` is enabled)\n    - Year-based patterns (when `-with-year` is enabled)\n    - Date-based patterns (when `-with-date` is enabled)\n3. Checks each URL to determine if it contains an actual archive\n4. Reports findings in real-time\n\n## Contributing\n\n1. Fork this repository\n2. Create a new branch: `git checkout -b feature/my-feature`\n3. Make your changes and commit them\n4. Push to your fork and open a pull request\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE). Feel free to use or modify for your own purposes.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsecuredcom%2Farchive-finder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdsecuredcom%2Farchive-finder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsecuredcom%2Farchive-finder/lists"}