{"id":37140543,"url":"https://github.com/caio-ishikawa/netscout","last_synced_at":"2026-01-14T16:28:26.898Z","repository":{"id":230931023,"uuid":"778938637","full_name":"caio-ishikawa/netscout","owner":"caio-ishikawa","description":"OSINT tool that finds domains, subdomains, directories, endpoints and files for a given seed URL.","archived":false,"fork":false,"pushed_at":"2024-04-05T11:15:00.000Z","size":871,"stargazers_count":151,"open_issues_count":0,"forks_count":15,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-06-19T20:54:00.247Z","etag":null,"topics":["cli","golang","osint","osint-tool"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/caio-ishikawa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-28T17:50:37.000Z","updated_at":"2024-06-18T03:10:26.000Z","dependencies_parsed_at":"2024-06-19T20:07:45.672Z","dependency_job_id":"a4695ae5-c89b-40e4-9fc3-7dc391341e10","html_url":"https://github.com/caio-ishikawa/netscout","commit_stats":null,"previous_names":["caio-ishikawa/netscout"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/caio-ishikawa/netscout","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/caio-ishikawa%2Fnetscout","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/caio-ishikawa%2Fnetscout/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/caio-ishikawa%2Fnetscout/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/caio-ishikawa%2Fnetscout/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/caio-ishikawa","download_url":"https://codeload.github.com/caio-ishikawa/netscout/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/caio-ishikawa%2Fnetscout/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28425941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T15:24:48.085Z","status":"ssl_error","status_checked_at":"2026-01-14T15:23:41.940Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","golang","osint","osint-tool"],"created_at":"2026-01-14T16:28:26.356Z","updated_at":"2026-01-14T16:28:26.881Z","avatar_url":"https://github.com/caio-ishikawa.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NetScout\n\u003cimg src=\"screenshot.png\"\u003e\n\nNetScout is an OSINT tool that finds domains, subdomains, directories, endpoints and files for a given seed URL.\nIt consists of the following components:\n- BinaryEdge client: Gets subdomains\n- DNS: Attempts to perform a DNS zone transfer to extract subdomains\n- Crawler: Gets URLs and directories from the seed URL\n- SERP client: Gets links for files. It uses Google dorking techniques to search for specific file types based on file extensions found by the crawler\n- Shortened URL scan: This module leverages the URLTeam's [lists of shortened URLs](https://archive.org/details/UrlteamWebCrawls). It downloads the list that was last uploaded, and checks every entry for a host that matches the seed URL's host. These text files can be very large (\u003e500mb), and this scan takes several minutes. This module was heavily inspired by [urlhunter](https://github.com/utkusen/urlhunter). \n\n## Setup\n### How to install\n- Go install:\n      - Run ```go install github.com/caio-ishikawa/netscout@latest```\n- Build from source:\n      - Clone repository\n      - Run ```make install```\n\n## External APIs\nNetScout uses two external APIs: BinaryEdge and SerpAPI.\nBinaryEdge is used to query for historical data of subdomains registered to the seed URL, and SERP API is used to collect Google Search results for speficic filetypes for the seed URL.\n\n### Setting API keys\nNetScout expects the API keys to be set as environment variables:\n- ```export BINARYEDGE_API_KEY=\"\u003ckey\u003e\"```\n- ```export SERP_API_KEY=\"\u003ckey\u003e\"```\n\n## Examples \nUsage:\n```\n=======================================================================\n ███▄    █ ▓█████▄▄▄█████▓  ██████  ▄████▄   ▒█████   █    ██ ▄▄▄█████▓\n ██ ▀█   █ ▓█   ▀▓  ██▒ ▓▒▒██    ▒ ▒██▀ ▀█  ▒██▒  ██▒ ██  ▓██▒▓  ██▒ ▓▒\n▓██  ▀█ ██▒▒███  ▒ ▓██░ ▒░░ ▓██▄   ▒▓█    ▄ ▒██░  ██▒▓██  ▒██░▒ ▓██░ ▒░\n▓██▒  ▐▌██▒▒▓█  ▄░ ▓██▓ ░   ▒   ██▒▒▓▓▄ ▄██▒▒██   ██░▓▓█  ░██░░ ▓██▓ ░ \n▒██░   ▓██░░▒████▒ ▒██▒ ░ ▒██████▒▒▒ ▓███▀ ░░ ████▓▒░▒▒█████▓   ▒██▒ ░ \n░ ▒░   ▒ ▒ ░░ ▒░ ░ ▒ ░░   ▒ ▒▓▒ ▒ ░░ ░▒ ▒  ░░ ▒░▒░▒░ ░▒▓▒ ▒ ▒   ▒ ░░   \n░ ░░   ░ ▒░ ░ ░  ░   ░    ░ ░▒  ░ ░  ░  ▒     ░ ▒ ▒░ ░░▒░ ░ ░     ░    \n   ░   ░ ░    ░    ░      ░  ░  ░  ░        ░ ░ ░ ▒   ░░░ ░ ░   ░      \n         ░    ░  ░              ░  ░ ░          ░ ░     ░              \n=======================================================================\nUsage:\n  -u string\n        A string representing the URL\n  -d int\n        An integer representing the depth of the crawl\n  -t int\n        An integer representing the amount of threads to use for the scans (default 5)\n  -delay-ms int\n        An integer representing the delay between requests in miliseconds\n  -lock-host\n        A boolean - if set, it will only save URLs with the same host as the seed\n  -o string\n        A string representing the name of the output file\n  -h string\n        A comma-separated key-value string representing request headers\n  -c string\n        A comma-separated key-value string representing the cookies\n  -v\n        A boolean - if set, it will display all found URLs\n\n\n  -skip-axfr\n        A bool - if set, it will skip the DNS zone transfer attempt\n  -skip-binaryedge\n        A bool - if set, it will skip BinaryEdge subdomain scan\n  -skip-google-dork \n        A bool - if set, it will skip the Google filetype scan\n  -headless\n        A bool - if set, all requests in the crawler will be made through a headless Chrome browser (requires Google Chrome)\n  -deep\n        A bool - if set, the shortened URL scan will be performed (can take several minutes)\n```\n\nSets seed url, depth, and output file:\n```sh\nnetscout -u https://crawler-test -d 2 -o netscout.txt\n```\n\nSkips BinaryEdge and Google dork:\n```sh\nnetscout -u https://crawler-test.com -d 2 --skip-binaryedge --skip-google-dork -o netscout.txt\n```\n\nSets thread count to 5, req delay to 1000ms, and forces requests to be made throught a headless Chrome browser.\n```sh\nnetscout -u https://crawler-test.com -d 2 -t 5 --delay-ms 1000 --headless -o netscout.txt\n```\n\nSets depth to 2, and adds cookies and header values\n```sh\nnetscout -u https://crawler-test.com --deep -d 2 -t 5 -h \"key=test,key_two=test_2\" -c \"key=test,key_two=test_2\"\n```\n\n\nEnables the shortened URL scan, sets crawler depth to 2, and threads to 5\n```sh\nnetscout -u https://crawler-test.com --deep -d 2 -t 5\n```\n\n## Development\nBefore submitting a PR, please ensure that the project builds successfully and that all existing tests pass. More info at [Testing](#Testing)\n\nThank you for your interest in contributing to this project!\n\n## Testing\nThe tests are placed in the same directory as the tested file, and the crawler tests require the [DVWA (Damn Vulnerable Web App)](https://github.com/citizen-stig/dockerdvwa/tree/master) to be running locally with port 80 exposed. \nBefore running the tests, the test files must be set up.\n\nAll the setup needed for the tests are handled in the Makefile: \n- To pull the DVWA image, run ```make test-container-pull```\n- To setup the test files ```make testfiles-setup```\n- To run the container ```make test-container-run```\n- To run the tests ```make test```\n- To teardown the test files ```make testfiles-teardown```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaio-ishikawa%2Fnetscout","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcaio-ishikawa%2Fnetscout","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaio-ishikawa%2Fnetscout/lists"}