{"id":25074751,"url":"https://github.com/rix4uni/uforall","last_synced_at":"2025-04-15T00:12:41.695Z","repository":{"id":151865792,"uuid":"583635685","full_name":"rix4uni/uforall","owner":"rix4uni","description":"uforall is a fast url crawler this tool crawl all URLs number of different sources, alienvault,WayBackMachine,urlscan,commoncrawl","archived":false,"fork":false,"pushed_at":"2024-12-09T14:50:22.000Z","size":52,"stargazers_count":40,"open_issues_count":0,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-15T00:12:24.380Z","etag":null,"topics":["alienvault","bugbounty","commoncrawl","crawler","osint","recon","reconnaissance","urlscan","wayback"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rix4uni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-30T11:55:43.000Z","updated_at":"2025-04-09T19:59:04.000Z","dependencies_parsed_at":"2024-12-09T15:41:23.790Z","dependency_job_id":null,"html_url":"https://github.com/rix4uni/uforall","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fuforall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fuforall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fuforall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fuforall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rix4uni","download_url":"https://codeload.github.com/rix4uni/uforall/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248981270,"owners_count":21193147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alienvault","bugbounty","commoncrawl","crawler","osint","recon","reconnaissance","urlscan","wayback"],"created_at":"2025-02-07T00:18:28.000Z","updated_at":"2025-04-15T00:12:41.672Z","avatar_url":"https://github.com/rix4uni.png","language":"Go","readme":"## uforall\n\n**uforall is a fast url crawler this tool crawl all URLs number of different sources**\n### Sources \n- [alienvault](https://otx.alienvault.com)\n- [WayBackMachine](http://web.archive.org)\n- [urlscan](https://urlscan.io)\n- [commoncrawl](https://index.commoncrawl.org/)\n\n## Installation\n```\ngo install github.com/rix4uni/uforall@latest\n```\n\n## Download prebuilt binaries\n```\nwget https://github.com/rix4uni/uforall/releases/download/v0.0.2/uforall-linux-amd64-0.0.2.tgz\ntar -xvzf uforall-linux-amd64-0.0.2.tgz\nrm -rf uforall-linux-amd64-0.0.2.tgz\nmv uforall ~/go/bin/uforall\n```\nOr download [binary release](https://github.com/rix4uni/uforall/releases) for your platform.\n\n## Compile from source\n```\ngit clone --depth 1 github.com/rix4uni/uforall.git\ncd uforall; go install\n```\n\n## Usage\n```\nUsage of uforall:\n  -silent\n        silent mode.\n  -t string\n        Comma-separated list of tools to run: 'otx', 'archive', 'urlscan', 'commoncrawl', or 'all' (default \"all\")\n  -version\n        Print the version of the tool and exit.\n```\n\n## Usage Examples\n\nSingle URL:\n```\necho \"testphp.vulnweb.com\" | uforall\n```\n\nMultiple URLs:\n```\ncat subs.txt | uforall\n```\n\nRun specific tools:\n```\ncat subs.txt | uforall -t otx, urlscan\n```\n\n### Use Domain name instead of subdomain list you can save lot of time\n```\n# cat withoutprotocolsubs.txt\nrest.vulnweb.com\ntestasp.vulnweb.com\ntesthtml5.vulnweb.com\ntestaspnet.vulnweb.com\ntestphp.vulnweb.com\nvulnweb.com\n\n\n# With subdomains\ntime cat withoutprotocolsubs.txt | uforall -silent -t all 2\u003e/dev/null | unew -el -i -t | wc -l\n12953\n\nreal    0m30.728s\nuser    0m0.120s\nsys     0m0.013s\n\n\n# With domain, saved 25 seconds and output is same\ntime echo \"vulnweb.com\" | uforall -silent -t all 2\u003e/dev/null | unew -el -i -t | wc -l\n12953\n\nreal    0m6.447s\nuser    0m0.060s\nsys     0m0.035s\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frix4uni%2Fuforall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frix4uni%2Fuforall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frix4uni%2Fuforall/lists"}