{"id":25074703,"url":"https://github.com/rix4uni/jscrawler","last_synced_at":"2026-04-18T00:32:21.366Z","repository":{"id":151865754,"uuid":"579967084","full_name":"rix4uni/jscrawler","owner":"rix4uni","description":"Fetches javascript file from a list of URLS or subdomains.","archived":false,"fork":false,"pushed_at":"2025-11-13T04:14:26.000Z","size":33,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-13T06:08:43.632Z","etag":null,"topics":["bugbounty","hacking","javascript","pentesting","recon","reconnaissance","urls"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rix4uni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-12-19T11:45:44.000Z","updated_at":"2025-11-13T04:11:03.000Z","dependencies_parsed_at":"2024-11-06T11:28:09.120Z","dependency_job_id":null,"html_url":"https://github.com/rix4uni/jscrawler","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/rix4uni/jscrawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fjscrawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fjscrawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fjscrawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fjscrawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rix4uni","download_url":"https://codeload.github.com/rix4uni/jscrawler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rix4uni%2Fjscrawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31951320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bugbounty","hacking","javascript","pentesting","recon","reconnaissance","urls"],"created_at":"2025-02-07T00:18:20.114Z","updated_at":"2026-04-18T00:32:21.352Z","avatar_url":"https://github.com/rix4uni.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## jscrawler\n\nA high-performance tool for extracting JavaScript files from websites using concurrent crawling techniques.\n\n## Installation\n\n### Using Go Install\n```\ngo install github.com/rix4uni/jscrawler@latest\n```\n\n### Download Prebuilt Binaries\n```\nwget https://github.com/rix4uni/jscrawler/releases/download/v0.0.4/jscrawler-linux-amd64-0.0.4.tgz\ntar -xvzf jscrawler-linux-amd64-0.0.4.tgz\nrm -rf jscrawler-linux-amd64-0.0.4.tgz\nmv jscrawler ~/go/bin/jscrawler\n```\n\nOr download [binary release](https://github.com/rix4uni/jscrawler/releases) for your platform.\n\n### Compile from Source\n```\ngit clone --depth 1 https://github.com/rix4uni/jscrawler.git\ncd jscrawler; go install\n```\n\n## Usage\n```yaml\nUsage of jscrawler:\n      --complete        Get Complete URL (default false)\n  -o, --output string   Output file to save results\n      --silent          Silent mode.\n  -t, --threads int     Number of threads to use (default 50)\n      --timeout int     Timeout (in seconds) for http client (default 15)\n      --verbose         Enable verbose output for debugging purposes.\n      --version         Print the version of the tool and exit.\n```\n\n### Basic Syntax\n```yaml\ncat targets.txt | jscrawler [options]\n```\n\nor\n\n```yaml\necho \"https://example.com\" | jscrawler [options]\n```\n\n### Options\n\n| Flag | Short | Description | Default |\n|------|-------|-------------|---------|\n| `--complete` | | Output complete URLs (absolute paths) | `false` |\n| `--output` | `-o` | Save results to specified file | |\n| `--silent` | | Suppress banner and non-essential output | `false` |\n| `--threads` | `-t` | Number of concurrent threads | `50` |\n| `--timeout` | | HTTP client timeout in seconds | `15` |\n| `--verbose` | | Enable detailed debug output | `false` |\n| `--version` | | Display version information and exit | |\n\n## Examples\n\n### Basic Usage\n```yaml\n# Single target\necho \"https://example.com\" | jscrawler\n\n# Multiple targets from file\ncat subdomains.txt | jscrawler\n```\n\n### Complete URL Extraction\n```yaml\necho \"https://example.com\" | jscrawler --complete\n```\n\n### Save Results to File\n```yaml\ncat targets.txt | jscrawler --complete -o javascript_files.txt\n```\n\n### Advanced Operations\n```yaml\n# Verbose mode for debugging\necho \"https://example.com\" | jscrawler --verbose --complete\n\n# Silent operation in pipelines\ncat targets.txt | jscrawler --silent --complete\n\n# Custom performance tuning\ncat targets.txt | jscrawler --threads 100 --timeout 30\n```\n\n## Performance Comparison\n\njscrawler demonstrates superior coverage compared to similar tools:\n\n```yaml\n# Test against example.com\necho \"https://www.example.com\" | getJS --complete | wc -l\n# Output: 4\n\necho \"https://www.example.com\" | subjs | wc -l  \n# Output: 8\n\necho \"https://www.example.com\" | jscrawler --silent --complete | wc -l\n# Output: 13\n```\n\n## Best Practices\n\n### Input Preparation\n```yaml\n# Filter active hosts before processing\ncat domains.txt | httpx -silent | jscrawler --complete\n```\n\n### Output Management\n```yaml\n# Remove duplicates and sort results\ncat targets.txt | jscrawler --complete | unew | unique_js.txt\n```\n\n### Resource Optimization\n```yaml\n# Balance between speed and resource usage\ncat large_target_list.txt | jscrawler --threads 50 --timeout 20\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frix4uni%2Fjscrawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frix4uni%2Fjscrawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frix4uni%2Fjscrawler/lists"}