{"id":15889145,"url":"https://github.com/arshadkazmi42/github-scanner-local","last_synced_at":"2025-08-12T18:13:14.431Z","repository":{"id":96463238,"uuid":"424967074","full_name":"arshadkazmi42/github-scanner-local","owner":"arshadkazmi42","description":"Locally scan all the repositories of a github organization","archived":false,"fork":false,"pushed_at":"2025-07-20T08:34:57.000Z","size":32,"stargazers_count":16,"open_issues_count":0,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-20T10:22:06.090Z","etag":null,"topics":["bounty","bug","bug-bounty","crawler","github","local","no-api","scanner"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arshadkazmi42.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-05T13:58:27.000Z","updated_at":"2025-07-20T08:35:01.000Z","dependencies_parsed_at":"2024-01-21T18:28:16.507Z","dependency_job_id":"d644d547-c572-4a0c-ae54-38089ce45530","html_url":"https://github.com/arshadkazmi42/github-scanner-local","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arshadkazmi42/github-scanner-local","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arshadkazmi42%2Fgithub-scanner-local","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arshadkazmi42%2Fgithub-scanner-local/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arshadkazmi42%2Fgithub-scanner-local/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arshadkazmi42%2Fgithub-scanner-local/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arshadkazmi42","download_url":"https://codeload.github.com/arshadkazmi42/github-scanner-local/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arshadkazmi42%2Fgithub-scanner-local/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270110033,"owners_count":24528963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bounty","bug","bug-bounty","crawler","github","local","no-api","scanner"],"created_at":"2024-10-06T06:41:25.145Z","updated_at":"2025-08-12T18:13:14.416Z","avatar_url":"https://github.com/arshadkazmi42.png","language":"Shell","readme":"# github-scanner-local\n\nLocally scan all the repositories of a github organization.\n\n## Setup\n\n- Clone below repositories as same location as github-scanner-local\n  - [gh-crawl](https://github.com/arshadkazmi42/gh-crawl) \n  - [bash-scripts](https://github.com/arshadkazmi42/bash-scripts)\n\n- Install [npm-name-cli](https://github.com/sindresorhus/npm-name-cli) globally\n- Install [get-dependencies](https://github.com/SharonGrossman/get-dependencies) globally \n- Install [pip-name](https://github.com/danishprakash/pip-name)\n\n```\n$ npm i npm-name-cli -g\n\n$ npm i get-dependencies -g\n\n$ pip install pip-name\n```\n\n## Usage\n\n```\n$ sh run.sh {GITHUB_ORGANIZATION_NAME} {GITHUB_TOKEN}\n\n$ sh runNpmDeps.sh {FOLDER_NAME}\n\n$ sh runPipDeps.sh {FOLDER_NAME}\n```\n\n## Working\n\n#### `run.sh` \n  \nUses `clone.sh`, `grepURL.sh` and `broken.sh`\n\n#### `clone.sh`\n\nClones all the repositories of the organanization (Note: It does not clones `archived` repositories)\n\n#### `grepURL.sh`\n\nFind all the urls from all the repositories of the organization (Searches in the cloned folder) and stores in results directory\n\n#### `broken.sh`\n\nChecks all the urls found by `grepURL.sh` and captures the status code of the URL\n\n\n## Commands\n\n- Search for string in all cloned repositories\n\n```\nls -d -1 target/* | grep -v target.txt | xargs -I {} sh search.sh {} \"{{STRING_TO_SEARCH}}\"\n```\n\n```\ngrep -r -o \"{{STRING_TO_SEARCH}}\" target/.\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farshadkazmi42%2Fgithub-scanner-local","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farshadkazmi42%2Fgithub-scanner-local","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farshadkazmi42%2Fgithub-scanner-local/lists"}