{"id":20294177,"url":"https://github.com/grab/secret-scanner","last_synced_at":"2025-04-11T11:42:44.368Z","repository":{"id":44330488,"uuid":"221413223","full_name":"grab/secret-scanner","owner":"grab","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-02T04:26:58.000Z","size":16206,"stargazers_count":49,"open_issues_count":4,"forks_count":12,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-25T08:03:15.915Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-13T08:47:37.000Z","updated_at":"2025-01-29T17:11:37.000Z","dependencies_parsed_at":"2024-11-14T15:35:34.896Z","dependency_job_id":"3886684a-b3b9-46d4-b069-349042bf1489","html_url":"https://github.com/grab/secret-scanner","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grab%2Fsecret-scanner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grab%2Fsecret-scanner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grab%2Fsecret-scanner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grab%2Fsecret-scanner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grab","download_url":"https://codeload.github.com/grab/secret-scanner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248385889,"owners_count":21094966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T15:28:08.354Z","updated_at":"2025-04-11T11:42:44.350Z","avatar_url":"https://github.com/grab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Secret Scanner\n\nSecret scanner is a command-line tool to scan Git repositories for any sensitive information such as private keys, API secrets and tokens, etc.\n\nIt does so by looking at file names, extensions, and content, attempting to match them against a list of signatures.\n\nThe tool is based on \u003ca href=\"https://github.com/michenriksen/gitrob\"\u003eGitrob\u003c/a\u003e, with added support for Gitlab and Bitbucket on top of Github.\n\n## Setup\n\n## Auth Tokens\n\nThe use of this tool requires you to set various Git provider (Github / Gitlab / Bitbucket) authentication token in your environment.\n\nYou can do so by:\n```\nexport GITHUB_TOKEN=my-token; secret-scanner -repos jquery/jquery\n```\n\nTo persist the various Git provider tokens, you can add them into your `.bash_profile` or create a `.env` file. See `.env.example`.\n\n### Skip Files\n\nYou can define paths to be excluded from scanning by defining them in a comma separated format in `.env` file.\n\n`SKIP_EXT` defines the file extensions to be excluded\n`SKIP_PATHS` defines the paths/files to be excluded if the path matches one of the patterns defined in the list\n`SKIP_TEST_PATHS` defines any test directories/files that you would like to skip. It is being kept separately from `SKIP_PATHS` because sometimes it may be useful to scan the test files as well. You can toggle to scan test files by giving `-skip-tests=false` in the CLI.\n\n## Usage\n\nFor `bool` CLI flags, use `=` between key-val pair. Eg `-ui=false`\n\n### Basic\n\nThe most basic usage requires a list of Github repository identifiers in the form of `org/repo`.\n\n```\n./secret-scanner -repos jquery/jquery\n```\n\nTo scan repositories in other Git providers, simply specify the Git provider name.\n\nFor Gitlab, provide the project ID instead of `org/repo`.\n\n```\n./secret-scanner -git bitbucket -repos litmis/mama\n./secret-scanner -git gitlab -repos 3836952\n```\n\nYou can scan multiple repositories from the same Git provider by providing multiple identifiers separated by commas.\n\n```\n./secret-scanner -repos jquery/jquery,lodash/lodash\n```\n\n### Local Scan\n\nBy default, the tool will attempt to make a clone before scanning the files.\n\nIf you already have a copy of the repository on local disk, you can do a local scan by specifying the `dir` parameter.\n\n```\n./secret-scanner -dir /dir/path/to/local/repository\n```\n\n### Sub-directory Scan\n\nIn instances where a repository contains multiple projects (i.e monorepo), or you simply want to scan specific sub-directory, you can do so by providing `sub-dir`.\n\nExample:\nhttps://github.com/user/awesome-projects contains\n- build/\n- dist/\n- src/\n- test/\n- ...\n\nCaveat: Only works with single `repos`\n\nTo scan `src` only:\n```\n./secret-scanner -repos jquery/jquery -sub-dir src\n```\n\n## Scan Results as Output\n\nBy default, findings found during the scan will be printed as console output. You can save it as JSON to path by specifying the `output` param\n\n```\n./secret-scanner -repos jquery/jquery -output ~/report.json\n```\n\nThe output file will contain the lines containing the potential secrets. In circumstances where you do not want to expose them, you can specify `-log-secret=false`\n\n## Scan State\n\nBy default, no scan state is being kept, meaning every scan on the same repository will start afresh.\n\nIf scan state is enabled, the scanner will save the latest scan session and commit hash in JSON format. From the next scan onwards for the same repository,the scanner will only scan changes since the last saved commit hash.\n\nThe default location of scan state JSON file is in `~/.secret-scanner/`.\n\n```\n./secret-scanner -repos jquery/jquery -use-state=true\n```\n\n## CLI Args\n\n```\n  -baseurl string\n        Specify Git provider base URL\n\n  -commit-depth int\n        Number of repository commits to process (default 500)\n\n  -debug\n        Print debugging information\n\n  -env string\n        .env file path containing Git provider base URLs and tokens\n\n  -git string\n        Name of git provider (Eg. github, gitlab, bitbucket) (default \"github\")\n\n  -load string\n        Load session file\n\n  -dir string\n        Specify the local git repo path to scan\n\n  -log-secret\n        If true, the matched secret will be included in output file (default true)\n\n  -output string\n        Save session to file\n\n  -repos string\n        Comma-separated list of repos to scan\n\n  -sub-dir string\n        Sub-directory within the repository to scan\n\n  -quiet\n        Suppress all output except for errors\n\n  -skip-tests\n        Skips possible test contexts (default true)\n\n  -use-state\n        If use-state is off, every scan will be treated as a brand new scan.\n\n  -threads int\n        Number of concurrent threads (default number of logical CPUs)\n\n  -token string\n        Specify Git provider token\n```\n\n## Credits\n\nProject is built upon the ground work laid in \u003ca href=\"https://github.com/michenriksen/gitrob\" target=\"_blank\"\u003eGitrob\u003c/a\u003e by \u003ca href=\"https://michenriksen.com/\" target=\"_blank\"\u003eMichael Henriksen\u003c/a\u003e.\n\nAnd many secret signatures was taken from \u003ca href=\"https://github.com/eth0izzle/shhgit/\" target=\"_blank\"\u003eshhgit\u003c/a\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrab%2Fsecret-scanner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrab%2Fsecret-scanner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrab%2Fsecret-scanner/lists"}