{"id":19652687,"url":"https://github.com/advanced-security/secret-scanning-tools","last_synced_at":"2025-04-28T17:30:53.120Z","repository":{"id":216368353,"uuid":"564779171","full_name":"advanced-security/secret-scanning-tools","owner":"advanced-security","description":"Testing Suite for GitHub Secret Scanning Custom Patterns","archived":false,"fork":false,"pushed_at":"2024-07-10T17:04:52.000Z","size":263,"stargazers_count":1,"open_issues_count":6,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-07-10T20:38:25.931Z","etag":null,"topics":["github-advanced-security","secret-scanning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/advanced-security.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-11T13:31:32.000Z","updated_at":"2024-04-05T16:22:58.000Z","dependencies_parsed_at":"2024-01-09T21:28:23.385Z","dependency_job_id":"12fc0b1e-4c00-47bc-a861-e5cff2a50df3","html_url":"https://github.com/advanced-security/secret-scanning-tools","commit_stats":null,"previous_names":["advanced-security/secret-scanning-tools"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advanced-security%2Fsecret-scanning-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advanced-security%2Fsecret-scanning-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advanced-security%2Fsecret-scanning-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advanced-security%2Fsecret-scanning-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/advanced-security","download_url":"https://codeload.github.com/advanced-security/secret-scanning-tools/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224124862,"owners_count":17259746,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["github-advanced-security","secret-scanning"],"created_at":"2024-11-11T15:11:45.401Z","updated_at":"2024-11-11T15:11:46.240Z","avatar_url":"https://github.com/advanced-security.png","language":"Python","funding_links":[],"categories":["Secret Scanning Custom Patterns"],"sub_categories":[],"readme":"# Secret Scanning Tools\n\n\u003e ℹ️ This is an _unofficial_ tool created by Field Security Services, and is not officially supported by GitHub.\n\nThis is a testing suite for GitHub Secret Scanning Custom Patterns.\n\nIt can be used in combination with GitHub Actions to test custom patterns before they are deployed.\n\nAn example repository that uses this Action is [advanced-security/secret-scanning-custom-patterns](https://github.com/advanced-security/secret-scanning-custom-patterns).\n\nA sample custom patterns config file compatible with this tool suite is provided in [`examples/config/patterns.yml`](examples/config/patterns.yml).\n\n## Usage in Actions\n\n```yaml\n- name: Secret Scanning Test Suite\n  uses: advanced-security/secret-scanning-tools@v1\n```\n\n### Advanced Configuration\n\n```yaml\n- name: Secret Scanning Test Suite\n  uses: advanced-security/secret-scanning-tools@v1\n  with:\n    # Modes to run\n    # \u003e 'validate' (default), 'all', 'snapshot', 'markdown'\n    mode: 'validate'\n```\n\n### Using GitHub App Token\n\n```yaml\n- name: Get Token\n  id: get_workflow_token\n  uses: peter-murray/workflow-application-token-action@v1\n  with:\n    application_id: ${{ secrets.ADVANCED_SECURITY_APP_ID }}\n    application_private_key: ${{ secrets.ADVANCED_SECURITY_APP_KEY }}\n\n- name: Secret Scanning Test Suite\n  uses: advanced-security/secret-scanning-tools@v1\n  with:\n    token: ${{ steps.get_workflow_token.outputs.token }}\n```\n\n### Using in Enterprise Server or with self-hosted Actions runners\n\nThe Action will set up Python on GitHub Hosted runners or on runners with a configured tool cache.\n\nIf you are using a self-hosted runner, or Enterprise Server, and you have not set up the tool cache, you will need to set up Python yourself (version 3.9+)\n\nYou can do that in the workflow, or by installing it permanently into your runner's image and including it in the path of the Actions runner.\n\n## Defining expected results for online testing\n\nCreate a snapshot of the results of a pattern by running the action with `snapshot`.\n\nThis is checked in the `validate` mode, to check for changes in the results of the pattern. Check that these changes are expected, and if so, update the snapshot, or fix the pattern, as necessary.\n\n\u003e ℹ️ the _earliest commit_ that a secret has been found at is reported by secret scanning, so it is not possible to cleanly define a current state of expected secrets in the repository, and test for those expected results. Instead, we use this overall snapshot approach.\n\u003e\n\u003e In contrast, [offline testing, below](#offline-testing-of-secret-scanning-custom-patterns) can be used to test for expected results, since it can be run on a single commit of a repository.\n\n## Using locally with pipenv\n\nInstall the requirements using `pipenv install`, then run `pipenv run \u003ccommand\u003e` to run the commands: `markdown`, `validate`, `snapshot`.\n\nSee [this sample script](./examples/update_custom_patterns_readme.sh) for how to update the `README.md` files for your custom patterns.\n\n## Offline testing of Secret Scanning custom patterns\n\nWe have a test Python script, `secretscanning/test.py` that uses Intel's `hyperscan` to test custom GitHub Advanced Security Secret Scanning patterns.\n\nThis is useful for thorough testing of patterns before they are deployed, whereas the rest of the test suite is primarily designed to be run in GitHub Actions for testing in CI.\n\n### Local test script usage\n\nChange directory to `secretscanning`.\n\nFirst run `make requirements` to install required dependencies.\n\n``` bash\n./test.py\n```\n\nBy default it searches the directory above the `testing` directory for `pattern.yml` files, and tests those patterns on the same directory that file was found in.\n\nor\n\n``` bash\n./test.py --tests \u003cdirectory\u003e\n```\n\nFor full usage use `./test.py --help`\n\n### Local test script requirements\n\nThis only works on Intel-compatible platforms, since `hyperscan` is an Intel application and written to use Intel-specific instructions.\n\nThe packages can be installed using `make requirements` on Ubuntu-compatible platforms. On other platforms, install the following dependencies:\n\n* Python 3.9+\n* Python packages (listed in `requirements.txt`)\n  * `hyperscan`\n  * `python-pcre` (requires `libpcre3`)\n  * `pygit2`\n  * `GitPython`\n\n### Development notes\n\nPlease run `make lint` after any changes\n\n## License\n\nThis project is licensed under the terms of the MIT open source license. Please refer to the [LICENSE](LICENSE) for the full terms.\n\n## Maintainers\n\nSee [CODEOWNERS](CODEOWNERS) for the list of maintainers.\n\n## Support\n\n\u003e ℹ️ This is an _unofficial_ tool created by Field Security Services, and is not officially supported by GitHub.\n\nSee the [SUPPORT](SUPPORT.md) file.\n\n## Background\n\nSee the [CHANGELOG](CHANGELOG.md), [CONTRIBUTING](CONTRIBUTING.md), [SECURITY](SECURITY.md), [SUPPORT](SUPPORT.md), [CODE OF CONDUCT](CODE_OF_CONDUCT.md) and [PRIVACY](PRIVACY.md) files for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvanced-security%2Fsecret-scanning-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadvanced-security%2Fsecret-scanning-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvanced-security%2Fsecret-scanning-tools/lists"}