{"id":43637960,"url":"https://github.com/austimkelly/secretsynth","last_synced_at":"2026-02-04T17:36:53.809Z","repository":{"id":218553808,"uuid":"746666950","full_name":"austimkelly/secretsynth","owner":"austimkelly","description":"A secret scanner wrapper to aggregate results across multiple secret scanning tools","archived":false,"fork":false,"pushed_at":"2024-02-09T14:54:26.000Z","size":450,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-02-09T15:32:00.340Z","etag":null,"topics":["code-scanning","ghas","gitleaks","secrets","secrets-detection","secrets-scanner","trufflehog"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/austimkelly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-01-22T13:24:24.000Z","updated_at":"2024-02-09T15:32:29.017Z","dependencies_parsed_at":"2024-02-09T15:42:18.216Z","dependency_job_id":null,"html_url":"https://github.com/austimkelly/secretsynth","commit_stats":null,"previous_names":["austimkelly/gitleaks-utils","austimkelly/secretsynth"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/austimkelly/secretsynth","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/austimkelly%2Fsecretsynth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/austimkelly%2Fsecretsynth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/austimkelly%2Fsecretsynth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/austimkelly%2Fsecretsynth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/austimkelly","download_url":"https://codeload.github.com/austimkelly/secretsynth/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/austimkelly%2Fsecretsynth/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29092037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T03:31:03.593Z","status":"ssl_error","status_checked_at":"2026-02-04T03:29:50.742Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-scanning","ghas","gitleaks","secrets","secrets-detection","secrets-scanner","trufflehog"],"created_at":"2026-02-04T17:36:53.120Z","updated_at":"2026-02-04T17:36:53.800Z","avatar_url":"https://github.com/austimkelly.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Python version](https://img.shields.io/badge/python-3.9.6-blue.svg)\n![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)\n[![Run Unit Tests](https://github.com/austimkelly/secretsynth/actions/workflows/unit_tests.yml/badge.svg)](https://github.com/austimkelly/secretsynth/actions/workflows/unit_tests.yml)\n\n# 🕵️ Secret Synth\n\n\u003cspan style='font-family: \"Lucida Console\";line-height: 14px;font-size: 14px;display: inline-block;'\u003e┏━━━┓━━━━━━━━━━━━━━━━┏┓━┏━━━┓━━━━━━━━━━┏┓━┏┓━━\u003cbr\u003e┃┏━┓┃━━━━━━━━━━━━━━━┏┛┗┓┃┏━┓┃━━━━━━━━━┏┛┗┓┃┃━━\u003cbr\u003e┃┗━━┓┏━━┓┏━━┓┏━┓┏━━┓┗┓┏┛┃┗━━┓┏┓━┏┓┏━┓━┗┓┏┛┃┗━┓\u003cbr\u003e┗━━┓┃┃┏┓┃┃┏━┛┃┏┛┃┏┓┃━┃┃━┗━━┓┃┃┃━┃┃┃┏┓┓━┃┃━┃┏┓┃\u003cbr\u003e┃┗━┛┃┃┃━┫┃┗━┓┃┃━┃┃━┫━┃┗┓┃┗━┛┃┃┗━┛┃┃┃┃┃━┃┗┓┃┃┃┃\u003cbr\u003e┗━━━┛┗━━┛┗━━┛┗┛━┗━━┛━┗━┛┗━━━┛┗━┓┏┛┗┛┗┛━┗━┛┗┛┗┛\u003cbr\u003e━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┏━┛┃━━━━━━━━━━━━━\u003cbr\u003e━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┗━━┛━━━━━━━━━━━━━\u003cbr\u003e\u003c/span\u003e\n    \nA python-based tool for discovering secrets with multiple secret scanning solutions across multiple orgs, and repositories. Secret Synth leverages the following secret scanning tools:\n\n* gitleaks\n* trufflehog\n* Github Advanced Security\n* Nosey Parker\n\nOnly pulls from GitHub are currently supported.\n\n## ❓ Why Secret Synth?\n\nIn short, to help you: \n\n* Identify what secrets scanning solution(s) and strategy you may want to deploy.\n* Know how many secrets may be discovered in your repositories (like a dry run).\n* Identify classes of secrets that are being identified in your source code.\n* Identify true positives and remediate them before rolling out a secrets alerting solution.\n* Identify false positives and tune your alerting system to eliminate noisy alerts (which decreases developer productivity).\n\n# 📄 License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details. As such, the software is provided \"as-is\" and without warranty. The authors are not responsible for any damages or liabilities that may arise from the use of the software.\n\n## Pre-requisites\n\n* Python 3.6+\n* A Github account with sufficient permissions to access the target repositories\n* A Github access token with sufficient permissions get a listing of repositories from the Github REST API\n* `git` installed and in your PATH\n* `gitleaks` installed and in your PATH\n* `trufflehog` installed and in your PATH\n* `noseyparker` installed and in your PATH\n\n### Versions\n\nThe following versions were used during development:\n\n* noseyparker, [v0.16.0](https://github.com/praetorian-inc/noseyparker/releases/tag/v0.16.0).\n* gitleaks, [v8.18.1](https://github.com/gitleaks/gitleaks/releases/tag/v8.18.1)\n* trufflehog, [v3.66.2](https://github.com/trufflesecurity/trufflehog/releases/tag/v3.66.2)\n* GitHub REST API, [API Version 2022-11-28](https://docs.github.com/en/rest?apiVersion=2022-11-28)\n\n## Installation\n\n1. Clone this repository:\n\n`git clone https://github.com/yourusername/yourrepository.git`\n\n2. Install the required Python packages:\n\n`pip install -r requirements.txt`\n\n3. Install the other required secrets scanners.\n\n## 🛠️ Usage\n\nHere's the command-line help:\n\n```\nusage: secretsynth.py [-h] [--clean] [--dry-run] [--keep-secrets-in-reports] [--repos-internal-type]\n                      [--org-type {users,orgs}] [--owners OWNERS] [--skip-noseyparker] [--skip-trufflehog]\n                      [--skip-ghas] [--skip-gitleaks] [--open-report-in-browser]\noptional arguments:\n  -h, --help            show this help message and exit\n  --clean               delete the directories ./checkouts and ./reports. When --clean is present all other commands are\n                        ignored.\n  --dry-run             run the script in dry run mode, don't execute any commands\n  --keep-secrets-in-reports\n                        Keep plain text secrets in the aggregated reports. By default the tool will hash secrets for final reports if this flag is missing.\n  --repos-internal-type\n                        If your repositories are internal, this flag will be added when fetching repositories from Github.\n  --open-report-in-browser\n                        Open the report in a browser after it's generated\n  --org-type {users,orgs}\n                        set the organization type\n  --owners OWNERS       comma-delimited list of owners\n  --skip-noseyparker    Skip the Noseyparker scan\n  --skip-trufflehog     Skip the TruffleHog scan\n  --skip-ghas           Skip the GitHub Advanced Security scan\n  --skip-gitleaks       Skip the Gitleaks scan\n  --open-report-in-browser\n                        Open the report in a browser after it's generated\n```\n\n1. Set your GitHub access token as an environment variable:\n\n`export GITHUB_ACCESS_TOKEN=youraccesstoken`\n\nSee [Managing your personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) for more information. You will only need the ability to list repositories so the script will know what to checkout via `git checkout`\n\n2. Review [.gitleaks.toml](./org-scan/.gitleaks.toml) for path and file exclusions. Modify as necessary.\n\nGitleaks can generate a lot of false positives out of the box. So review results carefully and add exclusions as necessary to minimize false positives.\n\n3. Run the `secretsynth` script from the `org-scan` directory:\n\n## Sample Command-Line Executions\n\nHere are some examples of use cases for running the script:\n\n**Example**: Running on a personal owner account:\n\n🚀🚀 This example uses some small open source repos with secrets in them. Hence, this is a good test for you to quickly evaluate the setup is correct for your environment. 🚀🚀\n\n`python3 secretsynth.py --org-type users --owners swell-consulting --skip-ghas --open-report-in-browser`\n\n**Example**: Running on a personal owner account and keeping plain text secrets in the output, but omit trufflehog from the execution:\n\n`python3 secretsynth.py --org-type users --owners austimkelly --keep-secrets-in-reports --skip-trufflehog`\n\n**Example**: Running on multiple organizations:\n\n`python3 secretsynth.py --org-type orgs --owners org1,org2,org3`\n\nNote: Multiple Github Personal Access Tokens are not supported yet.\n\n**Example**: Cleaning up source and scanning artifacts:\n\n`python3 secretsynth.py --clean`\n\n## 📈 Analyzing Results\n\nAfter the script has finished running, you can find the consolidated reports in the `./org-scan/reports/reports_\u003cYYYYMMDDHHMM\u003e` directory. An HTML file in that directory contains a short summary of the results, CSV artifacts with merged alerts, and an error log for any tool failures you want to investigate.\n\nHere's an example of the output:\n\n![html report](./doc/html_report.png)\n\nYou can further analyze the data in your favorite spreadsheet or data warehouse:\n\n![csv report](./doc/secrets_report.png)\n\n# 🏗️ Call sequence diagram\n\n```mermaid\nsequenceDiagram\n  participant User\n  participant Script as secretsynth.py\n  participant GitHub\n  participant FileSystem as Local File System\n  User-\u003e\u003eScript: Run script\n  loop for each organization\n    Script-\u003e\u003eGitHub: Request list of repositories for organization\n    GitHub--\u003e\u003eScript: Return list of repositories\n    loop for each repository\n      Script-\u003e\u003eGitHub: Clone repository\n      Script-\u003e\u003eFileSystem: Save repository to local file system\n      Script-\u003e\u003eScript: Run gitleaks on cloned repository\n      Script-\u003e\u003eFileSystem: Save gitleaks report to local file system\n      Script-\u003e\u003eScript: Run TruffleHog on cloned repository\n      Script-\u003e\u003eFileSystem: Save TruffleHog report to local file system\n      Script-\u003e\u003eScript: Run Nosey Parker on cloned repository\n      Script-\u003e\u003eFileSystem: Save Nosey Parker report to local file system\n      Script-\u003e\u003eGitHub: Fetch GHAS secret alerts for repository\n      GitHub--\u003e\u003eScript: Return GHAS secret alerts\n      Script-\u003e\u003eFileSystem: Save GHAS secret alerts to local file system\n    end\n  end\n```\n\n# ⚙️ Github Actions with Secrets Scanners\n\n## Gitleaks as a Github Action\n\nThis repository also contains a Github Action that can be used to scan a repository for secrets using Gitleaks. The action is located in the [.github/actions/gitleaks](.github/workflows/gitleaks.yml) directory. \n\nNOTE: That running gitleaks against a repo owned by a user is free. A repository owned by an organization will require a free API key. See [Obtaining a Gitleaks License](https://gitleaks.io/products.html)\n\n## Trufflehog as a Github Action\n\nSee [Trufflehog Github Action](https://github.com/marketplace/actions/trufflehog-oss)\n\n# 📚 References\n\n* [Gitleaks](https://gitleaks.io/)\n* [TruffleHog](https://github.com/trufflesecurity/trufflehog)\n* [Github Advanced Security](https://docs.github.com/en/github/getting-started-with-github/about-github-advanced-security)\n* [A Comparative Study of Software Secrets Reporting by Secret Detection Tools](https://arxiv.org/pdf/2103.01946.pdf)\n* [SecretBench](https://github.com/setu1421/SecretBench) - Reference data set used by the comparative paper above.\n* [secrets-magpie](https://github.com/punk-security/secret-magpie) - A similar tool to this one with more source scanning options.\n* [Meet Nosey Parker — An Artificial Intelligence Based Scanner That Sniffs Out Secrets](https://www.praetorian.com/blog/nosey-parker-ai-secrets-scanner-release/) - Some really good data science work here on the precision of Nosey Parker.\n\n# ⚠️ Limitations and Known Issues\n\n* The script does not support multiple Github Personal Access Tokens yet. When pulling GHAS Secert Alerts for multiple orgs, this will only use one token.\n* Internal repositories are treated with a separate flag. If you have a mix of internal, private, and public repositories in an org, you will have incomplete results.\n* Matching does not happen with GHAS Secret Alerts. The API does not return secrets, line or file numbers in the alerts. As such\n* Only clones from Github are supported. Adding filesystem and other repos could be done upon request.\n* Git fetch depth or custom scan depths on tools is not supported.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faustimkelly%2Fsecretsynth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faustimkelly%2Fsecretsynth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faustimkelly%2Fsecretsynth/lists"}