{"id":14008093,"url":"https://github.com/owenrumney/squealer","last_synced_at":"2025-05-14T16:06:39.245Z","repository":{"id":37719097,"uuid":"336026438","full_name":"owenrumney/squealer","owner":"owenrumney","description":"Telling tales on you for leaking secrets!","archived":false,"fork":false,"pushed_at":"2025-03-19T10:23:43.000Z","size":51899,"stargazers_count":232,"open_issues_count":2,"forks_count":13,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-05T19:06:47.720Z","etag":null,"topics":["aws","code-scanning","devsecops","git-tool","go","golang","hacktoberfest","leak-detection","leaking-secrets","sarif-report","secrets","security","security-tools","static-analysis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/owenrumney.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"owenrumney"}},"created_at":"2021-02-04T17:12:26.000Z","updated_at":"2025-03-20T15:37:32.000Z","dependencies_parsed_at":"2022-09-07T06:02:46.208Z","dependency_job_id":"c47295a9-4b29-4878-ad69-01ca4a5aaa9f","html_url":"https://github.com/owenrumney/squealer","commit_stats":{"total_commits":156,"total_committers":8,"mean_commits":19.5,"dds":0.5256410256410257,"last_synced_commit":"07378a349a961e998a67935f3b27ba84e62d6ea6"},"previous_names":["owenrum/squealer"],"tags_count":53,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owenrumney%2Fsquealer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owenrumney%2Fsquealer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owenrumney%2Fsquealer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owenrumney%2Fsquealer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/owenrumney","download_url":"https://codeload.github.com/owenrumney/squealer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631668,"owners_count":21136554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","code-scanning","devsecops","git-tool","go","golang","hacktoberfest","leak-detection","leaking-secrets","sarif-report","secrets","security","security-tools","static-analysis"],"created_at":"2024-08-10T11:01:13.169Z","updated_at":"2025-04-12T20:38:41.295Z","avatar_url":"https://github.com/owenrumney.png","language":"Go","funding_links":["https://github.com/sponsors/owenrumney"],"categories":["Go","\u003ca name=\"Go\"\u003e\u003c/a\u003eGo"],"sub_categories":[],"readme":"![Squealer](.github/images/squealer2.png)\n\n# Squealer\n\n### Telling tales on you for leaking secrets!\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/owenrumney/squealer)](https://goreportcard.com/report/github.com/owenrumney/squealer)\n[![Github Release](https://img.shields.io/github/release/owenrumney/squealer.svg)](https://github.com/owenrumney/squealer/releases)\n[![GitHub All Releases](https://img.shields.io/github/downloads/owenrumney/squealer/total)](https://github.com/owenrumney/squealer/releases)\n\nSquealer scans a git repository or filesystem for secrets that are being leaked deep within the commit history.\n\n![Squealer](.github/images/squealer.gif)\n\nThe built-in configuration includes the following checks;\n\n- AnsibleVault\n- AWS Manager ID\n- AWS MWS key\n- AWS Secret Key\n- Base64 Encoded Certificates\n- DomainPassword\n- DSA\n- Dynatrace token\n- Facebook Creds\n- GitCredential\n- Github\n- Google API key\n- Heroku API key\n- Keychain file\n- KeyStoreFile\n- LinkedIn Creds\n- MailChimp API key\n- Mailgun API key\n- NPM Token\n- OAuth Token\n- OpenAI Secret Key\n- OPENSSH\n- OpenVPN\n- Password literal text\n- PayPal Braintree access token\n- PGP\n- Picatic API key\n- Postgres password\n- PublishSettings\n- RSA\n- Shopify credentials\n- Slack credentials\n- SQL Connection Strings\n- Square credentials\n- Stripe API key\n- Twilio API key\n- Twitter credentials\n\nSometimes we have secrets committed to our projects, generally, we can invalidate them and move on. If Squealer is telling tales about a secret that you are aware of and has been mitigated, you can use the `exception` rule found in the output to register it as ignored.\n\n## Installation\n\n```bash\ncurl -s \"https://raw.githubusercontent.com/owenrumney/squealer/main/scripts/install.sh\" | bash\n```\n\n## Usage\n\nSquealer is intended to be run either locally or as part of a CI process.\n\n```shell\n./squealer --help\nTelling tales on your secret leaking\n\nUsage:\n  squealer [flags]\n\nFlags:\n      --concise                Reduced output.\n      --config-file string     Path to the config file with the rules.\n      --debug                  Include debug output.\n      --everything             Scan all commits.... everywhere.\n      --from-hash string       The hash to work back to from the starting hash.\n  -h, --help                   help for squealer\n      --no-git                 Scan as a directory rather than a git history.\n      --output-format string   The format that the output should come in (default, json, sarif.\n      --redacted               Display the results redacted.\n      --to-hash string         The most recent hash to start with.\n```\n\n### Scan remote repositories\n\n```shell\nsquealer git@github.com:owenrumney/woopsie.git\n```\n\n### Scan local repositories as a directory\n\n```shell\nsquealer --no-git /path/to/repo\n```\n\n### Config File\n\n```yaml\nrules:\n  - rule: (A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}\n    description: Check for AWS Access Key Id\n  - rule: (?i)aws(.{0,20})?(?-i)['\\\"][0-9a-zA-Z\\/+]{40}['\\\"]\n    description: Check for AWS Secret Access Key\n  - rule: (?i)github[_\\-\\.]?token[\\s:,=\"\\]']+?(?-i)[0-9a-zA-Z]{35,40}\n    description: Check for Github Token\n  - rule: https://hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24}\n    description: Check for Slack webhook\n  - rule: xox[baprs]-([0-9a-zA-Z]{10,48})?\n    description: Check for Slack token\n  - rule: \"-----BEGIN ((EC|PGP|DSA|RSA|OPENSSH) )?PRIVATE KEY( BLOCK)?-----\"\n    description: Check for Private Asymetric Key\nignore_paths:\n  - vendor\n  - node_modules\nignore_extensions:\n  - .zip\n  - .png\n  - .jpg\n  - .pdf\n  - .xls\n  - .doc\n  - .docx\nexceptions:\n  - exception: release/update.go:D2IDetI6aidl58GE6dv5uAaWmXM=\n    reason: This is a webhook that we got rid of - can be ignored in this file\n```\n\n### Config breakdown\n\nThe config file is made up of the `rules`, `ignore_prefixes`, `ignore_extensions` and `exceptions`.\n\n#### rules\n\nRules define the regular expression that is used to detect the secret. Requires a description for posterity.\n\n#### ignore_paths\n\nIgnore paths are folders that you don't want to look ing - generally `vendor` and the like.\n\n#### ignore_extensions\n\nIgnore extensions that have these file types that won't be scanned. Binaries are automatically ignored.\n\n#### exceptions\n\nExceptions are the entries that you've already handled and don't want to be reported any more.\n\n## Example Output\n\n```shell\nINFO[0000] Using a git scanner to process ../../tfsec/tfsec\nINFO[0000] starting at hash 3bd04e7e17f2aad9e5f38826d88325798534a289\n\nContent:      | access_key = \"AKIAABCD12ABCDEF1ABC\"\nFilename:     | internal/app/tfsec/checks/aws044.go\nLine No:      | 21\nSecret Hash:  | bcE9jU2WV11OYs63eGHPZf1l9v8=\nCommit:       | 4e68e1c5b3bc66982e4b7e6c5cc1c1642c87f83d\nCommitter:    | GitHub (noreply@github.com)\nCommitted:    | 2020-10-21 21:59:22 +0100 +0100\nExclude rule: | internal/app/tfsec/checks/aws044.go:bcE9jU2WV11OYs63eGHPZf1l9v8=\n\nContent:      | access_key = \"AKIAABCD12ABCDEF1ABC\"\nFilename:     | docs-website/docs/aws/AWS044.md\nLine No:      | 26\nSecret Hash:  | bcE9jU2WV11OYs63eGHPZf1l9v8=\nCommit:       | 8a7715f2cf5a2ac74a1e186792c476fd52ee1474\nCommitter:    | ¨Owen Rumney (owen.rumney@form3.tech)\nCommitted:    | 2021-01-24 19:04:27 +0000 +0000\nExclude rule: | docs-website/docs/aws/AWS044.md:bcE9jU2WV11OYs63eGHPZf1l9v8=\n\nProcessing:\n  duration:     2.99s\n  commits:      503\n  commit files: 4095\n\ntransgressionMap:\n  identified:   6\n  ignored:      0\n  reported:     2\n\n\nINFO[0002] Exit code: 1\n\n```\n\nIt's worth noting that these are known because they're examples in the documentation for tfsec - I can add them to the `config.yaml` as exclusions y using the `Exclude rule`\n\n## Using in Docker\n\n`squealer` is built in Docker images for `amd64`, `arm64`, and `armv7` as `owenrumney/squealer`. You can run it against a mounted volume using \n\n```bash\ndocker run --rm -v `pwd`:/src owenrumney/squealer:latest /src --redacted    \n```\n\nin this example it returns\n\n```bash\ntime=\"2025-01-29T13:38:20Z\" level=warning msg=\"Config file '' not found, using default config\"\ntime=\"2025-01-29T13:38:20Z\" level=info msg=\"Using a git scanner to process /src\"\ntime=\"2025-01-29T13:38:20Z\" level=info msg=\"starting at hash 27958219bc1ad0ecd052cd2e092aea945d69d5d4\"\n\nProcessing:\n  duration:     0.91s\n  commits:      66\n  files:        468\n\nTransgressions:\n  identified:   0\n  ignored:      0\n  reported:     0\n\n```\n\n## Using as a library\n\nSquealer can be used for scanning a specific string using either the default config or by passing your own file in.\n\n### Adding the library\n\n```\ngo get -u github.com/owenrumney/squealer\n```\n\n### Using as a library\n\n#### Git and Directory Scanning\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/owenrumney/squealer/pkg/squealer\"\n\t\"github.com/owenrumney/squealer/pkg/config\"\n)\n\nfunc main() {\n\n\t// create a new scanner (optionally load your own config in)\n\tscanner, err :=  squealer.New(\n\t\tsquealer.OptionWithConfig(config.DefaultConfig()), // if not supplied , config.DefaultConfig() used\n\t\tsquealer.OptionRedactedSecrets(true), // defaults to true, secrets in output redacted\n\t\tsquealer.OptionNoGitScan(false), // Treat Directories with .git in them as Directories, defaults to false\n\t\tsquealer.OptionWithBasePath(\".\"), // The path to scan, default is '.'\n\t\tsquealer.OptionWithFromHash(\"\"), // Specify the starting hash for the scan, useful for PRs\n\t\tsquealer.OptionWithToHash(\"\"), // Specify the hash to stop scanning, useful for PRs scanning\n\t\tsquealer.OptionWithScanEverything(false), // Scan everything in every branch, defaults to only the current branch\n\t\tsquealer.OptionWithCommitListFile(\"\"), // a file of commits that you want to explicitly scan in a text file.\n\t)\n\n\ttransgressions, err := scanner.Scan()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tfor _, t := range transgressions {\n\t\tfmt.Printf(\"%s[%d]\\n\", t.Filename, t.LineNo)\n    }\n}\n\n\n```\n\n#### String Scanning\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/owenrumney/squealer/pkg/squealer\"\n)\n\nfunc main() {\n\n  // create a new scanner (optionally load your own config in)\n\tscanner := squealer.NewStringScanner()\n\ttestString := `password = \"superSecretPassword\"`\n\n  // scan the string and if a transgression is found, report it.\n\tif result := scanner.Scan(testString); result.TransgressionFound {\n\t\tfmt.Printf(\"found an issue in [%s]. %s\\n\", testString, result.Description)\n\t}\n}\n\n```\n\n## Credits\n\nImage by George Rumney\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowenrumney%2Fsquealer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fowenrumney%2Fsquealer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowenrumney%2Fsquealer/lists"}