{"id":13542306,"url":"https://github.com/Skyscanner/whispers","last_synced_at":"2025-04-02T09:33:33.788Z","repository":{"id":37930584,"uuid":"260993812","full_name":"Skyscanner/whispers","owner":"Skyscanner","description":"Identify hardcoded secrets in static structured text","archived":true,"fork":false,"pushed_at":"2023-10-11T12:08:40.000Z","size":313,"stargazers_count":478,"open_issues_count":10,"forks_count":72,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-12-17T22:03:47.027Z","etag":null,"topics":["credentials","devsecops","hardcoded","json-parser","leaks","lint","linter","parser","passwords","pipeline","secrets","secrets-detection","secrets-scan","security","security-compliance","structured-data","structured-text","xml-parser","yaml-parser"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Skyscanner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-03T18:23:35.000Z","updated_at":"2024-11-28T05:49:50.000Z","dependencies_parsed_at":"2023-10-11T12:41:58.049Z","dependency_job_id":null,"html_url":"https://github.com/Skyscanner/whispers","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fwhispers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fwhispers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fwhispers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fwhispers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Skyscanner","download_url":"https://codeload.github.com/Skyscanner/whispers/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246789366,"owners_count":20834282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["credentials","devsecops","hardcoded","json-parser","leaks","lint","linter","parser","passwords","pipeline","secrets","secrets-detection","secrets-scan","security","security-compliance","structured-data","structured-text","xml-parser","yaml-parser"],"created_at":"2024-08-01T10:01:04.568Z","updated_at":"2025-04-02T09:33:33.246Z","avatar_url":"https://github.com/Skyscanner.png","language":"Python","funding_links":[],"categories":["Secrets Scanning","Miscellaneous","Python","Python (1887)","Application Security","security","Pre-commit \u0026 Secrets Detection"],"sub_categories":["Kubernetes Audit","Secrets","Secrets detection"],"readme":"# Whispers \u003cimg src=\"whispers.png\" width=\"40px\" alt=\"Whispers\" style=\"\"\u003e \n\n[![](https://img.shields.io/pypi/v/whispers.svg)](https://pypi.python.org/pypi/whispers/)\n[![](https://github.com/Skyscanner/whispers/workflows/build/badge.svg)](https://github.com/Skyscanner/whispers/actions)\n![](coverage.svg)\n[![](https://img.shields.io/github/issues/Skyscanner/whispers)](https://github.com/Skyscanner/whispers/issues)\n[![](https://img.shields.io/github/issues-pr/Skyscanner/whispers)](https://github.com/Skyscanner/whispers/pulls)\n[![](https://img.shields.io/pypi/dm/whispers)](https://img.shields.io/pypi/dm/whispers)\n\n\u003e \"My little birds are everywhere, even in the North, they whisper to me the strangest stories.\" - _Lord Varys_\n\nWhispers is a static code analysis tool designed for parsing various common data formats in search of hardcoded credentials and dangerous functions. Whispers can run in the CLI or you can integrate it in your CI/CD pipeline.\n\n\n## Detects\n* Passwords\n* API tokens\n* AWS keys\n* Private keys\n* Hashed credentials\n* Authentication tokens\n* Dangerous functions\n* Sensitive files\n\n## Supported Formats\nWhispers is intended to be a **structured text** parser, not a code parser.\n\nThe following commonly used formats are currently supported:\n* YAML\n* JSON\n* XML\n* .npmrc\n* .pypirc\n* .htpasswd\n* .properties\n* pip.conf\n* conf / ini\n* Dockerfile\n* Dockercfg\n* Shell scripts\n* Python3\n\nPython3 files are parsed as ASTs because of native language support.\n\n## Declaration \u0026 Assignment Formats\nThe following language files are parsed as text, and checked for common variable declaration and assignment patterns:\n* JavaScript\n* Java\n* Go\n* PHP\n\n## Special Formats\n* AWS credentials files\n* JDBC connection strings\n* Jenkins config files\n* SpringFramework Beans config files\n* Java Properties files\n* Dockercfg private registry auth files\n* Github tokens\n\n## Installation\n\n### From PyPI\n```\npip3 install whispers\n```\n\n### From GitHub\n```\ngit clone https://github.com/Skyscanner/whispers\ncd whispers\nmake install\n```\n\n## Usage\n### CLI\n```\nwhispers --help\nwhispers --info\nwhispers source/code/fileOrDir\nwhispers --config config.yml source/code/fileOrDir\nwhispers --output /tmp/secrets.yml source/code/fileOrDir\nwhispers --rules aws-id,aws-secret source/code/fileOrDir\nwhispers --severity BLOCKER,CRITICAL source/code/fileOrDir\nwhispers --exitcode 7 source/code/fileOrDir\n```\n### Python\n```python\nfrom whispers.cli import parse_args\nfrom whispers.core import run\n\nsrc = \"tests/fixtures\"\nconfigfile = \"whispers/config.yml\"\nargs = parse_args([\"-c\", configfile, src])\nfor secret in run(args):\n  print(secret)\n```\n\n## Config\nThere are several configuration options available in Whispers. It’s possible to include/exclude results based on file path, key, or value. File path specifications are interpreted as globs. Keys and values accept regular expressions and several other parameters. There is a default configuration file built-in that will be used if you don't provide a custom one.\n\n`config.yml` should have the following structure:\n```yaml\ninclude:\n  files:\n    - \"**/*.yml\"\n\nexclude:\n  files:\n    - \"**/test/**/*\"\n    - \"**/tests/**/*\"\n  keys:\n    - ^foo\n  values:\n    - bar$\n\nrules:\n  starks:\n    message: Whispers from the North\n    severity: CRITICAL\n    value:\n      regex: (Aria|Ned) Stark\n      ignorecase: True\n```\n\nThe fastest way to tweak detection (ie: remove false positives and unwanted results) is to copy the default [config.yml](whispers/config.yml) into a new file, adapt it, and pass it as an argument to Whispers.\n\n`whispers --config config.yml --rules starks src/file/or/dir`\n\n\n## Custom Rules\nRules specify the actual things that should be pulled out from key-value pairs. There are several common ones that come built-in, such as AWS keys and passwords, but the tool is made to be easily expandable with new rules.\n\n- Custom rules can be defined in the main config file under `rules:`\n- Custom rules can be added to [whispers/rules](whispers/rules/)\n\n```yaml\nrule-id:  # unique rule name\n  description: Values formatted like AWS Session Token\n  message: AWS Session Token  # report will show this message\n  severity: BLOCKER           # one of BLOCKER, CRITICAL, MAJOR, MINOR, INFO\n\n  key:        # specify key format\n    regex: (aws.?session.?token)?\n    ignorecase: True   # case-insensitive matching\n\n  value:      # specify value format\n    regex: ^(?=.*[a-z])(?=.*[A-Z])[A-Za-z0-9\\+\\/]{270,450}$\n    ignorecase: False  # case-sensitive matching\n    minlen: 270        # value is at least this long\n    isBase64: True     # value is base64-encoded\n    isAscii: False     # value is binary data when decoded\n    isUri: False       # value is not formatted like a URI\n\n  similar: 0.35        # maximum allowed similarity between key and value \n                       # (1.0 being exactly the same)\n```\n\n\n## Plugins\nAll parsing functionality is implemented via plugins. Each plugin implements a class with the `pairs()` method that runs through files and returns the key-value pairs to be checked with rules. \n\n```py\nclass PluginName:\n    def pairs(self, file):\n        yield \"key\", \"value\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSkyscanner%2Fwhispers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSkyscanner%2Fwhispers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSkyscanner%2Fwhispers/lists"}