{"id":21644460,"url":"https://github.com/samsung/credsweeper","last_synced_at":"2025-04-12T01:10:20.829Z","repository":{"id":36956250,"uuid":"419917201","full_name":"Samsung/CredSweeper","owner":"Samsung","description":"CredSweeper is a tool to detect credentials in any directories or files. CredSweeper could help users to detect unwanted exposure of credentials (such as token, passwords, api keys etc.) in advance. By scanning lines, filtering, and using AI model as option, CredSweeper reports lines with possible credentials, where the line is, and expected type o","archived":false,"fork":false,"pushed_at":"2025-04-10T07:29:47.000Z","size":56464,"stargazers_count":118,"open_issues_count":8,"forks_count":37,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-12T01:09:59.104Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Samsung.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-22T00:27:55.000Z","updated_at":"2025-04-10T07:29:50.000Z","dependencies_parsed_at":"2023-02-16T19:01:15.938Z","dependency_job_id":"7df58d41-50e2-4c71-bd93-65c01d1bea36","html_url":"https://github.com/Samsung/CredSweeper","commit_stats":{"total_commits":571,"total_committers":20,"mean_commits":28.55,"dds":0.6042031523642732,"last_synced_commit":"bcaf6019f67c55c226bf27ba1c4f9dcad28f758d"},"previous_names":[],"tags_count":62,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samsung%2FCredSweeper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samsung%2FCredSweeper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samsung%2FCredSweeper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samsung%2FCredSweeper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Samsung","download_url":"https://codeload.github.com/Samsung/CredSweeper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248501861,"owners_count":21114684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T05:40:19.239Z","updated_at":"2025-04-12T01:10:20.805Z","avatar_url":"https://github.com/Samsung.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CredSweeper\n\n[![GitHub release (latestSemVer)](https://img.shields.io/github/v/release/Samsung/CredSweeper)](https://github.com/Samsung/CredSweeper/releases)\n[![Documentation Status](https://readthedocs.org/projects/credsweeper/badge/?version=latest)](https://credsweeper.readthedocs.io/en/latest/?badge=latest)\n[![License](https://img.shields.io/badge/licence-MIT-green.svg?style=flat)](LICENSE)\n[![PyPI](https://img.shields.io/pypi/v/credsweeper)](https://pypi.org/project/credsweeper/)\n[![Python](https://img.shields.io/pypi/pyversions/credsweeper.svg)](https://badge.fury.io/py/credsweeper)\n[![Test](https://github.com/Samsung/CredSweeper/actions/workflows/test.yml/badge.svg)](https://github.com/Samsung/CredSweeper/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/Samsung/CredSweeper/branch/main/graph/badge.svg)](https://codecov.io/gh/Samsung/CredSweeper)\n[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/6055/badge)](https://bestpractices.coreinfrastructure.org/projects/6055)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/Samsung/CredSweeper/badge)](https://api.securityscorecards.dev/projects/github.com/Samsung/CredSweeper)\n\n\u003cimg src=\"https://raw.githubusercontent.com/Samsung/CredSweeper/main/docs/images/Logo.png\" width=\"500\"/\u003e\n\n- [CredSweeper](#credsweeper)\n  - [Introduction](#introduction)\n  - [How To Use](#how-to-use)\n    - [Main Requirements](#main-requirements)\n    - [Installation](#installation)\n    - [Run](#run)\n    - [Config](#config)\n  - [Develop](#develop)\n    - [Tests](#tests)\n    - [Benchmark](#benchmark)\n  - [Overall Architecture](#overall-architecture)\n  - [Retrain Model](#retrain-model)\n  - [License](#license)\n  - [How to Get Involved](#how-to-get-involved)\n    - [Project Roles](#project-roles)\n      - [Contributor](#contributor)\n      - [Maintainer](#maintainer)\n  - [How to Contact](#how-to-contact)\n\n## Introduction\n\nCredSweeper is a tool to detect credentials in any directories or files.\nCredSweeper could help users to detect unwanted exposure of credentials\n(such as tokens, passwords, api keys etc.) in advance.\nBy scanning lines, filtering, and using AI model as option,\nCredSweeper reports lines with possible credentials, where the line is,\nand expected type of the credential as a result.\n\nFull documentation can be found here: \u003chttps://credsweeper.readthedocs.io/\u003e\n\n## How To Use\n\n### Main Requirements\n\n- Python 3.9, 3.10, 3.11, 3.12\n\n### Installation\n\nDetails [here](https://credsweeper.readthedocs.io/en/latest/install.html).\n\n```bash\npip install credsweeper\n```\n\n### Run\n\n[How to use](https://credsweeper.readthedocs.io/en/latest/guide.html).\n\nGet all argument list:\n\n``` bash\npython -m credsweeper --help\n```\n\nRun CredSweeper:\n\n``` bash\npython -m credsweeper --path tests/samples/password.gradle --save-json output.json\n```\n\nTo check JSON file run:\n\n```bash\ncat output.json\n```\n\n``` json\n[\n    {\n        \"api_validation\": \"NOT_AVAILABLE\",\n        \"ml_validation\": \"VALIDATED_KEY\",\n        \"ml_probability\": 0.99755,\n        \"rule\": \"Password\",\n        \"severity\": \"medium\",\n        \"confidence\": \"moderate\",\n        \"line_data_list\": [\n            {\n                \"line\": \"password = \\\"cackle!\\\"\",\n                \"line_num\": 1,\n                \"path\": \"tests/samples/password.gradle\",\n                \"info\": \"\",\n                \"value\": \"cackle!\",\n                \"value_start\": 12,\n                \"value_end\": 19,\n                \"variable\": \"password\",\n                \"entropy_validation\": {\n                    \"iterator\": \"BASE64_CHARS\",\n                    \"entropy\": 2.120589933192232,\n                    \"valid\": false\n                }\n            }\n        ]\n    }\n]\n```\n\n### Config\n\n[credsweeper/secret/config.json](credsweeper/secret/config.json) - Configuration file for pre-processing of CredSweeper. For more details please check [here](https://credsweeper.readthedocs.io/en/latest/overall_architecture.html#pre-processing).\n\nYou can set the `pattern`, `extension` and `path` you want to exclude from scanning as below.\n\n``` json\n{\n    \"exclude\": {\n        \"pattern\": [\n            \"AKIA[0-9A-Z]{9}EXAMPLE\",\n            ...\n        ],\n        \"extension\": [\n            \"gif\",\n            \"jpg\",\n            ...\n        ],\n        \"path\": [\n            \"/.git/\",\n            \"/openssl/\",\n            ...\n        ]\n    },\n    ...\n}\n```\n\nAnd you can also set `source_ext`, `source_quote_ext`, `find_by_ext_list`, `check_for_literals`, `line_data_output`, and `candidate_output` as below.\n\n- `source_ext`: List of extensions for scanning categorized as source files.\n- `source_quote_ext`: List of extensions for scanning categorized as source files that using quote.\n- `find_by_ext_list`: List of extensions to detect only extensions.\n- `check_for_literals`: Bool value for whether to check line has string literal declaration or not.\n- `line_data_output`: List of attributes of [line_data](credsweeper/credentials/line_data.py) for output.\n- `candidate_output`: List of attributes of [candidate](credsweeper/credentials/candidate.py) for output.\n\n``` json\n{\n    ...\n    \"source_ext\": [\n        \".py\",\n        \".cpp\",\n        ...\n    ],\n    \"source_quote_ext\": [\n        \".py\",\n        \".cpp\",\n        ...\n    ],\n    \"find_by_ext_list\": [\n        \".pem\",\n        \".cer\",\n        ...\n    ],\n    \"check_for_literals\": true,\n    \"line_data_output\": [\n        \"line\",\n        \"line_num\",\n        ...\n    ],\n    \"candidate_output\": [\n        \"rule\",\n        \"severity\",\n        ...\n    ]\n}\n```\n\n[credsweeper/rules/config.yaml](credsweeper/secret/config.yaml) - Configuration file for setting Rule. For more details please check [here](https://credsweeper.readthedocs.io/en/latest/overall_architecture.html#rule).\n\n``` yaml\n...\n- name: API\nseverity: medium\nconfidence: moderate\ntype: keyword\nvalues:\n- api\nfilter_type: GeneralKeyword\nuse_ml: true\nvalidations: []\n- name: AWS Client ID\n...\n```\n\n## Develop\n\n### Tests\n\nTo run all tests:\n\n``` bash\npython -m pytest --cov=credsweeper --cov-report=term-missing -s tests/\n```\n\nTo run only tests independent of external api:\n\n``` bash\npython -m pytest -m \"not api_validation_test\" tests/\n```\n\nTo obtain manageable (without subprocesses) coverage:\n\n``` bash\npython -m pytest --cov=credsweeper --cov-report=html tests/ --ignore=tests/test_app.py\n```\n\n### Benchmark\n\nWe have a dataset for testing credential scanners that called [CredData](https://github.com/Samsung/CredData). If you want to test CredSweeper with this dataset please check [here](https://github.com/Samsung/CredData/blob/main/README.md#benchmark).\n\n## Overall Architecture\n\nTo check overall architecture of CredSweeper please check [here](https://credsweeper.readthedocs.io/en/latest/overall_architecture.html).\n\n## Retrain Model\n\nIf you want to check how model was trained or retrain it on your own data, please refer to the [experiment](experiment/README.md) folder\n\n## License\n\nThe CredSweeper is an Open Source project released under the terms of [MIT License V2](https://opensource.org/licenses/mit-license.php).\n\n## How to Get Involved\n\nIn addition to developing under an Open Source license, A use an Open Source Development approach, welcoming everyone to participate, contribute, and engage with each other through the project.\n\n### Project Roles\n\nA recognizes the following formal roles: Contributor and Maintainer. Informally, the community may organize itself and give rights and responsibilities to the necessary people to achieve its goals.\n\n#### Contributor\n\nA Contributor is anyone who wishes to contribute to the project, at any level. Contributors are granted the following rights, to:\n\n- Contribute code, documentation, translations, artwork, and etc.\n- Report defects (bugs) and suggestions for enhancement.\n- Participate in the process of reviewing contributions by others.\n\nIf you want to participate in the project development, check out the [how to contribute guideline](./docs/howto/how-to-contribute.md) in advance.\n\nContributors who show dedication and skill are rewarded with additional rights and responsibilities. Their opinions weigh more when decisions are made, in a fully meritocratic fashion.\n\n#### Maintainer\n\nA Maintainer is a Contributor who is also responsible for knowing, directing and anticipating the needs of a given a Module. As such, Maintainers have the right to set the overall organization of the source code in the Module, and the right to participate in the decision-making. Maintainers are required to review the contributor’s requests and decide whether to accept or not.\n\nName | E-Mail\n-- | --\n[Jaeku Yun](https://github.com/silentearth) | jk0113.yun@samsung.com\n[Shinhyung Choi](https://github.com/csh519) | sh519.choi@samsung.com\n[Roman Babenko](https://github.com/babenek) | r.babenko@samsung.com\n[Yuliia Tatarinova](https://github.com/Yullia) | yuliia.t@samsung.com\n\n## How to Contact\n\nPlease post questions, [issues, or suggestions in issues](https://github.com/Samsung/CredSweeper/issues). This is the best way to communicate with the developers.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamsung%2Fcredsweeper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamsung%2Fcredsweeper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamsung%2Fcredsweeper/lists"}