{"id":13539977,"url":"https://github.com/bishopfox/gitgot","last_synced_at":"2025-04-08T08:14:39.265Z","repository":{"id":40589204,"uuid":"191989621","full_name":"BishopFox/GitGot","owner":"BishopFox","description":"Semi-automated, feedback-driven tool to rapidly search through troves of public data on GitHub for sensitive secrets.","archived":false,"fork":false,"pushed_at":"2024-03-07T13:48:48.000Z","size":194,"stargazers_count":1490,"open_issues_count":3,"forks_count":213,"subscribers_count":41,"default_branch":"master","last_synced_at":"2025-04-08T08:14:19.943Z","etag":null,"topics":["fuzzy-matching","gist-search","gists","github-api","osint","python","recon","reconnaissance","security","security-scanner","security-tools","sensitive-data-exposure"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BishopFox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-14T18:33:16.000Z","updated_at":"2025-04-08T03:49:57.000Z","dependencies_parsed_at":"2024-08-01T09:23:47.946Z","dependency_job_id":"e8aa9f09-9644-4909-87fd-67ff3c320b4b","html_url":"https://github.com/BishopFox/GitGot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BishopFox%2FGitGot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BishopFox%2FGitGot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BishopFox%2FGitGot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BishopFox%2FGitGot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BishopFox","download_url":"https://codeload.github.com/BishopFox/GitGot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247801169,"owners_count":20998339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fuzzy-matching","gist-search","gists","github-api","osint","python","recon","reconnaissance","security","security-scanner","security-tools","sensitive-data-exposure"],"created_at":"2024-08-01T09:01:36.088Z","updated_at":"2025-04-08T08:14:39.239Z","avatar_url":"https://github.com/BishopFox.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"logo.png\" width=350/\u003e\n\u003c/p\u003e\n\n#\n\n\n![License](https://img.shields.io/badge/license-LGPLv3%2B-lightgrey.svg)\n![Python version](https://img.shields.io/badge/python-3.x-blue.svg)\n\n## Description\n\nGitGot is a semi-automated, feedback-driven tool to empower users to rapidly search through troves of public data on GitHub for sensitive secrets.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"example_usage.png\" width=80%/\u003e\n\u003c/p\u003e\n\n### How it Works\n\nDuring search sessions, users will provide feedback to GitGot about search results to ignore, and GitGot prunes the set of results. Users can blacklist files by filename, repository name, username, or a fuzzy match of the file contents.\n\nBlacklists generated from previous sessions can be saved and reused against similar queries (e.g.,\n`example.com` v.s. `subdomain.example.com` v.s. `Example Org`). Sessions can also be paused and resumed at any time.\n\nRead more about the semi-automated, human-in-the-loop design here: https://bishopfox.com/blog/semi-automated-vs-automated-security-tools.\n\n## Install Instructions\n\n### Manual Instructions\n\n[1] Install the `ssdeep` dependency for fuzzy hashing.\n\nUbuntu/Debian (or equivalent for your distro):\n```sh\napt-get install python3-dev libfuzzy-dev ssdeep\n```\n\nor, for Mac OSX:\n```sh\nbrew install ssdeep\n```\nFor Windows or *nix distributions without the `ssdeep` package, please see the [ssdeep installation instructions](https://ssdeep-project.github.io/ssdeep/index.html).\n\n[2] After installing `ssdeep`, install the Python dependencies using `pip`:\n```\npip3 install -r requirements.txt\n```\n\n### Docker Instructions\n\nRun `gitgot-docker.sh` to build the GitGot docker image (if it doesn't already exist) and execute the dockerized version of the GitGot tool.\n\nOn invocation, `gitgot-docker.sh` will create and mount `logs` and `states` directories from the host's current working directory. If this `gitgot-docker.sh` is executed from the GitGot project directory it will update the docker container with changes to `gitgot.py` or `checks/`:\n\n```sh\n./gitgot-docker.sh -q example.com\n```\n\n(See `gitgot-docker.sh` for specific docker commands)\n## Usage\n\nGitHub requires a token for rate-limiting purposes. Create a [GitHub API token](https://github.com/settings/tokens) with **no permissions/no scope**. This will be equivalent to public GitHub access, but it will allow access to use the GitHub Search API. Set this token at the top of `gitgot.py` as shown below:\n```sh\nACCESS_TOKEN = \"\u003cNO-PERMISSION-GITHUB-TOKEN-HERE\u003e\"\n```\n\n(Alternatively, this token can be set as the `GITHUB_ACCESS_TOKEN` environment variable)\n\nAfter adding the token, you are ready to go:\n```sh\n# Default RegEx list and logfile location (/logs/\u003cquery\u003e.log) are used when no others are specified.\n\n# Query for the string \"example.com\" using default GitHub search behavior (i.e., tokenization).\n# This will find com.example (e.g., Java) or example.com (Website)\n./gitgot.py -q example.com\n\n# Query self-hosted GitHub instance\n./gitgot.py -q example.com -u https://git.example.com\n\n# Query for the exact string \"example.com\". See Query Syntax in the next section for more details.\n./gitgot.py -q '\"example.com\"'\n\n# Query through GitHub gists\n./gitgot.py --gist -q CompanyName\n\n# Using GitHub advanced search syntax\n./gitgot.py -q \"org:github cats\"\n\n# Custom RegEx List and custom log files location\n./gitgot.py -q example.com -f checks/default.list -o example1.log\n\n# Recovery from existing session\n./gitgot.py -q example.com -r example.com.state\n\n# Using an existing session (w/blacklists) for a new query\n./gitgot.py -q \"Example Org\" -r example.com.state\n```\n### Query Syntax\n\nGitGot queries are fed directly into the GitHub code search API, so check out [GitHub's documentation](https://help.github.com/en/articles/searching-code) for more advanced query syntax.\n\n### UI Commands\n* **Ignore similar [c]ontent:** Blacklists a fuzzy hash of the file contents to ignore\nfuture results that are similar to the selected file\n* **Ignore [r]epo/[u]ser/[f]ilename:** Ignores future results by blacklisting selected strings\n* **Search [/(mykeyword)]:** Provides a custom regex expression with a capture group to searches on-the-fly (e.g., `/(secretToken)`)\n* **[a]dd to Log:** Add RegEx matches to log file, including all on-the-fly search results from search command\n* **Next[\\\u003cEnter\\\u003e], [b]ack:** Advances through search results, or returns to previous results\n* **[s]ave state:** Saves the blacklists and progress in the search results from the session\n* **[q]uit:** Quit\n","funding_links":[],"categories":["\u003ca id=\"a76463feb91d09b3d024fae798b92be6\"\u003e\u003c/a\u003e侦察\u0026\u0026信息收集\u0026\u0026子域名发现与枚举\u0026\u0026OSINT","\u003ca id=\"170048b7d8668c50681c0ab1e92c679a\"\u003e\u003c/a\u003e工具"],"sub_categories":["\u003ca id=\"dc74ad2dd53aa8c8bf3a3097ad1f12b7\"\u003e\u003c/a\u003e社交网络"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbishopfox%2Fgitgot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbishopfox%2Fgitgot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbishopfox%2Fgitgot/lists"}