Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/compasssecurity/pipeleak
Pipeleak is a tool designed to scan GitLab job output logs and artifacts for secrets.
https://github.com/compasssecurity/pipeleak
gitlab secrets
Last synced: 16 days ago
JSON representation
Pipeleak is a tool designed to scan GitLab job output logs and artifacts for secrets.
- Host: GitHub
- URL: https://github.com/compasssecurity/pipeleak
- Owner: CompassSecurity
- License: mit
- Created: 2024-07-31T08:56:38.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2024-12-10T07:21:48.000Z (16 days ago)
- Last Synced: 2024-12-10T08:19:46.392Z (16 days ago)
- Topics: gitlab, secrets
- Language: Go
- Homepage:
- Size: 521 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: readme.md
- License: LICENSE
Awesome Lists containing this project
README
# Pipeleak
Pipeleak is a tool designed to scan GitLab job output logs and artifacts for potential secrets.
## Getting Started
To begin using Pipeleak, download the latest binary from the [Releases](https://github.com/CompassSecurity/pipeleak/releases) page.
### Basic Usage
Run the following command to scan your GitLab instance:
```bash
pipeleak scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com
```
### Scanning Artifacts
In addition to logs, Pipeleak can also scan artifacts generated by your pipelines.
Note: Scanning artifacts uses a lot of memory!
```bash
pipeleak scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com -a
```
**Pro tip:**
> Note: [Dotenv artifacts](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsdotenv) are not accessible through the GitLab API. To scan these, you need to manually provide your session cookie after logging in via a web browser. The session cookie name is `_gitlab_session`. The cookie should be valid for [two weeks](https://gitlab.com/gitlab-org/gitlab/-/issues/395038).
```bash
pipeleak scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com -v -a -c [value-of-valid-_gitlab_session]
```
### Confidence
The following confidence levels for scans are used by default:
* `low`
* `medium`
* `high`
* `high-verified` these are validated by the corresponding TruffleHog validator.
However if you want to add custom ones, you can define any value in the `rules.yml` file
To scan only for a specific level, you can specify it via CLI flag e.g `--confidence=high,medium`. Define multiple by comma separating the values.
Note: `high-verified` cannot be removed.
### Memory Usage
As the scanner processes a lot of resoures (especially when using `--artifacts`) memory, CPU and disk usage can become hard to manage.
You can tweak `--threads`, `--max-artifact-size` and `--job-limit` to obtain a customized performance and achieve stable processing.
### Additional Features
`shodan` command: use the Shodan command to identify gitlab instances that might allow for anyone to register. this command assumes a JSON file from a Shodan export. example query: `product:"GitLab Self-Managed"`
`runners` command: Identify (shared) runners which are available globally or through a group where the user has at least developer access. Create projects with a default exploitation `.gitlab-ci.yml` file.
`register` command: Best effort automation to register a new user on an instance.
`vuln` command: Check the installed version for CVEs in the NIST Vulnerability database
`variables` command: Enumerate configured project/group/instance variables
`secureFiles` command: Enumerate secure files
Setting an HTTP proxy is possible by setting the environment variable `HTTP_PROXY` e.g. to route through Burp:
```bash
HTTP_PROXY=http://127.0.0.1:8080 pipeleak scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com
```
### Keybindings
In the `scan` mode you can change interactively between log levels by pressing `t`: Trace, `d`: Debug, `i`: Info, `w`: Warn, `e`: Error. Pressing `s` will output the current queue status.
## ELK Integration
To easily analyze the results you can [redirect the pipeleak](https://github.com/deviantony/docker-elk?tab=readme-ov-file#injecting-data) output using `nc` into Logstash.
Setup a local ELK stack using https://github.com/deviantony/docker-elk.
Then you can start a scan:
```bash
pipeleak scan --token glpat-xxxxxxxxxxx --gitlab https://gitlab.com --json | nc -q0 localhost 50000
```
Using Kibana you can filter for interesting messages, based on the JSON attributes of the output.
e.g. `message.level=warn and message.confidence=verified` to find verified results.
## Customizing Scan Rules
When you run Pipeleak for the first time, it generates a `rules.yml` file based on [this repository](https://github.com/mazen160/secrets-patterns-db/blob/master/db/rules-stable.yml). You can customize your scan rules by modifying this file as needed.