Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/brittonhayes/pillager
Pillage filesystems for sensitive information with Go 🔍
https://github.com/brittonhayes/pillager
cli gitleaks go golang hacking scanner security security-tools tool tui
Last synced: 9 days ago
JSON representation
Pillage filesystems for sensitive information with Go 🔍
- Host: GitHub
- URL: https://github.com/brittonhayes/pillager
- Owner: brittonhayes
- License: mit
- Created: 2020-12-06T01:40:41.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-10-30T16:01:11.000Z (9 days ago)
- Last Synced: 2024-10-30T16:38:34.458Z (9 days ago)
- Topics: cli, gitleaks, go, golang, hacking, scanner, security, security-tools, tool, tui
- Language: Go
- Homepage:
- Size: 26.7 MB
- Stars: 240
- Watchers: 7
- Forks: 21
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Pillager
[](https://pkg.go.dev/github.com/brittonhayes/pillager)
[](https://goreportcard.com/report/github.com/brittonhayes/pillager)
Pillage filesystems for sensitive information with Go.
## Table of Contents
1. [Summary](#summary)
1. [Installation](#installation)
1. [Usage](#usage)
1. [Documentation](#documentation)
## Summary
Pillager is designed to provide a simple means of leveraging Go's strong concurrency model to recursively search
directories for sensitive information in files. Pillager does this by standing on the shoulders
of [a few giants](#shoulders-of-giants). Once pillager finds files that match the specified pattern, the file is scanned
using a series of concurrent workers that each take a line of the file from the job queue and hunt for sensitive pattern
matches. The available pattern filters can be defined in a rules.toml file or you can use the default ruleset.
## Installation
### Go
If you have Go setup on your system, you can install Pillager with `go get`
```shell script
go get github.com/brittonhayes/pillager
```
### Scoop (Windows)
```shell
scoop bucket add pillager https://github.com/brittonhayes/pillager-scoop.git
scoop install pillager
```
### Homebrew (OSX/Linux)
```shell script
brew tap brittonhayes/homebrew-pillager
brew install pillager
```
### Docker Image
```
docker run --rm -it ghcr.io/brittonhayes/pillager:latest hunt .
```
If you're looking for a binary, check the latest releases for the executable that matches your system
## Usage
To see all the commands available with `pillager`
```shell
# To see instructions for the entire application
pillager
# From any subcommand
pillager [cmd] --help
```
## User Interface
Pillager provides a terminal user interface built with [bubbletea](https://github.com/charmbracelet/bubbletea) if you'd like to scan for secrets interactively.
[](https://asciinema.org/a/WISZMVvKsfbFkLLQIWBRotknU)
## Configuration
### Gitleaks Rules
Pillager provides full support for Gitleaks[^2] rules. This can either be passed
in with a rules.toml[^1] file, or you can use the default ruleset by leaving the rules flag blank.
[^1]: [Gitleaks Rules Reference](https://github.com/zricethezav/gitleaks/blob/57f9bc83d169bea363f2990a4de334b54efc3d7d/config/gitleaks.toml)
```toml
# rules.toml
title = "pillager rules"
[[rules]]
id = "gitlab-pat"
description = "GitLab Personal Access Token"
regex = '''glpat-[0-9a-zA-Z\-\_]{20}'''
[[rules]]
id = "aws-access-token"
description = "AWS"
regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
# Cryptographic keys
[[rules]]
id = "PKCS8-PK"
description = "PKCS8 private key"
regex = '''-----BEGIN PRIVATE KEY-----'''
```
### Built-in Output Formats
Pillager has a series of built-in output formats available. Pick your flavor!
#### Basic
```shell
pillager hunt .
```
#### JSON
```shell
pillager hunt ./example -f json | jq
```
> _JSON output is designed to work seamlessly with the amazing [jq](https://github.com/stedolan/jq) utility for easy parsing._
Click to view more output formats
#### YAML
```shell
pillager hunt . -f yaml
```
#### TOML
```shell
pillager hunt . -f toml
```
#### HTML
```shell
pillager hunt . -f html > results.html
```
#### HTML Table
```shell
pillager hunt . -f html-table > results.html
```
#### Markdown
```shell
pillager hunt . -f markdown > results.md
```
#### Markdown Table
```shell
pillager hunt . -f table > results.md
```
#### Custom Go Template
```shell
pillager hunt . --template "{{ range .}}Secret: {{.Secret}}{{end}}"
```
#### Custom Go Template from File
```shell
pillager hunt . -t "$(cat pkg/templates/simple.tmpl)"
```
### Custom Templates
Pillager allows you to use powerful `go text/template` to customize the output format. Here are a few template examples.
#### Basic
```gotemplate
{{ range . -}}
File: {{ .File }}
Secret: {{ .Secret}}
Description: {{ quote .Description }}
{{ end -}}
```
#### Markdown Styling
```gotemplate
# Results
{{ range . -}}
## {{ .File }}
- Location: {{.StartLine}}
{{end}}
```
> More template examples can be found in the [templates](./pkg/templates) directory.
## Documentation
:books: [View the docs](pkg/hunter)
GoDoc documentation is available on [pkg.go.dev for pillager](https://pkg.go.dev/github.com/brittonhayes/pillager) but
it is also available for all packages in the repository in markdown format. Just open the folder of any package, and
you'll see the GoDocs rendered in beautiful Github-flavored markdown thanks to the
awesome [gomarkdoc](https://github.com/princjef/gomarkdoc) tool.
## Development
To get involved developing features and fixes for Pillager, get started with the following:
- [Install Go](https://go.dev/doc/install)
- Install [Taskfile.dev](https://taskfile.dev/#/installation)
- Read the [CONTRIBUTING.MD](./CONTRIBUTING.md)
---
### Shoulders of Giants :star:
#### [spf13's Cobra](https://github.com/spf13/cobra)
**What is Cobra?**
> Cobra is a library providing a simple interface to create powerful modern CLI interfaces similar to git & go tools.
> Cobra is also an application that will generate your application scaffolding to rapidly develop a Cobra-based application.
If you've seen a CLI written in Go before, there's a pretty high chance it was built with Cobra. I can't recommend this
library enough. It empowers developers to make consistent, dynamic, and self-documenting command line tools with ease.
Some examples include `kubectl`, `hugo`, and Github's `gh` CLI.
#### [Gitleaks](https://github.com/zricethezav/gitleaks)
**What is Gitleaks?**
> Gitleaks[^2] is a SAST tool for detecting hardcoded secrets like passwords, api keys, and tokens in git repos.
Gitleaks is an amazing tool for secret leak prevention. If you haven't implemented Gitleaks as a pre-commit checker,
it's worth your time to check it out.
**Why is Gitleaks relevant to Pillager?**
[^2]: [Gitleaks](https://github.com/zricethezav/gitleaks)
Pillager implements the powerful [rules](https://github.com/zricethezav/gitleaks#rules-summary) functionality of
Gitleaks while taking a different approach to presenting and handling the secrets found. While I have provided a
baseline set of default rules, Pillager becomes much more powerful if you allow users to create rules for their own
use-cases.
Check out the included rules[^1] for a baseline ruleset.
---
> This goes without saying but I'm going to say it anyways: I am **not** responsible for any repercussions caused by your use of pillager.
> This tool is intended for defensive use, educational use, and security researcher use with the consent of all involved parties.
> Malicious behavior with pillager is in no way condoned, nor encouraged. Please use this tool responsibly and ensure you have permission
> to scan for secrets on any systems before doing so.
>
> At it's core, Pillager is designed to assist you in determining if a system is affected by common sources of credential leakage as documented
> by the MITRE ATT&CK[^3] framework.
>
> [^3]: [MITRE ATT&CK Website](https://attack.mitre.org)
>
> [MITRE ATT&CK Technique - T1552,003 - Unsecured Credentials: Bash History ](https://attack.mitre.org/techniques/T1552/003/)
>
> [MITRE ATT&CK Technique - T1552,001 - Unsecured Credentials: Credentials In Files](https://attack.mitre.org/techniques/T1552/001/)