https://github.com/mitodl/vault-raft-backup

A lean tool for creating Vault Raft snapshots and transferring them to AWS S3
https://github.com/mitodl/vault-raft-backup

Last synced: 6 days ago
JSON representation

A lean tool for creating Vault Raft snapshots and transferring them to AWS S3

• Host: GitHub
• URL: https://github.com/mitodl/vault-raft-backup
• Owner: mitodl
• Created: 2021-12-01T20:50:51.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2024-04-12T19:38:58.000Z (9 months ago)
• Last Synced: 2024-04-13T21:46:54.494Z (9 months ago)
• Language: Go
• Size: 22.5 KB
• Stars: 8
• Watchers: 6
• Forks: 1
• Open Issues: 7
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## vault-raft-backup

Vault Raft backup is a lean tool for creating snapshots of the Raft integrated storage in Hashicorp Vault, and transferring those backups to AWS S3.

### Prerequsities

If executing as an ad-hoc compile and run (i.e. `go run`), then the dependencies and requirements can be viewed in the [go.mod](go.mod) file. Additional setup requirements are as follows:

- Connectivity to a functioning Vault server cluster with Raft integrated storage.

- Authentication and authorization against the Vault server cluster for executing Raft snapshots.

  - Authentication can be input in general as a token.

  - Authentication can also specified as AWS IAM. In this situation, the Vault server cluster must have a role mapped to an AWS IAM role. This AWS IAM role must also be automatically accessible by this tool (e.g. executed on EC2 instance with appropriate IAM Instance profile).

- A local filesystem with permissions and storage capable of staging the snapshot.

- Authentication and authorization against an AWS account for listing, reading, and writing objects to a S3 bucket.

- A S3 bucket capable of storing the snapshot.

The Vault policy for executing Raft snapshots appears like:

```hcl

path "sys/storage/raft/snapshot" {

  capabilities = ["read"]

}

```

### Usage

The following environment variables are read for configuration of the backup tool. This usage is due to the expectation that this tool will be executed as part of automation e.g. pipeline, service, orchestator, etc., and also because some inputs are sensitive.

```

# equivalent to VAULT_ADDR with vault cli executable

export VAULT_ADDR=

# equivalent to VAULT_TOKEN with vault cli executable; if using aws iam auth, then set this equal to "aws-iam" instead

export VAULT_TOKEN=

# equivalent to VAULT_SKIP_VERIFY with vault cli executable

export VAULT_SKIP_VERIFY=

export VAULT_SNAPSHOT_PATH=

export S3_BUCKET=

# this is prepended to the base filename in VAULT_SNAPSHOT_PATH

export S3_PREFIX=

export AWS_REGION=

```

Additionally, AWS authentication must be provided with environment variables or other standard methods. The AWS Golang SDK will automatically read authentication information as per normal (i.e. IAM instance profile, `AWS_PROFILE`, etc.).