{"id":17531134,"url":"https://github.com/milosgajdos/vaultops","last_synced_at":"2025-04-23T19:25:32.066Z","repository":{"id":142256875,"uuid":"106702125","full_name":"milosgajdos/vaultops","owner":"milosgajdos","description":"Setup your vault servers and store the keys in an encrypted store of your choice","archived":false,"fork":false,"pushed_at":"2021-01-02T11:53:50.000Z","size":144,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-23T19:25:20.420Z","etag":null,"topics":["go","golang","kubernetes","kubernetes-secrets","vault","vault-cli"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/milosgajdos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-12T14:16:13.000Z","updated_at":"2022-01-10T23:47:03.000Z","dependencies_parsed_at":"2023-05-02T05:01:40.738Z","dependency_job_id":null,"html_url":"https://github.com/milosgajdos/vaultops","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milosgajdos%2Fvaultops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milosgajdos%2Fvaultops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milosgajdos%2Fvaultops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/milosgajdos%2Fvaultops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/milosgajdos","download_url":"https://codeload.github.com/milosgajdos/vaultops/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250498386,"owners_count":21440456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","kubernetes","kubernetes-secrets","vault","vault-cli"],"created_at":"2024-10-20T17:22:58.003Z","updated_at":"2025-04-23T19:25:32.046Z","avatar_url":"https://github.com/milosgajdos.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vaultops: automate vault setup\n\n[![Build Status](https://github.com/milosgajdos/vaultops/workflows/CI/badge.svg)](https://github.com/milosgajdos/vaultops/actions?query=workflow%3ACI)\n[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white\u0026style=flat-square)](https://pkg.go.dev/github.com/milosgajdos/vaultops)\n[![Go Report Card](https://goreportcard.com/badge/milosgajdos/vaultops)](https://goreportcard.com/report/github.com/milosgajdos/vaultops)\n[![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n`vaultops` is a command line utility which aims to simplify [vault](https://www.vaultproject.io/) server setup. At the moment it supports automatic initialization and unsealing.\n\n# Motivation\n\nTypical `vault` setup usually requires taking several steps before the server can be used:\n\n- initializing `vault` server\n- unsealing `vault` server(s)\n- mounting `vault` backends\n- creating `vault` backend roles\n- configuring `vault` policies\n\nThe above listed tasks are usually performed using the `vault` command line client which interacts with `vault` servers. Using the `vault` command line client requires writing a lot of `shell` scripts which often grow into unmanageable full fledged monsters which are often hard to debug and maintain.\n\n`vaultops` attempts to address this problem by providing a simple manifest file which can be used to specify all the tasks required to perform `vault` setup once the `vault` server/cluster is running. `vaultops` reads the manifest file and performs all the actions requsted by user.\n\n# Quick start\n\nGet the project:\n\n```console\n$ go get -u github.com/milosgajdos/vaultops\n```\n\nRun the tests\n```console\n$ cd $GOPATH/src/github.com/milosgajdos/vaultops\n$ make test\nfor pkg in github.com/milosgajdos/vaultops github.com/milosgajdos/vaultops/cipher github.com/milosgajdos/vaultops/cloud/aws github.com/milosgajdos/vaultops/cloud/gcp github.com/milosgajdos/vaultops/command github.com/milosgajdos/vaultops/manifest github.com/milosgajdos/vaultops/store github.com/milosgajdos/vaultops/store/local; do \\\n\t\tgo test -coverprofile=\"../../../$pkg/coverage.txt\" -covermode=atomic $pkg || exit; \\\n\tdone\n?   \tgithub.com/milosgajdos/vaultops\t[no test files]\n?   \tgithub.com/milosgajdos/vaultops/cipher\t[no test files]\nok  \tgithub.com/milosgajdos/vaultops/cloud/aws\t0.022s\tcoverage: 93.5% of statements\nok  \tgithub.com/milosgajdos/vaultops/cloud/gcp\t0.024s\tcoverage: 28.1% of statements\nok  \tgithub.com/milosgajdos/vaultops/command\t1.478s\tcoverage: 15.4% of statements\nok  \tgithub.com/milosgajdos/vaultops/manifest\t0.022s\tcoverage: 100.0% of statements\nok  \tgithub.com/milosgajdos/vaultops/store\t0.020s\tcoverage: 100.0% of statements\nok  \tgithub.com/milosgajdos/vaultops/store/local\t0.020s\tcoverage: 88.9% of statements\n```\n\nBuild the binary:\n```console\n$ make build\nmkdir -p ./_build\ngo build -ldflags=\"-s -w\" -o \"./_build/vaultops\"\n```\n\nOnce you have pulled in all the project dependencies you can also build the binary running the familiar command:\n```console\n$ go build -ldflags=\"-s -w\"\n```\n\n## Usage\n\n`vaultops` provides various commands following the same UX as `vault` command line utility. You can see the currently available commands below:\n\n```\nUsage: vaultops [--version] [--help] \u003ccommand\u003e [\u003cargs\u003e]\n\nAvailable commands are:\n    init      Initialize Vault cluster or server\n    unseal    Unseal a Vault server\n```\n\n`vaultops` reads **the same environment variables** as `vault` utility, so you can rely on the familiar `$VAULT_` environment variables when specifying the `vault` server URLs and tokens.\n\nAt the moment only `init` and `unseal` commands are implemented. The plan is to add a few more.\n\n## vaultops init\n\n`vaultops init` initializes the vault server. Besides providing the familiar `vault` command line utility options to connect to the `vault` server, it adds a few flags which allow to encrypt the `vault` master keys and root token and store them either on the workstation filesystem or encrypted in a remote storage:\n\n```console\n./vaultops init -help\nUsage: vaultops init [options]\n\n    Initialize a new Vault server or cluster.\n\n    This command connects to a Vault server and initializes it for the first time.\n    It sets up initial set of master keys and secret store.\n    Unless overridden init stores vault root token and keys on the local filesystem.\n\n    When init is called on already initialized server it will return error.\n...\n...\n...\n\n  -redact=true \t\t  Redacts sensitive information when printing into stdout\n  -kms-provider \t  KMS provider (aws, gcp)\n  -aws-kms-id\t\t  AWS KMS ID. KMS keys with given ID will be used to encrypt vault keys\n  -gcp-kms-crypto-key\t  GCP KMS crypto key ID\n  -gcp-kms-key-ring       GCP KMS keyring\n  -gcp-kms-region     \t  GCP region (eg. 'global', 'europe-west1')\n  -gcp-kms-project  \t  GCP project name\n  -storage-bucket         Cloud storage bucket\n  -storage-key            Cloud storage key\n  -key-store=local\t  Type of store where to loook up vault keys (default: local)\n    \t\t\t  Local store is ./.local/vault.json\n  -key-local-path         Path to locally stored keys\n\ninit Options:\n\n  -status \t\t\tDon't initialize the server, only check the init status\n  -key-shares=5 \t\tNumber of key shares to split the master key into\n  -key-threshold=3\t\tNumber of key shares required to reconstruct the master key\n  -config\t\t\tPath to a config file which contains a list of vault servers\n```\n\nWhen run with the default options, `init` command will store the `vault` keys **UNENCRYPTED** on your local filesystem in `.local` directory of your **current working directory** in a predefied `json` format which looks as follows:\n\n```json\n{\n  \"root_token\": \"your-root-token\",\n  \"master_keys\": [\n    \"master-key-1\",\n    \"master-key-2\",\n    \"master-key-3\",\n    \"master-key-4\",\n    \"master-key-5\"\n  ]\n}\n```\n\n**This is not what you should do when initialising your vault servers! This option is provided as convenience for local development!**\n\nFor real life setup, `vaultops` allows to encrypt the `vault` keys using the encryption keys provided by publc cloud providers such as [AWS KMS](https://aws.amazon.com/kms/) or [CGP Cloud KMS](https://cloud.google.com/security-key-management). These options are available via `-kms-provider` flag. Here is an example how to use the GCP Cloud KMS to encrypt the vault keys when initialising a new vault server:\n\n```console\n$ # export VAULT_ADDR environment variable\n$ export VAULT_ADDR=\"http://10.100.21.161:8200\"\n$ # initialize vault server and encrypt the vault keys\n$ ./vaultops init -kms-provider=\"gcp\" \\\n \t\t  -gcp-kms-project=\"kube-blog\" \\\n\t\t  -gcp-kms-region=\"europe-west1\" \\\n\t\t  -gcp-kms-key-ring=\"vaultops\" \\\n\t\t  -gcp-kms-crypto-key=\"vaultops\"\n\n[INFO] Attempting to initialize vault:\n[INFO] \thttp://10.100.21.161:8200\n[INFO] Host: http://10.100.21.161:8200 initialized. Master keys:\n[INFO] Key 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Initial Root Token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n```\n\nRunning the above `init` command will store the `vault` master keys and initial root token locally in `./.local/vault.json` file but now they're encrypted using the GCP Cloud KMS keys so if you try to inspect the local file you'll get a \"garbage\" pile of randomly generated bytes. If you want to see the actual unencrypted keys you need to decrypt the file using the same KMS keys you used when encrypting them.\n\n**At the moment only AWS KMS and GCP KMS are supported**\n\n#### Vault Keys redacting\n\nBy default `vaultops` tries to \"redact\" all sensitive information printed to `stdout`. You can disable this behavior via `-redact` command line switch by setting it to `false`. This will print the master keys and initial root token into `stdout` in plaintext.\n\n### Vault Key storage\n\n`vaultops` allows you to store `vault` keys remotely either in [AWS S3](https://aws.amazon.com/s3/), [Google Cloud Storage](https://cloud.google.com/storage/) or [kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/). You can choose the appropriate remote storage option via `-key-store` flag. Here is an example how to initialize `vault` using AWS KMS and store the keys in AWS S3 bucket of your choice:\n\n```console\n$ export VAULT_ADDR=\"http://${HostIP}:8200\"\n$ ./vaultops init -key-store=\"s3\" \\\n\t\t  -storage-bucket=\"vaultops-kms\" \\\n\t\t  -storage-key=\"vault.json\" \\\n\t\t  -kms-provider=\"aws\" \\\n\t\t  -aws-kms-id=\"your-kms-id\"\n```\n\n**NOTE:** when using kubernetes secrets storage, you can also specify a namespace for the secret; the default value is set to `default` namespace\n\n## vaultops unseal\n\n`vaultops unseal` unseals the vault cluster using the keys generated by `vault` during its initalisation. These keys can be stored encrypted or in plaintext either locally or remotely based on the command line switches you used when you initialized the server. `unseal` command allows you to read these keys from whatever location you stored them in during initialization and use them to unseal the `vault` server. See the available command line options listed below:\n\n```console\n$ ./vaultops unseal -help\nUsage: vaultops unseal [options]\n\n    Unseal the vault serve by entering master keys.\n\n    This command connects to a Vault server and attempts to unseal it.\n    first time. It sets up initial set of master keys and backend store.\n\n    When init is called on already initialized server it will error\n\n...\n...\n...\n\n  -redact=true \t\t  Redacts sensitive information when printing into stdout\n  -kms-provider \t  KMS provider (aws, gcp)\n  -aws-kms-id\t\t  AWS KMS ID. KMS keys with given ID will be used to encrypt vault keys\n  -gcp-kms-crypto-key\t  GCP KMS crypto key id\n  -gcp-kms-key-ring       GCP KMS key ring\n  -gcp-kms-region     \t  GCP region (eg. 'global', 'europe-west1')\n  -gcp-kms-project  \t  GCP project name\n  -storage-bucket         Cloud storage bucket\n  -storage-key            Cloud storage key\n  -key-store=local\t  Type of store where to loook up vault keys (default: local)\n    \t\t\t  Local store is ./.local/vault.json\n  -key-local-path         Path to locally stored keys\n\nunseal Options:\n\n    -status \t\t  Don't unseal the server, only check the seal status\n    -config\t\t  Path to a config file which contains a list of vault servers\n```\n\nHere is an example of how to unseal the vault server using the keys stored in AWS S3 which were encrypted using AWS KMS:\n\n```console\n$ export AWS_REGION=\"us-east-1\"\n$ export VAULT_ADDR=\"http://${HostIP}:8200\"\n$ # unseal the vault server\n$ ./vaultops unseal -key-store=\"s3\" \\\n\t\t    -storage-bucket=\"vaultops-kms\" \\\n\t\t    -storage-key=\"vault.json\" \\\n\t\t    -kms-provider=\"aws\" \\\n\t\t    -aws-kms-id=\"your-kms-id\"\n```\n\nObviously, you can create all kinds of crazy combination of storages and encryption keys i.e. store the keys in AWS S3, but encrypt them using GCP Cloud KMS\n\n# Manifest\n\n`vaultops` allows you to create a manifest file which can be used when running `vaultops` commands. The manifest is a simple `YAML` (woo, hoo! more `YAML` ᕕ( ᐛ )ᕗ) file which specifies a list of `vault` hosts for initialization and unsealing.\n\nLet's look at a simple example:\n\n```yaml\nhosts:\n  # URL of vault server to use for initialization\n  init:\n    - \"http://10.100.21.161:8200\"\n  # URLs of all vault servers that should be unsealed\n  unseal:\n    - \"http://10.100.21.161:8200\"\n    - \"http://10.100.21.162:8200\"\n    - \"http://10.100.21.163:8200\"\n```\n\nHopefully the `YAML` snippet above is straightforward to understand: we've got a server to initialize and a cluster of servers to unseal.\n\nYou don't have to have a full manifest at hand if you want to run just one command. Say, you just want to initialize the server -- you can supply the following `YAML` snippet to `init` command via `-config` command line switch:\n\n```yaml\nhosts:\n  # URL of vault server to use for initialization\n  init:\n    - \"http://10.100.21.161:8200\"\n```\n\nRun the `init` command using the manifest above stored in some file on the local filesystem:\n\n```console\n$ VAULT_ADDR=\"http://10.100.21.161:8200\" ./vaultops init -config init.yaml\n[INFO] Attempting to initialize vault:\n[INFO] \thttp://10.100.21.161:8200\n[INFO] Host: http://10.100.21.161:8200 initialized. Master keys:\n[INFO] Key 1: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 3: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 4: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Key 5: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n[INFO] Initial Root Token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n```\n\nSame rules apply when running the `unseal` command.\n\n# TODO\n\n* bigger test coverage\n* plenty of room for refactoring\n* setting up `vault` secret backends\n* setting up `vault` secret backend roles\n* setting up `vault` policies\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilosgajdos%2Fvaultops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmilosgajdos%2Fvaultops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmilosgajdos%2Fvaultops/lists"}