{"id":13469045,"url":"https://github.com/bruj0/vault_jenkins","last_synced_at":"2025-08-03T09:15:20.292Z","repository":{"id":169550685,"uuid":"116604099","full_name":"bruj0/vault_jenkins","owner":"bruj0","description":"How to use Vault to store secrets and use them in Jenkins","archived":false,"fork":false,"pushed_at":"2018-04-22T15:35:06.000Z","size":229,"stargazers_count":103,"open_issues_count":1,"forks_count":30,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-24T19:22:17.866Z","etag":null,"topics":["docker","docker-compose","jenkins","jenkins-pipeline","secret-management","vault","vault-client","vault-consul","vault-ui"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bruj0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-01-07T22:28:32.000Z","updated_at":"2024-10-21T01:37:34.000Z","dependencies_parsed_at":"2024-01-18T20:05:26.956Z","dependency_job_id":null,"html_url":"https://github.com/bruj0/vault_jenkins","commit_stats":null,"previous_names":["bruj0/vault_jenkins"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bruj0%2Fvault_jenkins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bruj0%2Fvault_jenkins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bruj0%2Fvault_jenkins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bruj0%2Fvault_jenkins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bruj0","download_url":"https://codeload.github.com/bruj0/vault_jenkins/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248311668,"owners_count":21082630,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","jenkins","jenkins-pipeline","secret-management","vault","vault-client","vault-consul","vault-ui"],"created_at":"2024-07-31T15:01:25.103Z","updated_at":"2025-04-10T22:36:27.772Z","avatar_url":"https://github.com/bruj0.png","language":"Shell","funding_links":[],"categories":["Shell","jenkins"],"sub_categories":[],"readme":"# How to use HashiCorp Vault to store secrets and read them from Jenkins\r\n\r\nby Rodrigo A. Diaz Leven\r\n\r\n- [How to use HashiCorp Vault to store secrets and read them from Jenkins](#how-to-use-hashicorp-vault-to-store-secrets-and-read-them-from-jenkins)\r\n  * [Description](#description)\r\n  * [Why do we want to use it.](#why-do-we-want-to-use-it)\r\n  * [The setup](#the-setup)\r\n- [Initial Vault configuration](#initial-vault-configuration)\r\n  * [Checking Vault is unsealed](#checking-vault-is-unsealed)\r\n  * [Testing](#testing)\r\n- [Reading secrets from Jenkins](#reading-secrets-from-jenkins)\r\n  * [AppRole](#approle)\r\n  * [Generating Policies and Roles](#generating-policies-and-roles)\r\n  * [Generating Role ID](#generating-role-id)\r\n  * [Generating Jenkins Token ID](#generating-jenkins-token-id)\r\n  * [Configuring Jenkins credentials to access Vault](#configuring-jenkins-credentials-to-access-vault)\r\n  * [Reading the secret](#reading-the-secret)\r\n  * [Retrieving secrets from source code](#retrieving-secrets-from-source-code)\r\n  * [Job Output](#job-output)\r\n- [Optional: Installing Vault plugin for Jenkins](#optional--installing-vault-plugin-for-jenkins)\r\n- [References](#references)\r\n\r\n## Description\r\n\r\nIn this article we will learn how to store secret or any other type of information you wish like Certificates in [Vault](https://www.vaultproject.io/) .\r\n\r\nVault works by exchanging **information** for **secrets**, for example  a client could havea RoleID and a SecretID or a temporary Token that it can trade for the actual Secret.\r\n\r\nIt also can create temporary access to a 3rd party services like AWS through the use of back ends.\r\n\r\n## Why do we want to use it.\r\n\r\n* Centralized secrets storage and role based access.\r\n\r\n* Replace secrets stored in source code or shell scripts which then are stored in places, difficult to replace with one time or temporary authentication credentials.\r\n\r\n* Create dynamic authentication for short lived jobs, ie AWS infrastructure creation. This means that a short lived set of AWS secret/key pair is created for a job and they won't need to be hard coded.\r\n\r\n* Optional, Human authentication to services like SSH, Jenkins, VPN, etc.\r\n\r\n## The setup\r\n\r\nWe will test this by using a docker-compose project that will orchestrate 3 containers:\r\n\r\n* Consul: service discovery and backend for storing encrypted secrets\r\n* Vault: Service for converting tokens to secrets\r\n* Jenkins: Automation\r\n\r\nWe persist Jenkins configuration in a Docker volume called \"jenkins-data\" and bind the local directory config to the Consul container so we can copy configuration later.\r\n\r\n```yaml\r\nversion: '2'\r\nservices:\r\n consul:\r\n   image: \"consul\"\r\n   hostname: \"consul\"\r\n   command: \"agent -dev -client 0.0.0.0\"\r\n   ports:\r\n     - \"8400:8400\"\r\n     - \"8500:8500\"\r\n     - \"8600:53/udp\"\r\n vault:\r\n   depends_on:\r\n     - consul\r\n   image: \"vault\"\r\n   hostname: \"vault\"\r\n   links:\r\n     - \"consul:consul\"\r\n   environment:\r\n     VAULT_ADDR: http://127.0.0.1:8200\r\n   ports:\r\n     - \"8200:8200\"\r\n   volumes:\r\n     - ./tools/wait-for-it.sh:/wait-for-it.sh\r\n     - ./config/vault:/config\r\n     - ./config/vault/policies:/policies\r\n   entrypoint: /wait-for-it.sh -t 20 -h consul -p 8500 -s -- vault server -config=/config/with-consul.hcl\r\n blueocean:\r\n   depends_on:\r\n     - vault\r\n   image: \"jenkinsci/blueocean:latest\"\r\n   ports:\r\n     - \"8080:8080\"\r\n   volumes:\r\n     - /var/run/docker.sock:/var/run/docker.sock\r\n     - 'jenkins-data:/var/jenkins_home'\r\nvolumes:\r\n jenkins-data:\r\n\r\n```\r\n\r\n# Initial Vault configuration\r\n\r\nThe first command starts the container and the second one logs us in the Vault container to \"unseal\" it.\r\n\r\nThis is the term used by Vault to say the service is ready to be used and to do so we need 3 of the 5 keys that were generated by the init command.\r\n\r\nThis is only needed the first time we setup vault and you should store the keys in a secure place.\r\n\r\n```bash\r\n$ docker-compose up -d\r\nCreating vaultjenkins_consul_1 ... \r\nCreating vaultjenkins_consul_1 ... done\r\nCreating vaultjenkins_vault_1 ... \r\nCreating vaultjenkins_vault_1 ... done\r\nCreating vaultjenkins_blueocean_1 ... \r\nCreating vaultjenkins_blueocean_1 ... done\r\n\r\n$ docker exec -it vaultjenkins_vault_1 sh\r\n\r\n(vaultjenkins_vault_1)$ vault init\r\nUnseal Key 1: d28dc3e20848c499749450b411bdc55416cefb0ff6ddefd01ec02088aa5c90aa01\r\nUnseal Key 2: ad2b7e9d02d0c1cb5b98fafbc2e3ea56bd4d4fa112a0c61882c1179d6c6585f302\r\nUnseal Key 3: c393269f177ba3d07b14dbf14e25a325205dfbf5c91769b8e55bf91aff693ce603\r\nUnseal Key 4: 87c605e5f766d2f76d39756b486cbdafbb1998e72d2f766c40911f1a288e53a404\r\nUnseal Key 5: e97e5de7e2cdb0ec4db55461c4aaf4dc26092cb3f698d9cc270bf19dbb82eab105\r\nInitial Root Token: 5a4a7e11-1e2f-6f76-170e-b8ec58cd2da5\r\n\r\n(vaultjenkins_vault_1)$ vault unseal (run this 3 times with 3 different keys from above)\r\n\r\n```\r\n\r\n## Checking Vault is unsealed\r\n\r\nIf the unseal was successful then you can login to the UI by going to:\r\n\r\n[http://localhost:8500/ui/#/dc1/services/vault](http://localhost:8500/ui/#/dc1/services/vault)\r\n\r\n![image alt text](image_0.png)\r\n\r\n## Testing\r\n\r\nThe first command sets an alias so we dont have to keep typing the same thing over and over.\r\n\r\nWe will use the Vault client from inside the container but you can use one installed in your local machine too.\r\n\r\n```bash\r\n$ alias vault='docker exec -it vaultjenkins_vault_1 vault \"$@\"'\r\n\r\n$ vault write -address=http://127.0.0.1:8200 secret/billion-dollars value=behind-super-secret-password\r\n\r\n$ vault read secret/billion-dollars\r\nKey             \tValue\r\n---             \t-----\r\nrefresh_interval\t2592000\r\nvalue           \tbehind-super-secret-password\r\n```\r\n\r\n\r\nHere we store a secret text \"behind-super-secret-password\" in the path \"secret/billion-dollars\" and check in the UI if it was recorded encrypted.\r\n\r\n\r\n\r\n\r\n![image alt text](image_1.png)\r\n\r\n# Reading secrets from Jenkins\r\n\r\n## AppRole\r\n\r\nAppRole is a secure introduction method to establish machine identity. \r\n\r\nIn AppRole, in order for the application to get a token, it would need to login using a Role ID (which is **static**, and associated with a policy), and a Secret ID  (which is **dynamic**, one time use, and can only be requested by a previously authenticated user/system. In this case, we have two options:\r\n\r\n* Store the Role IDs in Jenkins\r\n\r\n* Store the Role ID in the Jenkinsfile of each project\r\n\r\n## Generating Policies and Roles\r\n\r\nNow Jenkins will need permissions to retrieve Secret IDs for our newly created role. Jenkins shouldn’t be able to access the secret itself, list other Secret IDs, or even the Role ID.\r\n\r\nIn this case, tokens assigned to the **java-example** policy would have permission to read a secret on the **secret/hello** path.\r\n\r\n```bash\r\n$ echo 'path \"auth/approle/role/java-example/secret-id\" {\r\n  capabilities = [\"read\",\"create\",\"update\"]\r\n}' \u003e ./config/vault/policies/jenkins_policy.hcl\r\n\r\n$ vault policy-write jenkins ./config/vault/policies/jenkins_policy.hcl\r\n\r\n```\r\n\r\n\r\nNow we have to create a **Role **that will generate **tokens **associated with that **policy**, and retrieve the token:\r\n\r\n```bash\r\n$ echo 'path \"secret/hello\" {\r\n  capabilities = [\"read\", \"list\"]\r\n}' \u003e ./config/vault/policies/java-example_policy.hcl\r\n$ vault policy-write java-example ./config/vault/policies/java-example_policy.hcl\r\n\r\nPolicy 'java-example' written.\r\n```\r\n```bash\r\n$ vault auth-enable approle\r\n$ vault write auth/approle/role/java-example \\\r\n\u003e secret_id_ttl=60m \\\r\n\u003e token_ttl=60m \\\r\n\u003e token_max_tll=120m \\\r\n\u003e policies=\"java-example\"\r\nSuccess! Data written to: auth/approle/role/java-example\r\n\r\n$ vault read auth/approle/role/java-example\r\nKey                 Value\r\n---                 -----\r\nbind_secret_id      true\r\nbound_cidr_list\r\nperiod              0\r\npolicies            [default java-example]\r\nsecret_id_num_uses  0\r\nsecret_id_ttl       3600\r\ntoken_max_ttl       0\r\ntoken_num_uses      0\r\ntoken_ttl           3600\r\n\r\n```\r\n\r\n\r\n\r\n## Generating Role ID\r\n\r\nWe generate a **Role ID** for our application that can be used in conjunction with the Jenkins **Token ID** to generate a **Secret ID** that will allow us to access the secret at the path \"secret/hello\" per our policy.\r\n\r\n```bash\r\n$ vault read auth/approle/role/java-example/role-id\r\nKey    \tValue\r\n---    \t-----\r\nrole_id\t73ea4552-53da-6844-bab0-0d39d2dc06aa\r\n```\r\n\r\n## Generating Jenkins Token ID\r\n\r\nJenkins needs a long lived token but it needs to be eventually rotated.\r\n\r\n```bash\r\n$ vault token-create -policy=jenkins\r\nKey            \tValue\r\n---            \t-----\r\ntoken          \tc70b74af-0e46-40da-ae15-c09b95a636f9\r\ntoken_accessor \t1c1ad1bf-1b56-e398-169a-7e14600a3cb3\r\ntoken_duration \t768h0m0s\r\ntoken_renewable\ttrue\r\ntoken_policies \t[default jenkins]\r\n```\r\n\r\n## Configuring Jenkins credentials to access Vault\r\n\r\nThis is the long lived Token ID we generated for Jenkins use.\r\n\r\n![image alt text](image_4.png)\r\n\r\nThis is the Role ID for our example app and is restricted to \"secret/hello\" by our policy, it can also be hardcoded in the source code or job script.\r\n\r\n![image alt text](image_3.png)\r\n\r\n## Reading the secret\r\n\r\nLet’s write a secret that our application will consume:\r\n\r\n    $ vault write secret/hello value=\"You've Successfully retrieved a secret from Hashicorp Vault\"\r\n    Success! Data written to: secret/hello\r\n\r\nWe are going to exchange **ROLE_ID **(belonging only to this particular job) and a temporary **SECRET_ID **(generated with the Jenkins token) for another **SECRET_ID**.\r\n\r\nThis second **SECRET_ID** has a policy attached that will only let us access the secrets at the path \"secret/hello\".\r\n\r\nThis way we can authenticate our jobs and assign them specific access to our secrets, without leaving any secrets in our our source code or shell scripts.\r\n\r\n\r\n```groovy\r\npipeline {\r\n  agent any\r\nstages {  \r\n  stage('Integration Tests') {\r\n      steps {\r\n      sh 'curl -s -o vault.zip https://releases.hashicorp.com/vault/0.9.1/vault_0.9.1_linux_amd64.zip ; yes | unzip vault.zip '\r\n        withCredentials([string(credentialsId: 'JAVA_EXAMPLE_ROLE_ID', variable: 'ROLE_ID'),string(credentialsId: 'JENKINS_VAULT_TOKEN', variable: 'VAULT_TOKEN')]) {\r\n        sh '''\r\n  set +x\r\n  export VAULT_ADDR=http://vault:8200\r\n  export VAULT_SKIP_VERIFY=true\r\n  export SECRET_ID=$(./vault write -field=secret_id -f auth/approle/role/java-example/secret-id)\r\n  export JOB_VAULT_TOKEN=$(./vault write -field=token auth/approle/login role_id=${ROLE_ID}   secret_id=${SECRET_ID})\r\n./vault read -address=http://vault:8200 secret/hello \r\n        '''\r\n    }\r\n   }\r\n  }\r\n }\r\n}\r\n```\r\n\r\n\r\n\r\n* We retrieve a **SECRET_ID **A using Jenkins administrative **JENKINS_VAULT_TOKEN **(that we’ve manually generated before). That’s the only one that would be relatively longed lived, but can only generate SECRET_IDs. Notice that we store this SECRET_ID A in the environment variable called VAULT_TOKEN so the vault client can use it.\r\n\r\n    * ```\r\n      ./vault write -field=secret_id -f auth/approle/role/java-example/secret-id\r\n      ```\r\n\r\n      ​\r\n\r\n* We do an AppRole login with the **ROLE_ID **and the **SECRET_ID A **(short lived), this gives us a second **SECRET_ID B** that we store in the environment variable VAULT_TOKEN:\r\n\r\n    * ```\r\n      ./vault write -field=token auth/approle/login role_id=${ROLE_ID} secret_id=${SECRET_ID}\r\n      ```\r\n\r\n* We use this **SECRET_ID B** to read our plain text secret:\r\n\r\n    * ```\r\n      ./vault read -address=http://vault:8200 secret/hello \r\n      ```\r\n\r\n      ​\r\n\r\n## Retrieving secrets from source code\r\n\r\nWe can also retrieve the secrets from an application at run time, using environment variables to authenticate to Vault like this:\r\n\r\n```java\r\n\u003ctd\u003e  String vaulttoken = System.getenv(\"VAULT_TOKEN\");\r\n    String vaulthost = System.getenv(\"VAULT_ADDR\");\r\n    System.out.format( \"Using Vault Host %s\\n\", vaulthost);\r\n    System.out.format( \"With Vault Token %s\\n\", vaulttoken );\r\n    /* This should be a separate method called from Main, then\r\n     * again for simplicity...\r\n     */\r\n    final VaultConfig config = new VaultConfig().build();\r\n    final Vault vault = new Vault(config);\r\n    try {\r\n    final String value = vault.logical()\r\n                   .read(\"secret/hello\")\r\n                   .getData().get(\"value\");\r\n    System.out.format( \"value key in secret/hello is \" + value +\"\\n\");\r\n    } catch(VaultException e) {\r\n      System.out.println(\"Exception thrown: \" + e);\r\n    }\r\n```\r\n\r\n## Job Output\r\n\r\n![image alt text](image_5.png)\r\n\r\n# Optional: Installing Vault plugin for Jenkins\r\n\r\nMore information: [https://github.com/jenkinsci/hashicorp-vault-plugin](https://github.com/jenkinsci/hashicorp-vault-plugin)\r\n\r\n![image alt text](image_2.png)\r\n\r\n\r\n\r\nIf we decide to use this plugin then we will be using the \"[Response Wrapping](https://www.vaultproject.io/docs/concepts/response-wrapping.html)\" feature from vault.\r\n\r\nThis feature creates one time use tokens that are used to access specific secrets and limiting the time they can be used.\r\n\r\n```groovy\r\nnode {\r\n  // define the secrets and the env variables\r\n  def secrets = [\r\n      [$class: 'VaultSecret', path: 'secret/testing', secretValues: [\r\n          [$class: 'VaultSecretValue', envVar: 'testing', vaultKey: 'value_one'],\r\n          [$class: 'VaultSecretValue', envVar: 'testing_again', vaultKey: 'value_two']]],\r\n      [$class: 'VaultSecret', path: 'secret/another_test', secretValues: [\r\n          [$class: 'VaultSecretValue', envVar: 'another_test', vaultKey: 'value']]]\r\n  ]\r\n\r\n  // optional configuration, if you do not provide this the next higher configuration\r\n  // (e.g. folder or global) will be used\r\n  def configuration = [$class: 'VaultConfiguration',\r\n                       vaultUrl: 'http://my-very-other-vault-url.com',\r\n                       vaultCredentialId: 'my-vault-cred-id']\r\n\r\n  // inside this block your credentials will be available as env variables\r\n  wrap([$class: 'VaultBuildWrapper', configuration: configuration, vaultSecrets: secrets]) {\r\n      sh 'echo $testing'\r\n      sh 'echo $testing_again'\r\n      sh 'echo $another_test'\r\n  }\r\n}\r\n```\r\n\r\n# References\r\n\r\nA lot of information for this article was used from:\r\n\r\n- http://nicolas.corrarello.com/general/vault/security/ci/2017/04/23/Reading-Vault-Secrets-in-your-Jenkins-pipeline.html \r\n\r\nThe Docker compose project was based on:\r\n\r\n- https://github.com/tolitius/cault\r\n\r\n  ​","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbruj0%2Fvault_jenkins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbruj0%2Fvault_jenkins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbruj0%2Fvault_jenkins/lists"}