{"id":24702948,"url":"https://github.com/richlamdev/vault-pki-raft","last_synced_at":"2025-10-09T09:30:31.343Z","repository":{"id":140093742,"uuid":"540311415","full_name":"richlamdev/vault-pki-raft","owner":"richlamdev","description":"Ephemeral HashiCorp Vault Server (raft backend) TLS / PKI Engine for testing and learning","archived":false,"fork":false,"pushed_at":"2024-12-25T22:05:39.000Z","size":925,"stargazers_count":9,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-19T03:29:02.879Z","etag":null,"topics":["automation","bash","certificate","certificate-authority","deployment","elliptic-curve-cryptography","hashicorp-vault","hcl","openssl","pki","public-key-infrastructure","rsa-cryptography","rsa-encryption","scripting","ssl","ssl-certificate","tls","tls-certificate","vault","web-of-trust"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/richlamdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-23T06:38:01.000Z","updated_at":"2024-12-25T22:05:43.000Z","dependencies_parsed_at":"2024-07-27T22:43:33.155Z","dependency_job_id":"3fd0ee23-fe56-4b84-b9e6-476081f0b33e","html_url":"https://github.com/richlamdev/vault-pki-raft","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richlamdev%2Fvault-pki-raft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richlamdev%2Fvault-pki-raft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richlamdev%2Fvault-pki-raft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richlamdev%2Fvault-pki-raft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/richlamdev","download_url":"https://codeload.github.com/richlamdev/vault-pki-raft/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235807614,"owners_count":19047987,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","bash","certificate","certificate-authority","deployment","elliptic-curve-cryptography","hashicorp-vault","hcl","openssl","pki","public-key-infrastructure","rsa-cryptography","rsa-encryption","scripting","ssl","ssl-certificate","tls","tls-certificate","vault","web-of-trust"],"created_at":"2025-01-27T05:51:41.927Z","updated_at":"2025-10-09T09:30:25.694Z","avatar_url":"https://github.com/richlamdev.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HashiCorp Vault PKI with Raft backend\n\n- [HashiCorp Vault PKI with Raft backend](#hashicorp-vault-pki-with-raft-backend)\n  * [Introduction](#introduction)\n  * [Purpose](#purpose)\n  * [Prerequisites](#prerequisites)\n    + [Operating System](#operating-system)\n    + [Software](#software)\n    + [Optional](#optional)\n    + [Knowledge](#knowledge)\n  * [Quick Start](#quick-start)\n  * [Quick Start Explanation - short version](#quick-start-explanation---short-version)\n  * [Quick Start Explanation - long version](#quick-start-explanation---long-version)\n  * [Creating custom leaf certificates](#creating-custom-leaf-certificates)\n  * [Web Browswer Testing with Docker](#web-browswer-testing-with-docker)\n  * [Commands](#commands)\n    + [Save Vault state (save snapshot)](#save-vault-state--save-snapshot-)\n    + [Restore Vault state (restore snapshot)](#restore-vault-state--restore-snapshot-)\n    + [Stop Vault server](#stop-vault-server)\n    + [Clean up Vault](#clean-up-vault)\n    + [Store and retrieve secrets (non PKI)](#store-and-retrieve-secrets--non-pki-)\n  * [Revocation](#revocation)\n  * [Vault Configuration](#vault-configuration)\n  * [Security Concerns](#security-concerns)\n  * [Style Convention](#style-convention)\n  * [Why the name raft.sh?](#why-the-name-raftsh-)\n  * [Improvements](#improvements)\n  * [References](#references)\n\n\n## Introduction\n\nThis repo demonstrates [HashiCorp's Vault](https://www.hashicorp.com/products/vault)\nproduct as a Certificate Authority (CA) for Public Key Infrastructure (PKI).\n\nFollowing the Quick Start below, you're able to quickly genearate self-signed\ncertificates for learning or deployment in test environments.\n\nThis demo utilizes a raft as a [storage backend](https://www.vaultproject.io/docs/configuration/storage)\nas opposed to a file backend which is typical in many demos/tutorials.\nDeploying Vault with raft backend allows for simple backup and recovery of data\nvia built-in commands.  While there are other features with a raft deployment,\nconvienent backup and restoration were significant factors for using raft as a\nbackend.  Technically, raft is a [consensous algorithm](https://raft.github.io/),\nbut that is a digression.\n\n\n## Purpose\n\nThe primary purpose is to provide a development/test environment to learn PKI\nwith HashiCorp Vault.\n\nThis repo (platform) could potentially be used for personal/private networks,\nif you accept the shortcomings.  The limitations are the lack comprehensive\nfeatures, security best practices and proper deployment of HashiCorp\nraft backend infrastructure.\n\n\n## Prerequisites\n\n### Operating System\n\n- Developed on Ubuntu Linux 22.04 LTS.\n- Tested with Bash Shell\n\nNaturally this repo will work with other \\*nix Operating Systems and/or Shells\nwith modification.\n\n### Software\n\n- [HashiCorp Vault](https://www.vaultproject.io/downloads)\n    Note: Preferred to install Vault from Hashicorp, while Vault can be\n          installed via Snap package, the Snap version does not include\n          a GUI console.\n- [Jq](https://stedolan.github.io/jq/download/)\n- [OpenSSL](https://wiki.openssl.org/index.php/Binaries)\n\n### Optional\n*This enables convenient copy and paste of root token to login to Vault. (either CLI and/or GUI)*\n\n- [xclip](https://github.com/astrand/xclip)\n\n### Knowledge\n\n- Basic understanding of TLS certificates.  If knowledge is limited; this\n  platform is great to learn and play with TLS certificates and\n  Certificate Authority (CA)\n\n- Basic understanding of [HashiCorp Vault](https://www.vaultproject.io/).\n\n- Basic knowledge of Linux command line and BASH scripting.\n\n\n## Quick Start\n\nClone the Repo:\n\n```git clone https://github.com/richlamdev/vault-pki-raft.git```\\\n```cd vault-pki-raft```\n\nSteps:\\\n```./raft.sh start```\\\n```./create_root_inter_certs.sh```\\\n```./issue_cert_template.sh template```\n\n\n## Quick Start Explanation - short version\n\n1. Creates a Certificate Authority from a single Vault instance.\n\n2. Generates a signed leaf certificate, template.middleearth.test.\n   This certificate can be deployed to a web server.  Naturally,\n   this is just a demo, read below for instruction on how to\n   modify the output leaf certificate.\n\n3. Optional: Import the root certificate generated by Vault, to your browser\n   to allow the browser to trust the leaf certifcate presented.\n\n\n## Quick Start Explanation - long version\n\nThe above will perform the following:\n1. Deploys a single Vault instance with a raft backend. - [raft.sh]\n\n    a. This will unseal the vault and login in the current terminal (user)\n       as the root user.\n\n    b. This will save the unseal key in the file _unseal_key_ and root\n       token in the file _root_token_ in the current folder.  Naturally, this\n       not the most secure method to save the _unseal_key_ and _root_token_.\n       Actually, a single _unseal_key_ is not the best method for provisioning.\n       See Security Concerns section for more information.\n\n    c. After this is complete, the root token is saved to the system buffer.\n       This enables convenient login to the vault GUI console.  Visit\n       `http://127.0.0.1:8200/` via your browser.  Choose Token method to login\n       and paste the root token in the Token field to login.  This step is\n       optional, naturally the GUI provides a convenient way to explore\n       the certificates and/or actions as resuilt of the CLI commands.\n\n2. Enables Vault PKI Engine and creates a CA - [create_root_inter_certs.sh]\n\n    a. Creates a root certificate and self sign the certificate.\n       The root CA is designated by the variable ISSUER_NAME_CN.\n       By default the ISSUER_NAME_CN is \"Lord of the Rings\".\n       Change this value if you like.\n\n    b. Creates an intermediate certificate signing request, have the root\n       authority sign this certificate and store it within the CA.\n\n    c. Creates a role designated by the variable VAULT_ROLE to sign leaf\n       certificates.  Note the value of VAULT_ROLE, itself, is not critical.\n       However, the VAULT_ROLE value must be the same in both files,\n       _create_root_inter_certs.sh_ and _issue_cert_template.sh_.  This role\n       name is referenced (used) to sign leaf certificates.  If they do not\n       match, an error will occur. Change this value if you like, however keep\n       them consistent.  VAULT_ROLE is authorized to sign subdomains indicated\n       by the variable DOMAIN, in this case the Second Level Domain (SLD) and\n       Top Level Domain(TLD), the default value is \"middleearth.test\".  Again,\n       change this value as you like.\n\n    d. The self-signed CA root certificate and intermediate certificate chain\n       are stored in the directory as designated by the variable\n       $ROOT_INTER_DIR.  The directory default is _./root_inter_certs_.\n       Import the root certificate from this folder to your Operating System\n       Trusted Store or Web Browser.  If you're unaware how to import the root\n       certificate to either, a quick google search will help you.\n\n3. Issues a \\\"template\\\" certificate with a default Subject Common Name (CN)\n     template.middleearth.test - [issue_cert_template.sh]\n\n    a. The resulting public certificate, key file, as well as entire signed\n       json blob is stored in directory designated by the variable SUBJECT_CN.\n       Edit the HOST and DOMAIN variables to change the default value of\n       SUBJECT_CN.  Ensure the value of DOMAIN is the same in both files,\n       _create_root_inter_certs.sh_ and _issue_cert_template.sh_.  In this\n       example, the resulting certificate files will be stored in the directory\n       _template.middleearth.test_.\n\u003cbr/\u003e\n\nInspecting template.middleearth.test certificate via openssl command:\n\n![OpenSSL Inpection](images/openssl_inspection_certificate.png)\n\u003cbr/\u003e\n\u003cbr/\u003e\n\nOptionally deploy the template certificate to a web server for inspection via\nweb browser.  The below is taken from Nginx in a Ubuntu Virtual Machine\n(VM).  Naturally, alternatives would achieve similar results, such as Docker\nwith Apache or Nginx, Windows \u0026 IIS etc.  Refer to Web Browser Testing with\nDocker section.  The certificate inpsected via Firefox browser:\n![Firefox2](images/firefox_certificate2.png)\n\u003cbr/\u003e\n\u003cbr/\u003e\n\nIf you import the root certificate to your trusted store or browser update your\nlocal DNS (or update local /etc/hosts file) to resolve\ntemplate.middleaearth.test you will observe the certificate is trusted, denoted\nby the locked padlock symbol in your browser:\n\n![DNS](images/trusted_certificate_DNS.png)\n\u003cbr/\u003e\n\u003cbr/\u003e\n\nFurthermore, because the template script populates an IP address in the Subject\nAlternative Name (SAN) we have \"trust\" established when visiting the web URL\nvia IP.  Note, it's atypical to add an IP in the SAN for public\ncertificates, however, for internal/private networks this is your discretion.\n\n![IP SAN](images/trusted_certificate_SAN_IP.png)\n\u003cbr/\u003e\n\u003cbr/\u003e\n\nIf the root certificate is _not_ imported to the Web browser or added to the\nOperating System trusted store, then an error similar to this will appear:\n![Certificate error](images/not_trusted_certificate_dns.png)\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n\n## Creating custom leaf certificates\n\nWith minimal changes, custom leaf certificates can be generated with these\nscripts.\n\nEdit the following in the _env.sh_ file.\n\n**Host / Subdomain:**\\\nPass in the HOSTNAME (aka subdomain) for the certificate as an argument to\n./issue_cert_template.sh IE: `./issue_cert_template.sh template`\n\n**Second Level Domain:**\\\nEdit Second Level Domain (SLD_STRING) variable.\n\n**Top Level Domain:**\\\nEdit Top Level Domain (TLD_STRING) variable.\n\n**IP Address - Subject Alternative Name:**\\\nEdit IP_SAN1 variable.\nOptionally omit this variable to remove a IP entry in the SAN.\n\n**Common Name (CN) - Subject Common Name:**\\\nAs a best practice, the Subject Altnernative DNS name defaults to the value\nof $SUBJECT_CN_STRING\neg: hostname.secondleveldomain.topleveldomain or template.middleearth.test\n\n\n---OPTIONAL VARIABLES---\n\nedit ISSUER_NAME_CN for the Certificate Authority (CA) want to appear on certificates.\n\n**Certificate Authority (Issuer):**\\\nEdit the ISSUER_NAME_CN variable.\n\n**TTL:**\\\nEdit the TTL variable (aka expiry) to adjust the validity period of the\ncertificate in _issue_cert_template.sh_. This number is set in hours, per valid\nby Vault software.\n\nPresently the TTL is set for 9528 hours or 397 days.  Set to 397 days as per\ndefault expiry for public leaf certificates governed by major browser\nmanufacturers, Google, Apple, and Mozilla.  More often than not, many public\ncertificates have a one year maximum validity period. (sometimes less, in\nthe case of automated certifcate renewals, such as Let's Encrypt).\nNaturally, in the case of development or private environment set the length to\nyour desire.  Note that it cannot exceed the TTL of the root CA, which is\ndefault 10 years.  Refer to References section below for more information.\nReference: https://support.apple.com/en-ca/HT211025\n\n\n## Web Browswer Testing with Docker\n\nAfter the `issue_cert_template.sh` script has been run, you can test the\ncustom certificates in a web browser via docker container.  Change to the\n./docker folder and adjust the Dockerfile to your needs and execute the\n`test-certificate.sh` script.\n\nAdditionally after the `create_root_inter_certs.sh` script has been run, the\nroot certificate is copied to the ./docker folder.  This is not required for\nthe docker container, however, it's placed in this folder for convenience to\nimport to your trusted store or web browser.\n\n\n## Commands\n\n### Save Vault state (save snapshot)\n\nTo make a backup, or a snapshot of any certifcates (secrets) stored in Vault\nrun the following command:\\\n```./raft.sh save```\n\nThis will save the state in the current folder under a folder named\n*backup_xxxx*.  Where xxxx is a random identifier generated.  The purpose\nof the unique identifier is to distinguish between multiple saved states.\n\nEach backup folder will contain three files:\\\n-the snapshot file itself, named _snapshotxxxx_\\\n-the unseal key, saved in the file named _unseal_keyxxxx_\\\n-the root token, saved in the file named _root_tokenxxxx_\\\n\nWhen a snapshot is taken, the matching unseal key and root\ntoken must be used to access the vault when a snapshot is restored.  This is\nperformed transparently by this script.  Refer to below Restore section for\nmore information.\n\n### Restore Vault state (restore snapshot)\n\nTo restore a backup, or a snap of any previously saved state run the following\ncommand:\\\n```./raft.sh restore backup_xxxx```\n\nThis will unseal the vault and login in the current terminal (user) as the\nroot user with the associated backup unseal key and root token.\n\nAny previously generated certificates or other stored data within the Vault\nwill be restored.\n\n### Stop Vault server\n\nTo stop the Vault server process, run the following command:\\\n```./raft.sh stop```\n\nThis will stop the current Vault service, remove the storage folder, and\ndelete the associated unseal key and root token.\n\nIf there is data (secrets) to be kept with the working Vault session, run\nthe command before stopping the sessions.  Refer to Save Vault state section.\n\n### Clean up Vault\n\nTo clean up all certificates and backup folders, run the following command:\\\n```./raft.sh cleanup```\n\nIn addition to executing ```./raft.sh stop```, cleanup will also remove the\nroot and intermediate certificates, all domain certificates created and the all\nbackup folders.\n\nThis command is destructive and is primarily used to sanitize the\nworking folder.\n\n\n### Store and retrieve secrets (non PKI)\n\nA couple miscellaneous functions have been left in _raft.sh_ as random\nfunctions to demonstrate basic secret storage and retrieval within Vault.\nThese functions are irrelevant to PKI, arguably do not belong here,\nhowever they remain in this script as basic examples.\n\nTo enable a secrets engine named kv and store a value at kv/apikey with\nhardcoded example data _webapp=AAABBB238472320238CCC_, run the command:\n\n```./raft.sh putdata```\n\nTo retrieve and display the above stored data run the following command:\n\n```./raft.sh getdata```\n\n\n## Revocation\n\nIf this repo / deployment is used as an ephemeral instance, revocation\nwill not work without alternative configuration/additional servers.  Web\nbrowsers or clients need to verify validity of certificates against a\nCertificate Revocation List (CRL) or via a Online Certificate Status Protocol\n(OCSP) server.\n\nIf this repo is only used for learning/development, revocation is moot.  If\nthis is to be used for a personal home network, then you can determine your\nrisk tolerance and/or alternatives for maintaining certificate revocation and/\nor expiry.\n\nAn alternative to verifying certificate validity against a CRL or OCSP would be\nto set short certificate expiry (aka Time-To-Live (TTL)).  Doing so would\nrequire new certificates to be issued frequently, and preferably in an\nautomated fashion.  This would also require a Vault instance to deployed\npermanently...\n\nFor further information regarding certificate revocation, refer to the links\nin the Reference section.\n\n\n## Vault Configuration\n\nThe file _vault_config.hcl_ is the configuration file for the Vault server.\nThe configuration is fairly straight forward, for more information please refer\nto the [HashiCorp documentation](https://developer.hashicorp.com/vault/docs/configuration).\n\n\n## Security Concerns\n\nThis is *not* meant for a production environment for several reasons.\n\n1.  This deployment is a single instance.  This is not a reslient deployment.\n\n2.  Vault is configured with a single seal token.  Typically, five tokens are\ncreated and a requires a minimum of three tokens to unseal.\n\n3.  The unseal token is exported/displayed in plain-text.  A preferred method\nwould be to export and encrypt the token with a public PGP key of the person(s)\nresponsible for maintenance.\n\n4.  The root token is writen to disk.  For a typical deployment, according to\nHashicorp best practices, would be to [destroy the root token](https://developer.hashicorp.com/vault/docs/concepts/tokens#root-tokens)\nafter the root token has been utlized to setup Vault.\n\n5.  The secret certificate (key) for generated certificates are written to the\nlocal disk.  The intenion here was to have all generated certificates\nreadily available to inspect for learning purposes.  Naturally storing secret\ncertificates (keys) should be stored securely.  Exposed secret certificates\ncompromises secrecy/trust and defeats the purpose of a Public Key\nInfrastructure(PKI)!\n\n\n## Style Convention\n\nThe Bash scripts were written to adhere as close as possible to [Google's Shell\nStyle Guide](https://google.github.io/styleguide/shellguide.html)\n\n\n## Why the name raft.sh?\nThe reason the Vault script is name _raft.sh_ is for convenience of execution.\nStart to type ```./r``` followed by ```tab``` to autocomplete ```./raft.sh```\nfollowed by the subcommand to execute the desired command.\n\n\n## Improvements\n\n- Create a bash script to accept configuration parameters as command line\narguments instead of editing the script directly.\n\n\n## References\n\n[HashiCorp Build Your Own Certificate Authority](https://developer.hashicorp.com/vault/tutorials/secrets-management/pki-engine)\\\n[HashiCorp Storage Backend](https://www.vaultproject.io/docs/configuration/storage)\\\n[HashiCorp Vault Backup](https://learn.hashicorp.com/tutorials/vault/sop-backup)\\\n[HashiCorp Vault Restore](https://learn.hashicorp.com/tutorials/vault/sop-restore)\\\n[smallstep PKI article](https://smallstep.com/blog/everything-pki/)\\\n[Mozilla Blog - Public Certificate Expiry](https://blog.mozilla.org/security/2020/07/09/reducing-tls-certificate-lifespans-to-398-days/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichlamdev%2Fvault-pki-raft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frichlamdev%2Fvault-pki-raft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichlamdev%2Fvault-pki-raft/lists"}