{"id":19107272,"url":"https://github.com/govtechsg/kms-aes","last_synced_at":"2025-06-28T09:36:43.835Z","repository":{"id":74216798,"uuid":"125945714","full_name":"GovTechSG/kms-aes","owner":"GovTechSG","description":"Ansible roles and playbooks to encrypt and decrypt files with a key backed by a AWS KMS Customer Master Key","archived":false,"fork":false,"pushed_at":"2018-10-31T04:48:19.000Z","size":29,"stargazers_count":2,"open_issues_count":1,"forks_count":3,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-04-30T18:45:18.051Z","etag":null,"topics":["aes","ansible","aws","aws-kms","encryption","kms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GovTechSG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-20T01:55:32.000Z","updated_at":"2024-01-19T10:25:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"2f68334e-12b0-43ef-a6dd-ce3353e0b941","html_url":"https://github.com/GovTechSG/kms-aes","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/GovTechSG/kms-aes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Fkms-aes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Fkms-aes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Fkms-aes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Fkms-aes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GovTechSG","download_url":"https://codeload.github.com/GovTechSG/kms-aes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Fkms-aes/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262407042,"owners_count":23306304,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aes","ansible","aws","aws-kms","encryption","kms"],"created_at":"2024-11-09T04:11:59.216Z","updated_at":"2025-06-28T09:36:43.818Z","avatar_url":"https://github.com/GovTechSG.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AES Encryption with KMS\n\nThis repository contains Ansible roles and playbooks to encrypt and decrypt\nfiles with a key backed by a KMS Customer Master Key.\n\n## Pre-requisites\n\nFor the local machine executing the playbook, you will need to install at least\n[Ansible 2.4](https://docs.ansible.com/ansible/latest/intro_installation.html).\n\nFor the machine where the encryption takes place (i.e. Ansible remote -- this\ncan also be the local machine), you will need the following installed:\n\n- [AWS CLI](https://aws.amazon.com/cli/)\n- OpenSSL\n\nYou will need the following for your AWS acocunt:\n\n- AWS credentials on the machine performing the encryption. See\n  [this](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html)\n  for more information.\n- Customer Master Key (CMK) in Key Management Service (KMS)\n\n## General Concepts\n\nYou might want to [read](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html)\nsome concepts for KMS from the AWS documentation.\n\nIn this repository, we encrypt data with the symmetric algorithm AES 256 CBC.\nThe CMK in KMS acts as a Key encryption key (KEK) and the master key for the AES\nkey that will use to perform AES encryption.\n\nWe ask KMS to generate the AES key (known as a \"data key\" in AWS documentation)\nand then encrypt it with some\n[encryption context](https://docs.aws.amazon.com/kms/latest/developerguide/encryption-context.html).\n\nEach time we want to perform an encryption or decryption operation, we will ask\nKMS to decrypt the encrypted data key first. We will store the decrypted key in\nmemory for the duration of the operation only.\n\n## Playbooks\n\nThe playbooks contain full examples on how you might want to encrypt and decrypt\nfiles with a key backed by KMS. Each playbook will generate a data key from KMS\nand then use the data key to perform encryption and decryption. You can skip the\nvarious tasks using\n[tags](https://docs.ansible.com/ansible/latest/playbooks_tags.html).\n\nYou will also have to define\n[variables](https://docs.ansible.com/ansible/latest/playbooks_variables.html)\neither in the playbook themselves, or\n[override](https://docs.ansible.com/ansible/latest/playbooks_variables.html#passing-variables-on-the-command-line)\non the command line while executing the playbooks.\n\n### Tags\n\nUse the tags to skip or only execute certain tasks.\n\n- `generate_key`: KMS data key generation\n- `encrypt`: Encrypt data\n- `decrypt`: Decrypt data\n\n### Vault Playbook\n\nThis playbook is defined in `vault.yml`. This playbook is the most\nstraightforward and does not require any templates. The intent is to transform\nthe ansible encrypted vault file into AES key encrypted vault file in the\nencrypt step, and also to transform the AES key encrypted vault file to\ndecrypted vault file for direct `include_vars`.\n\nThe following variables are needed:\n\n- `key_id`: ID of the CMK.\n- `cli_json`: Path to a [CLI JSON parameter](https://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html#cli-using-param-json)\n  file. Use this to define the encryption context for your encrypted data key.\n  You can use the default `cli.json` as a base.\n- `key_output`: Path to output the encrypted data key to.\n- `vault_file`: Path to the ansible vault file. Assumes to be ansible-vault\n  encrypted, but also works for non-encrypted vault, for e.g. plain YAML file.\n- `encrypted_vault_file`: Path to the AES key encrypted vault file.\n- `decrypted_vault_file`: Path to the AES key decrypted vault file.\n\nTo test this playbook, run the following in the repository root to create your\nsecret:\n\n```bash\nprintf -- \"---\\nkey: value\" \u003e secrets.yml\nansible-vault encrypt secrets.yml\n```\n\nThink of a vault password and enter in this password twice to ansible-vault\nencrypt the `secrets.yml` vault file. In real use case, the ansible-vault\nencrypted vault file should have already been provided and is safe to check into\nany repository.\n\nNext we can test out the `vault.yml` playbook by first generating the AWS data\nkey.\n\n```bash\nansible-playbook -i inventory vault.yml -t generate_key\n```\n\nBy default, this should generate the AWS data key in `keys/kms.json`. The\n`KeyId` value points to the CMK stored in AWS, while the `CiphertextBlob` value\nis the generate AWS data key, but encrypted with CMK.\n\nNext we will transform the encryption on the vault file from ansible-vault\nencrypted version to a AWS data key encrypted version.\n\n```bash\nansible-playbook -i inventory --ask-vault-pass vault.yml -t encrypt\n```\n\nThis should generate the `secrets.encrypted.yml` file. In real use case, this is\nthe point where both the `keys/kms.json` and `secrets.encrypted.yml` should be\ncopied into the custom AMI.\n\nTo complete the picture, run the following to see the decrypted outcome:\n\n```bash\nansible-playbook -i inventory vault.yml -t decrypt\n```\n\nThis should generate the `secrets.decrypted.yml` file, where you should see\nthe original content of the `secrets.yml` without any encryption. In real use\ncase, this should only be done in the bootstrapping phase of the custom AMI to\nget the secrets.\n\n### Directory Playbook\n\nThis playbook is defined in `directory.yml`. Files are encrypted from a template\ndirectory to a destination encrypted directory, or decrypted from an encrypted\ndirectory to a destination decrypted directory. Subdirectories are preserved.\n\nThe following variables are needed:\n\n- `key_id`: ID of the CMK.\n- `cli_json`: Path to a [CLI JSON parameter](https://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html#cli-using-param-json)\n  file. Use this to define the encryption context for your encrypted data key.\n  You can use the default `cli.json` as a base.\n- `key_output`: Path to output the encrypted data key to.\n- `template_dir`: Directory where your templates are.\n- `encrypted_dir`: Directory to output encrypted files to. This will also be the\n  input directory when decrypting.\n- `decrypted_dir`: Directory to output the decrypted files to.\n- `secrets_file`: A secret file to include. This file can be a file encrypted\n  with [Ansible vault](https://docs.ansible.com/ansible/2.4/vault.html).\n\nFor example, to execute the playbook locally on your machine, you can do\nsomething like\n\n```bash\nansible-playbook --inventory inventory --ask-vault-pass directory.yml\n```\n\n### List playbook\n\nThis playbook is defined in `list.yml`. This playbook encrypts and decrypts file\nbased on a list defined.\n\nEach item in a list *must* contain the following keys:\n\n- `template`: Path to the source template file\n- `encrypted`: Path to the encrypted output or input\n- `decrypted`: Path to the decrypted output\n\nThe following variables are needed:\n\n- `key_id`: ID of the CMK.\n- `cli_json`: Path to a\n  [CLI JSON parameter](https://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html#cli-using-param-json)\n  file. Use this to define the encryption context for your encrypted data key.\n  You can use the default `cli.json` as a base.\n- `key_output`: Path to output the encrypted data key to.\n- `secrets_file`: A secret file to include. This file can be a file encrypted\n  with [Ansible vault](https://docs.ansible.com/ansible/2.4/vault.html).\n- `templates`: The list defined above.\n\nFor example, to execute the playbook locally on your machine, you can do\nsomething like\n\n```bash\nansible-playbook --inventory inventory --ask-vault-pass list.yml\n```\n\n## Roles\n\nThe `roles` directory contains some common tasks that you might be able to reuse\nin your own playbooks.\n\n- `filters`: Contains some Jinja filters that are used by the rest of the roles\n  and tasks.\n- `kms-data-key`: Use AWS KMS to generate a new data key\n- `kms-decrypt`: Use AWS KMS to decrypt some encrypted ciphertext.\n- `aes-encrypt`: Encrypt plaintext with AES 256 CBC. The IV is appended as the\n  final 16 bytes of the ciphertext.\n- `aes-decrypt`: Decrypt ciphertext with AES 256 CBC. The IV is assumed to be\n  the final 16 bytes of the ciphertext.\n\n## Tasks\n\nThe `tasks` directory contains some reusable tasks that you can use in your own\nplaybooks.\n\n- `secrets`: This simply includes a variable file to your play.\n- `generate_key`: Use KMS to generate a new data key.\n- `read_key`: Read and optionally decrypt a KMS JSON output containing the data\n  key for our encryption and decryption operations.\n- `encrypt_directory` and `decrypt_directory`: Encrypt and decrypt files on a\n  directory basis.\n- `encrypt_list` and `decrypt_list`: Encrypt and decrypt files based on a\n  provided list.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgovtechsg%2Fkms-aes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgovtechsg%2Fkms-aes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgovtechsg%2Fkms-aes/lists"}