{"id":20342494,"url":"https://github.com/jakubfijalkowski/kvenv","last_synced_at":"2025-04-11T23:43:29.948Z","repository":{"id":60116725,"uuid":"326062995","full_name":"jakubfijalkowski/kvenv","owner":"jakubfijalkowski","description":"A simple command-line utility written in Rust that allows running arbitrary commands within a custom environment that is loaded from Azure KeyVault, GCP Secret Manager, AWS Secrets Manager or Hashicorp Vault.","archived":false,"fork":false,"pushed_at":"2023-02-12T22:09:37.000Z","size":301,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-25T19:40:36.782Z","etag":null,"topics":["aws","azure-keyvault","gcp","hashicorp-vault","rust","secrets"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jakubfijalkowski.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-01T22:00:51.000Z","updated_at":"2024-07-07T18:19:12.000Z","dependencies_parsed_at":"2024-11-14T21:35:59.471Z","dependency_job_id":"5f328854-1de9-46d2-b695-4af7590bc7f4","html_url":"https://github.com/jakubfijalkowski/kvenv","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubfijalkowski%2Fkvenv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubfijalkowski%2Fkvenv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubfijalkowski%2Fkvenv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubfijalkowski%2Fkvenv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jakubfijalkowski","download_url":"https://codeload.github.com/jakubfijalkowski/kvenv/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248497903,"owners_count":21113982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","azure-keyvault","gcp","hashicorp-vault","rust","secrets"],"created_at":"2024-11-14T21:35:36.419Z","updated_at":"2025-04-11T23:43:29.916Z","avatar_url":"https://github.com/jakubfijalkowski.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kvenv\n\n`kvenv` is a simple command-line utility written in Rust that allows running arbitrary command with\na custom environment that is loaded from Azure KeyVault, GCP Secret Manager, AWS Secrets Manager or\nHashicorp Vault.\n\nThe main usage is in CI/CD pipelines - if your tool of choice does not support convenient,\nper-project secrets (or the functionality does not support multitenancy) management, you might store\nthe secrets in KV and then subsequently load it using the `kvenv` tool, without exposing secret\nstore credentials to the processes.\n\n## Usage\n\n### Basics\n\nThe app has three base commands:\n\n* `cache` - download an environment and store it in temporary file,\n* `run-with` - run the command with environment made with `cache` command, and\n* `run-in` - run the command with freshly downloaded environment.\n\n`cache` and `run-with` allow you to download the environment once, and use it for subsequent calls.\nThis can be used to optimize the number of network calls, sacrificing secrecy (because you store the\nsecret on-disk and, if not properly guarded, can be read by anyone).\n\n`run-in` can be used to run a command without storing anything to the disk. It downloads the\nenvironment and keeps it only in memory.\n\nThe environment downloaded from the secret storage is then joined with OS environment and pass it as\nthe process environment to the executed command.\n\n### Running command in fresh environment\n\n`run-in` can be used to start a command in an environment downloaded from the Cloud secret storage.\n\n```sh\nkvenv run-in [OPTIONS] \u003c--secret-name \u003cSECRET_NAME\u003e|--secret-prefix \u003cSECRET_PREFIX\u003e\u003e \u003c--aws|--azure|--google|--vault\u003e \u003cCOMMAND\u003e\n```\n\nExample:\n\n```sh\n$ kvenv run-in \\\n    --azure\n    --azure-tenant-id 00000000-0000-0000-0000-0000000000000\\\n    --azure-client-id 00000000-0000-0000-0000-0000000000000\\\n    --azure-client-secret appsecret \\\n    --azure-keyvault-name example-keyvault \\\n    --secret-name test \\\n    -- env\nDISPLAY=:0\nLANG=en_US.UTF-8\nPATH=/home/user\n...\nKEY_FROM_KV=Test\n```\n\n### Caching environment for faster subsequent runs\n\n`cache` + `run-with` pair can be used to first cache the environment and then run the commands with\nthat environment multiple times without accessing the storage at all.\n\nCache the environment\n\n```sh\n$ kvenv cache \\\n    --azure\n    --azure-tenant-id 00000000-0000-0000-0000-0000000000000\\\n    --azure-client-id 00000000-0000-0000-0000-0000000000000\\\n    --azure-client-secret appsecret \\\n    --azure-keyvault-name example-keyvault \\\n    --secret-name test \\\n    -- env\n/tmp/kvenv-xxxxx.json\n```\n\nRun a command with the environment\n\n```sh\n$ kvenv run-with --env-file /tmp/kvenv-xxxxx.json -- env\nDISPLAY=:0\nLANG=en_US.UTF-8\nPATH=/home/user\n...\nKEY_FROM_KV=Test\n```\n\nRemove the cached file\n\n```sh\n$ rm /tmp/kvenv-xxxxx.json\n```\n\n#### Snapshotting\n\nThe `cache` command supports `--snapshot-env` option that will store the `kvenv` process environment\nto the cached file and use it for subsequent runs instead of fresh process env.\n\n### Cloud secret storage selection\n\nEvery command that downloads environment (`cache` and `run-in`) takes one of the supported clouds:\n\n#### `--aws`\n\nUse AWS Secret Manager. It expects\n\n1. `--aws-region` - AWS region.\n\nIt uses [`rusoto`] crate underneath and supports any [AWS credentials]. You can also specify\ncredentials directly using:\n\n1. `--aws-access-key-id` (or `AWS_ACCESS_KEY_ID` environment variable), and\n2. `--aws-secret-access-key` (or `AWS_SECRET_ACCESS_KEY` environment variable).\n\n#### `--azure`\n\nUses Azure KeyVault. It expects:\n\n1. `--azure-keyvault-name` - the name of KeyVault, or\n2. `--azure-keyvault-url` - the full URL to KeyVault.\n\nIf name is provided, it constructs the KV url using `https://{name}.vault.azure.net`.\n\nThe app uses [`azure-sdk-for-rust`], thus supports all the [Azure authentication methods]. You can\nalso specify credentials directly:\n\n1. `--azure-tenant-id` (or `AZURE_TENATN_ID`),\n2. `--azure-client-id` (or `AZURE_CLIENT_ID`), and\n3. `--azure-client-secret` (or `AZURE_CLIENT_SECRET`).\n\n#### `--google`\n\nUses Google Secret Manager. It expects:\n\n1. `--google-project` - the GCP project name.\n\nThe app uses [`google-apis-rs`], thus supports all the [methods yup2-oauth supports]. You can\nalso specify credentials directly:\n\n1. `--google-credentials-file` (or `GOOGLE_APPLICATION_CREDENTIALS`), or\n2. `--google-credentials-json` (or `GOOGLE_APPLICATION_CREDENTIALS_JSON`).\n\nThe first one expects path to the credentials JSON file, the second one expects the **contents** of\nthe file.\n\n#### `--vault`\n\nUses Hashicorp Vault.\n\nIt expects:\n\n1. `--vault-address` - The address of the vault.\n\nIt does plain HTTPS requests, thus it expects:\n\n1. `--vault-token` (or `VAULT_TOKEN`), and\n2. `--vault-cacert` (or `VAULT_CACERT`).\n\n### Secret storage modes\n\nThere are two possible modes of secret storage:\n\n1. Environment stored as JSON in a single secret, or\n2. Secrets being environment variables.\n\n#### Env as JSON\n\nThis option is best if you want to store whole environment in a single place, or you want to store\nmultiple different environments in a single secret store and match it to the running program (e.g.\nper project/tenant configuration). It expects a single key-value pair in the underlying storage,\nwhere value is a JSON object with properties being non-complex values (so no arrays and no\nobjects).\n\nExample JSON environment:\n\n```json\n{\n    \"Variable_A\": false,\n    \"Variable_B\": \"Value\",\n    \"Variable_C\": 10\n}\n```\n\nTo get the environment as JSON, use the `--secret-name` option.\n\n#### Prefixed mode\n\nIf you prefer storing a single environment variable as a single secret in the storage, you can use\nprefixed mode. It finds all variables that start with a given prefix and interprets them as\nenvironment variables. The prefix will be stripped from secret name before using it as environment\nvariable.\n\nTo get the environment as a list of prefixed secrets, use the `--secret-prefix` option.\n\n##### A note on Azure KeyVault\n\nSince AKV secrets cannot have `_` in the name, all `-` will be replaced with `_` (to follow the\nconvention used by ASP.NET Core).\n\n##### A note on Hashicorp Vault\n\nSince Vault stores a list of values for a single secret, `kvenv` adheres to that - it does not try\nto force additional JSON encoding, it will get all pairs for a given secret directly.\n\nWhen in prefixed mode, it gets all pairs for all the secrets that match the prefix and concatenate\nthem.\n\n### Misc\n\n#### Masking\n\n`kvenv`, supports masking environment variables, i.e. hiding them from the destination process. This\ncan be achieved by using the `--mask` option, like so:\n\n```sh\n$ kvenv run-in ... --mask HOME --mask ANOTHER -- env\n# There will be no `HOME` nor `ANOTHER` in the output\n```\n\nor\n\n```sh\n$ kvenv cache ... --mask HOME --mask ANOTHER\n# There will be no `HOME` nor `ANOTHER` in the output cached file\n```\n\nSubsequent runs with the cached env file won't be able to see any of the mentioned variables.\n\n## Features\n\n* [x] Masking\n* [x] JSON-based env keys\n* [x] [ASP.NET Core-compatible](https://docs.microsoft.com/en-us/aspnet/core/security/key-vault-configuration?view=aspnetcore-5.0) key format\n* [x] `run-with` with direct env credentials (a.k.a `run-in`)\n* [x] Better documentation\n* [x] Integration tests\n* [x] GCP Secret Manager support\n* [x] AWS Secrets Manager support\n* [x] Hashicorp Vault support\n\n## Help\n\nThe cmdline is [Clap](https://clap.rs/)-based, so you have convenient help out of the box:\n\n```sh\n$ kvenv help\nA simple command-line utility that allows running arbitrary commands within a custom environment thatis loaded from Azure KeyVault, GCP Secret Manager, AWS Secrets Manager or Hashicorp Vault.\n\nUsage: kvenv \u003cCOMMAND\u003e\n\nCommands:\n  cache\n          Caches the environment variables from KeyVault into local file\n  run-with\n          Runs the command with the specified argument using cached environment\n  run-in\n          Runs the command with the specified argument using freshly downloaded environment\n  help\n          Print this message or the help of the given subcommand(s)\n\nOptions:\n  -h, --help\n          Print help\n  -V, --version\n          Print version\n```\n\n[`rusoto`]: https://github.com/rusoto/rusoto/\n[AWS Credentials]: https://github.com/rusoto/rusoto/blob/master/AWS-CREDENTIALS.md\n[`azure-sdk-for-rust`]: https://github.com/Azure/azure-sdk-for-rust\n[Azure authentication methods]: https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/identity/examples/default_credentials.rs\n[`google-apis-rs`]: https://github.com/Byron/google-apis-rs\n[methods yup2-oauth supports]: https://docs.rs/yup-oauth2/latest/yup_oauth2/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakubfijalkowski%2Fkvenv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjakubfijalkowski%2Fkvenv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakubfijalkowski%2Fkvenv/lists"}