{"id":20832853,"url":"https://github.com/udhos/boilerplate","last_synced_at":"2025-05-08T01:40:26.828Z","repository":{"id":150452700,"uuid":"622774292","full_name":"udhos/boilerplate","owner":"udhos","description":"boilerplate loads secrets from multiple sources, like AWS Secrets Manager, AWS Parameter Store, Hashicorp Vault, etc.","archived":false,"fork":false,"pushed_at":"2025-03-19T22:00:38.000Z","size":123,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-31T16:13:54.469Z","etag":null,"topics":["aws","environment-variables","go","golang","parameter-store","secrets-manager"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/udhos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-03T03:13:12.000Z","updated_at":"2025-03-19T22:00:37.000Z","dependencies_parsed_at":"2023-11-08T13:35:02.472Z","dependency_job_id":"7299af03-5e02-4af0-b64d-b7c022659393","html_url":"https://github.com/udhos/boilerplate","commit_stats":null,"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fboilerplate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fboilerplate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fboilerplate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fboilerplate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/udhos","download_url":"https://codeload.github.com/udhos/boilerplate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252983767,"owners_count":21835758,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","environment-variables","go","golang","parameter-store","secrets-manager"],"created_at":"2024-11-18T00:13:26.939Z","updated_at":"2025-05-08T01:40:26.818Z","avatar_url":"https://github.com/udhos.png","language":"Go","readme":"[![license](http://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/udhos/boilerplate/blob/main/LICENSE)\n[![Go Report Card](https://goreportcard.com/badge/github.com/udhos/boilerplate)](https://goreportcard.com/report/github.com/udhos/boilerplate)\n[![Go Reference](https://pkg.go.dev/badge/github.com/udhos/boilerplate.svg)](https://pkg.go.dev/github.com/udhos/boilerplate)\n\n# boilerplate\n\n[boilerplate](https://github.com/udhos/boilerplate) loads secrets from multiple sources, like AWS Secrets Manager, AWS Parameter Store, Hashicorp Vault, etc.\n\n* [envconfig](#envconfig)\n  * [Supported Stores](#supported-stores)\n    * [DynamoDB](#dynamodb)\n    * [Lambda](#lambda)\n    * [HTTP](#http)\n    * [Vault](#vault)\n  * [Usage](#usage)\n    * [Create a function to load app configuration from env vars](#create-a-function-to-load-app-configuration-from-env-vars)\n    * [How to define env var DB\\_URI](#how-to-define-env-var-db_uri)\n      * [Option 1: Literal value](#option-1-literal-value)\n      * [Option 2: Retrieve scalar value from AWS Secrets Manager](#option-2-retrieve-scalar-value-from-aws-secrets-manager)\n      * [Option 3: Retrieve JSON value from AWS Secrets Manager](#option-3-retrieve-json-value-from-aws-secrets-manager)\n* [References](#references)\n  * [Vault](#vault-1)\n    * [Curl](#curl)\n    * [Example](#example)\n    * [Testing AWS IAM Role](#testing-aws-iam-role)\n\nCreated by [gh-md-toc](https://github.com/ekalinin/github-markdown-toc.go)\n\n# envconfig\n\n## Supported Stores\n\n```\naws-secretsmanager: CONFIG_VAR=aws-secretsmanager:region:secret_name[:field_name]\naws-parameterstore: CONFIG_VAR=aws-parameterstore:region:parameter_name[:field_name]\naws-s3:             CONFIG_VAR=aws-s3:region:bucket_name,object_name[:field_name]\naws-dynamodb:       CONFIG_VAR=aws-dynamodb:region:table_name,key_name,key_value,value_attr[:field_name]\naws-lambda:         CONFIG_VAR=aws-lambda:region:func_name,key_name,key_value,body_field[:field_name]\n#http:              CONFIG_VAR=#http::method,proto,host,port,path,content_type,body_base64,token[:field_name]\nvault:              CONFIG_VAR=vault::token,token-value,proto,host,port,secret_path[:field_name]\nproxy:              CONFIG_VAR=proxy||proto,host,port,secret_name[|field_name]\n```\n\n`:field_name` is optional. If provided, the object will be decoded as JSON/YAML and the specified field name will be extracted.\n\nExamples:\n\n```\nexport DB_URI=aws-secretsmanager:us-east-1:database:uri\nexport DB_URI=aws-parameterstore:us-east-1:/microservice9/mongodb:uri\nexport DB_URI=aws-s3:us-east-1:bucketParameters,app7/mongodb.yaml:uri\nexport DB_URI=aws-dynamodb:us-east-1:parameters,parameter,mongodb,value:uri\nexport DB_URI=aws-lambda:us-east-1:parameters,parameter,mongodb,body:uri\nexport DB_URI=vault::token,dev-only-token,http,localhost,8200,secret/myapp1/mongodb:uri\nexport DB_URI=proxy||http,localhost,8080,vault::token,dev-only-token,http,localhost,8200,secret/myapp1/mongodb:uri\n\necho -n '{\"parameter\":\"mongodb\"}' | base64\neyJwYXJhbWV0ZXIiOiJtb25nb2RiIn0=\n\nexport DB_URI=#http::GET,https,tttt.lambda-url.us-east-1.on.aws,443,/,text/plain,eyJwYXJhbWV0ZXIiOiJtb25nb2RiIn0=,Bearer secret:uri\n```\n\n### DynamoDB\n\n    export DB_URI=aws-dynamodb:us-east-1:parameters,parameter,mongodb,value:uri\n    #           Table: parameters\n    #             Key: parameter=mongodb\n    #  Attribute name: value\n    # Attribute value: {\"uri\":\"mongodb://127.0.0.1:27001/?retryWrites=false\"}\n\n### Lambda\n\n    export DB_URI=aws-lambda:us-east-1:parameters,parameter,mongodb,body:uri\n    #       Function: parameters\n    #        Request: {\"parameter\":\"mongodb\"}\n    # Response field: body\n    #       Response: {\"statusCode\": 200,\"body\": \"{\\\"uri\\\": \\\"mongodb://localhost:27017/?retryWrites=false\\\"}\"}\n\n### HTTP\n\n    export DB_URI=#http::GET,https,tttt.lambda-url.us-east-1.on.aws,443,/,text/plain,eyJwYXJhbWV0ZXIiOiJtb25nb2RiIn0=,Bearer secret:uri\n    #       Method: GET\n    #     Protocol: https\n    #         Host: tttt.lambda-url.us-east-1.on.aws\n    #         Port: 443\n    #         Path: /\n    # Content-Type: text/plain\n    #         Body: {\"parameter\":\"mongodb\"} (base64 encoded as eyJwYXJhbWV0ZXIiOiJtb25nb2RiIn0=)\n    #        Token: Bearer secret\n    #     Response: {\"uri\":\"mongodb://127.0.0.1:27001/?retryWrites=false\"}\n\n### Vault\n\n    export DB_URI=vault::token,dev-only-token,http,localhost,8200,secret/myapp1/mongodb:uri\n    # Auth method:  token\n    # Token:        dev-only-token\n    # Protocol:     http\n    # Host:         localhost\n    # Port:         8200\n    # Secret path:  secret/myapp1/mongodb\n    # Secret key:   mongodb\n    # Response:     {\"uri\":\"abc\"}\n    # JSON Field:   uri\n\n## Usage\n\n### Create a function to load app configuration from env vars\n\nSee example function `newConfig()` below.\n\nOr look at [examples/envconfig-example/config.go](examples/envconfig-example/config.go).\n\n```go\nimport (\n\t\"github.com/udhos/boilerplate/envconfig\"\n)\n\ntype appConfig struct {\n\tdatabaseURI  string\n\tdatabaseCode int\n\tdatabaseTidy bool\n}\n\nfunc newConfig(env *envconfig.Env) appConfig {\n\treturn appConfig{\n\t\tdatabaseURI:  env.String(\"DB_URI\", \"http://test-db\"),\n\t\tdatabaseCode: env.Int(\"DB_CODE\", 42),\n\t\tdatabaseTidy: env.Bool(\"DB_TIDY\", false),\n\t}\n}\n```\n\n### How to define env var DB_URI\n\n#### Option 1: Literal value\n\n    export DB_URI=http://real-db\n\n#### Option 2: Retrieve scalar value from AWS Secrets Manager\n\nIf you prefix env var value with `aws-secretsmanager:`, the envconfig package will try to fetch it from AWS Secrets Manager.\n\n    Format:\n    export CONFIG_VAR=aws-secretsmanager:region:secret_name\n\n    Example:\n    export DB_URI=aws-secretsmanager::database_uri\n\n    # `database_uri` is the name of the secret stored in AWS Secrets Manager\n    # The secret `database_uri` could store any scalar value like: `http://real-db`\n\n#### Option 3: Retrieve JSON value from AWS Secrets Manager\n\nIf you append \":\u003cjson_field\u003e\" to env var value, after the secret name, the package envconfig will retrieve the secret from AWS Secrets Manager and will attempt to extract that specific JSON field from the value.\n\n    Format:\n    export CONFIG_VAR=aws-secretsmanager:region:secret_name:json_field\n\n    Example:\n    export DB_URI=aws-secretsmanager::database:uri\n\n    # `database` is the name of the secret stored in AWS Secrets Manager\n    # `uri` is the name of the field to be retrieved from the JSON value\n    # The secret `database` should store a JSON value like: `{\"uri\":\"http://real-db\"}`\n    # In this example, the env var DB_URI will be assigned the value of the JSON field `uri`: `http://real-db`.\n\n# References\n\n## Vault\n\nGet started: https://developer.hashicorp.com/vault/docs/get-started/developer-qs\n\nVault server version: `hashicorp/vault:1.17.5`\n\nVault cli version:\n\n```\n$ vault version\nVault v1.17.5 (4d0c53e84094b8017d32b6e5b7f8142035c8837f), built 2024-08-30T15:54:57Z\n```\n\n```bash\ndocker run --rm -p 8200:8200 -e 'VAULT_DEV_ROOT_TOKEN_ID=dev-only-token' hashicorp/vault:1.17.5\n\nexport VAULT_ADDR=http://127.0.0.1:8200\nvault login\n\n(Enter Root Token: dev-only-token)\n\nvault kv put secret/myapp1 mongodb='{\"uri\":\"abc\"}'\n\nvault kv get secret/myapp1\n```\n\n### Curl\n\n```\ncurl -H \"X-Vault-Token: dev-only-token\" http://127.0.0.1:8200/v1/secret/data/myapp1\n\n{\"request_id\":\"2a91f4df-64da-b585-c85b-f197869319b9\",\"lease_id\":\"\",\"renewable\":false,\"lease_duration\":0,\"data\":{\"data\":{\"mongodb\":\"{\\\"uri\\\":\\\"abc\\\"}\"},\"metadata\":{\"created_time\":\"2024-09-06T04:48:43.268293382Z\",\"custom_metadata\":null,\"deletion_time\":\"\",\"destroyed\":false,\"version\":1}},\"wrap_info\":null,\"warnings\":null,\"auth\":null,\"mount_type\":\"kv\"}\n\ndata --\u003e {\"mongodb\":\"{\\\"uri\\\":\\\"abc\\\"}\"}\n```\n\n### Example\n\n`secret/myapp1` is created as\n`secret/data/myapp1`, with key\n`mongodb='{\"uri\":\"abc\"}`, and should be queried like\n`vault::token,dev-only-token,http,localhost,8200,secret/myapp1/mongodb:uri`.\n\n```\nsecret/myapp1/mongodb/uri:abc\n\nsecret  = mount\nmyapp1  = secret\nmongodb = key\nuri     = json field\n```\n\n```basH\n$ vault kv get secret/myapp1\n=== Secret Path ===\nsecret/data/myapp1\n\n======= Metadata =======\nKey                Value\n---                -----\ncreated_time       2024-09-06T04:48:43.268293382Z\ncustom_metadata    \u003cnil\u003e\ndeletion_time      n/a\ndestroyed          false\nversion            1\n\n===== Data =====\nKey        Value\n---        -----\nmongodb    {\"uri\":\"abc\"}\n```\n\n### Testing AWS IAM Role\n\n1. Run vault server with permission to AWS STS\n\n```\nkey=$(echo $(grep aws_access_key_id ~/.aws/credentials | awk -F= '{print$2}'))\nsecret=$(echo $(grep aws_secret_access_key ~/.aws/credentials | awk -F= '{print$2}'))\n\ndocker run --rm -p 8200:8200 \\\n    -e 'VAULT_DEV_ROOT_TOKEN_ID=dev-only-token' \\\n    -e AWS_ACCESS_KEY_ID=$key \\\n    -e AWS_SECRET_ACCESS_KEY=$secret \\\n    hashicorp/vault:1.17.5\n```\n\n2. Use vault cli to configure the server\n\n```\nCLIENT_IAM_ROLE_ARN=... ;# fill this with client role arn\n\nexport VAULT_ADDR=http://127.0.0.1:8200\n\nvault login\n\n# enable aws auth\nvault auth enable aws\n\n# create policy for role dev-role-iam\nvault policy write \"example-policy\" -\u003c\u003cEOF\npath \"secret/*\" {\n  capabilities = [\"create\", \"read\"]\n}\nEOF\n\n# create role dev-role-iam\nvault write \\\n  auth/aws/role/dev-role-iam \\\n  auth_type=iam \\\n  policies=example-policy \\\n  max_ttl=24h \\\n  bound_iam_principal_arn=$CLIENT_IAM_ROLE_ARN\n\n# Put a secret to query later\nvault kv put secret/myapp1 mongodb='{\"uri\":\"abc\"}'\n```\n\n3. Test client\n\nNOTE: Vault client sdk has some limitations: (1) It does no support AWS_PROFILE. (2) It does not support credential files (`~/.aws/credentials`). It only worked with aws env vars (role credentials from env vars).\n\n```\n# Login into $CLIENT_IAM_ROLE_ARN with `aws sts assume-role` and put values into env vars.\n\naws sts assume-role --role-arn $CLIENT_IAM_ROLE_ARN --role-session-name vault-example | gojq -r .Credentials \u003e /tmp/creds.json\nexport AWS_ACCESS_KEY_ID=$(gojq -r .AccessKeyId \u003c /tmp/creds.json)\nexport AWS_SECRET_ACCESS_KEY=$(gojq -r .SecretAccessKey \u003c /tmp/creds.json)\nexport AWS_SESSION_TOKEN=$(gojq -r .SessionToken \u003c /tmp/creds.json)\naws sts get-caller-identity\n\n# Then run:\n\nexport VAULT=vault::,dev-role-iam,http,localhost,8200,secret/myapp1/mongodb:uri\nvault-example\n```\n\nIf everything works properly, look for these lines:\n\n```\n2024/09/07 23:27:01 ##### RESULT1: vault::,dev-role-iam,http,localhost,8200,secret/myapp1/mongodb:uri: abc\n\n2024/09/07 23:27:01 ##### RESULT2: vault::,dev-role-iam,http,localhost,8200,secret/myapp1/mongodb:uri: abc\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudhos%2Fboilerplate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fudhos%2Fboilerplate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudhos%2Fboilerplate/lists"}