{"id":21513065,"url":"https://github.com/turnerlabs/cstore","last_synced_at":"2025-10-25T06:46:26.726Z","repository":{"id":34710920,"uuid":"148514905","full_name":"turnerlabs/cstore","owner":"turnerlabs","description":"CLI for central file or configuration storage and sharing among team members and applications based on project context.","archived":false,"fork":false,"pushed_at":"2023-12-18T21:46:49.000Z","size":22039,"stargazers_count":5,"open_issues_count":4,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-23T20:43:48.800Z","etag":null,"topics":["aws","aws-parameter","aws-s3","aws-secret","cli","config","configuration","docker","lambda","secrets"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/turnerlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-12T17:12:40.000Z","updated_at":"2020-05-01T13:16:00.000Z","dependencies_parsed_at":"2024-06-20T22:03:12.328Z","dependency_job_id":"253dc566-73f7-4aae-b812-34c29065db8d","html_url":"https://github.com/turnerlabs/cstore","commit_stats":null,"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/turnerlabs","download_url":"https://codeload.github.com/turnerlabs/cstore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248092152,"owners_count":21046421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-parameter","aws-s3","aws-secret","cli","config","configuration","docker","lambda","secrets"],"created_at":"2024-11-23T22:52:30.156Z","updated_at":"2025-10-25T06:46:21.678Z","avatar_url":"https://github.com/turnerlabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# README\n\nSimple, secure, and flexible configuration management.\n\nThe cStore CLI provides a command to push config files to remote [storage](docs/STORES.md) using `$ cstore push service/dev/.env`. The pushed files are replaced by a, `cstore.yml` [file](docs/CATALOG.md), that remembers the storage location, file encryption, and other details making restoration locally or by a service as simple as `$ cstore pull -t dev`.\n\n`*.env` and `*.json` are special file types whose secrets can be [tokenized](docs/SECRETS.md), encrypted, stored separately from the configuration, and injected at runtime.\n\n\u003cdetails\u003e\n  \u003csummary\u003eSecurity Best Practices\u003c/summary\u003e\n\n  While cStore provides a simple and flexible way to store and retrieve configuration and secrets, the user has the responsibility to ensure the usage patterns and storage solution meet the oganization's security requirements.\n\n  ### Best Practices ###\n  * Understand your organizations security requirements.\n  * Understand what cStore does before using it.\n  * Never print or send cStore's `stdout` to logs.\n  * Always use encryption when storing secrets.\n  * Use your organization's approved vaults for storing secrets.\n  * Avoid exporting secrets into the environment when possible.\n  * Realize many security mistakes are made by users; so, be careful!\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eHow it Works\u003c/summary\u003e\n\n```\n├── project\n│   ├── components\n│   ├── models\n│   ├── main.go\n│   ├── Dockerfile \n│   ├── cstore.yml (catalog)\n│   └── service\n│       └── dev\n│       │   └── .env (stored)\n│       |   └── .cstore (ghost)\n│       |   └── fargate.yml\n│       |   └── docker-compose.yml\n│       │\n│       └── prod\n│           └── .env (stored)\n│           └── .cstore (ghost)\n│           └── fargate.yml\n│           └── docker-compose.yml\n```\nThe `cstore.yml` [catalog](docs/CATALOG.md) and hidden `.cstore` ghost files reference the stored `*.env` files. Secrets no longer need to be checked into source control.\n\nWhen the repository has been cloned or the project shared, running `$ cstore pull` in the same directory as the `cstore.yml` [catalog](docs/CATALOG.md) or any of the `.cstore` ghost files will locate, download, and decrypt the configuration files to their respective original location restoring the project's environment configuration.\n\nExample: `cstore.yml`\n```yml\nversion: v4\ncontext: project\nfiles:\n- path: service/dev/.env\n  store: aws-s3\n  type: env\n  data:\n    AWS_S3_BUCKET: my-bucket\n    AWS_STORE_KMS_KEY_ID: \"\"\n    AWS_VAULT_KMS_KEY_ID: aws/secretsmanager\n  tags:\n  - service\n  - dev\n  vaults:\n    access: env\n    secrets: aws-secrets-manager\n  versions: []\n- path: service/prod/.env\n  store: aws-parameter\n  type: env\n  data:\n    AWS_STORE_KMS_KEY_ID: aws/ssm\n    AWS_VAULT_KMS_KEY_ID: aws/secretsmanager\n  tags:\n  - service\n  - prod\n  vaults:\n    access: env\n    secrets: aws-secrets-manager\n  versions: []\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eInstall / Upgrade\u003c/summary\u003e\n\n| OS | CMD | Notes |\n|----|----|----|\n| Mac | ```$ sudo curl -L -o  /usr/local/bin/cstore https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_darwin_amd64 \u0026\u0026 sudo chmod +x /usr/local/bin/cstore``` ||\n| Linux | ```$ sudo curl -L -o  /usr/local/bin/cstore https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_linux_386 \u0026\u0026 sudo chmod +x /usr/local/bin/cstore``` ||\n| Windows | ```C:\\\u003e mkdir %HOMEPATH%\\cstore\\bin \u0026 wget -O %HOMEPATH%\\cstore\\bin\\cstore.exe https://github.com/turnerlabs/cstore/releases/download/v3.8.0-alpha/cstore_windows_amd64.exe``` (add `%HOMEPATH%\\cstore\\bin` to the `PATH` to make cstore executable from anywhere)|install requires wget v1.20 |\n\n\u003c/details\u003e\n\n## Authenticate ##\n\n[AWS credential chain](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) is used for Authentication.\n\n```bash\n$ export AWS_REGION=us-east-1\n$ export AWS_PROFILE=user-profile\n```\n\n## Store App Configuration ##\n\nEnsure a [storage](docs/STORES.md) solution is available and supports the configuration file type.\n\nDuring a push, [tokenized](docs/SECRETS.md) secrets are removed and stored in AWS Secrets Manager.\n\n\u003cdetails open\u003e\n  \u003csummary\u003eStore Env Configs\u003c/summary\u003e\n\n```bash\n$ cat service/dev/.env # example\n```\n```\nHEALTHCHECK=/ping\nMONGO_URL=mongodb://{{dev/user::appuser-dev}}:{{dev/password::3lkjr4kfdro4df}}@example-server.mongodb.net:30000/example-dev\nAPI_KEY={{dev/token::82f6f303-9e00-4a8c-be26-b9d06781d844}}\nAPI_URL=https://dev.api.example-service.com\nCONTACT=team@example-service.com\n```\n\nPush configs to one of the following storage solutions.\n```bash\n$ cstore push service/dev/.env -s aws-parameter \n```\n```bash\n$ cstore push service/dev/.env -s aws-s3\n```\n```bash\n$ cstore push service/dev/.env -s aws-secret\n```\n```bash\n$ cstore push service/dev/.env -s source-control\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eStore Json Configs\u003c/summary\u003e\n\n```bash\n$ cat service/dev/config.json # example\n```\n```json\n{\n    \"db_url\" : \"mongodb://{{dev/user::app_user}}:{{dev/password::4kdnow55jdjnk3nd}}@example-server.mongodb.net:30000/example-dev\",\n    \"api_key\": \"{{dev/key::82f6f303-9e00-4a8c-be26-b9d06781d844}}\",\n    \"healthcheck\": \"/ping\",\n    \"contact\": \"team@example-service.com\"\n}\n```\n\n```bash\n$ cstore push service/dev/config.json -s aws-s3\n```\n```bash\n$ cstore push service/dev/config.json -s aws-secret\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eStore Multiple Configs\u003c/summary\u003e\n\n```bash\n$ cstore push service/dev/.env service/qa/.env\n```\n\nAuto discover and push multiple files in `service` folder.\n```bash\n$ cstore push $(find service -name '*.env')\n```\n\u003c/details\u003e\n\u003cdetails\u003e\n  \u003csummary\u003eUpdate Configs\u003c/summary\u003e\n\n```bash\n$ cstore push # all configs\n```\n```bash\n$ cstore push service/dev/.env service/qa/.env \n```\n```bash\n$ cstore push -t \"dev\u0026qa\" # config must have both tags\n```\n```bash\n$ cstore push -t \"dev|qa\" # config must have either tag\n```\n\n\u003c/details\u003e\n\n## Restore App Configuration ##\n\nDuring a pull, `-i` will retrieve and inject [tokenized](docs/SECRETS.md) secrets from AWS Secrets Manager.\n\n\u003cdetails open=\"true\"\u003e\n  \u003csummary\u003eRestore Config Files Locally\u003c/summary\u003e\n\n```bash\n$ cstore pull # all configs\n```\n```bash\n$ cstore pull service/dev/.env service/qa/.env \n```\n```bash\n$ cstore pull -t \"dev\u0026qa\" # config must have both tags\n```\n```bash\n$ cstore pull -t \"dev|qa\" # config must have either tag\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eFormat/Send Configs to Stdout\u003c/summary\u003e\n\n```bash\n$ cstore pull -t dev -e # raw file contents\n```\n```bash\n$ cstore pull service/dev/.env -g json-object # JSON object format\n```\n```bash\n$ eval $( cstore pull service/dev/.env -g terminal-export ) # export environment variables\n```\n\nOutput Task Definition JSON [Env](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definition_environment)/[Secrets](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) Formats (`.env`)\n```bash\n$ cstore pull -t dev -g task-def-env # AWS Task Definition environment\n```\n```bash\n$ cstore pull -t dev -g task-def-secrets --store-command refs # AWS Task Definition secrets\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eLoading Configs in a Service\u003c/summary\u003e\n\n* [Docker Container](docs/DOCKER.md)\n* [Lambda Function](docs/LAMBDA.md)\n* [Application Memory](docs/LIBRARY.md)\n\n\u003c/details\u003e\n\n## More ##\n\n\u003cdetails\u003e\n  \u003csummary\u003eLearning Basics\u003c/summary\u003e\n\n* [Terminology](docs/TERMS.md)\n* [Storage Solutions](docs/STORES.md)\n* [Vault Solutions](docs/VAULTS.md)\n* [Catalog Fields](docs/CATALOG.md)\n\n| Demo |  |\n|---|---|\n| [watch](https://youtu.be/vpNii5Y0yNg) | Get Configs With Secrets Injected |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eUseful Options\u003c/summary\u003e\n\n* [Tagging Files](docs/TAGGING.md)\n* [Storing/Injecting Secrets](docs/SECRETS.md)\n* [Versioning Files](docs/VERSIONING.md)\n* [Linking Catalogs](docs/LINKING.md)\n* [CLI Commands and Flags](docs/CLI.md)\n* [S3 Bucket Store Terraform](docs/S3.md)\n* [Ghost Files (.cstore)](docs/GHOST.md)\n* [Terraform State Files](docs/TERRAFORM.md)\n* [Migrate from v1 to v3+](docs/MIGRATE.md) (breaking changes)\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eProject Details\u003c/summary\u003e\n\n* [Goals](docs/GOALS.md)\n* [Integration Testing](docs/TESTING.md)\n* [Publish Release](docs/PUBLISH.md)\n\u003c/details\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturnerlabs%2Fcstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fturnerlabs%2Fcstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturnerlabs%2Fcstore/lists"}