{"id":17132240,"url":"https://github.com/bitsofinfo/files-to-consul-kv","last_synced_at":"2025-04-13T07:56:03.825Z","repository":{"id":47670645,"uuid":"240073750","full_name":"bitsofinfo/files-to-consul-kv","owner":"bitsofinfo","description":"Simple script to push a filesystem of of consul kv files to consul via the transaction api","archived":false,"fork":false,"pushed_at":"2021-08-18T18:03:36.000Z","size":38,"stargazers_count":19,"open_issues_count":1,"forks_count":8,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-13T07:55:59.048Z","etag":null,"topics":["consul","devops","hashicorp-consul"],"latest_commit_sha":null,"homepage":"https://bitsofinfo.wordpress.com/2020/02/20/git2consul-alternatives-sync-consul-kv-file-filesystem-git/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitsofinfo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-12T17:32:20.000Z","updated_at":"2025-01-13T13:35:32.000Z","dependencies_parsed_at":"2022-08-21T14:40:15.827Z","dependency_job_id":null,"html_url":"https://github.com/bitsofinfo/files-to-consul-kv","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ffiles-to-consul-kv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ffiles-to-consul-kv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ffiles-to-consul-kv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ffiles-to-consul-kv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitsofinfo","download_url":"https://codeload.github.com/bitsofinfo/files-to-consul-kv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248681490,"owners_count":21144700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consul","devops","hashicorp-consul"],"created_at":"2024-10-14T19:26:28.046Z","updated_at":"2025-04-13T07:56:03.802Z","avatar_url":"https://github.com/bitsofinfo.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# files-to-consul-kv\n\nSimple utility for bulk loading sets of [Consul key-value](https://www.consul.io/docs/agent/kv.html) entries via the [transactions API](https://www.consul.io/api/txn.html) where the source of those values exist on disk in a directory structure. \n\nFor example given a simple directory structure like:\n```\ncd mykvs/\n\n$ find . -print\n.\n./sub\n./sub/key2\n./key1\n\n$ cat key1 \nval1\n\n$ cat sub/key2 \nval2\n```\n\nYou could use `fs2consulkv.py` to set all these in Consul under some root path:\n```\n ./fs2consulkv.py \\\n    --fs-kv-path ./mykvs \\\n    --consul-url https://[consul-fqdn][:port] \\\n    --consul-acl-token xxxxxxx \\\n    --consul-data-center optional-dc \\\n    --consul-kv-root some/root/path/\n ```\n\nWould result in your KVs in consul at:\n```\nhttps://[consul-fqdn][:port]/ui/mydc/kv/some/root/path/key1 = val1\nhttps://[consul-fqdn][:port]/ui/mydc/kv/some/root/path/sub/key2 = val2\n```\n\n## Docker\n\nRun via Docker:\nhttps://hub.docker.com/r/bitsofinfo/files-to-consul-kv\n\n```\ndocker run -i -v `pwd`/mykvs:/kvsource \\\n   bitsofinfo/files-to-consul-kv fs2consulkv.py \\\n   --fs-kv-path /kvsource \\\n   --consul-url https://[consul-fqdn][:port] \\\n   --consul-acl-token xxxxxxx \\\n   --consul-data-center optional-dc \\\n   --consul-kv-root some/root/path/\n```\n\n## Usage\n\n```\n$ ./fs2consulkv.py  -h\n\nusage: fs2consulkv.py [-h] [-p FS_KV_PATH] [-k CONSUL_KV_ROOT]\n                      [-z CONSUL_KV_ROOT_FILE] [-c CONSUL_URL]\n                      [-t CONSUL_ACL_TOKEN] [-f CONSUL_ACL_TOKEN_FILE]\n                      [-d CONSUL_DATA_CENTER] [-x] [-n] [-l LOG_LEVEL]\n                      [-b LOG_FILE]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -p FS_KV_PATH, --fs-kv-path FS_KV_PATH\n                        Full or relative path to filesystem directory\n                        containing the KV structure to send to consul\n                        (default: ./)\n  -k CONSUL_KV_ROOT, --consul-kv-root CONSUL_KV_ROOT\n                        Root path in Consul KV by which all new keys will be\n                        set, required. i.e. 'some/root/path' (default: None)\n  -z CONSUL_KV_ROOT_FILE, --consul-kv-root-file CONSUL_KV_ROOT_FILE\n                        Path to a file that contains the consul-kv-root\n                        argument value, optional, can be used instead of\n                        --consul-kv-root i.e. /path/to/consul-kv-root.txt\n                        where the file contents contains the value\n                        'some/root/path' (default: None)\n  -c CONSUL_URL, --consul-url CONSUL_URL\n                        Consul url, required. i.e. http[s]://[fqdn][:port]\n                        (default: None)\n  -t CONSUL_ACL_TOKEN, --consul-acl-token CONSUL_ACL_TOKEN\n                        Consul acl token, required (default: None)\n  -f CONSUL_ACL_TOKEN_FILE, --consul-acl-token-file CONSUL_ACL_TOKEN_FILE\n                        Consul acl token file, path to a file that contains\n                        the token value, required (default: None)\n  -d CONSUL_DATA_CENTER, --consul-data-center CONSUL_DATA_CENTER\n                        Consul data-center, optional. (default: None)\n  -x, --skip-prompt     Skip confirmation and prompt (default: False)\n  -n, --retain-trailing-newlines\n                        Retain trailing newline chars (\\n) in values files and\n                        do not strip them. Default behavior is to strip them\n                        (default: False)\n  -s SLEEP_DELAY, --sleep-delay SLEEP_DELAY\n                        Delay [in seconds] in kv upload loop, to avoid\n                        overwhelming the consul server. Default behavior is\n                        0.000 seconds (default: 0)\n  -u CHUNK_SIZE, --chunk-size CHUNK_SIZE\n                        Number of KV pairs uploaded at once. Default is 64,\n                        the maximum allowed. (default: 64)\n  -l LOG_LEVEL, --log-level LOG_LEVEL\n                        log level, DEBUG, INFO, etc (default: DEBUG)\n  -b LOG_FILE, --log-file LOG_FILE\n                        Path to log file; default None = STDOUT (default:\n                        None)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsofinfo%2Ffiles-to-consul-kv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitsofinfo%2Ffiles-to-consul-kv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsofinfo%2Ffiles-to-consul-kv/lists"}