{"id":15873744,"url":"https://github.com/mkeeler/consul-data","last_synced_at":"2025-07-14T02:37:54.954Z","repository":{"id":57584415,"uuid":"329079167","full_name":"mkeeler/consul-data","owner":"mkeeler","description":"Helpers to generate randomized data to shove into Consul for testing purposes.","archived":false,"fork":false,"pushed_at":"2022-09-21T19:45:15.000Z","size":61,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-01T13:11:23.443Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkeeler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-12T18:39:36.000Z","updated_at":"2022-09-29T16:43:26.000Z","dependencies_parsed_at":"2022-09-10T06:51:00.994Z","dependency_job_id":null,"html_url":"https://github.com/mkeeler/consul-data","commit_stats":null,"previous_names":["mkeeler/consul-data-gen"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mkeeler/consul-data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkeeler%2Fconsul-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkeeler%2Fconsul-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkeeler%2Fconsul-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkeeler%2Fconsul-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkeeler","download_url":"https://codeload.github.com/mkeeler/consul-data/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkeeler%2Fconsul-data/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265233753,"owners_count":23731825,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T01:05:53.940Z","updated_at":"2025-07-14T02:37:54.919Z","avatar_url":"https://github.com/mkeeler.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# consul-data\nThis repo contains a library for generating data to push into Consul, a command line client for outputting that generated data and some terraform code to use that data and push it to Consul.\n\n## Library\n\nThe `generate` subdirectory contains the top level package for all data generators. Subpackages are for generating a specific type of data.\n\n## CLI\n\n### Installation\n\n`go get github.com/mkeeler/consul-data/cmd/consul-data`\n\n### Usage (Top Level)\n\n```\nUsage: consul-data [--version] [--help] \u003ccommand\u003e [\u003cargs\u003e]\n\nAvailable commands are:\n    generate    Generate data for Consul\n    push        Pushes generated data to consul\n``` \n\n### Usage (Data Generation)\n\n```\nUsage: consul-data generate [OPTIONS] [output path]\n\n        Generate random data for consul.\n\n        By default the generated output is sent to the console but\n        an optional output path may be used to cause it to be written\n        to a file\n\nCommand Options\n\n  -config=\u003cstring\u003e\n     Path to the configuration to use for generating data\n\n  -seed=\u003cint\u003e\n     Value to use to seed the pseudo-random number generator with\n     instead of the current time\n```\n\n### Usage (Data Push)\n\n```\nUsage: consul-data push [OPTIONS]\n\n        Push data to Consul\n\n        By default this command will generate the random data and push it\n        to Consul. The data can be pregenerated if the -data flag is used\n        to specify a file which should be in the format outputted by the\n        consul-data generate command\n\nCommand Options\n\n  -ca-file=\u003cvalue\u003e\n     Path to a CA file to use for TLS when communicating with Consul.\n     This can also be specified via the CONSUL_CACERT environment\n     variable.\n\n  -ca-path=\u003cvalue\u003e\n     Path to a directory of CA certificates to use for TLS when\n     communicating with Consul. This can also be specified via the\n     CONSUL_CAPATH environment variable.\n\n  -client-cert=\u003cvalue\u003e\n     Path to a client cert file to use for TLS when 'verify_incoming'\n     is enabled. This can also be specified via the CONSUL_CLIENT_CERT\n     environment variable.\n\n  -client-key=\u003cvalue\u003e\n     Path to a client key file to use for TLS when 'verify_incoming'\n     is enabled. This can also be specified via the CONSUL_CLIENT_KEY\n     environment variable.\n\n  -config=\u003cstring\u003e\n     Path to the configuration to use for generating data\n\n  -data=\u003cstring\u003e\n     Path to data generated by consul-data generate to use as the data\n     source instead of generating new data\n\n  -datacenter=\u003cvalue\u003e\n     Name of the datacenter to query. If unspecified, this will default\n     to the datacenter of the queried agent.\n\n  -http-addr=\u003caddress\u003e\n     The `address` and port of the Consul HTTP agent. The value can be\n     an IP address or DNS address, but it must also include the port.\n     This can also be specified via the CONSUL_HTTP_ADDR environment\n     variable. The default value is http://127.0.0.1:8500. The scheme\n     can also be set to HTTPS by setting the environment variable\n     CONSUL_HTTP_SSL=true.\n\n  -namespace=\u003cdefault\u003e\n     Specifies the namespace to query. If not provided, the namespace\n     will be inferred from the request's ACL token, or will default\n     to the `default` namespace. Namespaces are a Consul Enterprise\n     feature.\n\n  -output=\u003cstring\u003e\n     Path to output the data file to if we generated it instead of\n     loading it in\n\n  -seed=\u003cint\u003e\n     Value to use to seed the pseudo-random number generator with\n     instead of the current time\n\n  -stale\n     Permit any Consul server (non-leader) to respond to this request.\n     This allows for lower latency and higher throughput, but can result\n     in stale data. This option has no effect on non-read operations.\n     The default value is false.\n\n  -tls-server-name=\u003cvalue\u003e\n     The server name to use as the SNI host when connecting via\n     TLS. This can also be specified via the CONSUL_TLS_SERVER_NAME\n     environment variable.\n\n  -token=\u003cvalue\u003e\n     ACL token to use in the request. This can also be specified via the\n     CONSUL_HTTP_TOKEN environment variable. If unspecified, the query\n     will default to the token of the Consul agent at the HTTP address.\n\n  -token-file=\u003cvalue\u003e\n     File containing the ACL token to use in the request instead of one\n     specified via the -token argument or CONSUL_HTTP_TOKEN environment\n     variable. This can also be specified via the CONSUL_HTTP_TOKEN_FILE\n     environment variable.\n```\n\n### Config Format\n\n```json\n{\n    \"KV\": {\n        \"NumEntries\": 500,\n        \"KeyType\": \"pet-name\",\n        \"ValueType\": \"random-b64\",\n        \n        \"PetName\": {\n            \"Prefix\": \"\",\n            \"Segments\": 3,\n            \"Separator\": \"-\",\n        },\n        \n        \"RandomB64\": {\n            \"MinSize\": 64,\n            \"MaxSize\": 128,\n        }\n    },\n    \"Catalog\": {\n        \"NumNodes\": 250,\n        \"MinServicesPerNode\": 4,\n        \"MaxServicesPerNode\": 8,\n        \"MinInstancesPerService\": 1,\n        \"MaxInstancesPerService\": 1,\n        \"MinMetaPerNode\": 2,\n        \"MaxMetaPerNode\": 16,\n        \"MinMetaPerService\": 5,\n        \"MaxMetaPerService\": 7,\n        \"NodeType\": \"pet-name\",\n        \"ServiceType\": \"pet-name\",\n        \"AddressType\": \"random-testing\",\n        \"MetaKeyType\": \"pet-name\",\n        \"MetaValueType\": \"random-b64\",\n        \n         \"NodePetNames\": {\n            \"Prefix\": \"\",\n            \"Segments\": 3,\n            \"Separator\": \"-\"\n        },\n        \n        \"ServicePetNames\": {\n            \"Prefix\": \"\",\n            \"Segments\": 3,\n            \"Separator\": \"-\"\n        },\n        \n        \"MetaKeyPetNames\": {\n            \"Prefix\": \"\",\n            \"Segments\": 3,\n            \"Separator\": \"-\"\n        },\n        \n        \"MetaValueRandomB64\": {\n            \"MinSize\": 64,\n            \"MaxSize\": 128\n        }\n    }\n}\n```\n\n## Terraform\n\nWe can use the `consul-data generate` subcommand to generate the data file required by the terraform provider. After that you are a simple `terraform apply` away from pushing all the data into Consul. Note that for very large\nnumbers of resources terraform has some bad performance issues and the `consul-data push` command will need to be used instead.\n\n### Variables\n\n* `consul_address` - **(Optional)** Allows you to override the Consul HTTP API addresses used by the Consul Terraform provider\n* `consul_token` - **(Optional)** Allows you to set a Consul ACL token for use by the Consul Terraform provider.\n* `data` - **(Required)** Path to the  data generated by the `consul-data generate` subcommand.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkeeler%2Fconsul-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkeeler%2Fconsul-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkeeler%2Fconsul-data/lists"}