Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mkeeler/consul-data
Helpers to generate randomized data to shove into Consul for testing purposes.
https://github.com/mkeeler/consul-data
Last synced: 19 days ago
JSON representation
Helpers to generate randomized data to shove into Consul for testing purposes.
- Host: GitHub
- URL: https://github.com/mkeeler/consul-data
- Owner: mkeeler
- License: mpl-2.0
- Created: 2021-01-12T18:39:36.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-09-21T19:45:15.000Z (about 2 years ago)
- Last Synced: 2024-10-13T01:20:49.839Z (about 1 month ago)
- Language: Go
- Size: 59.6 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# consul-data
This repo contains a library for generating data to push into Consul, a command line client for outputting that generated data and some terraform code to use that data and push it to Consul.
## Library
The `generate` subdirectory contains the top level package for all data generators. Subpackages are for generating a specific type of data.
## CLI
### Installation
`go get github.com/mkeeler/consul-data/cmd/consul-data`
### Usage (Top Level)
```
Usage: consul-data [--version] [--help] []
Available commands are:
generate Generate data for Consul
push Pushes generated data to consul
```
### Usage (Data Generation)
```
Usage: consul-data generate [OPTIONS] [output path]
Generate random data for consul.
By default the generated output is sent to the console but
an optional output path may be used to cause it to be written
to a file
Command Options
-config=
Path to the configuration to use for generating data
-seed=
Value to use to seed the pseudo-random number generator with
instead of the current time
```
### Usage (Data Push)
```
Usage: consul-data push [OPTIONS]
Push data to Consul
By default this command will generate the random data and push it
to Consul. The data can be pregenerated if the -data flag is used
to specify a file which should be in the format outputted by the
consul-data generate command
Command Options
-ca-file=
Path to a CA file to use for TLS when communicating with Consul.
This can also be specified via the CONSUL_CACERT environment
variable.
-ca-path=
Path to a directory of CA certificates to use for TLS when
communicating with Consul. This can also be specified via the
CONSUL_CAPATH environment variable.
-client-cert=
Path to a client cert file to use for TLS when 'verify_incoming'
is enabled. This can also be specified via the CONSUL_CLIENT_CERT
environment variable.
-client-key=
Path to a client key file to use for TLS when 'verify_incoming'
is enabled. This can also be specified via the CONSUL_CLIENT_KEY
environment variable.
-config=
Path to the configuration to use for generating data
-data=
Path to data generated by consul-data generate to use as the data
source instead of generating new data
-datacenter=
Name of the datacenter to query. If unspecified, this will default
to the datacenter of the queried agent.
-http-addr=
The `address` and port of the Consul HTTP agent. The value can be
an IP address or DNS address, but it must also include the port.
This can also be specified via the CONSUL_HTTP_ADDR environment
variable. The default value is http://127.0.0.1:8500. The scheme
can also be set to HTTPS by setting the environment variable
CONSUL_HTTP_SSL=true.
-namespace=
Specifies the namespace to query. If not provided, the namespace
will be inferred from the request's ACL token, or will default
to the `default` namespace. Namespaces are a Consul Enterprise
feature.
-output=
Path to output the data file to if we generated it instead of
loading it in
-seed=
Value to use to seed the pseudo-random number generator with
instead of the current time
-stale
Permit any Consul server (non-leader) to respond to this request.
This allows for lower latency and higher throughput, but can result
in stale data. This option has no effect on non-read operations.
The default value is false.
-tls-server-name=
The server name to use as the SNI host when connecting via
TLS. This can also be specified via the CONSUL_TLS_SERVER_NAME
environment variable.
-token=
ACL token to use in the request. This can also be specified via the
CONSUL_HTTP_TOKEN environment variable. If unspecified, the query
will default to the token of the Consul agent at the HTTP address.
-token-file=
File containing the ACL token to use in the request instead of one
specified via the -token argument or CONSUL_HTTP_TOKEN environment
variable. This can also be specified via the CONSUL_HTTP_TOKEN_FILE
environment variable.
```
### Config Format
```json
{
"KV": {
"NumEntries": 500,
"KeyType": "pet-name",
"ValueType": "random-b64",
"PetName": {
"Prefix": "",
"Segments": 3,
"Separator": "-",
},
"RandomB64": {
"MinSize": 64,
"MaxSize": 128,
}
},
"Catalog": {
"NumNodes": 250,
"MinServicesPerNode": 4,
"MaxServicesPerNode": 8,
"MinInstancesPerService": 1,
"MaxInstancesPerService": 1,
"MinMetaPerNode": 2,
"MaxMetaPerNode": 16,
"MinMetaPerService": 5,
"MaxMetaPerService": 7,
"NodeType": "pet-name",
"ServiceType": "pet-name",
"AddressType": "random-testing",
"MetaKeyType": "pet-name",
"MetaValueType": "random-b64",
"NodePetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"ServicePetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"MetaKeyPetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"MetaValueRandomB64": {
"MinSize": 64,
"MaxSize": 128
}
}
}
```
## Terraform
We can use the `consul-data generate` subcommand to generate the data file required by the terraform provider. After that you are a simple `terraform apply` away from pushing all the data into Consul. Note that for very large
numbers of resources terraform has some bad performance issues and the `consul-data push` command will need to be used instead.
### Variables
* `consul_address` - **(Optional)** Allows you to override the Consul HTTP API addresses used by the Consul Terraform provider
* `consul_token` - **(Optional)** Allows you to set a Consul ACL token for use by the Consul Terraform provider.
* `data` - **(Required)** Path to the data generated by the `consul-data generate` subcommand.