Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/letsencrypt/attache

A sidecar that allows for effortless scaling of a Redis Cluster
https://github.com/letsencrypt/attache

Last synced: 2 months ago
JSON representation

A sidecar that allows for effortless scaling of a Redis Cluster

Awesome Lists containing this project

README 

        
# Attaché

A sidecar that allows for effortless scaling of Redis Clusters using Hashicorp

Nomad and Consul.



#### Features

- Create a new cluster when no cluster is present

- Add new primary node and perform a shard slot rebalance

- Add new replica node to the primary node with the least replicas

- Full support for Redis mTLS and ACL Auth

- Full support for Consul mTLS and ACL Tokens



#### To Do

- [x] Redis ACL

- [x] Redis Password

- [x] Redis mTLS

- [ ] Drain, failover, and FORGET an existing primary node

- [ ] Remove and FORGET an existing replica node



### `attache-check`

A sidecar that servers an HTTP API that allows Consul to track the health of

Redis Cluster Nodes, route new nodes to the Await (introduction) Consul Service

for their Redis Cluster, then migrate them to the Destination Consul Service

once they've joined a cluster.



#### Usage

```shell

$ attache-check -help

Usage of attache-check:

  -check-serv-addr string

    	address this utility should listen on (e.g. 127.0.0.1:8080)

  -redis-auth-password-file string

    	redis-server password file path, (required)

  -redis-auth-username string

    	redis-server username, (required)

  -redis-node-addr string

    	redis-server listening address, (required)

  -redis-tls-ca-cert string

    	Redis client CA certificate file, (required)

  -redis-tls-cert-file string

    	Redis client certificate file, (required)

  -redis-tls-key-file string

    	Redis client key file, (required)

  -shutdown-grace duration

    	duration to wait before shutting down (e.g. '1s') (default 5s)

```



### `attache-control`

An ephemeral sidecar that acts as an agent for each Redis node when it's

started. If a node's `node info` reflects that of a new node, this agent will

attempt to introduce it to an existing Redis Cluster, if it exists, else it will

attempt to orchestrate the create a new Redis Cluster if there are enough new

Redis nodes (in the Await Consul Service) to do so.



#### Usage

```shell

$ ./attache-control -help

Usage of ./attache-control:

  -attempt-interval duration

    	Duration to wait between attempts to join or create a cluster (e.g. '1s') (default 3s)

  -await-service-name string

    	Consul Service for newly created Redis Cluster Nodes, (required)

  -consul-acl-token string

    	Consul client ACL token

  -consul-addr string

    	Consul client address (default "127.0.0.1:8501")

  -consul-dc string

    	Consul client datacenter (default "dev-general")

  -consul-tls-ca-cert string, (required)

    	Consul client CA certificate file

  -consul-tls-cert string, (required)

    	Consul client certificate file

  -consul-tls-key string, (required)

    	Consul client key file

  -dest-service-name string

    	Consul Service for healthy Redis Cluster Nodes, (required)

  -lock-kv-path string

    	Consul KV path to use as a leader lock for Redis Cluster operations (default "service/attache/leader")

  -log-level string

    	Set the log level (default "info")

  -redis-auth-password-file string

    	Redis password file path, (required)

  -redis-auth-username string

    	Redis username, (required)

  -redis-node-addr string

    	redis-server listening address, (required)

  -redis-tls-ca-cert string

    	Redis client CA certificate file, (required)

  -redis-tls-cert-file string

    	Redis client certificate file, (required)

  -redis-tls-key-file string

    	Redis client key file, (required)

```



### Running the Example Nomad Job

Note: these steps assume that you have the `nomad`, `consul`, and `terraform`

binaries installed on your machine and that they exist in your `PATH`.



Build the attache-control and attache-check binaries:

```shell

$ go build -o attache-check ./cmd/attache-check/main.go && go build -o attache-control ./cmd/attache-control/main.go ./cmd/attache-control/config.go

```



In another shell, start the Consul server in `dev` mode:

```shell

$ consul agent -dev -config-format=hcl -config-file consul.conf.hcl

```



In another shell, start the Nomad server in `dev` mode:

```shell

$ sudo nomad agent -dev -config nomad.conf.hcl

```



Start a Nomad job deployment using Terraform:

```shell

cd example

terraform init

terraform plan

terraform apply

```



Open the Nomad UI: http://localhost:4646/ui to view information about the Redis

Cluster deployment



Open the Consul UI: http://localhost:8501/ui to view health check information

for the Redis Cluster



### Useful Commands



#### Purge Nomad Job

This is useful for stopping and garbage collecting a job in Nomad immediately.

```shell

nomad job stop -purge ""

```



#### Count Primary Redis Nodes

```shell

redis-cli -p  --tls --cert ./example/tls/redis/cert.pem --key ./example/tls/redis/key.pem --cacert ./example/tls/ca-cert.pem --user replication-user --pass  cluster nodes | grep master | wc -l

```



#### Count Replica Redis Nodes

```shell

redis-cli -p  --tls --cert ./example/tls/redis/cert.pem --key ./example/tls/redis/key.pem --cacert ./example/tls/ca-cert.pem --user replication-user --pass  cluster nodes | grep slave | wc -l

```



#### Create New Consul CA-Cert and Key

```shell

consul tls ca create

```



#### Create and Sign New Consul Server Cert and Key

In the same directory as the CA Cert:

```shell

consul tls cert create -client -dc "dev-general"

```



#### Create and Sign New Consul Client Cert and Key

In the same directory as the CA Cert:

```shell

consul tls cert create -server -dc "dev-general"

```