https://github.com/kvz/riak-formation
Create/restore a riak cluster on any cloudprovider using Terraform
https://github.com/kvz/riak-formation
Last synced: 4 months ago
JSON representation
Create/restore a riak cluster on any cloudprovider using Terraform
- Host: GitHub
- URL: https://github.com/kvz/riak-formation
- Owner: kvz
- License: mit
- Created: 2014-08-12T12:10:15.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2015-08-21T16:48:49.000Z (almost 10 years ago)
- Last Synced: 2025-02-17T19:52:38.445Z (4 months ago)
- Language: Shell
- Size: 307 KB
- Stars: 3
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# riak-formation
Create/restore a riak cluster on any cloudprovider using Terraform
## What
- [Riak](http://basho.com/riak/) is a fine distributed key-value store built by [Basho](http://basho.com/)
- [Riak CS](http://basho.com/riak-cloud-storage/) is an addon so that you can build your own S3 compatible storage service, with Riak as a backend.
- [Terraform](http://www.terraform.io/) is a tool to launch clusters in the cloud, using nothing but simple configuration files.
**[riak-formation](https://github.com/kvz/riak-formation)** is a collection of config files and scripts that let you create/restore a Riak (CS) cluster in the cloud using a single command.
## Why
Launching a Riak cluster can be involved. Hence Basho has
[released a few scripts](https://github.com/basho/cloudformation-riak) for
[AWS CloudFormation](http://aws.amazon.com/cloudformation/)
to automate this.
While these scripts are more advanced than what **riak-formation** is aiming for - they tie you in with Amazon's EC2. The Terraform approach also works with other vendors such as e.g. Digital Ocean. You could even mix multiple providers to some extent.
Terraform works idempotent, meaning if it encounters servers that diverged from our config, we can just run it again and it will make all the required changes to restore our cluster as it was defined.
Terraform config uses the [HLC](https://github.com/hashicorp/hcl) configuration language and is more [dense](https://github.com/kvz/riak-formation/blob/master/scripts/riak.tf) than the `48.189 kb` CloudFormation JSON files, so it's easy to spot mistakes, and more fun to work on.
## How
First, **riak-formation** needs to know which cloud provider you wish to target, and the associated account. You can pass it these via cluster config. Either directly on the commandline, or add them to `clusters/production/config.sh`. `production` can be any name, you you could be deploying many completely different riak clusters from this project.
Since `config.sh` supports many configuration options, it is not required to define your own cluster's infrastructure definitions (`*.tf`), it instead will automatically borrow `default.tf` from the `example` cluster to reduce duplication. SSH keys, SSL keys, `config.sh`, state files, and plan files, do have to be administerd on a cluster-by-cluster basis.
Now type `make launch`. riak-formation will launch as many machines as you defined, set up firewalls, install riak nodes, connect them together, set up Riak Control, and end with this summary:
Click any of the links, and you're right inside the control panel:
## Todo
- [ ] Make env file dictate ingress, ports, server count, etc. So that you can launch different Riak clusters by just sourcing different env files.
- [ ] S3 emulation
- [ ] Can we serve read-only json files
- [ ] Multi datacenter replication? Can we do that for free somehow?
- [ ] Enable Search
- [ ] Research/Document CRDTs in 2.0
- [ ] Backup / Restore
- [x] Simplify directory layout
- [ ] Store assembly as MAP:
> "Maps are the richest of the Riak Data Types because within the fields of a map you can nest any of the five Data Types, including maps themselves (you can even embed maps within maps, and maps within those maps, and so on). " - http://docs.basho.com/riak/latest/theory/concepts/crdts/
## Credits
This project draws from
- [Installing on Debian and Ubuntu](http://docs.basho.com/riak/latest/ops/building/installing/debian-ubuntu/#Installing-From-Apt-Get)
- [How To Create a Riak Cluster on an Ubuntu VPS](https://www.digitalocean.com/community/tutorials/how-to-create-a-riak-cluster-on-an-ubuntu-vps)
- [Call Me Maybe: Carly Rae Jepsen and the Perils of Network Partitions - RICON East 2013](https://www.youtube.com/watch?v=mxdpqr-loyA)
- [Jepsen: ZK, NuoDB, Kafka, & Cassandra](https://www.youtube.com/watch?v=NsI51Mo6r3o)
- [Kyle Kingsbury and Al Tobey - Cassandra and Go Doubleheader](https://www.youtube.com/watch?v=oEFqxi_n1vU)
- [Readings in conflict-free replicated data types](http://christophermeiklejohn.com/crdt/2014/07/22/readings-in-crdts.html)
- [A comprehensive study of Convergent and Commutative Replicated Data Types](http://hal.upmc.fr/file/index/docid/555588/filename/techreport.pdf)