Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/CGamesPlay/infra

selfhosted infra scripts
https://github.com/CGamesPlay/infra

Last synced: 9 days ago
JSON representation

selfhosted infra scripts

Awesome Lists containing this project

README 

        
# Personal Cloud IaC



This is the repo I use for [my personal cloud server](https://cgamesplay.com/post/2021/10/27/creating-my-personal-cloud-with-hashicorp/), hosted as a VPS. At the time of writing this document, the system is a single node, therefore the security model is somewhat simpler than a production system and no container storage interfaces need to be configured.



**Features:**



- Single-node [Nomad](https://www.nomadproject.io), [Consul](https://www.consul.io), and [Vault](https://www.vaultproject.io) installation.

- mTLS configured between all services.

- [WireGuard](https://www.wireguard.com) used to secure communication between nodes (theoretically, since there's only one node).

- Access to GUI management portals via WireGuard (presently, the real use of WireGuard).

- A [variety of Nomad jobs](./nomad) that I've developed. Some highlights:

  - [backup](./nomad/backup) - back up the system to S3 using [Restic](https://restic.net) on a periodic basis.

  - [registry](./nomad/registry) - host a private [Docker](https://www.docker.com/) registry, which can be referenced by other Nomad jobs.

  - [traefik](./nomad/traefik) - expose Nomad jobs to the internet using [Traefik](https://traefik.io/traefik/) and [LetsEncrypt](https://letsencrypt.org).

  - See the full list [here](./nomad).



## System components



**Terraform**



[Terraform](https://www.terraform.io) is used to describe the very simple infrastructure requirements for the cluster. This is primarily intended to be a base for future improvements, if the cluster ever needs to move to a multi-node setup.



**Ansible**



[Ansible](https://www.ansible.com) is used to update all configuration files on all nodes. This includes the configuration for Vault, Vault Agent, Consul, Nomad, and WireGuard.



**Nomad**



The master node runs the Nomad server, and all other nodes run Nomad clients. Nomad is responsible for running Traefik and all of the actual workloads. Nomad needs a way to know which machines are Nomad clients, and what workloads they are running, for which it uses Consul.



**Consul**



Consul is used to store configuration and state information about the cluster. Each Nomad workload will register as a service in Consul, which in turn can be used to resolve the IP addresses and port information to reach those services from anywhere in the cluster.



**Vault**



Vault is used to store secrets and issue internal TLS certificates. It is not directly required by Nomad, but Nomad does have a tight Vault integration to allow workloads to securely receive secrets. Vault stores its data in Consul (:warning: in a real production system we would want to use a separate Consul cluster specifically to store Vault data). Vault Agent is a component of Vault which is used to update configuration files which contain secret data, and is used to rotate TLS certificates as well as to manage the encryption keys and tokens used by the other services.



**WireGuard**



WireGuard is used to secure communications between cluster nodes. This allows us to securely keep a private network even between multiple regions and cloud providers.



## Installation



1. Configure your environment. The requirements:

   - `HCLOUD_TOKEN` is the Hetzner Cloud token.

   - `ssh-add -L` needs to show at least one key (it will be used as the key for the created instances).

   - `python --version` needs to be 3. `apt install python-is-python3` on Ubuntu.

   - `pip3 install ansible hvac ansible-modules-hashivault`

2. Run `argc infra apply` to sync the infrastructure. This command requires confirmation before continuing, but you can also use `plan` or any other Terraform arguments.

3. Run `argc ansible` to apply the configuration. The change detection does not work correctly on the first run, so `-CD` cannot be used here. They will work after a run has completed at least once.

   - The Vault creation will drop `vault.txt` in the repository root, which contains the Vault unseal keys and root token. Store these safely and delete the file.

   - Optionally, `argc verify` can be used to diagnose some basic issues now and in the future.

4. Connect to the machine using ssh (use `argc master_ip` for the IP address) and follow the [WireGuard docs](./docs/wireguard.md) to set up the initial peer.

5. Deploy jobs with Nomad. Use `argc nomad apply`.

   - This will apply the terraform workspace in the `nomad` directory.



### Local environment setup



To access the cluster from your local machine:



1. Install the generated CA at `bootstrap/data/ca.crt` to configure SSL. You can install it using Keychain Access.app into the login keychain, but you will need to manually trust the certificate, which is done from the certificate info window under "Trust".

  - This also enables UI access in the browser: [Consul](https://172.30.0.1:8501/) | [Vault](https://172.30.0.1:8200/) | [Nomad](https://172.30.0.1:4646/)

2. Configure WireGuard and connect it.

3. Set up local DNS to use Consul.

4. Use `eval $(argc env)` to get the necessary environment variables.



### Accessing Consul DNS over Wireguard



The generated WireGuard configuration does not specify DNS servers for the tunnel. If you want to resolve `service.consul` addresses through the tunnel, you need to either route all DNS through the tunnel, or configure your machine to only route the desired queries through the tunnel.



**macOS**



You can configure your macOS system DNS to use the tunnel for the `.consul` TLD only using this snippet. [This StackExchange answer](https://apple.stackexchange.com/a/385218/14873) has more information and debugging tips.



```bash

sudo scutil <