https://github.com/fischerscode/my-k3s

The configuration and automatation of a k3s cluster.
https://github.com/fischerscode/my-k3s

Last synced: about 2 months ago
JSON representation

The configuration and automatation of a k3s cluster.

• Host: GitHub
• URL: https://github.com/fischerscode/my-k3s
• Owner: fischerscode
• License: other
• Created: 2021-08-04T19:39:02.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2024-06-28T20:08:18.000Z (3 months ago)
• Last Synced: 2024-06-29T20:56:09.710Z (3 months ago)
• Language: Jinja
• Size: 842 KB
• Stars: 2
• Watchers: 1
• Forks: 1
• Open Issues: 122
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
**THIS PROJECT IS STILL WORK IN PROGRESS**

# my-k3s

The configuration and automatation of a k3s cluster.

## Flux

Flux is used to automatically provision the manifests.

### Install Flux

MacOS: ```brew install fluxcd/tap/flux```

## New Cluster

1. Specify cluster name: `export CLUSTER=`

2. Setup ansible vault and store the vault password in `$CLUSTER-ansible.key`

3. Create inventory: `cp inventory-sample.yaml inventory-$CLUSTER.yaml`

4. Edit your inventory (Generate vault entries using `pbpaste | ansible-vault encrypt_string --vault-password-file $CLUSTER-ansible.key --name k3sToken`. `--name` has to be the name of the encrypted key.)

5. Copy cluster manifests: `cp -r clusters/sample clusters/$CLUSTER`

6. Edit `clusters/$CLUSTER/infrastructure.yaml` manifest

7. Copy infrastructure: `cp -r infrastructure/sample infrastructure/$CLUSTER`

8. Generate files: `ansible-playbook -i inventory-$CLUSTER.yaml tools/generate_files.yml --extra-vars=cluster_name=$CLUSTER`

9.  [Setup SOPS](#mozilla-sops)

10. Create secrets: `./generate-secrets.sh`

11. Store known hosts: `ansible-playbook -i inventory-$CLUSTER.yaml tools/store_known_hosts.yml`

12. Install k3s (wait until it hangs at 'Enable and check K3s service'): `ansible-playbook -i inventory-$CLUSTER.yaml main.yml --extra-vars=cluster_name=$CLUSTER --vault-password-file $CLUSTER-ansible.key`

13. Get access to the cluster:

    1.   New terminal and `export CLUSTER=` again.

    2.  Get kubeconfig: `ansible-playbook -i inventory-$CLUSTER.yaml tools/get_kubeconfig.yml --extra-vars=cluster_name=$CLUSTER`

    3.  Tunnel api server: `ssh -L 6443:10.1.0.1:6443 IP_OF_A_MASTER`

    4.  New terminal and `export CLUSTER=` again.

    5.  Replace IP at server in `kubeconfig-$CLUSTER.yaml` with `127.0.0.1`.

    6.  Use config: `KUBECONFIG=kubeconfig-$CLUSTER.yaml`

14. Setup Flux:

    1. Make sure GITHUB_TOKEN is set. Run `export GITHUB_TOKEN=$(pbpaste)` with a personal access token (everything in repo is enabled) in your clipboard.

    2. Setup flux: `flux bootstrap github --owner=fischerscode --repository=my-k3s --path=clusters/$CLUSTER --branch master --personal`

15. Playbook should finish now. If not check `flux get all`.

16. Replace IP at server in `kubeconfig-$CLUSTER.yaml` with `kubernetes_api_public_address`.

   

### Afterwards:

1. Add cluster to `.github/workflows/update-flux.yaml`

2. get grafana admin password: `kubectl get secret -n monitoring grafana-cred --template={{.data.ADMIN_PASSWORD}} | base64 -d | pbcopy`

## Mozilla SOPS

1. `brew install gnupg sops`

2. Generate a GPG/OpenPGP key with no passphrase (%no-protection):

   ```

   export KEY_NAME="$CLUSTER.my-k3s.fischerscode.com"

   export KEY_COMMENT="flux secrets"

   gpg --batch --full-generate-key < $CLUSTER.key` (Ansible will search for this key and apply it as a secret if present.)

   2. To clipboard: `gpg --export-secret-keys --armor ${KEY_NAME} | pbcopy`

6. Add public key to git: `gpg --export --armor ${KEY_NAME} > ./clusters/$CLUSTER/.sops.pub.asc`

7. ```

   cat <> .sops.yaml

     - path_regex: /$CLUSTER\/.*\.yaml$

       encrypted_regex: ^(data|stringData)$

       pgp: ${KEY_FP}

     - path_regex: /$CLUSTER\/.*\.encrypted$

       pgp: ${KEY_FP}

   EOF

   ```

### Optional:

9.  Remove private key: `gpg --delete-secret-keys ${KEY_NAME}`

10. Import secret key: `gpg --import $CLUSTER.key`