Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/giantswarm/aws-operator

Manages Kubernetes clusters running on AWS (before Cluster API)
https://github.com/giantswarm/aws-operator

aws kubernetes operator

Last synced: about 1 month ago
JSON representation

Manages Kubernetes clusters running on AWS (before Cluster API)

Lists

README 

        
[![CircleCI](https://dl.circleci.com/status-badge/img/gh/giantswarm/aws-operator/tree/master.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/giantswarm/aws-operator/tree/master)



# aws-operator



The aws-operator manages Kubernetes clusters running on AWS.



## Branches



- `thiccc`

    - Up to and including version v5.4.0.

    - Contains all versions of legacy controllers (reconciling AWSConfig CRs) up

      to and including v5.4.0.

- `legacy`

    - From version v5.5.0 up to and including v5.x.x.

    - Contains only the latest version of legacy controllers (reconciling

      AWSConfig CRs).

- `master`

    - From version v6.0.0.

    - Contains only the latest version of controllers (reconciling cluster API

      objects).



## Getting the Project



Download the latest release:

https://github.com/giantswarm/aws-operator/releases/latest



Clone the git repository: https://github.com/giantswarm/aws-operator.git



Download the latest docker image from here:

https://quay.io/repository/giantswarm/aws-operator



### How to build



Build the standard way.



```

go build github.com/giantswarm/aws-operator

```



## Architecture



The operator uses our [operatorkit][1] framework. It manages an `awsconfig`

CRD using a generated client stored in our [apiextensions][2] repo. Releases

are versioned using [version bundles][3].



The operator provisions guest Kubernetes clusters running on AWS. It runs in a

host Kubernetes cluster also running on AWS.



[1]:https://github.com/giantswarm/operatorkit

[2]:https://github.com/giantswarm/apiextensions

[3]:https://github.com/giantswarm/versionbundle



### CloudFormation



The guest Kubernetes clusters are provisioned using [AWS CloudFormation][4]. The

resources are split between CloudFormation stacks:



In control plane account

* tccpi - Tenant cluster control plane role setup.

* tccpf - Tenant cluster control plane routes setup.

* tcnpf - Tenant cluster nodepool peering.



In tenant account:

* tccp -  Tenant cluster network setup.

* tccpn - Tenant cluster control plane resources (masters).

* tcnp -  Tenant cluster nodepool resources (workers).



[4]:https://aws.amazon.com/cloudformation



### Other AWS Resources



As well as the CloudFormation stacks we also provision a KMS key and S3 bucket

per cluster. This is to upload cloudconfigs for the cluster nodes. The

cloudconfigs contain TLS certificates which are encrypted using the KMS key.



### Kubernetes Resources



The operator also creates a Kubernetes namespace per guest cluster with a

service and endpoints. These are used by the host cluster to access the guest

cluster.



### Certificates



Authentication for the cluster components and end-users uses TLS certificates.

These are provisioned using [Hashicorp Vault][5] and are managed by our

[cert-operator][6].



[5]:https://www.vaultproject.io/

[6]:https://github.com/giantswarm/cert-operator



## Secret



Here the AWS IAM credentials have to be inserted.

```

service:

  aws:

    accesskey:

      id: 'TODO'

      secret: 'TODO'

```



Here the base64 representation of the data structure above has to be inserted.

```

apiVersion: v1

kind: Secret

metadata:

  name: aws-operator-secret

  namespace: giantswarm

type: Opaque

data:

  secret.yml: 'TODO'

```



To create the secret manually do this.

```

kubectl create -f ./path/to/secret.yml

```



We also need a key to hold the SSH public key



```

apiVersion: v1

kind: Secret

metadata:

  name: aws-operator-ssh-key-secret

  namespace: giantswarm

type: Opaque

data:

  id_rsa.pub: 'TODO'

```



### Node VM Images (AMIs)



This operator holds a static mapping of versions and regions to AMI IDs (VM image IDs, region specific)

used for tenant cluster nodes in `service/controller/key/ami.go`. The file is generated by

`devctl` and should not be edited manually. When a new version of the OS is released and new

images have been published on AWS, this mapping can be updating using

`devctl gen ami --dir service/controller/key`.



## Live editing operator inside an installation



- Download Okteto latest release from https://github.com/okteto/okteto/releases

- `okteto init -n giantswarm`

- Set correct label `app.giantswarm.io/branch: $BRANCH` in the manifest

- Change your kubeconfig to the giantswarm namespace

- Modify PSP of the current operator `kubectl patch psp aws-operator-$BRANCH-psp -p '{"spec":{"runAsGroup":{"ranges":null,"rule":"RunAsAny"},"runAsUser":{"rule":"RunAsAny"},"volumes":["secret","configMap","hostPath","persistentVolumeClaim","emptyDir"]}}'`



- `okteto up`

- From this point on, you can modify files locally and will be synced to the remote pod



#### In order to start the operator, you can build it and execute it inside the pod

- `go build`

- `aws-operator daemon --config.dirs=/var/run/aws-operator/configmap/ --config.dirs=/var/run/aws-operator/secret/ --config.files=config --config.files=secret`



#### Live reload code

- `cd /tmp && go get -u github.com/cosmtrek/air && cd /okteto`

- `air -c air.conf`



#### For live debugging in VS Code

- Install delve debugger: `go get github.com/go-delve/delve/cmd/dlv`

- `dlv debug --headless --listen=:2345 --log --api-version=2 -- daemon --config.dirs=/var/run/aws-operator/configmap/ --config.dirs=/var/run/aws-operator/secret/ --config.files=config --config.files=secret` or `./debug_server.sh`

- Create debugging connection:

```

  {

    "version": "0.2.0",

    "configurations": [

        {

            "name": "Connect to okteto",

            "type": "go",

            "request": "attach",

            "mode": "remote",

            "remotePath": "/okteto",

            "port": 2345,

            "host": "127.0.0.1"

        }

    ]

  }

  ```

- Wait until debug server is up and create some breakpoints, start the debugger :)

- If you want to edit the code you will need to stop debugging session and stop the server

- `okteto down -v` (-v will delete volume with go cache)

- Revert psp with `kubectl patch psp aws-operator-$BRANCH-psp -p '{"spec":{"runAsGroup":{"ranges": [{"max":65535, "min":1}],"rule":"MustRunAs"},"runAsUser":{"rule":"MustRunAsNonRoot"},"volumes":["secret","configMap"]}}'` or redeploy application



## Contact



- Mailing list: [giantswarm](https://groups.google.com/forum/!forum/giantswarm)

- Bugs: [issues](https://github.com/giantswarm/aws-operator/issues)



## Contributing & Reporting Bugs



See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches, the

contribution workflow as well as reporting bugs.



For security issues, please see [the security policy](SECURITY.md).



## License



aws-operator is under the Apache 2.0 license. See the [LICENSE](LICENSE) file

for details.



## Credit



- https://github.com/giantswarm/microkit