https://github.com/khuedoan/cloudlab
Free cloud native platform for service hosting
https://github.com/khuedoan/cloudlab
cloud iaas k3s kubernetes kustomize oci terraform vpn
Last synced: about 2 months ago
JSON representation
Free cloud native platform for service hosting
- Host: GitHub
- URL: https://github.com/khuedoan/cloudlab
- Owner: khuedoan
- License: gpl-3.0
- Created: 2021-08-24T04:20:13.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2024-12-07T07:53:25.000Z (10 months ago)
- Last Synced: 2024-12-07T08:22:00.000Z (10 months ago)
- Topics: cloud, iaas, k3s, kubernetes, kustomize, oci, terraform, vpn
- Language: HCL
- Homepage:
- Size: 257 KB
- Stars: 39
- Watchers: 3
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Cloudlab
> [!IMPORTANT]
> This project is designed to manage my offsite setup, which is specific to my
> use cases, so it might not be directly useful to you. For a ready-to-use
> solution, please refer to my [homelab project](https://github.com/khuedoan/homelab).
## Project structure
```
.
├── flake.nix # Contains dependencies required by this project for both local and CI/CD
├── Makefile # Entry point for all manual actions
├── compose.yaml # Servers required for running locally
├── infra # Infrastructure definition
│ ├── modules # Terraform modules
│ │ ├── network
│ │ ├── instance
│ │ ├── cluster
│ │ └── ...
│ ├── local # Terragrunt configuration for the local environment
│ │ └── ...
│ └── ${ENV} # Terragrunt configuration for the ${ENV} environment
│ ├── root.hcl # Root config used by other Terragrunt files
│ ├── secrets.yaml # Encrypted secrets
│ ├── tfstate # Bootstrap Terraform state
│ ├── ${CLOUD}
│ │ └── ${REGION}
│ │ └── ${MODULE}
│ │ └── terragrunt.hcl
│ ├── metal
│ │ └── vn-southeast-1
│ │ ├── bootstrap
│ │ │ └── terragrunt.hcl
│ │ └── cluster
│ │ └── terragrunt.hcl
│ └── ...
├── platform # Highly privileged platform components
│ └── ${ENV}
│ ├── grafana.yaml
│ ├── temporal.yaml
│ ├── wireguard.yaml
│ └── ...
├── apps # User applications, standardized with strict controls
│ ├── ${NAMESPACE}
│ │ └── ${APP}
│ │ └── ${ENV}.yaml
│ └── khuedoan
│ └── blog
│ ├── local.yaml
│ └── production.yaml
├── controller # Automation controller for the entire project - think GitHub Actions, but better
│ ├── activities # Temporal activities (git clone, terragrunt apply, etc.)
│ │ ├── git.go
│ │ ├── terragrunt.go
│ │ └── ...
│ ├── workflows # Temporal workflows, define a sequence of activities
│ │ ├── infra.go
│ │ ├── app.go
│ │ └── ...
│ ├── worker # Worker process that executes the workflows
│ └── Dockerfile # Builds the image for the controller, can run locally or on a cluster
└── test # High level tests
```
## Features
- Unified hybrid cloud platform
- Temporal is used as the automation engine, providing the reliability and
performance that generic CI/CD engines can only dream of.
- Infra aka IaaS:
- Essentially `cd "infra/${ENV}" && terragrunt apply --all`
- Includes some graph pruning based on changed files for performance
- Bootstrap ArgoCD to apply the remaining
- Platform aka PaaS:
- Essentially `kubectl apply -f "platform/${ENV}"`
- However, the runtime doesn’t have access to Git - all manifests are pulled from an OCI registry
- Apps aka SaaS:
- Strict and standardized
- Uses the rendered manifests pattern, essentially `helm template && oras push`
## Estimated cost
| Provider | Service | Usage | Pricing |
| :-- | :-- | :-- | :-- |
| Cloudflare | R2 Bucket (Terraform state) | 2 | Free |
| Cloudflare | Domain | 2 | 1.67$/month |
| Cloudflare | Load Balancer | 1 | 5$/month |
| Cloudflare | Tunnel | 2 | Free |
| Oracle Cloud | Virtual Cloud Network | 1 | Free |
| Oracle Cloud | `VM.Standard.A1.Flex` (ARM) | 4 cores, 24GB mem | Free (yes, you read it right!) |
| Oracle Cloud | Block Storage | 200GB | Free |
| Metal | Hardware depreciation | | 6.36$/month |
| Metal | Electricity | | 3$/month |
| **Total** | | | 16.03$/month |
## Get started
### Prerequisites
- Fork this repository because you will need to customize it for your needs.
- A credit/debit card to register for the accounts.
- Basic knowledge on Terraform, Ansible and Kubernetes (optional, but will help a lot)
Configuration files:
Terraform Cloud
- Create a Terraform Cloud account at
Oracle Cloud
- Create an Oracle Cloud account at
- Generate an API signing key:
- Profile menu (User menu icon) -> User Settings -> API Keys -> Add API Key
- Select Generate API Key Pair, download the private key to `~/.oci/private.pem` and click Add
- Copy the Configuration File Preview to `~/.oci/config` and change `key_file` to `~/.oci/private.pem`
If you see a warning like this, try to avoid those regions:
> ⚠️ Because of high demand for Arm Ampere A1 Compute capacity in the Foo and Bar regions, A1 instance availability in these regions is limited.
> If you plan to create A1 instances, we recommend choosing another region as your home region
Install the following packages:
- [Nix](https://nixos.org/download.html)
That's it! Run the following command to open the Nix shell:
```sh
nix develop
```
### Provision
Build the infrastructure:
```sh
make
```
## TODOs
- Fix OCI plain HTTP for local development
- Config git username and email
- Credentials for the worker (SSH priv + pub + knowhosts?)
## Acknowledgments and References
- [Oracle Terraform Modules](https://github.com/oracle-terraform-modules)
- [Official k3s systemd service file](https://github.com/k3s-io/k3s/blob/master/k3s.service)
- [Sample Prometheus configuration for Istio](https://github.com/istio/istio/blob/master/samples/addons/extras/prometheus-operator.yaml)