{"id":16726201,"url":"https://github.com/deas/ka0s","last_synced_at":"2025-03-15T15:24:30.780Z","repository":{"id":142206086,"uuid":"607304873","full_name":"deas/ka0s","owner":"deas","description":"Building Chaos around LitmusChaos on Kubernetes","archived":false,"fork":false,"pushed_at":"2023-04-19T15:24:34.000Z","size":538,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-22T05:28:24.085Z","etag":null,"topics":["chaos-engineering","flux2","kubernetes","litmuschaos","sre"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-27T18:16:18.000Z","updated_at":"2023-04-19T15:15:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"6d3343a9-8bfc-4727-b344-20a761eedbaa","html_url":"https://github.com/deas/ka0s","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deas%2Fka0s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deas%2Fka0s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deas%2Fka0s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deas%2Fka0s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deas","download_url":"https://codeload.github.com/deas/ka0s/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243747614,"owners_count":20341526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chaos-engineering","flux2","kubernetes","litmuschaos","sre"],"created_at":"2024-10-12T22:52:28.483Z","updated_at":"2025-03-15T15:24:30.749Z","avatar_url":"https://github.com/deas.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ka0s - Building Chaos around LitmusChaos on Kubernetes 🧪\n\nThe primary goal of this project is to build a Chaos Engineering environment around the [LitmusChaos](https://litmuschaos.io/) platform. We try hard to provide a smooth development process including GitOps based deployment. Hence, we are leveraging `flux`, `terraform`, `nix` (using [devenv as a nix flake](https://devenv.sh/guides/using-with-flakes/)) and `kind` (maybe `k3s` soon). `nix` is no requirement, but strongly recommended as it should automatically provide you with the other tools - you should not have to worry about how to install things with your package manager.\n\nIf you just want to kick the Chaos the tires quickly, or if you want to build a long lasting Chaos environment : This might be a place to start.\n\nExperimentation is a natural element of Chaos Engineering. However, it should be just as natural in Software Development in general. That is why you might encounter bits (such as Knative) with no strong Chaos Engineering relationship in this repo. Those are meant to be optional.\n\nThe default `localhost` cluster environment has very few requirements. It should work on many types of clusters. However, it is optimized to work with just enough resources to run the whole Chaos Stack and the resilient Sock Shop. It aims to make things easiliy accessible.\n\nVarious things I built upon had minor issues (mostly because there where outdated). At this time, the \"fixes\" are here because I wanted to move on quickly. Would be happy to contribute back. \n\nThis repo is derived from [flux-conductr](https://github.com/deas/flux-conductr). Look at that, if you are after a similar experience, focused on `flux` specifically.\n\n## Features\n- [LitmusChaos](https://litmuschaos.io/) platform\n- This repo acts as a ChaosHub\n- We serve the [Sock Shop Microservices Demo Application](https://microservices-demo.github.io/) as a scenario (defaulting to `containerd` experiments)\n- Tightly integrated Prometheus Stack including Grafana provisioned for the Sock Shop Appliation\n- Loki\n- Istio Eventing/Serving/Tracing (zipkin)\n- Cilium\n- Knative\n- [Locust](https://locust.io/) load testing (supporting the UI)\n- Portal API usage examples\n- Support for deployment in proxy/custom CA environments\n- Flux-/Terraform Deployment\n- Nix Dev Experience\n- Doom (Opt-In/Next Gen)\n\n## Bootrapping\nEven though, we am trying to cover most things declaratively, some random bits may be covered by `make` targets. Simply calling the default target:\n\n```sh\nmake\n```\nshould output help hinting at what is covered.\n\nYou may also want to disable github actions to start.\n\nOptional: Generate ssh deployent keys and add public key to your repo\n\n```sh\nmake gen-keys\nmake gh-add-deploy-key\n```\n\nThere is a `terraform` + `kind` based bootstrap in [`tf`](./tf).\n\n```sh\ncp sample.tfvars terraform.tfvars\n# Set proper values in terraform.tfvars\nmake apply\n```\nThis should spin up the limus server. Once it is up\n\n```sh\nmake open-app\n```\n\nshould open it in your browser.\n\nAlternatively, you can bootstrap or even upgrade an existing cluster (be sure to have current kubecontext set properly). Also, make sure `flux --version` shows desired version.\n\n```sh\n./scripts/flux-bootstrap.sh\n```\n\n## Proxy / Custom CA support\nWe aim at supporting environments requiring a proxy (including custom CA certificate chains) to access external services.\n\nA proxy has to be introduced in various places. Many systems (including  `kind`) support configuration via environment variables, namely `HTTPS_PROXY`, `HTTP_PROXY` and `NO_PROXY`.\n\nFor `flux`, we ship a [`local-proxy`](./clusters/local-proxy/flux-system/kustomization.yaml) cluster adding that environment. Set this cluster in `tf/terraform.tfvars` to try it.\n\nFor `litmus`, we only ship a runtime patch at the moment.\n\nRegarding custom certificates, we simply overlay the compiled file in the containers using a `ConfigMap`. By default, we assume we can generate it on the host executing the initial deployment:\n\n```sh\nmake -n recreate-ca-res\n```\n\n```sh\nmake -n patch-litmus-ca-certs patch-litmus-proxy-env\n```\n\nshould give you an idea how we patch a system.\n\nThe `terraform` module provides a mechanism to patch the `coredns` `ConfigMap`. This may come in handy when working with a proxy.\n\nI use `mitmproxy` locally to try things out.\n\n## Misc\nThe `local` cluster uses metallb to provide a loadbalancer. It binds multiple services to a single IP using `metallb.universe.tf/allow-shared-ip`.\n\nThe following ports are used:\n- `9091` : Litmus Portal\n- `9002` : Litmus Server (for remote agents)\n- `3000` : Grafana\n- `9411` : Zipkin (Mesh/Tracing)\n- `20001` : Kiali (Mesh/Istio)\n\nActing as a ChaosHub, this repo serves the `sock-shop` [scenario/workflow](./workflows/sock-shop/workflow.yaml)\n\n### Autentication\nGrafana : `admin` / `prom-operator`.\nLitmus : `admin` / `litmus`.\n\n## TODO\n- There are TODO tags in the code\n- Leverage `kustomize` with remote repos/resources in workflow (`litmuschaos/k8s:latest` does not yet have `git`) \n- Leverage Istio for failure injection?\n- This repo can act as a ChaosHub - add it during setup\n- Add first class support for `mitmproxy` (ship deployment)\n- ~~Add first class support for remote agent?~~\n- ~~Try GitOps scenarios?~~\n- Manifests Naming\n- Fix annoying terraform plan ` yaml_incluster`\n- Add knative-serving/eventing/dns (using `nip.io`?)\n- Add mongodb/prometheus convenience (e.g. auth) targets to `Makefile`\n- Test drive 3.0-beta\n- `disk-fill` does not yet play with containerd?\n- Catchup `cron` scheduled sock-shop workflow\n- Introduce PrometheusRule Sock-Shop alerts\n- Recover chaos \"enabled\" in Sock Shop Dashboard\n- ~~Introduce istio based tracing~~\n- Introduce [deas/calendar_monkey](https://github.com/deas/calendar_monkey)? ;)\n- Use NodePort instead of LoadBalancer locally (just like we do it in `flux-conductr`)\n\n\n## Known Issues\n- Some experiments from `litmus-go` appear to rely on `/var/run/docker.sock` which does not exist with containerd based environments ([see](https://docs.litmuschaos.io/docs/troubleshooting))\n- Knative deployment straight from github deployment not possible\n- knative challenging, should probably merge `kustomize.toolkit.fluxcd.io/substitute: disabled` via `kustomize`. Other things need tweaks to upstream yaml to play with GitOps \"... configured\" / Managed fields)\n- Istio Ingress appears to have an image pulling issue, so it takes a while to come up\n- litmus `helm` release removal should remove default agent?\n\n## Misc/Random Bits\n- https://docs.cilium.io/en/stable/network/istio/\n- https://knative.dev/docs/install/installing-istio/#installing-istio\n- Deploy knative straight from github? like flux-monitoring.yaml?\n- [Running Knative with Istio in a Kind Cluster (old!)](https://www.arthurkoziel.com/running-knative-with-istio-in-kind/)\n- [Install Knative using quickstart](https://knative.dev/docs/getting-started/quickstart-install/)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeas%2Fka0s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeas%2Fka0s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeas%2Fka0s/lists"}