{"id":13688716,"url":"https://github.com/zer0def/aquarium","last_synced_at":"2025-05-01T20:30:31.690Z","repository":{"id":38275372,"uuid":"260899019","full_name":"zer0def/aquarium","owner":"zer0def","description":"Highly-opinionated Linux-centric scaffold for local multi-node Kubernetes development. Ages like fine milk - locally alive, but elegantly tying a bow on this mess while keeping it wholistically up to date is a circus performance to do.","archived":false,"fork":false,"pushed_at":"2023-06-19T12:36:59.000Z","size":246,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-08-02T15:11:58.870Z","etag":null,"topics":["ceph","docker","helmfile","incus","istio","k3s","kata-containers","kubeless","kubernetes","kubernetes-development","lxd","openstack","prometheus","rook","rook-ceph","thanos"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zer0def.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-03T11:42:47.000Z","updated_at":"2023-11-07T01:25:46.000Z","dependencies_parsed_at":"2024-01-14T16:12:43.278Z","dependency_job_id":null,"html_url":"https://github.com/zer0def/aquarium","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Faquarium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Faquarium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Faquarium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Faquarium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zer0def","download_url":"https://codeload.github.com/zer0def/aquarium/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224274722,"owners_count":17284637,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ceph","docker","helmfile","incus","istio","k3s","kata-containers","kubeless","kubernetes","kubernetes-development","lxd","openstack","prometheus","rook","rook-ceph","thanos"],"created_at":"2024-08-02T15:01:20.872Z","updated_at":"2025-05-01T20:30:31.684Z","avatar_url":"https://github.com/zer0def.png","language":"Shell","funding_links":[],"categories":["docker"],"sub_categories":[],"readme":"# Aquarium - highly-opinionated Linux-centric scaffold for local multi-node Kubernetes development\n\n## Summary\n\nAquarium's design goal is to leverage Docker (using k3s/k3d) or LXD (using proper Kubernetes through Kubedee) to provide a comparatively lightweight scaffold for launching multi-node Kubernetes development environments on your local machine, while trying to address some of a pain points of doing so.\n\nMost notably, this project tries really hard to avoid using operators where they're not necessary, as they usually serve as means to upsell the user on a product, while funnelling away from otherwise perfectly available configuration options and into each respective party's walled garden to gauge for.\n\n### Usage\n\n```\naquarium - Linux-centric scaffold for local K8S development\n\nUsage: aquarium.sh [options] \u003cup|down\u003e\n\nOptions:\n  --no-*, --with-*                    disable/enable installation of selected\n                                      component (choice of: registry-proxy,\n                                        monitoring, serverless, service-mesh,\n                                        storage, local-registry,\n                                        env: non-zero value on INSTALL_*)\n  -N \u003cname\u003e, --name \u003cname\u003e            cluster name (default: k3s-default,\n                                        env: CLUSTER_NAME)\n  -n \u003cnum\u003e, --num \u003cnum\u003e               number of workers (default: `nproc`/4,\n                                        env: NUM_WORKERS)\n  -r \u003cruntime\u003e, --runtime \u003cruntime\u003e   runtime choice (default: k3d,\n                                        choice of: k3d, kubedee,\n                                        env: K8S_RUNTIME)\n  -t \u003ctag\u003e, --tag \u003ctag\u003e               set runtime version (env: RUNTIME_TAG)\n  -s \u003cpool\u003e, --storage-pool \u003cpool\u003e    LXD storage pool to use with Kubedee\n                                        (default: default,\n                                        env: LXD_STORAGE_POOL)\n  --vm                                launch cluster in LXD VMs, instead of LXD\n                                        containers (requires `-r kubedee`)\n  -c \u003cmem\u003e, --controller-mem \u003cmem\u003e    memory to allocate towards K8S controller\n                                        (requires `--vm`, default: 2GiB,\n                                        env: CONTROLLER_MEMORY_SIZE)\n  -w \u003cmem\u003e, --worker-mem \u003cmem\u003e        memory to allocate per K8S worker\n                                        (requires `--vm`, default: 4GiB,\n                                        env: WORKER_MEMORY_SIZE)\n  -R \u003csize\u003e, --rootfs-size \u003csize\u003e     build rootfs image of provided size\n                                        (requires `--vm`, default: 20GiB,\n                                        env: ROOTFS_SIZE)\n\nEnvironment variables:\n\n  Registry proxy (ref: https://github.com/rpardini/docker-registry-proxy#usage ):\n    PROXY_REGISTRIES    space-delimited string listing registry domains to cache\n                        OCI image layers from\n    AUTH_REGISTRIES     space-delimited string listing \"domain:username:password\"\n                        information for the proxy to authenticate to registries\n```\n\n### Project status\n\nHighly bug-riddled alpha, YMMV. You probably should skim through, before using. You have been warned.\n\n### Project rationale\n\n#### Resource usage\n\nOther solutions targeted for Kubernetes development (taking Minikube \u0026 friends as an example) can be resource-taxing due to hypervisor overhead, which this avoids through usage of OCI/system containers. For Windows and MacOS X users, in terms of memory, that potentially means packing more into their Hyper-V/xhyve Docker VM or WSL-based VM, than Minikube or Docker-packaged Kubernetes might, though at possible cost of added CPU overhead, since those are still hypervised.\n\n#### Don't fight your tools when you don't need to\n\nEven with constant various improvements in upstream projects, over the years there has been a number of barely-addressed crippling corner cases, solutions to some of which have rotten away in experimental branches before eventually getting upstreamed after everyone has abandoned them by virtue of not wanting to endlessly fight the tools they need.\n\n#### Emulate your target environment without development/provider-specific cruft\n\nThere's also an issue of developing (and, perhaps more importantly, adequately testing) Kubernetes manifests for things like solution resilience or scalability, which you cannot properly do on a single-node environment without making development-specific additions/exceptions to your manifests. This allows you to focus on your goal, not how to make it work within artificial constraints, most of the time.\n\n## Dependencies\n\nBinaries/scripts (but not OCI images or Helm charts) listed below are expected to be in your system's `PATH`.\n\n### Hard dependencies\n\n- Docker\n- kubectl\n- [Helm](https://github.com/helm/helm)\n- [Helmfile](https://github.com/roboll/helmfile)\n- either of:\n    - [k3d](https://github.com/rancher/k3d)\n    - [kubedee](https://github.com/schu/kubedee) (along with [cfssl](https://github.com/cloudflare/cfssl), [jq](https://github.com/stedolan/jq) and [lxd](https://github.com/lxc/lxd))\n\n### Optional dependencies (enabled by default!)\n\n- [docker-volume-loopback](https://github.com/ashald/docker-volume-loopback) (when Docker root is running on a filesystem not supporting overlays)\n- [docker-registry-proxy](https://github.com/rpardini/docker-registry-proxy) (transparent proxy for caching OCI image layers)\n- [Kata Containers](https://github.com/kata-containers/kata-containers)\n\n### Charts/software used, depending on component selection\n\nLogical components are split into namespaces according to the following logic:\n\n- storage:\n  - [External-Secrets](https://github.com/external-secrets/external-secrets)\n  - [OpenEBS](https://github.com/openebs/openebs) in local volume provisioner mode\n  - [SeaweedFS](https://github.com/chrislu/seaweedfs) for object storage (MinIO replacement)\n  - [Patroni](https://github.com/zalando/patroni)/[Spilo](https://github.com/zalando/spilo) for PostgreSQL deployment\n  - [RedPanda](https://github.com/redpanda-data/redpanda) (Kafka replacement)\n  - [KeyDB](https://github.com/snapchat/keydb) (Redis replacement)\n  - [Scylla](https://github.com/scylladb/scylladb) (Cassandra replacement)\n- network/service mesh: [Istio](https://github.com/istio/istio)\n- monitoring:\n  - [Loki](https://github.com/grafana/loki)\n  - [OpenSearch](https://github.com/opensearch-project/OpenSearch)\n  - [Prometheus-Operator](https://github.com/coreos/prometheus-operator) with [Thanos](https://github.com/thanos-io/thanos) or [Cortex](https://github.com/cortexproject/cortex)\n- serverless: [OpenFAAS](https://docs.openfaas.com/), possibly [Fission](https://github.com/fission/fission)\n- development:\n  - [GitLab](https://gitlab.com/gitlab-org/gitlab)\n  - [Harbor](https://github.com/goharbor/harbor) for cluster-local registry\n  - [Sentry](https://github.com/getsentry/self-hosted)\n  - [Zulip](https://github.com/zulip/zulip)\n\n## Known issues\n\n- Kata's available only through Kubedee\n- Kubedee: Registry proxy not deployed as an LXD container, making Docker a harder dependency than it genuinely needs to be\n- most likely inconsistent whitespace handling, deal with it\n\n## Legalese\n\nReality says \"put it under WTFPL\", but sure, let's try LGPL3.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0def%2Faquarium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzer0def%2Faquarium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0def%2Faquarium/lists"}