{"id":27248882,"url":"https://github.com/bakdata/citus-k8s-membership-manager","last_synced_at":"2025-07-10T20:32:35.370Z","repository":{"id":39846438,"uuid":"157416218","full_name":"bakdata/citus-k8s-membership-manager","owner":"bakdata","description":"A manager for connecting Citus worker nodes with the Citus master node.","archived":false,"fork":false,"pushed_at":"2022-12-08T04:51:32.000Z","size":180,"stargazers_count":11,"open_issues_count":12,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-03T01:25:31.809Z","etag":null,"topics":["citusdb","kubernetes","postgres","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bakdata.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-13T17:04:37.000Z","updated_at":"2024-09-16T23:39:43.000Z","dependencies_parsed_at":"2022-07-26T10:18:01.563Z","dependency_job_id":null,"html_url":"https://github.com/bakdata/citus-k8s-membership-manager","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/bakdata/citus-k8s-membership-manager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fcitus-k8s-membership-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fcitus-k8s-membership-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fcitus-k8s-membership-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fcitus-k8s-membership-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bakdata","download_url":"https://codeload.github.com/bakdata/citus-k8s-membership-manager/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fcitus-k8s-membership-manager/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264652633,"owners_count":23644292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["citusdb","kubernetes","postgres","python"],"created_at":"2025-04-10T23:48:24.193Z","updated_at":"2025-07-10T20:32:35.310Z","avatar_url":"https://github.com/bakdata.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# citus-k8s-membership-manager\n[![Build Status](https://travis-ci.com/bakdata/citus-k8s-membership-manager.svg?branch=master)](https://travis-ci.com/bakdata/citus-k8s-membership-manager)\n[![Code Climate](https://codeclimate.com/github/bakdata/citus-k8s-membership-manager/badges/gpa.svg)](https://codeclimate.com/github/bakdata/citus-k8s-membership-manager)\n[![](https://img.shields.io/docker/automated/jrottenberg/ffmpeg.svg)](https://hub.docker.com/r/bakdata/citus-k8s-membership-manager)\n\n\nThis project aims to provide a service which helps running PostgreSQL with the [Citus](https://github.com/citusdata/citus) extension on kubernetes.\n\nHereby, it supports the following features:\n\n- Register/unregister worker nodes on master during startup/teardown\n- Wait until worker threshold is reached before provisioning\n- Running provision scripts (SQL) on master/worker node startup\n\n## Setup\n\nFirst, it is recommended to create a dedicated namespace for the membership-member or if your citus cluster is already running you can reuse this namespace. Furthermore, you have to create a service account allowing the membership-manager to list all pods and their status in the current namespace.\nThis service account and the needed privileges can be created according to [tests/test\\_yaml/pods-list-role-binding.yaml](tests/test\\_yaml/pods-list-role-binding.yaml)\n\n### Provisioning\n\nIn addition to the service account, you also have to create a ConfigMap to provision worker and master nodes. An example can be found here [tests/test\\_yaml/provision-map.yaml](tests/test\\_yaml/provision-map.yaml).\n\n**IMPORTANT:** Keep the same file structure with all its keys and only change the two value strings for `master.setup` and `worker.setup`. The membership-manager will check for these file names specifically.\n\n### Labels\n\nWe use pod labels to distinguish between worker and master nodes. Therefore you have to create a pod label called `citusType`.   \nEither you create your nodes accordingly to [tests/test\\_yaml/citus-master.yaml](tests/test\\_yaml/citus-master.yaml), [tests/test\\_yaml/citus-worker.yaml](tests/test\\_yaml/citus-worker.yaml) or you patch your existing cluster with the following command.\n\n```\nkubectl patch statefulset  \u003cstatefulset-name\u003e -n \u003cnamespace\u003e --patch '{\"spec\": {\"template\": {\"metadata\": {\"labels\": {\"citusType\": \u003cyour-label\u003e}}}}}'\n```\n\n\n### Installation\n\nFinally to deploy the membership-manager, you have to edit the yaml file replacing the template variables: \n\n```\n\u003cyour-service-account\u003e\n\u003cyour-namespace\u003e\n\u003cyour-config-map-name\u003e\n```\n\nwith the corresponding names in your setup.\n\nThen you can run:\n\n```\nkubectl create -f manager-deployment.yaml\n``` \n\n\n### GKE\n\nIf you want to do the same on Google's Kubernetes Engine you might have to create a cluster admin binding first which sets your current google user as cluster admin\n\n```shell\nkubectl create clusterrolebinding \u003cyour--binding-name\u003e --clusterrole=cluster-admin --user=\u003cemail-address-used-for-this-google-account\u003e\n```\n\n## Manager Pod Environment Config\n\nThe following environment variables are configurable on pod startup. Before the deployment you can set those in the `manager-deployment.yaml` file.\n\n```yml\nenv:\n- name: NAMESPACE\n  value: \u003cyour-namespace\u003e # Namespace where the manager runs and the citus cluster is supposed to be\n- name: MASTER_LABEL\n  value: \u003cdefault: citus-master\u003e # Label of the citus master pods\n- name: MASTER_SERVICE\n  value: \u003cdefault: pg-citus-master\u003e # Service of the citus master pods\n- name: WORKER_LABEL\n  value: \u003cdefault: citus-worker\u003e # Label of the citus worker pods\n- name: WORKER_SERVICE\n  value: \u003cdefault: pg-citus-worker\u003e # Service of the citus worker pods\n- name: PG_DB\n  value: \u003cdefault: postgres\u003e # Database name for postgres db\n- name: PG_USER\n  value: \u003cdefault: postgres\u003e # Database user for postgres db\n- name: PG_PORT\n  value: \u003cdefault: 5432\u003e # Database port for postgres db\n- name: PG_PASSWORD\n  value: \u003cdefault: None\u003e # If present it is used for all the connections to the pg nodes\n- name: MINIMUM_WORKERS\n  value: \u003cdefault: 0\u003e # Threshold until the manager waits with node provisioning\n- name: SHORT_URL\n  value: \u003cdefault: False\u003e # If set {pod_name}.{service_name} is used as host pattern instead of {pod_name}.{service_name}.{namespace}.svc.cluster.local\n- name: SSL_MODE\n  value: \u003cdefault: None\u003e # Supports PostgreSQL sslmodes https://www.postgresql.org/docs/current/libpq-ssl.html\n```\n\n## Development\n\nSince the main development for this tool is done in Python we decided to use [black](https://github.com/ambv/black) as formatting tool and [mypy](http://mypy-lang.org/) as type hinting tool. If you want to contribute please install these tools in your favorite IDE or use them as cli tools to keep the code consistent. When you want to make your first changes you can install the needed dependencies with running the following commands in the root directory of the repository.\n\n```shell\npipenv install --dev\npipenv shell\n```\n\nTo run the tests locally you first have to install [minikube](https://kubernetes.io/docs/setup/minikube/) on your machine. We have tested the following cluster configuration:\n\n```shell\nminikube start --memory 4096 --cpus 4 --vm-driver hyperkit --bootstrapper=kubeadm\n```\n\nAfterward, you can run `pytest`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbakdata%2Fcitus-k8s-membership-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbakdata%2Fcitus-k8s-membership-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbakdata%2Fcitus-k8s-membership-manager/lists"}