{"id":21384445,"url":"https://github.com/daljitdokal/stackgress-postgres-ha-solution","last_synced_at":"2025-03-16T11:41:54.010Z","repository":{"id":167743659,"uuid":"530478530","full_name":"daljitdokal/stackgress-postgres-ha-solution","owner":"daljitdokal","description":"An enterprise-grade PostgreSQL stack needs several other ecosystem components and significant tuning. It’s not only PostgreSQL. It requires connection pooling, automatic failover and HA, monitoring, backups and DR, centralized logging… we have built them all: a Postgres Stack.","archived":false,"fork":false,"pushed_at":"2022-11-07T21:49:07.000Z","size":45,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T23:41:12.652Z","etag":null,"topics":["automation","ci-cd","helm","high-availability","openshift","pipeline","postgresql","serviceaccount","serviceaccounts","stackgres"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daljitdokal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-30T03:10:21.000Z","updated_at":"2024-04-22T18:19:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"59283a47-c193-4bf6-baac-cc41782a9395","html_url":"https://github.com/daljitdokal/stackgress-postgres-ha-solution","commit_stats":null,"previous_names":["daljitdokal/stackgress-postgres-ha-solution"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daljitdokal%2Fstackgress-postgres-ha-solution","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daljitdokal%2Fstackgress-postgres-ha-solution/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daljitdokal%2Fstackgress-postgres-ha-solution/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daljitdokal%2Fstackgress-postgres-ha-solution/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daljitdokal","download_url":"https://codeload.github.com/daljitdokal/stackgress-postgres-ha-solution/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243864611,"owners_count":20360355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","ci-cd","helm","high-availability","openshift","pipeline","postgresql","serviceaccount","serviceaccounts","stackgres"],"created_at":"2024-11-22T11:40:57.826Z","updated_at":"2025-03-16T11:41:53.989Z","avatar_url":"https://github.com/daljitdokal.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Stackgress\n\n[StackGres](https://stackgres.io/) is a full-stack (connection pooling, automatic failover and HA, monitoring, backups and DR, centralized logging) PostgreSQL distribution for Kubernetes, packed into an easy deployment unit. With a carefully selected and tuned set of surrounding PostgreSQL components. We recommend that you check our [documentation](https://stackgres.io/doc/latest/) and have a look at the Demo / Quickstart section to know how to start using StackGres.\n\n# How to upgrade\n\n**Latest releases:** https://gitlab.com/ongresinc/stackgres/-/releases\n\n```bash\nmkdir tmp\ncd tmp\nexport latestVersion=\"1.2.1\"\nwget \"https://gitlab.com/ongresinc/stackgres/-/archive/${latestVersion}/stackgres-${latestVersion}.zip\"\nunzip \"stackgres-${latestVersion}.zip\"\nrm \"stackgres-${latestVersion}.zip\"\n\nrm -rf ~/stackgress-postgres-ha-solution/stackgres-operator\nrm -rf ~/stackgress-postgres-ha-solution/stackgres-cluster\n\nmv  ~/tmp/stackgres-${latestVersion}/stackgres-k8s/install/helm/stackgres-operator/ ~/stackgress-postgres-ha-solution/\nmv  ~/tmp/stackgres-${latestVersion}/stackgres-k8s/install/helm/stackgres-cluster/ ~/stackgress-postgres-ha-solution/\n\nrm -rf ~/tmp/stackgres-${latestVersion}\n```\n\n# Stackgres Operator\n\nAn Operator is a method of packaging, deploying and managing a Kubernetes application. Some applications, such as databases, required more hand-holding, and a cloud-native Postgres requires an operator to provide additional knowledge of how to maintain state and integrate all the components.\n\nThis operator is built in pure-Java and uses the [Quarkus](https://quarkus.io/) framework a Kubernetes Native Java stack tailored for GraalVM \u0026 OpenJDK HotSpot, crafted from the best of breed Java libraries and standards.\n\nThe container image of StackGres is built on `Red Hat Universal Base Image` and compiled as a native binary with GraalVM allowing amazingly fast boot time and incredibly low RSS memory.\n\n#### Helm Chart to create the StackGres Operator\n\n```bash\nexport appName=\"stackgres-operator\"\nexport namespace=\"stackgres\"\nexport password=\"password\"\n\n# Create namespace\noc create ${namespace}\n\n# Deploy\nhelm upgrade --install --namespace ${namespace} \\\n    --set containerRegistry=containerRegistry \\\n    --set authentication.password=${password} \\\n    --set adminui.service.exposeHTTP=true \\\n    --set extensions.repositoryUrls[0]=\"\" \\\n    ${appName} ~/stackgress-postgres-ha-solution/stackgres-operator/\n\n# Create route\noc create route edge admin-ui --service=stackgres-restapi --port=http --insecure-policy=Redirect --namespace ${namespace}\nexport routeUrl=$(oc get routes admin-ui -o jsonpath={.spec.host} --namespace ${namespace});\necho \"Plese use the following details to access admin-ui for stackgres-operator\"\necho \"URL: https://${routeUrl}/admin/index.html\"\necho \"USERNAME: admin\"\n```\n\n#### How to delete\n\n```bash\nexport appName=\"stackgres-operator\"\nexport namespace=\"stackgres\"\n\noc delete route admin-ui -n ${namespace}\nhelm delete ${appName} -n ${namespace}\noc delete validatingwebhookconfigurations.admissionregistration.k8s.io stackgres-operator\noc delete mutatingwebhookconfigurations.admissionregistration.k8s.io stackgres-operator\noc get crds -n ${namespace} -o name | egrep stackgres.io | xargs kubectl delete\n```\n\n# Stackgres Database HA Cluster\n\nTo add only required permissions to a service account, please assign following `ClusterRole` to `service account`.\n\nCreate `ClusterRole`:\n\n```yaml\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  name: stackgres-admin\nrules:\n- apiGroups: [\"\"]\n  resources:\n    - namespaces\n    - pods\n    - secrets\n  verbs: [\"get\", \"list\"]\n- apiGroups: [\"storage.k8s.io\"]\n  resources:\n    - storageclasses\n  verbs: [\"get\", \"list\"]\n- apiGroups: [\"apiextensions.k8s.io\"]\n  resources:\n    - customresourcedefinitions\n  verbs: [\"get\", \"list\"]\n- apiGroups: [\"stackgres.io\"]\n  resources:\n    - sgclusters\n    - sgpgconfigs\n    - sgbackupconfigs\n    - sgbackups\n    - sgdistributedlogs\n    - sginstanceprofiles\n    - sgpoolconfigs\n    - sgscripts\n  verbs: [\"get\", \"list\", \"create\", \"update\", \"patch\", \"delete\"]\n```\n\nAssign `ClusterRole` to service account:\n\n```yaml\napiVersion: rbac.authorization.k8s.io/v1\nkind: RoleBinding\nmetadata:\n  name: my-sa-stackgres-admin\n  namespace: my-namespace\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: stackgres-admin\nsubjects:\n- kind: ServiceAccount\n  name: my-sa\n  namespace: my-namespace\n```\n\nNow we can use this service account in `CI/CD` pipelines for deplyment of each database cluster.\n\n**Deploy database cluster**\n\n```bash\nexport appName=\"mydb-postgres\"\nexport namespace=\"my-namespace\"\nexport instances=3\nexport dbName=\"mydb\"\nexport dbUserName=\"mydbuser\"\nexport dbUserPass=\"pass\"\nexport version=latest\nexport cpu=500m\nexport memory=512Mi\nexport backup_retention=3\nexport backup_cronSchedule=\"*/30 * * * *\" # every 30 minutes\nexport backup_path_in_minio=\"/my-namespace\"\nexport restore_backup_name=\"\" # leave it empty if restore is not required or set the value as example below\n# export restore_backup_name=\"mydb-postgres-2022-11-08-20-00-09\" # i.e. oc get sgbackups -n ${namespace}\n# export setDate=\"2022-11-07T20:00:15.00Z\" # \"2022-11-08T20:00:09.00Z\" # to restore data to specific date and time with WAL logs files.\n\n# Custom database name with user\noc --namespace ${namespace} create secret generic create-${appName}-user \\\n    --from-literal=create-${appName}-user.sql=\"CREATE USER ${dbUserName} PASSWORD '${dbUserPass}';\"\n\n# MinIO instance details for backups\noc create -n ${namespace} secret generic minio-details \\\n     --from-literal=accesskey=\"xxxxxx\" \\\n     --from-literal=secretkey=\"xxxxxx\"\n\nexport restore_script=\"\"\nif [[ ${restore_backup_name} != \"\" ]]; then\n    export restore_script=\"--set cluster.initialData.restore.fromBackup.name=${restore_backup_name}\"\n    #export restore_script=\"--set cluster.initialData.restore.fromBackup.name=${restore_backup_name} --set cluster.initialData.restore.fromBackup.pointInTimeRecovery.restoreToTimestamp=${setDate}\"\nfi\n\n# Deploy\nhelm upgrade --install --namespace ${namespace} \\\n    --set cluster.instances=${instances} \\\n    --set cluster.postgres.version=${version} \\\n    --set cluster.sgInstanceProfile=size-${appName} \\\n    --set cluster.pods.disableMetricsExporter=true \\\n    --set cluster.prometheusAutobind=false \\\n    --set instanceProfiles[0].name=size-${appName} \\\n    --set instanceProfiles[0].cpu=${cpu} \\\n    --set instanceProfiles[0].memory=${memory} \\\n    --set cluster.configurations.sgPostgresConfig=postgresconf-${appName} \\\n    --set cluster.configurations.sgPoolingConfig=pgbouncerconf-${appName} \\\n    --set cluster.configurations.sgBackupConfig=sgbackupconfig-${appName} \\\n    --set cluster.managedSql.scripts[0].name=create-${appName}-user \\\n    --set cluster.managedSql.scripts[0].scriptFrom.secretKeyRef.name=create-${appName}-user \\\n    --set cluster.managedSql.scripts[0].scriptFrom.secretKeyRef.key=create-${appName}-user.sql \\\n    --set cluster.managedSql.scripts[1].name=create-${appName}-database \\\n    --set cluster.managedSql.scripts[1].script=\"CREATE DATABASE ${dbName} WITH OWNER ${dbUserName};\" \\\n    --set configurations.backupconfig.create=true \\\n    --set configurations.backupconfig.baseBackups.retention=${backup_retention} \\\n    --set configurations.backupconfig.baseBackups.cronSchedule=\"${backup_cronSchedule}\" \\\n    --set configurations.backupconfig.baseBackups.performance.maxNetworkBandwidth=\"26214400\" \\\n    --set configurations.backupconfig.baseBackups.performance.maxDiskBandwidth=\"52428800\" \\\n    --set configurations.backupconfig.baseBackups.performance.uploadDiskConcurrency=2 \\\n    --set configurations.backupconfig.storage.s3Compatible.bucket=\"bucket-name\" \\ # minio bucket name i.e. us-east-1\n    --set configurations.backupconfig.storage.s3Compatible.region=\"region\" \\ # minio region i.e. us-east-1\n    --set configurations.backupconfig.storage.s3Compatible.endpoint=\"http://minio-instance-url.com\" \\\n    --set configurations.backupconfig.storage.s3Compatible.enablePathStyleAddressing=true \\\n    --set configurations.backupconfig.storage.s3Compatible.awsCredentials.secretKeySelectors.accessKeyId.name=\"minio-details\" \\\n    --set configurations.backupconfig.storage.s3Compatible.awsCredentials.secretKeySelectors.accessKeyId.key=\"accesskey\" \\\n    --set configurations.backupconfig.storage.s3Compatible.awsCredentials.secretKeySelectors.secretAccessKey.name=\"minio-details\" \\\n    --set configurations.backupconfig.storage.s3Compatible.awsCredentials.secretKeySelectors.secretAccessKey.key=\"secretkey\" \\\n    --set configurations.backupconfig.storage.s3Compatible.path=${backup_path_in_minio} ${restore_script} \\\n    ${appName} ~/stackgress-postgres-ha-solution/stackgres-cluster/\n```\n\n## Restore backup from another namespace\n\n```bash\nexport source_backup_name=\"mydb-postgres-2022-11-08-20-00-09\"\nexport source_namespace=\"my-namespace\"\nexport target_namespace=\"new-namespace\"\n\n# Import backup refrence to new namespace\noc get sgbackup -n ${source_namespace} ${source_backup_name} -o json | jq '.metadata.namespace = \"stackgres2\" | .spec.sgCluster = \"${target_namespace}.\" + .spec.sgCluster' | kubectl create -f -\n\n# View backup refence in target namespace\noc get sgbackup -n ${target_namespace}\n\n# Now you shoud be able to restore backup to new namespace. Please update the `restore_backup_name` before you creating a new cluster in deplyment script.\n```\n\n\n#### How to delete\n\n```bash\nexport appName=\"mydb-postgres\"\nexport namespace=\"namespace\"\n\nhelm delete ${appName} -n ${namespace}\n\noc --namespace ${namespace} --ignore-not-found=true delete secret create-${appName}-user\noc --namespace ${namespace} --ignore-not-found=true delete sginstanceprofile size-${appName}\noc --namespace ${namespace} --ignore-not-found=true delete sgpoolconfigs pgbouncerconf-${appName}\noc --namespace ${namespace} --ignore-not-found=true delete sgpgconfigs postgresconf-${appName}\noc --namespace ${namespace} --ignore-not-found=true delete sgobjectstorages objectstorage-${appName}\noc --namespace ${namespace} --ignore-not-found=true delete sgbackupconfigs sgbackupconfig-${appName}\n\n# oc --namespace ${namespace} get sgbackups -o name | grep ${appName} | xargs oc delete\n# oc --namespace ${namespace} get sginstanceprofile -o name | grep generated-from-default | xargs oc delete\n# oc --namespace ${namespace} get sgpgconfigs -o name | grep generated-from-default | xargs oc delete\n# oc --namespace ${namespace} get sgpoolconfigs -o name | grep generated-from-default | xargs oc delete\n```\n\n\n# Known issue raised with `StackGres`\n\nI have raised an issue with `StackGres`: https://gitlab.com/ongresinc/stackgres/-/issues/1910\n\n**Summary:**\nAdmin UI making too many xhr.js calls and stackgres-restapi pod unable to handle the requests. Start getting 500 Gateway time-out errors once there are  more that 4 pending 'can-i` jobs.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaljitdokal%2Fstackgress-postgres-ha-solution","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaljitdokal%2Fstackgress-postgres-ha-solution","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaljitdokal%2Fstackgress-postgres-ha-solution/lists"}