{"id":30237737,"url":"https://github.com/dalibo/cnpg-plugin-pgbackrest","last_synced_at":"2025-08-15T02:09:25.654Z","repository":{"id":286066578,"uuid":"958416817","full_name":"dalibo/cnpg-plugin-pgbackrest","owner":"dalibo","description":"pgBackRest CNPG-I plugin for CloudNativePG","archived":false,"fork":false,"pushed_at":"2025-04-11T12:10:11.000Z","size":130,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-19T00:04:49.269Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dalibo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-01T06:58:34.000Z","updated_at":"2025-04-11T12:10:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"f0ac7437-7995-4d06-924e-7d0cce45bb0e","html_url":"https://github.com/dalibo/cnpg-plugin-pgbackrest","commit_stats":null,"previous_names":["dalibo/cnpg-plugin-pgbackrest"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dalibo/cnpg-plugin-pgbackrest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalibo%2Fcnpg-plugin-pgbackrest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalibo%2Fcnpg-plugin-pgbackrest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalibo%2Fcnpg-plugin-pgbackrest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalibo%2Fcnpg-plugin-pgbackrest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dalibo","download_url":"https://codeload.github.com/dalibo/cnpg-plugin-pgbackrest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dalibo%2Fcnpg-plugin-pgbackrest/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270511293,"owners_count":24597666,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-15T02:09:23.465Z","updated_at":"2025-08-15T02:09:25.628Z","avatar_url":"https://github.com/dalibo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\nSPDX-FileCopyrightText: 2025 Dalibo\n\nSPDX-License-Identifier: Apache-2.0\n--\u003e\n\n# pgBackRest CNPG plugin\n\nExperimental CNPG operator plugin to backup PostgreSQL instances with\npgBackRest.\n\n## Features\n\n-   Data Directory Backup\n-   WALs archiving\n\nThis plugin is currently only compatible with `s3` storage and have been\ntested with:\n\n-   [minIO](https://min.io)\n-   [Scaleway Object\n    Storage](https://www.scaleway.com/en/object-storage/)\n\n## Dependencies\n\nTo use this plugin, these dependencies should be installed on the target\nKubernetes cluster:\n\n-   [CloudNativePG](https://cloudnative-pg.io/) 1.25 or newer (those\n    versions add the cnpg-i support).\n-   [Cert-Manager](https://cert-manager.io/)\n\n## How install and use the pgbackrest plugin\n\n### Install\n\nTo install and use this plugin, Kubernetes and CNPG users should:\n\n-   Build the Docker images and load them to a registry that is\n    accessible by the target Kubernetes cluster. You can build them with\n    the `make images` command, which will execute the appropriate docker\n    build commands.\n\n-   Install the plugin by applying the manifest located in the\n    `kubernetes` directory\n\n    ``` console\n    $ kubectl apply -f ./kubernetes\n    ```\n\n-   The installation can be verified by inspecting the presence and\n    status of the `pgbackrest-controller` deployment in the namespace\n    dedicated to the CloudNativePG operator (Eg: `cnpg-system`).\n\n### Initiate an instance with pgBackRest\n\nThe `examples` directory contains several pre-configured manifests\ndesigned to work with kind (Eg: the pull policy is set to `Never`).\nThese files may require modifications to run on other types of\nKubernetes clusters.\n\nTo use this plugin with a **Cluster**, CNPG users must:\n\n-   Create a secret named `pgbackrest-s3-secret` in the namespace of the\n    PostgreSQL Cluster, this secret must contain the `key` and\n    `secret-key` for the `S3` bucket.\n\n    Example:\n\n    ``` yaml\n    ---\n    apiVersion: v1\n    kind: Secret\n    metadata:\n      name: pgbackrest-s3-secret\n    type: Opaque\n    stringData:\n      key: \u003ckey_to_replace\u003e\n      key-secret: \u003csecret_to_replace\u003e\n    ```\n\n-   Adapt the PostgreSQL Cluster manifest by:\n\n    -   Adding the plugin definition `pgbackrest.dalibo.com` under the\n        `plugins` entry.\n    -   Specifying the `s3` parameters directly below the plugin\n        declaration\n\n    Example:\n\n    ``` yaml\n    ---\n    apiVersion: postgresql.cnpg.io/v1\n    kind: Cluster\n    metadata:\n      name: cluster-demo\n    spec:\n      instances: 1\n      plugins:\n        - name: cnpg-i-pgbackrest.dalibo.com\n          parameters:\n            s3-bucket: demo-pgbackrest\n            s3-endpoint: s3.fr-par.scw.cloud\n            s3-region: fr-par\n            s3-repo-path: /demo-pgbackrest-5/cluster-demo\n            stanza: pgbackrest\n      storage:\n        size: 1Gi\n    ```\n\n-   Then apply the manifest (`kubectl apply -f instance.yml`)\n\nIf it runs without errors, the Pod dedicated to the PostgreSQL Cluster\nshould have now two containers. One for the `postgres` service (which is\nthe default setup), an other one for the pgbackrest plugin, named\n`pgbackrest-plugin`. The injected container now holds the responsibility\nfor archiving the WALs and triggering backups when a backup request is\nmade.\n\n### Backup an instance\n\nThere are two ways to backup a PostgreSQL Cluster managed by the\nCloudNativePG operator:\n\n-   One shot backup, equivalent to running it by hand but through a\n    Backup object definition\n-   Scheduled backup, equivalent to defining a crontab entry to run a\n    backup periodically\n\nWhatever the kind of backup, users can list and see them with the\nappropriate kubectl command:\n\n``` console\n$ kubectl get backups.postgresql.cnpg.io\n```\n\n#### One shot backup\n\nBackup can be requested through a Backup object, using the default CNPG\nCRD backup definition. The pgbackrest plugin can be specified when\ndeclaring the backup object, for that the `method` should be set to\n`plugin` and the `pluginConfiguration.name` field to\n`pgbackrest.dalibo.com`.\n\nHere is a full example of a backup definition using the pgbackrest\nplugin:\n\n``` yaml\n---\napiVersion: postgresql.cnpg.io/v1\nkind: Backup\nmetadata:\n  name: backup-example\nspec:\n  method: plugin\n  cluster:\n    name: cluster-demo\n  pluginConfiguration:\n    name: cnpg-i-pgbackrest.dalibo.com\n```\n\n#### Scheduled backup\n\nA scheduled backup uses almost the same definition as a \"simple\" backup,\nonly the kind should be adapted (to `ScheduledBackup`). When using that\nkind of object, the schedule field (with a `crontab` annotation) should\nalso be defined under the specification (`spec`).\n\nHere is a full example of a scheduled backup definition using the\npgbackrest plugin:\n\n``` yaml\n---\napiVersion: postgresql.cnpg.io/v1\nkind: ScheduledBackup\nmetadata:\n  name: backup-example\nspec:\n  schedule: \"0 30 * * * *\"\n  method: plugin\n  cluster:\n    name: cluster-demo\n  pluginConfiguration:\n    name: cnpg-i-pgbackrest.dalibo.com\n```\n\n## How it works\n\nHere are some basic informations about how this plugin should work:\n\n-   When installing the plugin, a new deployment is created to run a Pod\n    for the controller (`pgbackrest-controller`) of our plugin in the\n    same namespace as the CNPG operator.\n\n-   The CNPG operator detects the plugin when a dedicated Kubernetes\n    Service (with some specific annotations) is created.\n\n-   Our specialized controller exposes the supported capabilities (at\n    least those required to manage the\n    [lifecycle](https://pkg.go.dev/github.com/cloudnative-pg/cnpg-i@v0.1.0/pkg/lifecycle)\n    of our CNPG instances) to the CNPG operator.\n\n-   When initializing a new Cluster configured with our plugin, the\n    pgBackRest controller will be called by the CloudNativePG operator.\n\n-   The plugin controller modifies the resources (Deployment / Pods /\n    Jobs) requested by the CNPG operator (this is done before requesting\n    the Kubernetes API), and inject some configuration if needed.\n\n    For our pgbackrest plugin, the controller inject a sidecar container\n    for `pgBackRest` within the PostgreSQL Pods. This sidecar container\n    executes a manager dedicated to `pgBackRest` (which expose the\n    required capabilities archive the WAL and backup the PostgreSQL\n    instance).\n\n-   Our newly created PostgreSQL instance will call the dedicated\n    `pgBackRest` manager (on the side container) when the archive\n    command is triggered.\n\n\u003chttps://github.com/cloudnative-pg/cnpg-i/blob/main/docs/protocol.md#cnpgi-wal-v1-WALCapability-RPC\u003e\n\n## Dev\n\nTo contribute and test the pgbackrest plugin a dedicated Kubernetes\ncluster with the CNPG operator is required. Contributors can use the dev\nversion of the CNPG operator and follow those steps to prepare the\nrequired environment.\n\n-   Clone the main CNPG operator repository :\n    `$ git clone https://github.com/cloudnative-pg/cloudnative-pg`\n\n-   Move to the newly created directory: `$ cd cloudenative-pg`\n\n-   Install the required dependencies (please follow the instruction\n    within the README.md file, at least you will need\n    [**kind**](https://kind.sigs.k8s.io/))\n\n-   Run a Kubernetes cluster with the development version of CNPG:\n    `$ ./hack/setup-cluster.sh create load deploy`\n\n-   Then install cert-manager, CNPG operator and the plugin will use\n    certificates to communicate securely:\n    `kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml`\n\nContributors and users can also refer to the CNPG documentation for more\ndetails on how it works and how to run the operator on\n[**Kind**](https://kind.sigs.k8s.io/).\n\nThe plugin can now be deployed on that Kubernetes cluster:\n\n-   Go back to the plugin directory `$ cd -`\n\n-   Build the container images for the plugin (One for the controller\n    and another one for the sidecar container):\n\n    ``` console\n    $ docker build --tag pgbackrest-controller:latest --target pgbackrest-controller  -f containers/pgbackrestPlugin.containers .\n    $ docker build --tag pgbackrest-sidecar:latest --target pgbackrest-sidecar -f containers/pgbackrestPlugin.containers .\n    ```\n\n-   The images should now be loaded into the registry dedicated the\n    development environment:\n\n    ``` console\n    $ kind load docker-image pgbackrest-{controller,sidecar}:latest --name pg-operator-e2e-v1-31-2\n    ```\n\n    If needed, it's possible to retrieve the name of the cluster by\n    running:\n\n    ``` console\n    $ kind get clusters\n    ```\n\n-   The plugin controller can now be deployed within the `cnpg-system`\n    namespace. For that the manifests on the `kubernetes` should be\n    applied:\n\n    ``` console\n    $ kubectl apply -f ./kubernetes/\n    ```\n\n-   Then the deployment can be verified by inspecting the objects\n    (Deployments, Pods,...) on the `cnpg-system` namespace. A\n    `pgbackrest-controller` deployment must be present. The plugin\n    controller should run on a dedicated Pod alongside the CNPG operator\n    Pod.\n\n### Executing E2E tests\n\nE2E Tests can be run automatically, for that the easiest approach is to\nuse [**kind**](https://kind.sigs.k8s.io/) and the appropriate make\ntarget:\n\n``` console\n$ make test-e2e\n```\n\nThat command will:\n\n-   Create a dedicated Kubernetes cluster (managed by **kind** and named\n    `e2e-cnpg-pgbackrest`)\n-   Build the container images for the pgbackrest plugin\n-   Load them on the Kubernetes Cluster\n-   Run the tests defined on `test/e2e/e2e_test.go`, which also install\n    the dependencies and our plugin.\n\nTo only run the tests (`test/e2e/e2e_test.go`), the `test-e2e-run-tests`\ntarget can be used:\n\n``` console\n$ make test-e2e-run-tests\n```\n\n## Resources\n\nThis plugin is inspired by the:\n\n- [Barman Cloud plugin](https://github.com/cloudnative-pg/plugin-barman-cloud)\n- [Hello World plugin](https://github.com/cloudnative-pg/cnpg-i-hello-world)\n\n\u003c!--\n    vim: spelllang=en spell\n  --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdalibo%2Fcnpg-plugin-pgbackrest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdalibo%2Fcnpg-plugin-pgbackrest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdalibo%2Fcnpg-plugin-pgbackrest/lists"}