{"id":19236888,"url":"https://github.com/crunchydata/postgres-realtime-demo","last_synced_at":"2025-06-17T12:36:33.909Z","repository":{"id":89933704,"uuid":"419782230","full_name":"CrunchyData/postgres-realtime-demo","owner":"CrunchyData","description":null,"archived":false,"fork":false,"pushed_at":"2021-10-24T22:13:57.000Z","size":21,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-23T13:49:38.271Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CrunchyData.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-21T15:41:26.000Z","updated_at":"2021-10-24T22:14:00.000Z","dependencies_parsed_at":"2023-05-30T14:15:44.844Z","dependency_job_id":null,"html_url":"https://github.com/CrunchyData/postgres-realtime-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CrunchyData/postgres-realtime-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrunchyData%2Fpostgres-realtime-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrunchyData%2Fpostgres-realtime-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrunchyData%2Fpostgres-realtime-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrunchyData%2Fpostgres-realtime-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CrunchyData","download_url":"https://codeload.github.com/CrunchyData/postgres-realtime-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrunchyData%2Fpostgres-realtime-demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260359616,"owners_count":22997327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T16:23:51.696Z","updated_at":"2025-06-17T12:36:28.876Z","avatar_url":"https://github.com/CrunchyData.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Building a Complex, Realtime Data Management Application with PostgreSQL](https://www.slideshare.net/jkatz05/build-a-complex-realtime-data-management-app-with-postgres-14) - Demo Code\n\nCompliments the [Building a Complex, Realtime Data Management Application with PostgreSQL](https://www.slideshare.net/jkatz05/build-a-complex-realtime-data-management-app-with-postgres-14) presentation:\n\n[https://www.slideshare.net/jkatz05/build-a-complex-realtime-data-management-app-with-postgres-14](https://www.slideshare.net/jkatz05/build-a-complex-realtime-data-management-app-with-postgres-14)\n\n## How to Use This Repository\n\nThis provides all of the examples that are used in the [Building a Complex, Realtime Data Management Application with PostgreSQL](https://www.slideshare.net/jkatz05/build-a-complex-realtime-data-management-app-with-postgres-14) presentation. You can follow along as a reference, or run the examples on your own.\n\nThis README provides a guide for setting up all the resources you need in a Kubernetes environment and guidance for running the demo. You do not necessarily need to use Kubernetes to set up the various components of the demo, but this is utilizing tools that make it easier to set up distributed environments.\n\nNote that the demo is broken up into several different parts. This README will provide guidance on where each demo is run in the context of the presentation.\n\n## General Notes\n\n- All of the examples are run in the `postgres-operator` namespace. You may need to create this:\n\n```\nkubectl create ns\n```\n\n- The example uses PostgreSQL 14. There is a dependency on `wal2json`. This is included in the `crunchy-postgres` container.\n- The database used in the application is called `realtime`.\n- When running the example application, you will need to have a logical replication slot defined in the database you are running the examples (e.g. `realtime`). For example:\n\n```\nSELECT * FROM pg_create_logical_replication_slot('schedule', 'wal2json');\n```\n\n## Installation\n\n### `postgres-realtime-demo`\n\nYou may need to make some changes to the `manifests` provided in this repository. As such, you should [fork this repository](https://github.com/CrunchyData/postgres-operator-examples/fork) and clone it to your host machine. For example:\n\n```\nYOUR_GITHUB_UN=\"\u003cyour GitHub username\u003e\"\ngit clone --depth 1 \"git@github.com:${YOUR_GITHUB_UN}/postgres-operator-examples.git\"\n```\n\n### [PGO](https://github.com/CrunchyData/postgres-operator), the open source [Postgres Operator](https://github.com/CrunchyData/postgres-operator) from [Crunchy Data](https://www.crunchydata.com/)\n\n[PGO](https://github.com/CrunchyData/postgres-operator), the [Postgres Operator](https://github.com/CrunchyData/postgres-operator) from [Crunchy Data](https://www.crunchydata.com), gives you a **declarative Postgres** solution that automatically manages your [PostgreSQL](https://www.postgresql.org) clusters.\n\nPGO is easy to set up. You can follow the [quickstart](https://access.crunchydata.com/documentation/postgres-operator/v5/quickstart/), or follow the instructions below:\n\n1. [Fork the Postgres Operator examples repository](https://github.com/CrunchyData/postgres-operator-examples/fork) and clone it to your host machine. For example:\n\n```\nYOUR_GITHUB_UN=\"\u003cyour GitHub username\u003e\"\ngit clone --depth 1 \"git@github.com:${YOUR_GITHUB_UN}/postgres-operator-examples.git\"\ncd postgres-operator-examples\n```\n\n2. Run `kubectl apply -k kustomize/install`\n\n3. Go back to this repository:\n\n```\ncd ../postgres-realtime-demo\n```\n\n4. Deploy the Postgres cluster example from the `manifests` directory:\n\n```\nkubectl apply -k manifests/postgres\n```\n\nYour PostgreSQL cluster will now be deployed.\n\n5. There are several ways to [connect to your Postgres](https://access.crunchydata.com/documentation/postgres-operator/v5/tutorial/connect-cluster/) cluster. The examples are already configured to connect automatically to your Postgres cluster.\n\nIf you want to manually connect to the cluster, you can do so with the following command:\n\n```\nkubectl exec -it -c database -n postgres-operator \\\n  $(kubectl get pods -o name \\\n    -n postgres-operator \\\n    --selector=postgres-operator.crunchydata.com/role=master,postgres-operator.crunchydata.com/cluster=realtime) \\\n  -- psql realtime\n```\n\nOr follow one of the [`psql` methods in the quickstart](https://access.crunchydata.com/documentation/postgres-operator/v5/quickstart/#connect-via-psql-in-the-terminal).\n\n### Kafka\n\nKafka can be installed using the [Strimzi](https://strimzi.io/) Kubernetes Operator.\n\n1. Install the Strimzi Operator into the `postgres-operator` namespace:\n\n```\nkubectl create -f 'https://strimzi.io/install/latest?namespace=postgres-operator' -n postgres-operator\n```\n\n2. Once the Strimzi Operator is installed, create a Kafka broker and topics:\n\n```\nkubectl apply -k manifests/kafka\n```\n\n### Application\n\n1. Before installing the application, you will need to ensure that you have created the schema needed to run the application in the database. You can do this by running the SQL in [demo5.sql](./examples/demo/demo5.sql) in the `realtime` database.\n\n2. You will also need to create a logical replication slot in the `realtime` database:\n\n```\nSELECT * FROM pg_create_logical_replication_slot('schedule', 'wal2json');\n```\n\n3. You may need to build the container and push it to your registry. You can do this from the `Makefile`. There are several variables you can tune to build the container for your registry:\n\n| Variable          | Usage                           | Default                  |\n|-------------------|---------------------------------|--------------------------|\n| `BUILDER`         | The container image builder     | `podman`                 |\n| `IMAGE_NAME`      | The name of the image           | `postgres-realtime-demo` |\n| `IMAGE_REGISTRY`  | The registry to store the image | `crunchydata`            |\n| `IMAGE_TAG`       | The tag of the image            | `latest`                 |\n\nFor example, to build an image for your own registry (giving it the name `your-registry`):\n\n```\nIMAGE_REGISTRY=your-registry make\n```\n\nTo build the image with docker:\n\n```\nBUILDER=docker make\n```\n\nTo build and push the image to your registry:\n\n```\nIMAGE_REGISTRY=your-registry make build push\n```\n\n4. Once the image is pushed to your registry, you can deploy the application.\n\nFirst, set the name of the image in the `manifests/app/kustomization.yaml` file. You can do this by uncommenting this section:\n\n```yaml\nimages:\n  - name: crunchydata/postgres-realtime-demo\n    newName: your-registry/postgres-realtime-demo\n```\n\nand setting `newName` to the name of your image. Assuming you kept the tag name of `latest`, you do not need to make any additional changes.\n\n## Running the Example\n\nOnce you have deployed PostgreSQL, Kafka, and the application, you can run the example found in [demo6.sql](./examples/demo/demo6.sql).\n\n## Advanced\n\n### Application Configuration\n\nThere are several environmental variables available for the application that you can set on the Pod.\n\n| Variable                           | Usage                           |\n|------------------------------------|---------------------------------|\n| `POSTGRES_URI`                     | A PostgreSQL URI. You can use this, or break out each Postgres connection parameter into its own setting. |\n| `POSTGRES_HOST`                    | The host representing a PostgreSQL instance.           |\n| `POSTGRES_PORT`                    | The port representing a PostgreSQL instance.           |\n| `POSTGRES_USER`                    | The PostgreSQL user to login as.           |\n| `POSTGRES_PASSWORD`                | The password of the PostgreSQL user.           |\n| `POSTGRES_DBNAME`                  | The PostgreSQL database to login to.           |\n| `STRIMZI_KAFKA_BOOTSTRAP_SERVERS`  | Reference to the `advertised-hostnames.config` that Strimzi creates for Kafka bootstrap servers. Use this, or the breakout Kafka settings. |\n| `KAFKA_BOOTSTRAP_SERVERS`          | A comma-separated list of Kafka bootstrap servers.            |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrunchydata%2Fpostgres-realtime-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrunchydata%2Fpostgres-realtime-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrunchydata%2Fpostgres-realtime-demo/lists"}