{"id":48610949,"url":"https://github.com/decodableco/examples","last_synced_at":"2026-04-09T00:34:25.896Z","repository":{"id":38685287,"uuid":"487037179","full_name":"decodableco/examples","owner":"decodableco","description":"🌟 Examples of use cases that utilize Decodable, as well as demos for related open-source projects such as Apache Flink, Debezium, and Postgres.","archived":false,"fork":false,"pushed_at":"2025-06-20T11:16:02.000Z","size":20852,"stargazers_count":76,"open_issues_count":4,"forks_count":34,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-06-20T12:29:07.828Z","etag":null,"topics":["datamesh","dataproducts","etl","flink","flink-stream-processing","kafka","pyflink","python","stream-processing"],"latest_commit_sha":null,"homepage":"https://decodable.co","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/decodableco.png","metadata":{"files":{"readme":"README.md","changelog":"change-streams/.gitignore","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-04-29T16:18:39.000Z","updated_at":"2025-06-20T11:16:06.000Z","dependencies_parsed_at":"2024-02-19T16:09:40.739Z","dependency_job_id":"b464b8f6-923f-4ff8-9be6-60cd4177be3f","html_url":"https://github.com/decodableco/examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/decodableco/examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/decodableco%2Fexamples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/decodableco%2Fexamples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/decodableco%2Fexamples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/decodableco%2Fexamples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/decodableco","download_url":"https://codeload.github.com/decodableco/examples/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/decodableco%2Fexamples/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31580174,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datamesh","dataproducts","etl","flink","flink-stream-processing","kafka","pyflink","python","stream-processing"],"created_at":"2026-04-09T00:34:25.354Z","updated_at":"2026-04-09T00:34:25.871Z","avatar_url":"https://github.com/decodableco.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Decodable Examples\n\n![Real-time Data: Simplified](https://img.shields.io/badge/real%E2%80%93time%20data-simplified-%232F74F9?labelColor=%2306091A\u0026link=https%3A%2F%2Fdecodable.co)\n![GitHub last commit](https://img.shields.io/github/last-commit/decodableco/examples)\n![GitHub License](https://img.shields.io/github/license/decodableco/examples)\n![Static Badge](https://img.shields.io/badge/we%20love-apache%20flink-%23E6526F?logo=apacheflink)\n\n![A squirrel holding a laptop that says \"Examples repo\"](images/examples_repo.webp)\n\n## Introduction\n\nThis repository contains examples of use cases that utilize Decodable streaming solution as well as demos for related open-source projects such as Apache Flink, Debezium, and Postgres.\n\nExamples are presented \"as-is\" and are maintained on a best effort basis. PRs for updating existing (or adding new) examples are welcome!\n\nFor help with any of the examples, or using Decodable in general, please [join our Slack group](https://join.slack.com/t/decodablecommunity/shared_invite/zt-uvow71bk-Uf914umgpoyIbOQSxriJkA).\n\n## About Decodable\n\n_Decodable radically simplifies real-time data, making it easier to access the freshest, high-quality data. Reduce infrastructure overhead, connect data sources, transform, and deliver data reliably to any destination._\n\n_Learn more [here](https://decodable.co), and [sign up for a free trial](https://app.decodable.co/-/accounts/create) today!_\n\n## Contents\n\n### Stream Processing Techniques\n\n| Example                                     | Description                                                                                                    |\n|---------------------------------------------|----------------------------------------------------------------------------------------------------------------|\n| [Change Streams](change-streams)            | Using change streams to build materialized views in Postgres                                                   |\n| [XML Processing](xml)                       | Parse XML and transform to JSON                                                                                |\n| [Masking](masking)                          | Ways to mask data                                                                                              |\n| [Array Aggregation (1)](array-agg-postgres) | Demonstrating how to aggregate the elements of the many side of 1:n join into an array with data from Postgres |\n| [Array Aggregation (2)](array-agg)          | Using the `array_agg()` UDF for denormalizing data in a pipeline from MySQL to OpenSearch                      |\n\n### Data Pipelines\n\n| Example                                                 | Description                                                                            |\n|---------------------------------------------------------|----------------------------------------------------------------------------------------|\n| [Opinionated Data Pipelines](opinionated-pipelines)     | Building data pipelines with schema on write streams.                                  |\n| [Postman](postman)                                      | Building data pipelines with Postman.                                                  |\n| [Postgres to Snowflake](postgres-to-snowflake-with-cdc) | Getting data from Postgres to Snowflake using Decodable.                               |\n| [Flink CDC](flink-cdc)                                  | Trying out Flink CDC and comparing it to Flink SQL.                                    |\n| [Flink SQL and Custom Pipelines](sql-cupi-hybrid)       | Bridging Flink SQL and Custom Java Pipelines with the Decodable SDK.                   |\n| [Testing Custom Pipelines on Decodable](testing-cupi)   | How to write more modular Flink jobs with pluggable components to improve testability. |\n\n\n### PyFlink\n\n_Decodable provides a managed PyFlink service. Learn more [here](https://docs.decodable.co/pipelines/create-pipelines-using-your-own-apache-flink-jobs.html#_create_a_custom_pipeline_python)._\n\n| Example                                                         | Description                                                                             |\n|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------|\n| [PyFlink](pyflink)                                              | Running a basic PyFlink job on Kubernetes                                               |\n| [PyFlink on Decodable](pyflink-decodable)                       | Running a PyFlink job as a Custom Pipeline on Decodable                                 |\n| [PyFlink and MongoDB Vector Search](pyflink-vector-embeddings)  | End-to-end example for PyFlink Vector Ingestion on Decodable with MongoDB Vector Search |\n| [PyFlink Intro](pyflink-intro)                                  | A Hands-On Introduction to PyFlink                                                      |\n\n### Integrations\n\n| Example                                                                    | Description                                                                                                                                                                                                          |\n|----------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|\n| [Apache Druid](druid)                                                      | Sending COVID-19 data to Decodable using its REST API, cleaning it with SQL, and then sending it to Apache Druid      |\n| [Apache Kafka / Flink / Iceberg](kafka-iceberg/apache-flink)               | Integrating Apache Kafka with Apache Iceberg through Apache Flink. _As presented at Kafka Summit London 2024_         |\n| [Apache Kafka / Flink / Iceberg](kafka-iceberg/decodable) (with Decodable) | Streaming from Apache Kafka to Apache Iceberg with Decodable                                                          |\n| [Apache Kafka Upsert connector](kafka-upsert/)                             | Explaining the difference between the Flink Kafka and Kafka Upsert connectors                                         |\n| [Apache Kafka mTLS](mtls)                                                  | Installing Apache Kafka on EC2 and configuring it with mTLS                                                           |\n| [Apache Kafka with ngrok](kafka-ngrok)                                     | Using Docker Compose for running Apache Kafka locally, accessible from the internet using ngrok                       |\n| [Apache Kafka](kafka2s3)                                                   | Installing Apache Kafka on EC2 and writing to S3 with Decodable                                                       |\n| [Apache Pinot](pinot)                                                      | Transforming osquery logs to Apache Pinot and Superset                                                                |\n| [AsyncAPI](asyncapi)                                                       | Publishing Data Products with AsyncAPI                                                                                |\n| [Confluent](confluent)                                                     | Clickstream from Confluent Cloud joined with CDC user data from Postgres                                              |\n| [Delta Lake / Flink](flink-delta-lake)                                     | Writing to Delta Lake with Apache Flink                                                                               |\n| [GitHub Webhooks](github-webhooks)                                         | Processing GitHub Webhook events using the Decodable REST source connector                                            |\n| [OSQuery Routing](osquery)                                                 | Routing OSQuery logs with SQL                                                                                         |\n| [Redpanda](redpanda)                                                       | Reading and writing data to Redpanda from Flink                                                                       |\n| [S3 Events in a Lambda Function](s3events/)                                | Configuring an S3 bucket with a Lambda notification to send data to Kinesis to be processed in Decodable              |\n| [Tinybird](tinybird)                                                       | Writing data to Tinybird from Decodable                                                                               |\n\n### Changed Data Capture (CDC)\n\n| Example                                                          | Description                                                                                                          |\n|------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|\n| [MSSQL CDC](mssql_cdc/)                                          | Enabling MSSQL in Docker with CDC, reading from it with Debezium, writing change events into AWS Kinesis             |\n| [Oracle CDC](oracle_cdc/)                                        | Configuring Oracle AWS RDS with LogMiner, reading from it with Debezium, writing change events into AWS Kinesis      | \n| [DynamoDb CDC](dynamodb_cdc/)                                    | Configure DynamoDB to send change data to Kinesis, reading changes into Decodable for transformation or replication. |\n| [Logical Decoding Message Examples](postgres-logical-decoding)   | How to retrieve logical decoding messages from the Postgres WAL                                                      |\n| [Logical Replication on Postgres 16 Stand-By Servers](postgres-logical-replication-standby) | How to use logical replication on Postgres 16 stand-by servers                            |\n| [Postgres 17 Fail-Over Slots](failover-slots)                    | How to use fail-over slots with Postgres 17                                                                          |\n| [Transactional CDC Event Aggregation](tx-aware-cdc-buffering)    | Aggregating Change Data Capture Events based on Transactional Boundaries                                             |\n\n### Flink SQL\n\n| Example                                               | Description |\n|-------------------------------------------------------|-------------|\n| [Flink SQL Troubleshooting](troubleshooting-flinksql) | A set of Docker Compose environments for demonstrating various Flink SQL troubleshooting scenarios (see [related blog](https://www.decodable.co/blog/flink-sql-misconfiguration-misunderstanding-and-mishaps?utm_medium=github\u0026utm_source=examples_repo\u0026utm_campaign=blog\u0026utm_content=troubleshooting-flinksql))|\n\n### Decodable tools\n\n| Example                                               | Description |\n|-------------------------------------------------------|-------------|\n| [Decodable CI/CD](declarative-cicd) | An example of using Decodable with GitHub Actions|\n| [Decodable CLI Docker image](cli-docker) | An example Dockerfile for running the Decodable CLI under Docker.|\n\n### Kubernetes\n\n| Example                                               | Description |\n|-------------------------------------------------------|-------------|\n| [Fink on Kubernetes](flink-on-kubernetes) | An example of running Flink via the Flink Kubernetes Operator|\n\n## License\n\nThis code base is available under the Apache License, version 2.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecodableco%2Fexamples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdecodableco%2Fexamples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecodableco%2Fexamples/lists"}