{"id":16367820,"url":"https://github.com/nkonev/flink-pipe","last_synced_at":"2026-02-25T08:06:09.949Z","repository":{"id":220204204,"uuid":"751013538","full_name":"nkonev/flink-pipe","owner":"nkonev","description":"A [single|uber|fat] jar standalone Apache Flink connected to PostgreSQL via Ververica CDC connector. Usage Flink SQL to replicate data from PostgreSQL to Elasticsearch/something else.","archived":false,"fork":false,"pushed_at":"2024-05-26T00:22:35.000Z","size":80,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-01T02:41:55.287Z","etag":null,"topics":["cdc","debezium","debezium-connector","debezium-embedded","elastic","elasticsearch","flink","flink-cdc","flink-sql","flink-stream-processing","flink-streaming","postgres","postgresql","replication","ververica-connector"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nkonev.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-31T19:06:03.000Z","updated_at":"2024-05-26T00:22:38.000Z","dependencies_parsed_at":"2024-02-11T05:20:00.258Z","dependency_job_id":"5ac0bb4b-d483-4317-877e-6ae12a96f157","html_url":"https://github.com/nkonev/flink-pipe","commit_stats":null,"previous_names":["nkonev/flink-pipe"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nkonev/flink-pipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nkonev%2Fflink-pipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nkonev%2Fflink-pipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nkonev%2Fflink-pipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nkonev%2Fflink-pipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nkonev","download_url":"https://codeload.github.com/nkonev/flink-pipe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nkonev%2Fflink-pipe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29815020,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T05:36:42.804Z","status":"ssl_error","status_checked_at":"2026-02-25T05:36:31.934Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdc","debezium","debezium-connector","debezium-embedded","elastic","elasticsearch","flink","flink-cdc","flink-sql","flink-stream-processing","flink-streaming","postgres","postgresql","replication","ververica-connector"],"created_at":"2024-10-11T02:50:57.267Z","updated_at":"2026-02-25T08:06:09.923Z","avatar_url":"https://github.com/nkonev.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# What is it ?\nIt is Apache Flink with [Ververica Postgres CDC](https://github.com/ververica/flink-cdc-connectors) built as one fat|uber jar to be launched via `java -jar` as a standalone application.\n\nThe purpose is - declaratively (via Flink SQL) replicate data from PostgreSQL to any other datastores.\n\nA docker image of PostgreSQL 16 configured by Debuzium  included to be run via `docker-compose up -d`.\n\nSample data is placed to the predefined by Postgres Docker `docker-entrypoint-initdb.d` directory.\n\nConcrete example to replicate data to Elasticsearch 7 derived from [example](https://www.ververica.com/blog/how-to-guide-build-streaming-etl-for-mysql-and-postgres-based-on-flink-cdc) is in branch `elastic`.\n\n# Build\n```\n./gradlew clean shadowJar\n```\n\n# Run\n```\njava -jar /home/nkonev/javaWorkspace/flink-pipe/build/libs/flink-pipe-0.1.0-all.jar\n```\n\n# Links\n* [Streaming SQL with Apache Flink: A Gentle Introduction](https://blog.rockthejvm.com/flink-sql-introduction/)\n* [How-to guide: Build Streaming ETL for MySQL and Postgres based on Flink CDC](https://www.ververica.com/blog/how-to-guide-build-streaming-etl-for-mysql-and-postgres-based-on-flink-cdc)\n* [Streaming ETL for MySQL and Postgres with Flink CDC](https://ververica.github.io/flink-cdc-connectors/release-3.0/content/quickstart/mysql-postgres-tutorial.html)\n* [Потоковый захват изменений из PostgreSQL/MySQL с помощью Apache Flink](https://habr.com/ru/companies/neoflex/articles/567930/)\n* [Как использовать Spring в качестве фреймворка для Flink-приложений](https://habr.com/ru/companies/ru_mts/articles/775970/)\n* [debezium/container-images](https://github.com/debezium/container-images/tree/main/examples/postgres)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnkonev%2Fflink-pipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnkonev%2Fflink-pipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnkonev%2Fflink-pipe/lists"}