{"id":22357126,"url":"https://github.com/gordonmurray/apache_flink_and_docker_compose","last_synced_at":"2025-07-30T10:33:01.829Z","repository":{"id":205190901,"uuid":"713603039","full_name":"gordonmurray/apache_flink_and_docker_compose","owner":"gordonmurray","description":"Running Apache Flink containers using Docker Compose is a convenient way to get up and running to try out some Flink workloads.","archived":false,"fork":false,"pushed_at":"2023-11-15T21:25:31.000Z","size":26189,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-30T08:42:31.750Z","etag":null,"topics":["apache-flink","cdc","mariadb","redis"],"latest_commit_sha":null,"homepage":"https://gordonmurray.com/data/2023/11/02/deploying-flink-cdc-jobs-with-docker-compose.html","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gordonmurray.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-02T21:27:11.000Z","updated_at":"2024-11-05T19:29:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"bc9095e2-ebf9-4c50-8b13-b7f88f2a5658","html_url":"https://github.com/gordonmurray/apache_flink_and_docker_compose","commit_stats":null,"previous_names":["gordonmurray/apache_flink_and_docker_compose"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gordonmurray%2Fapache_flink_and_docker_compose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gordonmurray%2Fapache_flink_and_docker_compose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gordonmurray%2Fapache_flink_and_docker_compose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gordonmurray%2Fapache_flink_and_docker_compose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gordonmurray","download_url":"https://codeload.github.com/gordonmurray/apache_flink_and_docker_compose/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228124578,"owners_count":17873170,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-flink","cdc","mariadb","redis"],"created_at":"2024-12-04T14:13:16.592Z","updated_at":"2024-12-04T14:13:17.131Z","avatar_url":"https://github.com/gordonmurray.png","language":null,"readme":"# Apache Flink containers using Docker Compose\nRunning Apache Flink containers using Docker Compose is a convenient way to get up and running to try out some Flink workloads.\n\nThere is a post to go along with this project here [https://gordonmurray.com/data/2023/11/02/deploying-flink-cdc-jobs-with-docker-compose.html](https://gordonmurray.com/data/2023/11/02/deploying-flink-cdc-jobs-with-docker-compose.html)\n\nAssuming docker compose is installed you can start the containers using the following command in the same folder as the docker compose file:\n\n```\ndocker compose up -d\n```\n\nThis will start the containers in the background and you can check that the containers are running using\n\n```\ndocker ps\n```\n\nYou should see something like the following showing a jobmanager and 2 task managers.\n\n```\nCONTAINER ID   IMAGE             COMMAND                  CREATED          STATUS          PORTS                                                 NAMES\neb87408560be   flink:1.17.1      \"/docker-entrypoint.…\"   35 minutes ago   Up 35 minutes   6123/tcp, 8081/tcp                                    apache_flink_and_docker_compose-taskmanager-2\n565fd52d250a   flink:1.17.1      \"/docker-entrypoint.…\"   35 minutes ago   Up 35 minutes   6123/tcp, 8081/tcp                                    apache_flink_and_docker_compose-taskmanager-1\n0b3e3eaa5c06   flink:1.17.1      \"/docker-entrypoint.…\"   35 minutes ago   Up 35 minutes   6123/tcp, 0.0.0.0:8081-\u003e8081/tcp, :::8081-\u003e8081/tcp   jobmanager\n```\n\nTo start adding some work to Flink you can access the Flink console using the following command and from there you can try out various jobs like creating tables.\n\n```bash\ndocker exec -it jobmanager /opt/flink/bin/sql-client.sh\n```\n\nTo run a Flink job to CDC from a Mariadb container and sink some data in to Redis, use the following:\n\n```bash\ndocker exec -it jobmanager /opt/flink/bin/sql-client.sh embedded -f job.sql\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgordonmurray%2Fapache_flink_and_docker_compose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgordonmurray%2Fapache_flink_and_docker_compose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgordonmurray%2Fapache_flink_and_docker_compose/lists"}