{"id":22204367,"url":"https://github.com/innogames/flink-real-time-crm","last_synced_at":"2025-06-18T00:34:25.390Z","repository":{"id":207936333,"uuid":"717821923","full_name":"innogames/flink-real-time-crm","owner":"innogames","description":"Demo Flink and Kafka project to show how to react on tracking events in real-time and trigger offer for customer engagement based on campaign configurations. The project also utilizes the Broadcast State Pattern in order to update the rules (campaigns) at runtime without restarting the project, using a dedicated, low-frequency, Kafka topic.","archived":false,"fork":false,"pushed_at":"2024-11-19T18:11:09.000Z","size":1277,"stargazers_count":14,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-16T04:28:59.437Z","etag":null,"topics":["apacheflink","apachekafka","flink","flink-examples","flink-stream-processing","flink-streaming","java","kafka","stream-processing"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/innogames.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-12T17:52:45.000Z","updated_at":"2025-01-14T08:14:59.000Z","dependencies_parsed_at":"2023-11-18T15:48:05.628Z","dependency_job_id":"21c60dd7-75be-4a00-96f6-e1f0bf148966","html_url":"https://github.com/innogames/flink-real-time-crm","commit_stats":null,"previous_names":["innogames/flink-real-time-crm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/innogames/flink-real-time-crm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innogames%2Fflink-real-time-crm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innogames%2Fflink-real-time-crm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innogames%2Fflink-real-time-crm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innogames%2Fflink-real-time-crm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/innogames","download_url":"https://codeload.github.com/innogames/flink-real-time-crm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innogames%2Fflink-real-time-crm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260462348,"owners_count":23013082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apacheflink","apachekafka","flink","flink-examples","flink-stream-processing","flink-streaming","java","kafka","stream-processing"],"created_at":"2024-12-02T17:17:04.669Z","updated_at":"2025-06-18T00:34:20.361Z","avatar_url":"https://github.com/innogames.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Real-time Customer Engagement in Gaming Using Kafka and Flink\n\n## Description\nDemo Flink project to show how to react on interaction events in real-time and trigger CRM campaigns for customer\nengagement based on campaign configurations.\n\nThe project also utilizes the [Broadcast State Pattern](https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/fault-tolerance/broadcast_state/) \nin order to update the rules (_campaigns_) at runtime without restarting the project, using a dedicated, low-frequency, Kafka topic.\n\n## Business Logic\n\n### Customer Engagement Campaigns\nEach customer engagement campaign runs for a specific product (game) and can have a start and end date.\nAdditionally, a campaign can be enabled and disabled and has a unique ID.\n\nThe key element of a campaign is the filter. The filter is a JavaScript function that is executed for each event and\nmust return a boolean value. With this filter, we can define the conditions that must be met in order to trigger the\ncampaign based on an event.\n\nCampaigns can be configured by simply writing the JSON representation to the `crm-campaigns` Kafka topic.\n\n### Trigger Campaigns\nInteraction events are written as JSON to the `events-valid` Kafka topic. For each event, the Flink Application will\nevaluate all active campaigns and check if the filter matches. If that is the case, a trigger is written to the\n`crm-triggers` Kafka topic, indicating that the user is eligible to receive the offer connected to the campaign.\n\nPlease refer to the local execution example to get a better understanding of how the system works.\n\n## Local Execution\nRequirements:\n* Docker\n* (_optional_) kcat (see: https://github.com/edenhill/kcat)\n\n### Configuration\nDefault configuration is defined in `src/main/resources/config.json`. You can override it by creating a\ncustom config file and pass the path as a parameter named `config` when running the Flink job.\n\n### Start Kafka\nJust run the following command in the project dir to start ZooKeeper and Kafka as defined in `docker-compose.yml`:\n```bash\ndocker-compose up -d\n```\n\n### Create topics\nWe are using `kcat` to interact with Kafka. We start the producer and exit (CTRL+D) again to just create the\nnecessary topics:\n```bash\nkcat -b localhost:9093 -t events-valid -P\nkcat -b localhost:9093 -t crm-campaigns -P\nkcat -b localhost:9093 -t crm-triggers -P\n```\n\nThen use `kcat` to list topics:\n```bash\nkcat -b localhost:9093 -L | grep topic\n```\n\nThe output should include the three topics above:\n```\nMetadata for all topics (from broker 1: localhost:9093/1):\n 4 topics:\n  topic \"crm-triggers\" with 1 partitions:\n  topic \"events-valid\" with 1 partitions:\n  topic \"crm-campaigns\" with 1 partitions:\n  topic \"__consumer_offsets\" with 50 partitions:\n```\n\n### Run Flink job\nRun `com.innogames.analytics.rtcrm.App` and use `kcat` to create a campaign, send and event and check the trigger:\n\n### Open Flink Web UI\nOpen http://localhost:8081/ to see the Flink Web UI.\n\n### Create campaign\n```bash\nkcat -b localhost:9093 -t crm-campaigns -P\n```\n```\n{\"campaign_id\":4711,\"enabled\":true,\"game\":\"foe\",\"event_name\":\"fight\",\"start_date\":\"2023-09-01T10:00:00.00Z\",\"end_date\":\"2023-12-01T10:00:00.00Z\",\"filter\":\"function(event) { return event.getData().get('result') === 'loss' }\"}\n```\n\n### Send event\n```bash\nkcat -b localhost:9093 -t events-valid -P\n```\n```\n{\"schema_version\":\"2023110110010101\",\"event_id\":\"7a8d0614-4e8e-4464-b439-36e31441a040\",\"system_type\":\"game\",\"system_name\":\"foe-backend\",\"game\":\"foe\",\"market\":\"de\",\"player_id\":12345,\"event_type\":\"progress\",\"event_name\":\"fight\",\"event_scope\":\"user\",\"created_at\":\"2023-11-20T11:00:00.00Z\",\"received_at\":\"2023-11-20T11:00:02.00Z\",\"hostname\":\"foe-backend-1.ig.com\",\"context\":{},\"data\":{\"result\":\"loss\"}}\n```\n\n### Consume trigger\n```bash\nkcat -b localhost:9093 -t crm-triggers -C -o beginning -u\n```\n\n## Demo UI\n\n![Demo UI](doc/flink-real-time-crm-ui.png)\n\nWhen Kafka and Flink are running, you can also use the dedicated [Demo UI project](https://github.com/innogames/flink-real-time-crm-ui) to\ninteract with the system via a web frontend.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnogames%2Fflink-real-time-crm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finnogames%2Fflink-real-time-crm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnogames%2Fflink-real-time-crm/lists"}