{"id":18602122,"url":"https://github.com/build-on-aws/real-time-streaming-analytics-application-using-apache-kafka","last_synced_at":"2026-05-18T10:11:00.425Z","repository":{"id":178369158,"uuid":"650651373","full_name":"build-on-aws/real-time-streaming-analytics-application-using-apache-kafka","owner":"build-on-aws","description":"Sample code repository to build a real-time streaming analytics application using Apache Kafka on AWS","archived":false,"fork":false,"pushed_at":"2023-06-28T04:31:27.000Z","size":53,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-10-20T06:09:58.965Z","etag":null,"topics":["amazon-ecs","aws","aws-opensearch","flink","kafka"],"latest_commit_sha":null,"homepage":"https://www.buildon.aws/tutorials/building-real-time-streaming-analytics-application-on-apache-kafka","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/build-on-aws.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-07T14:11:55.000Z","updated_at":"2024-06-17T07:20:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"a17c8419-321b-4deb-b4e3-e22da85eb77e","html_url":"https://github.com/build-on-aws/real-time-streaming-analytics-application-using-apache-kafka","commit_stats":null,"previous_names":["build-on-aws/real-time-streaming-analytics-application-using-apache-kafka"],"tags_count":0,"template":false,"template_full_name":"amazon-archives/__template_MIT-0","purl":"pkg:github/build-on-aws/real-time-streaming-analytics-application-using-apache-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/build-on-aws","download_url":"https://codeload.github.com/build-on-aws/real-time-streaming-analytics-application-using-apache-kafka/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33174091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-18T09:27:30.708Z","status":"ssl_error","status_checked_at":"2026-05-18T09:27:28.300Z","response_time":71,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-ecs","aws","aws-opensearch","flink","kafka"],"created_at":"2024-11-07T02:10:23.694Z","updated_at":"2026-05-18T10:11:00.402Z","avatar_url":"https://github.com/build-on-aws.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Real-time streaming analytics application using Apache Kafka\n\nIn today’s fast-paced digital world, real-time streaming analytics has become increasingly important as companies require to understand what customers, application and products are doing right now and react promptly. For example, companies need to analyse data in real-time to continuously monitor an application to ensure high service uptime and personalize promotional offers and product recommendations to customers. \n\nThis repository contains the sample code to build a real-time streaming analytics application using Apache Kafka on AWS. It forms the basis of the following tutorial:\n* [Learn how to build a real-time streaming analytics application on Apache Kafka](https://www.buildon.aws/tutorials/building-real-time-streaming-analytics-application-on-apache-kafka)\n\nThe source code builds Apache Flink application and creates the following resources on AWS using the provided AWS CloudFormation:\n* Amazon OpenSearch Cluster\n* Amazon ECS Cluster and a Task definition\n* Amazon Kinesis Data Analytics streaming application\n* Amazon EC2 Instance to serve as Kafka client\n* Amazon EC2 Instance to serve as Nginx proxy \n* Security groups and AWS IAM roles \n\n## Requirements\n\n* [Java 11+](https://openjdk.org/install)\n* [Maven 3.8.6+](https://maven.apache.org/download.cgi)\n* [AWS Account](https://aws.amazon.com/resources/create-account)\n\n## ⚙️ Building the Apache Flink application\n\nFollow the following steps to build the Apache Flink application:\n\n1. Install the following dependencies:\n\n- [Java 11+](https://openjdk.java.net)\n- [Apache Maven](https://maven.apache.org)\n\n2. Move to the right directory:\n\n```bash\ncd flink-clickstream-consumer\n```\n\n3. Build the Apache Flink application file:\n\n```bash\nmvn clean package\n```\n\n💡 A file named `flink-clickstream-consumer/target/ClickStreamProcessor-1.0.jar` will be created. This is your Apache Flink application.\n\n## 🌩 Deploying into AWS\n\nOnce you created built the Flink application, you can deploy the deploy the required resources on AWS to start building your real-time streaming analytics application. The required steps are detailed in the following tutorial:\n* [Learn how to build a real-time streaming analytics application on Apache Kafka](https://www.buildon.aws/tutorials/building-real-time-streaming-analytics-application-on-apache-kafka)\n\n\n## Security\n\nSee [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.\n\n## License\n\nThis project is licensed under the MIT-0 License. See the [LICENSE](./LICENSE) file.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuild-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuild-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuild-on-aws%2Freal-time-streaming-analytics-application-using-apache-kafka/lists"}