https://github.com/iamkushvanth/real-time-data-analysis-using-kafka
In this project, you will execute an End-To-End Data Engineering Project on Real-Time Stock Market Data using Kafka. We are going to use different technologies such as Python, Amazon Web Services (AWS), Apache Kafka, Glue, Athena, and SQL.
https://github.com/iamkushvanth/real-time-data-analysis-using-kafka
athena aws aws-ec2 aws-s3 crawler glue kafka kafka-consumer python sql
Last synced: 5 days ago
JSON representation
In this project, you will execute an End-To-End Data Engineering Project on Real-Time Stock Market Data using Kafka. We are going to use different technologies such as Python, Amazon Web Services (AWS), Apache Kafka, Glue, Athena, and SQL.
- Host: GitHub
- URL: https://github.com/iamkushvanth/real-time-data-analysis-using-kafka
- Owner: iamkushvanth
- License: mit
- Created: 2025-02-05T06:11:11.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-11T17:35:41.000Z (over 1 year ago)
- Last Synced: 2025-02-23T16:49:39.110Z (over 1 year ago)
- Topics: athena, aws, aws-ec2, aws-s3, crawler, glue, kafka, kafka-consumer, python, sql
- Language: Jupyter Notebook
- Homepage:
- Size: 2.46 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Stock Market Kafka Real Time Data Engineering Project
## Introduction
In this project, you will execute an End-To-End Data Engineering Project on Real-Time Stock Market Data using Kafka.
We are going to use different technologies such as Python, Amazon Web Services (AWS), Apache Kafka, Glue, Athena, and SQL.
## Architecture
## Technology Used
- Programming Language - Python
- Amazon Web Service (AWS)
1. S3 (Simple Storage Service)
2. Athena
3. Glue Crawler
4. Glue Catalog
5. EC2
- Apache Kafka
