https://github.com/omerap12/kafka_assignment

https://github.com/omerap12/kafka_assignment

python3

Last synced: 20 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/omerap12/kafka_assignment
• Owner: omerap12
• Created: 2023-10-04T16:53:14.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-10-08T14:03:57.000Z (about 1 year ago)
• Last Synced: 2024-10-29T23:43:54.172Z (2 months ago)
• Topics: python3
• Language: Python
• Homepage:
• Size: 7.81 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Apache Kafka Demo

Link to docs: [Google sheets](https://drive.google.com/drive/folders/1WgUmNJzBZ12FlMsoAMJObUDGawCpizyN?usp=sharing)

## Overview

This project comprises three essential components:

1. **Pokemon API Fetcher (Producer):** This component acts as a producer within the cluster. The script employs the requests package to establish a connection with the Pokemon API (https://pokeapi.co/) and stream a comprehensive list of Pokemons to the Apache Kafka cluster.

2. **Kafka Consumer:** This program is designed to seamlessly connect to the Apache Kafka cluster, receiving real-time events and logging them to the standard output (stdout).

3. **Kafka Cluster:** The heart of the system, this cluster includes a Zookeeper and three Brokers, ensuring robust data management and distribution.

## Run Locally

Clone the project

```bash

  git clone https://github.com/omerap12/kafka_assignment.git

```

Go to the project directory

```bash

  cd kafka_assignment

```

Start the project

```bash

  docker compose up --build

```