https://github.com/devops-360-online/go-with-me-kafka
a comprehensive guide to understanding Apache Kafka, including its architecture, installation steps, and a practical use case demonstration .
https://github.com/devops-360-online/go-with-me-kafka
kafka kraft kubernetes
Last synced: about 1 month ago
JSON representation
a comprehensive guide to understanding Apache Kafka, including its architecture, installation steps, and a practical use case demonstration .
- Host: GitHub
- URL: https://github.com/devops-360-online/go-with-me-kafka
- Owner: devops-360-online
- Created: 2024-07-06T06:33:43.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-03-28T09:23:30.000Z (about 1 month ago)
- Last Synced: 2025-03-28T10:29:08.186Z (about 1 month ago)
- Topics: kafka, kraft, kubernetes
- Language: Python
- Homepage:
- Size: 2.41 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Kafka-Guide
A comprehensive guide to understanding Apache Kafka, including its architecture, installation steps, and a practical use case demonstration.
Kafka is a message broker using a commit log, often associated with infrastructure data, microservices, and event-driven architectures.
## Architecture
Schema: 
### Kafka Cluster Components
- **Broker**: Acts as a node with SSD storage.
- **Partition**: Ensures high availability and fault tolerance of data. We could have multiple partitions replicated across brokers (Leader and Followers).
- **Topic**: Similar to a table in a database, it defines a data stream.
- **Offset**: The unique identifier for a message/event within a topic.
- **Producer**: Publishes messages to Kafka.
- **Consumer**: Reads messages from a topic.
- **Consumer Group**: Multiple instances of the same application (e.g., App C with 3 instances) can consume messages from specific partitions with coordination.
### Key Features
- **Replications**: Ensures data redundancy and availability.
- **Partitions**: Allows for data distribution and parallel processing.
- **Scalability**: Easily scalable to handle large volumes of data.
- **Performance**: Optimized for high-throughput and low-latency message processing.
Kafka operates asynchronously, and consumers can reread messages since events are stored with offsets.
### Partitioning
Partitions logically divide topics and are distributed across the cluster. Each message within a partition is associated with a unique key.
### Alternatives to Kafka
- RabbitMQ
- ActiveMQ
- ZeroMQ
## Installing Kafka with Kubernetes
Instructions for installing Kafka using Kubernetes will be provided in this section.
## Real-World Example
Tracking a bus to see its real-time position using Flask, JavaScript, and leafletjs .