https://github.com/emanahmed55/weather-forecaster

A simple project of kafka and python to demonstrate basic concepts of producer, consumer and topic
https://github.com/emanahmed55/weather-forecaster

consumer kafka producer python topic weather

Last synced: 3 months ago
JSON representation

A simple project of kafka and python to demonstrate basic concepts of producer, consumer and topic

• Host: GitHub
• URL: https://github.com/emanahmed55/weather-forecaster
• Owner: EmanAhmed55
• Created: 2025-02-24T18:38:16.000Z (8 months ago)
• Default Branch: master
• Last Pushed: 2025-02-24T19:34:03.000Z (8 months ago)
• Last Synced: 2025-02-24T20:29:55.840Z (8 months ago)
• Topics: consumer, kafka, producer, python, topic, weather
• Language: Python
• Homepage:
• Size: 11.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Weather Forecaster

A simple project of kafka and python to demonstrate basic concepts of producer, consumer and topic

## Table of Contents

- [Installation](#installation)

- [Main idea of the project](#usage)

- [How to run the project](#contributing)

## Installation

To run this project, you need to have the JDK of java and following Python packages installed:



  
matplotlib

  
kafka-python



You can install these packages using pip:


pip install matplotlib kafka-python



## Usage



  


Producer


    It generates random weather data , including weatherstate , temperature and humidity.It sends this data as messages to kafka.

  

  


Topic


    This is where the weather data gets snt.It receives weather data messages from the producer and makes them available for consumers.

  

  


Consumer


    It fetches data from kafka topic and displays them in a real-time graph or chart.

  



## How to run the project

Steps to run the project:



  


Install kafka


    Extract the folder and copy the folder at any place ( here in D Partition)

    Go to kakafolder \config\server.properties.

    Edit log.dirs=D:/kafkafoldername/kafka-logs.

    Go to kafkafoldername\config\zookeeper.properties

    Edit dataDir=D:/kafkafoldername/zookeeper-data

    

  

  


Start zookeeper server


    Go to kafkafoldername\bin\windows and Open command propmpt and write "D:\kafka_2.12-3.7.0\bin\windows\zookeeper-server-start.bat D:\kafka_2.12-3.7.0\config\zookeeper.properties"

    Don't close the cmd window

  

  


Start kafka server in another new CMD window at the same previous path


    Write "D:\kafka_2.12-3.7.0\bin\windows\kafka-server-start.bat D:\kafka_2.12-3.7.0\config\server.properties"

    Don't close this CMD window

  

  


Create a kafka topic in  another new CMD window at the same previous path


    Write "kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic weather_forecasts"

    Don't close this CMD window

  

    


Create a kafka producer in the same previous CMD window at the same previous path


    Write "kafka-console-producer.bat --broker-list localhost:9092 --topic weather_forecasts"

    Don't close this CMD window

  

      


Create a kafka consumer in another new CMD window at the same previous path


    Write "kafka-console-consumer.bat --topic weather_forecasts --bootstrap-server localhost:9092 --from-beginning    D:\kafka_2.12-3.7.0\data\kafka"

    Don't close this CMD window

  
   

      


GO To visual studio code and run producer.py


  

  


GO To visual studio code and run consumer.py in another terminal


  

  


To stop the project 


    Go to the opened zookeeper server CMD window and write "D:\kafka_2.12-3.7.0\bin\windows\zookeeper-server-stop.bat D:\kafka_2.12-3.7.0\config\zookeeper.properties" Then Go to the opened kafka server CMD window and write "D:\kafka_2.12-3.7.0\bin\windows\kafka-server-stop.bat  D:\kafka_2.12-3.7.0\config\server.properties"