Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with kafka-producer

A curated list of projects in awesome lists tagged with kafka-producer .

https://github.com/rafaelrok/ms-order-pattern-saga-orchestrator

Projeto de microservices no padrão Saga de maneira de gerenciar a consistência de dados entre microservices em cenários de transação distribuída1. Uma saga é uma sequência de transações que atualiza cada serviço e publica uma mensagem ou evento para disparar a próxima etapa de transação utilizando o kafka.

api-rest docker docker-compose java kafka kafka-consumer kafka-producer micro postgresql redpanda spring-boot

Last synced: 11 Jan 2025

https://github.com/kamon-io/kamon-kafka

Kamon Integration for Kafka(Producers|Consumers|Streams)

kafka kafka-client kafka-consumer kafka-producer kafka-streams kamon metrics tracing

Last synced: 11 Nov 2024

https://github.com/danielprinz/kafka-streaming-vertx

Vert.x Kafka clients playground

kafka kafka-consumer kafka-producer vertx

Last synced: 19 Dec 2024

https://github.com/tashi-2004/fma-a-dataset-for-music-analysis

🎶 Scripts for music feature analysis, model training, and real-time recommendation using Apache Kafka. Extract features with Librosa 🎹, store them in MongoDB 🗄️, and process the data with Apache Spark ⚡. A 🌐 web interface 💻✨ is also included. Contributors: Tashfeen Abbasi 👤, Laiba Mazhar 👤, and Rafia Khan 👤.

html kafka kafka-consumer kafka-producer kafka-streaming linux mongodb mongodb-compass python3 spark ubuntu web-application

Last synced: 03 Dec 2024

https://github.com/tashi-2004/apache-kafka-and-frequent-item-sets

This Bash script 🌟 automates the setup and execution of a data processing pipeline 🔄 using Apache Kafka 📦 and Python scripts 🐍, ensuring fault tolerance 🛡️ and streamlined management 🚀 of Kafka-based data pipelines 📊💡.

kafka kafka-consumer kafka-producer kafka-streams mongodb mongodbcompass

Last synced: 03 Dec 2024

https://github.com/math-ferreira/apache-kafka-series

Projeto desenvolvido em Java, para consolidar conhecimentos adquiridos com o Kafka em curso realizado pela Udemy. Neste Projeto foi desenvolvido Productor e Consumer integrados com a API do Twitter

java kafka kafka-consumer kafka-producer

Last synced: 13 Jan 2025

https://github.com/dbgjerez/workshop-amq-streams

An example with AMQ Streams, SQL Database and GitOps

amq-streams kafka kafka-consumer kafka-producer kafkaconnect strimzi

Last synced: 15 Oct 2024

https://github.com/ciphertron/kafka-streamer

A microservice that uses Kafka to listens to an event involving order creation and then insert the product orders to a PostgreSQL database.

docker docker-c flask kafka kafka-consumer kafka-producer postgresql python-kafka zook

Last synced: 06 Jan 2025

https://github.com/wbarnha/bachelor

Producing large volumes of data over Kafka made easy!

kafka kafka-producer

Last synced: 25 Nov 2024

https://github.com/NashTech-Labs/kafka-unit-testing.g8

A Sample application of Kafka Producer/Consumer API and its unit testing using Embedded Kafka.

embedded-kafka kafka kafka-consumer kafka-producer unit-testing

Last synced: 23 Oct 2024

https://github.com/hihouhou/huginn_kafka_producer_agent

kafka producer agent for huginn

huginn huginn-agent kafka-producer

Last synced: 13 Jan 2025

https://github.com/soumyadip007/distributed-system-message-broking-log-monitoring-using-websocket-netflix-oss-kafka-and-microservice

A message broker is an architectural pattern for message validation, transformation, and routing. It mediates communication among applications, minimizing the mutual awareness that applications should have of each other in order to be able to exchange messages, effectively implementing decoupling.

java kafka kafka-connector kafka-producer kafka-streams log message-broker microservices microservices-architecture realtime-logger rest spring-boot system-design websocket websocket-server zookeeper zookeeper-cluster

Last synced: 09 Nov 2024

https://github.com/snexus/streaming-playground

Exploring streaming design patterns with Kafka and Spark Structural Streaming

kafka kafka-producer python spark spark-streaming

Last synced: 22 Nov 2024

https://github.com/prpundge/kafka-demo-ce

This is a Spring Boot application that integrates with Apache Kafka to demonstrate message production and consumption. The application serves as a foundational example for building scalable and resilient messaging systems.

kafka kafka-consumer kafka-producer maven springboot

Last synced: 20 Dec 2024

https://github.com/alexandramartinez/asyncapis-accounts-email

All the resources you need to implement a functional simple architecture with an Accounts and an Email services using AsyncAPI, Anypoint Code Builder, and the available message brokers: Anypoint MQ, Kafka, Solace PubSub+, and Salesforce CDC/Platform Events

acb anypoint-code-builder anypointmq async async-api asyncapi asyncapi-specification eda kafka kafka-consumer kafka-producer kafka-topic mule mule4 mulesoft queue queues topic

Last synced: 29 Dec 2024

https://github.com/eltonsandre/intellij-kafka

Intellij plugin Kafka Client: A powerful client for Apache Kafka. It provides support for multiple clusters, allowing you to efficiently manage topics, publish and consume messages, and perform various Kafka-related operations right within IntelliJ.

intellij intellij-idea intellij-plugin intellijidea java kafka kafka-consumer kafka-producer kafka-streams kotlin

Last synced: 28 Dec 2024

https://github.com/hoangtien2k3/microservice-webflux

Microservice-webflux with spring boot, reactor-core, Liquibase database migration, docker, apache kafka.

api-multiplexer docker kafka kafka-consumer kafka-producer liquibase maven-plugin microservice mysql r2dbc reactor-core spring-webflux-reactor

Last synced: 09 Nov 2024

https://github.com/chetanxpro/kafka-basics

Explore Kafka Basics: Set up topics, partitions, consumers, and producers using Node.js scripts.

kafka kafka-admin kafka-consumer kafka-producer nodejs

Last synced: 15 Jan 2025

https://github.com/james-kennet/kafka-project

Spring Boot, Kafka, Zookeeper, Kafdrop(kafka viewer), Kafka producer, Kafka consumer, Spring Data/JPA, MySQL, Microservices, Rest API

kafdrop kafka kafka-consumer kafka-producer kafka-viewer microservices mysql rest-api springboot2 zookeeper

Last synced: 13 Jan 2025

https://github.com/wajahat-husain/kakfa-docker

Code and files for an Apache Kafka project with a Kafka producer, consumer, cluster setup, and Docker instructions.

apache-kafka docker docker-compose kafka-consumer kafka-docker kafka-producer kafka-streams kafkajs

Last synced: 19 Nov 2024

https://github.com/fjord-framework/producer

Kafka producer to simulate producing records into a Fjord user's cluster

api-proxy aws kafka kafka-producer real-time sse

Last synced: 19 Nov 2024

https://github.com/carloscazelattojr/kafka-example

Simple example of Kafka - Producer and Consumer

java kafka kafka-consumer kafka-producer spring-boot

Last synced: 13 Jan 2025

https://github.com/deepakbhalla/microservices-with-kafka

This repository demonstrates the use of Kafka in microservices applications

java kafka kafka-consumer kafka-producer kafka-producer-consumer spring-boot

Last synced: 17 Nov 2024

https://github.com/robinhosz/apache-kafka

Examples of microservices using the apache kafka messageria architecture, which consists of the Publisher/Consumer style.

docker docker-compose kafka kafka-brokers kafka-consumer kafka-producer maven message-broker microservices spring-boot

Last synced: 07 Dec 2024

https://github.com/josue-lubaki/kafka-beginners-course

introduction to the concept related to the use of Kafka

event-handling event-sourcing kafka-consumer kafka-producer kafka-streams kakfa

Last synced: 16 Jan 2025

https://github.com/brianpursley/kafka-producer-consumer-java

Java example of a Kafka producer and consumer

java kafka kafka-consumer kafka-producer

Last synced: 01 Dec 2024

https://github.com/idsulik/kafka-concepts-visualizer

Interactive visualization tool that helps developers understand Apache Kafka core concepts through animations. See how messages flow through topics and partitions, how consumer groups work, and how partition rebalancing happens when scaling consumers up or down.

kafka kafka-consumer kafka-producer kafka-topic visualization

Last synced: 09 Dec 2024

https://github.com/castaglia/proftpd-mod_kafka

ProFTPD module for logging via Kafka

kafka-client kafka-producer proftpd

Last synced: 09 Dec 2024

https://github.com/valandro/kafka-producer-consumer

A simple example of producer/consumer application using Apache Kafka as message broker for testing their perfomance.

kafka kafka-consumer kafka-producer spring-boot

Last synced: 09 Dec 2024

https://github.com/cploutarchou/go-kafka-rest

JWT-based Kafka REST API with Golang, GORM, and Fiber

api asynchronous golang kafka kafka-producer kafka-rest orm rest rest-api

Last synced: 23 Dec 2024

https://github.com/nashtech-labs/kafka-unit-testing.g8

A Sample application of Kafka Producer/Consumer API and its unit testing using Embedded Kafka.

embedded-kafka kafka kafka-consumer kafka-producer unit-testing

Last synced: 23 Dec 2024

https://github.com/canelmas/data-producer

Fake data producer for Kafka, console and http endpoints

data fake-content fake-data fakerjs kafka kafka-producer

Last synced: 18 Dec 2024

https://github.com/jonasfschuh/kafka-spring

This project uses Event-Driven Architecture with Java 17, Spring Boot 3 and Kafka. Two microservices were created, ms-kafka-producer and ms-kafka-consumer. Both communicate with Kafka through Spring Boot to create and access topics and partitions.

event-driven java-17-maven kafka kafka-consumer kafka-producer microservices springboot

Last synced: 18 Dec 2024

https://github.com/ronaldkanyepi/log-realtime-analysis

A scalable architecture for real-time log processing and visualization. Built with a Kafka-Spark ETL pipeline, DynamoDB for storing aggregate real-time metrics, and Python Dash for interactive dashboards. Designed for high-throughput log ingestion, real-time monitoring, and long-term storage.

dash docker docker-compose docker-container dynamodb etl etl-pipeline hdfs kafka kafka-consumer kafka-producer kafka-streams kafka-topic logs python realtime spark spark-streaming streaming visualization

Last synced: 25 Dec 2024

https://github.com/fabogit/node-kafka-conduktor

Kafka Consumer/Producer microservices

fastify kafka kafka-consumer kafka-producer mongoose

Last synced: 26 Dec 2024

https://github.com/jveiiga/go-docker-and-kafka-classes

Aulas sobre desenvolvimento Full Cycle - Imersão Full Stack && Full Cycle.

dto entity golang kafka kafka-consumer kafka-producer kafka-topic transformer

Last synced: 02 Jan 2025

https://github.com/poudel/raindrip

Collect various operating system metrics and stream it

cli kafka kafka-consumer kafka-producer metrics python

Last synced: 04 Jan 2025

https://github.com/laurosilveira/hexagonal-architecture

A hexagonal-architecture project to learn about types of software architecture

docker-compose hexagonal-architecture kafka-consumer kafka-producer mongodb openapi3 rest-api spring-boot

Last synced: 13 Dec 2024

https://github.com/pierrekieffer/akka-kafka-producer

Akka http with kafka producer service

akka akka-http kafka kafka-producer scala

Last synced: 13 Dec 2024

https://github.com/cse0518/spring-eureka

MSA 서버 구조에 Spring Cloud Gateway와 Spring Eureka를 적용해봅니다

kafka-producer netty spring-cloud-gateway spring-eureka webflux

Last synced: 21 Dec 2024

https://github.com/tk04/java_kafka

Implementation of a kafka producer + consumer using kafka-clients java library

java kafka kafka-client kafka-consumer kafka-producer

Last synced: 06 Jan 2025

https://github.com/telkomdev/kafkademojava

Kafka Demo using Java with multiple data Serialization (Protobuf, JSON, Avro etc)

java kafka kafka-consumer kafka-java kafka-producer spring-boot

Last synced: 16 Jan 2025

https://github.com/alifruliarso/springboot-kafka

Create Kafka consumer and producer using Spring Boot

kafka kafka-consumer kafka-producer spring-boot-kafka spring-kafka

Last synced: 10 Jan 2025

https://github.com/travelxml/kafka-python-aws-crawler-amazon-athena

A comprehensive tutorials / steps / scripts for setting up Apache Kafka on an Amazon EC2 instance, streaming logs to S3, and querying data with AWS Glue and Amazon Athena. Includes Zookeeper configuration, producer and consumer setup, and automated data catalog creation

aws aws-glue-crawler aws-s3 kafka kafka-connect kafka-consumer kafka-producer kafka-streams message message-broker message-queue

Last synced: 19 Nov 2024

https://github.com/teta42/anti_duplicate_system

Система противодействующая обработки дубликатов, работает исключительно на Kafka

kafka kafka-consumer kafka-producer package python python3

Last synced: 30 Nov 2024

https://github.com/lhmzhou/hairy-lemon

kafka producer implementation

kafka-producer

Last synced: 14 Dec 2024

https://github.com/dulajkavinda/kafka-node

Kafka Producer & Consumer using Node.js.

docker kafka kafka-consumer kafka-producer node

Last synced: 14 Dec 2024

https://github.com/wesleysbmartins/node_kafka

Implementação de integração entre uma aplicação Node TS com Kafka.

backend factory kafka kafka-consumer kafka-producer kafkajs node node-js nodejs singleton typescript usecase

Last synced: 24 Nov 2024

https://github.com/wesleysbmartins/go_kafka

Este é um exemplo de como uma aplicação Golang pode trabalhar integrada ao Kafka.

adapters entities factory golang kafka kafka-client kafka-connect kafka-consumer kafka-consumer-group kafka-producer kafka-topic sarama singleton usecases

Last synced: 24 Nov 2024

https://github.com/scblur869/mqtt-to-kafka-alt

maps mqtt payloads based on topic to Kafka using a lower level calls

kafka kafka-producer mqtt mqtt-to-kafka

Last synced: 13 Dec 2024

https://github.com/memojja/kafka-spark-examples

Learning kafka-spark entegration and streaming data analyze

kafka-consumer kafka-producer spark-streaming spark-streaming-kafka

Last synced: 21 Nov 2024

https://github.com/samiksha-khare/crypto-real-time-analysis-using-kafka

Real-time analysis, visualization, and storage of cryptocurrency data with Kafka, Apache Spark, and MongoDB

api cryptocurrency kafka kafka-consumer kafka-producer kafka-topic matplotlib mongodb nosql python real-time spark visualization zookeeper

Last synced: 20 Nov 2024

https://github.com/eclipsemode/java-spring-kafka-producer

Rest api minimal project with postgresql, mongodb and kafka producer

kafka-producer keycloak mongodb postgresql spring-boot

Last synced: 18 Nov 2024

https://github.com/sanzharanarbay/golang_kafka_example

Golang implementation of the kafka producer and kafka consumer with the docker deployment

docker docker-compose go golang kafka kafka-connect kafka-consumer kafka-producer kafka-topic

Last synced: 21 Nov 2024

https://github.com/himash79/spring-cloud-stream-with-apache-kafka

Build restful webservices utilizing Spring boot as well as intergrate Apache Kafka also manipulate with Spring Cloud Stream.

apache-kafka kafka kafka-consumer kafka-producer restful-api spring-boot

Last synced: 21 Nov 2024

https://github.com/himash79/spring-boot-apache-kafka

Build restful webservices utilizing Spring boot as well as intergrate with Apache Kafka.

apache-kafka kafka kafka-consumer kafka-producer restful-api spring-boot

Last synced: 21 Nov 2024

https://github.com/rafael-vasconcellos/venda-service

basic example for study of using kafka with spring

backend kafka kafka-producer server-side spring spring-boot

Last synced: 11 Jan 2025

https://github.com/nitesh232/kafka_producer_with_sasl_ssl_security

This project consists of Kafka Producer with SASL_SSL security enabled

kafka kafka-producer kafka-security sasl-ssl springboot3

Last synced: 21 Nov 2024

https://github.com/eccenca/cmem-plugin-kafka

This eccenca Corporate Memory plugin allows for sending and receiving messages from Apache Kafka.

corporate-memory eccenca kafka kafka-consumer kafka-producer plugin

Last synced: 21 Nov 2024

https://github.com/pouyapouryaie/springboot-kafka

This project demonstrates a simple kafka producer and consumer alongside another feature like error-handling, integration test with testcontainer and more

avro kafka-consumer kafka-producer retryable schema-registry springboot springboot3-x testcontainers

Last synced: 20 Dec 2024

https://github.com/priyanka387/ml-data-pipeline

The project aims to read the data from sensors attached in heavy vehicles and push it to kafka. Apache Kafka is a distributed event store and stream-processing platform. We are finally consuming the data from kafka to Mongodb.

kafka kafka-cluster kafka-consumer kafka-producer kafkatopic mongodb-atlas mongodb-database

Last synced: 17 Jan 2025

https://github.com/cunkz/nodejs-kafka-example

Example Kafka Producer and Consumer via NodeJS

kafka kafka-consumer kafka-producer nodejs nodejs-example

Last synced: 25 Nov 2024

https://github.com/wjc-van-es/ps-eda-kafka-docker-1rst-app

This will be an adaptation of the example of the second module of the PluralSight Course:

apache-kafka confluent-platform cp-community-components docker-compose kafka-consumer kafka-producer pluralsight-course

Last synced: 25 Nov 2024

https://github.com/wjc-van-es/ps-eda-kafka-docker-avro-app

Example from Designing Event-driven Applications Using Apache Kafka Ecosystem By Bogdan Sucaciu m 4

apache-kafka avro confluent-platform confluent-schema-registry cp-community-components docker-compose kafka-consumer kafka-producer pluralsight-course

Last synced: 25 Nov 2024

https://github.com/siddharth-nandagopal/real-time-fraud-detection

detect potentially fraudulent transactions in real-time by analyzing payment transactions as they occur.

apache asdf kafka kafka-consumer kafka-producer poetry python python-kafka

Last synced: 13 Jan 2025