Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/datastax/starlight-for-kafka

DataStax - Starlight for Kafka

kafka pulsar

Last synced: 03 Jul 2024

https://github.com/cmcornejocrespo/elastic-stack-k8s

Deploys the elastic stack on k8s using terraform. Showcases elastic stack 6.8.0 features

aks apm-server beats elasticsearch kafka kibana kubernetes logstash terraform zookeeper

Last synced: 03 Jul 2024

https://github.com/kaiwaehner/kafka-streams-machine-learning-examples

This project contains examples which demonstrate how to deploy analytic models to mission-critical, scalable production environments leveraging Apache Kafka and its Streams API. Models are built with Python, H2O, TensorFlow, Keras, DeepLearning4 and other technologies.

deep-learning deeplearning4j dl4j h2o h2oai java kafka kafka-client kafka-streams keras keras-tensorflow ksql machine-learning open-source python tensorflow

Last synced: 02 Jul 2024

https://github.com/TurboWay/spiderman

基于 scrapy-redis 的通用分布式爬虫框架

hbase hive kafka rdbm scapy-redis scrapy spiderman

Last synced: 02 Jul 2024

https://github.com/danthelion/twitter-trending

A small example visualizing live Twitter trending data using Redpanda and Materialize

kafka materialize metabase redpanda stream-processing

Last synced: 02 Jul 2024

https://github.com/Aiven-Open/journalpump

systemd journald to aws_cloudwatch, elasticsearch, google cloud logging, kafka, rsyslog or logplex log sender

elasticsearch journald kafka logplex message-pump

Last synced: 02 Jul 2024

https://github.com/typestreamio/typestream

Open Source streaming platform. Write and run typed data pipelines with a minimal, familiar syntax.

kafka kafka-streams pipelines

Last synced: 01 Jul 2024

https://github.com/daneshzaki/kokpit

Basic Kafka client that does publish subscribe and runs as a Windows application

electron-app javascript kafka kafkajs nodejs windows

Last synced: 01 Jul 2024

https://github.com/raystack/raccoon

Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols.

clickstream dataops eventsourcing kafka

Last synced: 01 Jul 2024

https://github.com/raystack/firehose

Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.

apache-kafka bigquery dataops firehose influxdb kafka postgresql prometheus sink streaming

Last synced: 01 Jul 2024

https://github.com/childe/hangout

用java实现一下Logstash的几个常用input/filter/output, 希望能有效率上面的大提升. 现在我们迁移到golang了 https://github.com/childe/gohangout

elasticsearch kafka logstash

Last synced: 29 Jun 2024

https://github.com/apache/camel-kafka-connector-examples

Apache Camel Kafka Connector Examples

camel integration java kafka

Last synced: 29 Jun 2024

https://github.com/apache/camel-kafka-connector

Camel Kafka Connector allows you to use all Camel components as Kafka Connect connectors

camel integration java kafka

Last synced: 29 Jun 2024

https://github.com/marceloboeira/voik

♒︎ [WIP] An experimental ~distributed~ commit-log

append-only commit-log distributed immutable kafka kinesis nats streaming

Last synced: 29 Jun 2024

https://github.com/pemrouz/fero

light, fast, scalable, streaming microservices made easy

fast kafka lightweight microservices scalable stateful stream-processing

Last synced: 27 Jun 2024

https://github.com/bluxmit/alnoda-workspaces

:fireworks: Flexible and extendable containerized workspaces. Now. with free offline chat GPT!!! 🚀🚀🚀

ansible containers docker elasticsearch kafka postgres python redis terraform web-based-ide workspace

Last synced: 27 Jun 2024

https://github.com/NHadi/Pos

Sample Application DDD, Reactive Microservices, CQRS Event Sourcing Powered by DERMAYON LIBRARY

cqrs cqrs-es ddd-architecture event-driven event-sourcing generic-repository kafka microservices microservices-architecture netcore reactive

Last synced: 27 Jun 2024

https://github.com/nitishm/go-micro-framework

A simplistic microservice framework written in Golang

framework golang golang-framework kafka microservice

Last synced: 27 Jun 2024

https://github.com/hemin1003/spring-boot-study

SpringBoot框架源码实战(已更新到springboot2版本实现)~基本用法,Rest,Controller,事件监听,连接数据库MySQL,jpa,redis集成,mybatis集成(声明式与xml两种方式~对应的添删查改功能),日志处理,devtools配置,拦截器用法,资源配置读取,测试集成,Web层实现请求映射,security安全验证,rabbitMq集成,kafka集成,分布式id生成器等。项目实战:https://github.com/hemin1003/yfax-parent 已投入生产线上使用

kafka mybatis oauth2 redis restful-api spring-boot springsecurity

Last synced: 27 Jun 2024

https://github.com/ThreeDotsLabs/event-driven-example

An example Event-Driven application in Go built with Watermill library.

docker-compose event-driven go golang grafana kafka prometheus rabbitmq watermill

Last synced: 27 Jun 2024

https://github.com/team-ide/teamide

Team IDE 集成MySql、Oracle、金仓、达梦、神通等数据库、SSH、FTP、Redis、Zookeeper、Kafka、Elasticsearch、Mongodb、小工具等管理工具

database elasticsearch file-manager kafka mongodb redis ssh terminal tools zookeeper

Last synced: 27 Jun 2024

https://github.com/capitalone/cqrs-manager-for-distributed-reactive-services

Experimental CQRS and Event Sourcing service

cqrs event-sourcing kafka

Last synced: 25 Jun 2024

https://github.com/alibaba/Alink

Alink is the Machine Learning algorithm platform based on Flink, developed by the PAI team of Alibaba computing platform.

apriori classification clustering data-mining feature-engineering flink flink-machine-learning flink-ml fm graph-algorithms graph-embedding kafka machine-learning recommender recommender-system regression statistics word2vec xgboost

Last synced: 23 Jun 2024

https://github.com/alonsoir/hello-kafka-twitter-scala

twitter, kafka,spark-streaming, mongodb,scala

kafka mongodb scala twitter-api

Last synced: 22 Jun 2024

https://github.com/DataCater/datacater

The developer-friendly ETL platform for transforming data in real-time. Based on Apache Kafka® and Kubernetes®.

apache-kafka cloud-native data-pipelines etl kafka kubernetes python

Last synced: 21 Jun 2024

https://github.com/fengzhizi715/NetDiscovery

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

coroutines crawler disruptor dsl htmlunit kafka kotlin lettuce middleware redis rxjava2 selenium spider vertx3

Last synced: 20 Jun 2024

https://github.com/sasou/syncClient

syncClient,数据实时同步中间件(同步mysql到kafka、redis、elasticsearch、httpmq)!

binlog canal elasticsearch httpmq java kafka redis

Last synced: 20 Jun 2024

https://github.com/javahongxi/whatsmars

Java生态研究(Spring Boot + Redis + Dubbo + RocketMQ + Elasticsearch)🔥🔥🔥🔥🔥

distributed-rpc dubbo elastic-job elasticsearch java kafka lambda log4j2 microservices redis rocketmq rpc rpc-framework sentinel sharding-jdbc spring spring-boot spring-mvc tomcat zookeeper

Last synced: 20 Jun 2024

https://github.com/2227324689/gpmall

【咕泡学院实战项目】-基于SpringBoot+Dubbo构建的电商平台-微服务架构、商城、电商、微服务、高并发、kafka、Elasticsearch

canal dubbo elasticsearch elk java kafka nacos sentinel springboot springcloud-alibaba zookeeper

Last synced: 20 Jun 2024

https://github.com/lensesio/stream-reactor

A collection of open source Apache 2.0 Kafka Connector maintained by Lenses.io.

awss3 azure-datalake cassandra connector cosmosdb elasticsearch eventhubs gcp http influxdb jms kafka kafka-connect mongodb mqtt pubsub s3

Last synced: 19 Jun 2024

https://github.com/ajanata/PretendYoureXyzzy

A web clone of the card game Cards Against Humanity.

card-game cards-against-humanity game hibernate java javascript kafka kafka-producer

Last synced: 19 Jun 2024

https://github.com/terascope/teraslice

Scalable data processing pipelines in JavaScript

elasticsearch hadoop hdfs json kafka

Last synced: 19 Jun 2024

https://github.com/Chabane/bigdata-playground

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

angular apache-flink apache-spark avro big-data docker graphql hadoop hbase kafka kops machine-learning mongodb nodejs parquet python scala spark-sql spark-streaming twitter-api

Last synced: 17 Jun 2024

https://github.com/iamabug/BigDataParty

大数据组件 All-in-One 的 Dockerfile

big-data dockerfile hadoop kafka spark

Last synced: 16 Jun 2024

https://github.com/piomin/sample-terraform-kubernetes-argocd

Repository with configuration for Terraform and Argo CD to create and manage Kubernetes cluster locally with Kind

argocd kafka kind kubernetes strimzi terraform terraform-kubernetes-provider

Last synced: 16 Jun 2024

https://github.com/agalue/opennms-drift-kubernetes

OpenNMS Drift Deployment in Kubernetes for testing and learning purposes

aws azure cassandra ingress-controller kafka kops kubernetes opennms

Last synced: 16 Jun 2024

https://github.com/streamdal/plumber

A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

event-bus event-driven golang hacktoberfest kafka message-bus message-queue protobuf rabbitmq

Last synced: 14 Jun 2024

https://github.com/abhithube/chatterbox-microservices

Microservices web application for group messaging.

aws docker javascript kafka microservices mongodb nodejs react redis terraform typescript

Last synced: 13 Jun 2024

https://github.com/Speccy-Rom/SpeccyTV

​💻​👨‍💻​⌨️️​ Streaming service with ETL on steroids

airflow clickhouse django docker docker-compose elasticsearch fastapi ffmpeg flask kafka microservice mongodb nginx postgresql pytest python uvicorn

Last synced: 13 Jun 2024

https://github.com/hitachienergy/epiphany

Cloud and on-premises automation for Kubernetes centered industrial grade solutions.

ansible docker elasticsearch grafana infrastructure-as-code kafka keycloak kubernetes prometheus rabbitmq terraform

Last synced: 13 Jun 2024

https://github.com/smallcase/smalllinks

Simple URL shortener built with Fastify. Uses PostgreSQL, Redis, and Kafka.

docker-compose fastify hacktoberfest kafka nanoid postgresql redis url-shortener

Last synced: 12 Jun 2024

https://github.com/dynamicheart/DTSS

Distributed Transaction Settlement System

hadoop kafka se347 spark zookeeper

Last synced: 12 Jun 2024

https://github.com/Wellers0n/node-kafka

nodejs application using kafka streams

express javascript kafka node typescript

Last synced: 12 Jun 2024

https://github.com/adyliu/jafka

a fast and simple distributed publish-subscribe messaging system (mq)

jafka-mq jms kafka message-queue mq

Last synced: 12 Jun 2024

https://github.com/Anant/Cassandra.Realtime

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

akka cassandra flink flink-stream-processing flink-streaming kafka kafka-connect spark spark-streaming

Last synced: 12 Jun 2024

https://github.com/maxwellpeterson/kafka-worker

Kafka 0.8.0 broker implementation on top of Cloudflare Workers

cloudflare-workers kafka

Last synced: 11 Jun 2024

https://github.com/tisonkun/kafka-api

Generic abstractions for Kafka API.

kafka rust-lang

Last synced: 11 Jun 2024

https://github.com/qunarcorp/qmq

QMQ是去哪儿网内部广泛使用的消息中间件,自2012年诞生以来在去哪儿网所有业务场景中广泛的应用,包括跟交易息息相关的订单场景; 也包括报价搜索等高吞吐量场景。

consumer distributed-transactions event event-bus event-driven kafka message message-bus message-driven message-queue mq producer rabbitmq xa

Last synced: 11 Jun 2024

https://github.com/Amazingwujun/mqttx

MQTTX Project 完整实现 mqttv3.1.1 协议,旨在提供易于使用且性能优异的 mqtt broker

bridge broker cluster docker iot-application kafka mqtt mqtt-broker mqttx netty redis share-topic springboot topic websocket

Last synced: 11 Jun 2024

https://github.com/HariSekhon/Dockerfiles

50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak

apache-drill cassandra consul devops docker dockerhub hacktoberfest hadoop hbase kafka kubernetes linux nagios-plugins presto rabbitmq rabbitmq-cluster solr solrcloud spark zookeeper

Last synced: 11 Jun 2024

https://github.com/memgraph/memgraph

Open-source graph database, built for real-time streaming data, compatible with Neo4j.

cypher graph graph-algorithms graph-analysis graph-database kafka kafka-streams nosql opencypher stream-processing streaming-data

Last synced: 11 Jun 2024

https://github.com/wangweianger/zanePerfor

前端性能监控系统,消息队列,高可用,集群等相关架构

egg kafka mongodb nodejs redis vuejs2

Last synced: 09 Jun 2024

https://github.com/wangshibiaoFlytiger/apiproject

[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+jwt+json日志库zap(支持日志收集到kafka或mongo)+消息队列kafka+微信支付宝支付gopay+api加密+api反向代理+go modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+livereload热加载)

alipay api-server compress cors gin-framework golang gomodule gorm headless jwt kafka livereload makefile mongo redis reverseproxy spider wxpay zap

Last synced: 09 Jun 2024

https://github.com/dushixiang/kafka-map

A beautiful, concise and powerful kafka web management tool. 一个美观简洁且强大的kafka web管理工具。

kafka kafka-manager kafka-monitor kafka-ui kafka-viewer kafka-web-ui

Last synced: 09 Jun 2024

https://github.com/sanshengshui/netty-learning-example

:egg: Netty实践学习案例,见微知著!带着你的心,跟着教程。我相信你行欧。

fastjson http im iot kafka mqtt netty protobuf rpc zero-copy

Last synced: 09 Jun 2024

https://github.com/kevwan/go-stash

go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch.

elasticsearch elk kafka logstash

Last synced: 09 Jun 2024

https://github.com/Mikaelemmmm/go-zero-looklook

🔥基于go-zero(go zero) 微服务全技术栈开发最佳实践项目。Develop best practice projects based on the full technology stack of go zero (go zero) microservices.

asynq dtm elasticsearch filebeat gitlab go-queue go-stash go-zero goctl golang gozero grafana harbor jaeger jenkins k8s kafka microservices modd prometheus

Last synced: 09 Jun 2024

https://github.com/xuwujing/java-study

java-study 是本人学习Java过程中记录的一些代码!从Java基础的数据类型、jdk1.8的Lambda、Stream和日期的使用、 IO流、数据集合、多线程使用、并发编程、23种设计模式示例代码、常用的工具类, 以及一些常用框架,netty、mina、springboot、kafka、storm、zookeeper、redis、elasticsearch、hbase、hive等等。

commons concurrent design easticsearch java jdk8 kafka mina netty redis thread utils

Last synced: 09 Jun 2024

https://github.com/CoderLeixiaoshuai/java-eight-part

『Java八股文』Java面试套路,Java进阶学习,打破内卷拿大厂Offer,升职加薪!

java juc kafka mysql redis spring

Last synced: 09 Jun 2024

https://github.com/fooinha/nginx-json-log

Highly configurable JSON format logging per Location - nginx logging module - aka. kasha 🍲

elk http jansson json json-format kafka kasha logging nginx nginx-http rdkafka

Last synced: 09 Jun 2024

https://github.com/dinivas/dinivas

AWS, GCP alternative on premise. Dinivas manage your private Cloud (OpenStack) infrastructure by providing many features based on popular Open Source projects

ansible aws-alternative cloud gcp-alternative infrastructure kafka keycloak kubernetes mosquitto openstack packer rabbitmq terraform

Last synced: 08 Jun 2024

https://github.com/ozontech/file.d

A blazing fast tool for building data pipelines: read, process and output events. Our community: https://t.me/file_d_community

actions clickhouse elasticsearch events file gelf go http input json kafka logs observability output pipeline processing reading sre throttle tracing

Last synced: 08 Jun 2024

https://github.com/xaecbd/KCenter

KCenter(KafkaCenter) is a unified platform for kafka cluster management and maintenance, producer / consumer monitoring, and use of ecological components(ksql/kafka connect).it's kafkacenter(kafka center).

cmak connect-ui kafka kafka-center kafka-connect kafka-connect-ui kafka-eagle kafka-manager kafka-monitor kafkacenter kafkamanager kafkaoffsetmonitor kc kmanager ksql

Last synced: 08 Jun 2024

https://github.com/childe/gohangout

使用 golang 模仿的 Logstash。用于消费 Kafka 数据,处理后写入 ES、Clickhouse 等。

elasticsearch golang kafka logstash

Last synced: 08 Jun 2024

https://github.com/syslog-ng/syslog-ng

syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.

c elastic kafka log-management logging python syslog syslog-ng

Last synced: 08 Jun 2024

https://github.com/collabH/bigdata-growth

大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

bigdata bigdatalearning debezium flink hadoop hbase hdfs hive hudi kafka kudu mapreduce olap spark

Last synced: 08 Jun 2024

https://github.com/confluentinc/demo-scene

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

debezium demo elasticsearch kafka kafka-connect kafka-streams ksql ksqldb mysql syslog

Last synced: 08 Jun 2024

https://github.com/tmcgrath/kafka-connect-examples

Kafka Connect Examples

kafka kafka-connect

Last synced: 08 Jun 2024

https://github.com/opentracing-contrib/java-kafka-client

OpenTracing Instrumentation for Apache Kafka Client

java kafka kafka-client kafka-streams opentracing spring-kafka

Last synced: 08 Jun 2024

https://github.com/jcustenborder/connect-utils

Utility project for working with Kafka Connect.

conversion kafka kafka-connect parsing schema

Last synced: 07 Jun 2024

https://github.com/hpgrahsl/kafka-connect-mongodb

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

avro azure-cosmosdb bson cdc change-data-capture confluent-hub connector cosmosdb debezium json kafka kafka-connect mongodb sink sink-connector

Last synced: 07 Jun 2024

https://github.com/dvoraka/av-service

Network file service with replication and anti-virus checking

amqp anti-virus database file-services java jms kafka replication rest solr spring spring-boot storage

Last synced: 07 Jun 2024

https://github.com/confluentinc/kafka-connect-storage-cloud

Kafka Connect suite of connectors for Cloud storage (Amazon S3)

aws aws-s3 cloud confluent kafka kafka-connect s3

Last synced: 07 Jun 2024

https://github.com/wangzhiwubigdata/God-Of-BigData

专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

azkaban bigdata flink flume hadoop hbase hdfs hive kafka spark zookeeper

Last synced: 07 Jun 2024

https://github.com/immerok/recipes

The Immerok Apache Flink Cookbook is a collection of examples of Apache Flink applications in the format of "recipes". Each recipe explains how you can solve a specific problem by leveraging one or more of the APIs of Apache Flink. The recipes can be extended or provide a basis for solving your requirements with Apache Flink.

apacheflink cookbook datastream flink kafka streaming

Last synced: 07 Jun 2024

https://github.com/MoRan1607/BigDataGuide

大数据学习,从零开始学习大数据,包含大数据学习各阶段学习视频、面试资料

bigdata flink flume hadoop hbase hive javase kafka scala spark zookeeper

Last synced: 07 Jun 2024

https://github.com/streamnative/kop

Kafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar

apache-kafka apache-pulsar dotnet event-streaming kafka kafka-on-pulsar kop pub-sub pubsub pulsar

Last synced: 07 Jun 2024

https://github.com/redpanda-data/kminion

KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.

consumer consumer-group exporter group kafka lag prometheus prometheus-exporter

Last synced: 07 Jun 2024

https://github.com/Aiven-Open/tiered-storage-for-apache-kafka

RemoteStorageManager for Apache Kafka® Tiered Storage

apache-kafka kafka tiered-storage

Last synced: 07 Jun 2024

https://github.com/ccamel/kynaptik

λ Fission Function which evaluates incoming stimulus (event/message) and determines an appropriate (configured) action (http, graphQL, ...)

faas fission function graphql kafka kubernetes lambda notify rest serverless-functions stimulus webhook

Last synced: 07 Jun 2024

https://github.com/bsm/sarama-cluster

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 [DEPRECATED]

consumer deprecated go kafka sarama

Last synced: 07 Jun 2024

https://github.com/argoproj-labs/old-argo-dataflow

Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.

data jetstream kafka kubernetes pipeline

Last synced: 07 Jun 2024

https://github.com/lensesio/lenses-docker

❤for real-time DataOps - where the application and data fabric blends - Lenses

dataops docker enterprise governance kafka kubernetes openshift security

Last synced: 07 Jun 2024

https://github.com/Telefonica/prometheus-kafka-adapter

Use Kafka as a remote storage database for Prometheus (remote write only)

cdco kafka kafka-producer prometheus prometheus-adapter

Last synced: 07 Jun 2024