Projects in Awesome Lists tagged with flink

https://github.com/izhangzhihao/real-time-data-warehouse

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

cdc change-data-capture data-warehouse data-warehousing datalake debezium delta delta-lake deltalake elasticsearch flink flink-sql hoodie hudi iceberg kafka real-time-data-warehouse spark spark-sql sql

Last synced: 20 Dec 2024

https://github.com/WeBankFinTech/Streamis

Streaming application development and management system, based on Linkis and DSS, planning to provide the workflow-like graphical drag-and-drop development capability.

datalake dataspherestudio deltalake flink hudi iceberg kafka linkis streaming streamis warehouse wedatasphere

Last synced: 23 Nov 2024

https://github.com/pravega/flink-connectors

Apache Flink connectors for Pravega.

flink pravega stream-processing

Last synced: 31 Dec 2024

https://github.com/mbode/flink-prometheus-example

Example setup to demonstrate Prometheus integration of Apache Flink

flink prometheus

Last synced: 15 Nov 2024

https://github.com/ververica/ververica-platform-playground

Instructions for getting started with Ververica Platform on minikube.

flink ververica-platform

Last synced: 12 Nov 2024

https://github.com/LinMingQiang/flink-learn

Learning Flink : Flink CEP,Flink Core,Flink SQL

bigdata flink sql stream

Last synced: 05 Nov 2024

https://github.com/zuinnote/hadoopoffice

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

analyze-office-documents bigdata excel flink hadoop hadoop-ecosystem hadoopoffice hive office poi spark

Last synced: 14 Oct 2024

https://github.com/apache/flink-connector-elasticsearch

Apache Flink connector for ElasticSearch

connector datastream elasticsearch flink sql table

Last synced: 29 Dec 2024

https://github.com/hifly81/kafka-examples

Practical examples with Apache Kafka®

avro avro-schema avroserializer confluent confluent-kafka flink graalvm kafka kafka-client kafka-consumer kafka-consumers kafka-events kafka-installation kafka-producer kafka-producers kafka-streams kafka-topic ksql ktable microprofile

Last synced: 27 Dec 2024

https://github.com/nextbreakpoint/flink-client

Java client for managing Apache Flink via REST API

flink java openapi

Last synced: 31 Dec 2024

https://github.com/nextbreakpoint/flink-k8s-toolbox

Flink Kubernetes Toolbox is the Swiss Army knife for deploying and managing Apache Flink on Kubernetes

flink infrastructure-as-code kotlin kubernetes kubernetes-controller kubernetes-operator stream-processing

Last synced: 17 Nov 2024

https://github.com/twalthr/flink-api-examples

Examples for using Apache Flink® with DataStream API, Table API, Flink SQL and connectors such as MySQL, JDBC, CDC, Kafka.

apache-flink data-engineering flink flink-examples flink-sql stream-processing

Last synced: 05 Nov 2024

https://github.com/mikeroyal/apache-flink-guide

Apache Flink Guide

data-science database flink flink-kafka flink-stream-processing flink-streaming stream-processing streaming

Last synced: 11 Dec 2024

https://github.com/zmyzheng/tweetmap

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud

angular aws bootstrap cloud data-stream-processing docker elaticsearch flink kafka kibana kubernetes microservice sentiment-analysis spring-boot

Last synced: 02 Nov 2024

https://github.com/collabh/flink-connector-kudu

基于Apache-bahir-kudu-connector的flink-connector-kudu，支持Flink1.11.x DynamicTableSource/Sink，支持Range分区等

datastream flink flink-dynamic-source flink-sql kudu kudu-connector

Last synced: 28 Oct 2024

https://github.com/SophiaData/Bigdata_Code_Tutorial

Flink cdc 整库同步 & flink 代码 demo

flink

Last synced: 05 Nov 2024

https://github.com/MyLanPangzi/flink-demo

Flink Demo

flink flink-examples flink-sql

Last synced: 05 Nov 2024

https://github.com/ververica/jupyter-vvp

Jupyter Integration for Flink SQL via Ververica Platform

apache-flink flink jupyter

Last synced: 12 Nov 2024

https://github.com/apache/flink-connector-aws

Apache flink

connector datastream dynamodb flink sql table

Last synced: 05 Nov 2024

https://github.com/aikuyun/bigdata-doc

大数据学习笔记，学习路线，技术案例整理。

bigdata flink hadoop hdfs hive kafka mapreduce

Last synced: 30 Oct 2024

https://github.com/LB-Yu/data-systems-learning

Learning summary and examples about data systems.

antlr big-data calcite distributed-systems flink hadoop hbase spark

Last synced: 05 Nov 2024

https://github.com/pathwaycom/pathway-benchmarks

Benchmarks for data processing systems: Pathway, Spark, Flink, Kafka Streams

benchmark-framework flink kafka-streams latency pagerank pathway spark-streaming streaming streaming-data wordcount

Last synced: 13 Nov 2024

https://github.com/aws-samples/amazon-managed-service-for-apache-flink-examples

Collection of code examples for Amazon Managed Service for Apache Flink

amazon-managed-service-for-apache-flink flink flink-examples flink-stream-processing

Last synced: 05 Nov 2024

https://github.com/nmoncho/helenus

An Apache Cassandra Client for Scala 2 inspired by Anorm and Quill

akka-streams cassandra client database flink pekko-streams reactive reactive-streams scala

Last synced: 11 Nov 2024

https://github.com/klarna-incubator/flink-connector-dynamodb

Apache Flink connector sink for AWS DynamoDB

dynamodb flink flink-connector-dynamodb klarna-featured

Last synced: 14 Nov 2024

https://github.com/Anant/Cassandra.Realtime

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

akka cassandra flink flink-stream-processing flink-streaming kafka kafka-connect spark spark-streaming

Last synced: 08 Nov 2024

https://github.com/apache/flink-connector-hbase

Apache flink

connector datastream flink hbase sql table

Last synced: 30 Dec 2024

https://github.com/anant/cassandra.realtime

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

akka cassandra flink flink-stream-processing flink-streaming kafka kafka-connect spark spark-streaming

Last synced: 18 Nov 2024

https://github.com/aiven-open/sql-cli-for-apache-flink-docker

SQL CLI for Apache Flink® via docker-compose

apacheflink flink sql-client

Last synced: 11 Nov 2024

https://github.com/1ambda/lakehouse

Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)

airflow cdc dbt debezium docker flink hudi iceberg kafka spark trino

Last synced: 18 Nov 2024

https://github.com/lightbend/flink-k8s-operator

An example of building kubernetes operator (Flink) using Abstract operator's framework

flink kubernetes operator scala

Last synced: 12 Nov 2024

https://github.com/aikuyun/2018-flink-forward-china

Flink Forward China 2018 第一届记录，视频记录 | 文档记录 | 不仅仅是流计算 | More than streaming

china flink

Last synced: 17 Dec 2024

https://github.com/cjermain/flink-onnx-pytorch

Streaming machine learning using PyTorch, Flink, and ONNX

flink flink-stream-processing flink-streaming machine-learning onnx onnxruntime pytorch streaming

Last synced: 23 Oct 2024

https://github.com/spektom/realtime-dashboard-example

This is a real-time dashboard example using Spark Streaming and Node.js

dashboard-application flink kafka meetup rethinkdb spark spark-streaming

Last synced: 19 Nov 2024

https://github.com/chloro-pn/tunnel

Tunnel is a Pipeline Execution Engine based on C++20 coroutine

asynchronous-programming clickhouse coroutines cpp20-coroutine flink olap parallel-computing pipeline taskflow workflow

Last synced: 15 Nov 2024

https://github.com/snowplow/dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR

amazon-emr flink golang-application hadoop spark

Last synced: 09 Nov 2024

https://github.com/streamnative/pulsar-hub

The canonical source of StreamNative Hub.

apache-pulsar connector data-processing event-streaming flink messaging offloader opentracing prestosql pubsub pulsar-functions pulsar-io spark tracing

Last synced: 01 Dec 2024

https://github.com/singgel/flink-kafka-hbase

功能：实现kafka消息实时落地hbase，支持csv/json字符串两种格式的消息，支持自定义组合rowkey,列簇和列名，支持按照kafka消息流中不同字段join不同的hbase表，并自定义写入列簇和列(join时需评估一下性能)，支持at least once语义外部依赖：apollo配置中心，本项目依靠配置驱动，配置存储在apollo配置中心

apollo flink hbase kafka

Last synced: 14 Oct 2024

https://github.com/spotify/flyte-flink-plugin

Flyte Flink k8s plugin.

flink flyte kubernetes

Last synced: 07 Nov 2024

https://github.com/cyberdelia/flink-kotlin

Kotlin support for Apache Flink

apache-flink flink kotlin serialization

Last synced: 07 Nov 2024

https://github.com/jaehyeon-kim/flink-demos

Apache Flink (Pyflink) and Related Projects

aws data-streaming docker docker-compose flink kafka kinesis-data-analytics pyflink python real-time-analytics

Last synced: 30 Oct 2024

https://github.com/apache/flink-connector-cassandra

Apache flink

cassandra connector datastream flink

Last synced: 02 Jan 2025

https://github.com/melin/flink-jobserver

REST job server for Apache Flink

flink hadoop hive java kerberos kubernetes yarn

Last synced: 05 Nov 2024

https://github.com/qiushisun/distributed-computing-systems

2021 Spring (Distributed Computing Systems) 分布式系统与编程

distributed-computing distributed-systems ecnu-dase flink hadoop-mapreduce spark

Last synced: 19 Dec 2024

https://github.com/isopropylcyanide/websockets-vertx-flink-kafka

A simple request response cycle using Websockets, Eclipse Vert-x server, Apache Kafka, Apache Flink.

eventbus flink flink-kafka flink-kafka-consumer kafka vertx vertx-eventbus vertx-kafka websocket

Last synced: 12 Nov 2024

https://github.com/innogames/flink-real-time-crm

Demo Flink and Kafka project to show how to react on tracking events in real-time and trigger offer for customer engagement based on campaign configurations. The project also utilizes the Broadcast State Pattern in order to update the rules (campaigns) at runtime without restarting the project, using a dedicated, low-frequency, Kafka topic.

apacheflink apachekafka flink flink-examples flink-stream-processing flink-streaming java kafka stream-processing

Last synced: 02 Dec 2024

https://github.com/cclient/flink-connector-elasticsearch-source

Flink Hadoop Compatibility + Elasticsearch for Apache Hadoop = Flink Connector Elasticsearch Source Table。结合flink+hadoop+es 实现的es table source,从es下载数据后应用flink sql，小数据agg，大数据etl.不支持谓词下推

elasticsearch flink

Last synced: 16 Nov 2024

https://github.com/jorokr21/flink-shapeless

ADT support for Flink with Shapeless

algebraic-data-types flink serialization shapeless type-classes

Last synced: 13 Oct 2024

https://github.com/collabh/reasearch-bigdata

看书看源码看第三方学习视频

flink hadoop hive spark

Last synced: 28 Oct 2024

https://github.com/bptlab/cepta

CEPTA - Complex Event Processing Transportation Analysis

cep flink kafka monitoring real-time transportation

Last synced: 10 Nov 2024

https://github.com/chaokunyang/bigdata-examples

bigdata examples about spark and flink

bigdata flink hadoop monitor python samples spark spark-sql sparkml

Last synced: 19 Nov 2024

https://github.com/ahmetfurkandemir/flink-example

Flink Example

apache apache-flink data-engineering data-stream-processing debezium debezium-client debezium-connector debeziumkafkaconnector docker flink flink-sql flink-stream-processing flink-streaming kafka kafka-streams scala scala2 ubuntu

Last synced: 16 Nov 2024

https://github.com/ververica/flink-ecosystem

Ecosystem website for Apache Flink

flink

Last synced: 12 Nov 2024

https://github.com/codefeedr/codefeedr

CodeFeedr core infrastructure

flink kafka sbt scala stream-processing

Last synced: 10 Oct 2024

https://github.com/ev2900/flink_kinesis_data_analytics

Apache Flink examples designed to be run by AWS Kinesis Data Analytics (KDA).

aws flink flink-examples flink-sql flink-stream-processing flink-streaming flinksql kinesis

Last synced: 05 Nov 2024

https://github.com/nmoncho/helenus3

An Apache Cassandra Client for Scala 3 inspired by Anorm and Quill

akka-streams cassandra client database flink pekko-streams reactive reactive-streams scala

Last synced: 12 Oct 2024

https://github.com/fiware/tutorials.big-data-flink

:blue_book: FIWARE 305: Real-time Processing of Context Data using Apache Flink

apache-flink big-data-analytics fiware fiware-cosmos flink orion-flink-connector tutorial

Last synced: 17 Nov 2024

https://github.com/duhanmin/bigdata-sql-parser

数据血缘，支持spark sql,hive sql,pg sql,presto sql,mysql sql,tidb sql, flink sql, datax血缘,spark/flink jar 运行命令的血缘解析；支持with语法

datax flink hive mysql postgresql presto spark tidb trino

Last synced: 05 Nov 2024

https://github.com/confluentinc/vscode

Confluent for Visual Studio Code

apache-flink apache-kafka confluent confluent-cloud confluent-platform flink kafka kafka-consumer kafka-producer kstreams message schema schema-registry scripts templates vscode vscode-extension

Last synced: 11 Oct 2024

https://github.com/da91666/daph

Daph是一个通用的数据同步与数据处理平台级工具，既具有丰富的数据同步能力，又具有强大的数据处理能力，一站式满足数据开发所有需求，可用于构建可视化配置化的数据同步与数据处理平台。

bigdata etl flink spark

Last synced: 11 Oct 2024

https://github.com/dylanmei/statefun-ops

Use Flink's Stateful Functions as a control-plane technology for operating a streaming-platform

flink flink-statefun kafka platform-engineering prometheus statefun

Last synced: 28 Dec 2024

https://github.com/jyothsnasrinivas/eta-flink

Apache Flink Example in Eta

big-data eta flink haskell

Last synced: 29 Nov 2024

https://github.com/hexnn/stark

基于Spark+Debezium打造的简单易用、超高性能大数据治理引擎，适用于批流一体的数据集成和数据分析场景，支持CDC实时数据采集，支持海量数据同步、数据建模和OLAP数据分析

cdc datax debezium flink hadoop seatunnel spark

Last synced: 11 Oct 2024

https://github.com/uk0/rocketmq-flink-plugin

rocketmq-flink-plugin sink and source

flink rocketmq

Last synced: 15 Nov 2024

https://github.com/touk/nussknacker-quickstart

Docker Demo for Nussknacker - A visual tool to define and run real-time decision algorithms. Brings agility to business teams, liberates developers to focus on technology.

flink kafka lowcode nussknacker streaming

Last synced: 22 Dec 2024

https://github.com/java-edge/flink-tutorial

第四代大数据计算引擎Flink - 从入门到实战

flink flink-dataset flink-examples flink-streaming

Last synced: 28 Oct 2024

https://github.com/garystafford/flink-kafka-demo

Apache Flink/Apache Kafka streaming data analytics demonstration using Streaming Synthetic Sales Data Generator

analytics apache-flink apache-kafka flink kafka streaming-data

Last synced: 06 Dec 2024

https://github.com/trk54ylmz/musik

Musik: Music recognition application

audio-processing cassandra flink kafka stream-processing

Last synced: 18 Nov 2024

https://github.com/naupio/pical

(Work In Process) pita is a general distributed computation system with Erlang language base on DAG model. This project is inspired by DouBan 's DPark and Apache Spark.

big-data bigdata dag data distributed distributed-computing distributed-systems erlang erlang-otp flink spark

Last synced: 13 Nov 2024

https://github.com/isopropylcyanide/flink-couchbase-data-sink

A Flink job that reads a Json file (either one-time or continous poll) as its source and dumps it to couchbase as a sink using the asynchronous Couchbase SDK.

apache-flink couchbase couchbase-server couchbase-sink database-store flink flink-examples flink-job

Last synced: 12 Nov 2024

https://github.com/haoxins/flink-apis

The Golang APIs for Apache Flink K8s Operator APIs (CRDs) and Flink Rest APIs.

api-k8s api-rest flink flink-operator golang

Last synced: 08 Nov 2024

https://github.com/frett27/osm-flink-tools

Tools for helping integrate OSM data in hdfs using csv or avro files, using flink (implements PBF stream reading). This project reconstruct the relations for ways, polygons to be used in GIS Analysis

bigdata csv flink gis import java openstreetmap

Last synced: 15 Oct 2024

https://github.com/sergio11/traffic_sentinel_architecture

🚀 Traffic Sentinel: A scalable IoT system using Fog nodes and Apache Flink to process 📷 IP camera streams, powered by YOLO for intelligent 🚗 traffic monitoring on highways. 🛣️

apache-nifi flask flink flink-stream-processing fog fog-computing iot-platform minio minio-server mongodb nifi nifi-processor redis tkinter ultralytics vault vehicle-detection vehicle-tracking yolo

Last synced: 14 Dec 2024

https://github.com/mikeroyal/apache-beam-guide

Apache Beam Guide

beam domain-specific-language enterprise etl-pipeline flink flink-stream-processing python

Last synced: 24 Oct 2024

https://github.com/apache/flink-connector-prometheus

Apache flink

flink prometheus

Last synced: 07 Oct 2024

https://github.com/nduytg/flink_prometheus_sd

A simple service for discovering Flink cluster on Hadoop Yarn

flink flink-clusters flink-prometheus-sd go golang hadoop hadoop-yarn prometheus service-discovery yarn

Last synced: 09 Dec 2024

https://github.com/collabh/flink-deployer

flink部署器,支持flink on yarn/k8s,基于Flink自带ClusterDescriptor的不同实现进行通用封装

deployer flink k8s yarn

Last synced: 12 Oct 2024

https://github.com/jorgermduarte/java-kafka-cluster-avro

This project sets up a Kafka environment for development purposes. It includes multiple Kafka brokers, Zookeeper, a Schema Registry, and Kafka UI for easy management and visualization.

avro cluster docker docker-compose flink java kafka kafka-ui schema-registry zookeeper