Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-modern-bigdata

A list of awesome modern big data libraries, frameworks and platforms.
https://github.com/yangliuyu/awesome-modern-bigdata

Last synced: 1 day ago
JSON representation

  • Ingestion

  • File Storage

    • Fluid - native Distributed Dataset Orchestrator and Accelerator for data-intensive applications, such as big data and AI applications.
    • MINIO - performance, S3 compatible object storage.

  • OLAP Query Engine

  • Database

    • StarRocks - gen sub-second MPP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics and ad-hoc query.
    • TiKV - source, distributed, and transactional key-value database. Unlike other traditional NoSQL systems, TiKV not only provides classical key-value APIs, but also transactional APIs with ACID compliance.
    • Clickhouse - oriented database management system (DBMS) for online analytical processing of queries (OLAP).

  • Data Lake

    • Flink Table Store
    • Iceberg - performance format for huge analytic tables. Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, and Hive to safely work with the same tables, at the same time.
    • Flink Table Store

  • Data Analytics

    • Zeppelin - based notebook that enables interactive data analytics. You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala and more.

  • Data Visualization

  • Orchestration

    • StreamPipes - service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
Programming Languages
Java 8 TypeScript 1 Rust 1 Go 1
Categories
Database 3 Data Lake 3 File Storage 2 Ingestion 2 Data Visualization 2 OLAP Query Engine 1 Orchestration 1 Data Analytics 1
Sub Categories
Keywords
big-data 6 flink 5 paimon 4 real-time-analytics 3 spark 3 database 2 sql 2 lakehouse 2 java 2 data-ingestion 2 streaming-datalake 2 table-store 2 schema-evolution 2 real-time 2 postgresql 2 data-visualization 2 mysql 2 kafka 2 etl 2 elt 2 change-data-capture 2 distributed 2 data-integration 2 data-pipeline 2 batch 2 cdc 2 query 1 analytics 1 cloudnative 1 datalake 1 delta-lake 1 distributed-database 1 hudi 1 presto 1 hive 1 hadoop 1 data 1 kubernetes 1 distributed-cache 1 data-abstraction 1 alluxio 1 tidb 1 tikv 1 javascript 1 nosql 1 scala 1 zeppelin 1 dashboard 1 davinci 1 display 1