Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/risingwavelabs/risingwave

Stream processing and management platform.
https://github.com/risingwavelabs/risingwave

data-engineering database kafka materialized-view postgresql rust stream-processing

Last synced: about 1 month ago
JSON representation

Stream processing and management platform.

Awesome Lists containing this project

README 

        




  

    

    

  







### šŸŒŠ Ride the Wave of Streaming Data.







  Docs | Benchmarks | Demos









  

    Release

  

  

    Slack

  

  

    X

  

  

    YouTube

  




RisingWave is a stream processing and management platform designed to offer the simplest and most cost-effective way to process, analyze, and manage real-time event data ā€” with built-in support for the [Apache Icebergā„¢](https://iceberg.apache.org/) open table format. It provides both a Postgres-compatible [SQL interface](https://docs.risingwave.com/sql/overview) and a DataFrame-style [Python interface](https://docs.risingwave.com/python-sdk/intro).



RisingWave can ingest millions of events per second, continuously join and analyze live streams with historical data, serve ad-hoc queries at low latency, and persist fresh, consistent results to Apache Icebergā„¢ or any other downstream system.



![RisingWave](./docs/dev/src/images/architecture_20250127.png)



## Try it out in 60 seconds



Install RisingWave standalone mode:

```shell

curl -L https://risingwave.com/sh | sh

```



To learn about other installation options, such as using a Docker image, see [Quick Start](https://docs.risingwave.com/docs/current/get-started/).



## Stream, Store, and Query ā€” All in One



RisingWave delivers a full **end-to-end streaming data platform** ā€” combining real-time processing with built-in storage and open-format persistence.



It supports:



* **Ingestion:** Ingest millions of events per second from streaming and batch sources.

* **Stream processing:** Perform real-time incremental processing to join and analyze live data with historical tables.

* **Delivery:** Deliver fresh, consistent results to data lakes (e.g., Apache Icebergā„¢) or any destination.



What sets RisingWave apart is its integrated storage engine:



* **Online serving:** Row-based storage optimized for point and range queries with single-digit millisecond latency.

* **Offline persistence:** Built-in Apache Icebergā„¢ integration for low-cost, durable storage with open access for external query engines.



With RisingWave, real-time data isnā€™t just processed ā€” itā€™s stored, queried, and shared across your entire stack.



## Key design decisions



RisingWave is designed to be easier to use and more cost-efficient:



### PostgreSQL compatibility



* **Seamless integration:** Connects via the PostgreSQL wire protocol, working with psql, JDBC, and any Postgres tool.

* **Expressive SQL:** Supports structured, semi-structured, and unstructured data with a familiar SQL dialect.

* **No manual state tuning:** Eliminates complex state management configurations.



### S3 as primary storage



RisingWave stores tables, materialized views, and internal states of stream processing jobs in S3 (or equivalent object storage), providing:

- **High performance:** Optimized for complex queries, including joins and time windowing.

- **Fast recovery:** Restores from system failures within seconds.

- **[Dynamic scaling](https://docs.risingwave.com/deploy/k8s-cluster-scaling):** Instantly adjusts resources to handle workload spikes.



### Elastic disk cache



Beyond caching hot data in memory, RisingWave supports [**elastic disk cache**](https://docs.risingwave.com/get-started/disk-cache), a powerful performance optimization that uses local disks or EBS for efficient data caching. This minimizes access to S3, lowering processing latency and cutting S3 access costs.



### Apache Icebergā„¢ native support

RisingWave [**natively integrates with Apache Icebergā„¢**](https://docs.risingwave.com/iceberg/overview), enabling continuous ingestion of streaming data into Iceberg tables. It can also read directly from Iceberg, perform automatic compaction, and maintain table health over time. Since Iceberg is an open table format, results are accessible by other query engines ā€” making storage not only cost-efficient, but interoperable by design.



## In what use cases does RisingWave excel?

RisingWave is particularly effective for the following use cases:



* **Streaming analytics**: Achieve sub-second data freshness in live dashboards, ideal for high-stakes scenarios like stock trading, sports betting, and IoT monitoring.

* **Event-driven applications**: Develop sophisticated monitoring and alerting systems for critical applications such as fraud and anomaly detection.

* **Real-time data enrichment**: Continuously ingest data from diverse sources, conduct real-time data enrichment, and efficiently deliver the results to downstream systems.

* **Feature engineering**: Transform batch and streaming data into features in your machine learning models using a unified codebase, ensuring seamless integration and consistency.



## Production deployments



[**RisingWave Cloud**](https://cloud.risingwave.com) offers the easiest way to run RisingWave in production.



For **Docker deployment**, please refer to [Docker Compose](https://docs.risingwave.com/docs/current/risingwave-docker-compose/).



For **Kubernetes deployment**, please refer to [Kubernetes with Helm](https://docs.risingwave.com/docs/current/risingwave-k8s-helm/) or [Kubernetes with Operator](https://docs.risingwave.com/docs/current/risingwave-kubernetes/).



## Community



Looking for help, discussions, collaboration opportunities, or a casual afternoon chat with our fellow engineers and community members? Join our [Slack workspace](https://risingwave.com/slack)!



## Notes on telemetry



RisingWave uses [Scarf](https://scarf.sh/) to collect anonymized installation analytics. These analytics help support us understand and improve the distribution of our package. The privacy policy of Scarf is available at [https://about.scarf.sh/privacy-policy](https://about.scarf.sh/privacy-policy).



RisingWave also collects anonymous usage statistics to better understand how the community is using RisingWave. The sole intention of this exercise is to help improve the product. Users may opt out easily at any time. Please refer to the [user documentation](https://docs.risingwave.com/docs/current/telemetry/) for more details.



## License



RisingWave is distributed under the Apache License (Version 2.0). Please refer to [LICENSE](LICENSE) for more information.



## Contributing



Thanks for your interest in contributing to the project! Please refer to [RisingWave Developer Guide](https://risingwavelabs.github.io/risingwave/) for more information.