Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/apache/streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://github.com/apache/streampark

apache development-framework easy-to-use etl-pipeline operation-platform streaming streampark

Last synced: 3 days ago
JSON representation

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.

Host: GitHub
URL: https://github.com/apache/streampark
Owner: apache
License: apache-2.0
Created: 2019-05-27T05:56:03.000Z (over 5 years ago)
Default Branch: dev
Last Pushed: 2025-02-11T04:10:04.000Z (10 days ago)
Last Synced: 2025-02-11T04:26:37.698Z (10 days ago)
Topics: apache, development-framework, easy-to-use, etl-pipeline, operation-platform, streaming, streampark
Language: Java
Homepage: https://streampark.apache.org/
Size: 59.2 MB
Stars: 3,966
Watchers: 76
Forks: 1,021
Open Issues: 246
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        

Apache StreamPark

==========



**StreamPark**: a streaming application development framework and cloud-native real-time computing platform.

[![GitHub stars](https://img.shields.io/github/stars/apache/streampark?style=for-the-badge&label=stars)](https://github.com/apache/streampark/stargazers)

[![Latest release](https://img.shields.io/github/v/release/apache/streampark.svg?style=for-the-badge&label=release)](https://github.com/apache/streampark/releases)

[![X Follow](https://img.shields.io/badge/2K%2B-follow?style=for-the-badge&logo=X&label=%40ASFStreamPark)](https://x.com/ASFStreamPark)

## 📊 Overview

**StreamPark** is a streaming application development framework and cloud-native real-time computing platform. Designed to simplify the development and management of streaming applications, StreamPark provides a development framework for constructing stream processing applications using Apache Flink and Apache Spark, along with a professional streaming application management platform. The platform encompasses application development, debugging, interactive querying, deployment, operation, maintenance, and more. Originally named StreamX, the project was renamed StreamPark in August 2022 and officially graduated as an Apache Top-Level Project (TLP) in January 2025.

* **Streaming Application Development Framework**

    * StreamPark provides a Streaming application framework to simplify Apache Flink and Apache Spark application development, offering plug-and-play connectors to reduce the learning curve and development complexity.

* **Real-Time Computing Platform**

    * StreamPark delivers a one-stop real-time computing platform with core capabilities including application development, deployment, management and monitoring, etc.

* **Supports Batch & Streaming**

    * Supports both Apache Flink and Apache Spark, enabling seamless integration of streaming and batch processing, with multi-engine/multi-version support on a single platform.

* **Supports multi-engine/multi-version**

    * StreamPark supports multiple versions of Apache Flink and Apache Spark, enabling users to develop and manage applications for different versions of these streaming engines within a single framework.

* **Multi-environment Compatibility**

    * Compatible with various cluster environments, users can submit Flink and Spark applications to standalone clusters, YARN (Hadoop 2.x/3.x), and Kubernetes.

* **Rich Ecosystems**

    * Compatible with mainstream open-source technologies (e.g., Apache Flink, Apache Spark, Apache Paimon, Apache Doris) and ML/AI ecosystems, ensuring flexible technology adoption.

* **Easy to use**

    * StreamPark is designed to lower the learning curve and entry barrier. Only one service, deployment easy, allowing even beginners to get started within minutes.



## 🚀 QuickStart

```shell

curl -L https://streampark.apache.org/quickstart.sh | sh

```

https://github.com/user-attachments/assets/dd7d5a89-bc28-4ccc-9ad5-258925fc4f34

more:

- [Start with Docker](docker/README.md)

- [Start with Kubernetes](helm/README.md)

## 🔨 How to Build

```shell

./build.sh

```

🗄 how to [Development](https://streampark.apache.org/docs/development/development)

## ⬇️ Downloads

Please head to the [releases page](https://streampark.apache.org/download) to download a release of Apache StreamPark.

## 📚 Docs

[Official Documentation](https://streampark.apache.org/docs/get-started)

## 💋 Our users

Various companies and organizations use Apache StreamPark for research, production and commercial products. Are you using this project? [Welcome to add your company](https://github.com/apache/streampark/issues/163)!

![Our users](https://streampark.apache.org/image/users.png?20250214)

## 🤝 Contribution

[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/apache/streampark/pulls)

### 🙋 Submit Pull Request and Issues

You can submit any ideas as [pull requests](https://github.com/apache/streampark/pulls) or as [issues](https://github.com/apache/streampark/issues/new/choose).

> If you're new to posting issues, we ask that you read [*How To Ask Questions The Smart Way*](http://www.catb.org/~esr/faqs/smart-questions.html) (**This guide does not provide actual support services for this project!**), [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html) prior to posting. Well written bug reports help us help you!

### 🍻 How to Contribute

We welcome your suggestions, comments (including criticisms), comments and contributions. See [How to Contribute](https://streampark.apache.org/community/submit_guide/submit_code) and [Code Submission Guide](https://streampark.apache.org/community/submit_guide/code_style_and_quality_guide)

Thank you to all the people who already contributed to Apache StreamPark!

## 💬 Contact Us

Contact us through the following mailing list.

| Name                                                          | Mailing list                                                                                                                                                                               | 

|:--------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

| [[email protected]](mailto:[email protected]) | [Subscribe](mailto:[email protected])、[Unsubscribe](mailto:[email protected]) 、[Archives](http://mail-archives.apache.org/mod_mbox/streampark-dev/) |

## 💿 Social Media

- [X (Twitter)](https://twitter.com/ASFStreamPark)

- [Zhihu](https://www.zhihu.com/people/streampark) (in Chinese)

- [bilibili](https://space.bilibili.com/455330087) (in Chinese)

- WeChat Official Account (in Chinese, scan the QR code to follow)




## 📜 License

Licensed under the [Apache License, Version 2.0](LICENSE)