Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/apache/paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://github.com/apache/paimon

big-data data-ingestion flink paimon real-time-analytics spark streaming-datalake table-store

Last synced: about 1 month ago
JSON representation

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

Host: GitHub
URL: https://github.com/apache/paimon
Owner: apache
License: apache-2.0
Created: 2022-01-12T03:13:15.000Z (almost 3 years ago)
Default Branch: master
Last Pushed: 2024-05-21T07:03:43.000Z (6 months ago)
Last Synced: 2024-05-21T07:21:09.576Z (6 months ago)
Topics: big-data, data-ingestion, flink, paimon, real-time-analytics, spark, streaming-datalake, table-store
Language: Java
Homepage: https://paimon.apache.org/
Size: 31.4 MB
Stars: 1,987
Watchers: 72
Forks: 798
Open Issues: 360
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

awesome-datalake - Apache Paimon - Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations. (Open Table Formats)
awesome-datalake - Apache Paimon - Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations. (Open Table Formats)

README

![Paimon](https://github.com/apache/paimon/blob/master/docs/static/paimon-simple.png)

[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://the-asf.slack.com/archives/C053Q2NCW8G)

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark
for both streaming and batch operations. Paimon innovatively combines lake format and LSM structure, bringing realtime
streaming updates into the lake architecture.

Background and documentation are available at https://paimon.apache.org

`Paimon`'s former name was `Flink Table Store`, developed from the Flink community. The architecture refers to some
design concepts of Iceberg. Thanks to Apache Flink and Apache Iceberg.

## Collaboration

Paimon tracks issues in GitHub and prefers to receive contributions as pull requests.

## Mailing Lists

Name
Subscribe
Digest
Unsubscribe
Post
Archive

user@paimon.apache.org

User support and questions mailing list

Subscribe
Subscribe
Unsubscribe
Post

Archives

dev@paimon.apache.org

Development related discussions

Subscribe
Subscribe
Unsubscribe
Post

Archives

Please make sure you are subscribed to the mailing list you are posting to! If you are not subscribed to the mailing list, your message will either be rejected (dev@ list) or you won't receive the response (user@ list).

## Slack

You can join the Paimon community on Slack. Paimon channel is in ASF Slack workspace.

- Anyone with an @apache.org email address can become a full member of the ASF Slack workspace.
Search [Paimon channel](https://the-asf.slack.com/archives/C053Q2NCW8G) and join it.
- If you don't have an @apache.org email address, you can email to `[email protected]` to apply for an
[ASF Slack invitation](https://infra.apache.org/slack.html). Then join [Paimon channel](https://the-asf.slack.com/archives/C053Q2NCW8G).

## Building

JDK 8/11 is required for building the project. Maven version >=3.3.1.

- Run the `mvn clean install -DskipTests` command to build the project.
- Run the `mvn spotless:apply` to format the project (both Java and Scala).
- IDE: Mark `paimon-common/target/generated-sources/antlr4` as Sources Root.

## How to Contribute

[Contribution Guide](https://paimon.apache.org/docs/master/project/contributing/).

## License

The code in this repository is licensed under the [Apache Software License 2](LICENSE).