https://github.com/cyberdelia/flink-kotlin

Kotlin support for Apache Flink
https://github.com/cyberdelia/flink-kotlin

apache-flink flink kotlin serialization

Last synced: about 2 months ago
JSON representation

Kotlin support for Apache Flink

• Host: GitHub
• URL: https://github.com/cyberdelia/flink-kotlin
• Owner: cyberdelia
• License: mit
• Created: 2022-10-18T02:44:47.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-09-09T00:02:40.000Z (4 months ago)
• Last Synced: 2024-09-10T01:49:09.178Z (4 months ago)
• Topics: apache-flink, flink, kotlin, serialization
• Language: Kotlin
• Homepage: https://github.com/cyberdelia/flink-kotlin
• Size: 211 KB
• Stars: 16
• Watchers: 6
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Kotlin support for Apache Flink

This package provides type information and

specialized [type serializers](https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/datastream/fault-tolerance/serialization/types_serialization/)

for kotlin data classes (including `Pair` and `Triple`), as well as for Kotlin `Map` and `Collection` types.

## Installation

The package is available on [Maven Central](https://search.maven.org/search?q=g:com.lapanthere%20AND%20a:flink-kotlin)

and [Github](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-gradle-registry#using-a-published-package),

using Gradle:

```kotlin

implementation("com.lapanthere:flink-kotlin:0.1.0")

```

## Usage

### TypeInformation

Using the `createTypeInformation` that will return a Kotlin friendly `TypeInformation` for data classes, collections,

maps, etc:

```kotlin

dataStream.process(

    processFunction(),

    createTypeInformation()

)

```

It also supports fields name in the definition of keys, i.e. you will be able to use name of fields directly:

```kotlin

dataStream.join(another).where("name").equalTo("personName")

```

You can also annotate your data classes with the `@TypeInfo` annotation:

```kotlin

@TypeInfo(DataClassTypeInfoFactory::class)

data class Record(

    val name: String,

    val value: Long

)

```

## Schema evolution

Schema evolution for data classes follow this set of rules:

1. Fields can be removed. Once removed, the previous value for the removed field will be dropped in future checkpoints

   and savepoints.

2. New fields can be added. The new field will be initialized to the default value for its type, as defined by Java.

3. Declared fields types cannot change.

4. Class name of type cannot change, including the namespace of the class.

5. Null fields are not supported.