https://github.com/streamnative/pulsar-flink-state-migrate
pulsar flink state migrate tools
https://github.com/streamnative/pulsar-flink-state-migrate
Last synced: 8 months ago
JSON representation
pulsar flink state migrate tools
- Host: GitHub
- URL: https://github.com/streamnative/pulsar-flink-state-migrate
- Owner: streamnative
- Created: 2021-03-22T12:52:00.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2021-04-20T08:56:45.000Z (almost 5 years ago)
- Last Synced: 2025-03-24T01:18:19.958Z (11 months ago)
- Language: Java
- Size: 35.2 KB
- Stars: 1
- Watchers: 20
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pulsar-flink-state-migrate
pulsar flink state migrate tools
## Usage
```
Usage:
-uid flink uid for pulsar source
-savepointPath flink savepoint path
-newStatePath new flink savepoint path
```
This tool is used to migrate from pulsar-flink-connector versions prior to `2.4.27` to [pulsar-flink-connector version `2.5.8.3`](https://repo1.maven.org/maven2/io/streamnative/connectors/).
At the same time, the Flink client version will be upgraded from `1.9.x` to `1.11.x`, the Pulsar client version will be upgraded from `2.5.2` to `2.7.0`.
This upgrade tool does not support batch flink job upgrade. When you need batch, please call it externally from batch.
It needs to be submitted as a Flink job for upgrade.
example:
`bin/flink run pulsar-flink-state-migrate-1.11-0.0.1.jar -uid pulsar-source-id -savepointPath savepoint/savepoint-1af2b3-a7db17bbd4b1 -newStatePath /new-savepoint`
`uid` is the multiple uid set for Pulsar Source in the user code. multiple uid use `,` join, example: `pulsar-source-id,pulsar-source-id2,pulsar-source-id3`
`savepointPath` parameter is the old savepoint directory or flink state`_metadata` file
The `newStatePath` parameter is the output directory, where the `_metadata` file will be generated