https://github.com/redis/spring-batch-redis

Spring Batch extension for Redis
https://github.com/redis/spring-batch-redis

batch redis spring

Last synced: 3 months ago
JSON representation

Spring Batch extension for Redis

• Host: GitHub
• URL: https://github.com/redis/spring-batch-redis
• Owner: redis
• License: apache-2.0
• Created: 2020-03-19T22:59:24.000Z (almost 5 years ago)
• Default Branch: main
• Last Pushed: 2024-04-25T09:04:53.000Z (8 months ago)
• Last Synced: 2024-04-25T09:47:31.589Z (8 months ago)
• Topics: batch, redis, spring
• Language: Java
• Homepage:
• Size: 3.26 MB
• Stars: 43
• Watchers: 15
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.adoc
  • License: LICENSE

Awesome Lists containing this project

README

        
= Spring Batch Redis

:linkattrs:

:project-owner:   redis

:project-name:    spring-batch-redis

:project-group:   com.redis

:project-version: 4.4.9

:artifact-id:     spring-batch-redis-core

image:https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml/badge.svg["Build Status", link="https://github.com/{project-owner}/{project-name}/actions/workflows/early-access.yml"]

image:https://img.shields.io/maven-central/v/{project-group}/{project-name}[Download, link="https://search.maven.org/#search|ga|1|{project-name}"]

image:https://codecov.io/gh/{project-owner}/{project-name}/branch/main/graph/badge.svg["Coverage", link="https://codecov.io/gh/{project-owner}/{project-name}"]

ItemReader and ItemWriter implementations for Redis.

== Getting Started

=== Maven

Add Spring Batch Redis dependency to your POM file:

[source,xml]

[subs="verbatim,attributes"]

.pom.xml

----

    {project-group}

    {artifact-id}

    {project-version}

----

=== Gradle

Add Spring Batch Redis dependency to your `build.gradle` file

[source,groovy]

[subs="attributes"]

.build.gradle

----

dependencies {

    implementation '{project-group}:{artifact-id}:{project-version}'

}

----

== Data Types

`RedisItemReader` and `RedisItemWriter` support a single data type called `KeyValue` that has the following fields:

* `key` of type `K` representing the Redis key.

* `type` of type `String` representing the type of that Redis key (`string`, `hash`, `list`, ...).

* `ttl` of type `long` representing the key's absolute expiration epoch time in milliseconds.

* `value` of type:

** `byte[]` for `dump` reader.

** `Object` for `struct` reader. The type of the object depends on the type of the Redis key (`V` for `string`, `Map` for `hash`, `List` for `list`, ...). See <<_data_structures,Data Structures>> below for more details.

* `mem` of type `long` representing the memory usage of that key in Redis. This is only populated when `memUsageLimit` on `MemKeyValueRead` operation is strictly greater than 0.

[[_data_structures]]

=== Data Structures

Values for `struct` readers `RedisItemReader` are Java object representations of the underlying Redis data-structures:

* `hash`: `java.util.Map`

* `list`: `java.util.List`

* `set`: `java.util.Set`

* `stream`: `java.util.List>`

* `string`: `V`

* `zset`: `java.util.Set>`

* `json`: `V`

* `timeseries`: `java.util.List>`

These generic classes `` depend on the `RedisCodec` used to initialize the reader:

* `StringCodec`: `java.lang.String`

* `ByteArrayCodec`: `byte[]`

== Item Readers

`RedisItemReader` exposes 2 operation modes:

* Snapshot: relies on SCAN command to iterate over the keys whose values will be read to produce key/values.

* Live (AKA continuous): listens to changes in the keyspace via notifications and produces corresponding key/values.

== Item Writers

`RedisItemWriter` can perform both inserts or deletes depending on the value and TTL in the incoming object.

If value is null or TTL is -2 then the `DEL` command is called, otherwise a write is performed.

Item writers support two different payload (value) types:

=== Key Dump

The writer accepts key dumps (`KeyValue` with a `byte[]` value) and calls the RESTORE command with the byte array and TTL if any.

=== Data Structure

The writer takes `KeyValue` objects and calls the write command specific to the data type:

* `hash`: `HSET`

* `json`: `JSON.SET`

* `list`: `RPUSH`

* `set`: `SADD`

* `stream`: `XADD`

* `string`: `SET`

* `zset`: `ZADD`

* `timeseries`: `TS.ADD`

If TTL >= 0 then an additional call is made to the `EXPIREAT` command.

== Usage

Refer to https://github.com/redis/spring-batch-redis/blob/main/subprojects/spring-batch-redis-test/src/test/java/com/redis/spring/batch/test/BatchTests.java[unit tests] for usage examples.