https://github.com/mawngo/batch4j

Simple batch processing library for java
https://github.com/mawngo/batch4j

batch-processing java

Last synced: 7 months ago
JSON representation

Simple batch processing library for java

• Host: GitHub
• URL: https://github.com/mawngo/batch4j
• Owner: mawngo
• License: mit
• Created: 2024-06-13T12:37:29.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2024-06-15T05:07:46.000Z (almost 2 years ago)
• Last Synced: 2025-01-22T07:19:31.424Z (over 1 year ago)
• Topics: batch-processing, java
• Language: Java
• Homepage: https://central.sonatype.com/artifact/io.github.mawngo/batch4j
• Size: 95.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Batch4j

Simple batch processing library for java.

## Installation

Require java 8+.

Add library to gradle dependencies.

```groovy

dependencies {

    implementation 'io.github.mawngo:batch4j:1.0.0'

}

```

# Usage

Example usage:

```java

import java.util.concurrent.Executors;

import java.util.concurrent.TimeUnit;

import io.github.mawngo.batch4j.BatchProcessors;

import io.github.mawngo.batch4j.RunningProcessor;

import io.github.mawngo.batch4j.ParallelProcessor;

import io.github.mawngo.batch4j.handlers.BatchMerger;

@Test

void example() {

    final AtomicInteger sum = new AtomicInteger(0);

    final RunningProcessor processor = BatchProcessors

        .newBuilder((List list) -> sum.updateAndGet(i -> i + list.size()))

        .maxWait(1000, TimeUnit.MILLISECONDS)       // maximum wait time before processing

        .maxItem(100)                               // maximum item in batch before processing

        .build(BatchMerger.mergeToList())

        .run();

    for (int i = 0; i < 1_000_000; i++) {

        processor.put(i);

    }

    processor.close();

    Assertions.assertEquals(1_000_000, sum.get());

}

@Test

void exampleParallel() {

    final AtomicInteger sum = new AtomicInteger(0);

    final ParallelProcessor processor = BatchProcessors

        .newBuilder((List list) -> sum.updateAndGet(i -> i + list.size()))

        .maxWait(1000, TimeUnit.MILLISECONDS)

        .maxItem(100)

        .parallelTo(Executors.newCachedThreadPool()) // specify executor for parallel processing

        //.parallel("2-10")                          // or specify parallel for auto creating one.

        .build(BatchMerger.mergeToList())

        .run();

    for (int i = 0; i < 1_000_000; i++) {

        processor.put(i);

    }

    processor.close();

    processor.closeExecutor();                        // optional, closing backing executor.

    Assertions.assertEquals(1_000_000, sum.get());

}

```