https://github.com/atleon/atleon
Lightweight infinite reactive stream processing framework
https://github.com/atleon/atleon
java kafka rabbitmq reactive-streams reactor sns sns-client sqs sqs-client
Last synced: 2 months ago
JSON representation
Lightweight infinite reactive stream processing framework
- Host: GitHub
- URL: https://github.com/atleon/atleon
- Owner: atleon
- License: apache-2.0
- Created: 2021-08-24T22:21:50.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2026-04-02T16:52:36.000Z (3 months ago)
- Last Synced: 2026-04-03T01:10:50.733Z (3 months ago)
- Topics: java, kafka, rabbitmq, reactive-streams, reactor, sns, sns-client, sqs, sqs-client
- Language: Java
- Homepage: http://atleon.io
- Size: 3.1 MB
- Stars: 36
- Watchers: 3
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
# Atleon
[](https://opensource.org/licenses/Apache-2.0)
[](https://github.com/atleon/atleon/actions/workflows/main.yml)
[](https://search.maven.org/artifact/io.atleon/atleon-core)
[](https://javadoc.io/doc/io.atleon/atleon-core)
Atleon is a lightweight reactive stream processing framework that scalably transforms data from any supported infrastructure, and allows sending that data nearly anywhere, while _seamlessly_ maintaining **at** **le**ast **on**ce processing guarantees.
Atleon is based on [Reactive Streams](https://www.reactive-streams.org/) and backed by [Project Reactor](https://projectreactor.io/). There are two levels of client APIs offered:
- **Low-Level**: Low-level client APIs are thin reactive wrappers around third-party native infrastructure clients. These APIs are defined in terms of Reactor-native types (e.g. [`Flux`](https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html) and [`Mono`](https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html)), and emit elements that contain (or otherwise reference) native infrastructure types. Consumption of messages is facilitated via `Receiver` implementations, with emitted elements containing callbacks for acknowledgement (both positive/successful and negative/failed). Production of messages is facilitated via `Sender` implementations, with emitted elements (implementations of [`SenderResult`](../../blob/main/base/core/src/main/java/io/atleon/core/SenderResult.java)) containing metadata about successes or errors indicating production failure.
- **High-Level**: High-level client APIs decorate low-level clients with an Atleon-native abstraction called [`Alo`](../../blob/main/base/core/src/main/java/io/atleon/core/Alo.java) (short for At Least Once). `Alo` facilitates per-element context which (at minimum) provides access to acknowledgement, and allows for decorating its context with functionality like metrics, distributed tracing, and application-specific metadata. In order to simplify building streams that emit/consume `Alo`, high-level clients produce a special [`AloFlux`](../../blob/main/base/core/src/main/java/io/atleon/core/AloFlux.java) type that bridges operations to an underlying `Flux>`, and allows for defining reactive pipelines purely in terms of data typing (`T`), while providing automatic context propagation.
## Documentation and Getting Started
Atleon documentation and instructions on how to get started are available in the [Wiki](../../wiki).
### Low-Level Client Example
The following is an example of using low-level clients in Atleon:
```java
import io.atleon.kafka.KafkaReceiver;
import io.atleon.kafka.KafkaReceiverOptions;
import io.atleon.kafka.KafkaReceiverRecord;
import io.atleon.kafka.KafkaSender;
import io.atleon.kafka.KafkaSenderOptions;
import io.atleon.kafka.KafkaSenderRecord;
import org.apache.kafka.clients.CommonClientConfigs;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
import reactor.core.Disposable;
import reactor.core.publisher.Flux;
import java.time.Duration;
import java.util.Collections;
public class Example {
public static void main(String[] args) throws Exception {
KafkaSenderOptions senderOptions = KafkaSenderOptions.newBuilder()
.producerProperty(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")
.producerProperty(CommonClientConfigs.CLIENT_ID_CONFIG, "example")
.producerProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName())
.producerProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName())
.build();
KafkaReceiverOptions receiverOptions = KafkaReceiverOptions.newBuilder()
.consumerProperty(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")
.consumerProperty(CommonClientConfigs.CLIENT_ID_CONFIG, "example")
.consumerProperty(ConsumerConfig.GROUP_ID_CONFIG, "consumer-group-id")
.consumerProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())
.consumerProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName())
.build();
// Periodically produce records
KafkaSender sender = KafkaSender.create(senderOptions);
Disposable production = Flux.interval(Duration.ofMillis(100))
.map(it -> KafkaSenderRecord.create("source-topic", it.toString(), it.toString(), it))
.transform(sender::send)
.doFinally(__ -> sender.close())
.subscribe();
// Consume produced records
Disposable consumption = KafkaReceiver.create(receiverOptions)
.receiveManual(Collections.singletonList("source-topic"))
.doOnNext(it -> System.out.printf("Consumed record with key=%s and value=%s", it.key(), it.value()))
.subscribe(KafkaReceiverRecord::acknowledge);
System.in.read(); // Added for posterity - Everything above will execute asynchronously
production.dispose(); // Stop producing
consumption.dispose(); // Stop consuming
}
}
```
### High-Level AloStream Example
The next example builds on the availability of low-level clients by switching to high-level `Alo` clients, removing manual production of data in favor of demonstrating sending/producing transformed messages, and wrapping the resulting stream definition in an implementation of `AloStream`:
```java
import io.atleon.core.SelfConfigurableAloStream;
import io.atleon.core.DefaultAloSenderResultSubscriber;
import io.atleon.kafka.AloKafkaReceiver;
import io.atleon.kafka.AloKafkaSender;
import io.atleon.kafka.KafkaConfigSource;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
import reactor.core.Disposable;
public class MyStream extends SelfConfigurableAloStream {
private final KafkaConfigSource configSource;
private final String sourceTopic;
private final String destinationTopic;
public MyStream(KafkaConfigSource configSource, String sourceTopic, String destinationTopic) {
this.configSource = configSource;
this.sourceTopic = sourceTopic;
this.destinationTopic = destinationTopic;
}
@Override
public Disposable startDisposable() {
AloKafkaSender sender = buildKafkaSender();
return buildKafkaReceiver()
.receiveAloRecords(sourceTopic)
.mapNotNull(it -> it.value() != null ? it.value().toUpperCase() : null) // Business logic goes here
.transform(sender.sendAloValues(destinationTopic, message -> message.substring(0, 1)))
.resubscribeOnError(name())
.doFinally(sender::close)
.subscribeWith(new DefaultAloSenderResultSubscriber<>());
}
private AloKafkaSender buildKafkaSender() {
return configSource
.withClientId(name())
.withKeySerializer(StringSerializer.class)
.withValueSerializer(StringSerializer.class)
.as(AloKafkaSender::create);
}
private AloKafkaReceiver buildKafkaReceiver() {
return configSource
.withClientId(name())
.withConsumerGroupId("consumer-group-id")
.withKeyDeserializer(StringSerializer.class)
.withValueDeserializer(StringSerializer.class)
.as(AloKafkaReceiver::create);
}
}
```
### Spring AloStream Example
Atleon has built-in integration with Spring, where a fully configured `AloStream` looks like the following:
`pom.xml`:
```xml
io.atleon
atleon-kafka
${atleon.version}
io.atleon
atleon-spring
${atleon.version}
```
`application.yml`:
```yaml
atleon:
config.sources:
- name: kafkaConfigSource
type: kafka
bootstrap.servers: localhost:9092
stream:
kafka:
destination.topic: output
source.topic: input
```
`MyStream.java`:
```java
import io.atleon.core.DefaultAloSenderResultSubscriber;
import io.atleon.kafka.AloKafkaReceiver;
import io.atleon.kafka.AloKafkaSender;
import io.atleon.kafka.KafkaConfigSource;
import io.atleon.spring.AutoConfigureStream;
import io.atleon.spring.SpringAloStream;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.context.ApplicationContext;
import reactor.core.Disposable;
@AutoConfigureStream
public class MyStream extends SpringAloStream {
private final KafkaConfigSource configSource;
public MyStream(ApplicationContext context) {
super(context);
this.configSource = context.getBean("kafkaConfigSource", KafkaConfigSource.class);
}
@Override
public Disposable startDisposable(MyStreamConfig config) {
AloKafkaSender sender = buildKafkaSender();
String destinationTopic = getRequiredProperty("stream.kafka.destination.topic");
return buildKafkaReceiver()
.receiveAloRecords(getRequiredProperty("stream.kafka.source.topic"))
.mapNotNull(it -> it.value() != null ? it.value().toUpperCase() : null) // Business logic goes here
.transform(sender.sendAloValues(destinationTopic, message -> message.substring(0, 1)))
.resubscribeOnError(name())
.doFinally(sender::close)
.subscribeWith(new DefaultAloSenderResultSubscriber<>());
}
private AloKafkaSender buildKafkaSender() {
return configSource
.withClientId(name())
.withKeySerializer(StringSerializer.class)
.withValueSerializer(StringSerializer.class)
.as(AloKafkaSender::create);
}
private AloKafkaReceiver buildKafkaReceiver() {
return configSource
.withClientId(name())
.withConsumerGroupId("consumer-group-id")
.withKeyDeserializer(StringDeserializer.class)
.withValueDeserializer(StringDeserializer.class)
.as(AloKafkaReceiver::create);
}
}
```
### Runnable Examples
The [examples module](examples) contains runnable classes showing Atleon in action and intended usage.
## Building
Atleon is built using Maven. Installing Maven locally is optional as you can use the Maven Wrapper:
```$bash
./mvnw clean verify
```
#### Docker
Atleon makes use of [Testcontainers](https://www.testcontainers.org/) for some unit tests. Testcontainers is based on Docker, so successfully building Atleon requires Docker to be running locally.
## Contributing
Please refer to [CONTRIBUTING](CONTRIBUTING.md) for information on how to contribute to Atleon
## Legal
This project is available under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0.html).