https://github.com/superstreamlabs/memphis.java
https://github.com/superstreamlabs/memphis.java
Last synced: 9 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/superstreamlabs/memphis.java
- Owner: superstreamlabs
- License: apache-2.0
- Created: 2022-12-01T11:28:58.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-05-20T11:46:42.000Z (about 2 years ago)
- Last Synced: 2025-06-09T04:32:33.710Z (about 1 year ago)
- Language: Java
- Size: 136 KB
- Stars: 8
- Watchers: 1
- Forks: 4
- Open Issues: 41
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
Simple as RabbitMQ, Robust as Apache Kafka, and Perfect for microservices.
Sandbox - Docs - Twitter - YouTube
**[Memphis](https://memphis.dev)** is a next-generation alternative to traditional message brokers.
A simple, robust, and durable cloud-native message broker wrapped with
an entire ecosystem that enables cost-effective, fast, and reliable development of modern queue-based use cases.
Memphis enables the building of modern queue-based applications that require
large volumes of streamed and enriched data, modern protocols, zero ops, rapid development,
extreme cost reduction, and a significantly lower amount of dev time for data-oriented developers and data engineers.
## Installation
```sh
$ gradle install memphis-dev
```
## Importing
```java
import com.memphis.Memphis;
```
### Connecting to Memphis
First, we need to create a connection to the broker using `MemphisConnection`.
```java
ClientOptions opts = new ClientOptions.Builder()
.host("")
.username("")
.token("")
.port("") // defaults to 6666
.reconnect(true) // defaults to true
.maxReconnect(3) // defaults to 3
.reconnectInterval(1500) // defaults to 1500 ms
.timeOut(1500) // defaults to 1500 ms
.keyFile("") // key_file, for TLS connection
.certFile("") // cert_file, for TLS connection
.caFile("") // ca_file, for TLS connection
.build();
MemphisConnection memphisConnection = MemphisConnection(opts);
```
Once connected, the entire functionalities offered by Memphis are available.
### Disconnecting from Memphis
To disconnect from Memphis, call `close()` on the memphis object.
```java
memphisConnection.close();
```
### Creating a Station
_If a station already exists nothing happens, the new configuration will not be applied_
```java
Station station = memphisConnection.createStation(
"",
"",
Retention.MAX_MESSAGE_AGE_SECONDS, // MAX_MESSAGE_AGE_SECONDS/MESSAGES/BYTES. Defaults to MAX_MESSAGE_AGE_SECONDS
604800, // defaults to 604800
Storage.DISK, // Storage.DISK/Storage.MEMORY. Defaults to DISK
1, // defaults to 1
120000, // defaults to 2 minutes
true, // defaults to true
true, // defaults to true
false // defaults to false
).get();
```
### Retention types
Memphis currently supports the following types of retention:
```java
RetentionTypes.MAX_MESSAGE_AGE_SECONDS
```
Means that every message persists for the value set in retention value field (in seconds)
```java
RetentionTypes.MESSAGES
```
Means that after max amount of saved messages (set in retention value), the oldest messages will be deleted
```java
RetentionTypes.BYTES
```
Means that after max amount of saved bytes (set in retention value), the oldest messages will be deleted
### Retention Values
The `retention values` are directly related to the `retention types` mentioned above, where the values vary according to the type of retention chosen.
All retention values are of type `Integer` but with different representations as follows:
`RetentionTypes.MAX_MESSAGE_AGE_SECONDS` is represented **in seconds**, `RetentionTypes.MESSAGES` in a **number of messages** and finally `RetentionTypes.BYTES` in a **number of bytes**.
After these limits are reached oldest messages will be deleted.
### Storage types
Memphis currently supports the following types of messages storage:
```java
StorageTypes.DISK
```
Means that messages persist on disk
```java
StorageTypes.MEMORY
```
Means that messages persist on the main memory
### Destroying a Station
Destroying a station will remove all its resources (producers/consumers)
```java
station.destroy().get()
```
### Attaching a Schema to an Existing Station
```java
memphisConnection.attachSchema("", "").get();
```
### Detaching a Schema from Station
```java
memphisConnection.detachSchema("").get();
```
### Produce and Consume messages
The most common client operations are `produce` to send messages and `consume` to
receive messages.
Messages are published to a station and consumed from it by creating a consumer.
Consumers are pull based and consume all the messages in a station unless you are using a consumers group, in this case messages are spread across all members in this group.
Memphis messages are payload agnostic. Payloads are `byte[]`.
In order to stop getting messages, you have to call `consumer.destroy()`. Destroy will terminate regardless
of whether there are messages in flight for the client.
### Creating a Producer
```java
ProducerOptions pOpts = new ProducerOptions.Builder()
.stationName("")
.producerName("")
.build();
MemphisProducer producer = memphisConnection.createProducer(pOpts);
```
### Producing a message and blocking until acknowledgment
```java
producer.produce(byte[] message);
```
### Producing a message without blocking
This method will add the message to an internal queue and return.
If the queue is full, this method will block until the queue has been
drained some.
```java
producer.produceNonblocking(byte[] message);
```
### Stopping a Producer
```java
producer.stop();
```
### Creating an Asynchronous Consumer
The asynchronous consumer executes a provided function on each batch of messages received in a background thread.
```java
ConsumerOptions opts = new ConsumerOptions.Builder()
.consumerName("test-runner")
.stationName("example-station")
.build();
MemphisAsyncConsumer consumer = memphisConnection.createAsyncConsumer(
opts,
messages -> {
for(MemphisMessage msg : messages) {
System.out.println(new String(msg.getData(), StandardCharsets.UTF_8));
msg.ack();
}
});
);
```
### Processing messages
```java
consumer.start();
```
### Stopping the consumer
```java
consumer.stop();
```
### Creating a Synchronous Consumer
The synchronous consumer checks for messages when its `fetch()` method is called.
The call blocks until messages are available or the wait timeout has been exceeded.
```java
ConsumerOptions opts = new ConsumerOptions.Builder()
.consumerName("test-runner")
.stationName("example-station")
.build();
MemphisSyncConsumer consumer = connection.createSyncConsumer(opts);
```
### Processing messages
```java
var messages = consumer.fetch();
for(var msg : messages) {
msg.ack();
}
```
### Check connection status
```java
memphisConnection.isConnected();
```