https://github.com/meilisearch/MeiliES
A Rust based event store using the Redis protocol
https://github.com/meilisearch/MeiliES
event-store redis-protocol
Last synced: about 1 year ago
JSON representation
A Rust based event store using the Redis protocol
- Host: GitHub
- URL: https://github.com/meilisearch/MeiliES
- Owner: meilisearch
- License: mit
- Archived: true
- Created: 2019-03-02T16:59:43.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2021-12-06T15:54:53.000Z (over 4 years ago)
- Last Synced: 2025-04-05T14:56:03.779Z (about 1 year ago)
- Topics: event-store, redis-protocol
- Language: Rust
- Homepage:
- Size: 303 KB
- Stars: 322
- Watchers: 14
- Forks: 19
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
MeiliES
A Rust based event store using the Redis protocol
## Introduction
_This project is no more maintained, please consider finding another tool instead._
MeiliES is an _Event Sourcing database_ that uses the _RESP_ (REdis Serialization Protocol) to communicate.
This way it is possible to create clients by reusing the already available protocol. For example, it is possible to use [the official Redis command line interface](https://redis.io/topics/rediscli) program to communicate with MeiliES. There is [a release blog post](https://blog.meilisearch.com/meilies-release/) if you want to know more.
An event store is like a Kafka or a Rabbit MQ but it stores events on disk indefinitely. The first purpose of the server is to publish events of a stream to all subscribed clients, note that events are saved in reception order. A client can also specify from which event number (incrementing) it wants to read, therefore it is possible to recover from crashing by reading and reconstructing a state with only new events. Keep in mind that [a message queue is not made for event-sourcing](https://medium.com/serialized-io/apache-kafka-is-not-for-event-sourcing-81735c3cf5c).
[](https://asciinema.org/a/KX6qpkfDd1RWxS8NDQQYoo26r?speed=2&autoplay=1)
## Features
- Event publication
- TCP stream subscriptions from an optional event number
- Resilient connections (reconnecting when closed)
- Redis based protocol
- Full Rust, using [sled as the internal storage](http://sled.rs)
- Takes near 2min to compile
## Building MeiliES
To run MeiliES you will need Rust, you can install it by following the steps on https://rustup.rs.
Once you have Rust in your `PATH` you can clone and build the MeiliES binaries.
```bash
git clone https://github.com/meilisearch/MeiliES.git
cd MeiliES
cargo install --path meilies-server
cargo install --path meilies-cli
```
## Basic Event Store Usage
Once MeiliES is installed and available in your `PATH`, you can run it by executing the following command.
```bash
meilies-server --db-path my-little-db.edb
```
There is now a MeiliES server running on your machine and listening on `127.0.0.1:6480`.
In another terminal window, you can specify to a client to listen to only new events.
```bash
meilies-cli subscribe 'my-little-stream'
```
And in another one again you can send new events.
All clients which are subscribed to that same stream will see the events.
```bash
meilies-cli publish 'my-little-stream' 'my-event-name' 'Hello World!'
meilies-cli publish 'my-little-stream' 'my-event-name' 'Hello Cathy!'
meilies-cli publish 'my-little-stream' 'my-event-name' 'Hello Kevin!'
meilies-cli publish 'my-little-stream' 'my-event-name' 'Hello Donut!'
```
But that is not a really interesting usage of Event Sourcing, right?!
Let's do a more interesting usage of it.
## Real Event Store Usage
MeiliES stores all the events of all the streams that were sent by all the clients in the order they were received.
So let's check that and specify to one client the point in time where we want to start reading events.
Once there are no more events in the stream, the server starts sending events at the moment it receives them.
### Stream name specification
A stream name is composed as follow.
`{name}{:from}{:to}`
- name: the name of the stream, case sensitive, must not contain space (prefer dash-separated words).
- from: Specifies the first event number to start reading from. Optional, if it's not set MeiliES, will start from the end.
- to: Specifies the last event number to send (exclusive range). Optional value, will never stop if it's not given.
### Examples
We can do that by prepending the start event number separated by a colon.
```bash
meilies-cli subscribe 'my-little-stream:0'
```
In this example, we will start reading from the first event of the stream.
But we can also specify to start reading from the third event too.
```bash
meilies-cli subscribe 'my-little-stream:2'
```
Or from events which do not exists yet!
Try sending events to this stream and you will see: only events from the fifth one will appear.
```bash
meilies-cli subscribe 'my-little-stream:5'
```
It's also possible to read a stream until an event number.
```bash
meilies-cli subscribe 'my-little-stream:3:5'
```
## Current Limitations
The current implementation has some limitations related to the whole number of streams subscribed. The problem is that one thread is spawned for each stream and for each client. For example, if two clients subscribe to the same stream, the server will spawn two threads, one for each client instead of spawning only one thread and sending new events to a clients pool.
Even uglier, if a client is closing the connection, the spawned threads will not stop immediately but after some stream activity.
## Support
For commercial support, drop us an email at bonjour@meilisearch.com.