https://github.com/jstotz/jstreams
A distributed streaming platform for Ruby built on top of Redis Streams
https://github.com/jstotz/jstreams
pubsub redis redis-streams ruby streams
Last synced: 16 days ago
JSON representation
A distributed streaming platform for Ruby built on top of Redis Streams
- Host: GitHub
- URL: https://github.com/jstotz/jstreams
- Owner: jstotz
- License: mit
- Created: 2019-04-01T14:07:55.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-04-03T22:06:41.000Z (about 1 year ago)
- Last Synced: 2025-03-29T06:11:32.342Z (about 1 month ago)
- Topics: pubsub, redis, redis-streams, ruby, streams
- Language: Ruby
- Homepage:
- Size: 85 KB
- Stars: 4
- Watchers: 3
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.txt
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# jstreams
[](https://rubygems.org/gems/jstreams)
[](https://circleci.com/gh/jstotz/jstreams)
[](https://codeclimate.com/github/jstotz/jstreams/test_coverage)
[](https://codeclimate.com/github/jstotz/jstreams/maintainability)
[](https://www.rubydoc.info/github/jstotz/jstreams/master)
A distributed streaming platform for Ruby built on top of Redis Streams.
Provides a multi-threaded publisher/subscriber.
## Project Status
This is alpha software and not suitable for production use.
## Features
- Load balancing among subscribers within a group
- Automatically message reassignment when consumers go away
- Multi-threaded subscribers
- Configurable message serialization
## Roadmap
- Configurable retry logic
- Replay a stream from a given point
- Wildcard subscriptions
## Installation
Add this line to your application's Gemfile:
```ruby
gem 'jstreams'
```
And then execute:
$ bundle
Or install it yourself as:
$ gem install jstreams
## Usage
### Publisher
```ruby
jstreams = Jstreams::Context.new
jstreams.publish(
:users,
event: 'user_created',
user_id: 1,
name: 'King Buzzo',
email: '[email protected]'
)
jstreams.publish(:users, event: 'user_logged_in', user_id: 1)
```
### Subscriber
```ruby
jstreams = Jstreams::Context.new
jstreams.subscribe(
:user_activity_logger,
:users
) do |message, _stream, _subscriber|
case message['event']
when 'user_created'
logger.info "User #{message['name']} created"
when 'user_logged_in'
logger.info "User #{message['id']} logged in"
end
end
jstreams.subscribe(
:send_welcome_email,
:users
) do |message, _stream, _subscriber|
send_user_welcome_email(message['id']) if message['event'] == 'user_created'
end
# Spawns subscriber threads and blocks
jstreams.run
```
### Replay
Starts a temporary copy of the given subscriber until messages have been replayed up to the checkpoint stored at the time replay is called.
```ruby
jstreams.replay(:user_activity_logger, from: message_id)
```
### Retries
By default subscribers will process messages indefinitely until successful.
```ruby
# TODO
```
### Serialization
```ruby
class Serializer
MESSAGE_TYPES = {
user_created: UserCreatedMessage, user_logged_in: UserLoggedInMessage
}
def serialize(type, message)
message_class(type).serialize(message)
end
def deserialize(type, message)
message_class(type).deserialize(message)
end
private
def message_class(type)
MESSAGE_TYPES.fetch(type) { raise "Unknown message type: #{type}" }
end
end
jstreams = Jstreams::Context.new(serializer: Serializer)
```
## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/jstotz/jstreams. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
## License
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
## Code of Conduct
Everyone interacting in the jstreams project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/jstotz/jstreams/blob/master/CODE_OF_CONDUCT.md).