https://github.com/abhirockzz/redis-streams-in-action

Go, Rust, Java services to showcase Redis Streams and RediSearch in action
https://github.com/abhirockzz/redis-streams-in-action

azure docker go java redis redis-streams rust serverless

Last synced: over 1 year ago
JSON representation

Go, Rust, Java services to showcase Redis Streams and RediSearch in action

• Host: GitHub
• URL: https://github.com/abhirockzz/redis-streams-in-action
• Owner: abhirockzz
• Created: 2021-05-13T14:58:05.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-05-18T16:25:07.000Z (over 5 years ago)
• Last Synced: 2025-03-24T00:24:41.993Z (over 1 year ago)
• Topics: azure, docker, go, java, redis, redis-streams, rust, serverless
• Language: Java
• Homepage: https://itnext.io/redis-streams-in-action-part-1-intro-and-overview-135f66d3ab58
• Size: 46.9 KB
• Stars: 39
• Watchers: 4
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Redis Streams in Action

A practical example to showcase [Redis Streams](https://redis.io/topics/streams-intro) in action. The goal is to process Twitter data in real-time for search and query tweets based on some criteria such as hashtags, user, location etc. [RediSearch](https://redisearch.io/) and [Redis Streams](https://redis.io/topics/streams-intro) serve as the backbone of this solution. 

![](arch.jpg)

It consists of several co-operating components:

1. Twitter Stream Consumer: A [Rust](https://rustlang.org/) application to consume streaming Twitter data and pass them on to Redis Streams. I will demonstrate how to run this as a Docker container in [Azure Container Instances](https://docs.microsoft.com/azure/container-instances/container-instances-overview?WT.mc_id=data-17927-abhishgu)

2. Tweets Processor: The tweets from Redis Streams are processed by a Java application - this too will be deployed (and scaled) using Azure Container Instances.

3. Monitoring service: The last part is a [Go](https://golang.org/) application to monitor the progress of the tweets processor service  and ensure that any failed records are re-processed. This is a Serverless component which will be deployed to [Azure Functions](https://docs.microsoft.com/azure/azure-functions/functions-overview?WT.mc_id=data-17927-abhishgu) where you can run it based on a [Timer trigger](https://docs.microsoft.com/azure/azure-functions/functions-bindings-timer?tabs=csharp&WT.mc_id=data-17927-abhishgu) and only pay for the duration it runs for.