Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/zmyzheng/tweetmap

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud
https://github.com/zmyzheng/tweetmap

angular aws bootstrap cloud data-stream-processing docker elaticsearch flink kafka kibana kubernetes microservice sentiment-analysis spring-boot

Last synced: 12 days ago
JSON representation

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud

Awesome Lists containing this project

README

        

# Tweet Map with Trends

1. Build a `java` Tweet Collector to collect real-time tweets with Twitter Streaming API
2. Push collected tweets to `Kafka` cluster
3. Utilize `Apache Flink` Streaming to process (parse, filter and tranform) tweets
4. Ingest processed tweets to `Elasticsearch` for data persistance and index
5. Develop `SpringBoot` `RESTful API server` to query tweets from Elasticsearch
6. Visualize real time tweet trends with Frontend `Angular` web application
7. Show Geographical Tweet Heat Map with `Kibana`
8. Create `Docker` image and deploy `microservices` to `Kubernetes` cluster

![](TweetMap-Architecture.png)

![](ScaledCircle.png)
![](HeatMap.png)

## Project structure
- **tweet-collector**: collect real-time tweets with Twitter Streaming API and push to Kafka cluster.
- To support other social media, implement *SocialMediaCollector* interface.
- To support other message queues, implement *Sinkable* interface.

- **rest-api-server**: A RESTful API server querying tweets from Elasticsearch using SpringBoot framework.
- **flink-processor**: Streaming process tweets and ingest into Elasticsearch.
- **frontend-website**: Visualize Tweet Map with Angular framework. (developing)

## Build

JDK version:
1. *master* branch and *dev-jdk11* branch: JDK 11
2. *dev-jdk8* branch: JDK 8

./gradlew :tweet-collector:clean :tweet-collector:build :tweet-collector:dockerPush

java -jar tweet-collector/build/libs/tweet-collector-1.1-SNAPSHOT.jar

./gradlew :flink-processor:clean :flink-processor:build