Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/zmyzheng/tweetmap
A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud
https://github.com/zmyzheng/tweetmap
angular aws bootstrap cloud data-stream-processing docker elaticsearch flink kafka kibana kubernetes microservice sentiment-analysis spring-boot
Last synced: 12 days ago
JSON representation
A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud
- Host: GitHub
- URL: https://github.com/zmyzheng/tweetmap
- Owner: zmyzheng
- License: apache-2.0
- Created: 2017-10-21T00:33:34.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2023-03-05T21:02:48.000Z (over 1 year ago)
- Last Synced: 2023-10-21T00:04:30.929Z (about 1 year ago)
- Topics: angular, aws, bootstrap, cloud, data-stream-processing, docker, elaticsearch, flink, kafka, kibana, kubernetes, microservice, sentiment-analysis, spring-boot
- Language: Java
- Homepage:
- Size: 40.9 MB
- Stars: 48
- Watchers: 3
- Forks: 21
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Tweet Map with Trends
1. Build a `java` Tweet Collector to collect real-time tweets with Twitter Streaming API
2. Push collected tweets to `Kafka` cluster
3. Utilize `Apache Flink` Streaming to process (parse, filter and tranform) tweets
4. Ingest processed tweets to `Elasticsearch` for data persistance and index
5. Develop `SpringBoot` `RESTful API server` to query tweets from Elasticsearch
6. Visualize real time tweet trends with Frontend `Angular` web application
7. Show Geographical Tweet Heat Map with `Kibana`
8. Create `Docker` image and deploy `microservices` to `Kubernetes` cluster![](TweetMap-Architecture.png)
![](ScaledCircle.png)
![](HeatMap.png)## Project structure
- **tweet-collector**: collect real-time tweets with Twitter Streaming API and push to Kafka cluster.
- To support other social media, implement *SocialMediaCollector* interface.
- To support other message queues, implement *Sinkable* interface.
- **rest-api-server**: A RESTful API server querying tweets from Elasticsearch using SpringBoot framework.
- **flink-processor**: Streaming process tweets and ingest into Elasticsearch.
- **frontend-website**: Visualize Tweet Map with Angular framework. (developing)## Build
JDK version:
1. *master* branch and *dev-jdk11* branch: JDK 11
2. *dev-jdk8* branch: JDK 8./gradlew :tweet-collector:clean :tweet-collector:build :tweet-collector:dockerPush
java -jar tweet-collector/build/libs/tweet-collector-1.1-SNAPSHOT.jar
./gradlew :flink-processor:clean :flink-processor:build