https://github.com/zmyzheng/tweetmap

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud
https://github.com/zmyzheng/tweetmap

angular aws bootstrap cloud data-stream-processing docker elaticsearch flink kafka kibana kubernetes microservice sentiment-analysis spring-boot

Last synced: about 2 months ago
JSON representation

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud

• Host: GitHub
• URL: https://github.com/zmyzheng/tweetmap
• Owner: zmyzheng
• License: apache-2.0
• Created: 2017-10-21T00:33:34.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2023-03-05T21:02:48.000Z (almost 2 years ago)
• Last Synced: 2023-10-21T00:04:30.929Z (about 1 year ago)
• Topics: angular, aws, bootstrap, cloud, data-stream-processing, docker, elaticsearch, flink, kafka, kibana, kubernetes, microservice, sentiment-analysis, spring-boot
• Language: Java
• Homepage:
• Size: 40.9 MB
• Stars: 48
• Watchers: 3
• Forks: 21
• Open Issues: 16
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Tweet Map with Trends

1. Build a `java` Tweet Collector to collect real-time tweets with Twitter Streaming API

2. Push collected tweets to `Kafka` cluster

3. Utilize `Apache Flink` Streaming to process (parse, filter and tranform) tweets

4. Ingest processed tweets to `Elasticsearch` for data persistance and index

5. Develop `SpringBoot` `RESTful API server` to query tweets from Elasticsearch 

6. Visualize real time tweet trends with Frontend `Angular` web application

7. Show Geographical Tweet Heat Map with `Kibana`

8. Create `Docker` image and deploy `microservices` to `Kubernetes` cluster

![](TweetMap-Architecture.png)

![](ScaledCircle.png)

![](HeatMap.png)

## Project structure

- **tweet-collector**: collect real-time tweets with Twitter Streaming API and push to Kafka cluster. 

  - To support other social media, implement *SocialMediaCollector* interface.

  - To support other message queues, implement *Sinkable* interface.

  

- **rest-api-server**: A RESTful API server querying tweets from Elasticsearch using SpringBoot framework.

- **flink-processor**: Streaming process tweets and ingest into Elasticsearch.

- **frontend-website**: Visualize Tweet Map with Angular framework. (developing)

## Build

JDK version:

1. *master* branch and *dev-jdk11* branch: JDK 11

2. *dev-jdk8* branch: JDK 8

./gradlew :tweet-collector:clean :tweet-collector:build :tweet-collector:dockerPush

java -jar tweet-collector/build/libs/tweet-collector-1.1-SNAPSHOT.jar

./gradlew :flink-processor:clean :flink-processor:build