{"id":17649567,"url":"https://github.com/zmyzheng/tweetmap","last_synced_at":"2025-05-01T01:50:34.715Z","repository":{"id":39004889,"uuid":"107739146","full_name":"zmyzheng/TweetMap","owner":"zmyzheng","description":"A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud","archived":false,"fork":false,"pushed_at":"2023-03-05T21:02:48.000Z","size":42894,"stargazers_count":50,"open_issues_count":16,"forks_count":22,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-30T20:52:08.054Z","etag":null,"topics":["angular","aws","bootstrap","cloud","data-stream-processing","docker","elaticsearch","flink","kafka","kibana","kubernetes","microservice","sentiment-analysis","spring-boot"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zmyzheng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-10-21T00:33:34.000Z","updated_at":"2025-01-03T21:46:34.000Z","dependencies_parsed_at":"2023-02-06T08:01:13.500Z","dependency_job_id":"be04cff3-1438-426d-9faa-06dd469ccf59","html_url":"https://github.com/zmyzheng/TweetMap","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zmyzheng%2FTweetMap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zmyzheng%2FTweetMap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zmyzheng%2FTweetMap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zmyzheng%2FTweetMap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zmyzheng","download_url":"https://codeload.github.com/zmyzheng/TweetMap/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251810897,"owners_count":21647626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angular","aws","bootstrap","cloud","data-stream-processing","docker","elaticsearch","flink","kafka","kibana","kubernetes","microservice","sentiment-analysis","spring-boot"],"created_at":"2024-10-23T11:27:50.175Z","updated_at":"2025-05-01T01:50:34.681Z","avatar_url":"https://github.com/zmyzheng.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tweet Map with Trends\n\n1. Build a `java` Tweet Collector to collect real-time tweets with Twitter Streaming API\n2. Push collected tweets to `Kafka` cluster\n3. Utilize `Apache Flink` Streaming to process (parse, filter and tranform) tweets\n4. Ingest processed tweets to `Elasticsearch` for data persistance and index\n5. Develop `SpringBoot` `RESTful API server` to query tweets from Elasticsearch \n6. Visualize real time tweet trends with Frontend `Angular` web application\n7. Show Geographical Tweet Heat Map with `Kibana`\n8. Create `Docker` image and deploy `microservices` to `Kubernetes` cluster\n\n![](TweetMap-Architecture.png)\n\n![](ScaledCircle.png)\n![](HeatMap.png)\n\n\n## Project structure\n- **tweet-collector**: collect real-time tweets with Twitter Streaming API and push to Kafka cluster. \n  - To support other social media, implement *SocialMediaCollector* interface.\n  - To support other message queues, implement *Sinkable* interface.\n  \n- **rest-api-server**: A RESTful API server querying tweets from Elasticsearch using SpringBoot framework.\n- **flink-processor**: Streaming process tweets and ingest into Elasticsearch.\n- **frontend-website**: Visualize Tweet Map with Angular framework. (developing)\n\n## Build\n\nJDK version:\n1. *master* branch and *dev-jdk11* branch: JDK 11\n2. *dev-jdk8* branch: JDK 8\n\n./gradlew :tweet-collector:clean :tweet-collector:build :tweet-collector:dockerPush\n\njava -jar tweet-collector/build/libs/tweet-collector-1.1-SNAPSHOT.jar\n\n./gradlew :flink-processor:clean :flink-processor:build\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzmyzheng%2Ftweetmap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzmyzheng%2Ftweetmap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzmyzheng%2Ftweetmap/lists"}