https://github.com/iskyzh/hadoop-spark-job

Course project for CS236
https://github.com/iskyzh/hadoop-spark-job

Last synced: about 1 month ago
JSON representation

Course project for CS236

• Host: GitHub
• URL: https://github.com/iskyzh/hadoop-spark-job
• Owner: iskyzh
• License: apache-2.0
• Archived: true
• Created: 2020-12-17T15:04:13.000Z (almost 4 years ago)
• Default Branch: master
• Last Pushed: 2021-04-06T00:19:25.000Z (over 3 years ago)
• Last Synced: 2024-06-11T07:34:44.735Z (6 months ago)
• Language: Java
• Homepage:
• Size: 12.7 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-cs - @skyzh, 2020 Fall

README

        
# hadoop-spark-job

Inside this repo, we:

* one-click setup a hadoop / spark cluster `hadoop-docker`

    * with the help of https://github.com/big-data-europe/docker-hadoop

    * and https://github.com/big-data-europe/docker-hadoop

* use map-reduce to sort and group temperature data `map-reduce-temperature`

* use spark to to sort and group temperature data `spark-temperature`