https://github.com/maziyarpanahi/spark2-template
Intellij template to develop Apache Spark 2.x applications
https://github.com/maziyarpanahi/spark2-template
spark-ml spark-sql spark-streaming spark2
Last synced: 9 months ago
JSON representation
Intellij template to develop Apache Spark 2.x applications
- Host: GitHub
- URL: https://github.com/maziyarpanahi/spark2-template
- Owner: maziyarpanahi
- License: mit
- Created: 2018-02-10T21:27:19.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2022-01-18T14:40:06.000Z (over 4 years ago)
- Last Synced: 2025-04-09T08:04:04.486Z (about 1 year ago)
- Topics: spark-ml, spark-sql, spark-streaming, spark2
- Language: Scala
- Homepage:
- Size: 20.5 KB
- Stars: 7
- Watchers: 1
- Forks: 41
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# spark2-template
This is a Intellij template for developing Apache Spark 2.4.x applications.
**NOTE:** This repo is for teaching :)
## Requirements
Install `scala` and `sbt` (or maybe Java as well)
Ubuntu:
```bash
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
sudo apt-get update
sudo apt-get install sbt
sudo apt-get install scala
```
macOS:
```bash
brew update
brew install scala
brew install sbt
```
In any case, please make sure you have JAVA and it is up to date:
```bash
java --version
```
## Evaluation
You should fork this repository, commit your changes and push them into your own GitHub account.
## IntelliJ IDEA
Please download the community version of IntelliJ IDEA from here:
https://www.jetbrains.com/idea/download/