https://github.com/fpopic/gg-interview-challenge
(Interview) GG Interview Challenge in Scala/Spark
https://github.com/fpopic/gg-interview-challenge
apache-spark json logstash parsing regex scala spark sparksql
Last synced: 7 months ago
JSON representation
(Interview) GG Interview Challenge in Scala/Spark
- Host: GitHub
- URL: https://github.com/fpopic/gg-interview-challenge
- Owner: fpopic
- Created: 2017-06-03T12:53:51.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-06-03T13:06:41.000Z (over 8 years ago)
- Last Synced: 2025-01-10T19:43:01.386Z (9 months ago)
- Topics: apache-spark, json, logstash, parsing, regex, scala, spark, sparksql
- Language: Scala
- Homepage:
- Size: 45.8 MB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
**This application uses Spark 2.1.0 version which is marked as 'provided' inside ```build.sbt``` file.**
### Run command usage
```
spark-submit --class com.fp.GumGumJoiner --master local[*]
```
### Run command with prepared file paths
```
spark-submit
--class com.fp.GumGumJoiner \
--master local[*] \
target/scala-2.11/gumgum-challenge-assembly-1.0.jar \
src/main/resources/adevents \
src/main/resources/assets \
src/main/resources/output
```
### Artifacts
1. jar with classes => ```target/scala-2.11/*```
2. source code => ```src/main/scala/*```
3. instructions => ```README.md```
4. output file => ```src/main/resources/myoutput/part-00000-e2377d2b-fff6-4d6b-a721-72355bb4562c.txt```