Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/chapsuk/spark_examples

Last synced: 9 days ago
JSON representation

Host: GitHub
URL: https://github.com/chapsuk/spark_examples
Owner: chapsuk
Created: 2020-10-02T18:53:59.000Z (over 4 years ago)
Default Branch: master
Last Pushed: 2020-10-02T23:51:09.000Z (over 4 years ago)
Last Synced: 2024-11-10T20:36:26.853Z (2 months ago)
Language: Scala
Size: 3.91 KB
Stars: 0
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

        # Spark Examples

## Build

```shell

$ sbt assembly

```

## JsonReader 

Spark on scala example.

Read json file, decode to User case class, print to output

### Dataset

[:bookmark_tabs:](https://storage.googleapis.com/otus_sample_data/winemag-data.json.tgz)

### Run

```shell

$ spark-submit                      \

    --master local[*]               \

    --class com.example.JsonReader  \

                      \

    

```

## CrimeStrict

Aggregate data from boston crimes dataset and write to parquet file

## Dataset

[:bookmark_tabs:](https://www.kaggle.com/AnalyzeBoston/crimes-in-boston)

### Run

```shell

$ spark-submit                      \

    --master local[*]               \

    --class com.example.CrimeStrict \

                      \

                        \ 

                       \

    

```