https://github.com/vsimko/spark-scala-training

https://github.com/vsimko/spark-scala-training

Last synced: 26 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/vsimko/spark-scala-training
• Owner: vsimko
• Created: 2024-04-30T17:53:09.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-04-30T20:49:42.000Z (6 months ago)
• Last Synced: 2024-05-14T06:29:07.612Z (6 months ago)
• Size: 2.93 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# spark-scala-training

## Scala recap

- we're gonna use Scala 2.13.9 becuase of Spark (latest Scala version is currently 3.4.1 see https://www.scala-lang.org/download/all.html) 

Topics:

  - vals/vars, lazy

  - statement vs expression, if-expr

  - functions and named arguments

  - for-comprehensions vs flatmapping

  - throws vs Try/Either/Option

  - generics, variance, Unit

  - traits, final case classes, companion objects

  - extension methods in scala 2

  - implicits in scala 2

  - package objects (e.g. in testing)

  - multi-word identifiers (e.g. in tests, constants)

  - single-argument functione e.g. Try{...}

  - immutable vs mutable containers

  - update() syntax sugar

  - apply method

  - variable arguments + expanding a collection in place of varargs

  - logging

  - scala-testing

Other topics not covered here:

  - collections, futures, type classes, akka, zio, cats

  - opaque types (scala 3) / value classes (scala 2)

  - tagless final

## Spark : Basic Concepts

Master

Workers

Cluster, Serialization, Plans, Jobs, Tasks, Web Console

Spark SQL, Dataframes vs Datatypes

## Spark : Type-Unsafe API

## Spark : Type-Safe API

## Spark : Special Stuff

- Enumerations

- MdmsDecimal