Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/univalence/spark-tools

Last synced: about 1 month ago
JSON representation

Host: GitHub
URL: https://github.com/univalence/spark-tools
Owner: univalence
License: apache-2.0
Created: 2019-03-11T23:15:59.000Z (almost 6 years ago)
Default Branch: master
Last Pushed: 2020-04-27T12:23:20.000Z (over 4 years ago)
Last Synced: 2023-07-16T20:49:46.116Z (over 1 year ago)
Language: Scala
Homepage: https://univalence.github.io/spark-tools/
Size: 3.13 MB
Stars: 43
Watchers: 8
Forks: 8
Open Issues: 14
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        ![Spark-tools Logo](./logo.png)

[![CircleCI](https://circleci.com/gh/univalence/spark-tools.svg?style=svg)](https://circleci.com/gh/univalence/spark-tools)

Spark-tools is a set of tools dedicated to Spark and aims to

make the life of data engineers easier.

## The tools

* [Spark-Test](https://github.com/univalence/spark-tools/tree/master/spark-test), testing tools for Dataframe and Dataset from Spark

* [Parka](https://github.com/univalence/spark-tools/tree/master/parka), data quality on Spark

* [Fenek](https://github.com/univalence/spark-tools/tree/master/fenek), a DSL for *semi-typed* transformation in Scala for Spark

* [Plumbus](https://github.com/univalence/spark-tools/tree/master/plumbus), light misc things for Spark

* [Spark-ZIO](https://github.com/univalence/spark-tools/tree/master/spark-zio), Spark in ZIO environment