https://github.com/dfdx/spark.jl

Julia binding for Apache Spark
https://github.com/dfdx/spark.jl

big-data julia spark

Last synced: about 12 hours ago
JSON representation

Julia binding for Apache Spark

• Host: GitHub
• URL: https://github.com/dfdx/spark.jl
• Owner: dfdx
• License: other
• Created: 2015-05-25T22:33:25.000Z (over 9 years ago)
• Default Branch: main
• Last Pushed: 2024-10-02T22:23:33.000Z (4 months ago)
• Last Synced: 2025-01-15T00:17:12.676Z (8 days ago)
• Topics: big-data, julia, spark
• Language: Julia
• Homepage:
• Size: 480 KB
• Stars: 208
• Watchers: 13
• Forks: 39
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# Spark.jl

A Julia interface to Apache Spark™

| **Latest Version** | **Documentation** | **PackageEvaluator** | **Build Status** |

|:------------------:|:-----------------:|:--------------------:|:----------------:|

| [![][version-img]][version-url] | [![][docs-latest-img]][docs-latest-url] | [![PkgEval][pkgeval-img]][pkgeval-url]  | [![][gh-test-img]][gh-test-url]  |

Spark.jl provides an interface to Apache Spark™ platform, including SQL / DataFrame and Structured Streaming. It closely follows the PySpark API, making it easy to translate existing Python code to Julia.

Spark.jl supports multiple cluster types (in client mode), and can be considered as an analogue to PySpark or RSpark within the Julia ecosystem. It supports running within on-premise installations, as well as hosted instance such as Amazon EMR and Azure HDInsight.

**[Documentation][docs-latest-url]**

## Trademarks

Apache®, [Apache Spark and Spark](http://spark.apache.org) are registered trademarks, or trademarks of the [Apache Software Foundation](http://www.apache.org/) in the United States and/or other countries.

[docs-latest-img]: https://img.shields.io/badge/docs-latest-blue.svg

[docs-latest-url]: http://dfdx.github.io/Spark.jl/dev

[gh-test-img]: https://github.com/dfdx/Spark.jl/actions/workflows/test.yml/badge.svg

[gh-test-url]: https://github.com/dfdx/Spark.jl/actions/workflows/test.yml

[codecov-img]: https://codecov.io/gh/dfdx/Spark.jl/branch/master/graph/badge.svg

[codecov-url]: https://codecov.io/gh/dfdx/Spark.jl

[issues-url]: https://github.com/dfdx/Spark.jl/issues

[pkgeval-img]: https://juliahub.com/docs/Spark/pkgeval.svg

[pkgeval-url]: https://juliahub.com/ui/Packages/Spark/zpJEw

[version-img]: https://juliahub.com/docs/Spark/version.svg

[version-url]: https://juliahub.com/ui/Packages/Spark/zpJEw