https://github.com/exasol/spark-connector

A connector for Apache Spark to access Exasol
https://github.com/exasol/spark-connector

apache-spark connector exasol exasol-integration spark streaming

Last synced: 3 months ago
JSON representation

A connector for Apache Spark to access Exasol

• Host: GitHub
• URL: https://github.com/exasol/spark-connector
• Owner: exasol
• License: apache-2.0
• Created: 2018-09-10T09:09:11.000Z (over 6 years ago)
• Default Branch: main
• Last Pushed: 2024-08-28T13:39:43.000Z (5 months ago)
• Last Synced: 2024-10-15T18:41:05.170Z (3 months ago)
• Topics: apache-spark, connector, exasol, exasol-integration, spark, streaming
• Language: Java
• Homepage:
• Size: 584 KB
• Stars: 12
• Watchers: 17
• Forks: 7
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Spark Exasol Connector

[![Build Status](https://github.com/exasol/spark-connector/actions/workflows/ci-build.yml/badge.svg)](https://github.com/exasol/spark-connector/actions/workflows/ci-build.yml)

[![Maven Central – The Spark Exasol Connector](https://img.shields.io/maven-central/v/com.exasol/spark-connector_2.13)](https://mvnrepository.com/artifact/com.exasol/spark-connector_2.13)

[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=alert_status)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=security_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=sqale_index)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=code_smells)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=coverage)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)

[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=com.exasol%3Aspark-connector&metric=ncloc)](https://sonarcloud.io/dashboard?id=com.exasol%3Aspark-connector)



## Overview

Spark Exasol Connector (SEC) supports an integration between [Exasol][exasol]

and [Apache Spark][spark]. It allows creating Spark DataFrame from Exasol

queries and saving Spark DataFrame as an Exasol table.

## Features

* Creates Spark DataFrame from Exasol query results

* Saves Spark DataFrame to an Exasol table

* Allows configuring the Spark tasks for parallel connection

* Supports Spark DataFrame column pruning and filter push down

## Spark Streaming

At the moment, Spark Exasol Connector does not support integration with Spark

Streaming. Please check out the [Spark regular JDBC

integration](https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html).

## Information for Users

* [User Guide](doc/user_guide/user_guide.md)

* [Changelog](doc/changes/changelog.md)

## Information for Contributors

* [Developer Guide](doc/development/developer_guide.md)

* [Dependencies](dependencies.md)

[exasol]: https://www.exasol.com/en/

[spark]: https://spark.apache.org/