Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mobiletelesystems/spark-dialect-extension

Package extending the default dialect capabilities for Spark.
https://github.com/mobiletelesystems/spark-dialect-extension

etl etl-components plugin-system spark

Last synced: 4 months ago
JSON representation

Package extending the default dialect capabilities for Spark.

Host: GitHub
URL: https://github.com/mobiletelesystems/spark-dialect-extension
Owner: MobileTeleSystems
License: apache-2.0
Created: 2024-06-19T10:59:24.000Z (8 months ago)
Default Branch: develop
Last Pushed: 2024-09-26T11:23:36.000Z (5 months ago)
Last Synced: 2024-09-29T04:42:02.636Z (4 months ago)
Topics: etl, etl-components, plugin-system, spark
Language: Scala
Homepage:
Size: 101 KB
Stars: 3
Watchers: 3
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md

Awesome Lists containing this project

README

        # Spark Dialect Extension Project Documentation

[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)

[![Maven Central Version](https://img.shields.io/maven-central/v/io.github.mtsongithub.doetl/spark-dialect-extension_2.12)](https://central.sonatype.com/artifact/io.github.mtsongithub.doetl/spark-dialect-extension_2.12)

[![Tests](https://github.com/MobileTeleSystems/spark-dialect-extension/actions/workflows/tests_clickhouse.yml/badge.svg)](https://github.com/MobileTeleSystems/spark-dialect-extension/actions)

[![Coverage](https://codecov.io/gh/MobileTeleSystems/spark-dialect-extension/branch/develop/graph/badge.svg?token=99FBR0VZ4F)](https://codecov.io/gh/MobileTeleSystems/spark-dialect-extension)

This repository adds custom Spark JDBC dialects, enhansing or fixing Apache Spark issues with handling some JDBC types.

**Note**: for now only Spark 3.5.x is supported.

## Documentation Index

- [**Using the Dialect**](docs/using_the_dialect.md)

  - How to configure and use the dialect in Spark applications.

- [**Data Type Mappings**](docs/data_type_mappings.md)

  - Detailed mappings between ClickHouse data types and Spark data types.

- [**Contributing to the project**](CONTRIBUTING.md)

  - Detailed instructions on how to build the project.