https://github.com/mobiletelesystems/spark-dialect-extension

Extend JDBC types support for Apache Spark.
https://github.com/mobiletelesystems/spark-dialect-extension

dialects etl jdbc spark

Last synced: 3 months ago
JSON representation

Extend JDBC types support for Apache Spark.

• Host: GitHub
• URL: https://github.com/mobiletelesystems/spark-dialect-extension
• Owner: MobileTeleSystems
• License: apache-2.0
• Created: 2024-06-19T10:59:24.000Z (11 months ago)
• Default Branch: develop
• Last Pushed: 2025-01-09T09:03:11.000Z (5 months ago)
• Last Synced: 2025-02-01T21:12:07.044Z (4 months ago)
• Topics: dialects, etl, jdbc, spark
• Language: Scala
• Homepage:
• Size: 122 KB
• Stars: 4
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# Spark Dialect Extension Project Documentation

[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)

[![Maven Central Version](https://img.shields.io/maven-central/v/io.github.mtsongithub.doetl/spark-dialect-extension_2.12)](https://central.sonatype.com/artifact/io.github.mtsongithub.doetl/spark-dialect-extension_2.12)

[![Tests](https://github.com/MobileTeleSystems/spark-dialect-extension/actions/workflows/tests_clickhouse.yml/badge.svg)](https://github.com/MobileTeleSystems/spark-dialect-extension/actions)

[![Coverage](https://codecov.io/gh/MobileTeleSystems/spark-dialect-extension/branch/develop/graph/badge.svg?token=99FBR0VZ4F)](https://codecov.io/gh/MobileTeleSystems/spark-dialect-extension)

This repository adds custom Spark JDBC dialects, enhansing or fixing Apache Spark issues with handling some JDBC types.

**Note**: for now only Spark 3.5.x is supported.

## Documentation Index

- [**Using the Dialect**](docs/using_the_dialect.md)

  - How to configure and use the dialect in Spark applications.

- [**Data Type Mappings**](docs/data_type_mappings.md)

  - Detailed mappings between ClickHouse data types and Spark data types.

- [**Contributing to the project**](CONTRIBUTING.md)

  - Detailed instructions on how to build the project.