https://github.com/sbcd90/kafka-connect-storage-cloud-formats

This project provides an Apache ORC writer to Amazon S3 for kafka-connect-s3 connector.
https://github.com/sbcd90/kafka-connect-storage-cloud-formats

Last synced: 9 days ago
JSON representation

This project provides an Apache ORC writer to Amazon S3 for kafka-connect-s3 connector.

• Host: GitHub
• URL: https://github.com/sbcd90/kafka-connect-storage-cloud-formats
• Owner: sbcd90
• Created: 2019-07-23T20:44:30.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2022-08-11T21:16:15.000Z (over 2 years ago)
• Last Synced: 2024-04-14T18:21:10.511Z (7 months ago)
• Language: Java
• Size: 29.3 KB
• Stars: 4
• Watchers: 2
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
kafka-connect-storage-cloud-formats

===================================

- This project provides an [Apache ORC](https://orc.apache.org/) writer to S3

for [kafka-connect-s3](https://docs.confluent.io/current/connect/kafka-connect-s3/index.html) connector.

## Installation

- Generate jar using maven

```

mvn clean install

```

- Download [confluent cloud](https://www.confluent.io/download/).

- Put the jar in plugins directory.

- Start Zookeeper, Kafka, schema-registry.

## Configuration

- In `quickstart-s3.properties` change,

```

format.class=io.confluent.connect.s3.format.orc.OrcFormat

```

## Test cases

- Full test cases are available [here](src/test/java/io/confluent/connect/s3/DataWriterOrcTest.java)