https://github.com/hden/datastore-offsetbackingstore

Implementation of Kafka Connect OffsetBackingStore that persists data to a GCP Cloud Datastore
https://github.com/hden/datastore-offsetbackingstore

Last synced: 28 days ago
JSON representation

Implementation of Kafka Connect OffsetBackingStore that persists data to a GCP Cloud Datastore

• Host: GitHub
• URL: https://github.com/hden/datastore-offsetbackingstore
• Owner: hden
• License: epl-1.0
• Created: 2018-04-28T00:52:40.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2023-12-15T05:49:41.000Z (about 1 year ago)
• Last Synced: 2024-10-03T12:16:44.173Z (3 months ago)
• Language: Clojure
• Size: 7.81 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# datastore-offsetbackingstore

Using Google Cloud Datastore as a Kafka Connect OffsetBackingStore.

Offset data are flushed to a Cloud Datastore entity and restored upon restart.

## Usage

Add the following values to the worker configuration.

* `offset.storage`: `datastore-offsetbackingstore.core.DatastoreOffsetBackingStore`

* `datastore.kind`: String. The kind of the Cloud Datastore entity.

* `datastore.id`: Integer or String. The id or key for the worker.

Also refer to the following documentations:

* [Entities, Properties, and Keys](https://cloud.google.com/datastore/docs/concepts/entities) - Cloud Datastore Documentation.

* [Kafka Connect User Guide](https://docs.confluent.io/2.0.0/connect/userguide.html#configuring-workers) for general concepts of Kafka Connect.

* [Debezium-embedded](https://github.com/debezium/debezium/tree/master/debezium-embedded) for Running Debezium without a Kafka cluster.

## License

Copyright © 2018 Haokang Den

Distributed under the Eclipse Public License, the same as Clojure.