https://github.com/confluentinc/kafka-connect-elasticsearch

Kafka Connect Elasticsearch connector
https://github.com/confluentinc/kafka-connect-elasticsearch

confluent elasticsearch kafka kafka-connect kafka-connector

Last synced: 4 months ago
JSON representation

Kafka Connect Elasticsearch connector

• Host: GitHub
• URL: https://github.com/confluentinc/kafka-connect-elasticsearch
• Owner: confluentinc
• License: other
• Created: 2016-02-10T09:44:07.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2024-04-13T02:25:54.000Z (7 months ago)
• Last Synced: 2024-04-14T01:06:38.036Z (7 months ago)
• Topics: confluent, elasticsearch, kafka, kafka-connect, kafka-connector
• Language: Java
• Size: 2.46 MB
• Stars: 744
• Watchers: 307
• Forks: 429
• Open Issues: 140
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

• awesome-kafka - Elasticsearch

README

        
# Kafka Connect Elasticsearch Connector

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch.svg?type=shield)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch?ref=badge_shield)

Changelog for this connector can be found [here](https://docs.confluent.io/kafka-connect-elasticsearch/current/changelog.html).

kafka-connect-elasticsearch is a [Kafka Connector](http://kafka.apache.org/documentation.html#connect)

for copying data between Kafka and Elasticsearch.

# Development

To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the [FAQ](https://github.com/confluentinc/kafka-connect-elasticsearch/wiki/FAQ) for guidance on this process.

You can build kafka-connect-elasticsearch with Maven using the standard lifecycle phases.

# Configuring

## Creating an Elasticsearch user and assigning required privileges

### Create an Elasticsearch role

```

curl -u elastic:elastic -X POST "localhost:9200/_security/role/es_sink_connector_role?pretty" -H 'Content-Type: application/json' -d'

{

  "indices": [

    {

      "names": [ "*" ],

      "privileges": ["create_index", "read", "write", "view_index_metadata"]

    }

  ]

}'

```

### Create an Elasticsearch user

```

curl -u elastic:elastic -X POST "localhost:9200/_security/user/es_sink_connector_user?pretty" -H 'Content-Type: application/json' -d'

{

  "password" : "seCret-secUre-PaSsW0rD",

  "roles" : [ "es_sink_connector_role" ]

}'

```

# Contribute

- Source Code: https://github.com/confluentinc/kafka-connect-elasticsearch

- Issue Tracker: https://github.com/confluentinc/kafka-connect-elasticsearch/issues

- Learn how to work with the connector's source code by reading our [Development and Contribution guidelines](CONTRIBUTING.md).

# License

This project is licensed under the [Confluent Community License](LICENSE).

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch.svg?type=large)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch?ref=badge_large)