https://github.com/1ambda/docker-kafka-connect
Docker Image for kafka-connect
https://github.com/1ambda/docker-kafka-connect
docker kafka-connect
Last synced: about 1 year ago
JSON representation
Docker Image for kafka-connect
- Host: GitHub
- URL: https://github.com/1ambda/docker-kafka-connect
- Owner: 1ambda
- License: apache-2.0
- Created: 2016-07-07T15:15:22.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2018-06-30T10:34:54.000Z (about 8 years ago)
- Last Synced: 2025-04-01T09:04:24.194Z (over 1 year ago)
- Topics: docker, kafka-connect
- Language: Shell
- Homepage: https://hub.docker.com/r/1ambda/kafka-connect/
- Size: 12.7 KB
- Stars: 19
- Watchers: 3
- Forks: 18
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://hub.docker.com/r/1ambda/kafka-connect/)
[](https://hub.docker.com/r/1ambda/kafka-connect/)
docker-kafka-connect
============
Dockerized [Apache Kafka Connect](http://kafka.apache.org/documentation.html#connect) (distributed mode)
## Supported Tags
- `0.10.0.0` (2.11) [(0.10.0.0/Dockerfile)](https://github.com/1ambda/docker-kafka-connect/blob/master/0.10.0.0/Dockerfile)
- `0.10.1.1` (2.11) [(0.10.1.1/Dockerfile)](https://github.com/1ambda/docker-kafka-connect/blob/master/0.10.1.1/Dockerfile)
- **latest** `0.10.2.0` (2.12) [(0.10.2.0/Dockerfile)](https://github.com/1ambda/docker-kafka-connect/blob/master/0.10.2.0/Dockerfile)
## Quick Start
### with Docker Compose
Write `docker-compose.yml` like and then execute `docker-compose up`
```yaml
version: '2'
services:
zk:
image: 31z4/zookeeper:3.4.8
kafka:
image: ches/kafka
links:
- zk
environment:
KAFKA_BROKER_ID: 0
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ADVERTISED_PORT: 9092
ZOOKEEPER_CONNECTION_STRING: zk:2181
ZOOKEEPER_CHROOT: /broker-0
connect:
image: 1ambda/kafka-connect
links:
- kafka
ports:
- "8083:8083"
environment:
CONNECT_BOOTSTRAP_SERVERS: kafka:9092
CONNECT_GROUP_ID: connect-cluster-A
```
### with Docker CLI
```shell
$ docker run -d --name zk 31z4/zookeeper:3.4.8
$ docker run -d --name kafka \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e ZOOKEEPER_CONNECTION_STRING=zk:2181 \
--link zk:zk ches/kafka
$ docker run --rm --name connect \
-p 8083:8083 \
-e CONNECT_BOOTSTRAP_SERVERS=kafka:9092 \
-e CONNECT_GROUP_ID=connect-cluster-A \
--link kafka:kafka 1ambda/kafka-connect
```
## Environment Variables
Pass env variables starting with `CONNECT_` to configure `connect-distributed.properties`.
For example, If you want to set `offset.flush.interval.ms=15000`, use `CONNECT_OFFSET_FLUSH_INTERVAL_MS=15000`
- (**required**) `CONNECT_BOOTSTRAP_SERVERS`
- (*recommended*): `CONNECT_GROUP_ID` (default value: `connect-cluster`)
- (*recommended*) `CONNECT_REST_ADVERTISED_HOST_NAME`
- (*recommended*) `CONNECT_REST_ADVERTISED_PORT`
Other connect configuration fields are optional. (see also [Kafka Connect Configs](http://kafka.apache.org/documentation.html#connectconfigs))
## How To Extend This Image
If you want to run additional connectors, add connector JARs to `${KAFKA_HOME}/connectors` in container.
```
FROM 1ambda/kafka-connect:latest
# same as `cp -R connectors/ $KAFKA_HOME/`
# the entrypoint will extends `$CLASSPATH`
# like `export CLASSPATH=${CLASSPATH}:${KAFKA_HOME}/connectors/*`
COPY connectors $KAFKA_HOME/connectors
```
## Development
- **SCALA_VERSION**: `2.11`
- **KAFKA_VERSION**: `0.10.0.0`
- **KAFKA_HOME**: `/opt/kafka_${SCALA_VERSION}-${KAFKA_VERSION}`
- **CONNECT_CFG**: `${KAFKA_HOME}/config/connect-distributed.properties`
- **CONNECT_BIN**: `${KAFKA_HOME}/bin/connect-distributed.sh`
- **CONNECT_PORT**: `8083` (exposed)
- **JMX_PORT**: `9999` (exposed)
# License
Apache 2.0