https://github.com/anoskov/kafka-consumer

Consumer for Kafka using brod and elixir (production ready)
https://github.com/anoskov/kafka-consumer

Last synced: 8 days ago
JSON representation

Consumer for Kafka using brod and elixir (production ready)

• Host: GitHub
• URL: https://github.com/anoskov/kafka-consumer
• Owner: anoskov
• License: mit
• Created: 2016-08-11T07:37:19.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2020-03-03T21:24:46.000Z (over 4 years ago)
• Last Synced: 2024-10-20T14:04:33.177Z (19 days ago)
• Language: Elixir
• Homepage:
• Size: 39.1 KB
• Stars: 26
• Watchers: 4
• Forks: 5
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• freaking_awesome_elixir - Elixir - Consumer for Kafka using kafka_ex. (Queue)

README

        
# Kafka consumer

Scalable consumer for Kafka using brod.

## Installation

If [available in Hex](https://hex.pm/docs/publish), the package can be installed as:

* Add `kafka_consumer` to your list of dependencies in `mix.exs`:

```elixir

def deps do

  [{:kafka_consumer, "~> 2.0"}]

end

```

* Ensure `kafka_consumer` is started before your application:

```elixir

def application do

  [applications: [:kafka_consumer]]

end

```

## Usage

* Set config or pass default attributes

```elixir

config :brod,

  clients: [

    kafka_client: [

      endpoints: [{'localhost', 9092}],

      auto_start_producers: true

    ]

  ]

```

* Write your own Consumer. Functions handle_async/1 and handle_sync/2 is overridable

```elixir

defmodule KafkaConsumer.ExampleConsumer do

  use KafkaConsumer.Consumer

  def handle_async(_message) do

    :ok

  end

end

```

* Set event handlers in config.

```elixir

config :kafka_consumer,

  consumers: [

    [client: :kafka_client,

     group_id: "messaging",

     topics: ["message-events", "system-events"],

     callback: KafkaConsumer.ExampleConsumer,

     callback_args: []]

  ]

```

* Define KafkaConsumer.Supervisor as child supervisor of your app.

```elixir

children = [

  supervisor(KafkaConsumer.Supervisor, []),

]

opts = [strategy: :one_for_one, name: YourApplication.Supervisor]

Supervisor.start_link(children, opts)

```

* Start your app.