https://github.com/dashbitco/broadway_kafka

A Broadway connector for Kafka
https://github.com/dashbitco/broadway_kafka

broadway genstage kafka

Last synced: 5 months ago
JSON representation

A Broadway connector for Kafka

• Host: GitHub
• URL: https://github.com/dashbitco/broadway_kafka
• Owner: dashbitco
• Created: 2019-07-05T14:11:41.000Z (over 6 years ago)
• Default Branch: main
• Last Pushed: 2025-03-28T09:41:29.000Z (6 months ago)
• Last Synced: 2025-05-07T19:47:27.424Z (5 months ago)
• Topics: broadway, genstage, kafka
• Language: Elixir
• Homepage:
• Size: 177 KB
• Stars: 239
• Watchers: 13
• Forks: 53
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md

Awesome Lists containing this project

README

          
# BroadwayKafka

[![Build Status](https://github.com/dashbitco/broadway_kafka/actions/workflows/ci.yml/badge.svg)](https://github.com/dashbitco/broadway_kafka/actions/workflows/ci.yml)

A Kafka connector for [Broadway](https://github.com/dashbitco/broadway).

Documentation can be found at [https://hexdocs.pm/broadway_kafka](https://hexdocs.pm/broadway_kafka).

## Installation

Add `:broadway_kafka` to the list of dependencies in `mix.exs`:

```elixir

def deps do

  [

    {:broadway_kafka, "~> 0.4.1"}

  ]

end

```

## Usage

Configure Broadway's producer using `BroadwayKafka.Producer`:

```elixir

  defmodule MyBroadway do

    use Broadway

    def start_link(_opts) do

      Broadway.start_link(__MODULE__,

        name: __MODULE__,

        producer: [

          module: {BroadwayKafka.Producer, [

            hosts: [localhost: 9092],

            group_id: "group_1",

            topics: ["test"],

          ]},

          concurrency: 1

        ],

        processors: [

          default: [

            concurrency: 10

          ]

        ]

      )

    end

    def handle_message(_, message, _) do

      IO.inspect(message.data, label: "Got message")

      message

    end

  end

```

## License

Copyright 2019 Plataformatec\

Copyright 2020 Dashbit

  Licensed under the Apache License, Version 2.0 (the "License");

  you may not use this file except in compliance with the License.

  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.