https://github.com/sami12rom/kafka-connect-gitlab

Building Kafka Connectors for Gitlab with Kotlin
https://github.com/sami12rom/kafka-connect-gitlab

confluent gitlab kafka kafka-connector streaming

Last synced: 4 months ago
JSON representation

Building Kafka Connectors for Gitlab with Kotlin

• Host: GitHub
• URL: https://github.com/sami12rom/kafka-connect-gitlab
• Owner: sami12rom
• License: apache-2.0
• Created: 2024-02-04T15:31:25.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-03-17T16:56:45.000Z (over 2 years ago)
• Last Synced: 2024-03-18T13:39:49.147Z (over 2 years ago)
• Topics: confluent, gitlab, kafka, kafka-connector, streaming
• Language: Kotlin
• Homepage: https://events.bizzabo.com/559905/agenda/session/1284973
• Size: 74.2 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-kafka-connect - sami12rom/kafka-connect-gitlab - GitLab source connector built with Kotlin (Developer Tools / GitLab)

README

          
# Building a Kafka Connector for GitLab using Kotlin

This repository provides a Kafka connector implemented in Kotlin to streamline data integration between GitLab and your Kafka topics. Leverage the power of Kafka Connect for scalable and reliable data pipelines, and harness GitLab's rich source of events and metrics.

![Kotlin](https://img.shields.io/badge/Kotlin-0095D5?&style=for-the-badge&logo=kotlin&logoColor=white)

![Apache Kafka](https://img.shields.io/badge/Apache%20Kafka-000?style=for-the-badge&logo=apachekafka)

![Gitlab](https://img.shields.io/badge/GitLab-330F63?style=for-the-badge&logo=gitlab&logoColor=white)

![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge&logo=docker&logoColor=white)

## Getting Started

This guide helps you quickly set up and use the connector:

1. **Prerequisites:**

    - Java 8 or later

    - Apache Kafka installation

    - GitLab instance with API access

2. **Build and Run:**

    - Clone this repository: `git clone https://github.com/sami12rom/kafka-connect-gitlab.git`

    - Build the connector: `./gradlew build`

    - Start the connector using a Kafka Connect worker configuration file (see `examples/worker.properties`).

3. **Configuration:**

    - Customize the connector settings (like GitLab API URL, topics, events) in the worker configuration file.

## Features

- Capture GitLab events:

    - Push events

    - Merge request events

    - Issue events

- Choose data transformation options for structured or raw data.

- Flexible configuration via the worker properties file.

## Usage Examples

- Send GitLab push events to a Kafka topic for real-time CI/CD pipeline updates.

- Track GitLab issue activity and analyze trends using Kafka Stream processing.

- Integrate GitLab data with other systems like data lakes or BI tools.

## Contributions

We welcome any contributions to this project, whether it is bug reports, feature requests, code improvements, documentation updates, or feedback

## Authors

| # | Name           | Contact |

|---|----------------|---------|

| 1 | Sami Alashabi  |[![](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/sami-alashabi)|

| 2 | Ramzi Alashabi |[![](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/ramzialashabi)|

## License

This project is licensed under the Apache License 2.0.