{"id":23020973,"url":"https://github.com/osskit/dafka-producer","last_synced_at":"2025-04-02T18:43:28.882Z","repository":{"id":37543328,"uuid":"373035508","full_name":"osskit/dafka-producer","owner":"osskit","description":"Dockerized kafka producer","archived":false,"fork":false,"pushed_at":"2024-03-26T06:45:24.000Z","size":394,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-08T09:23:16.610Z","etag":null,"topics":["dockerized","kafka","messaging","producer"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/osskit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-06-02T04:08:20.000Z","updated_at":"2023-06-08T17:33:05.000Z","dependencies_parsed_at":"2023-11-16T21:06:44.663Z","dependency_job_id":"ba53f0fb-26bc-4a48-bdb5-63857043ff6e","html_url":"https://github.com/osskit/dafka-producer","commit_stats":null,"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osskit%2Fdafka-producer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osskit%2Fdafka-producer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osskit%2Fdafka-producer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osskit%2Fdafka-producer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/osskit","download_url":"https://codeload.github.com/osskit/dafka-producer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246871421,"owners_count":20847477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dockerized","kafka","messaging","producer"],"created_at":"2024-12-15T12:16:01.063Z","updated_at":"2025-04-02T18:43:28.862Z","avatar_url":"https://github.com/osskit.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg width=\"300\" height=\"200\" src=\"https://user-images.githubusercontent.com/15312980/175078334-f284f44e-0366-4e24-8f09-5301b098ea64.svg\"/\u003e\n \u003c/p\u003e\n \n\u003cdiv align=\"center\"\u003e\nDockerized kafka producer\n  \n\u003c/div\u003e\n\n## Overview\nDafka-producer is a dockerized Kafka producer used to abstract producing messages to a kafka topic.\n\nProducing messages is as simple as sending a POST request from your service to the dafka-producer, with the topic, value, key and headers of the request being the kafka message.\n\n## Motivation\nWhy use this over just a Kafka client?\n* Abstracts away the messaging layer, could be replaced with RabbitMQ or any other producer.\n* Separates configuration, everything that's related to Kafka is encapsulated in Dafka and not the service itself.\n* When testing your service you only test your service's logic and not the messaging layer implementation details.\n\n## Design Diagram\n\u003cimg width=\"790\" alt=\"image\" src=\"https://user-images.githubusercontent.com/15312980/175814041-9991f7d5-830c-4e3f-9b2b-ad3e33228946.png\"\u003e\n\n## Usage \u0026 Examples\n\n### docker-compose\n```\nversion: '3.9'\n\nservices:\n    producer:\n        image: osskit/dafka-producer\n        ports:\n            - 6000:6000\n        environment:\n            - PORT=6000\n            - KAFKA_BROKER=kafka:9092\n        depends_on:\n            - kafka\n    # Generic Kafka Setup Containers\n    zookeeper:\n        image: wurstmeister/zookeeper\n    kafka:\n        image: wurstmeister/kafka:2.12-2.2.0\n        ports:\n            - '9092:9092'\n        environment:\n            - KAFKA_ADVERTISED_HOST_NAME=kafka\n            - KAFKA_CREATE_TOPICS=foo:1:1\n            - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181\n        depends_on:\n            - zookeeper\n```\nin joint with [`dafka-consumer`](https://github.com/osskit/dafka-consumer):\n```\nversion: '3.9'\n\nservices:\n    producer:\n        image: osskit/dafka-producer\n        ports:\n            - 6000:6000\n        environment:\n            - PORT=6000\n            - KAFKA_BROKER=kafka:9092\n        depends_on:\n            - kafka\n    consumer:\n        image: osskit/dafka-consumer\n        ports:\n            - 4001:4001\n        environment:\n            - KAFKA_BROKER=kafka:9092\n            - GROUP_ID=consumer_1\n            - TARGET_BASE_URL=http://target:8080\n            - TOPICS_ROUTES=foo:/consume\n            - MONITORING_SERVER_PORT=4001\n        depends_on:\n             - kafka\n    # Generic Kafka Setup Containers\n    zookeeper:\n        image: wurstmeister/zookeeper\n    kafka:\n        image: wurstmeister/kafka:2.12-2.2.0\n        ports:\n            - '9092:9092'\n        environment:\n            - KAFKA_ADVERTISED_HOST_NAME=kafka\n            - KAFKA_CREATE_TOPICS=foo:1:1\n            - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181\n        depends_on:\n            - zookeeper\n```\n\n### Kubernetes\nYou can use the provided [Helm Chart](https://github.com/osskit/dafka-producer-helm-chart), this gives you a `Deployment` separated from your service's `Pod`.\n\nIt's also possible to use this as a `Sidecar`.\n## Parameters\n\nContainer images are configured using parameters passed at runtime.\n\n| Parameter | Default Values | Description\n| :----: | --- | ---- |\n| `PORT` | `required` | Incoming requests' endpoint port | \n| `KAFKA_BROKER` | `required` | URL of the Kafka Broker | \n| `READINESS_TOPIC` | `null` | Producing to this topic will provide an healthcheck of the producer container |\n| `LINGER_TIME_MS` | `0` | [Description for LINGER_TIME_MS](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#producerconfigs_linger.ms)|\n| `COMPRESSION_TYPE` | `\"none\"` | [Description for COMPRESSION_TYPE](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#producerconfigs_compression.type)|\n| `USE_SASL_AUTH` | `false` | Use SASL authentication |\n| `SASL_USERNAME` | `required` if `USE_SASL_AUTH=true` | SASL Usernanme to authenticate |\n| `SASL_PASSWORD` | `required` if `USE_SASL_AUTH=true` | SASL Password to authenticate |\n| `TRUSTSTORE_FILE_PATH` | `null` | Truststore certificate file path |\n| `TRUSTSTORE_PASSWORD` | `required` if `TRUSTORE_FILE_PATH != null` | Truststore's password |\n| `USE_PROMETHEUS` | `false` | Export metrics to Prometheus |\n| `PROMETHEUS_BUCKETS` | `0.003,0.03,0.1,0.3,1.5,10` | A list of Prometheus buckets to use |\n\n## License\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosskit%2Fdafka-producer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fosskit%2Fdafka-producer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosskit%2Fdafka-producer/lists"}