{"id":43672322,"url":"https://github.com/sarkarshuvojit/kafka-sync-proxy","last_synced_at":"2026-02-05T00:13:56.980Z","repository":{"id":152127462,"uuid":"622525747","full_name":"sarkarshuvojit/kafka-sync-proxy","owner":"sarkarshuvojit","description":"Make http calls to Kafka that waits for responses.","archived":false,"fork":false,"pushed_at":"2025-06-23T15:43:08.000Z","size":51,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-23T16:53:42.511Z","etag":null,"topics":["golang","kafka","kafka-consumer","kafka-producer","microservices","request-reply"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sarkarshuvojit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-02T11:33:37.000Z","updated_at":"2025-06-23T15:43:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"34d5c9c8-6f4e-4f2d-97be-40fa3c8fff95","html_url":"https://github.com/sarkarshuvojit/kafka-sync-proxy","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/sarkarshuvojit/kafka-sync-proxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sarkarshuvojit%2Fkafka-sync-proxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sarkarshuvojit%2Fkafka-sync-proxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sarkarshuvojit%2Fkafka-sync-proxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sarkarshuvojit%2Fkafka-sync-proxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sarkarshuvojit","download_url":"https://codeload.github.com/sarkarshuvojit/kafka-sync-proxy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sarkarshuvojit%2Fkafka-sync-proxy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29102306,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T22:44:52.815Z","status":"ssl_error","status_checked_at":"2026-02-04T22:44:16.428Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","kafka","kafka-consumer","kafka-producer","microservices","request-reply"],"created_at":"2026-02-05T00:13:56.920Z","updated_at":"2026-02-05T00:13:56.974Z","avatar_url":"https://github.com/sarkarshuvojit.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Kafka Synchronous Proxy\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/sarkarshuvojit/kafka-sync-proxy.svg)](https://pkg.go.dev/github.com/sarkarshuvojit/kafka-sync-proxy)\n[![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)](https://hub.docker.com/r/sarkarshuvojit/kafka-sync-proxy)\n\nA web service to make traditional rest api calls which publishes a messsage into a topic \u0026 wait for the response in a response channel.\n\n\n## How does it work?\n\nThe following steps are followed \n\n- Create a random string  to be passed as **key** (currently done using uuidv4)\n- Prepare a kafka message containing the **key**, payload as the **value** and **headers** as is.\n- Produce it to the **requestTopic**\n- Consume from the **responseTopic** until a message with the exact **key** is received\n- If nothing is received in **REQ_TIMEOUT** seconds then the circuit breaks and **HTTP 408 Request Timeout** is sent back.\n*Note: Currently, **REQ_TIMEOUT** is set to 5s.*\n\n```mermaid\nsequenceDiagram\n    http-client-\u003e\u003e+kafka-sync-proxy: POST /v1 \n    note left of kafka-sync-proxy: requestTopic\u003cbr\u003eresponseTopic\u003cbr\u003epayload\u003cbr\u003eheaders  \n    note right of kafka-sync-proxy: requestTopic\u003cbr\u003ekey [coorelationId]\u003cbr\u003epayload\u003cbr\u003eheaders  \n    kafka-sync-proxy--\u003e\u003ekafka: produce\n\n    kafka--\u003e\u003emicroservice:consumes\n    note right of microservice: business logic\n    microservice--\u003e\u003ekafka: produces\n\n    kafka--\u003e\u003ekafka-sync-proxy: consume \n    note right of kafka-sync-proxy: responseTopic\u003cbr\u003ekey [coorelationId]\u003cbr\u003eresponse payload\u003cbr\u003eresponse headers  \n    note left of kafka-sync-proxy: application/json\u003cbr\u003e{respConse, headers}  \n    kafka-sync-proxy-\u003e\u003e-http-client: HTTP 200 OK\n\n\n    note left of kafka-sync-proxy: after REQ_TIMEOUT seconds have passed  \n    kafka-sync-proxy-\u003e\u003ehttp-client: HTTP 408 Request Timeout\n```\n## Installation\n\n### Go\n\nInstall from [pkg.go.dev](https://pkg.go.dev/github.com/sarkarshuvojit/kafka-sync-proxy)\n\n```bash\ngo install github.com/sarkarshuvojit/kafka-sync-proxy\n```\n\n### Docker \n\nRun docker image.\n\n```bash\ndocker pull sarkarshuvojit/kafka-sync-proxy\n```\n\n## Usage/Examples\n\n### Running\n\nIt can be run as a standalone as well as inside a docker container.\n\n#### standalone\n\nAfter installing from [pkg.go.dev](https://pkg.go.dev/github.com/sarkarshuvojit/kafka-sync-proxy)\n\n```bash\n$ kafka-sync-proxy\n\n ┌───────────────────────────────────────────────────┐\n │                   Fiber v2.43.0                   │\n │               http://127.0.0.1:8420               │\n │       (bound on host 0.0.0.0 and port 8420)       │\n │                                                   │\n │ Handlers ............. 1  Processes ........... 1 │\n │ Prefork ....... Disabled  PID ............ 536624 │\n └───────────────────────────────────────────────────┘\n\n\n```\n\n#### docker\n\n```bash\ndocker run -p 8420:8420 -d sarkarshuvojit/kafka-sync-proxy\n```\n\n#### docker-compose\n\nUse it with docker-compose\n\n```docker-compose\nversion: '2'\nservices:\n  kafka-sync-proxy:\n    container_name: kafka-sync\n    image: sarkarshuvojit/kafka-sync-proxy\n    ports:\n      - 8420:8420\n    networks:\n      - common-network\n\nnetworks:\n  common-network:\n```\n\n### Calling the API\n\nThere is only route hosted  at `/v1/` which only listens to `POST` \n\nLet's say you have a microservice listening to `com.org.events.testInitiated` which processes the message and responds to the `com.org.events.testCompleted` you can possibly test it by the following curl request.\n\n```bash\n$ curl --location --request POST 'http://localhost:8420/v1/' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{\n    \"requestTopic\": \"com.org.events.testInitiated\",\n    \"responseTopic\": \"com.org.events.testCompleted\",\n    \"payload\": {\n        \"message\": \"testing the proxy\",\n        \"nested\": {\n            \"child\": {\n                \"subChild\": 42\n            }\n        }\n    },\n    \"headers\": {\n        \"customAuthHeader\": \"ggg\"\n    },\n    \"brokers\": [\n        \"localhost:29092\"\n    ]\n}'\n```\n\nShould respond with:\n\n```json\nHTTP 200 OK\n\n{\n   \"headers\" : {\n      \"customAuthHeader\" : \"ggg\",\n      \"newHeaderKey\" : \"header val\"\n   },\n   \"response\" : {\n      \"message\" : \"testing the proxy\",\n      \"nested\" : {\n         \"child\" : {\n            \"subChild\" : 42\n         }\n      }\n   }\n}\n```\n\n\n*Note: This system requires that the response sends the message with the same **key** that was sent.*\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsarkarshuvojit%2Fkafka-sync-proxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsarkarshuvojit%2Fkafka-sync-proxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsarkarshuvojit%2Fkafka-sync-proxy/lists"}