{"id":18771688,"url":"https://github.com/majidgolshadi/kafka-rest-proxy","last_synced_at":"2025-12-12T12:30:12.838Z","repository":{"id":85103871,"uuid":"91468031","full_name":"majidgolshadi/kafka-rest-proxy","owner":"majidgolshadi","description":"Stand-alone Kafka rest proxy written in go","archived":false,"fork":false,"pushed_at":"2018-07-14T11:40:13.000Z","size":16,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-22T14:51:16.557Z","etag":null,"topics":["golang","http-proxy","kafka","rest-api","zookeeper"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/majidgolshadi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-16T14:29:22.000Z","updated_at":"2021-01-05T16:57:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"4c3912f1-3c48-42ec-bc4f-01149ead0e28","html_url":"https://github.com/majidgolshadi/kafka-rest-proxy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majidgolshadi%2Fkafka-rest-proxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majidgolshadi%2Fkafka-rest-proxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majidgolshadi%2Fkafka-rest-proxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majidgolshadi%2Fkafka-rest-proxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/majidgolshadi","download_url":"https://codeload.github.com/majidgolshadi/kafka-rest-proxy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239679559,"owners_count":19679500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","http-proxy","kafka","rest-api","zookeeper"],"created_at":"2024-11-07T19:25:56.493Z","updated_at":"2025-12-12T12:30:12.793Z","avatar_url":"https://github.com/majidgolshadi.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"Kafka rest proxy\n================\nStand-alone Kafka rest proxy written in go.\nAny instance register himself on zookeeper (register namespace is configurable) and connect to all kafka brokers.\nyou can pass your content on `\u003cURI\u003e:\u003cPORT\u003e/topic/\u003cTOPIC_NAME\u003e` uri as message body to put on kafka topics\n\nConfiguration\n-------------\nIn order to configure service it need to make a `config.toml` file and put it near the application binary\n\n```toml\nadvertised_listener=\"127.0.0.1:8090\" #Optional ip and port that http proxy bind on (default: 0.0.0.0:8080)\ndebug_port=\":6060\"                   #Optional debugging port (default: 6060)\n\n[log]\nformat=\"json\"           #text,json (default: text)\nlog_level=\"debug\"       #Optiona debug,info,warning,error (default: warning)\nlog_point=\"/directory/path/to/store/logs\" #Optional (default: stdout)\n\n[kafka-producer]\nzookeeper=\"127.0.0.1:2181/kafka\"\nzookeeper_timeout=5             #Second\nmax_retries=10                  #Optional (default: 10)\nproducer_registration_namespace=\"/kafka-rest-proxy\" #Optional (default: /kafka-rest-proxy)\n```\n\nBenchmark test\n--------------\nBenchmark was given on a system with 2 core cpu and 2 gig ram and a kafka instance was installed on\n```bash\nab -kl  -n 1000000 -c 100 -p post.json  192.168.120.84:8080/topic/archive-messages\n\nBenchmarking 192.168.120.84 (be patient)\nCompleted 100000 requests\nCompleted 200000 requests\nCompleted 300000 requests\nCompleted 400000 requests\nCompleted 500000 requests\nCompleted 600000 requests\nCompleted 700000 requests\nCompleted 800000 requests\nCompleted 900000 requests\nCompleted 1000000 requests\nFinished 1000000 requests\n\n\nServer Software:        \nServer Hostname:        192.168.120.84\nServer Port:            8080\n\nDocument Path:          /topic/archive-messages\nDocument Length:        Variable\n\nConcurrency Level:      100\nTime taken for tests:   79.293 seconds\nComplete requests:      1000000\nFailed requests:        0\nKeep-Alive requests:    1000000\nTotal transferred:      202001114 bytes\nTotal body sent:        4279000000\nHTML transferred:       61001114 bytes\nRequests per second:    12611.45 [#/sec] (mean)\nTime per request:       7.929 [ms] (mean)\nTime per request:       0.079 [ms] (mean, across all concurrent requests)\nTransfer rate:          2487.82 [Kbytes/sec] received\n                        52699.61 kb/s sent\n                        55187.43 kb/s total\n\nConnection Times (ms)\n              min  mean[+/-sd] median   max\nConnect:        0    0   0.0      0       4\nProcessing:     1    8   4.1      8     214\nWaiting:        1    8   4.1      8     214\nTotal:          1    8   4.1      8     214\n\nPercentage of the requests served within a certain time (ms)\n  50%      8\n  66%      9\n  75%     10\n  80%     10\n  90%     12\n  95%     13\n  98%     16\n  99%     21\n 100%    214 (longest request)\n```\n\nDebugging\n---------\nDebugging rest APIs\n\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/goroutine\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/heap\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/threadcreate\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/block\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/mutex\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/profile\n- http://`\u003cSERVER_IP\u003e:\u003cdebug_port\u003e`/debug/pprof/trace?seconds=5\n\nCall `http://\u003cSERVER_IP\u003e:\u003cdebug_port\u003e/debug/pprof/trace?seconds=5` to get 5 second of application trace file and then you can see application trace. With\n`go tool trace \u003cDOWNLOADED_FILE_PATH\u003e` command you can see what's happen in application on that period of time\n\nCall `http://\u003cSERVER_IP\u003e:\u003cdebug_port\u003e/debug/pprof/profile` to get service profile and then run `go tool pprof \u003cDOWNLOADED_FILE_PATH\u003e` command go see more details about appli   cation processes\n\nTo get more information you can see [How to use pprof](https://www.integralist.co.uk/posts/profiling-go/) article","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmajidgolshadi%2Fkafka-rest-proxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmajidgolshadi%2Fkafka-rest-proxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmajidgolshadi%2Fkafka-rest-proxy/lists"}