{"id":27063566,"url":"https://github.com/hb-go/micro-mq","last_synced_at":"2026-01-12T02:38:31.901Z","repository":{"id":57487915,"uuid":"133653393","full_name":"hb-chen/gmqtt","owner":"hb-chen","description":"Golang MQTT Broker.","archived":false,"fork":false,"pushed_at":"2023-02-25T03:06:58.000Z","size":959,"stargazers_count":80,"open_issues_count":3,"forks_count":12,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-18T12:29:13.233Z","etag":null,"topics":["broker","im","iot","kafka","microservice","mq","mqtt","mqtt-proxy"],"latest_commit_sha":null,"homepage":"http://gmqtt.k8s.hbchen.com:80","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hb-chen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-05-16T11:07:57.000Z","updated_at":"2024-03-30T09:07:42.000Z","dependencies_parsed_at":"2023-07-15T04:17:16.381Z","dependency_job_id":null,"html_url":"https://github.com/hb-chen/gmqtt","commit_stats":null,"previous_names":["hb-chen/micro-mq","hb-go/micro-mq"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hb-chen%2Fgmqtt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hb-chen%2Fgmqtt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hb-chen%2Fgmqtt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hb-chen%2Fgmqtt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hb-chen","download_url":"https://codeload.github.com/hb-chen/gmqtt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247361593,"owners_count":20926641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broker","im","iot","kafka","microservice","mq","mqtt","mqtt-proxy"],"created_at":"2025-04-05T16:01:26.922Z","updated_at":"2025-04-05T16:01:47.706Z","avatar_url":"https://github.com/hb-chen.png","language":"Go","funding_links":[],"categories":["kafka"],"sub_categories":[],"readme":"# Micro MQ\n以微服务+MQ构建支持高并发连接的分布式消息服务系统\n\n\u003e [消息持久化及写放大模式设计](https://github.com/hb-chen/gmqtt/issues/1#issuecomment-518517597)\n\n![micro-mq](/doc/img/architecture.jpg \"micro-mq\")\n\n- Gateway节点`Node`通过订阅MQ消息的方式，完成消息在节点间的转发\n- 根据业务场景的需求，需要考虑Node节点消息消费与生产速度的匹配\n    - Client间的pub/sub关系比较多，如`n`个node, 发送`m`条消息/topic/node，节点消费的需求是`n`*`m`条/topic\n    - Client端多为pub操作，而系统下发消息较少的情况，节点消费需求则比较低\n    \nGateway编程模型\n\n![micro-mq](/doc/img/gateway_model.jpg \"gateway_modem\")\n### Features\n\n## 运行\n#### 服务依赖\n\u003e 根据配置选择 conf/conf.toml\n- MQ `conf.broker`\n    - [x] Kafka \n- 服务注册与发现 `conf.auth`\n    - [x] Etcd\n    - [ ] Consul\n    \n#### 启动服务\n```bash\n# 启动Gateway，[-h]帮助查看可选参数\n$ cd gateway\n$ go run -tags \"consul\" main.go\n\n# RPC Auth服务，[-h]帮助查看可选参数\n$ cd api\n$ go run -tags \"consul\" main.go\n```\n\n#### 在线演示\n- [MQTT Web Client](http://mqtt-client.hbchen.com/)\n  - [Github源码](https://github.com/hb-chen/hivemq-mqtt-web-client)\n- Gateway 测试服务\n  - `host`=`gmqtt.k8s.hbchen.com`\n  - `port`=`80`\n\n## 组件\n- gateway\n    - sessions\n        - [x] mock\n        - [x] redis\n    - topic\n        - [x] mem\n    - client auth\n        - [x] mock\n        - [x] rpc\n    - pub/sub auth\n        - [ ] mock\n        - [ ] rpc\n- broker\n    - [x] kafka\n    \n- \u003e Developing\n    - api\n        - auth\n            - [ ] RPC服务间的访问控制：RBAC\n        - client\n            - auth\n                - [x] client auth\n                - [ ] pub/sub auth\n            - register\n                - [ ] register\n                - [ ] unregister\n        - cluster\n            - nodes\n                - 节点列表\n            - clients\n                - 终端列表\n            - sessions\n                - 会话列表\n            - topics\n                - 话题信息\n            - subscriptions\n                - 订阅信息\n    - console\n    - deploy\n\n## Frameworks\n- [rpcx](https://github.com/smallnest/rpcx)\n- [Echo](https://github.com/labstack/echo)\n\n## Benchmark\n- MBP开发环境\n- mock broker\n- pub timeout 3000 ms, size 64\n```bash\n$ go test -run=TestClient$\n# 1 client pub, sent 100000 messages, no sub\n# qos 0\nTotal sent 100000 messages dropped 0 in 3096.074512 ms, 0.030961 ms/msg, 32298 msgs/sec\n# qos 1\nTotal sent 100000 messages dropped 0 in 10411.318733 ms, 0.104113 ms/msg, 9604 msgs/sec\n```\n```bash\n$ go test -run=TestClients$\n# 1000 clients, sent 1000 messages/client, no sub\n# qos 0\nTotal Sent 1000000 messages dropped 0 in 14628.666153 ms, 0.014629 ms/msg, 68358 msgs/sec\n# qos 1\nTotal Sent 1000000 messages dropped 0 in 38669.812430 ms, 0.038670 ms/msg, 25859 msgs/sec\n\n# 1000 clients, sent 1000 messages/client, 1 client sub\n# qos 1\nTotal Sent 1000000 messages dropped 0 in 65403.199238 ms, 0.065403 ms/msg, 15289 msgs/sec\nTotal Received 1000000 messages in 65403.199238 ms, 0.065403 ms/msg, 15289 msgs/sec\n# qos 2\nTotal Sent 1000000 messages dropped 0 in 68339.624216 ms, 0.068340 ms/msg, 14632 msgs/sec\nTotal Received 1000000 messages in 68339.624216 ms, 0.068340 ms/msg, 14632 msgs/sec\n```\n\n## 备用命令\n#### Protobuf\n```bash\n# 在.proto有import时注意相对的路径\nprotoc -I=$GOPATH/src:.  --go_out=.  api/proto/define.proto\n```\n#### Kafka\n```bash\n# Start the server\n$ bin/zookeeper-server-start.sh config/zookeeper.properties\n$ bin/kafka-server-start.sh config/server.properties\n\n# List topic\nbin/kafka-topics.sh --list --zookeeper localhost:2181\n\n# Start a consumer\nbin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic --from-beginning\n```\n#### Kafka Manager\n```bash\n# 启动报错，需修改conf/application.conf\nkafka-manager.zkhosts=\"localhost:2181\"\n```\n\n## 参考内容\n- [[转][译]百万级WebSockets和Go语言](https://colobu.com/2017/12/13/A-Million-WebSockets-and-Go/)\n    - [[原]A Million WebSockets and Go](https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb)\n- [SurgeMQ](https://github.com/surgemq/surgemq)\n- [Micro](http://github.com/micro)\n- [Managing IoT devices with Kafka and MQTT](https://www.ibm.com/blogs/bluemix/2017/01/managing-iot-devices-with-kafka-and-mqtt/)\n- [MQTT QoS（服务质量）介绍](https://www.emqx.cn/blog/introduction-to-mqtt-qos)\n- [StreamNative 宣布开源 MoP：Apache Pulsar 支持原生 MQTT 协议](https://segmentfault.com/a/1190000025167243)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhb-go%2Fmicro-mq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhb-go%2Fmicro-mq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhb-go%2Fmicro-mq/lists"}