{"id":22588437,"url":"https://github.com/654894017/reliable-message","last_synced_at":"2025-10-09T10:36:40.214Z","repository":{"id":129028632,"uuid":"380250719","full_name":"654894017/reliable-message","owner":"654894017","description":"通过消息的半提交机制，实现最终一致性的事务。同时通过消息回调检测机制，实现事务消息与TCC模型的完美融合。","archived":false,"fork":false,"pushed_at":"2022-06-09T05:50:52.000Z","size":3981,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-02T17:45:02.278Z","etag":null,"topics":["distributed-transaction","dubbo","grpc","kafka","reliable-message","rocketmq","sharding-jdbc","springcloud","tcc","thrift"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/654894017.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-06-25T13:43:22.000Z","updated_at":"2025-01-13T07:32:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"951a771e-5338-4472-aa84-a512f81be4c7","html_url":"https://github.com/654894017/reliable-message","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/654894017/reliable-message","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/654894017%2Freliable-message","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/654894017%2Freliable-message/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/654894017%2Freliable-message/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/654894017%2Freliable-message/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/654894017","download_url":"https://codeload.github.com/654894017/reliable-message/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/654894017%2Freliable-message/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001269,"owners_count":26083040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-transaction","dubbo","grpc","kafka","reliable-message","rocketmq","sharding-jdbc","springcloud","tcc","thrift"],"created_at":"2024-12-08T08:09:29.725Z","updated_at":"2025-10-09T10:36:40.198Z","avatar_url":"https://github.com/654894017.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"------------\n\n## 介绍\n\n**reliable-message**是基于可靠消息的**最终一致性**的分布式事务解决方案（需要MQ），同时也提供了Tcc事务模型解决方案，基于事务消息半提交原理，结合消息的回查机制（不依赖MQ的情况）。\n\n- RMQ不同于seata、tcc-transaction、Hmily等类似框架。RMQ给予用户最灵活的选择，不局限于dubbo、spring\n  cloud，对方接口可以是grpc、thrift、http等类似接口。只要业务方接口提供类似Try、Commit、Cancel接口，或Commit、Cancel接口。我们在业务层面通过硬编码的形式实现类型TCC或CC的效果。\n\n## 框架定位\n\n- RMQ本身不生产消息队列，只是消息的搬运工。\n- RMQ框架提供消息预发送、消息发送、消息确认、消息恢复、消息管理等功能，结合成熟的消息中间件（Kafka、RocketMQ），解决分布式事务，达到数据最终一致性。\n- RMQ基于事务消息半提交原理，结合消息的回查机制，实现类似TCC的事务模型（业务硬编码）。\n\n## 系统架构\n![架构图](images/architecture.png)\n\n| 角色 | 描述 |\n| --- | --- |\n| 消息确认模块 | 对于长时间未被确认的业务消息，与上游系统确认是否需要发送业务消息 |\n| 上游业务系统 | 根据消息确认模块回查的业务消息，判断业务是否完成，如果完成告诉消息确认模块已处理成功，如果失败回滚上游业务系统资源，告诉消息确认模块业务处理失败，可以删除消息 |\n| 下游业务系统 | 消费MQ数据，完成业务处理成功后，删除业务消息 |\n| 系统管理模块 | 消息可视化管理后台，消息队列配置、消息查看、重发、删除等 |\n| 消息恢复模块 | 对于长时间未消费的业务消息，重新发送业务消息给下层业务系统 |\n| 死信消息检测模块 | 对于重发次数达到一定值的业务消息标记未死亡消息，不在被发送，转人工重发处理或者删除 |\n| 消息服务模块 | 暴露服务的服务提供方，业务服务实现（预发送业务消息、确认发送业务消息、删除业务消息等） |\n| MQ | 提供消息队列功能，Kafka、RocketMQ等|\n\n\n------------\n\n## Maven模块描述\n\n| 模块名称 | 描述 |\n| --- | --- |\n| rmq-api | 提供业务系统调用的RMQ服务接口 |\n| rmq-service-api | 基础消息服务接口、系统工具类、实体类封装 |\n| rmq-service | RMQ服务接口实现、基础消息服务接口实现、消息管理子系统服务接口实现 |\n| rmq-schedule-api | 消息确认模块、消息恢复模块、死信消息检测模块服务接口 |\n| rmq-schedule | 消息确认模块，与上游业务系统确认消息是否发送\u003cbr\u003e消息恢复模块，重新发送消息给下游业务 |\n| rmq-admin-api | 系统管理模块服务接口、实体类封装 |\n| rmq-admin | 系统管理模块，提供消息管理后台 |\n| rmq-storage | 消息持久化层|\n\n------------\n\n## 在业务代码中引入RMQ的Dubbo服务\n\n```\nimport org.apache.dubbo.config.annotation.DubboReference;\nimport com.damon.rmq.api.service.IReliableMessageService;\n\n@DubboReference\nprivate IReliableMessageService reliableMessageService;\n```\n\n## 编写消息发送方业务方法\n\n结合事务消息实现TCC效果，如果不需要使用MQ传递领域消息到其他业务模块，可以在完成业务后删除事务消息，不需要confirm它。\n\n- 案例1\n\n```\npublic void doBusiness() {\n        // 自定义消息队列名称\n        String queue = \"test_queue\";\n        // 消息内容, 如果传输对象，建议转换成json字符串\n        String messageContent = \"......\";\n        // 调用RMQ，预发送消息\n        String messageId = reliableMessageService.createPreMessage(queue, messageContent);\n\n        try{\n          //以下步骤可以异步多线程执行\n          //执行业务1 Try(业务层面需要做好幂等、悬挂)\n          //执行业务2 Try(业务层面需要做好幂等、悬挂)\n          //执行业务3 Commit(业务层面需要做好幂等、悬挂)    \n        }catch(Throwable e){\n         RpcContext.getContext().asyncCall(() -\u003e \n            // 回滚业务1 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n            // 回滚业务2 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n            // 执行业务3 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n            // 删除预发送消息\n            reliableMessageService.deleteMessage(queue, messageId)\n         );\n         return;\n        }\n        RpcContext.getContext().asyncCall(() -\u003e \n          //执行业务1 Commit(业务层面需要做好幂等、悬挂)\n          //执行业务2 Commit(业务层面需要做好幂等、悬挂)    \n          //异步调用RMQ，确认发送消息(如果是当做分布式事务框架使用，不需要对外发送消息，则不需要进行消息confirm操作，直接调用deleteMessage删除事务消息即可)\n          reliableMessageService.confirmAndSendMessage(queue, messageId);\n        );\n    }\n```\n\n## 案例1编写消息消费方业务方法（RocketMQ）\n\n```\n@Component\n@Slf4j\npublic class PayQueueRocketmqConsumer {\n\n    @DubboReference\n    private IRechargeOrderService rechargeOrderService;\n    @DubboReference\n    private IReliableMessageService reliableMessageService;\n\n    @PostConstruct\n    public void handler() throws MQClientException {\n        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(\"pay_queue_consumer_group\");\n        consumer.setNamesrvAddr(\"localhost:9876\");\n        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);\n        consumer.setConsumeThreadMax(1);\n        consumer.setConsumeThreadMin(1);\n        consumer.setMaxReconsumeTimes(1);\n        consumer.subscribe(\"pay_queue\", \"*\");\n        consumer.registerMessageListener(new MessageListenerConcurrently() {\n\n            @Override\n            public ConsumeConcurrentlyStatus consumeMessage(List\u003cMessageExt\u003e msgs, ConsumeConcurrentlyContext context) {\n                try {\n                    System.out.printf(\"%s Receive New Messages: %s %n\", Thread.currentThread().getName(), msgs);\n                    for (MessageExt message : msgs) {\n                        String body = new String(message.getBody(), Charset.forName(\"UTF-8\"));\n                        TransactionMessage msg = JSONUtil.toBean(body, TransactionMessage.class);\n                        log.info(\"【payQueue】开始处理消息\" + msg);\n                        PayOrder payOrder = JSONUtil.toBean(msg.getMessageBody(), PayOrder.class);\n                        rechargeOrderService.rechargeSuccess(payOrder);\n                        reliableMessageService.deleteMessage(message.getTopic(), msg.getMessageId());\n                        log.info(\"【payQueue】处理消息成功\");\n                    }\n                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;\n                } catch (Throwable e) {\n                    log.info(\"【payQueue】处理消息失败\", e);\n                    return ConsumeConcurrentlyStatus.RECONSUME_LATER;\n                }\n\n            }\n        });\n        consumer.start();\n        System.out.printf(\"pay queue consumer started.\");\n    }\n}\n        \n```\n\n## 案例1编写消息消费方业务方法（Kafka）\n\n```\n@Component\n@Slf4j\n@Configuration\npublic class PayQueueKafkaConsumer {\n\n    @Bean\n    public ConcurrentKafkaListenerContainerFactory\u003cString, String\u003e kafkaListenerContainerFactory() {\n        ConcurrentKafkaListenerContainerFactory\u003cString, String\u003e factory =\n            new ConcurrentKafkaListenerContainerFactory\u003cString, String\u003e();\n        factory.setConsumerFactory(consumerFactory());\n        return factory;\n    }\n\n    @Bean\n    public ConsumerFactory\u003cString, String\u003e consumerFactory() {\n        return new DefaultKafkaConsumerFactory\u003cString, String\u003e(consumerConfigs());\n    }\n\n    @Bean\n    public Map\u003cString, Object\u003e consumerConfigs() {\n        HashMap\u003cString, Object\u003e props = new HashMap\u003cString, Object\u003e();\n        props.put(\"bootstrap.servers\", \"localhost:9092\");\n        props.put(\"group.id\", \"pay_queue_consumer_group\");\n        props.put(\"enable.auto.commit\", \"true\");\n        props.put(\"auto.commit.interval.ms\", \"1000\");\n        props.put(\"key.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n        props.put(\"value.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n        return props;\n    }\n\n    @DubboReference\n    private IRechargeOrderService rechargeOrderService;\n    @DubboReference\n    private IReliableMessageService reliableMessageService;\n\n    @KafkaListener(id = \"pay\", topics = \"pay_queue\")\n    public void listen(ConsumerRecord\u003cString, String\u003e record) {\n        String body = record.value();\n        TransactionMessage msg = JSONUtil.toBean(body, TransactionMessage.class);\n        log.info(\"【payQueue】开始处理消息\" + msg);\n        PayOrder payOrder = JSONUtil.toBean(msg.getMessageBody(), PayOrder.class);\n        rechargeOrderService.rechargeSuccess(payOrder);\n        reliableMessageService.deleteMessage(\"pay_queue\", msg.getMessageId());\n        log.info(\"【payQueue】处理消息成功\");\n    }\n\n}\n        \n```\n\n- 案例2\n\n```\npublic void doBusiness() {\n        // 自定义消息队列名称\n        String queue = \"test_queue\";\n        // 消息内容, 如果传输对象，建议转换成json字符串\n        String messageContent = \"......\";\n        // 调用RMQ，预发送消息\n        String messageId = reliableMessageService.createPreMessage(queue, messageContent);\n\n        try{\n         \n         // 执行业务1 Try(业务层面需要做好幂等、悬挂)\n         // 执行业务2 Try(业务层面需要做好幂等、悬挂)\n         // 执行业务3 Commit(业务层面需要做好幂等、悬挂)    \n        }catch(Throwable e){\n         // 回滚业务1 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\t                     \n         // 回滚业务2 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)                      \n         // 执行业务3 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n         RpcContext.getContext().asyncCall(() -\u003e reliableMessageService.deleteMessage(queue, messageId));\n         return;\n        } \n        // 异步调用RMQ，确认发送消息(如果是当做分布式事务框架使用，不需要对外发送消息，则不需要进行消息confirm操作，直接调用deleteMessage删除事务消息即可)\n        RpcContext.getContext().asyncCall(() -\u003e reliableMessageService.confirmAndSendMessage(queue, messageId));\n    }\n```\n\n## 案例2消息消费方业务方法，（RocketMQ）\n\n```\n@Component\n@Slf4j\npublic class PayQueueRocketmqConsumer {\n\n    @DubboReference\n    private IRechargeOrderService rechargeOrderService;\n    @DubboReference\n    private IReliableMessageService reliableMessageService;\n\n    @PostConstruct\n    public void handler() throws MQClientException {\n        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(\"pay_queue_consumer_group\");\n        consumer.setNamesrvAddr(\"localhost:9876\");\n        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);\n        consumer.setConsumeThreadMax(1);\n        consumer.setConsumeThreadMin(1);\n        consumer.setMaxReconsumeTimes(1);\n        consumer.subscribe(\"pay_queue\", \"*\");\n        consumer.registerMessageListener(new MessageListenerConcurrently() {\n\n            @Override\n            public ConsumeConcurrentlyStatus consumeMessage(List\u003cMessageExt\u003e msgs, ConsumeConcurrentlyContext context) {\n                try {\n                    System.out.printf(\"%s Receive New Messages: %s %n\", Thread.currentThread().getName(), msgs);\n                    for (MessageExt message : msgs) {\n                       \n                        String body = new String(message.getBody(), Charset.forName(\"UTF-8\"));\n                        TransactionMessage msg = JSONUtil.toBean(body, TransactionMessage.class);\n                        \n                        log.info(\"【payQueue】开始处理消息\" + msg);\n                        PayOrderCancel cancel = JSONUtil.toBean(msg.getMessageBody(), PayOrderCancel.class);\n                        // 执行业务1 Commit(业务层面需要做好幂等、悬挂)\n                        // 执行业务2 Commit(业务层面需要做好幂等、悬挂)   \n                        //处理下游业务\n                        rechargeOrderService.rechargeSuccess(payOrder);\n                        //删除rmq消息\n                        reliableMessageService.deleteMessage(message.getTopic(), msg.getMessageId());\n                        \n                        log.info(\"【payQueue】处理消息成功\");\n                    }\n                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;\n                } catch (Throwable e) {\n                    log.info(\"【payQueue】处理消息失败\", e);\n                    return ConsumeConcurrentlyStatus.RECONSUME_LATER;\n                }\n\n            }\n        });\n        consumer.start();\n        System.out.printf(\"pay queue consumer started.\");\n    }\n}\n        \n```\n\n## 编写业务回调check方法\n\n当执行doBusiness异常回滚业务时或业务在Commit时，系统奔溃，消息确认子系统定时发起消息确认\n\n```\n@RequestMapping(\"check\")\n@ResponseBody\npublic CheckStatus checkBusStatus(BusReq req) {\n   \n   //如果业务执行成功 \n   //return new CheckStats(0,1||2)\n   \n   \n   //如果业务执行失败\n   //回滚业务1 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n   //回滚业务2 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n   //执行业务3 Cancel(业务层面需要做好幂等、悬挂、空回滚问题)\n   //return new CheckStats(0,0)\n   \n}\n\nCheckStatus 格式\n{\t\n  \"code\": 0,  // 0 成功  1 失败 \n  \"data\": 1   // 0 业务处理失败，删除半提交消息 1 业务处理成功，RMQ发送半消息到MQ中间件 2 业务处理成功，RMQ删除半提交消息 \n}\n\n```\n\n### 为什么会有3种状态？\n\n- 0 业务没有处理成功，回滚完所有业务后，半提交消息需要删除。\n- 1 业务处理成功了，只是刚好在消息confirm时系统宕机了，此时消息确认子系统check业务系统后需要重新发送。\n- 2 不需要传递领域消息到其他业务模块，业务已经完成了，需要删除了（虽然和0状态码效果是一样的，还是区分开来好一点）。\n\n## 业务接口注意事项\n\n- 幂等性：不管是MQ消费服务，还是业务提供的Try、Commit、Cancel接口都需要满足幂等性要求，因为极端异常情况下，消息确认子系统会check业务系统做数据一致性修正。存在重复调用的情况，也存在消息重复发送MQ的情况。\n- 空回滚：业务系统有可能没执行Try，结果被执行Cancle的情况。需要保障不允许空回滚的情况。\n- 悬挂：由于网络问题业务先被Cancel了然，后又收到Try的动作。需要保证Try不会被执行。\n\n基于以上三个问题是分布式事务中一定会遇到的：可以引入一个**业务幂等表**来解决消息的幂等性、空回滚、悬挂问题。\n\n------------\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F654894017%2Freliable-message","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F654894017%2Freliable-message","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F654894017%2Freliable-message/lists"}