{"id":19036183,"url":"https://github.com/tx7do/kratos-cqrs","last_synced_at":"2025-04-23T19:07:19.261Z","repository":{"id":112433650,"uuid":"466024881","full_name":"tx7do/kratos-cqrs","owner":"tx7do","description":null,"archived":false,"fork":false,"pushed_at":"2023-06-20T16:58:49.000Z","size":602,"stargazers_count":25,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T19:07:07.004Z","etag":null,"topics":["consul","cqrs","go-kratos","golang","kafka","kratos-transport","zookeeper"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tx7do.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-04T07:26:30.000Z","updated_at":"2024-12-20T04:07:04.000Z","dependencies_parsed_at":"2025-04-17T15:08:28.408Z","dependency_job_id":null,"html_url":"https://github.com/tx7do/kratos-cqrs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tx7do%2Fkratos-cqrs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tx7do%2Fkratos-cqrs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tx7do%2Fkratos-cqrs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tx7do%2Fkratos-cqrs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tx7do","download_url":"https://codeload.github.com/tx7do/kratos-cqrs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250496990,"owners_count":21440231,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consul","cqrs","go-kratos","golang","kafka","kratos-transport","zookeeper"],"created_at":"2024-11-08T21:53:40.708Z","updated_at":"2025-04-23T19:07:19.255Z","avatar_url":"https://github.com/tx7do.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kratos-cqrs\n\n这是一个实验性质的项目,主要是为了实验在微服务框架 [Kratos](https://go-kratos.dev/) 里面调用Kafka进行数据库写操作.\n\n因为现在的Kratos不原生支持对队列的支持,所以我自己捯饬了一个项目 [kratos-transport](https://github.com/tx7do/kratos-transport) 做了支持,现在支持:kafka,RabbitMQ,MQTT,Redis,Websocket,NATS,Websocket.\n\n抄了 [go-micro](https://github.com/asim/go-micro) 里边的Broker代码,但是我发现asim似乎只管写不管测,老多bug了.\n\n## 什么是 CQRS\n\n命令查询的责任分离Command Query Responsibility Segregation (简称CQRS)模式是一种架构体系模式，能够使改变模型的状态的命令和模型状态的查询实现分离。这属于DDD应用领域的一个模式，主要解决DDD在数据库报表输出上处理方式。\n\n曾经是 读写分离:\n\n![](docs/rws.png)\n\n在微服务的时代,变成了这样:\n\n![](docs/cqrs.png)\n\n## 项目简要说明\n\n项目分为两个微服务:\n\n- 日志查询服务(kratos.logger.service)\n- 日志写入服务(kratos.logger.job)\n\n## 涵盖的技术点\n\n- 使用Kratos开发微服务\n- 使用Kratos框架实验CQRS\n- 使用TimeScaleDB(其实质为PostgreSQL的插件)存取时序数据\n- 使用Facbook的Entgo操作TimeScaleDB的时序数据\n- 集成Kafka,并将Kafka封装成Kratos的一个Server,用于接收数据\n- 将服务注册进入Consul,用于服务发现\n- 将服务注册进入Jaeger,用于链路跟踪\n- 从远程配置中心读取配置,Consul和Nacos可以直接用\n\n### 日志查询服务\n\n- 开了gRPC服务用来给其他服务查询之用(Postman已经支持调试gRPC)\n- 使用ORM框架Entgo进行TimeScaleDB的普通以及时序查询\n\n### 日志写入服务\n\n连接Kafka,订阅以下几个Topic:\n\n- `logger.sensor.ts` 用于写入传感器的遥感数据\n- `logger.sensor.instance` 用于写入传感器的实体数据(实际设计上,它不应该存在在这里)\n\n## 技术栈\n\n- [Kratos](https://go-kratos.dev/)\n- [TimeScaleDB](https://www.timescale.com/)\n- [Kafka](https://kafka.apache.org/)\n- [Consul](https://www.consul.io/)\n- [Jaeger](https://www.jaegertracing.io/)\n- [Entgo](https://entgo.io/)\n\n## Docker部署开发服务器\n\n### TimeScaleDB\n\n```shell\ndocker pull timescale/timescaledb:latest-pg14\ndocker pull timescale/timescaledb-postgis:latest-pg13\ndocker pull timescale/pg_prometheus:latest-pg11\n\ndocker run -itd \\\n    --name timescale-test \\\n    -p 5432:5432 \\\n    -e POSTGRES_PASSWORD=123456 \\\n    timescale/timescaledb-postgis:latest-pg13\n```\n\n### Kafka\n\n```shell\ndocker pull bitnami/kafka:latest\ndocker pull bitnami/zookeeper:latest\ndocker pull hlebalbau/kafka-manager:latest\n\ndocker run -itd \\\n    --name zookeeper-test \\\n    -p 2181:2181 \\\n    -e ALLOW_ANONYMOUS_LOGIN=yes \\\n    bitnami/zookeeper:latest\n\ndocker run -itd \\\n    --name kafka-standalone \\\n    --link zookeeper-test \\\n    -p 9092:9092 \\\n    -v /home/data/kafka:/bitnami/kafka \\\n    -e KAFKA_BROKER_ID=1 \\\n    -e KAFKA_LISTENERS=PLAINTEXT://:9092 \\\n    -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092 \\\n    -e KAFKA_ZOOKEEPER_CONNECT=zookeeper-test:2181 \\\n    -e ALLOW_PLAINTEXT_LISTENER=yes \\\n    --user root \\\n    bitnami/kafka:latest\n\ndocker run -itd \\\n     -p 9000:9000  \\\n     -e ZK_HOSTS=\"localhost:2181\" \\\n     hlebalbau/kafka-manager:latest\n```\n\n### Consul\n\n```shell\ndocker pull bitnami/consul:latest\n\ndocker run -itd \\\n    --name consul-server-standalone \\\n    -p 8300:8300 \\\n    -p 8500:8500 \\\n    -p 8600:8600/udp \\\n    -e CONSUL_BIND_INTERFACE='eth0' \\\n    -e CONSUL_AGENT_MODE=server \\\n    -e CONSUL_ENABLE_UI=true \\\n    -e CONSUL_BOOTSTRAP_EXPECT=1 \\\n    -e CONSUL_CLIENT_LAN_ADDRESS=0.0.0.0 \\\n    bitnami/consul:latest\n```\n\n### Jaeger\n\n```shell\ndocker pull jaegertracing/all-in-one:latest\n\ndocker run -d \\\n    --name jaeger \\\n    -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \\\n    -p 5775:5775/udp \\\n    -p 6831:6831/udp \\\n    -p 6832:6832/udp \\\n    -p 5778:5778 \\\n    -p 16686:16686 \\\n    -p 14268:14268 \\\n    -p 14250:14250 \\\n    -p 9411:9411 \\\n    jaegertracing/all-in-one:latest\n```\n\n## 测试\n\n下载 [Postman](https://www.postman.com/downloads/) 向日志服务发起gRPC请求进行查询.\n\n下载 [Offset Explorer](https://www.kafkatool.com/download.html) 向 Topic ```logger.sensor.ts``` 发送JSON数据:\n\n```json\n[{\"ts\": 1646409307, \"sensor_id\": 1, \"temperature\":30, \"cpu\":20}]\n```\n\n## 参考资料\n\n- [淺談 CQRS 的實現方法](https://medium.brobridge.com/%E6%B7%BA%E8%AB%87-cqrs-%E5%AF%A6%E7%8F%BE%E6%96%B9%E6%B3%95-3b4fcb8d5c86)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftx7do%2Fkratos-cqrs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftx7do%2Fkratos-cqrs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftx7do%2Fkratos-cqrs/lists"}