https://github.com/didi/DDMQ
DDMQ is a distributed messaging product with low latency, high throughput and high availability.
https://github.com/didi/DDMQ
Last synced: 6 days ago
JSON representation
DDMQ is a distributed messaging product with low latency, high throughput and high availability.
- Host: GitHub
- URL: https://github.com/didi/DDMQ
- Owner: didi
- License: apache-2.0
- Created: 2018-12-27T03:51:21.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-10-09T06:49:51.000Z (over 1 year ago)
- Last Synced: 2024-10-29T17:55:55.469Z (6 months ago)
- Language: Java
- Homepage:
- Size: 17.6 MB
- Stars: 1,303
- Watchers: 80
- Forks: 338
- Open Issues: 24
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-message-queue - DDMQ - commit/didi/DDMQ) |  |  | (Message Queue Middlewares)
README
**English** | [中文](./README_CN.md)
**DDMQ** is a distributed messaging product built by DiDi Infrastructure Team based on [Apache RocketMQ](https://rocketmq.apache.org/). As a distributed messaging middleware, DDMQ provides low latency, high throughput and high available messaging service to many important large-scale distributed systems inside [DiDi](https://www.didiglobal.com/). DDMQ provides realtime messaging, delay-time messaging and transactional messaging to satisfy different scenarios. Through an easy-to-use Web Console and simple SDK Client, developers can experience producing and consuming messages in the most simple and stable way.
----------
### Features
* Messaging model: support both P2P and Pub/Sub messaging model
* Massive message storage, support both RocketMQ and Kafka as storage engine
* Low latency & High throughput
* Delay message, use RocksDB as storage engine
* Transactional message: provide transaction similar to X/Open XA
* Multiple language client SDK: provide client SDK in PHP, Java, Go, C/C++, Python
* Message transition and filter with user-defined Groovy script
* An easy-to-use Web Console
----------
### Architecture Overview
----------
### Modules
* carrera-common: common code for other modules, such as encapsulate zk operations.
* carrera-producer: message producer proxy with built-in Thrift Server, responsible for forwarding message from SDK client to broker.
* carrera-consumer: message consumer proxy with built-in Thrift Server, provide SDK PULL and HTTP PUSH for message consumption.
* carrera-chronos: delay message module, use RocksDB as storage engine.
* carrera-sdk: producer and consumer SDK, support Java/C/C++/Go/PHP/Python.
* rocketmq: based on RocketMQ (Ver 4.2.0)，add new features such as broker auto fail-over.
* carrera-console: a Spring-based User Web Console.
* carrera-monitor: consumer lag monitor and DDMQ cluster monitor.
* carrera-docker: provide a DDMQ docker image that runs in standalone mode.
----------
### Quick Start
We provide a standalone version of DDMQ as Docker image, read [this](carrera-docker/README.md) for more information.
----------
### Deployment
#### Dependencies
* 64bit OS, Linux/Unix/Mac
* 64bit JDK 1.8+
* Maven 3.2.x
* MySQL 5.7.x
* Tomcat 7/8/9
* Zookeeper 3.4.x
#### Deployment Procedures
* Deploy MySQL & Zookeeper
> Install MySQL 5.7:
> Install Zookeeper 3.4.x:
* Deploy RocketMQ
>Guide: [RocketMQ](rocketmq/README.md)
* Init MySQL Tables
>Guide: [MySQL](carrera-console/README.md)
* Deploy Console
>Guide: [Console](carrera-console/README.md)
* Init Zookeeper Node and Data
> call console api：
> * curl http://console_addr:8080/carrera/api/odin/internal/v4/initZkPath
> * curl http://console_addr:8080/carrera/api/odin/internal/v4/initAllZk
* Deploy Producer Proxy
>Guide: [Deploy PProxy](carrera-producer/README.md)
* Deploy Consumer Proxy
>Guide: [Deploy CProxy](carrera-consumer/README.md)
* Deploy Chronos
>Guide: [Deploy Chronos](carrera-chronos/README.md)
----------
### Usage
* read [DDMQ Console Manual](carrera-console/USAGE.md) for more information.
----------
### Contributing
Welcome to contribute by creating issues or sending pull requests. See [Contributing Guide](CONTRIBUTING.md) for guidelines.
----------
### Contact
Feel free to contact DDMQ developers via
----------
### License
DDMQ is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file.