{"id":28995313,"url":"https://github.com/jonefeewang/stonemq","last_synced_at":"2025-06-25T05:02:41.171Z","repository":{"id":234742465,"uuid":"736642928","full_name":"jonefeewang/stonemq","owner":"jonefeewang","description":"A high-performance and efficient message queue developed   in Rust","archived":false,"fork":false,"pushed_at":"2025-02-19T01:14:27.000Z","size":14,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-19T01:31:49.236Z","etag":null,"topics":["message-queue-rust-kafka-pulsar-stonemq"],"latest_commit_sha":null,"homepage":"https://wangjunfei.com/2025/02/10/Announcing-Stonemq-A-high-performance-and-efficient-message-queue-developed-in-Rust/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonefeewang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-28T13:13:38.000Z","updated_at":"2025-02-19T01:14:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"bebd9c5f-22c3-451a-be90-d53569bab2c0","html_url":"https://github.com/jonefeewang/stonemq","commit_stats":null,"previous_names":["jonefeewang/stonemq"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jonefeewang/stonemq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonefeewang%2Fstonemq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonefeewang%2Fstonemq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonefeewang%2Fstonemq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonefeewang%2Fstonemq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonefeewang","download_url":"https://codeload.github.com/jonefeewang/stonemq/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonefeewang%2Fstonemq/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261807929,"owners_count":23212683,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["message-queue-rust-kafka-pulsar-stonemq"],"created_at":"2025-06-25T05:01:02.551Z","updated_at":"2025-06-25T05:02:41.161Z","avatar_url":"https://github.com/jonefeewang.png","language":"Rust","readme":"# Stonemq\n\n[![License](https://img.shields.io/badge/license-Apache%202-blue)](LICENSE)\n[![Build Status](https://img.shields.io/github/actions/workflow/status/username/stonemq/ci.yml)](https://github.com/username/stonemq/actions)\n[![Crates.io](https://img.shields.io/crates/v/stonemq)](https://crates.io/crates/stonemq)\n\nStonemq aims to outperform Kafka in scenarios with massive-scale queue clusters, delivering enhanced performance and efficiency to achieve cost reduction and operational optimization.\n\nIn use cases involving large clusters with countless queues—particularly in enterprise business services and public cloud services—there can be tens of thousands of partitions and partition leaders. Regardless of whether the queues contain messages, the volume of messages, or the flow rate, the presence or absence of active consumers in each partition poses a significant burden for cluster operators. Partition growth negatively impacts cluster throughput, while node failures or restarts often result in partition leader and controller switching, creating critical operational challenges. Stonemq addresses these inefficiencies.\n\nCluster performance should remain consistent regardless of partition growth. Queues with varying traffic volumes need consolidation to enable more efficient message flow—akin to containerized shipping for multiple clients. This is precisely the vision of Stonemq. While solutions like Pulsar utilize journaling for centralized message handling, Stonemq seeks to retain Kafka’s replication , which simplifies and standardizes cluster operation and maintenance.We believe this protocol is both straightforward and highly efficient, forming the backbone of our solution.Additionally, by reusing Kafka's client-server communication protocol, StoneMQ ensures seamless migration without requiring any changes to the user's client applications. This approach significantly reduces adoption costs for users, enabling a smooth transition to StoneMQ while retaining the familiar and reliable interface they are accustomed to. \n\nFor a more detailed analysis and introduction to the project's inception, please refer to the blog: [Blog Link](https://wangjunfei.com/2025/02/10/Announcing-Stonemq-A-high-performance-and-efficient-message-queue-developed-in-Rust/).\n\n---\n\n## Current Status\n\n- version 0.1.0 released \n- Supports single-node message sending and receiving.  \n- Implements group consumption functionality.  \n\n---\n\n## Installation and Usage\n\n### Build From Source\n\n1. Make sure you have Rust installed (recommended via [Rustup](https://rustup.rs/)):\n\n   ```bash\n   rustc --version\n   ```\n\n2. Clone the repository:\n\n   ```bash\n   git clone https://github.com/jonefeewang/stonemq.git\n   cd stonemq\n   ```\n\n3. Build the release version using Cargo:\n\n   ```bash\n   cargo build --release\n   ```\n\n4. Run the application:\n\n   ```bash\n   ./target/release/stonemq\n   ```\n\n## **Quick Start**\n\n### **1. Start the StoneMQ Server**  \n\nRun the following command to launch the StoneMQ server:  \n\n```bash\n./stonemq\n```\n\n---\n\n### **2. Connect to StoneMQ**  \n\nUse a **Kafka client library** (or REST API) with a minimum supported version of **Kafka v0.11** to connect to the StoneMQ server for publishing and subscribing to messages. StoneMQ uses the same default server port as Kafka: **9092**. This port can be customized in the configuration file.\n\n---\n\n### **3. Publish a Message**  \n\nStoneMQ comes with a built-in topic, **`topic_a`**, which is pre-configured with **2 partitions**:  \n\n- `topic_a-0`  \n- `topic_a-1`  \n\nBy default, the system sets up with **2 journals**.\n\n---\n\n### **4. Configuration**  \n\nThe configuration file, **`conf.toml`**, is located in the project’s root directory. Adjust settings as needed to customize the server's behavior.\n\n### **5. Server Graceful Shutdown**  \n\nStoneMQ provides a graceful shutdown mechanism that allows the server to complete all pending tasks before exiting. This is particularly useful in scenarios where the server needs to ensure all messages are processed or stored before shutting down.\n\nTo trigger a graceful shutdown, you can send a termination signal to the server. In a Unix-like environment, you can use `Ctrl+C` to send the `SIGINT` signal or use the following command to send the `SIGINT` signal:\n\n```bash\nkill -2 \u003cpid\u003e\n```\n\nThis will allow the server to perform a graceful shutdown, ensuring all pending messages are processed and stored.\n\n\n---\n\n## Development Guide\n\n### Local Development Setup\n\n1. **Install Rust Toolchain**:\n   Ensure Rust is installed on your system. Use [Rustup](https://rustup.rs/) to get started:\n\n   ```bash\n   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n   rustup default stable\n   ```\n\n2. **Clone the Repository**:\n\n   ```bash\n   git clone https://github.com/jonefeewang/stonemq.git\n   cd stonemq\n   ```\n\n3. **Run Tests**:\n   Run all unit tests to ensure everything is working correctly:\n\n   ```bash\n   cargo test\n   ```\n\n4. **Check Code Formatting**:\n   Ensure the code adheres to the formatting rules:\n\n   ```bash\n   cargo fmt -- --check\n   ```\n\n5. **Lint (Static Analysis)**:\n\n   Run clippy to check for potential issues:\n\n   ```bash\n   cargo clippy\n   ```\n\n6. **Log Parsing**  \n\n   An executable log_parser, source file is **`log_parser.rs`**, is available in the project's `bin` directory. This tool can be used to parse `log`,`index`,and `checkpoint` files.\n\n7. **Benchmarking and Monitoring**  \n\n   The entry file **`stonemq.rs`** in the `bin` directory allows you to specify the current running mode: **prod**, **dev**, or **perf**. The default \tmode is **prod**.  \n\n   - **Dev Mode**: Suitable for development.  \n   - **Perf Mode**: Enables the use of **tokio-console** to monitor the state of Tokio tasks within the process.  \n     \n   Additionally, you can specify whether to enable **trace**. When enabled, added spans can be observed in **Open Jaeger** for in-depth monitoring. \n\n   You can also specify the **log output level** through environment variables. The project uses **dotenv** crate to configure environment variables for ease of management and flexibility.\n\n---\n\n## Contribution Guide\n\nContributions are welcome and greatly appreciated! Whether it's reporting a bug, submitting code, improving documentation, or suggesting ideas, your help is valuable.\n\n### Steps to Contribute\n\n1. Fork the repository and create a new branch for your feature/bugfix.\n2. Ensure all tests pass, and your code adheres to the project's standards.\n3. Submit a pull request with a clear description of your changes.\n\n---\n\n## Community Support\n\nIf you encounter issues or have questions, feel free to reach out to the community:\n\n- 🐛 Submit Issues: [GitHub Issues](https://github.com/jonefeewang/stonemq/issues)\n- 📢 Join Discussions: [GitHub Discussions](https://github.com/jonefeewang/stonemq/discussions)\n\n---\n\n## License\n\nStonemq is distributed under the **Apache License 2.0**. See the [LICENSE](./LICENSE) file for more details.\n\n","funding_links":[],"categories":["Rust","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonefeewang%2Fstonemq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonefeewang%2Fstonemq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonefeewang%2Fstonemq/lists"}