{"id":13629420,"url":"https://github.com/Qihoo360/huststore","last_synced_at":"2025-04-17T09:34:09.805Z","repository":{"id":151565190,"uuid":"57210915","full_name":"Qihoo360/huststore","owner":"Qihoo360","description":"High-performance Distributed Storage","archived":false,"fork":false,"pushed_at":"2018-06-11T08:08:41.000Z","size":31900,"stargazers_count":826,"open_issues_count":1,"forks_count":177,"subscribers_count":138,"default_branch":"master","last_synced_at":"2024-08-01T22:43:41.291Z","etag":null,"topics":["architecture","benchmark","database-engine","distributed-storage","message-queue","nginx","redis"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Qihoo360.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-04-27T12:16:12.000Z","updated_at":"2024-06-14T02:07:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"46bdbbf3-119e-44fa-9621-20837391195c","html_url":"https://github.com/Qihoo360/huststore","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2Fhuststore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2Fhuststore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2Fhuststore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2Fhuststore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Qihoo360","download_url":"https://codeload.github.com/Qihoo360/huststore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223751137,"owners_count":17196579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["architecture","benchmark","database-engine","distributed-storage","message-queue","nginx","redis"],"created_at":"2024-08-01T22:01:10.158Z","updated_at":"2024-11-08T20:31:02.261Z","avatar_url":"https://github.com/Qihoo360.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"[简体中文](README_ZH.md)\n\n# huststore - High-performance Distributed Storage #\n![huststore logo](res/logo.png)\n\n`huststore` is a open source high performance distributed database system. It not only provides key-value storage service with extremely high performance, up to **hundreds of thousands QPS**, but also supports data structures like `hash`, `set`, `sorted set`, etc. Also, it can store **binary** data as value from a key-value pair, and thus can be used as an alternative of Redis.\n\nIn addtion, `huststore` implements a distributed message queue by integrating a special `HA` module, features including message Push Stream, and message Publish-SubScribe.\n\n## Features ##\n`huststore` has two core components, `hustdb` and `HA`. `hustdb` is a database engine developed by our own, in the fundamental architecture. `HA` is implemented as a `nginx` module. It is well-known that `nginx` is a industry-proven high quality code base, thus by inheriting it `huststore` gains the below advantages:\n\n* High Throughput  \n`hustdb` uses [libevhtp](https://github.com/ellzey/libevhtp), a open source network library, as the inner network communication system, by incorporating it with high-performance storage engine, `hustdb` achieves a extremely high performance, the benchmark shows that `QPS` hits **hundreds of thousands** and even more.\n\n* High Concurrency  \nPlease refer to concurrency report of `nginx` for more details.\n\n* High Availability  \n`huststore` architecture provides `Replication` (master-master) and `load balance` support. Therefore, the availability of `HA` is guaranteed by `master-worker` design. When one of `worker` process is down, the `master` will load another `workder` process, since multiple `worker`s work independently, the `HA` is guaranteed to work steadily.\nThe fundamental design architecture of `huststore` guarantees the high availability, by using `master-master` architecture, when one of the storage node fails, `HA` module will re-direct the request to another living `master` node. Also, when a node failure happens, `HA` cluster will automatically re-balance the data distribution, thus avoid single point of failure.\nIn addition, `HA` cluster uses a distributed architecture design by incorporating LVS as the director, each `HA` node is separated and work independently. When one of the `HA` node is down, `LVS` will re-direct the request to other available `HA` node, thus avoids `HA`'s failure on single point node.\n\n* Language-free Interface  \n`huststore` use `http` as the communication protocol, therefore the client side implementation is not limited in any specific programming language.\n\n* Persistence  \n**You do not need to worry about the loss of data** as most of interfaces will persist data to disk.  \n\n* Support Binary Key-Value  \n* Support Version Clock\n\n## Operation and Maintenance ##\n\n### Architect ###\n![architect](res/architect.png)\n\n### Deployment ###\n* Distributed KV storage  : HA (hustdb ha) + DB (hustdb)\n* Distributed Message Queue  : HA (hustmq ha) + DB (hustdb)\n\n## Database Engine ##\n![hustdb](res/hustdb.png)\n\n## Dependency ##\n* [cmake](https://cmake.org/download/)\n* [leveldb](https://github.com/google/leveldb)\n* [libcurl](https://curl.haxx.se/libcurl/)\n* [libevent2](http://libevent.org/)\n* [libevhtp](https://github.com/ellzey/libevhtp)\n* [zlog](https://github.com/HardySimpson/zlog)\n* [zlib](https://zlib.net/)\n\n## Platforms ##\n\nTested platforms so far:\n\nPlatform         | Description\n-----------------|----------------------------------------------------------\nCentOS 6.x \u0026 7.x | kernel \u003e= 2.6.32 (GCC 4.4.7)\n\n## Quick Start ##\n\nRead the [Quick Start](quickstart.md).\n\n## Documents ##\n\n* [hustdb](hustdb/doc/doc/en/index.md)\n* [hustmq](hustmq/doc/doc/en/index.md)\n\nAbove includes detailed documents of design, deployments, `API` usage and test samples. You can refer quickly to common problems in `FAQ` part.\n\n## Performance ##\n\n### Environment ###\n\n    CPU: Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz (2socket*16cores)\n    Memory: 192G\n    Disk: Intel SSD DC S3520 Series (800GB, 2.5in SATA 6Gb/s, 3D1, MLC)\n    Network Adapter: Intel Ethernet 10G 2P X520 Adapter\n    OS: CentOS Linux release 7.2.1511 (3.10.0-327.el7.x86_64)\n\n### Test 1 -  Worker ###\n\n#### Purpose ####\n\nWith different number of hustdb worker threads, we test hustdb's max QPS.\n\n#### Condition ####\n\nstorage capacity : 512GB  \n\nconcurrent connection : 1000\n\nvalue : **1KB**\n\nmd5db cache : **disabled**  \n\ndata compression : **disabled**  \n\nThreads are not bound to CPU cores.\n\n#### Result ####\n\nDescription : Horizontal axis is the number of worker threads; Vertical axis is the QPS. The value size is **1KB** .\n\n![benchmark_workers](res/benchmark_workers.png)\n\n#### Conclusion ####\n\nThe best hustdb worker threads number is 36-40.  \n\n### Test 2 -  RTT ###\n\n#### Purpose ####\n\nWith the optimal worker threads number, we test hustdb's round-trip time.  \n\n#### Condition ####\n\nstorage capacity : 512GB  \n\nconcurrent connection : 200\n\nvalue : **1KB**\n\nmd5db cache : **disabled**  \n\ndata compression : **disabled**  \n\nThreads are not bound to CPU cores.\n\n#### Result ####\n\n    # GET\n    Thread Stats   Avg      Stdev     Max   +/- Stdev\n        Latency   238.94us   95.42us   9.33ms   81.03%\n        Req/Sec    31.44k     1.09k   36.30k    63.68%\n    Latency Distribution\n        50%  220.00us\n        75%  288.00us\n        90%  343.00us\n        99%  482.00us\n    11333230 requests in 15.10s, 11.10GB read\n    Requests/sec: 750635.81\n    Transfer/sec:    752.65MB\n    --------------------------------------------------\n    [Latency Distribution]  0.01%  0.07ms\n    [Latency Distribution]   0.1%  0.07ms\n    [Latency Distribution]   0.5%  0.09ms\n    [Latency Distribution]     1%  0.09ms\n    [Latency Distribution]     3%  0.12ms\n    [Latency Distribution]     5%  0.13ms\n    [Latency Distribution]    10%  0.15ms\n    [Latency Distribution]    20%  0.17ms\n    [Latency Distribution]    30%  0.19ms\n    [Latency Distribution]    40%  0.20ms\n    [Latency Distribution]    50%  0.22ms\n    [Latency Distribution]    60%  0.25ms\n    [Latency Distribution]    70%  0.28ms\n    [Latency Distribution]    80%  0.30ms\n    [Latency Distribution]    90%  0.34ms\n    [Latency Distribution]    91%  0.35ms\n    [Latency Distribution]    92%  0.36ms\n    [Latency Distribution]    93%  0.37ms\n    [Latency Distribution]  93.5%  0.37ms\n    [Latency Distribution]    94%  0.38ms\n    [Latency Distribution]  94.5%  0.38ms\n    [Latency Distribution]    95%  0.39ms\n    [Latency Distribution]  95.5%  0.39ms\n    [Latency Distribution]    96%  0.40ms\n    [Latency Distribution]  96.5%  0.40ms\n    [Latency Distribution]    97%  0.41ms\n    [Latency Distribution]  97.5%  0.42ms\n    [Latency Distribution]    98%  0.43ms\n    [Latency Distribution]  98.5%  0.45ms\n    [Latency Distribution]    99%  0.48ms\n    [Latency Distribution]  99.1%  0.49ms\n    [Latency Distribution]  99.2%  0.50ms\n    [Latency Distribution]  99.3%  0.51ms\n    [Latency Distribution]  99.4%  0.52ms\n    [Latency Distribution]  99.5%  0.53ms\n    [Latency Distribution]  99.6%  0.56ms\n    [Latency Distribution]  99.7%  0.59ms\n    [Latency Distribution]  99.8%  0.64ms\n    [Latency Distribution]  99.9%  0.76ms\n    [Latency Distribution]  99.99%  1.85ms\n    [Latency Distribution]  99.999%  4.07ms\n\n    # PUT\n    Thread Stats   Avg      Stdev     Max   +/- Stdev\n        Latency   495.13us  393.71us  21.29ms   93.06%\n        Req/Sec    16.37k     1.33k   23.72k    74.26%\n    Latency Distribution\n        50%  447.00us\n        75%  623.00us\n        90%  815.00us\n        99%    1.28ms\n    17628712 requests in 45.10s, 1.53GB read\n    Requests/sec: 390880.11\n    Transfer/sec:     34.67MB\n    --------------------------------------------------\n    [Latency Distribution]  0.01%  0.09ms\n    [Latency Distribution]   0.1%  0.10ms\n    [Latency Distribution]   0.5%  0.12ms\n    [Latency Distribution]     1%  0.12ms\n    [Latency Distribution]     3%  0.14ms\n    [Latency Distribution]     5%  0.17ms\n    [Latency Distribution]    10%  0.20ms\n    [Latency Distribution]    20%  0.26ms\n    [Latency Distribution]    30%  0.33ms\n    [Latency Distribution]    40%  0.39ms\n    [Latency Distribution]    50%  0.45ms\n    [Latency Distribution]    60%  0.51ms\n    [Latency Distribution]    70%  0.58ms\n    [Latency Distribution]    80%  0.67ms\n    [Latency Distribution]    90%  0.81ms\n    [Latency Distribution]    91%  0.84ms\n    [Latency Distribution]    92%  0.86ms\n    [Latency Distribution]    93%  0.89ms\n    [Latency Distribution]  93.5%  0.90ms\n    [Latency Distribution]    94%  0.92ms\n    [Latency Distribution]  94.5%  0.93ms\n    [Latency Distribution]    95%  0.95ms\n    [Latency Distribution]  95.5%  0.97ms\n    [Latency Distribution]    96%  0.99ms\n    [Latency Distribution]  96.5%  1.02ms\n    [Latency Distribution]    97%  1.05ms\n    [Latency Distribution]  97.5%  1.08ms\n    [Latency Distribution]    98%  1.13ms\n    [Latency Distribution]  98.5%  1.19ms\n    [Latency Distribution]    99%  1.28ms\n    [Latency Distribution]  99.1%  1.30ms\n    [Latency Distribution]  99.2%  1.33ms\n    [Latency Distribution]  99.3%  1.37ms\n    [Latency Distribution]  99.4%  1.41ms\n    [Latency Distribution]  99.5%  1.47ms\n    [Latency Distribution]  99.6%  1.56ms\n    [Latency Distribution]  99.7%  1.73ms\n    [Latency Distribution]  99.8%  2.24ms\n    [Latency Distribution]  99.9%  4.23ms\n    [Latency Distribution]  99.99%  7.22ms\n    [Latency Distribution]  99.999%  9.62ms\n\n\n### Test 3 -  vs Redis ###\n\n#### Version ####\n\n* [redis 4.0.9](https://redis.io/)\n\n#### Tools ####\n\n* [redis-benchmark](https://redis.io/topics/benchmarks)\n* [wrk](https://github.com/wg/wrk)\n\n#### Condition ####\n\nstorage capacity : 512GB  \n\nmd5db cache : **disabled**  \n\ndata compression : **disabled**  \n\nThreads are not bound to CPU cores.\n\n#### Arguments ####\n\nabbr       |concurrent connection |value\n-----------|----------------------|--------------\nC1000-512B |1000                  |512B\nC1000-1K   |1000                  |1KB\nC1000-4K   |1000                  |4KB\nC2000-512B |2000                  |512B\nC2000-1K   |2000                  |1KB\nC2000-4K   |2000                  |4KB\n\n#### PUT ####\n\n![benchmark_put](res/benchmark_put.png)\n\n#### GET ####\n\n![benchmark_get](res/benchmark_get.png)\n\nSee more details in [here](benchmark/README.md)\n\n## LICENSE ##\n\n`huststore` is licensed under [LGPL-3.0](https://www.gnu.org/licenses/lgpl-3.0.en.html), a very flexible license to use.\n\n## Authors ##\n\n* XuRuibo（hustxrb, hustxrb@163.com)  \n* ChengZhuo（jobs, yao050421103@163.com)  \n\n## More ##\n\n- Nginx module development kit - [hustngx](https://github.com/jobs-github/hustngx)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQihoo360%2Fhuststore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FQihoo360%2Fhuststore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQihoo360%2Fhuststore/lists"}