{"id":13619790,"url":"https://github.com/YoungPioneers/mgq","last_synced_at":"2025-04-14T18:32:26.489Z","repository":{"id":90956143,"uuid":"53328456","full_name":"YoungPioneers/mgq","owner":"YoungPioneers","description":"memcached go queue","archived":false,"fork":false,"pushed_at":"2016-09-26T02:55:36.000Z","size":39,"stargazers_count":81,"open_issues_count":0,"forks_count":20,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-11-08T06:38:22.278Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YoungPioneers.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-07T13:41:11.000Z","updated_at":"2023-07-23T07:12:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"1e4e0ffe-0f32-4c21-adf5-60b9cd2ce8e7","html_url":"https://github.com/YoungPioneers/mgq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YoungPioneers%2Fmgq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YoungPioneers%2Fmgq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YoungPioneers%2Fmgq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YoungPioneers%2Fmgq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YoungPioneers","download_url":"https://codeload.github.com/YoungPioneers/mgq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248936919,"owners_count":21186124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T21:00:48.731Z","updated_at":"2025-04-14T18:32:26.157Z","avatar_url":"https://github.com/YoungPioneers.png","language":"Go","funding_links":[],"categories":["Message Queue Middlewares"],"sub_categories":[],"readme":"# Introduction\n\nMemcached Go Queue, 简称mgq, 是一个用[Go](https://golang.org)语言写的，基于memcached协议的消息队列。其父亲[mcq](https://github.com/stvchu/memcacheq.git)是最早应用于[weibo](http://weibo.com)的基础消息中间件，有着高性能，解耦的优点，使得其广泛应用于微博\n\n\n\n\n\n# Features\n\n\nmgq是一个基于NOSQL数据库[BerkeleyDB](http://www.oracle.com/technetwork/cn/database/database-technologies/berkeleydb/overview/index.html)写的FIFO消息队列，目前支持的特性如下：\n\n* 一写多读：举个例子，set myqueue 'one message'，只要get的时候，myqueue开头，#分隔，如myqueue#1，多个客户端之间读是彼此独立的，是不受影响的\n* 默认的get是读取队列中未读取的最旧消息\n* 支持getc操作，支持获取队列中某一个cursor位置的数据：举个例子，假设myqueue已经有1000条数据，getc myqueue 99,就可以获取队列当中cursor为99的消息\n* 支持getr操作，支持获取队列中某一个start cursor位置开始，到end的数据：举个例子，假设myqueue已经有20条数据，getr myqueue 1 10 ,就可以获取队列当中cursor为[1-10]的消息 「内测中」\n\n* getn支持timeout机制的阻塞api来获取队列中的最新消息，举个例子：getn queue 10,意味着10s内有数据则立马返回，否则会10s后立马返回数据不存在的错误，默认getn的timeout是0s，永不超时（需要注意的是如果客户端有getn的操作，则set的另一个客户端需要调用setn）\n\n\n# Benchmark\n针对消息的丢失率，做了一下单个set和get的测试，下面的是消息数为30w，50w，100w时候的结果\n\nmessage total:30w\n\n```\nmgq message set total:300000, cost total time:123518222205 ns, 2428 per/s ,fail set total:0\nmgq message get total:300000, cost total time:51103619703 ns, 5870 per/s ,fail get total:0\n```\n\nmessage total:50w\n\n```\nmgq message set total:500000, cost total time:210480212729 ns, 2375 per/s ,fail set total:0\nmgq message get total:500000, cost total time:87694742059 ns, 5701 per/s ,fail get total:0\n```\n\nmessage total:100w\n\n```\nmgq message set total:1000000, cost total time:422339921379 ns, 2367 per/s ,fail set total:0\nmgq message get total:1000000, cost total time:173768683759 ns, 5754 per/s ,fail get total:0\n```\n\n同时简单做了一下压测，下面的结果依次是1，2，3，4个routine，单个set和get的相对平均耗时时间,[Benchmark code](https://github.com/YoungPioneers/mgq/blob/master/benchmark/mgqPerformance_test.go)\n\n```\nBenchmark_MgqMultiSetAndGet-4 \t    2000\t    546617 ns/op (1829 per/s)\nBenchmark_MgqMultiSetAndGet-4 \t    2000\t    583259 ns/op (1714 per/s)\nBenchmark_MgqMultiSetAndGet-4 \t    2000\t    723603 ns/op (1381 per/s)\nBenchmark_MgqMultiSetAndGet-4 \t    2000\t    754741 ns/op (1324 per/s)\n\n```\n\n# Installation\n\n\n* 首先要安装BerkeleyDB，以版本6.1.26为例，下载[db-6.1.26.tar.gz](http://www.oracle.com/technetwork/cn/database/database-technologies/berkeleydb/downloads/index.html),执行\n\n```\ntar -zxvf db-6.1.26.tar.gz\ncd db-6.1.26;dist/configure --includedir=/usr/include --libdir=/usr/lib64/ (64位的机器)\nmake \u0026\u0026 sudo make install\n```\n\n默认安装到/usr/local/BerkeleyDB.6.1/,之后执行\n\n```\ngit clone https://github.com/YoungPioneers/mgq\ncd mgq \u0026\u0026 make\ncd bin \u0026\u0026 ./mgq\n```\n* 默认端口为22201，可通过./mgq -h查看更多帮助\n\n\n\n# Usage\n\n查看队列的统计情况\n\nstats命令\n\n```\ntelnet localhost 22201\nTrying 127.0.0.1...\nConnected to localhost.\nEscape character is '^]'.\nstats\nSTAT q 0/0\nSTAT queue 122389/80000\nEND\n```\n\nset 队列名（q为队列名，按下\\r\\n后输入的'helloworld'为消息内容）\n\n```\nset q\nhelloworld\nSTORED\n```\n\nget 队列名\n\n```\nget q\nVALUE q 0 10\nhelloworld\nEND\n```\n\nsetn 队列名 \n\n```\nsetn q\nhelloworld\nSTORED\n```\n\ngetn 队列名 timeout_second\n\n```\ngetn q 5\nVALUE q 0 10\nhelloworld\nEND\n```\n\ngetc 队列名 cursor\n\n```\ngetc q 9\nVALUE q 0 11\nhelloworld9\nEND\n```\n\ndelete 队列名\n\n```\ndelete queue\nDELETED\n```\n\nclient example in [examples](https://github.com/YoungPioneers/mgq/blob/master/examples)\n\n\n# ToDo\n[todo](https://github.com/YoungPioneers/mgq/blob/master/todo)\n\n# ChangeLog\n[ChangeLog](https://github.com/YoungPioneers/mgq/blob/master/ChangeLog)\n\n# License\nunder the MIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYoungPioneers%2Fmgq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYoungPioneers%2Fmgq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYoungPioneers%2Fmgq/lists"}