{"id":21296937,"url":"https://github.com/reddec/iot_queue","last_synced_at":"2025-03-15T17:27:43.427Z","repository":{"id":81702518,"uuid":"199586497","full_name":"reddec/iot_queue","owner":"reddec","description":"Lightweight library and tools to build append-only queues","archived":false,"fork":false,"pushed_at":"2019-07-31T05:31:11.000Z","size":22,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-09T00:06:39.005Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reddec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-30T06:11:08.000Z","updated_at":"2019-07-31T05:31:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"ba4a0852-07a0-4df5-8c2b-a89e2592bb4b","html_url":"https://github.com/reddec/iot_queue","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reddec%2Fiot_queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reddec%2Fiot_queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reddec%2Fiot_queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reddec%2Fiot_queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reddec","download_url":"https://codeload.github.com/reddec/iot_queue/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243765044,"owners_count":20344525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-21T14:31:11.404Z","updated_at":"2025-03-15T17:27:43.392Z","avatar_url":"https://github.com/reddec.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IOT-queue\n\nSimple, lightweight and easy-to-use set of libraries and tools to make queues.\n\nArchitecture design principles (ADP):\n\n1. SHOULD be capable to handle extreme amount of messages (billions)\n2. Single message always can fit into memory (it means that messages usually relatively small)\n3. SHOULD be reliable: if procedure (append) successfully completes the data and the index already persisted\n4. SHOULD be consistent and handle situations like:\n   a. power off/system corruption between appends\n   b. power off/system corruption during writing data\n   c. power off/system corruption during writing index\n5. SHOULD be fast enough but speed is not a main concern (expected performance should be about: 300-500 messages/second)\n6. SHOULD be append-only queue. Truncating is optional and supposed to be an offline process (still not yet clean)\n7. Fetch newest messages SHOULD be faster than fetch historical messages\n\nCoding design principles (CDP):\n\n1. No memory allocations. At all. All buffers should be stack based or provided by external code\n2. No \"destructors\": all resource allocation should be done by external code (like file descriptors)\n3. Zero (or positive in case of buffer operations) return code means success, negative - error\n4. Should be easy-to-use interfaces \"by default\" with advanced options\n\n\n## iot_queued\n\nSimple daemon for single queue with ZMQ interface.\n\nRuntime flags:\n\n```\n-C, --cache-size \u003cuint\u003e [default: 65536] size of cached index chunks\n-L, --line-size \u003cuint\u003e  [default: 8192] maximum size of line (incoming message)\n-i, --index-file \u003cpath\u003e [default: index.bin] path to index file\n-d, --data-file \u003cpath\u003e  [default: data.bin] path to data file\n-b, --bind \u003czmq\u003e        [default: tcp://*:9888] URL to bind API (rep) socket\n-h, --help      show this help\n```\n\nRequirements:\n\n* zeromq\n* iot_queue\n\n### API\n\n#### PUSH\n\nAppend multiple messages to the queue.\n\n* **Input**: \u003cPUSH:string\u003e, [\u003cMSG:bytes\u003e, ... ]\n* **Output**: \u003cOK:string\u003e | \u003cFAIL:string\u003e\n\n#### FETCH\n\nFetch messages from the queue starting from `OFFSET` with `LIMIT`.\n\n* **Input**: \u003cFETCH:string\u003e, \u003cOFFSET:string\u003e, \u003cLIMIT:string\u003e\n* **Output**: \u003cOK:string\u003e, [\u003cMSG:bytes\u003e, ... ] | \u003cFAIL:string\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freddec%2Fiot_queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freddec%2Fiot_queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freddec%2Fiot_queue/lists"}