{"id":15556087,"url":"https://github.com/pupudu/queuep","last_synced_at":"2025-04-23T20:23:15.872Z","repository":{"id":72502926,"uuid":"85397076","full_name":"pupudu/queuep","owner":"pupudu","description":"An intelligent queue for NodeJs backed by Redis for handling a heavy load of data","archived":false,"fork":false,"pushed_at":"2018-05-11T02:21:31.000Z","size":284,"stargazers_count":23,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-18T04:55:35.717Z","etag":null,"topics":["caching","congestion-control","dirty-checkers","event-driven","javascript","loadbalancing","memoization","nodejs","optimization","performance","pubsub","queue"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pupudu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-03-18T11:43:26.000Z","updated_at":"2023-07-07T11:24:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"cb7654a0-f8e5-47f9-ad11-b09e4885d174","html_url":"https://github.com/pupudu/queuep","commit_stats":{"total_commits":61,"total_committers":3,"mean_commits":"20.333333333333332","dds":0.06557377049180324,"last_synced_commit":"48ae28c21b335ce24510daad0c7f313bc9f7a6cc"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pupudu%2Fqueuep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pupudu%2Fqueuep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pupudu%2Fqueuep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pupudu%2Fqueuep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pupudu","download_url":"https://codeload.github.com/pupudu/queuep/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250506868,"owners_count":21441865,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caching","congestion-control","dirty-checkers","event-driven","javascript","loadbalancing","memoization","nodejs","optimization","performance","pubsub","queue"],"created_at":"2024-10-02T15:11:53.630Z","updated_at":"2025-04-23T20:23:15.851Z","avatar_url":"https://github.com/pupudu.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003ca href='http://queuep.netlify.com/'\u003e\u003cimg src='http://i.imgur.com/24TwZl4.png' height='100'\u003e\u003c/a\u003e\n\n[![Build Status](https://travis-ci.org/pupudu/queuep.svg?branch=master)](https://travis-ci.org/pupudu/queuep) \n[![Code Climate](https://codeclimate.com/github/pupudu/queuep/badges/gpa.svg)](https://codeclimate.com/github/pupudu/queuep)\n[![Coverage Status](https://coveralls.io/repos/github/pupudu/queuep/badge.svg?branch=master)](https://coveralls.io/github/pupudu/queuep?branch=master)\n\nPronounced: \"queue-pea\" https://pupudu.gitbooks.io/queuep https://pupudu.gitbooks.io/queuep\n\nAn API which will be consumed by multiple clients will eventually reach a point when the resources\nof the hosting server is insufficient to handle the incoming load. \n\n**QueueP to the Rescue !!**\n\nQueueP can be used in any context where performance is affected by a heavy load of data.\n \n## Demo\nA live demo of how QueueP works can be found at https://queuep-fruit-salad.stackblitz.io\n\nThe source code of the demo can be found at https://stackblitz.com/edit/queuep-fruit-salad\n\nFeel free to edit the demo code in realtime and fork it on Stack Blitz.\n \n## Table of Contents\n\n- [Why QueueP](#why-queuep)\n- [Basic Usage](#basic-usage)\n- [Installation](#installation)\n- [Documentation](#documentation)\n- [Background](#background)\n- [License](#license) \n \n### Why QueueP?\nNone of the similar queue libraries have a concept of avoiding duplicate requests. \nQueueP filters out redundant requests and allows to process useful data.\n \nQueueP internally uses concepts such as memoization, throttling and producer-consumer pattern \nto provide a premium user experience.\n \nQueueP allows you to customize the memoization logic (deciding whether or not to process a data chunk) via the \ndirty checkers. While you can write the dirty checkers all by yourself, QueueP provides several\ndirty checker templates which can be quite handy to use. \n \n### Basic Usage\nLet's assume that multiple devices are sending data about their online status.\n\nFirst, let's create a queue with the minimal required configurations. \n\n```js\nimport qp from 'queuep';\n\nqp.initQueue(\"app_online_status\", {\n    consumer: function(key, data, done) {\n        // Logic to process the online status data goes here\n        \n        // key - to identify the device corresponding to the online status data\n        // data - online status of the corresponding device\n        // done - an error-first callback to signal the queue, that the operation has been completed\n    }\n});\n```\n\nThen we can publish data to the queue.\n\n```js\nqp.publish(\"app_online_status\", deviceId, onlineStatus);\n```\n\n#### Notes about initQueue\n* 1: The first argument is used to identify the queue. An application can have any number of queuep queues.\n\n* 2: The second argument is an options object. \n**consumer** is the only compulsory attribute \u0026 should be a function which accepts 3 arguments. \n\nFirst two arguments will give the **key** and the **data** published from the publish method. \nThe optional 3rd argument is an **error-first** callback that should be used to signal the queue that the consume task \nhas finished.\n\n```js\nfunction consumer(key, data, done) {\n\n    let onlineStatus = data,\n        deviceId = key;\n\n    myDbModule.updateOnlineStatus(deviceId, onlineStatus, function (err) {\n        if (err) {\n            return done(err)\n        }\n        return done();\n    });\n}\n```\n    \nIf the 3rd argument is not provided, then the function should return a promise to signal that the operation has finished.\nIf the actual code which processes the online status returns a promise, then you can return the function call directly. \n\n```js\nfunction consumer(key, data) {\n    return myDbModule.updateOnlineStatus(key, data); // calling updateOnlineStatus should return a promise \n    // key: deviceId, data: onlineStatus\n}\n```\n\n### Installation\nTo install the stable version:\n\n    npm install --save queuep\n\n### Documentation\nWe have started writing a gitbook to give the users a thorough understanding about the framework. \nThe book is still not complete, but do visit  https://pupudu.gitbooks.io/queuep/content/ or http://queuep.netlify.com/  and have a look\nto see where it is heading. We promise to finish it soon. \n\n### Background\nI wrote queuep to fix an issue in a project I was working on. \nThe story in brief and the fundamental advantages of using QueueP can be found at \nhttps://pupudu.gitbooks.io/queuep/content/background.html\n\n### License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpupudu%2Fqueuep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpupudu%2Fqueuep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpupudu%2Fqueuep/lists"}