{"id":15646233,"url":"https://github.com/threez/file-queue","last_synced_at":"2025-10-06T06:09:24.668Z","repository":{"id":11625292,"uuid":"14124731","full_name":"threez/file-queue","owner":"threez","description":"File system based queue for node.js","archived":false,"fork":false,"pushed_at":"2016-11-17T22:03:30.000Z","size":75,"stargazers_count":51,"open_issues_count":3,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-28T15:58:37.444Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/threez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-04T21:55:51.000Z","updated_at":"2025-04-05T07:55:29.000Z","dependencies_parsed_at":"2022-08-07T06:16:27.910Z","dependency_job_id":null,"html_url":"https://github.com/threez/file-queue","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threez%2Ffile-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threez%2Ffile-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threez%2Ffile-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threez%2Ffile-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/threez","download_url":"https://codeload.github.com/threez/file-queue/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251342720,"owners_count":21574244,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T12:11:56.706Z","updated_at":"2025-10-06T06:09:24.583Z","avatar_url":"https://github.com/threez.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File-Queue\n\nA simple file system based queuing implementation. Based on the maildir format,\nthe queues are lockfree and therefore work in most situatuions. Look\n[here](http://cr.yp.to/proto/maildir.html) for the maildir specification.\nThe queue organises everything in FIFO (first-in first-out) order, because of\nthat it is not possible to get the messages of the queue with random access.\nAll objects are encoded using json and therefore can be easily read manually.\n\nThis queuing library has a specialty, its supports transactions for popping items.\n\n## Create a queue\n\nThe queue is created based on a folder. The passed folder will contain have\nthree directories `cur`, `tmp` and `new`. The names of the files follow the \nmaildir conventions.\n\n    var Queue = require('file-queue').Queue,\n        queue = new Queue('.', callback);\n\n### Dealing with many files and common filesystem errors\n\nIf you deal with lots of files EMFILE errors (too many open files errors)\ncan occur. Issacs wrote the `graceful-fs` package to deal with these errors.\nTo use it simply pass the filesystem library that you prefer:\n\n    var queue = new Queue({\n        path: 'tmp',\n        fs: require('graceful-fs')\n    }, done);\n\n## Pushing and popping messages from the queue\n\nPopping a message can be done at any time. If the queue doesn't contain an item at the\nmoment it 'blocks' until it does. If there was an error, while removing the\nmessage `err` will contain an error message.\n\n    queue.pop(function(err, message) {\n      if (err) throw err;\n      console.log(message);\n    });\n\nPushing an item into the queue could cause an error if e.g. no disk space is\nleft on the device.\n\n    queue.push('Hello World', function(err) {\n      if (err) throw err;\n    });\n\n## Getting the length of the queue\n\nThe queue length can easily be determined with the following call:\n\n    queue.length(function(err, length) {\n      console.log(length);\n    });\n\n## Transactional popping\n\nA transactional pop means, that the element is taken from the queue, but will\nnot be removed until commit is called. The rollback action makes the item again\navailable for popping.\n\n    queue.tpop(function(err, message, commit, rollback) {\n      if (Processor.process(message) === true) {\n        commit(function(err) { if (err) throw err; });\n      } else {\n        rollback(function(err) { if (err) throw err; });\n      }\n    });\n\nThere can be multiple layers of transactions. Since transactional pops (tpops) don't block the popping in general, there can be multiple inside of each other.\nThe downside is that it can't be assured that the messages are processed in order.\n\n## Clearing\n\nTo remove all items from a queue, for example for testing purposes, use clear.\n\n    queue.clear(function(err) { if (err) throw err; });\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreez%2Ffile-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreez%2Ffile-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreez%2Ffile-queue/lists"}