{"id":30141973,"url":"https://github.com/solzimer/fileq","last_synced_at":"2026-03-10T04:34:03.682Z","repository":{"id":57235814,"uuid":"92421214","full_name":"solzimer/fileq","owner":"solzimer","description":"File based FIFO queue. High-performance queue that stores JSON objects in a file-based FIFO","archived":false,"fork":false,"pushed_at":"2019-09-17T09:12:39.000Z","size":88,"stargazers_count":4,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-09T06:23:32.827Z","etag":null,"topics":["buffer","cache","fifo-queue","file","javascript","json","nodejs","queue"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/solzimer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-25T16:11:01.000Z","updated_at":"2022-08-05T11:24:28.000Z","dependencies_parsed_at":"2022-08-23T15:50:55.172Z","dependency_job_id":null,"html_url":"https://github.com/solzimer/fileq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/solzimer/fileq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solzimer%2Ffileq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solzimer%2Ffileq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solzimer%2Ffileq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solzimer%2Ffileq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/solzimer","download_url":"https://codeload.github.com/solzimer/fileq/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solzimer%2Ffileq/sbom","scorecard":{"id":837359,"data":{"date":"2025-08-11","repo":{"name":"github.com/solzimer/fileq","commit":"00d7361477f4a7111e0b5ea485a9d5e3b2d544a3"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-23T19:28:06.811Z","repository_id":57235814,"created_at":"2025-08-23T19:28:06.811Z","updated_at":"2025-08-23T19:28:06.811Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30324450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T01:36:58.598Z","status":"online","status_checked_at":"2026-03-10T02:00:06.579Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buffer","cache","fifo-queue","file","javascript","json","nodejs","queue"],"created_at":"2025-08-11T05:21:21.111Z","updated_at":"2026-03-10T04:34:03.660Z","avatar_url":"https://github.com/solzimer.png","language":"JavaScript","readme":"# fileq\n\nFile based FIFO queue.\nHigh-performance queue that stores JSON objects in a file-based FIFO, so the reads and writes are independent, allowing them to each have their own rhythm without increasing the memory usage.\n\n## Features\n* Multiple writers and readers on the same queue\n* Callback and promise modes\n* Can recover previous queue if process is restarted\n* Recover queue position on process restart\n* Persitent or truncate modes on process restart\n* Fault tolerant, and fine-tunning\n\n## Installation\n```\nnpm install fileq\n```\n\n## Usage\n```javascript\nconst FileQueue = require(\"fileq\");\n\n// Each queue stores its files in a folder\nvar queue = FileQueue.from(\"queue\");\nvar i=0;\n\n// Callback mode\nsetInterval(()=\u003e{\n\tqueue.push({key:i, message:\"This is the entry for \"+i});\n\ti++;\n},100);\n\nsetInterval(()=\u003e{\n\tqueue.peek((err,entry)=\u003e{\n\t\tconsole.log(entry);\n\t});\n},100);\n\n// Promise mode\nsetInterval(async ()=\u003e{\n\tawait queue.push({key:i, message:\"This is the entry for \"+(i++)});\n},100);\n\nsetInterval(async ()=\u003e{\n\tlet item = await queue.peek();\n\tconsole.log(item);\n},100);\n\n```\n\n## API\n### FileQueue.from([path],[options]) =\u003e FileQueue\nRetrieve a queue stored in *path* folder. If the queue doesn't exist, it is\ncreated. The *options* parameter will be described later. If path is not\nspecified, an *anonymous* queue will be created in the base path defined in\nthe base options.\n\n### FileQueue.configure(options)\nSets default options that will be passed to every new created queue\n\n### queue.push(json,callback) =\u003e Promise\nPushes a JSON object to the queue. Callback takes the typical *err* and\n*result* arguments. If no callback is provided, it returns a promise.\n\n### queue.peek(time,callback,commit) =\u003e Promise\nRetrieves a JSON object from the queue, in a FIFO manner. Callback takes the\nusual *err* and *result* arguments. If no callback is provided, it returns a\npromise. The argument **time** specifies a wait for data timeout. If no data\ncould be read before **time**, then callback is called with \"timeout\" error\n(promise is rejected). The *commit* parameter specifies a transactional requirement.\nWhen *commit* is *true*, callback function receives a third argument (function *done*)\nthat must be called in order to remove the item from the queue.\nCommit mode only works when callback function is passed:\n\n```javascript\n\t// Promise mode\n\t// Waits forever for an entry. Entry is returned and removed from queue\n\tlet item = await queue.peek();\n\t// Waits 100 ms for an entry. Entry is returned and removed from queue, or timeout error\n\tlet item = await queue.peek(10);\n\n\t// Callback mode\n\t// Waits forever for an entry. Entry is returned and removed from queue\n\tqueue.peek((err,item)=\u003e{\n\t\tconsole.log(item);\n\t});\n\t// Waits 100 ms for an entry. Entry is returned and removed from queue, or timeout error\n\tqueue.peek((err,item)=\u003e{\n\t\tif(err) console.error(err);\n\t\telse console.log(entry);\n\t},100);\n\t// Waits forever for an entry. Entry is returned but not remove until done is called\n\tqueue.peek((err,item,done)=\u003e{\n\t\tlet error = doSomething(item);\n\t\tif(error) done(error); // If done is called with arguments, item is not removed\n\t\telse done();\t// done called without arguments remove the item from the queue\n\t},100,true);\n```\n**Important:** If commit mode is used, no more reads will be done until *done*\nhas been called (queue will block further reads to avoid inconsistency):\n```javascript\n\tqueue.peek((err,item,done)=\u003e{\n\t\tconsole.log(item);\n\t\tsetTimeout(done,1000);\t// Queue will be locked 1 sec\n\t},0,true);\n\n\t// Cannot retrieve next item until previous call ends\n\tqueue.peek((err,item)=\u003e{\n\t\tconsole.log(item);\n\t});\n```\n\n### queue.poll(time,callback) =\u003e Promise\nThe same as **queue.peek** but without the *commit* feature\n\n### queue.head(time,callback) =\u003e Promise\nRetrieves the head of the queue, without removing the element, as\noposed to **peek** and **poll**\n\n### queue.lock(callback) =\u003e Promise\nLocks the queue so no other callers can read from it until **queue.unlock**\nis called. Note that this is a soft lock (other readers can ignore the lock). The only time where a lock cannot be ignored if is **queue.peek** is called with *commit* feature (It's a different hard lock):\n\n```javascript\n\t// Locks the queue for 100 reads\n\tasync function reader1() {\n\t\tawait queue.lock();\n\t\tfor(let i=0;i\u003c100;i++) {\n\t\t\tlet item = await queue.poll();\n\t\t}\n\t\tqueue.unlock();\n\t}\n\n\t// Same as reader1: If reader1 has the lock, reader2 must wait\n\tasync function reader2() {\n\t\tawait queue.lock();\n\t\tfor(let i=0;i\u003c10;i++) {\n\t\t\tlet item = await queue.poll();\n\t\t}\n\t\tqueue.unlock();\n\t}\n\n\t// reader3 doesn't ask for lock, so it can read without waiting\n\tasync function reader3() {\n\t\tfor(let i=0;i\u003c100;i++) {\n\t\t\tlet item = await queue.poll();\n\t\t}\n\t}\n\n\t// reader4 doesn't ask for lock, but uses commit feature, so nobody\n\t// can read until commit is applied\n\tasync function reader4() {\n\t\tfor(let i=0;i\u003c10;i++) {\n\t\t\tqueue.peek((err,item,done)=\u003e{\n\t\t\t\tsetTimeout(done,1000);\n\t\t\t});\n\t\t}\n\t}\n```\n\n### queue.unlock()\nUnlocks queue reads\n\n### queue.close() =\u003e Promise\nCloses de queue\n\n### queue.locked =\u003e Boolean\nReturns *true* if queue has a virtual lock; *false* otherwise.\n\n### queue.closed =\u003e Boolean\nReturns *true* if the queue has been closed\n\n## Options\nWhen creating a queue, data are stored in several files in a folder.\n\nThe *options* object allows us to *fine-tune* the queue files to better match\nthe needs of our process:\n* **truncate** : If *true*, previous queue status is reset, and a new empty\nqueue is created. If *false*, a previously created queue is recovered. By\ndefault is set to *false*.\n* **path** : Base folder to store *anonymous* queues when the path is not\nspecified. By default, the base path is the os temporal folder.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolzimer%2Ffileq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolzimer%2Ffileq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolzimer%2Ffileq/lists"}