{"id":25381945,"url":"https://github.com/pzingg/dirq-r","last_synced_at":"2025-04-09T12:40:43.896Z","repository":{"id":276641920,"uuid":"732218419","full_name":"pzingg/dirq-r","owner":"pzingg","description":"A port of the Perl module Directory::Queue::Simple for the R language","archived":false,"fork":false,"pushed_at":"2023-12-16T00:31:12.000Z","size":10,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T04:15:04.833Z","etag":null,"topics":["fifo-queue","filesystem","queues"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pzingg.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-16T00:30:45.000Z","updated_at":"2023-12-16T00:33:28.000Z","dependencies_parsed_at":"2025-02-09T16:06:49.850Z","dependency_job_id":"3d16d3af-f645-4c14-96a0-3d66ccbf1bdd","html_url":"https://github.com/pzingg/dirq-r","commit_stats":null,"previous_names":["pzingg/dirq-r"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzingg%2Fdirq-r","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzingg%2Fdirq-r/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzingg%2Fdirq-r/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzingg%2Fdirq-r/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pzingg","download_url":"https://codeload.github.com/pzingg/dirq-r/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045230,"owners_count":21038552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fifo-queue","filesystem","queues"],"created_at":"2025-02-15T06:35:50.263Z","updated_at":"2025-04-09T12:40:43.876Z","avatar_url":"https://github.com/pzingg.png","language":"R","readme":"---\noutput:\n  github_document:\n    html_preview: false\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r knitrsetup, echo = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"README-\"\n)\n```\n\n```{r mainexample, echo = FALSE}\nsuppressMessages(suppressWarnings(library(dirq)))\n```\n\n# dirq - A Filesystem Directory Based Queue\n\nA port of Perl module [Directory::Queue::Simple]\n  (http://search.cpan.org/dist/Directory-Queue/)\nThe goal of this package is to offer a simple queue system\nusing the underlying filesystem for storage, security and\nto prevent race conditions via atomic operations.\n\nA simple producer:\n\n```{r producer}\n# Define a path to your queue\n# In production, you wouldn't use a temporary directory!\npath \u003c- tempfile()\n# Create a new queue or recover an existing one\nq \u003c- dirq(path)\ncat(paste(\"top level directory is\", q$path(), \"\\n\"))\n# Add some messages\nfor (i in seq.int(1, 6)) {\n  name \u003c- q$add(list(data = paste(\"element\", i)), format = \"json\")\n  cat(paste(\"added element\", name, \"\\n\"))\n}\n```\n\nA simple consumer:\n\n```{r consumer}\n# Recover the previous queue\nq \u003c- dirq(path)\n# Get the first element in the queue\ncursor \u003c- q$iter_first()\nwhile (!is.null(cursor)) {\n  if (q$lock(cursor)) {\n    cat(paste(\"reading element\", cursor, \"\\n\"))\n    data \u003c- q$get(cursor, format = \"json\")\n    q$remove(cursor)\n    # Or use q$unlock(cursor) to just peek at data\n  }\n  # Get the next element in the queue\n  cursor \u003c- q$iter_next(cursor)\n}\n```\n\n# Similar work\n\n## txtq\n\n[Will Landau](https://github.com/wlandau)'s [`txtq`](https://github.com/wlandau/txtq) is also a pure filesystem-based queue.\n\n## liteq\n\n[Gábor Csárdi](https://github.com/gaborcsardi)'s [`liteq`](https://github.com/r-lib/liteq) package offers essentially the same functionality implemented with SQLite. It has a some additional features, such as the ability to detect crashed workers and re-queue failed messages, but it was in an early stage of development at the time `dirq` was released.\n\n## Other message queues\n\nThere is a plethora of message queues beyond R, most notably [ZeroMQ](https://zeromq.org) and [RabbitMQ](https://www.rabbitmq.com/). In fact, [Jeroen Ooms](https://github.com/jeroen) and [Whit Armstrong](https://github.com/armstrtw) maintain [`rzmq`](https://github.com/ropensci/rzmq), a package to work with [ZeroMQ](https://zeromq.org) from R. Even in this landscape, `dirq` has advantages.\n\n  1. The `dirq` user interface is friendly, and its internals are simple.\n    No prior knowledge of sockets or message-passing is required.\n  2. `dirq` is lightweight, R-focused, and easy to install. It only depends\n    on R and a few packages on [CRAN](https://cran.r-project.org).\n  3. Because `dirq` it is file-based, the queue persists even if your work\n    crashes.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpzingg%2Fdirq-r","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpzingg%2Fdirq-r","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpzingg%2Fdirq-r/lists"}