{"id":32167481,"url":"https://github.com/cantido/queutils","last_synced_at":"2025-10-21T15:32:17.456Z","repository":{"id":41785133,"uuid":"267187197","full_name":"Cantido/queutils","owner":"Cantido","description":"Handy little queues and producers.","archived":true,"fork":false,"pushed_at":"2024-03-06T17:54:44.000Z","size":86,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-07T01:06:05.243Z","etag":null,"topics":["broadway","elixir","hacktoberfest","queue"],"latest_commit_sha":null,"homepage":"https://hex.pm/packages/queutils/","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cantido.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-05-27T01:05:37.000Z","updated_at":"2024-03-16T02:28:04.000Z","dependencies_parsed_at":"2024-03-08T06:48:35.849Z","dependency_job_id":null,"html_url":"https://github.com/Cantido/queutils","commit_stats":{"total_commits":47,"total_committers":3,"mean_commits":"15.666666666666666","dds":"0.44680851063829785","last_synced_commit":"90b1d6ff07d600625797aa2844f597130d0f49d7"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/Cantido/queutils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cantido%2Fqueutils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cantido%2Fqueutils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cantido%2Fqueutils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cantido%2Fqueutils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cantido","download_url":"https://codeload.github.com/Cantido/queutils/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cantido%2Fqueutils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280286295,"owners_count":26304700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broadway","elixir","hacktoberfest","queue"],"created_at":"2025-10-21T15:31:16.042Z","updated_at":"2025-10-21T15:32:17.446Z","avatar_url":"https://github.com/Cantido.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Queutils\n\n[![Hex.pm](https://img.shields.io/hexpm/v/queutils)](https://hex.pm/packages/queutils/)\n![Elixir CI](https://github.com/Cantido/queutils/workflows/Elixir%20CI/badge.svg)\n[![standard-readme compliant](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg)](https://github.com/RichardLitt/standard-readme)\n[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md)\n\nHandy little queues and producers that make using `GenStage` or `Broadway` a breeze.\n\nGetting events into a `GenStage` or `Broadway` pipeline is more difficult than it seems at first glance.\nAny message producer based on `GenStage` needs to track demand,\nand emit events as it receives them if there is surplus demand.\nAlso, any producer implementation needs to provide back-pressure,\na key part of `GenStage` and `Broadway`'s design.\nThat's why a standalone blocking queue process is most the most ideal producer for these libraries.\n\n- Decouples message producers and consumers, so producers do not need to wait for the consumer to be ready\n- Blocks when full, so producers don't keep producing messages if the consumer is overwhelmed\n- Tracks demand, and immediately emits events if demand is pending\n\nThis library also provides a blocking queue implementation that can be used in pure Elixir.\n\n## Installation\n\nThis library is [available in Hex](https://hex.pm/docs/publish), and the package can be installed\nby adding `queutils` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:queutils, \"~\u003e 1.2\"}\n  ]\nend\n```\n\nDocumentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)\nand can be found online at [https://hexdocs.pm/queutils](https://hexdocs.pm/queutils).\n\n## Usage\n\nIf you just want a queue to communicate between processes, use a `Queutils.BlockingQueue`.\nThis module implements a queue with a fixed length,\nand any calls to `Queutils.BlockingQueue.push/2` will block until the queue has room again.\n\nIf you're working with `GenStage`, you probably want to use `Queutils.BlockingProducer`.\nThis module is just like a `Queutils.BlockingQueue`,\nbut it also provides callbacks that let a `GenStage` consumer subscribe to it.\nThis way you can push messages into a `GenStage` pipeline.\n\nLastly, a `Queutils.BlockingQueueProducer` acts just like a `Queutils.BlockingProducer`,\nexcept you need to provide the queue yourself.\nThis is the module to use if you're working with `Broadway`,\nbecause `Broadway` stages needs to start up their producers themselves.\n\n### With plain Elixir\n\nAdd `Queutils.BlockingQueue` to your application supervisor's `start/2` function, like this:\n\n```elixir\ndef start(_type, _args) do\n  children = [\n    {Queutils.BlockingQueue, name: MessageQueue, max_length: 10_000},\n  ]\n\n  opts = [strategy: :one_for_one, name: MyApplication.Supervisor]\n  Supervisor.start_link(children, opts)\nend\n```\n\nYou can now push messages to the queue like this:\n\n```elixir\n:ok = Queutils.BlockingQueue.push(MessageQueue, :my_message)\n```\n\nand pop from it like this:\n\n```elixir\n:my_message = Queutils.BlockingQueue.pop(MessageQueue)\n```\n\n### With GenStage\n\nAdd `Queutils.BlockingProducer` to your application supervisor's `start/2` function, like this:\n\n```elixir\ndef start(_type, _args) do\n  children = [\n    {Queutils.BlockingProducer, name: MessageProducer, max_length: 10_000}\n  ]\n\n  opts = [strategy: :one_for_one, name: MyApplication.Supervisor]\n  Supervisor.start_link(children, opts)\nend\n```\n\nThen, subscribe a `GenStage` to it.\n\n```elixir\ndef init(:ok) do\n  {:consumer, :the_state_does_not_matter, subscribe_to: [MessageProducer]}\nend\n```\n\nYou can now push messages to the queue like this:\n\n```elixir\n:ok = Queutils.BlockingProducer.push(MessageProducer, :my_message)\n```\n\n### With Broadway\n\nAdd `Queutils.BlockingQueue` to your application supervisor's `start/2` function,\njust like we're using it with plain Elixir:\n\n```elixir\ndef start(_type, _args) do\n  children = [\n    {Queutils.BlockingQueue, name: MessageQueue, max_length: 10_000},\n  ]\n\n  opts = [strategy: :one_for_one, name: MyApplication.Supervisor]\n  Supervisor.start_link(children, opts)\nend\n```\n\nThen, add a `Queutils.BlockingQueueProducer` as your `Broadway` stage's producer,\npointing it to the queue you just created.\n\n```elixir\ndef start_link(_opts) do\n  Broadway.start_link(__MODULE__,\n    name: __MODULE__,\n    producer: [\n      module: {Queutils.BlockingQueueProducer, queue: MessageQueue},\n      transformer: {MyApplication.Transformer, :transform, []}\n    ],\n    processors: [\n      default: []\n    ]\n  )\nend\n```\n\nYou will need to add a `:transformer` option to your `Broadway` stage in order to wrap messages in a `Broadway.Message` struct.\nIt's easy, but needs to be done.\nSee `Broadway`'s [Custom Producers](https://hexdocs.pm/broadway/custom-producers.html) documentation for details.\n\nYou can now push to the queue like this, and your `Broadway` stage will pick it up:\n\n```elixir\n:ok = Queutils.Blockingqueue.push(MessageQueue, :my_message)\n```\n\n## Maintainer\n\nThis project was developed by [Rosa Richter](https://github.com/Cantido).\nYou can get in touch with her on [Keybase.io](https://keybase.io/cantido).\n\n## Contributing\n\nQuestions and pull requests are more than welcome.\nI follow Elixir's tenet of bad documentation being a bug,\nso if anything is unclear, please [file an issue](https://github.com/Cantido/queutils/issues/new)!\nIdeally, my answer to your question will be in an update to the docs.\n\nPlease see [CONTRIBUTING.md](CONTRIBUTING.md) for all the details you could ever want about helping me with this project.\n\nNote that this project is released with a Contributor [Code of Conduct](code_of_conduct.md).\nBy participating in this project you agree to abide by its terms.\n\n## License\n\nMIT License\n\nCopyright 2020 Rosa Richter.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is furnished to do\nso, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcantido%2Fqueutils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcantido%2Fqueutils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcantido%2Fqueutils/lists"}