{"id":22092865,"url":"https://github.com/toyokumo/gluttony","last_synced_at":"2025-07-24T20:32:26.949Z","repository":{"id":38461655,"uuid":"232930096","full_name":"toyokumo/gluttony","owner":"toyokumo","description":"A consumer library using core.async and aws-api based on AWS SQS","archived":false,"fork":false,"pushed_at":"2024-06-17T06:06:31.000Z","size":90,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-11-28T14:18:58.557Z","etag":null,"topics":["clojure","consumer","sqs"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/toyokumo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-10T00:03:39.000Z","updated_at":"2024-06-17T06:06:33.000Z","dependencies_parsed_at":"2024-06-17T03:33:28.772Z","dependency_job_id":null,"html_url":"https://github.com/toyokumo/gluttony","commit_stats":{"total_commits":92,"total_committers":4,"mean_commits":23.0,"dds":"0.48913043478260865","last_synced_commit":"a9a9a47cfe379ee9b1cd142396bcad83a91afee3"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toyokumo%2Fgluttony","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toyokumo%2Fgluttony/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toyokumo%2Fgluttony/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toyokumo%2Fgluttony/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/toyokumo","download_url":"https://codeload.github.com/toyokumo/gluttony/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227476038,"owners_count":17779417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","consumer","sqs"],"created_at":"2024-12-01T03:11:10.463Z","updated_at":"2024-12-01T03:11:11.100Z","avatar_url":"https://github.com/toyokumo.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gluttony\nA consumer library using [core.async](https://github.com/clojure/core.async)\nbased on AWS SQS.\n\nYou can use this library with Standard queue but it is almost designed for FIFO queue.\n\n[![Build and Test](https://github.com/toyokumo/gluttony/actions/workflows/build-and-test.yml/badge.svg)](https://github.com/toyokumo/gluttony/actions/workflows/build-and-test.yml)\n[![cljdoc badge](https://cljdoc.org/badge/toyokumo/gluttony)](https://cljdoc.org/d/toyokumo/gluttony/CURRENT)\n[![Clojars Project](https://img.shields.io/clojars/v/toyokumo/gluttony.svg)](https://clojars.org/toyokumo/gluttony)\n\n## Installation\nTo install, add the following to your project `:dependencies`:\n\n[![Clojars Project](https://clojars.org/toyokumo/gluttony/latest-version.svg)](https://clojars.org/toyokumo/gluttony)\n\nand add [AWS SDK for Java 2.0](https://github.com/aws/aws-sdk-java-v2) to your dependencies.\n(If you want to use [aws-api](https://github.com/cognitect-labs/aws-api) instead of AWS SDK, add the dependency on it.)\n\n## Usage\n### Basis\nGluttony mainly offer two APIs, `start-consumer` and `stop-consumer`.\n\n`start-consumer` makes a consumer instance.\nIt has some `go-loop` processes internally and they receive messages from AWS SQS\nand invoke your `consume` function.\n\n`stop-consumer` takes a consumer created by `start-consumer`, and stop all processes and the client\nif you don't provide it.\n\nAfter calling `stop-consumer`, the consumer doesn't make a new `consume` function call,\nbut it doesn't immediately stop long polling access to AWS SQS.\nThe message acquired by long polling is not processed by the consumer and becomes an \"invisible message\" until the visibility timeout elapses.\n\nTo prevent that, call `stop-receivers` before calling `stop-consumer` and wait for the number of seconds specified by `long-polling` option (default: 20).\n\n```clojure\n(require '[clojure.core.async :as a]\n         '[gluttony.core :as gluttony]\n         '[gluttony.record.aws-sqs-client :as g.client])\n\n(defn consume\n  \"Your consume function takes three arguments.\n  `message` is a instance received from SQS.\n  You MUST call `respond` which is 2nd argument or `raise` which is 3rd argument.\n  respond delete the message from the SQS, so call it when your process has done successfully.\n  raise doesn't delete the message but change the limit of time that the message can be seen\n  from other receivers. raise takes zero or one argument, which control the limit of time.\n  default limit is zero, which means that retry will be executed as soon as possible.\"\n  [message respond raise]\n  (let [success? (do-my-computation-use-cpu message)]\n    (if success?\n      (do (respond)\n          (println \"success!\"))\n      ;; You can pass to the raise a integer determines retry delay seconds\n      (raise))))\n\n(def queue-url \"https://...\")\n\n(defonce consumer (atom nil))\n\n;; Start consumer connects to assigned queue\n(reset! consumer (gluttony/start-consumer queue-url consume (g.client/make-client)))\n\n;; Stop receiver and worker\n(when @consumer\n  (gluttony/stop-consumer @consumer)\n  (reset! consumer nil))\n```\n\n`start-consumer` can take some optional arguments which control consumer work\nsuch as the number of worker processes, long polling duration and so on.\nSee [API docs](https://cljdoc.org/d/toyokumo/gluttony/CURRENT) for detail.\n\n### Heartbeat\nIf you don't know how long it takes to process a message, pass `:hearbeat` and `:heartbeat-timeout` options.\n\nThen Gluttony extends the message visibility per `:hearbeat` seconds to `:hearbeat + 1` seconds.\n(Extended seconds is configurable by `:visibility-timeout-in-heartbeat` option)\n\nSee [AWS documents](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/working-with-messages.html) \nfor more detail. \n\n## What for?\nThere already has been [Squeedo](https://github.com/TheClimateCorporation/squeedo) for the same purpose\nand it provides nice APIs and work as intended. Gluttony use it as reference so much.\n\nGluttony deffer from Squeedo in following points.\n\nFirst, Gluttony only uses asynchronous http requests with aws-api, which uses Jetty http client internally,\nbut Squeedo depends on [AmazonSQS](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html) interface\ninstead of [AmazonSQSAsync](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html) and\ndoesn't depend on AWS SDK Ver2 but Ver1.11.\n\nSecond, Gluttony does focus on being as a consumer, so doesn't have APIs such as creating a new queue\nor setting attributes.\n\n## Test\nYou may have to fix `dev-resources/test-config.edn`.\n\n## License\n\nCopyright 2024 TOYOKUMO,Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoyokumo%2Fgluttony","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftoyokumo%2Fgluttony","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoyokumo%2Fgluttony/lists"}