{"id":18284472,"url":"https://github.com/pplu/sqs-worker","last_synced_at":"2026-03-07T23:31:45.126Z","repository":{"id":56831200,"uuid":"71585932","full_name":"pplu/SQS-Worker","owner":"pplu","description":"A framework to make SQS workers easy to program","archived":false,"fork":false,"pushed_at":"2019-06-19T15:26:41.000Z","size":55,"stargazers_count":3,"open_issues_count":6,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-21T00:26:11.573Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pplu.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-10-21T17:48:03.000Z","updated_at":"2020-12-03T09:43:35.000Z","dependencies_parsed_at":"2022-09-09T16:24:04.415Z","dependency_job_id":null,"html_url":"https://github.com/pplu/SQS-Worker","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2FSQS-Worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2FSQS-Worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2FSQS-Worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2FSQS-Worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pplu","download_url":"https://codeload.github.com/pplu/SQS-Worker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305872,"owners_count":20917197,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T13:13:40.355Z","updated_at":"2025-12-12T02:48:34.777Z","avatar_url":"https://github.com/pplu.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SQS::Worker\n\nThis project is a light framework that allows you to just code asyncronous tasks\nthat consume messages from an SQS Queue. The framework takes care of launching the \nnecessary processes (workers), and executes your code on incoming messages.\n\nAlso, since you're surely going to be deserializing the messages that come from the\nqueue, SQS::Worker provides you with ways to easily consume JSON messages, for example.\n\n# Architecture\n\n## The worker\n\nThe worker is the unit of work. Each worker is launched independently of other workers, \nwhich fits the asynchronous and independent nature of messaging. In fact, each worker \nwill be a full process, that will dispatch messages from the queue to your code.\n\nA worker is a role that your code will consume, and that will let the framework know how \nto send messages to it.\n\nThe Moose role ```SQS::Worker``` is to be used for this purpose.\n\n```\npackage YourWorker;\n\nuse Moose;\nwith 'SQS::Worker';\n\nsub process_message {\n\tmy ($self, $message) = @_;\n\n\t# do something with that message: f. ex:\n\n  $self-\u003elog-\u003edebug(\"I'm going to split the message\");\n  my @parts = split /:/, $message;\n\n  ...\n}\n```\n\nYou get a logger attached to the worker by default, so you don't need to worry about logging.\n\nBy default, if something goes badly in `process_message` (it raises an uncontrolled \nexception) the message will not be deleted from the queue, thus being retried after\nthe visibility timeout: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html\n\nOnce you have a worker class, there's a loader (installed as part of the SQS::Worker framework) \ncalled `spawn_worker` that, along with some extra configuration, will launch a process that \nwill receive messages from the queue and pass them to the worker class you've created.\n\n```\nspawn_worker --worker YourClass --queue_url sqs_endpoint_url --region aws_sqs_region --log_conf log4perl_config_file_path\n```\n\nYou can control if the message should be deleted upon reception (before the message is actually processed) with:\n\n```\nspawn_worker --worker YourClass --queue_url sqs_endpoint_url --region aws_sqs_region --log_conf log4perl_config_file_path --consumer DeleteAlways\n```\n\n`spawn_worker` is just a convenience loader: you don't necessarily need to use it: you can write your own worker (just look at it's code)\n\n## Composable interceptors for workers\n\nWhile the basic worker role will provide your code with a raw sqs message (as a string), there are interceptors that can be composed into your class that will pre-process the message. Among them:\n\n- SQS::Worker::DecodeJson\n- SQS::Worker::DecodeStorable\n- SQS::Worker::Multipex\n- SQS::Worker::SNS\n\nFor example, if you compose your worker with the role SQS::Worker::DecodeJSON, the message received by the process_message will be a perl datastructure, instead of a string.\n\nLook at the documentation of each interceptor too see what each does, and how to be used.\n\nYou can compose more than one Worker role into your worker. If you recieve a message in JSON format, and later want to dispatch it to a series\nof actions, you can:\n\n```\nwith 'SQS::Worker', 'SQS::Worker::DecodeJson', 'SQS::Worker::Multiplex';\n```\n\n## Credentials handling\n\nSQS::Worker is an abstraction over Paws, it thus uses the same credential system that Paws does, which means the three ways you can provide the access key and secret key for the code to use:\n\n- having the credentials in the home of the user launching the script, in the ~/.aws/credentials file.\n- by assigning an IAM role to the EC2 instance that is running the code (if deploying the code inside an EC2 instance)\n- by using environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpplu%2Fsqs-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpplu%2Fsqs-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpplu%2Fsqs-worker/lists"}