{"id":18916802,"url":"https://github.com/dusterio/laravel-aws-worker","last_synced_at":"2025-05-14T12:06:46.396Z","repository":{"id":8915956,"uuid":"60133263","full_name":"dusterio/laravel-aws-worker","owner":"dusterio","description":"Run Laravel (or Lumen) tasks and queue listeners inside of AWS Elastic Beanstalk workers","archived":false,"fork":false,"pushed_at":"2025-03-27T15:35:18.000Z","size":97,"stargazers_count":310,"open_issues_count":0,"forks_count":60,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-12T23:29:42.845Z","etag":null,"topics":["aws","beanstalk","laravel","worker"],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dusterio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-06-01T00:37:46.000Z","updated_at":"2025-03-27T15:27:08.000Z","dependencies_parsed_at":"2024-05-29T07:02:40.159Z","dependency_job_id":"688b97b7-d385-4a3a-aed9-ce5fd12d5462","html_url":"https://github.com/dusterio/laravel-aws-worker","commit_stats":{"total_commits":64,"total_committers":13,"mean_commits":4.923076923076923,"dds":0.5,"last_synced_commit":"7a56a3727a7924ed4afed14cea685ff0d23f5686"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusterio%2Flaravel-aws-worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusterio%2Flaravel-aws-worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusterio%2Flaravel-aws-worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusterio%2Flaravel-aws-worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dusterio","download_url":"https://codeload.github.com/dusterio/laravel-aws-worker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254140740,"owners_count":22021218,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","beanstalk","laravel","worker"],"created_at":"2024-11-08T10:22:05.170Z","updated_at":"2025-05-14T12:06:46.320Z","avatar_url":"https://github.com/dusterio.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# laravel-aws-worker\n[![Code Climate](https://codeclimate.com/github/dusterio/laravel-aws-worker/badges/gpa.svg)](https://codeclimate.com/github/dusterio/laravel-aws-worker/badges)\n[![Total Downloads](https://poser.pugx.org/dusterio/laravel-aws-worker/d/total.svg)](https://packagist.org/packages/dusterio/laravel-aws-worker)\n[![Latest Stable Version](https://poser.pugx.org/dusterio/laravel-aws-worker/v/stable.svg)](https://packagist.org/packages/dusterio/laravel-aws-worker)\n[![Latest Unstable Version](https://poser.pugx.org/dusterio/laravel-aws-worker/v/unstable.svg)](https://packagist.org/packages/dusterio/laravel-aws-worker)\n[![License](https://poser.pugx.org/dusterio/laravel-aws-worker/license.svg)](https://packagist.org/packages/dusterio/laravel-plain-sqs)\n\nRun Laravel tasks and queue listeners inside of AWS Elastic Beanstalk workers\n\n\u003e We've dropped future support for Lumen, however you can still use [v0.1.40](https://github.com/dusterio/laravel-aws-worker/releases/tag/v0.1.40) for Lumen.\n\n## Overview\n\nLaravel documentation recommends to use supervisor for queue workers and *IX cron for scheduled tasks. However, when deploying your application to AWS Elastic Beanstalk, neither option is available.\n\nThis package helps you run your Laravel jobs in AWS worker environments.\n\n![Standard Laravel queue flow](https://www.mysenko.com/images/queues-laravel.png)\n![AWS Elastic Beanstalk flow](https://www.mysenko.com/images/queues-aws_eb.png)\n\n## Dependencies\n\n* PHP \u003e= 5.5\n* Laravel \u003e= 5.1\n\n## Scheduled tasks - option 1\n\nOption one is to use Kernel.php as the schedule and run Laravel schedule runner every minute.\nYou remember how Laravel documentation advised you to invoke the task scheduler? Right, by running ```php artisan schedule:run``` on regular basis, and to do that we had to add an entry to our cron file:\n\n```bash\n* * * * * php /path/to/artisan schedule:run \u003e\u003e /dev/null 2\u003e\u00261\n```\n\nAWS doesn't allow you to run *IX commands or to add cron tasks directly. Instead, you have to make regular HTTP (POST, to be precise) requests to your worker endpoint.\n\nAdd cron.yaml to the root folder of your application (this can be a part of your repo or you could add this file right before deploying to EB - the important thing is that this file is present at the time of deployment):\n\n```yaml\nversion: 1\ncron:\n - name: \"schedule\"\n   url: \"/worker/schedule\"\n   schedule: \"* * * * *\"\n```\n\nFrom now on, AWS will do POST /worker/schedule to your endpoint every minute - kind of the same effect we achieved when editing a UNIX cron file. The important difference here is that the worker environment still has to run a web process in order to execute scheduled tasks.\nBehind the scenes it will do something very similar to a built-in `schedule:run` command.\n\nYour scheduled tasks should be defined in ```App\\Console\\Kernel::class``` - just where they normally live in Laravel, eg.:\n\n```php\nprotected function schedule(Schedule $schedule)\n{\n    $schedule-\u003ecommand('inspire')\n              -\u003eeveryMinute();\n}\n```\n\n## Scheduled tasks - option 2\n\nOption two is to use AWS schedule defined in the cron.yml:\n\n```yaml\nversion: 1\ncron:\n - name: \"run:command\"\n   url: \"/worker/schedule\"\n   schedule: \"0 * * * *\"\n\n - name: \"do:something --param=1 -v\"\n   url: \"/worker/schedule\"\n   schedule: \"*/5 * * * *\"\n```\n\nNote that AWS will use UTC timezone for cron expressions. With the above example,\nAWS will hit /worker/schedule endpoint every hour with `run:command` artisan command and every\n5 minutes with `do:something` command. Command parameters aren't supported at this stage.\n\nPick whichever option is better for you!\n\n## Queued jobs: SQS\n\nNormally Laravel has to poll SQS for new messages, but in case of AWS Elastic Beanstalk messages will come to us – inside of POST requests from the AWS daemon. \n\nTherefore, we will create jobs manually based on SQS payload that arrived, and pass that job to the framework's default worker. From this point, the job will be processed the way it's normally processed in Laravel. If it's processed successfully,\nour controller will return a 200 HTTP status and AWS daemon will delete the job from the queue. Again, we don't need to poll for jobs and we don't need to delete jobs - that's done by AWS in this case.\n\nIf you dispatch jobs from another instance of Laravel or if you are following Laravel's payload format ```{\"job\":\"\",\"data\":\"\"}``` you should be okay to go. If you want to receive custom format JSON messages, you may want to install \n[Laravel plain SQS](https://github.com/dusterio/laravel-plain-sqs) package as well.\n\n## Configuring the queue\n\nEvery time you create a worker environment in AWS, you are forced to choose two SQS queues – either automatically generated ones or some of your existing queues. One of the queues will be for the jobs themselves, another one is for failed jobs – AWS calls this queue a dead letter queue.\n\nYou can set your worker queues either during the environment launch or anytime later in the settings:\n\n![AWS Worker queue settings](https://www.mysenko.com/images/worker_settings.jpg)\n\nDon't forget to set the HTTP path to ```/worker/queue``` – this is where AWS will hit our application. If you chose to generate queues automatically, you can see their details later in SQS section of the AWS console:\n\n![AWS SQS details](https://www.mysenko.com/images/sqs_details.jpg)\n\nYou have to tell Laravel about this queue. First set your queue driver to SQS in ```.env``` file:\n\n```\nQUEUE_DRIVER=sqs\n```\n\nThen go to ```config/queue.php``` and copy/paste details from AWS console:\n\n```php\n        ...\n        'sqs' =\u003e [\n            'driver' =\u003e 'sqs',\n            'key' =\u003e 'your-public-key',\n            'secret' =\u003e 'your-secret-key',\n            'prefix' =\u003e 'https://sqs.us-east-1.amazonaws.com/your-account-id',\n            'queue' =\u003e 'your-queue-name',\n            'region' =\u003e 'us-east-1',\n        ],\n        ...\n```\n\nTo generate key and secret go to Identity and Access Management in the AWS console. It's better to create a separate user that ONLY has access to SQS.\n\n## Installation via Composer\n\nTo install simply run:\n\n```\ncomposer require dusterio/laravel-aws-worker\n```\n\nOr add it to `composer.json` manually:\n\n```json\n{\n    \"require\": {\n        \"dusterio/laravel-aws-worker\": \"~0.1\"\n    }\n}\n```\n\n### Usage in Laravel 5\n\n```php\n// Add in your config/app.php\n\n'providers' =\u003e [\n    '...',\n    'Dusterio\\AwsWorker\\Integrations\\LaravelServiceProvider',\n];\n```\n\nAfter adding service provider, you should be able to see two special routes that we added:\n\n```bash\n$ php artisan route:list\n+--------+----------+-----------------+------+----------------------------------------------------------+------------+\n| Domain | Method   | URI             | Name | Action                                                   | Middleware |\n+--------+----------+-----------------+------+----------------------------------------------------------+------------+\n|        | POST     | worker/queue    |      | Dusterio\\AwsWorker\\Controllers\\WorkerController@queue    |            |\n|        | POST     | worker/schedule |      | Dusterio\\AwsWorker\\Controllers\\WorkerController@schedule |            |\n+--------+----------+-----------------+------+----------------------------------------------------------+------------+\n```\n\nEnvironment variable ```REGISTER_WORKER_ROUTES``` is used to trigger binding of the two routes above. If you run the same application in both web and worker environments,\ndon't forget to set ```REGISTER_WORKER_ROUTES``` to ```false``` in your web environment. You don't want your regular users to be able to invoke scheduler or queue worker.\n\nThis variable is set to ```true``` by default at this moment.\n\nSo that's it - if you (or AWS) hits ```/worker/queue```, Laravel will process one queue item (supplied in the POST). And if you hit ```/worker/schedule```, we will run the scheduler (it's the same as to run ```php artisan schedule:run``` in shell).\n\n### Usage in Lumen 5\n\n```php\n// Add in your bootstrap/app.php\n$app-\u003eregister(Dusterio\\AwsWorker\\Integrations\\LumenServiceProvider::class);\n```\n\n## Errors and exceptions\n\nPlease make sure that two special routes are not mounted behind a CSRF middleware. Our POSTs are not real web forms and CSRF is not necessary here. If you have a global CSRF middleware, add these routes to exceptions, or otherwise apply CSRF to specific routes or route groups.\n\nIf your job fails, we will throw a ```FailedJobException```. If you want to customize error output – just customise your exception handler.\nNote that your HTTP status code must be different from 200 in order for AWS to realize the job has failed.\n\n## Job expiration (retention)\n\nA new nice feature is being able to set a job expiration (retention in AWS terms) in seconds so \nthat low value jobs get skipped completely if there is temporary queue latency due to load.\n\nLet's say we have a spike in queued jobs and some of them don't even make sense anymore\nnow that a few minutes passed – we don't want to spend computing resources processing them\nlater.\n\nBy setting a special property on a job or a listener class:\n```php\nclass PurgeCache implements ShouldQueue\n{\n    public static int $retention = 300; // If this job is delayed more than 300 seconds, skip it\n}\n```\n\nWe can make sure that we won't run this job later than 300 seconds since it's been queued.\nThis is similar to AWS SQS \"message retention\" setting which can only be set globally for\nthe whole queue.\n\nTo use this new feature, you have to use provided ```Jobs\\CallQueuedHandler``` class that\nextends Laravel's default ```CallQueuedHandler```. A special ```ExpiredJobException``` exception\nwill be thrown when expired jobs arrive and then it's up to you what to do with them.\n\nIf you just catch these exceptions and therefore stop Laravel from returning code 500\nto AWS daemon, the job will be deleted by AWS automatically.\n\n## ToDo\n\n1. Add support for AWS dead letter queue (retry jobs from that queue?)\n\n## Video tutorials\n\nI've just started a educational YouTube channel that will cover top IT trends in software development and DevOps: [config.sys](https://www.youtube.com/channel/UCIvUJ1iVRjJP_xL0CD7cMpg)\n\nAlso I'm glad to announce a new cool tool of mine – [GrammarCI](https://www.grammarci.com/), an automated typo/grammar checker for developers, as a part of the CI/CD pipeline.\n\n## Implications\n\nNote that AWS cron doesn't promise 100% time accuracy. Since cron tasks share the same queue with other jobs, your scheduled tasks may be processed later than expected. \n\n## Post scriptum\n\nI wrote a [blog post](https://web.archive.org/web/20210616063916/https://blog.menara.com.au/2016/06/running-laravel-in-amazon-elastic-beanstalk/) explaining how this actually works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdusterio%2Flaravel-aws-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdusterio%2Flaravel-aws-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdusterio%2Flaravel-aws-worker/lists"}