https://github.com/ostark/craft-async-queue
Async Queue Handler for Craft 3 and 4
https://github.com/ostark/craft-async-queue
craft craft3 craftcms craftcms-plugin plugin
Last synced: 6 months ago
JSON representation
Async Queue Handler for Craft 3 and 4
- Host: GitHub
- URL: https://github.com/ostark/craft-async-queue
- Owner: ostark
- License: mit
- Created: 2017-08-24T10:52:37.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2024-05-21T10:14:32.000Z (over 1 year ago)
- Last Synced: 2025-03-29T21:05:33.621Z (6 months ago)
- Topics: craft, craft3, craftcms, craftcms-plugin, plugin
- Language: PHP
- Homepage:
- Size: 1.02 MB
- Stars: 95
- Watchers: 2
- Forks: 8
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Async (Background) Queue
[](https://packagist.org/packages/ostark/craft-async-queue)
[](https://packagist.org/packages/ostark/craft-async-queue)
[](https://packagist.org/packages/ostark/craft-async-queue)
[](https://plant.treeware.earth/ostark/craft-async-queue)
With Craft's job queue you can run heavy tasks in the background. Unfortunately, this is not entirely true. When `runQueueAutomatically => true` (default), the job queue is handled by a ajax (FPM) call.
With many jobs in the queue and limited PHP-FPM processes this can break your site. This plugin replaces Craft's default queue handler and moves queue execution to a non-blocking background process. The command `craft queue/run` gets executed right after you push a job to the queue.
[Here](https://github.com/craftcms/cms/issues/1952) you can find the initial discussion I started at `craftcms/cms`.
## Sponsor 🐇
Development happens in my free time, but also during working hours. Thanks [fortrabbit.com](https://www.fortrabbit.com/craft-hosting)!
## Licence 🌳
This package is [Treeware](https://treeware.earth). If you use it in production, then we ask that you [**buy the world a tree**](https://plant.treeware.earth/ostark/craft-async-queue) to thank me for my work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.
## Requirements
* Craft 3 or 4
* Permissions to execute a php binary
* proc_open()
* **PHP >=7.1** (for PHP 7.0 use `ostark/craft-async-queue:1.3.*`)
## Installation
```shell
cd your/craft-project
composer require ostark/craft-async-queue
php craft install/plugin async-queue
```
If you run into Composer version conflicts:
```
composer config platform --unset
composer update
php craft migrate/all
composer require ostark/craft-async-queue
php craft install/plugin async-queue
```
## Configuration (optional)
The plugin uses [symfony/process](https://github.com/symfony/process) to execute the `php` binary. Usually the binary is located in `/usr/bin/`, but other common locations are auto detected as well. With the ENV var `PHP_BINARY` you can explicitly set the path, e.g. in your .env file like this:
```
PHP_BINARY="/usr/local/Cellar/php71/7.1.0_11/bin/php"
```
By default `1` background process handles the queue. With the `ASYNC_QUEUE_CONCURRENCY` ENV var you can modify this behaviour.
```
# No concurrency
ASYNC_QUEUE_CONCURRENCY=1
# Or max 5 background processes
ASYNC_QUEUE_CONCURRENCY=5
```
To disable the plugin in certain environments, like on Windows which is not supported yet, set the `DISABLE_ASYNC_QUEUE` ENV var.
```
DISABLE_ASYNC_QUEUE=1
```
Note that the plugin will overwrite the `runQueueAutomatically` setting with Craft CMS. No configurartion required.
## Tests
Beside the test suite you can run from the command line with this shortcut: `composer tests`, you can perform a test in the Craft CP.
Navigate to `Utilities` > `Async Queue Test` and hit the `Run test` button.
## Events
The command that runs in the background is basically `php craft queue/run`, however we add some linux specific syntax that executes the command in a non-blocking way.
By setting `useDefaultDecoration` to `false` you prevent this. You have also the ability to modify the command itself.
```
// Add handler
\yii\base\Event::on(
\ostark\AsyncQueue\QueueCommand::class,
\ostark\AsyncQueue\QueueCommand::EVENT_PREPARE_COMMAND,
function(\ostark\AsyncQueue\Events\QueueCommandEvent $event) {
$event->useDefaultDecoration = false;
$event->commandLine = "BEFORE {$event->commandLine} AFTER";
}
);
```
## Under the hood: Process list
**Empty queue** (only php-fpm master is running)
```
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
````
**New job pushed** (php-fpm master + child + /usr/bin/php daemon started)
```
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
app 3031 2.2 0.2 718520 45992 ? S 12:31 0:00 \_ php-fpm: pool www
app 3033 1.2 0.2 280936 32808 ? S 12:31 0:00 /usr/bin/php craft queue/run
app 3034 0.0 0.0 4460 784 ? S 12:31 0:00 \_ sh -c /usr/bin/php craft queue/exec "1234" "0" "1"
app 3035 1.2 0.2 280928 32280 ? S 12:31 0:00 \_ /usr/bin/php craft queue/exec 1234 0 1
```