Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/vtsykun/cron-bundle

:clock3: Docker friendly Symfony Cron Bundle for handling scheduled tasks consistently, parallel or via message queue
https://github.com/vtsykun/cron-bundle

cron-bundle cron-jobs hacktoberfest lock message-queue scheduled-tasks symfony-messenger

Last synced: 2 months ago
JSON representation

:clock3: Docker friendly Symfony Cron Bundle for handling scheduled tasks consistently, parallel or via message queue

Awesome Lists containing this project

README 

        
# Okvpn - Cron Bundle



This bundle provides interfaces for registering and handle scheduled tasks within your Symfony application.



[![Latest Stable Version](https://poser.okvpn.org/okvpn/cron-bundle/v/stable)](https://packagist.org/packages/okvpn/cron-bundle)

[![Total Downloads](https://poser.okvpn.org/okvpn/cron-bundle/downloads)](https://packagist.org/packages/okvpn/cron-bundle) 

[![Latest Unstable Version](https://poser.okvpn.org/okvpn/cron-bundle/v/unstable)](https://packagist.org/packages/okvpn/cron-bundle) 

[![License](https://poser.okvpn.org/okvpn/cron-bundle/license)](https://packagist.org/packages/okvpn/cron-bundle)



## Purpose

This is a simpler alternative of existing cron bundle without doctrine deps.

Here also added support middleware for customization handling cron jobs across a cluster install: 

(Send jobs to message queue, like Symfony Messenger; locking; etc.).

This allows to limit the number of parallel running processes and prioritized it.



Features

--------



- Not need doctrine/database.

- Docker friendly, runs as background command without `crond`.

- Schedule tasks with one-millisecond precision.

- More ways to randomize crons with `@random 3600` and `jitter`.

- Integration with Symfony Messenger.

- Load a cron job from a different storage (config.yml, tagged services, commands).

- Support many engines to run cron (in parallel process, message queue, consistently).

- Support many types of cron handlers/command: (services, symfony commands, UNIX shell commands).

- Can be used along with timers, subscriber and async I/O with React EventLoop, like Redis subscriber [clue/redis-react](https://github.com/clue/reactphp-redis).

- Middleware and customization.



## Table of Contents



 - [Install](#install)

 - [Commands](#commands)

 - [Registration a new scheduled task](#registration-a-new-scheduled-task)

 - [Configuration](#full-configuration-reference)

 - [Symfony Messenger Integration](#handle-cron-jobs-via-symfony-messenger)

 - [Your own Scheduled Tasks Loader](#your-own-scheduled-tasks-loaders)

 - [Handling cron jobs across a cluster](#handling-cron-jobs-across-a-cluster-or-custom-message-queue)

 - [Use ReactPHP EventLoop](#use-reactphp-eventloop)



Install

------



Install using composer:



```

composer require okvpn/cron-bundle

```



For Symfony 4+ add bundle to `config/bundles.php`



```php

 ['all' => true],

]

```



##  Quick Usage 



You can use `AsCron` or `AsPeriodicTask` attribute for autoconfigure.



```php

 /dev/null

```



#### Second way. Using supervisor



Setup Supervisor to run cron on demand.



```

sudo apt -y --no-install-recommends install supervisor

```



Create a new supervisor configuration.



```

sudo vim /etc/supervisor/conf.d/app_cron.conf

```

Add the following lines to the file.



```

[program:app-cron]

command=/path/to/bin/console okvpn:cron --env=prod --demand

process_name=%(program_name)s_%(process_num)02d

numprocs=1

autostart=true

autorestart=true

startsecs=0

redirect_stderr=true

priority=1

user=www-data

```



## Registration a new scheduled task



To add a new scheduled task you can use tag `okvpn.cron` or using `autoconfigure`

with interface `Okvpn\Bundle\CronBundle\CronSubscriberInterface`.



#### Services.



```php

 /root/renew.txt" # Shell command

      group: root # Filtering by group. You can run `bin/console okvpn:cron --group=root` under the root user 

      cron: "0 0 * * *"

    -

      command: 'App\Cron\YouServiceName' # Your service name

      cron: "0 0 * * *"

    -

      command: 'app:cron:sync-amazon-orders' # Your symfony console command name

      cron: "*/30 * * * *"

      async: true

      arguments: { '--transport': 15 } # command arguments or options

      jitter: 60 # 60 sec random delay 



    -

      command: 'app:cron:wrfda-grib2' # run the command with 20 sec interval and 10 sec random delay 

      interval: "20 seconds"

      jitter: 10

```



## Full Configuration Reference



```yaml

# Your config file

okvpn_cron:

    lock_factory: ~ # The Service to create lock. Default lock.factory, see Symfony Lock component.

    timezone: ~ # default timezone, like Europe/Minsk. if null will use php.ini default

    messenger:

        enable: false # Enable symfony messenger

        

    # Default options allow to add define default policy for all tasks, 

    # For example to always run commands with locking and asynchronously

    default_policy:

        async: true # Default false

        lock: true # Default false

        messenger: true # Handle all jobs with symfony messenger bus.

    

    # Stamps it's markers that will add to each tasks.

    with_stamps:

        - 'Packagist\WebBundle\Cron\WorkerStamp'

    

    # service name for run cron in demand (Okvpn\Bundle\CronBundle\Runner\ScheduleLoopInterface)

    loop_engine: ~



    tasks: # Defined tasks via configuration

      - 

        command: 'app:noaa:gfs-grib-download'

        cron: '34,45 */6 * * *'

        messenger: { routing: lowpriority } # See Messenger configuration

        lock: true

        arguments: { '--transport': '0p25' }

        # Here you can also add other custom options and create your own middleware.

      -

        command: "bash /root/renew.sh > /root/renew.txt" # Shell command

        group: root # Group filter. You can run `bin/console okvpn:cron --group=root` under the root user 

        cron: "0 0 * * *"

```



## Handle Cron Jobs via Symfony Messenger 



To limit the number of parallel running processes you can handle the cron jobs in the queue using Symfony Messenger.



1. Install Symfony Messenger

2. Enable default route for cron job



```yaml

# config/packages/messenger.yaml

framework:

    messenger:

        transports:

            async: "%env(MESSENGER_TRANSPORT_DSN)%"

            lowpriority: "%env(MESSENGER_TRANSPORT_LOW_DSN)%"



        routing:

            # async is whatever name you gave your transport above

            'Okvpn\Bundle\CronBundle\Messenger\CronMessage':  async

```



3. Enable Messenger for cron.



```yaml

# config/packages/cron.yaml

okvpn_cron:

    # Required. messenger middleware is disable

    messenger:

        enable: true



    # Optional

    default_options:

        messenger: true # For handle all cron jobs with messenger

    

    # Optional 

    tasks:

        - 

            command: 'app:noaa:gfs-grib-download'

            cron: '34,45 */6 * * *'

#           messenger: true # OR

            messenger: { routing: lowpriority } # Send to lowpriority transport

```



More information how to use [messenger here](https://symfony.com/doc/current/messenger.html)



## Your own Scheduled Tasks Loaders



You can create custom tasks loaders, see example



```php

 1]), // Command arguments

            new Model\AsyncStamp() // If you want to run asynchronously

        );



        yield new ScheduleEnvelope(

            'ls -l', // shell command

            new Model\ScheduleStamp('*/10 * * * *'),  // Cron expression

            new Model\ShellStamp(['timeout'=> 300]), // Run command as shell

        );



        // ...

    }

}



```



And register your loader.



```yaml

services:

    Packagist\WebBundle\DoctrineCronLoader:

        tags: [okvpn_cron.loader]



```



## Handling cron jobs across a cluster or custom message queue 



You can use the cron `$group` to split many scheduled tasks between clusters, see example:



```php

registry->getAllRepos($chunkId) as $name => $repo) {

            $expr = '@random ' . $this->getSyncInterval($repo);

            

            yield new ScheduleEnvelope(

                'sync:mirrors',

                new Model\ScheduleStamp($expr),

                new WorkerStamp(true),

                new Model\ArgumentsStamp(['mirror' => $name,])

            );

        }

    }

}

```



```

[program:app-cron]

command=/path/to/bin/console okvpn:cron --env=prod --demand --group=chunk_%process_num%

process_name=%(program_name)s_%(process_num)02d

numprocs=1

```



See example of customization 

[one](https://github.com/vtsykun/packeton/tree/master/src/Cron/WorkerMiddleware.php), 

[two](https://github.com/vtsykun/packeton/tree/master/src/Cron/CronWorker.php)



## Use ReactPHP EventLoop



You can add your own periodic tasks directly to `Loop`. 

The bundle uses a simple wrapper `Okvpn\Bundle\CronBundle\Runner\ScheduleLoopInterface` for the library `react/event-loop`



```php

getDDog();

        $event->getLoop()->addPeriodicTimer(6.0, static function () use ($dataDogS) {

            $dataDogS->set('crond', getmypid());

        });

    }

}

```



#### Configure ReactPHP adapter



Need to install [react/event-loop](https://github.com/reactphp/event-loop) if you want to use with async I/O, for example for handle websockets, redis.



```

composer req react/event-loop 

```



```yaml

# Add file to config/packages/*

okvpn_cron:

    loop_engine: okvpn_cron.react_loop # service name

```



```php

timers;

        $loop = $event->getLoop();



        $redis->on('message', static function (string $channel, string $payload) use ($timers, $loop) {

            [$command, $args] = unserialize($payload);

            if ($timers->hasTimer($envelope = $timers->find($command, $args))) {

                [$timer] = $timers->getTimer($envelope);

                $loop->futureTick($timer);

            }

        });



        Loop::addPeriodicTimer(5.0, static function () use ($redis) {

            $redis->ping();

        });

    }

}

```



License

-------



MIT License.