Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/grinnz/minion-backend-sqlite

Minion::Backend::SQLite - SQLite backend for Minion job queue
https://github.com/grinnz/minion-backend-sqlite

Last synced: 2 months ago
JSON representation

Minion::Backend::SQLite - SQLite backend for Minion job queue

Awesome Lists containing this project

README 

        
=pod



=encoding utf8



=head1 NAME



Minion::Backend::SQLite - SQLite backend for Minion job queue



=head1 SYNOPSIS



  use Minion::Backend::SQLite;

  my $backend = Minion::Backend::SQLite->new('sqlite:test.db');



  # Minion

  use Minion;

  my $minion = Minion->new(SQLite => 'sqlite:test.db');



  # Mojolicious (via Mojolicious::Plugin::Minion)

  $self->plugin(Minion => { SQLite => 'sqlite:test.db' });



  # Mojolicious::Lite (via Mojolicious::Plugin::Minion)

  plugin Minion => { SQLite => 'sqlite:test.db' };



  # Share the database connection cache

  helper sqlite => sub { state $sqlite = Mojo::SQLite->new('sqlite:test.db') };

  plugin Minion => { SQLite => app->sqlite };



=head1 DESCRIPTION



L is a backend for L based on L.

All necessary tables will be created automatically with a set of migrations

named C. If no connection string or C<:temp:> is provided, the database

will be created in a temporary directory.



=head1 ATTRIBUTES



L inherits all attributes from L and

implements the following new ones.



=head2 dequeue_interval



  my $seconds = $backend->dequeue_interval;

  $backend    = $backend->dequeue_interval($seconds);



Interval in seconds between L"dequeue"> attempts. Defaults to C<0.5>.



=head2 sqlite



  my $sqlite = $backend->sqlite;

  $backend   = $backend->sqlite(Mojo::SQLite->new);



L object used to store all data.



=head1 METHODS



L inherits all methods from L and

implements the following new ones.



=head2 new



  my $backend = Minion::Backend::SQLite->new;

  my $backend = Minion::Backend::SQLite->new(':temp:');

  my $backend = Minion::Backend::SQLite->new('sqlite:test.db');

  my $backend = Minion::Backend::SQLite->new->tap(sub { $_->sqlite->from_filename('C:\\foo\\bar.db') });

  my $backend = Minion::Backend::SQLite->new(Mojo::SQLite->new);



Construct a new L object.



=head2 broadcast



  my $bool = $backend->broadcast('some_command');

  my $bool = $backend->broadcast('some_command', [@args]);

  my $bool = $backend->broadcast('some_command', [@args], [$id1, $id2, $id3]);



Broadcast remote control command to one or more workers.



=head2 dequeue



  my $job_info = $backend->dequeue($worker_id, 0.5);

  my $job_info = $backend->dequeue($worker_id, 0.5, {queues => ['important']});



Wait a given amount of time in seconds for a job, dequeue it and transition

from C to C state, or return C if queues were empty.

Jobs will be checked for in intervals defined by L"dequeue_interval"> until

the timeout is reached.



These options are currently available:



=over 2



=item id



  id => '10023'



Dequeue a specific job.



=item min_priority



  min_priority => 3



Do not dequeue jobs with a lower priority.



=item queues



  queues => ['important']



One or more queues to dequeue jobs from, defaults to C.



=back



These fields are currently available:



=over 2



=item args



  args => ['foo', 'bar']



Job arguments.



=item id



  id => '10023'



Job ID.



=item retries



  retries => 3



Number of times job has been retried.



=item task



  task => 'foo'



Task name.



=back



=head2 enqueue



  my $job_id = $backend->enqueue('foo');

  my $job_id = $backend->enqueue(foo => [@args]);

  my $job_id = $backend->enqueue(foo => [@args] => {priority => 1});



Enqueue a new job with C state.



These options are currently available:



=over 2



=item attempts



  attempts => 25



Number of times performing this job will be attempted, with a delay based on

L after the first attempt, defaults to C<1>.



=item delay



  delay => 10



Delay job for this many seconds (from now).



=item expire



  expire => 300



Job is valid for this many seconds (from now) before it expires. Note that this

option is B and might change without warning!



=item lax



  lax => 1



Existing jobs this job depends on may also have transitioned to the C

state to allow for it to be processed, defaults to C. Note that this

option is B and might change without warning!



=item notes



  notes => {foo => 'bar', baz => [1, 2, 3]}



Hash reference with arbitrary metadata for this job.



=item parents



  parents => [$id1, $id2, $id3]



One or more existing jobs this job depends on, and that need to have

transitioned to the state C before it can be processed.



=item priority



  priority => 5



Job priority, defaults to C<0>. Jobs with a higher priority get performed first.



=item queue



  queue => 'important'



Queue to put job in, defaults to C.



=back



=head2 fail_job



  my $bool = $backend->fail_job($job_id, $retries);

  my $bool = $backend->fail_job($job_id, $retries, 'Something went wrong!');

  my $bool = $backend->fail_job(

    $job_id, $retries, {msg => 'Something went wrong!'});



Transition from C to C state with or without a result, and if

there are attempts remaining, transition back to C with an

exponentially increasing delay based on L.



=head2 finish_job



  my $bool = $backend->finish_job($job_id, $retries);

  my $bool = $backend->finish_job($job_id, $retries, 'All went well!');

  my $bool = $backend->finish_job($job_id, $retries, {msg => 'All went well!'});



Transition from C to C state with or without a result.



=head2 history



  my $history = $backend->history;



Get history information for job queue.



These fields are currently available:



=over 2



=item daily



  daily => [{epoch => 12345, finished_jobs => 95, failed_jobs => 2}, ...]



Hourly counts for processed jobs from the past day.



=back



=head2 list_jobs



  my $results = $backend->list_jobs($offset, $limit);

  my $results = $backend->list_jobs($offset, $limit, {states => ['inactive']});



Returns the information about jobs in batches.



  # Get the total number of results (without limit)

  my $num = $backend->list_jobs(0, 100, {queues => ['important']})->{total};



  # Check job state

  my $results = $backend->list_jobs(0, 1, {ids => [$job_id]});

  my $state = $results->{jobs}[0]{state};



  # Get job result

  my $results = $backend->list_jobs(0, 1, {ids => [$job_id]});

  my $result = $results->{jobs}[0]{result};



These options are currently available:



=over 2



=item before



  before => 23



List only jobs before this id.



=item ids



  ids => ['23', '24']



List only jobs with these ids.



=item queues



  queues => ['important', 'unimportant']



List only jobs in these queues.



=item states



  states => ['inactive', 'active']



List only jobs in these states.



=item tasks



  tasks => ['foo', 'bar']



List only jobs for these tasks.



=back



These fields are currently available:



=over 2



=item args



  args => ['foo', 'bar']



Job arguments.



=item attempts



  attempts => 25



Number of times performing this job will be attempted.



=item children



  children => ['10026', '10027', '10028']



Jobs depending on this job.



=item created



  created => 784111777



Epoch time job was created.



=item delayed



  delayed => 784111777



Epoch time job was delayed to.



=item expires



  expires => 784111777



Epoch time job is valid until before it expires.



=item finished



  finished => 784111777



Epoch time job was finished.



=item id



  id => 10025



Job id.



=item lax



  lax => 0



Existing jobs this job depends on may also have failed to allow for it to be

processed.



=item notes



  notes => {foo => 'bar', baz => [1, 2, 3]}



Hash reference with arbitrary metadata for this job.



=item parents



  parents => ['10023', '10024', '10025']



Jobs this job depends on.



=item priority



  priority => 3



Job priority.



=item queue



  queue => 'important'



Queue name.



=item result



  result => 'All went well!'



Job result.



=item retried



  retried => 784111777



Epoch time job has been retried.



=item retries



  retries => 3



Number of times job has been retried.



=item started



  started => 784111777



Epoch time job was started.



=item state



  state => 'inactive'



Current job state, usually C, C, C or C.



=item task



  task => 'foo'



Task name.



=item time



  time => 78411177



Current time.



=item worker



  worker => '154'



Id of worker that is processing the job.



=back



=head2 list_locks



  my $results = $backend->list_locks($offset, $limit);

  my $results = $backend->list_locks($offset, $limit, {names => ['foo']});



Returns information about locks in batches.



  # Get the total number of results (without limit)

  my $num = $backend->list_locks(0, 100, {names => ['bar']})->{total};



  # Check expiration time

  my $results = $backend->list_locks(0, 1, {names => ['foo']});

  my $expires = $results->{locks}[0]{expires};



These options are currently available:



=over 2



=item names



  names => ['foo', 'bar']



List only locks with these names.



=back



These fields are currently available:



=over 2



=item expires



  expires => 784111777



Epoch time this lock will expire.



=item name



  name => 'foo'



Lock name.



=back



=head2 list_workers



  my $results = $backend->list_workers($offset, $limit);

  my $results = $backend->list_workers($offset, $limit, {ids => [23]});



Returns information about workers in batches.



  # Get the total number of results (without limit)

  my $num = $backend->list_workers(0, 100)->{total};



  # Check worker host

  my $results = $backend->list_workers(0, 1, {ids => [$worker_id]});

  my $host    = $results->{workers}[0]{host};



These options are currently available:



=over 2



=item before



  before => 23



List only workers before this id.



=item ids



  ids => ['23', '24']



List only workers with these ids.



=back



These fields are currently available:



=over 2



=item id



  id => 22



Worker id.



=item host



  host => 'localhost'



Worker host.



=item jobs



  jobs => ['10023', '10024', '10025', '10029']



Ids of jobs the worker is currently processing.



=item notified



  notified => 784111777



Epoch time worker sent the last heartbeat.



=item pid



  pid => 12345



Process id of worker.



=item started



  started => 784111777



Epoch time worker was started.



=item status



  status => {queues => ['default', 'important']}



Hash reference with whatever status information the worker would like to share.



=back



=head2 lock



  my $bool = $backend->lock('foo', 3600);

  my $bool = $backend->lock('foo', 3600, {limit => 20});



Try to acquire a named lock that will expire automatically after the given

amount of time in seconds. An expiration time of C<0> can be used to check if a

named lock already exists without creating one.



These options are currently available:



=over 2



=item limit



  limit => 20



Number of shared locks with the same name that can be active at the same time,

defaults to C<1>.



=back



=head2 note



  my $bool = $backend->note($job_id, {mojo => 'rocks', minion => 'too'});



Change one or more metadata fields for a job. Setting a value to C will

remove the field. It is currently an error to attempt to set a metadata field

with a name containing the characters C<.>, C<[>, or C<]>.



=head2 receive



  my $commands = $backend->receive($worker_id);



Receive remote control commands for worker.



=head2 register_worker



  my $worker_id = $backend->register_worker;

  my $worker_id = $backend->register_worker($worker_id);

  my $worker_id = $backend->register_worker(

    $worker_id, {status => {queues => ['default', 'important']}});



Register worker or send heartbeat to show that this worker is still alive.



These options are currently available:



=over 2



=item status



  status => {queues => ['default', 'important']}



Hash reference with whatever status information the worker would like to share.



=back



=head2 remove_job



  my $bool = $backend->remove_job($job_id);



Remove C, C or C job from queue.



=head2 repair



  $backend->repair;



Repair worker registry and job queue if necessary.



=head2 reset



  $backend->reset({all => 1});



Reset job queue.



These options are currently available:



=over 2



=item all



  all => 1



Reset everything.



=item locks



  locks => 1



Reset only locks.



=back



=head2 retry_job



  my $bool = $backend->retry_job($job_id, $retries);

  my $bool = $backend->retry_job($job_id, $retries, {delay => 10});



Transition job back to C state, already C jobs may also be

retried to change options.



These options are currently available:



=over 2



=item attempts



  attempts => 25



Number of times performing this job will be attempted.



=item delay



  delay => 10



Delay job for this many seconds (from now).



=item expire



  expire => 300



Job is valid for this many seconds (from now) before it expires. Note that this

option is B and might change without warning!



=item lax



  lax => 1



Existing jobs this job depends on may also have transitioned to the C

state to allow for it to be processed, defaults to C. Note that this

option is B and might change without warning!



=item parents



  parents => [$id1, $id2, $id3]



Jobs this job depends on.



=item priority



  priority => 5



Job priority.



=item queue



  queue => 'important'



Queue to put job in.



=back



=head2 stats



  my $stats = $backend->stats;



Get statistics for the job queue.



These fields are currently available:



=over 2



=item active_jobs



  active_jobs => 100



Number of jobs in C state.



=item active_locks



  active_locks => 100



Number of active named locks.



=item active_workers



  active_workers => 100



Number of workers that are currently processing a job.



=item delayed_jobs



  delayed_jobs => 100



Number of jobs in C state that are scheduled to run at specific time

in the future.



=item enqueued_jobs



  enqueued_jobs => 100000



Rough estimate of how many jobs have ever been enqueued.



=item failed_jobs



  failed_jobs => 100



Number of jobs in C state.



=item finished_jobs



  finished_jobs => 100



Number of jobs in C state.



=item inactive_jobs



  inactive_jobs => 100



Number of jobs in C state.



=item inactive_workers



  inactive_workers => 100



Number of workers that are currently not processing a job.



=item uptime



  uptime => undef



Uptime in seconds. Always undefined for SQLite.



=item workers



  workers => 200;



Number of registered workers.



=back



=head2 unlock



  my $bool = $backend->unlock('foo');



Release a named lock.



=head2 unregister_worker



  $backend->unregister_worker($worker_id);



Unregister worker.



=head1 BUGS



Report any issues on the public bugtracker.



=head1 AUTHOR



Dan Book 



=head1 COPYRIGHT AND LICENSE



This software is Copyright (c) 2015 by Dan Book.



This is free software, licensed under:



  The Artistic License 2.0 (GPL Compatible)



=head1 SEE ALSO



L, L



=cut