Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/sysread/anyevent-processpool

An asynchronous process pool for Perl using AnyEvent
https://github.com/sysread/anyevent-processpool

Last synced: about 1 month ago
JSON representation

An asynchronous process pool for Perl using AnyEvent

Awesome Lists containing this project

README 

        
=pod



=encoding UTF-8



=head1 NAME



AnyEvent::ProcessPool - Asynchronously runs code concurrently in a pool of perl processes



=head1 VERSION



version 0.07



=head1 SYNOPSIS



  use AnyEvent::ProcessPool;



  my $pool = AnyEvent::ProcessPool->new(

    workers => 8,

    limit   => 10,

    include => ['lib', 'some/lib/path'],

  );



  my $condvar = $pool->async(sub{

    # do task type stuff...

  });



  # Block until result is ready

  my $result = $condvar->recv;



=head1 DESCRIPTION



Executes code using a pool a forked Perl subprocesses. Supports configurable

pool size, automatically restarting processes after a configurable number of

requests, and closures (with the caveat that changes are not propagated back to

the parent process).



=head1 CONSTRUCTOR



=head2 workers



Required attribute specifying the number of worker processes to launch.

Defaults to the number of CPUs.



=head2 limit



Optional attribute that causes a worker process to be restarted after

performing C tasks. This can be useful when calling code which may be

leaky. When unspecified or set to zero, worker processes will only be restarted

if it unexpectedly fails.



=head2 include



An optional array ref of paths to add to the perl command string used to start

the sub-process worker.



=head1 METHODS



=head2 async



Executes the supplied code ref in a worker sub-process. Remaining (optional)

arguments are passed unchanged to the code ref in the worker process. Returns a

L that will block and return the task

result when C is called on it.



Alternately, the name of a task class may be supplied. The class must implement

the methods 'new' (as a constructor) and 'run'. When using a task class, the

arguments will be passed to the constructor (new) and the result of 'run' will

be returned.



  # With an anonymous subroutine

  my $cv = $pool->async(sub{ ... });



  # With a code ref

  my $cv = $pool->async(\&do_stuff);



  # With optional parameter list

  my $cv = $pool->async(sub{ ... }, $arg1, $arg2, ...);



  # With a task class

  my $cv = $pool->async('My::Task', $arg1, ...);



=head2 join



Blocks until all pending tasks have completed. This does not prevent new tasks

from being queued while waiting (for example, in the callback of an already

queued task's condvar).



=head1 PIPELINES



Pipelinelines are alternative way of using the process pool. See

L for details.



  use AnyEvent::ProcessPool::Pipeline;



  pipeline workers => 4,

    in  { get_next_task() }

    out { do_stuff_with_result(shift->recv) };



=head1 DIAGNOSTICS



=head2 Task errors



Error messages resulting from a C or C in task code executed in a

worker process are rethrown in the parent process when the condition variable's

C method is called.



=head2 "AnyEvent::ProcessPool::Worker: ..." (warning)



When a worker sub-process emits output to C, the process pool warns

the message out to its own C.



=head2 "error launching worker process: ..."



Thrown when a worker sub-process failed to launch due to an execution error.



=head2 "worker terminated in response to signal: ..."



Thrown when a worker sub-process exits as a result of a signal received.



=head2 "worker terminated with non-zero exit status: ..."



Thrown when a worker sub-process terminates with a non-zero exit code. The

worker will be automatically restarted.



=head1 INCOMPATIBILITIES



Will not work on MSWin32 (although Cygwin should be fine) due to lack of

support for non-blocking writes to process pipes (see notes in

L.



=head1 SEE ALSO



=over



=item L



Highly reliable, but somewhat arcane, blocking, and can be tricky to integrate

into non-blocking code.



=item L



Similar in function, but runs only under L (which as of 6.513 has

experimental support for 5.22).



=back



=head1 AUTHOR



Jeff Ober 



=head1 COPYRIGHT AND LICENSE



This software is copyright (c) 2018 by Jeff Ober.



This is free software; you can redistribute it and/or modify it under

the same terms as the Perl 5 programming language system itself.



=cut