{"id":20315238,"url":"https://github.com/sysread/argon","last_synced_at":"2025-04-11T17:23:06.784Z","repository":{"id":5113134,"uuid":"6277297","full_name":"sysread/Argon","owner":"sysread","description":"A distributed parallel processing system for Perl","archived":false,"fork":false,"pushed_at":"2017-11-30T18:31:46.000Z","size":501,"stargazers_count":4,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-10T09:19:43.171Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"cloudbase/windows-openstack-imaging-tools","license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sysread.png","metadata":{"files":{"readme":"README.pod","changelog":"Changes","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-10-18T11:19:50.000Z","updated_at":"2018-01-28T15:29:19.000Z","dependencies_parsed_at":"2022-07-08T21:17:49.702Z","dependency_job_id":null,"html_url":"https://github.com/sysread/Argon","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysread%2FArgon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysread%2FArgon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysread%2FArgon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysread%2FArgon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sysread","download_url":"https://codeload.github.com/sysread/Argon/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248448124,"owners_count":21105243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T18:18:25.597Z","updated_at":"2025-04-11T17:23:06.756Z","avatar_url":"https://github.com/sysread.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"=pod\n\n=encoding UTF-8\n\n=head1 NAME\n\nArgon - Simple, fast, and flexible distributed computing\n\n=head1 VERSION\n\nversion 0.19\n\n=head1 DESCRIPTION\n\nArgon is a distributed processing platform built for Perl. It is designed to\noffer a simple, flexible, system for quickly building scalable systems with the\nminimum impact on existing workflows and code structure.\n\n=head1 QUICK START\n\nAn argon system is controlled by a I\u003cmanager\u003e process, whose job it is to\nschedule tasks among registered I\u003cworkers\u003e.\n\n=head2 MANAGER\n\nA manager process is started with C\u003car-manager\u003e. The manager, workers, and\nclients must all use the same key (a file containing a key phrase used for\nencryption).\n\n  ar-manager --host localhost --port 8000 --key path/to/secret --verbose 7\n\n=head2 WORKER\n\nWorkers are started with C\u003car-worker\u003e and must use the same C\u003ckey\u003e as the\nmanager.\n\n  ar-worker --mgr mgrhost:8000 --capacity 8 --key path/to/secret --verbose 7\n\n=head2 CLIENT\n\nConnecting to an Argon service is straightforward.\n\n  use Argon::Client;\n  use AnyEvent;\n\n  # Connect\n  my $cv = AnyEvent-\u003econdvar;\n\n  my $ar = Argon::Client-\u003enew(\n    host    =\u003e 'mgrhost',\n    port    =\u003e 8000,\n    keyfile =\u003e 'path/to/key',\n    ready   =\u003e $cv,\n    retry   =\u003e 1,\n  );\n\n  # Connected!\n  $cv-\u003erecv;\n\nA code ref (or any callable reference) may be passed using the C\u003cready\u003e\nparameter which will be called once the client is connected. The example uses a\ncondition variable (see L\u003cAnyEvent/CONDITION VARIABLES\u003e) to sleep until it\nis called, making the connection blocking.\n\n=head1 RUNNING TASKS\n\nOnce connected, there are a number of ways to schedule tasks with the manager,\nthe most basic being the L\u003cArgon::Client/queue\u003e method.\n\n  $client-\u003equeue('My::Task::Class', $arg_list, sub {\n    my $reply = shift;\n    my $result = $reply-\u003eresult;\n  });\n\nThere are a couple things to note here. The task class is any class that\ndefines both a C\u003cnew\u003e and a C\u003crun\u003e method. The C\u003c$arg_list\u003e will be passed to\nC\u003cnew\u003e. A subroutine is passed which is called when the result is ready. The\ncall to the C\u003cresult\u003e method will C\u003ccroak\u003e if the task failed.\n\nIf C\u003cretry\u003e was set when the client was connected, the task will retry on a\nlogarithmic backoff until the server has the capacity to process the task\n(see L\u003cArgon/PREDICTABLE PERFORMANCE DEGREDATION\u003e).\n\nIf the workers were started with the C\u003c--allow-eval\u003e switch, the client may\npass code references directly to be evaluated by the workers using the\nL\u003cArgon::Client/process\u003e method.\n\n  local $Argon::ALLOW_EVAL = 1;\n\n  $client-\u003eprocess(sub { ... }, $arg_list, sub {\n    ...\n  });\n\n=head1 PREDICTABLE PERFORMANCE DEGREDATION\n\nOne of the key problems with many task queue implementations is the manner in\nwhich the system recovers from an interruption in service. In most cases, tasks\ncontinue to pile up while the system is unavailable. Once the service is again\nready to process tasks, a significant backlog has built up, creating further\ndelay for new tasks added to the queue. This creates a log jam that is often\naccompanied by incidental service slowdowns that can be difficult to diagnose\n(for example, overloaded workers clearing out the backlog tie up the database,\ncausing other services to slow down).\n\nArgon prevents these cases by placing the responsibility for the backlog on the\nclient. When the manager determines that the system has reached max capacity,\nnew tasks are I\u003crejected\u003e until there is room in the queue. From the\nperspective of the client, there is still a delay in the processing of tasks,\nbut the task queue itself never becomes overloaded and the performance\ndegredation will never overflow onto neighborhing systems as a result.\n\nAnother adavantage of having a bounded queue is that clients are aware of the\nbacklog and may report this to callers. System administrators may effectively\nplan for and respond to increased load by spinning up new servers as needed\nbecause they can reliably predict the performance of the system under load\ngiven a reliable estimate of the cost imposed by the tasks being performed.\n\n=head1 AUTHOR\n\nJeff Ober \u003csysread@fastmail.fm\u003e\n\n=head1 COPYRIGHT AND LICENSE\n\nThis software is copyright (c) 2017 by Jeff Ober.\n\nThis is free software; you can redistribute it and/or modify it under\nthe same terms as the Perl 5 programming language system itself.\n\n=cut\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsysread%2Fargon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsysread%2Fargon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsysread%2Fargon/lists"}