{"id":24545435,"url":"https://github.com/greyside/errand-boy","last_synced_at":"2025-10-25T04:39:48.779Z","repository":{"id":13400081,"uuid":"16088489","full_name":"greyside/errand-boy","owner":"greyside","description":"A memory-conscious alternative to os.fork() and subprocess.Popen().","archived":false,"fork":false,"pushed_at":"2020-02-05T08:57:16.000Z","size":54,"stargazers_count":36,"open_issues_count":2,"forks_count":6,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-28T21:35:14.705Z","etag":null,"topics":["fork","linux","memory-management","pool","popen","python","unix","unix-like"],"latest_commit_sha":null,"homepage":"https://pypi.python.org/pypi/errand-boy/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/greyside.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-01-21T00:58:13.000Z","updated_at":"2025-02-28T18:14:35.000Z","dependencies_parsed_at":"2022-09-14T03:23:16.647Z","dependency_job_id":null,"html_url":"https://github.com/greyside/errand-boy","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greyside%2Ferrand-boy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greyside%2Ferrand-boy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greyside%2Ferrand-boy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greyside%2Ferrand-boy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/greyside","download_url":"https://codeload.github.com/greyside/errand-boy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248571841,"owners_count":21126522,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fork","linux","memory-management","pool","popen","python","unix","unix-like"],"created_at":"2025-01-22T21:19:48.441Z","updated_at":"2025-10-25T04:39:48.683Z","avatar_url":"https://github.com/greyside.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"==========\nerrand-boy\n==========\n\n.. image:: https://travis-ci.org/greyside/errand-boy.svg?branch=master\n    :target: https://travis-ci.org/greyside/errand-boy\n.. image:: https://coveralls.io/repos/greyside/errand-boy/badge.png?branch=master\n    :target: https://coveralls.io/r/greyside/errand-boy?branch=master\n\n----------------\nWhat does it do?\n----------------\n\nUses Python multiprocessing to maintain a pool of worker processes used to execute arbitrary terminal commands.\n\n-------------------------------\nWhy not use subprocess.Popen()?\n-------------------------------\n\nUnder the hood subprocess.Popen() uses os.fork(), which copies the currently running process' memory before launching the command you want to run. If your process uses a lot of memory, such as a Celery worker using an eventlet pool, this can cause a \"Cannot allocate memory\" error.\n\nerrand-boy still uses subprocess.Popen(), but tries to keep a low memory footprint. Your celery greenthread workers can communicate with it via asynchronous network calls.\n\nFurther reading:\n\n#. http://stackoverflow.com/a/13329386/241955\n#. http://stackoverflow.com/a/14942111/241955\n\n-----\nSetup\n-----\n\nInstall:\n\n    pip install errand-boy\n\n    # optional\n    pip install setproctitle\n\n-----\nUsage\n-----\n\nRun tests::\n\n    cd errand-boy/\n    python -m unittest discover\n\nRun server::\n\n    python -m errand_boy.run\n\nRun client (useful for testing/debugging)::\n\n    python -m errand_boy.run 'ls -al'\n\nUse the client in your code::\n\n    from errand_boy.transports.unixsocket import UNIXSocketTransport\n    \n    \n    errand_boy_transport = UNIXSocketTransport()\n    \n    stdout, stderr, returncode = errand_boy_transport.run_cmd('ls -al')\n    \n    print stdout\n    print stderr\n    print returncode\n\nUse a subprocess.Popen-like interface::\n\n    from errand_boy.transports.unixsocket import UNIXSocketTransport\n    \n    \n    errand_boy_transport = UNIXSocketTransport()\n    \n    # Attribute accesses and function calls on objects retrieved via a session\n    # result in a call to the errand-boy server, unless that object is a string\n    # or number type.\n    with errand_boy_transport.get_session() as session:\n        subprocess = session.subprocess\n        \n        # Here, subprocess.Popen is actually a reference to the actual objects\n        # on the errand-boy server. subprocess.PIPE is an int.\n        process = subprocess.Popen('ls -al', stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, close_fds=True)\n        \n        # Here, process_stdout and process_stderr are strings and returncode is\n        # an int, so their actual values are returned instead of references to\n        # the remote objects. This means it's safe to use these values later on\n        # outside of the current session.\n        process_stdout, process_stderr = process.communicate()\n        returncode = process.returncode\n    \n    print stdout\n    print stderr\n    print returncode\n    \n    # Since the session has been closed, trying this will result in an error:\n    print process.returncode\n    # raised errand_boy.exceptions.SessionClosedError()\n\nRun load tests::\n\n    python -m errand_boy.run --max-accepts=0\n\n    pip install Fabric locustio\n    cd errand-boy/\n    fab locust_local\n\n--------------------------------\nDoes it work in other languages?\n--------------------------------\n\nThe client/server use an HTTP-inspired protocol, but the data that's sent back and forth is currently serialized using Python's Pickle format. Support could be added for other serialization types though.\n\n-----------\nDevelopment\n-----------\n\nFurther reading:\n\n* http://stackoverflow.com/questions/18414020/memory-usage-keep-growing-with-pythons-multiprocessing-pool\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreyside%2Ferrand-boy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgreyside%2Ferrand-boy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreyside%2Ferrand-boy/lists"}