{"id":24714310,"url":"https://github.com/nvsl/delegate-function","last_synced_at":"2026-04-19T05:31:28.288Z","repository":{"id":85666331,"uuid":"588297101","full_name":"NVSL/delegate-function","owner":"NVSL","description":"Flexibly execute python methods in an external process","archived":false,"fork":false,"pushed_at":"2023-10-25T22:34:32.000Z","size":74,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-10-30T10:44:17.540Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NVSL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-12T19:44:13.000Z","updated_at":"2023-01-12T19:46:52.000Z","dependencies_parsed_at":"2023-05-05T15:17:05.023Z","dependency_job_id":"24ddb632-d542-44d1-a98c-bf6d3c76d2d4","html_url":"https://github.com/NVSL/delegate-function","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NVSL/delegate-function","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVSL%2Fdelegate-function","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVSL%2Fdelegate-function/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVSL%2Fdelegate-function/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVSL%2Fdelegate-function/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NVSL","download_url":"https://codeload.github.com/NVSL/delegate-function/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVSL%2Fdelegate-function/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31996265,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-27T08:16:16.491Z","updated_at":"2026-04-19T05:31:28.262Z","avatar_url":"https://github.com/NVSL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Delegate Function\n\nDelegate Function is a simple Python package that lets you invoke a\nmethod on an object but have the code run in a different process.\n\nIt differs from Python's `multiprocess`  modules in two ways:\n\n1. It allows much more flexibilty in how the delegate process is created and run.\n2. It doesn't support shared state between the current process and subprocess.\n\n#1 means that you can, for instance, run the method over `ssh`,\nthrough a batch system like `slurm`, in a different directory, under\n`sudo`, etc.\n\nYou can also easily compose these methods, so you run a method in a\ndocker container started on a remote machine via a batch system.\n\n#2 means there are quite a few restrictions in what changes the method\ncan make.  It also means it's not useful for things like parallel\nexecution.\n\n# Motivation\n\nThis package grew out a very specific use case:  A user would write some untrusted code that needed to be run on the user's behalf in a\nsandboxed environment.  The challenge was that the path to accessing the sandboxed environment was somewhat complex.  For instance, it might involve:\n\n1.  Connecting to a remote machine via `ssh`\n2.  Then using `sudo` to run a priviliged script that would...\n3.  Submit a job to a batch runnning service (e.g., Slurm) that would...\n4.  Create a sandboxed docker container that would...\n5.  Run the user's code.\n\nTo handle thing, I built the notion of \"delegate\" objects that could be chained together.  There are delegates `ssh`, `docker`, `sudo`, and `slurm`, and implementing new ones is reasonably easy.\nThey can be \"stacked\" arbitrarily. \n\n# Limitations\n\nThe implementation is not super-sophisticated, so it\nshould really only modify the object's state.  \n\nIn particular, changes to files will not be preserved.  However, Python's `io.StringIO` and `io.BytesIO` objects are both picklable, \nwhich allows for some neat tricks.  For instance, you can create a zip archive in an `io.BytesIO` object, unpack it on the other side, \nrun your code, and zip up the results. \n\n\n# Installation\n\n```\npip install .\n```\n\n# How to Use It\n\nUntil I write some docs, checkout `tests/test_delegate_function.py`.\n\n# Running The Tests\n\nTesting everything is quite involved because it needs a slurm cluster that you can access, a couple of test accounts, and a ssh host.\n\nA dockerized version of all of that is in `testing-setup`.\n\nI assume you are developing from inside docker:\n\nTo get things going:\n\n1. Create a docker volume called `shared_scratch` and mount it on your dev container at `/scratch`  make sure `/scratch` is readable/writeable by everyone.\n2. `cd testing-setup; docker-compose build`\n3. Run `testing-setup/setup_test.sh`.  This will create some users, and make it possible for your dev container to submit slurm jobs.\n4. `cd testing-setup; SCRATCH=cse141pp-root_shared_scratch DEV_NETWORK=cse141pp-root_default docker-compose up --detach` -- 'DEV_NETWORK` should be network the slurm containers should connect to.  It should be accessible from your dev container.\n5. `cd tests; pytest`\n\nIf the SSH tests fail due to host key issues, you can try running `testing-setup/fix_sh.sh`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvsl%2Fdelegate-function","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvsl%2Fdelegate-function","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvsl%2Fdelegate-function/lists"}