{"id":51505747,"url":"https://github.com/jakobbossek/expyrimenter","last_synced_at":"2026-07-08T00:01:59.083Z","repository":{"id":369963004,"uuid":"1291070089","full_name":"jakobbossek/expyrimenter","owner":"jakobbossek","description":"Easy management of parallel commputing jobs using different parallelisation backends.","archived":false,"fork":false,"pushed_at":"2026-07-07T16:32:46.000Z","size":35,"stargazers_count":0,"open_issues_count":15,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-07T17:14:47.069Z","etag":null,"topics":["benchmarking","experimentation","multi-core","optimisation-algorithms","parallel-computing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jakobbossek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-06T13:29:31.000Z","updated_at":"2026-07-07T16:33:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/jakobbossek/expyrimenter","commit_stats":null,"previous_names":["jakobbossek/expyrimenter"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/jakobbossek/expyrimenter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakobbossek%2Fexpyrimenter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakobbossek%2Fexpyrimenter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakobbossek%2Fexpyrimenter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakobbossek%2Fexpyrimenter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jakobbossek","download_url":"https://codeload.github.com/jakobbossek/expyrimenter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakobbossek%2Fexpyrimenter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35246561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-07T02:00:07.222Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","experimentation","multi-core","optimisation-algorithms","parallel-computing"],"created_at":"2026-07-08T00:01:58.257Z","updated_at":"2026-07-08T00:01:59.043Z","avatar_url":"https://github.com/jakobbossek.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ExPyrimenter\n\nLightweight Python library for **managing and executing parameterizable, inherently parallel computing jobs**.\n\nThe typical workflow is as follows:\n\n1. Create a **registry**. The registry is the core component of the library. It manages the collection of jobs and serves as the primary interface for interacting with them.\n2. Populate the registry with an **experimental design**. The design is a table in which each row defines a single job (or experiment). Each job specifies the parameters required to execute a computational task.\n3. Define a **runner**. A runner is a function that accepts an integer job identifier and a dictionary of parameters. It executes the corresponding job and writes the results to the job's designated output directory.\n4. Execute the jobs in parallel using either a built-in or custom parallelization backend (for example, on a multi-core local machine).\n5. Monitor job progress from another Python process, including the number of running, completed, and failed jobs.\n6. Collect and analyze the results. For benchmarking optimization algorithms, the library provides built-in utilities for result aggregation.\n\n## Key Features\n\n* **Lightweight:** Focuses on the essential functionality without unnecessary complexity.\n* **Extensible:** Supports multiple parallelization backends. Currently, the library includes an implementation based on the [joblib library](https://pypi.org/project/joblib/), with additional backends easily added.\n\n## Installation\n\nThe package is currently under development. At a later stage it will be available via PyPI. For the time being you can install the development version.\nYou will need *conda* to get it running easily. On Macs I suggest to use [homebrew 🍺](https://brew.sh) to install *miniconda*\n```bash\nbrew install --cask miniconda \n```\n\nOnce done you can proceed. First, clone the repository and navigate into the respective folder in your favourite terminal application. Once there type the following to install and activate the environment:\n```bash\nconda env create -f environment.yml\nconda activate codebase\n```\n\n## Example\n\n```python\nfrom Experiments.Registry import Registry\nimport os\nimport random\n\n# Registry folder in the file system \npath = \"expyrimenter-registry\"\n\n# Build the registry\nreg = Registry(path = path, overwrite = True)\n\n# Runner function. Expects the job's ID and a dictionary of parameters.\ndef my_runner(jobid, params):\n    # For showcasing we force job 1 to fail\n    if jobid == 1:\n        raise Exception()\n\n    # Some arbitrary result\n    return random.uniform(1, 2)\n\n# Load the test design\nreg.load_design(\"designs/test_design.csv\")\n\nprint(f\"No. of jobs in registry: {reg.size()}\")\nprint(reg.get_job(400))\n\n# Run some jobs\nreg.run(my_runner, jobids = [1, 4, 6])\n\n# Note that job 1 failed by design\nprint(reg.get_failed())\n\n# Jobs 4 and 6 finished\nprint(reg.get_done())\n\n# Try to run again. Jobs 4 and 6 are already finished and will thus be skipped!\nreg.run(my_runner, jobids = [1, 4, 6])\nprint(reg.get_failed())\nprint(reg.get_done())\n\n# Force jobs 4 and 6 to re-run\nreg.run(my_runner, jobids = [4, 6, 10], rerun = True)\n```\n\n## Contact\n\nPlease address questions to the main author Jakob Bossek \u003cj.bossek@gmail.com\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakobbossek%2Fexpyrimenter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjakobbossek%2Fexpyrimenter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakobbossek%2Fexpyrimenter/lists"}