{"id":13639721,"url":"https://github.com/matthewfeickert/distributed-inference-with-pyhf-and-funcX","last_synced_at":"2025-04-19T22:33:46.039Z","repository":{"id":40459613,"uuid":"342961649","full_name":"matthewfeickert/distributed-inference-with-pyhf-and-funcX","owner":"matthewfeickert","description":"Example code for vCHEP 2021 paper \"Distributed statistical inference with pyhf enabled through funcX\"","archived":false,"fork":false,"pushed_at":"2024-10-07T22:36:39.000Z","size":70,"stargazers_count":0,"open_issues_count":5,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-08T09:53:19.682Z","etag":null,"topics":["chep2021","funcx","pyhf"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matthewfeickert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-27T21:26:23.000Z","updated_at":"2021-10-05T02:46:52.000Z","dependencies_parsed_at":"2024-10-16T04:41:07.557Z","dependency_job_id":null,"html_url":"https://github.com/matthewfeickert/distributed-inference-with-pyhf-and-funcX","commit_stats":{"total_commits":27,"total_committers":5,"mean_commits":5.4,"dds":"0.40740740740740744","last_synced_commit":"0ed2617dd601888f3304bc1779c0f20ccc6af8c9"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matthewfeickert","download_url":"https://codeload.github.com/matthewfeickert/distributed-inference-with-pyhf-and-funcX/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223810680,"owners_count":17206812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chep2021","funcx","pyhf"],"created_at":"2024-08-02T01:01:03.820Z","updated_at":"2024-11-09T09:31:34.704Z","avatar_url":"https://github.com/matthewfeickert.png","language":"Python","funding_links":[],"categories":["Applications"],"sub_categories":["High Energy Physics"],"readme":"# Distributed Inference with `pyhf` and `funcX`\n\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/matthewfeickert/distributed-inference-with-pyhf-and-funcX/main.svg)](https://results.pre-commit.ci/latest/github/matthewfeickert/distributed-inference-with-pyhf-and-funcX/main)\n\nExample code for vCHEP 2021 paper \"Distributed statistical inference with pyhf enabled through funcX\"\n\n## Setup\n\nCreate a Python 3 virtual environment and then install the `pyhf` and `funcX` dependencies in `requirements.txt`.\n\n```\n(distributed-inference) $ python -m pip install --upgrade pip setuptools wheel\n(distributed-inference) $ python -m pip install -r requirements.txt\n```\n\n### Reproducible environment\n\nTo install a reproducible environment that is consistent down to the hash level, use `pip-compile` to compile a lock file from `requirements.txt` and install it following the [`pip-secure-install`][pip-secure-install] recommendations.\n\n```\n(distributed-inference) $ bash compile_dependencies.sh\n(distributed-inference) $ bash secure_install.sh\n```\n\n[pip-secure-install]: https://github.com/brettcannon/pip-secure-install\n\n### On XSEDE's EXPANSE\n\nOn EXPANSE, to use a Python 3.7+ runtime Conda must be used, so create a Conda environment from the `expanse-environment.yml` provided, which uses the different `requirements.txt` files to provide the dependencies.\n\n```console\n$ conda env create -f expanse-environment.yml\n$ conda activate distributed-inference\n```\n\nOnce a GPU session has been entered, source the `setup_expanse_funcx_test_env.sh` shell script to activate the environment and load all required modules\n\n```\n(distributed-inference) $ . setup_expanse_funcx_test_env.sh\n```\n\n#### Machine Configuration\n\nEXPANSE has the following Nvidia drivers and GPUs:\n\n```console\n$ nvcc --version\nnvcc: NVIDIA (R) Cuda compiler driver\nCopyright (c) 2005-2020 NVIDIA Corporation\nBuilt on Thu_Jun_11_22:26:38_PDT_2020\nCuda compilation tools, release 11.0, V11.0.194\nBuild cuda_11.0_bu.TC445_37.28540450_0\n$ nvidia-smi --list-gpus\nGPU 0: Tesla V100-SXM2-32GB (UUID: GPU-XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX)\n```\n\n## Run\n\nCreate a file named `endpoint_id.txt` in the top level of this repository and save your funcX endpoint ID into the file.\n\n```\n(distributed-inference) $ touch endpoint_id.txt\n```\n\nThis will be read in during the run.\n\nPass the config JSON file for the analysis you want to run to `fit_analysis.py`\n\n```\n(distributed-inference) $ python fit_analysis.py -c config/1Lbb.json -b numpy\n```\n\n```console\n$ python fit_analysis.py --help\nusage: fit_analysis.py [-h] [-c CONFIG_FILE] [-b BACKEND]\n\nconfiguration arguments provided at run time from the CLI\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -c CONFIG_FILE, --config-file CONFIG_FILE\n                        config file\n  -b BACKEND, --backend BACKEND\n                        pyhf backend str alias\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatthewfeickert%2Fdistributed-inference-with-pyhf-and-funcX/lists"}