{"id":16953804,"url":"https://github.com/ivotron/popper-canopie-paper","last_synced_at":"2025-06-24T10:05:43.751Z","repository":{"id":37625264,"uuid":"207291022","full_name":"ivotron/popper-canopie-paper","owner":"ivotron","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-22T04:06:36.000Z","size":20368,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-29T11:43:05.446Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivotron.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-09T11:14:01.000Z","updated_at":"2020-10-10T09:47:24.000Z","dependencies_parsed_at":"2023-01-21T11:46:46.245Z","dependency_job_id":null,"html_url":"https://github.com/ivotron/popper-canopie-paper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivotron%2Fpopper-canopie-paper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivotron%2Fpopper-canopie-paper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivotron%2Fpopper-canopie-paper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivotron%2Fpopper-canopie-paper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivotron","download_url":"https://codeload.github.com/ivotron/popper-canopie-paper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247354948,"owners_count":20925526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T22:07:58.255Z","updated_at":"2025-04-05T15:22:59.348Z","avatar_url":"https://github.com/ivotron.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CANOPIE paper artifacts\n\nRepository for the submission:\n\n\u003e Jayjeet Chakraborty, Carlos Maltzahn and Ivo Jimenez. _Enabling \n\u003e Seamless Execution of Computational and Data Science Workflows on \n\u003e HPC and Cloud with the Popper Container-native Automation Engine_.\n\nThe repository is structured as follows:\n\n* `workflows/`. Contains the workflow used in the \"Case Study\" section \n  of the paper. This workflow trains a model for the MNIST dataset, \n  and can be executed locally or on Kubernetes or Slurm clusters.\n\n* `paper/`. Contains the paper and the static assets used in the paper \n  along with a Popper workflow to plot the results and build the paper \n  after running an experiment.\n\n* `results/`. The location where result files get generated after \n  running the experiment workflows. It also contains a Jupyter \n  notebook to plot the results which is executed by the workflow here.\n\n## Installing Popper\nTo install Popper, run the following in your terminal:\n\n```bash\npython3 -m venv venv\nsource venv/bin/activate\npip install popper\n```\n\n## Building the paper\n\n```bash\npopper run -f paper/.popper.yml\n```\n\n## Running the workflow\n\n### Local\n\n```bash\npopper run -f workflows/mnist/.popper.yml\n```\n\n### Kubernetes\n\nGet access to a Kubernetes cluster if you don't already have access to \none. Popper assumes that a [`kubectl` configuration](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file is \navailable.\n\nTo execute the workflow:\n\n```bash\npopper run -f workflows/mnist/.popper.yml -c workflows/mnist/config_k8s.yml\n```\n\n### Slurm\n\nSingularity is required on the Slurm nodes. After logging into the \nSlurm cluster, installing popper and cloning this repository, execute:\n\nExecute the workflow,\n\n```bash\npopper run -f workflows/mnist/.popper.yml -c workflows/mnist/config_slurm.yml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivotron%2Fpopper-canopie-paper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivotron%2Fpopper-canopie-paper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivotron%2Fpopper-canopie-paper/lists"}