{"id":26694484,"url":"https://github.com/deeplearnphysics/spine_workshop_2024","last_synced_at":"2025-04-13T00:35:23.475Z","repository":{"id":248945089,"uuid":"830258810","full_name":"DeepLearnPhysics/spine_workshop_2024","owner":"DeepLearnPhysics","description":"Resources for the 2024 SPINE ML Workshop at Tufts","archived":false,"fork":false,"pushed_at":"2024-11-22T17:20:10.000Z","size":47143,"stargazers_count":1,"open_issues_count":1,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-13T00:35:00.937Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DeepLearnPhysics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-17T23:32:19.000Z","updated_at":"2024-09-27T15:34:06.000Z","dependencies_parsed_at":"2025-03-26T18:44:16.429Z","dependency_job_id":null,"html_url":"https://github.com/DeepLearnPhysics/spine_workshop_2024","commit_stats":null,"previous_names":["deeplearnphysics/spine_workshop_2024"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Fspine_workshop_2024","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Fspine_workshop_2024/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Fspine_workshop_2024/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Fspine_workshop_2024/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DeepLearnPhysics","download_url":"https://codeload.github.com/DeepLearnPhysics/spine_workshop_2024/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650460,"owners_count":21139670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-26T18:29:46.258Z","updated_at":"2025-04-13T00:35:23.455Z","avatar_url":"https://github.com/DeepLearnPhysics.png","language":"Jupyter Notebook","readme":"# SPINE Workshop 2024\n\nThis repository contains all necessary resources to participate in the 2024 SPINE workshop organized for the SBN and 2x2 ML reconstruction groups. This workshop aims to train new comers to use [SPINE](https://github.com/DeepLearnPhysics/spine), our machine-learning-based particle imaging detector reconstruction chain. You can find the workshop agenda [here](https://indico.slac.stanford.edu/event/8926/).\n\n## Software environment\n\nFor the workshop, we will use [this \"Docker container\"](https://hub.docker.com/layers/deeplearnphysics/larcv2/ub20.04-cuda11.6-pytorch1.13-larndsim/images/sha256-afe799e39e2000949f3f247ab73fe70039fb411cb301cb3c78678b68c22e37fb?context=explore).\n\nSome notes below:\n\n* The image is fairly large (multiple GBs). Please download in advance if you are using it locally. It is used in both NVIDIA GPU and CPU running mode of our software.\n* Supported GPUs include those with NVIDIA Volta (e.g. V100), Turing (e.g. RTX 2080Ti), and Ampere architectures (e.g. A100, RTX 3080). If you want an older architectures to be supported, such as Pascal, please [contact Kazu](mailto:kterao@slac.stanford.edu).\n* We assume basic knowledge about _software container_, in particular `Docker`. If you are learning for the first time, we recommend to use/learn about `Apptainer` ([website](https://apptainer.org/docs/user/latest/)) instead of `Docker`.\n    * You can pull a apptainer image as follows\n```shell\n$ apptainer pull docker://deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim\n```\nYou can now launch a shell inside the apptainer with\n```shell\n$ apptainer exec --bind /path/to/workshop/folder/ larcv2_ub20.04-cuda11.6-pytorch1.13-larndsim.sif bash\n```\nFor nersc:\n```shell\n$ salloc --nodes 1 --qos shared_interactive --time 00:30:00 --constraint gpu --gpus 1 --account=dune --image=deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim shifter /bin/bash\n```\n### Docker alternative\n\nYou can also pull the docker image using docker (easier on Mac and Windows) directly with:\n```shell\n$ docker pull deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim\n```\nTo see which images are present on your system, you can use docker images. It will look something like this:\n```shell\n$ docker images\nREPOSITORY                TAG                                     IMAGE ID       CREATED        SIZE\ndeeplearnphysics/larcv2   ub20.04-cuda11.6-pytorch1.13-larndsim   cd28cb3cd04b   2 months ago   20.8GB\n```\nto run a shell in your image, simply do:\n```shell\n$ docker run -i -t cd28cb3cd04b bash\n```\n\n* [Ask Francois](mailto:drielsma@slac.stanford.edu) for questions or a request for a separate tutorial if interested.\n\n## Resources\n\n1. The *configuration files* are packages with this repository.\n\n2. You can find *data files* for the examples used in this workshop under:\n- S3DF\n```shell\n/sdf/data/neutrino/public_html/spine_workshop/larcv/ # Example MPV/MPR LArCV files prior to reconstruction\n/sdf/data/neutrino/public_html/spine_workshop/reco/  # Reconstructed HDF5 files\n```\n- NERSC\n```shell\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/larcv/ # Example MPV/MPR LArCV files prior to reconstruction\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/reco/  # Reconstructed HDF5 files\n```\n- Public\n  - [Small LArCV files](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/larcv/) (Day 1)\n    - [Generic](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/larcv/generic_small.root)\n    - [ICARUS](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/larcv/icarus_small.root)\n    - [SBND](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/larcv/sbnd_small.root)\n    - [2x2](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/larcv/2x2_small.root)\n  - [Small corresponding reconstructed HDF5 files](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/) (Day 2)\n    - [Generic](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/generic_small_spine.h5)\n    - [ICARUS](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/icarus_small_spine.h5)\n    - [SBND](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/sbnd_small_spine.h5)\n    - [2x2](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/2x2_small_spine.h5)\n  - [Medium reconstructed HDF5 files](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/) (Days 3-4)\n    - [Generic](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/generic_medium_spine.h5)\n    - [ICARUS](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/icarus_medium_spine.h5)\n    - [SBND](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/sbnd_medium_spine.h5)\n  - [BNB nu + cosmics](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/icarus_bnb_corsika_small_spine.h5) (Day 5)\n  - [BNB nue + cosmics](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/reco/icarus_bnb_nue_corsika_small_spine.h5) (Day 5)\n\n3. The *network model parameters* for the inference tutorial can be found at:\n- S3DF\n```shell\n/sdf/data/neutrino/public_html/spine_workshop/weights/generic_snapshot-4999.ckpt # Generic\n/sdf/data/neutrino/public_html/spine_workshop/weights/icarus_snapshot-7999.ckpt # ICARUS\n/sdf/data/neutrino/public_html/spine_workshop/weights/sbnd_snapshot-1999.ckpt # SBND\n/sdf/data/neutrino/public_html/spine_workshop/weights/2x2_snapshot-3999.ckpt # 2x2\n```\n- NERSC\n```shell\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/weights/generic_snapshot-4999.ckpt # Generic\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/weights/icarus_snapshot-7999.ckpt # ICARUS\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/weights/sbnd_snapshot-1999.ckpt # SBND\n/global/cfs/cdirs/dune/users/drielsma/spine_workshop/weights/2x2_snapshot-3999.ckpt # 2x2\n```\n- Public\n  - Generic: [generic_snapshot-2999.ckpt](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/weights/generic_snapshot-4999.ckpt)\n  - ICARUS: [icarus_snapshot-7999.ckpt](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/weights/icarus_snapshot-7999.ckpt)\n  - SBND: [sbnd_snapshot-1999.ckpt](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/weights/sbnd_snapshot-1999.ckpt)\n  - 2x2: [2x2_snapshot-3999.ckpt](https://s3df.slac.stanford.edu/data/neutrino/spine_workshop/weights/2x2_snapshot-3999.ckpt)\n\n## Computing resource\nMost of the notebooks can be run strictly on CPU. The following notebooks will run significantly slower on CPU, however:\n- Training/validation notebook\n- Inference and HDF5 file making notebook\n\nFor all other notebooks, you can run them locally, provided that you download:\n- Apptainer container\n- Necessary data\n- [SPINE v0.1.0](https://github.com/DeepLearnPhysics/spine)\n\nTo gain access to GPUs:\n- Everyone participating in this workshop should have access to both S3DF or NERSC, if you do not, please reach out to [Francois](mailto:drielsma@slac.stanford.edu).\n  - SDF Jupyter ondemand: https://sdf.slac.stanford.edu/public/doc/#/\n  - S3DF Jupyter ondemand: https://s3df.slac.stanford.edu/public/doc/#/\n\n* SBN collaborators also have access to the Wilson Cluster at FNAL, equipped with GPUs. Below is a few commands to log-in and load `Apptainer` with which you can run a container image for the workshop (see the previous section). For how-to utilize the Wilson Cluster, refer to [their website](https://computing.fnal.gov/wilsoncluster/slurm-job-scheduler/) as well as [this](https://cdcvs.fnal.gov/redmine/projects/nova_reconstruction/wiki/The_Wilson_Cluster) and [that](https://cdcvs.fnal.gov/redmine/projects/nova_reconstruction/wiki/Step-by-step_guide_to_running_on_the_WC) documentation from NOvA (replace `nova` with `icarus` or `sbnd` and most commands should just work).\n\n```shell\n$ ssh $USER@wc.fnal.gov\n$ module load apptainer\n$ apptainer --version\napptainer version 3.6.4\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeplearnphysics%2Fspine_workshop_2024","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeeplearnphysics%2Fspine_workshop_2024","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeplearnphysics%2Fspine_workshop_2024/lists"}