{"id":26694520,"url":"https://github.com/deeplearnphysics/lartpc_mlreco3d","last_synced_at":"2025-04-13T00:35:34.963Z","repository":{"id":38080966,"uuid":"184113799","full_name":"DeepLearnPhysics/lartpc_mlreco3d","owner":"DeepLearnPhysics","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-09T23:48:22.000Z","size":4882,"stargazers_count":9,"open_issues_count":20,"forks_count":32,"subscribers_count":6,"default_branch":"develop","last_synced_at":"2025-04-13T00:35:18.375Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DeepLearnPhysics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-29T17:24:01.000Z","updated_at":"2024-10-14T03:38:51.000Z","dependencies_parsed_at":"2024-03-02T01:25:29.060Z","dependency_job_id":"8674f86e-917d-45ff-ac4c-71638ab5d73d","html_url":"https://github.com/DeepLearnPhysics/lartpc_mlreco3d","commit_stats":null,"previous_names":[],"tags_count":67,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Flartpc_mlreco3d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Flartpc_mlreco3d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Flartpc_mlreco3d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepLearnPhysics%2Flartpc_mlreco3d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DeepLearnPhysics","download_url":"https://codeload.github.com/DeepLearnPhysics/lartpc_mlreco3d/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650451,"owners_count":21139671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-26T18:30:00.806Z","updated_at":"2025-04-13T00:35:34.939Z","avatar_url":"https://github.com/DeepLearnPhysics.png","language":"Python","readme":"[![Build Status](https://travis-ci.com/DeepLearnPhysics/lartpc_mlreco3d.svg?branch=develop)](https://travis-ci.com/DeepLearnPhysics/lartpc_mlreco3d)\n\n[![Documentation Status](https://readthedocs.org/projects/lartpc-mlreco3d/badge/?version=latest)](https://lartpc-mlreco3d.readthedocs.io/en/latest/?badge=latest)\n\n# A Machine Learning Pipeline for LArTPC Data\n\nThis repository contains code used for training and running machine learning models on LArTPC data.\n\n![Full chain](./images/full_chain.png)\n\n## Installation\nWe recommend using a Singularity or Docker containers pulled from `deeplearnphysics/larcv2`: https://hub.docker.com/r/deeplearnphysics/larcv2. It needs to have at least\n* MinkowskiEngine,\n* larcv2,\n* pytorch_geometric,\n* PyTorch,\n* standard Python scientific libraries.\n\nThen git clone this repository and have fun!\n\n## Usage\nBasic example:\n```python\n# assume that lartpc_mlreco3d folder is on python path\nfrom mlreco.main_funcs import process_config, train\nimport yaml\n# Load configuration file\nwith open('lartpc_mlreco3d/config/train_uresnet.cfg', 'r') as f:\n    cfg = yaml.load(f, Loader=yaml.Loader)\nprocess_config(cfg)\n# train a model based on configuration\ntrain(cfg)\n```\n\n* Some tutorials are available at https://deeplearnphysics.org/lartpc_mlreco3d_tutorials/.\n* More technical documentation is available at https://lartpc-mlreco3d.readthedocs.io/.\n\n### Example Configuration Files\n\nFor your inspiration, the following standalone configurations are available in the `config` folder:\n\n| Configuration name            | Model          |\n| ------------------------------|----------------|\n| `train_uresnet.cfg`           | UResNet alone  |\n| `train_uresnet_ppn.cfg`       | UResNet + PPN  |\n| `train_graph_spice.cfg`       | GraphSpice     |\n| `train_grappa_shower.cfg`     | GrapPA for shower fragments clustering (particle fragments -\u003e particle clusters) |\n| `train_grappa_interaction.cfg`| GrapPA for interaction clustering (particle clusters -\u003e interactions) |\n\nSwitching from train to test mode is as simple as switching `trainval.train: False` for all models. The only exception at the moment is GraphSpice, for which an example test configuration is provided (`test_graph_spice.cfg`).\n\nTypically in a configuration file the first things you may want to edit will be:\n* `batch_size` (in 2 places)\n* `weight_prefix` (where to save the model checkpoints)\n* `log_dir` (where to save the logs)\n* `iterations`\n* `model_path` (checkpoint to load, optional)\n* `train` (boolean)\n* `gpus` (leave empty '' if you want to run on CPU)\n\n\nIf you want more information stored, such as network output tensors and post-processing outcomes, you can use `analysis` (scripts) and `outputs` (formatters)\nto store them in CSV format and run your custom analysis scripts (see folder `analysis`).\n\nThis section has described how to use the contents of this repository to train variations of what has already been implemented.  To add your own models and analysis, you will want to know how to contribute to the `mlreco` module.\n\n### Running A Configuration File\n\nMost basic usage is to use the `run` script.  From the `lartpc_mlreco3d` folder:\n```bash\nnohup python3 bin/run.py train_gnn.cfg \u003e\u003e log_gnn.txt \u0026\n```\nThis will train a GNN specified in `config/train_gnn.cfg`, save checkpoints and logs to specified directories in the `cfg`, and output `stderr` and `stdout` to `log_gnn.txt`\n\nYou can generally load a configuration file into a python dictionary using\n```python\nimport yaml\n# Load configuration file\nwith open('lartpc_mlreco3d/config/train_uresnet.cfg', 'r') as f:\n    cfg = yaml.load(f, Loader=yaml.Loader)\n```\n\n### Reading a Log\n\nA quick example of how to read a training log, and plot something\n```python\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfname = 'path/to/log.csv'\ndf = pd.read_csv(fname)\n\n# plot moving average of accuracy over 10 iterations\ndf.accuracy.rolling(10, min_periods=1).mean().plot()\nplt.ylabel(\"accuracy\")\nplt.xlabel(\"iteration\")\nplt.title(\"moving average of accuracy\")\nplt.show()\n\n# list all column names\nprint(df.columns.values)\n```\n\n### Recording network output or running analysis\nWe use [LArTPC MLReco3D Analysis Tools](./analysis/README.md) for all inference and high-level analysis related work. \n\n## Repository Structure\n* `bin` contains very simple scripts that run the training/inference functions.\n* `config` has various example configuration files.\n* `docs` Documentation (in progress)\n* `mlreco` the main code lives there!\n* `test` some testing using Pytest\n* `analysis`: [LArTPC MLReco3D Analysis Tools](./analysis/README.md), a pure python interface for inference, high-level analysis, and visualization using the full chain. \n\nPlease consult the README of each folder respectively for more information.\n\n## Contributing\n\nBefore you start contributing to the code, please see the [contribution guidelines](contributing.md).\n\n### Adding a new model\nYou may be able to re-use a fair amount of code, but here is what would be necessary to do everything from scratch:\n\n1. Make sure you can load data you need.\n\nParsers already exist for a variety of sparse tensor outputs as well as particle outputs.\n\nThe most likely place you would need to add something is to `mlreco/iotools/parsers.py`.\n\nIf the data you need is fundamentally different from data currently used, you may also need to add a collation function to `mlreco/iotools/collates.py`\n\n2. Include your model\n\nYou should put your model in a new file in the `mlreco/models` folder.\n\nAdd your model to the dictionary in `mlreco/models/factories.py` so it can be found by the configuration parsers.\n\nAt this point, you should be able to train your model using a configuration file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeplearnphysics%2Flartpc_mlreco3d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeeplearnphysics%2Flartpc_mlreco3d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeplearnphysics%2Flartpc_mlreco3d/lists"}