{"id":21079312,"url":"https://github.com/davidzhang73/pytorch-lightning-template","last_synced_at":"2025-05-16T08:32:30.245Z","repository":{"id":163371925,"uuid":"610113095","full_name":"DavidZhang73/pytorch-lightning-template","owner":"DavidZhang73","description":"A template for simple deep learning projects using Lightning","archived":false,"fork":false,"pushed_at":"2024-12-17T05:52:21.000Z","size":51,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T22:36:21.174Z","etag":null,"topics":["best-practices","config","deep-learning","pytorch","pytorch-lightning","reproducibility","template"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DavidZhang73.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-06T05:41:00.000Z","updated_at":"2025-03-31T03:41:53.000Z","dependencies_parsed_at":"2024-01-16T01:54:52.409Z","dependency_job_id":"910e2daa-74e4-44c3-af3b-b61f7fa6fb16","html_url":"https://github.com/DavidZhang73/pytorch-lightning-template","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidZhang73%2Fpytorch-lightning-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidZhang73%2Fpytorch-lightning-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidZhang73%2Fpytorch-lightning-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidZhang73%2Fpytorch-lightning-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DavidZhang73","download_url":"https://codeload.github.com/DavidZhang73/pytorch-lightning-template/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254496320,"owners_count":22080691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["best-practices","config","deep-learning","pytorch","pytorch-lightning","reproducibility","template"],"created_at":"2024-11-19T19:45:39.711Z","updated_at":"2025-05-16T08:32:26.192Z","avatar_url":"https://github.com/DavidZhang73.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Pytorch Lightning Template\n\n[![Pytorch](https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white)](https://pytorch.org/get-started/locally/)\n[![Pytorch Lightning](https://img.shields.io/badge/-Lightning-ffffff?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iNDEiIGhlaWdodD0iNDgiIHZpZXdCb3g9IjAgMCA0MSA0OCIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTIwLjQ5OTIgMEwwIDEyVjM2TDIwLjUgNDhMNDEgMzZWMTJMMjAuNDk5MiAwWk0xNi45NTAxIDM2LjAwMTZMMTkuMTA4OSAyNi42ODU2TDE0LjI1NDggMjEuODkyTDI0LjA3OTEgMTEuOTk5MkwyMS45MTYzIDIxLjMyOTZMMjYuNzQ0NCAyNi4wOTc2TDE2Ljk1MDEgMzYuMDAxNloiIGZpbGw9IiM3OTJFRTUiLz4KPC9zdmc+Cg==)](https://lightning.ai/docs/pytorch/stable/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)\n\nA template for simple deep learning projects using Lightning\n\nEnglish | [中文](./README_ZH.md)\n\n\u003c/div\u003e\n\n## Introduction\n\n[PyTorch Lightning](https://lightning.ai/docs/pytorch/stable/) is to deep learning project development as [MVC](https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller) frameworks (such as [Spring](https://spring.io/), [Django](https://www.djangoproject.com/), etc.) are to website development. While it is possible to implement everything from scratch and achieve maximum flexibility (especially since [PyTorch](https://pytorch.org/) and its ecosystem are already quite straightforward), using a framework can help you quickly implement prototypes with guidance from [\"best practices\"](#best-practice) (personal opinion) to save a lot of boilerplate code through re-usability, and focus on scientific innovation rather than engineering challenges. This template is built using the full Lightning suite, follows the principle of [Occam's razor](https://en.wikipedia.org/wiki/Occam%27s_razor), and is friendly to researchers. It also includes a simple handwritten digit recognition task using the MNIST dataset. The repository also contains some [Tips](#tips), for reference.\n\n## \"Best Practice\"\n\n### Using [Pytorch Lightning](https://lightning.ai/docs/pytorch/stable/) as a deep learning framework:\n\nMost of the deep learning code can be divided into the following three parts([Reference [Chinese]](https://zhuanlan.zhihu.com/p/120331610)):\n\n1. Research code: This part pertains to the model and generally deals with customizations of the model's structure and training. In `Lightning`, this code is abstracted as the `pl.LightningModule` class. While dataset definition can also be included in this part, it is not recommended as it is not relevant to the experiment and should be included in `pl.LightningDataModule` instead.\n\n2. Engineering code: This part of the code is essential for its high repeatability, such as setting early stopping, 16-bit precision, and GPU distributed training. In `Lightning`, this code is abstracted as the `pl.Trainer` class.\n\n3. Non-essential code: This code is helpful in conducting experiments but is not directly related to the experiment itself, and can even be omitted. For example, gradient checking and outputting logs to `TensorBoard`. In Lightning, this code is abstracted as the `Callbacks` class, which is registered to `pl.Trainer`.\n\nThe advantages of using `Lightning`:\n\n1. Custom training processes and learning rate adjustment strategies can be implemented through various [hook functions](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#hooks) in `pl.LightningModule`.\n\n2. The model and data no longer need to be explicitly designated for devices (`tensor.to`, `tensor.cuda`, etc.). `pl.Trainer` handles this automatically, thereby supporting various [acceleration devices such as CPU, GPU, and TPU](https://lightning.ai/docs/pytorch/latest/extensions/accelerator.html).\n\n3. `pl.Trainer` implements various [training strategies](https://lightning.ai/docs/pytorch/latest/extensions/strategy.html), such as automatic mixed precision training, multi-GPU training, and distributed training.\n\n4. `pl.Trainer` implements multiple [callbacks](https://lightning.ai/docs/pytorch/latest/extensions/callbacks.html) such as automatic model saving, automatic config saving, and automatic visualization result saving.\n\n### Using [Pytorch Lightning CLI](https://lightning.ai/docs/pytorch/stable/cli/lightning_cli.html#lightning-cli) as a command-line tool:\n\n1. Using `lightning_cli` as the program entry point, model, data, and training parameters can be set through configuration files or command-line parameters, thereby achieving quick switching between multiple experiments.\n\n2. `pl.LightningModule.save_hyperparameters()` saves the model's hyperparameters and automatically generates a command-line parameter table, eliminating the need for tools such as [`argparse`](https://docs.python.org/3/library/argparse.html) or [`hydra`](https://hydra.cc/).\n\n### Using [Torchmetrics](https://torchmetrics.readthedocs.io/en/stable/) as a metric computation tool:\n\n1. `Torchmetrics` provides multiple metric calculation methods such as `Accuracy`, `Precision`, and `Recall`.\n\n2. It is integrated with `Lightning` and is compatible with parallel training strategies. Data is automatically aggregated to the main process for metric computation.\n\n### [Optional] Using [WanDB](https://wandb.ai/) to track experiments\n\n### Project Architecture\n\n```mermaid\n    graph TD;\n        A[LightningCLI]---B[LightningModule]\n        A---C[LightningDataModule]\n        B---D[models]\n        B---E[metrics]\n        B---F[...]\n        C---G[dataloaders]\n        G---H[datasets]\n```\n\n### File Structure\n\n```text\n├── configs                  # Configuration files\n│   ├── data                 # Dataset configuration\n│   │   └── mnist.yaml       # Example configuration for MNIST dataset\n│   ├── model                # Model configuration\n│   │   └── simplenet.yaml   # Example configuration for SimpleNet model\n│   └── default.yaml         # Default configuration\n├── data                     # Dataset directory\n├── logs                     # Log directory\n├── notebooks                # Jupyter Notebook directory\n├── scripts                  # Script directory\n│   └── clear_wandb_cache.py # Example script to clear wandb cache\n├── src                      # Source code directory\n│   ├── callbacks            # Callbacks directory\n│   │   └── __init__.py\n│   ├── data_modules         # Data module directory\n│   │   ├── __init__.py\n│   │   └── mnist.py         # Example data module for MNIST dataset\n│   ├── metrics              # Metrics directory\n│   │   └── __init__.py\n│   ├── models               # Model directory\n│   │   ├── __init__.py\n│   │   └── simplenet.py     # Example SimpleNet model\n│   ├── modules              # Module directory\n│   │   ├── __init__.py\n│   │   └── mnist_module.py  # Example MNIST module\n│   ├── utils                # Utility directory\n│   │   ├── __init__.py\n│   │   └── cli.py           # CLI tool\n│   ├── __init__.py\n│   └── main.py              # Main program entry point\n├── .env.example             # Example environment variable file\n├── .gitignore               # Ignore files for git\n├── .project-root            # Project root indicator file for pyrootutils\n├── LICENSE                  # Open source license\n├── pyproject.toml           # Configuration file for Black and Ruff\n├── README.md                # Project documentation\n├── README_PROJECT.md        # Project documentation template\n├── README_ZH.md             # Project documentation in Chinese\n└── requirements.txt         # Dependency list\n```\n\n## Usage\n\n### Installation\n\n```bash\n# Clone project\ngit clone https://github.com/DavidZhang73/pytorch-lightning-template \u003cproject_name\u003e\ncd \u003cproject_name\u003e\n\n# [Optional] Create a conda virtual environment\nconda create -n \u003cenv_name\u003e python=\u003c3.8|3.9|3.10\u003e\nconda activate \u003cenv_name\u003e\n\n# [Optional] Use mamba instead of conda to speed up\nconda install mamba -n base -c conda-forge\n\n# [Optional] Install PyTorch according to the website to get GPU support\n# https://pytorch.org/get-started/\n\n# Install dependencies\npip install -r requirements.txt\n```\n\n### Configuration\n\n1. Define dataset by inheriting `pl.LightningDataModule` in `src/data_module`.\n2. Define dataset configuration file in `configs/data` as parameters for the custom `pl.LightningDataModule`.\n3. Define the model by inheriting `nn.Module` in `src/models`.\n4. Define metrics by inheriting `torchmetrics.Metric` in `src/metrics`.\n5. Define training module by inheriting `pl.LightningModule` in `src/modules`.\n6. Define the configuration file for the training module in `configs/model` as parameters for the custom `pl.LightningModule`.\n7. Configure `pl.trainer`, logs and other parameters in `configs/default.yaml`.\n\n### Run\n\n**Fit**\n\n```bash\npython src/main.py fit -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --trainer.logger.name exp1\n```\n\n**Validate**\n\n```bash\npython src/main.py validate -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --trainer.logger.name exp1\n```\n\n**Test**\n\n```bash\npython src/main.py test -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --trainer.logger.name exp1\n```\n\n**Inference**\n\n```bash\npython src/main.py predict -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --trainer.logger.name exp1\n```\n\n**Debug**\n\n```bash\npython src/main.py fit -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --trainer.fast_dev_run true\n```\n\n**Resume**\n\n```bash\npython src/main.py fit -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --ckpt_path \u003cckpt_path\u003e --trainer.logger.id exp1_id\n```\n\n## Tips\n\n### `print_config`\n\nUsing the `print_config` functionality of `jsonargparse`, you can obtain the parsed arguments and generate default `yaml` files. However, it is necessary to first configure the `yaml` files for `data` and `model`.\n\n```bash\npython src/main.py fit -c configs/data/mnist.yaml -c configs/model/simplenet.yaml --print_config\n```\n\n[Prepare a config file for the CLI](https://lightning.ai/docs/pytorch/stable/cli/lightning_cli_advanced.html#prepare-a-config-file-for-the-cli)\n\n### Customized `LightningCLI`\n\nThis template implements a custom `CLI` (`CustomLightningCLI`) to achieve the following functions,\n\n- When starting the program, the configuration file is automatically saved to the corresponding log directory, for `WandbLogger` only.\n- When starting the program, save configurations for optimizer and scheduler to loggers.\n- When starting the program, the default configuration file is automatically loaded.\n- After the test is completed, the `checkpoint_path` used for testing is printed.\n- Add some command line parameters:\n    - `--ignore_warnings` (default: `False`): Ignore all warnings.\n    - `--test_after_fit` (default: `False`): Automatically test after each training.\n    - `--git_commit_before_fit` (default: `False`): `git commit` before each training, the commit message is `{logger.name}_{logger.version}`, for `WandbLogger` only.\n\n[CONFIGURE HYPERPARAMETERS FROM THE CLI (EXPERT)](https://lightning.ai/docs/pytorch/stable/cli/lightning_cli_expert.html)\n\n### Limit `numpy` number of threads\n\nWhen running on a server, especially when the CPU has a lot of cores (\u003e=24), you may encounter the problem of too many `numpy` processes, which may cause the experiment to inexplicably hang. You can limit the number of `numpy` processes by setting environment variables (in the `.env` file).\n\n```text\nOMP_NUM_THREADS=8\nMKL_NUM_THREADS=8\nGOTO_NUM_THREADS=8\nNUMEXPR_NUM_THREADS=8\nOPENBLAS_NUM_THREADS=8\nMKL_DOMAIN_NUM_THREADS=8\nVECLIB_MAXIMUM_THREADS=8\n```\n\n\u003e `.env` file is automatically loaded to environment by [`pyrootutils`](https://github.com/ashleve/pyrootutils) via [`python-dotenv`](https://github.com/theskumar/python-dotenv).\n\n[Stack Overflow: Limit number of threads in numpy](https://stackoverflow.com/questions/30791550/limit-number-of-threads-in-numpy)\n\n### Clear `wandb` cache\n\nWhen you delete an experiment from the `wandb` web page, the cache of the experiment still exists in the local `wandb` directory, you can use the `scripts/clear_wandb_cache.py` script to clear the cache.\n\n[Wandb Python Documentation](https://docs.wandb.ai/ref/python/)\n\n## References\n\nInspired by,\n\n- [deep-learning-project-template](https://github.com/Lightning-AI/deep-learning-project-template)\n- [lightning-hydra-template](https://github.com/ashleve/lightning-hydra-template)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidzhang73%2Fpytorch-lightning-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidzhang73%2Fpytorch-lightning-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidzhang73%2Fpytorch-lightning-template/lists"}