{"id":18354678,"url":"https://github.com/nextbitlabs/pytorch-template","last_synced_at":"2025-09-23T22:05:14.685Z","repository":{"id":98590716,"uuid":"239481092","full_name":"nextbitlabs/pytorch-template","owner":"nextbitlabs","description":"Template for a deep learning project based on PyTorch","archived":false,"fork":false,"pushed_at":"2024-07-25T11:01:29.000Z","size":122,"stargazers_count":11,"open_issues_count":2,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-06T12:37:07.918Z","etag":null,"topics":["deep-learning","pytorch","template"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nextbitlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-10T10:08:51.000Z","updated_at":"2024-05-07T19:07:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"de62d308-734d-4e47-8009-4f423ec4e4b8","html_url":"https://github.com/nextbitlabs/pytorch-template","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"purl":"pkg:github/nextbitlabs/pytorch-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextbitlabs%2Fpytorch-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextbitlabs%2Fpytorch-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextbitlabs%2Fpytorch-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextbitlabs%2Fpytorch-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nextbitlabs","download_url":"https://codeload.github.com/nextbitlabs/pytorch-template/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nextbitlabs%2Fpytorch-template/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276655671,"owners_count":25680971,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-23T02:00:09.130Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","pytorch","template"],"created_at":"2024-11-05T22:04:51.113Z","updated_at":"2025-09-23T22:05:14.631Z","avatar_url":"https://github.com/nextbitlabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyTorch Template\n\nCode and documentation template for PyTorch research projects.\nThis repository is intended to be cloned at the beginning of any\nnew research deep learning project based on PyTorch.\n\nEvery TODO comment in the code indicates a portion of the code\nthat should be adapted for every specific project.\nThe rest of the code should usually remain almost unchanged.\n\n## Getting Started\n\nThese instructions will get you a copy of the project up and running\non your local machine for development and testing purposes.\n\nThe project folder, including also files excluded from git versioning,\nhas the following structure:\n\n```\npytorch-template/                   [main folder]\n│   .gitignore                      [files ignored by git]\n│   LICENSE                         [code license]\n│   README.md                       [this file]\n│   requirements.txt                [package dependencies]\n│   setup.py                        [package setup script]\n│\n├───data                            [data folder excluded from git tracking]\n│   │   targets.csv                 [targets for train, dev and test data]\n│   │\n│   ├───train\n│   │       ...\n│   ├───dev\n│   │       ...\n│   └───test\n│           ...\n│\n├───docs                            [documentation folder]\n│       ...\n│\n└───pytorch_template                [package source code folder]\n        __main__.py                 [package command-line interface]\n        ...\n```\n\nYou should comply to this structure in all your projects,\nin particular you should structure the `data` folder containing your dataset\naccording to the hierarchy shown. \n\n### Prerequisites\n\nIn order to run the code you need to have Python 3.6 installed.\n\n### Installing\n\nYou can install the package on MacOS/Linux with the following commands:\n```\ngit clone https://github.com/nextbitlabs/pytorch-template.git\ncd pytorch-template\npython3 setup.py sdist\npython3 setup.py bdist_wheel\npip3 install --no-index --find-links=dist pytorch_template -r requirements.txt\n```\n\nHere data are synthetic so, in order to generate them run:\n```\npython3 generate_data.py\n```\n\n## Usage\n\nA command line interface is available to easily interact with the package.\nIt is included inside the package in the file `__main__.py`.\n\nTo see more details about the command line interface\nit is possible to show the help page using the command:\n```\npython3 -m pytorch_template --help\n``` \n\nThe available commands are:\n- `ingest`: preprocess raw data and export it in a suitable format for model\ntraining;\n- `train`: train the deep learning model on ingested data;\n- `restore`: restore the training from a saved checkpoint;\n- `eval`: evaluate the model on ingested validation data;\n- `test`: produce model output on a single raw data sample.\n\nEvery command has its separate help page that can be visualized with\n```\npython3 -m pytorch_template \u003ccommand\u003e --help\n```\n\n### Command `ingest`\n\nThe ingestion phase is useful if preprocessing is computationally expensive and\nmany transformations are required. Here, for example, it is not really necessary\nbut it is included to show the code structure.\n\nIn some cases an additional `safe-ingest` can be used to check and assure labels \ncoherence among the different dataset splits or to perform transformations\nthat depend on other splits. Here it is not needed because the\nset of labels is not fixed since the example task is a regression.\n\n#### Examples\n\nOnly the training set and the development set have to be ingested\nand that can be do with the following lines:\n```\npython3 -m pytorch_template ingest data train\npython3 -m pytorch_template ingest data dev\n```\n\nFor more details on the usage you can access the help page with the command\n```\npython3 -m pytorch_template ingest --help\n```\n\n### Command `train`\n\nThe training phase has always the same structure and the template is built\nto keep all the tried models in files separated from the main training function.\n\nThe path to the best weight checkpoint according to the metric is printed\nto console at the end of the computation.\n\n#### Examples\n\nThe command has many optional training-related parameters commonly tuned by the \nexperimenter, like `batch-size`, `epochs`, `lr`.\n\nThe most basic training can be performed specifying just the directory containing\nthe dataset, already split in `train` (compulsory) and `dev` (optional) folders\nusing the default values for the other parameters.\n```\npython3 -m pytorch_template train data/tensors\n```\n\nAn equivalent form of the previous command with all the default values\nmanually specified is:\n```\npython3 -m pytorch_template train \\\n    data/tensors \\\n    --output-dir . \\\n    --batch-size 20 \\\n    --epochs 40 \\\n    --lr 0.1\n```\n\nFor more details on the usage you can access the help page with the command\n```\npython3 -m pytorch_template train --help\n```\n\n### Command `restore`\n\nWhen the model has not converged at the end of the training phase, it\ncan be useful to restore it from the last saved checkpoint and that is exactly\nthe role of this command.\n\n#### Examples\n\n\nThe command has the same optional parameters of the `train` command.\nIt just has an additional compulsory parameter: the path to the checkpoint model\nto be restored.\n\nThe most basic restored training can be performed specifying just the directory\ncontaining the dataset, already split in `train` (compulsory) and `dev` (optional)\nfolders, and the checkpoint path using the default values for the other parameters.\n```\npython3 -m pytorch_template restore \\\n    runs/\u003csecfromepochs\u003e/checkpoints/model-\u003cepoch\u003e-\u003cmetric\u003e.ckpt data/tensors\n```\n\nAn equivalent form of the previous command with all the default values\nmanually specified is:\n```\npython3 -m pytorch_template restore \\\n    runs/\u003csecfromepochs\u003e/checkpoints/model-\u003cepoch\u003e-\u003cmetric\u003e.ckpt \\\n    data/tensors \\\n    --output-dir . \\\n    --batch-size 20 \\\n    --epochs 40 \\\n    --lr 0.1\n```\n\nFor more details on the usage you can access the help page with the command\n```\npython3 -m pytorch_template restore --help\n```\n\n### Command `eval`\n\nThe `eval` command reproduces the validation performed at the end of every epoch during the training phase.\nIt is particularly useful when many datasets are available to evaluate the transfer learning performances.\n\n#### Examples\n\nThe evaluation can be performed specifying just the model checkpoint\nto be evaluated and the directory containing the dataset, provided of a `dev` sub-folders.\nThe batch size of evaluation batches can be manually specified otherwise its default\nvalue is 20.\n\nA full call to the command is:\n```\npython3 -m pytorch_template eval \\\n    runs/\u003csecfromepochs\u003e/checkpoints/model-\u003cepoch\u003e-\u003cmetric\u003e.ckpt \\\n    data/tensors \\\n    --batch-size 20\n```\n\nFor more details on the usage you can access the help page with the command\n```\npython3 -m pytorch_template eval --help\n```\n\n### Command `test`\n\nThe `test` command preforms the inference on a single file.\n\n#### Examples\n\nThe test of the model is performed specifying the model checkpoint to be evaluated\nand the path to a sample, for example:\n```\npython3 -m pytorch_template test \\\n    runs/\u003csecfromepochs\u003e/checkpoints/model-\u003cepoch\u003e-\u003cmetric\u003e.ckpt \\\n    data/test/\u003csample\u003e.pt\n```\n\nFor more details on the usage you can access the help page with the command\n```\npython3 -m pytorch_template test --help\n```\n\n## Performances\n\nThe model converges to perfect predictions using default parameters.\n\n## Deployment\n\nThe template can be deployed on an NGC optimized instance, here we list\nthe steps necessary to configure it on a AWS EC2 **g4dn.xlarge** instance\non the **NVIDIA Deep Learning AMI** environment.\n\n1. Log in via ssh following the instructions on the EC2 Management Dashboard.\n2. Clone the repo `pytorch-template` in the home directory.\n3. Download the most update PyTorch container running \n`docker pull nvcr.io/nvidia/pytorch:YY.MM-py3`\n(replace “YY” and “MM” with the most recent version updated monthly)\n4. Create a container with\n```\ndocker run --gpus all --name template -e HOME=$HOME -e USER=$USER \\\n    -v $HOME:$HOME -p 6006:6006 --shm-size 60G -it nvcr.io/nvidia/pytorch:YY.MM-py3\n```\nAt the end of the procedure you will gain access to a terminal on a Docker\ncontainer configured to work on the GPU and you could simply run the commands\nabove leveraging the speed of parallel computing.\n\nThe `$HOME` directory on the Docker container is linked to the `$HOME` directory\nof the host machine, so the repository can be found in the `$HOME`, similarly the\nport 6006 used by TensorBoard is remapped from the container to the port 6006\nof the host machine.\n\nUseful commands to interact with the Docker container are:\n- `docker start template`: start the container;\n- `docker exec -it template bash`: open a terminal on the container;\n- `docker stop template`: stop the container;\n- `docker rm template`: remove the container.\n\nIn order to monitor training you can run the following commands from the container console:\n- `watch -n 1 nvidia-smi` to monitor GPU usage;\n- `tensorboard --logdir runs/\u003crun_id\u003e --bind_all` to start Tensorboard.\n\n## Other \nThe template also includes an implementation of a cool new optimizer,\n[Ranger](https://medium.com/@lessw/2dc83f79a48d). \nRanger uses the [Lookahead](https://arxiv.org/abs/1907.08610) optimization method \ntogether with the [RAdam](https://arxiv.org/abs/1908.03265) optimizer.\nIt is not used here, as it is way too slow for such a simple model,\nbut it reportedly performs better than other Adam variants on deeper models. \n\nYou can use it simply by calling:\n ```\nfrom pytorch_template.models.optimizer.ranger import Ranger\n\noptimizer = Ranger(module.parameters())\n```\n\nIf you want, you can specify many more hyper-parameters. \nIf you use a learning rate scheduler, you should make sure that the learning rate remains\nconstant for a rather long time, in order to let RAdam start correctly and to take\nadvantage of LookAhead exploration.\n\nAn implementation of the [Mish](https://arxiv.org/abs/1908.08681) activation function is\nalso included. Mish seems to perform slightly better than ReLu when training deep models,\nand works well in conjunction with Ranger. \n\nTo use Mish, you just need to call it:\n```\nfrom pytorch_template.models.mish import Mish\n\nmish = Mish()\n```\n\n## License\n\nThis project is licensed under Apache License 2.0,\nsee the [LICENSE](LICENSE) file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnextbitlabs%2Fpytorch-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnextbitlabs%2Fpytorch-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnextbitlabs%2Fpytorch-template/lists"}