{"id":21525908,"url":"https://github.com/sraashis/easytorch","last_synced_at":"2025-04-09T23:23:23.884Z","repository":{"id":38150823,"uuid":"210070612","full_name":"sraashis/easytorch","owner":"sraashis","description":"EasyTorch is a research-oriented pytorch prototyping framework with a straightforward learning curve. It is highly robust and contains almost everything needed to perform any state-of-the-art experiments. ","archived":false,"fork":false,"pushed_at":"2023-12-06T00:17:30.000Z","size":125906,"stargazers_count":30,"open_issues_count":4,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T01:12:43.538Z","etag":null,"topics":["convolutional-neural-networks","deep-learning","deep-learning-example","deep-learning-for-computer-vision","fast-pytorch-training","fundus-image-analysis","getting-started-with-pytorch","k-fold-cross-validation","pytorch","pytorch-for-medical-images","pytorch-google-colab","roc-auc","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sraashis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-09-22T00:40:30.000Z","updated_at":"2025-03-05T21:00:25.000Z","dependencies_parsed_at":"2023-12-26T06:37:58.664Z","dependency_job_id":"a5a8b551-332b-48ba-93a1-183363cf75a8","html_url":"https://github.com/sraashis/easytorch","commit_stats":{"total_commits":722,"total_committers":1,"mean_commits":722.0,"dds":0.0,"last_synced_commit":"ebdf8f9a2dc3e1795e95065a4bcd4c4e91a9f610"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sraashis%2Feasytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sraashis%2Feasytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sraashis%2Feasytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sraashis%2Feasytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sraashis","download_url":"https://codeload.github.com/sraashis/easytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248126857,"owners_count":21052066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","deep-learning","deep-learning-example","deep-learning-for-computer-vision","fast-pytorch-training","fundus-image-analysis","getting-started-with-pytorch","k-fold-cross-validation","pytorch","pytorch-for-medical-images","pytorch-google-colab","roc-auc","transfer-learning"],"created_at":"2024-11-24T01:39:18.737Z","updated_at":"2025-04-09T23:23:23.837Z","avatar_url":"https://github.com/sraashis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## A very lightweight framework on top of PyTorch with full functionality.\n\n#### **Just one way of doing things means no learning curve.**  ✅\n\n![Logo](assets/easytorchlogo.png)\n\n![PyPi version](https://img.shields.io/pypi/v/easytorch)\n[![YourActionName Actions Status](https://github.com/sraashis/easytorch/workflows/build/badge.svg)](https://github.com/sraashis/easytorch/actions)\n![Python versions](https://img.shields.io/pypi/pyversions/pybadges.svg)\n\n\u003chr /\u003e\n\n#### Installation\n\n1. `pip install --upgrade pip`\n2. `Install latest pytorch and torchvision from` [Pytorch](https://pytorch.org/)\n3. `pip install easytorch`\n\n* [MNIST](./examples/MNIST_easytorch_CNN.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github//sraashis/easytorch/blob/master/examples/MNIST_easytorch_CNN.ipynb)\n* [Retinal fundus image transfer learning tasks.](https://github.com/sraashis/retinal-fundus-transfer)\n* [Covid-19 chest x-ray classification.](https://github.com/sraashis/covidxfactory)\n* [DCGAN.](https://github.com/sraashis/gan-easytorch-celeb-faces)\n\n#### Let's start with something simple like MNIST digit classification:\n\n```python\nfrom easytorch import EasyTorch, ETRunner, ConfusionMatrix, ETMeter\nfrom torchvision import datasets, transforms\nimport torch.nn.functional as F\nimport torch\nfrom examples.models import MNISTNet\n\ntransform = transforms.Compose([\n    transforms.ToTensor(),\n    transforms.Normalize((0.1307,), (0.3081,))\n])\n\n\nclass MNISTTrainer(ETRunner):\n    def _init_nn_model(self):\n        self.nn['model'] = MNISTNet()\n\n    def iteration(self, batch):\n        inputs, labels = batch[0].to(self.device['gpu']).float(), batch[1].to(self.device['gpu']).long()\n\n        out = self.nn['model'](inputs)\n        loss = F.nll_loss(out, labels)\n        _, pred = torch.max(out, 1)\n\n        meter = self.new_meter()\n        meter.averages.add(loss.item(), len(inputs))\n        meter.metrics['cfm'].add(pred, labels.float())\n\n        return {'loss': loss, 'meter': meter, 'predictions': pred}\n\n    def init_experiment_cache(self):\n        self.cache['log_header'] = 'Loss|Accuracy,F1,Precision,Recall'\n        self.cache.update(monitor_metric='f1', metric_direction='maximize')\n\n    def new_meter(self):\n        return ETMeter(\n            cfm=ConfusionMatrix(num_classes=10),\n            device=self.device['gpu']\n        )\n\n\nif __name__ == \"__main__\":\n    train_dataset = datasets.MNIST('../data', train=True, download=True, transform=transform)\n    val_dataset = datasets.MNIST('../data', train=False, transform=transform)\n\n    dataloader_args = {'train': {'dataset': train_dataset}, 'validation': {'dataset': val_dataset}}\n    runner = EasyTorch(phase='train', batch_size=512,\n                       epochs=10, gpus=[0], dataloader_args=dataloader_args)\n    runner.run(MNISTTrainer)\n```\n\n### Run as:\n\n`python script.py -ph train -b 512 -e 10 -gpus 0`\n\n### ... with **20+** useful options. Check [here](./easytorch/config/) for full list.\n\n\u003chr /\u003e\n\n## General use case:\n\n#### 1. Define your trainer\n\n```python\nfrom easytorch import ETRunner, Prf1a, ETMeter, AUCROCMetrics\n\n\nclass MyTrainer(ETRunner):\n\n    def _init_nn_model(self):\n        self.nn['model'] = NeuralNetModel(out_size=self.conf['num_class'])\n\n    def iteration(self, batch):\n        \"\"\"Handle a single batch\"\"\"\n        \"\"\"Must have loss and meter\"\"\"\n        meter = self.new_meter()\n        ...\n        return {'loss': ..., 'meter': ..., 'predictions': ...}\n\n    def new_meter(self):\n        return ETMeter(\n            num_averages=1,\n            prf1a=Prf1a(),\n            auc=AUCROCMetrics(),\n            device=self.device['gpu']\n        )\n\n    def init_cache(self):\n        \"\"\"Will plot Loss in one plot, and Accuracy,F1_score in another.\"\"\"\n        self.cache['log_header'] = 'Loss|Accuracy,F1_score'\n\n        \"\"\"Model selection using validation set if present\"\"\"\n        self.cache.update(monitor_metric='f1', metric_direction='maximize')\n\n````\n\n* Method new_meter() returns ETMeter that takes any implementation of easytorch.meter.ETMetrics. Provided ones:\n    * easytorch.metrics.Prf1a() for binary classification that computes accuracy,f1,precision,recall, overlap/IOU.\n    * easytorch.metrics.ConfusionMatrix(num_classes=...) for multiclass classification that also computes global\n      accuracy,f1,precision,recall.\n    * easytorch.metrics.AUCROCMetrics for binary ROC-AUC score.\n\n#### 2. Define specification for your datasets:\n\n* EasyTorch automatically splits the training data in _**data_source**_ as specified by\n  _**split_ratio(-spl or --split-ratio 0.7, 0.15, 0.15, for train validation and test portion)**_ OR Custom splits in\n    1. Text files:\n        * data_source = \"/some/path/*.txt\", where it looks for 'train.txt', 'validation.txt', and 'test.txt' if phase is\n          _train_, and only 'test.txt' if phase is _test_\n    2. Json files:\n        * data_source = \"some/path/split.json\", where each split key has list of files as {'train': [], '\n          validation' :[], 'test':[]}\n    3. Just glob as data_source = \"some/path/**/*.txt\", must also provide split_ratio if phase = _train_\n\n```python\nfrom easytorch import ETDataset\n\n\nclass MyDataset(ETDataset):\n    def load_index(self, file):\n        \"\"\"(Optional) Load/Process something and add to diskcache as:\n                self.diskcahe.add(file, value)\"\"\"\n        \"\"\"This method runs in multiple processes by default\"\"\"\n\n        self.indices.append([file, 'something_extra'])\n\n    def __getitem__(self, index):\n        file = self.indices[index]\n        \"\"\"(Optional) Retrieve from diskcache as self.diskcache.get(file)\"\"\"\n\n        image =  # Todo # Load file/Image. \n        label =  # Todo # Load corresponding label.\n\n        # Extra preprocessing, if needed.\n        # Apply transforms, if needed.\n\n        return image, label\n```\n\n#### 3. Entry point (say main.py)\n### Run as:\n\n`python main.py -ph train -b 512 -e 10 -gpus 0`\n\nOne can also directly pass arguments as below which overrides all.\n```python\nfrom easytorch import EasyTorch\n\nrunner = EasyTorch(phase=\"train\", batch_size=4, epochs=10,\n                   gpus=[0], num_channel=1, \n                   num_class=2, data_source=\"\u003csome_data\u003e/data_split.json\")\nrunner.run(MyTrainer, MyDataset)\n```\n\n### All the best! Cheers! 🎉\n\n#### Cite the following papers if you use this library:\n\n```\n@article{deepdyn_10.3389/fcomp.2020.00035,\n\ttitle        = {Dynamic Deep Networks for Retinal Vessel Segmentation},\n\tauthor       = {Khanal, Aashis and Estrada, Rolando},\n\tyear         = 2020,\n\tjournal      = {Frontiers in Computer Science},\n\tvolume       = 2,\n\tpages        = 35,\n\tdoi          = {10.3389/fcomp.2020.00035},\n\tissn         = {2624-9898}\n}\n\n@misc{2202.02382,\n        Author       = {Aashis Khanal and Saeid Motevali and Rolando Estrada},\n        Title        = {Fully Automated Tree Topology Estimation and Artery-Vein Classification},\n        Year         = {2022},\n        Eprint       = {arXiv:2202.02382},\n}\n```\n\n### Feature Higlights:\n\n* DataHandle that is always available, and decoupled from other modules enabling easy\n  customization ([ETDataHandle](easytorch/data/data.py)).\n    * Use custom \u0026 complex data handling mechanism.\n* Simple lightweight logger/plotter.\n    * **Plot:** set log_header = 'Loss,F1,Accuracy' to plot in same plot or set log_header = 'Loss|F1,Accuracy' to plot\n      Loss in one plot, and F1,Accuracy in another plot.\n    * **Logs:** all arguments/generated data will be saved in logs.json file after the experiment finishes.\n* Gradient accumulation, automatic logging/plotting, model checkpointing, save everything.\n* Multiple metrics implementation at easytorch.metrics: Precision, Recall, Accuracy, Overlap, F1, ROC-AUC, Confusion\n  matrix\n* **For advanced training with multiple networks, and complex training steps,\n  click [here](assets/AdvancedTraining.md):**\n* **Implement custom metrics as [here](assets/CustomMetrics.md).**\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsraashis%2Feasytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsraashis%2Feasytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsraashis%2Feasytorch/lists"}