{"id":13738476,"url":"https://github.com/tchaton/pytorch2lightning","last_synced_at":"2025-09-11T20:33:24.006Z","repository":{"id":48284166,"uuid":"389645060","full_name":"tchaton/pytorch2lightning","owner":"tchaton","description":null,"archived":false,"fork":false,"pushed_at":"2021-08-03T12:07:21.000Z","size":13377,"stargazers_count":15,"open_issues_count":2,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-30T07:51:23.228Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tchaton.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-26T13:36:26.000Z","updated_at":"2024-02-27T03:38:27.000Z","dependencies_parsed_at":"2022-08-26T07:51:02.471Z","dependency_job_id":null,"html_url":"https://github.com/tchaton/pytorch2lightning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2Fpytorch2lightning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2Fpytorch2lightning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2Fpytorch2lightning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2Fpytorch2lightning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tchaton","download_url":"https://codeload.github.com/tchaton/pytorch2lightning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232666171,"owners_count":18557991,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T03:02:23.558Z","updated_at":"2025-01-06T02:55:02.767Z","avatar_url":"https://github.com/tchaton.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Converting PyTorch 2 Lightning Examples\n\nThe repository will show you how to:\n* Convert a pure `PyTorch Convolutional Neural Network Classifier` trained on MNIST to [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning).\n* Extend Pure PyTorch trivially with Lightning best practice features.\n* Seamlessly scale your training in the cloud with [Grid.ai](https://www.grid.ai/) - No code changes.\n* Learn about [Lighting Flash](https://github.com/PyTorchLightning/lightning-flash) and its 15+ production ready tasks. \n\nFind below PyTorch Community Voices | PyTorch Lightning | William Falcon \u0026 Thomas Chaton presenting this repository.\n\n[![Alt text](https://img.youtube.com/vi/A1bkh4gNDJA/0.jpg)](https://www.youtube.com/watch?v=A1bkh4gNDJA)\n\n## Bare MNIST Classifier\n![Minst Dataset](https://miro.medium.com/max/800/1*LyRlX__08q40UJohhJG9Ow.png)\n\n* [PyTorch](bare_mnist/pytorch.py) | 127 lines \n* [Lightning](bare_mnist/lightning.py) | 101 lines \n\n## Add DDP Support\n* [PyTorch](ddp_mnist/pytorch.py) | 184 lines\n* [Lightning](ddp_mnist/lightning.py) | 102 lines: -82 lines\n\n## Add DDP Spawn Support\n* [PyTorch](ddp_mnist_spawn/pytorch.py) | 196 lines\n* [Lightning](ddp_mnist_spawn/lightning.py) | 105 lines: -91 lines\n\n## Add Accumulated Gradients Support\n* [PyTorch](ddp_mnist_accumulate_gradients/pytorch.py) | +198 lines \n* [Lightning](ddp_mnist_accumulate_gradients/lightning.py) | 106 lines: -92 lines\n\n## Add Profiling Support\n* [PyTorch](ddp_mnist_accumulate_gradients_profiler/pytorch.py) | +226 lines \n* [Lightning](ddp_mnist_accumulate_gradients_profiler/lightning.py) | 106 lines: -120 lines\n\n## Add DeepSpeed, FSDP, Multiple Loggers, Mutliple Profilers, TorchScript, Loop Customization, Fault Tolerant Training, etc ....\n* [PyTorch](https://github.com/PyTorchLightning/pytorch-lightning) | requires a huge number of addtional lines. You `definitely` do not  want to do that :tired_face: \n* [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) | Still ~ 106 lines. Let's keep it simple. :rocket:\n\nLearn more with [Lighting Docs](https://pytorch-lightning.readthedocs.io/en/stable/).\n\nPyTorch Lightning 1.4 is out ! Here is our [CHANGELOG](https://github.com/PyTorchLightning/pytorch-lightning/releases/tag/1.4.0).\n\nDon't forget to :star: [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning).\n\n# Training on [Grid.ai](https://www.grid.ai/)\n\n[Grid.ai](https://www.grid.ai/) is a ML Platform from the creators of [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) that enables you to train Machine Learning code without worrying about infrastructure. \n\nLearn more with [Grid.ai Docs](https://docs.grid.ai/platform/about-these-features/multi-node)\n\n### 1. Install Lightning-Grid\n\n```bash\npip install lightning-grid --upgrade\n```\n\n### 2. SEAMLESSLY TRAIN 100s OF MACHINE LEARNING MODELS ON THE CLOUD FROM YOUR LAPTOP - NO CODE CHANGES\n\n\n```bash\ngrid run --instance_type 4_M60_8gb ddp_mnist_grid/lightning.py --trainer.max_epochs 2 --trainer.gpus 4 --trainer.accelerator ddp\n```\n\nWith [Grid DataStores](https://docs.grid.ai/products/global-cli-configs/cli-api/grid-datastores), low-latency, highly-scalable auto-versioned dataset.\n\n```bash\ngrid datastore create --name mnist --source data\ngrid run --instance_type 4_M60_8gb --datastore_name mnist --datastore_mount_dir data ddp_mnist_grid/lightning.py  --trainer.max_epochs 2 --trainer.gpus 4 --trainer.accelerator ddp\n```\n\nPure PyTorch:\n\n```bash\ngrid datastore create --name mnist --source data\ngrid run --instance_type g4dn.xlarge --gpus 2 ddp_mnist_grid/boring_pytorch.py\n```\n\nAdd `--use_spot` to use interruptible machines.\n\n[Grid.ai](https://www.grid.ai/) makes scaling multi node training easy :rocket: Train on 2+ nodes with 4 GPUS using [DDP Sharded](https://medium.com/pytorch/pytorch-lightning-1-1-model-parallelism-training-and-more-logging-options-7d1e47db7b0b) :fire:\n\n\n```bash\ngrid run --instance_type 4_M60_8gb --gpus 8 --datastore_name mnist --datastore_mount_dir data  ddp_mnist_grid/lightning.py  --trainer.max_epochs 2 --trainer.num_nodes 2 --trainer.gpus 4 --trainer.accelerator ddp_sharded\n```\n\nTrain [Andrej Karpathy](https://karpathy.ai) [minGPT](https://github.com/karpathy/minGPT) converted to [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) by [@williamFalcon](https://github.com/williamFalcon) and bencharmked with DeepSpeed by [@SeanNaren](https://github.com/SeanNaren)\n\n```\ngit clone https://github.com/SeanNaren/minGPT.git\ngit checkout benchmark\ngrid run --instance_type g4dn.12xlarge --gpus 8 benchmark.py --n_layer 6 --n_head 16 --n_embd 2048 --gpus 4 --num_nodes 2 --precision 16 --batch_size 32 --plugins deepspeed_stage_3\n```\n\nLearn how to scale your scripts with [PyTorch Lighting + DeepSpeed](https://devblog.pytorchlightning.ai/accessible-multi-billion-parameter-model-training-with-pytorch-lightning-deepspeed-c9333ac3bb59)\n\n# [Lighting Flash](https://github.com/PyTorchLightning/lightning-flash).\n\n[Lighting Flash](https://github.com/PyTorchLightning/lightning-flash) is collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning built on top of PyTorch Lightning.\n\nTrain a [PyTorchVideo Classifier](https://github.com/PyTorchLightning/lightning-flash/blob/master/flash_examples/video_classification.py) with [Lighting Flash](https://github.com/PyTorchLightning/lightning-flash). Check out [Grid.ai](https://www.grid.ai/) reproducible button: \n[![Grid](https://img.shields.io/badge/rid_AI-run-78FF96.svg?labelColor=black\u0026logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iNDgiIGhlaWdodD0iNDgiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHBhdGggZD0iTTEgMTR2MjBhMTQgMTQgMCAwMDE0IDE0aDlWMzYuOEgxMi42VjExaDIyLjV2N2gxMS4yVjE0QTE0IDE0IDAgMDAzMi40IDBIMTVBMTQgMTQgMCAwMDEgMTR6IiBmaWxsPSIjZmZmIi8+PHBhdGggZD0iTTM1LjIgNDhoMTEuMlYyNS41SDIzLjl2MTEuM2gxMS4zVjQ4eiIgZmlsbD0iI2ZmZiIvPjwvc3ZnPg==)](https://platform.grid.ai/#/runs?script=https://github.com/aribornstein/KineticsDemo/blob/188f1948725506914b67d3814073a7bec152ac0a/train.py\u0026cloud=grid\u0026instance=g4dn.xlarge\u0026accelerators=1\u0026disk_size=200\u0026framework=lightning\u0026script_args=train.py%20--gpus%201%20--max_epochs%203)\n\n\n```py\nimport os\n\nimport flash\nfrom flash.core.data.utils import download_data\nfrom flash.video import VideoClassificationData, VideoClassifier\n\n# 1. Create the DataModule\n# Find more datasets at https://pytorchvideo.readthedocs.io/en/latest/data.html\ndownload_data(\"https://pl-flash-data.s3.amazonaws.com/kinetics.zip\", \"./data\")\n\ndatamodule = VideoClassificationData.from_folders(\n    train_folder=os.path.join(os.getcwd(), \"data/kinetics/train\"),\n    val_folder=os.path.join(os.getcwd(), \"data/kinetics/val\"),\n    clip_sampler=\"uniform\",\n    clip_duration=1,\n    decode_audio=False,\n)\n\n# 2. Build the task\nmodel = VideoClassifier(backbone=\"x3d_xs\", num_classes=datamodule.num_classes, pretrained=False)\n\n# 3. Create the trainer and finetune the model\ntrainer = flash.Trainer(max_epochs=3)\ntrainer.finetune(model, datamodule=datamodule, strategy=\"freeze\")\n\n# 4. Make a prediction\npredictions = model.predict(os.path.join(os.getcwd(), \"data/kinetics/predict\"))\nprint(predictions)\n\n# 5. Save the model!\ntrainer.save_checkpoint(\"video_classification.pt\")\n```\n\n## Credits\n\nCredit to PyTorch Team for providing the [Bare Mnist example](https://github.com/pytorch/examples/blob/master/mnist/main.py).\n\nCredit to Andrej Karpathy for providing an implementation of minGPT.\n\n\n## Troubleshooting\n\nKill ddp processes\n\n```bash\nsudo kill -9 $(ps -aef | grep -i 'ddp' | grep -v 'grep' | awk '{ print $2 }')\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftchaton%2Fpytorch2lightning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftchaton%2Fpytorch2lightning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftchaton%2Fpytorch2lightning/lists"}