{"id":15009475,"url":"https://github.com/autuanliu/pytorch-ml","last_synced_at":"2025-04-09T17:24:49.805Z","repository":{"id":196754947,"uuid":"126927840","full_name":"AutuanLiu/PyTorch-ML","owner":"AutuanLiu","description":":books::orange: Implement DNN or ML models and advanced policies with PyTorch.(Include experiment)","archived":false,"fork":false,"pushed_at":"2019-12-10T07:28:41.000Z","size":4640,"stargazers_count":11,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-23T19:38:41.708Z","etag":null,"topics":["benchmark","circular-learning-rate","deep-learning","dnn","neural-network","python-3-6","pytorch","research","warm-restart"],"latest_commit_sha":null,"homepage":"http://bit.ly/2RufRK5","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AutuanLiu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"ko_fi":"moliang"}},"created_at":"2018-03-27T04:16:26.000Z","updated_at":"2023-06-15T22:59:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"f30e2638-c23e-4ced-b6e0-ada7e6f8a70b","html_url":"https://github.com/AutuanLiu/PyTorch-ML","commit_stats":null,"previous_names":["autuanliu/pytorch-ml"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutuanLiu%2FPyTorch-ML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutuanLiu%2FPyTorch-ML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutuanLiu%2FPyTorch-ML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutuanLiu%2FPyTorch-ML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AutuanLiu","download_url":"https://codeload.github.com/AutuanLiu/PyTorch-ML/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248075693,"owners_count":21043634,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","circular-learning-rate","deep-learning","dnn","neural-network","python-3-6","pytorch","research","warm-restart"],"created_at":"2024-09-24T19:25:28.828Z","updated_at":"2025-04-09T17:24:49.778Z","avatar_url":"https://github.com/AutuanLiu.png","language":"Python","funding_links":["https://ko-fi.com/moliang"],"categories":[],"sub_categories":[],"readme":"# PyTorch-DNN\nImplement DNN models and advanced policy with PyTorch.\n\n## Requirements\n1. torch \u003e= 0.4.0\n2. torchvision \u003e= 0.2.1\n\n## Content\n1. [Cyclical Learning Rates](CLR_example.py)\n```python\noptimizer = optim.Adam(model.parameters(), lr=1.)\n# initial lr should be 1\nclr = cyclical_lr(step_size, min_lr=0.001, max_lr=1, scale_func=clr_func, scale_md='iterations')\nscheduler = lr_scheduler.LambdaLR(optimizer, [clr])\n```\n2. [SGDR(has been committed to PyTorch)](WarmRestart_example.py)\n\n* [Warm restart policy is available now by AutuanLiu · Pull Request #6130 · pytorch/pytorch](https://github.com/pytorch/pytorch/pull/6130)\n* [Cosine Annealing with warm restarts by roveo · Pull Request #7821 · pytorch/pytorch](https://github.com/pytorch/pytorch/pull/7821)\n* [Cosine annealing with restarts by striajan · Pull Request #11104 · pytorch/pytorch](https://github.com/pytorch/pytorch/pull/11104)\n\n```python\ntorch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=100, eta_min=1e-8, T_mult=2)\n# T_max \u003c training epochs if you want to use restart policy\n```\n3. [An abstract class for establish network](models/BaseNet_calss.py)\n```python\nfrom models.BaseNet_class import BaseNet\n# some configs setting\nconfigs = {\n    'model': net,\n    'opt': opt,\n    'criterion': nn.CrossEntropyLoss(),\n    'dataloaders': ...,\n    'data_sz': ...,\n    'lrs_decay': lr_scheduler.StepLR(opt, step_size=50),\n    'prt_freq': 5,\n    'epochs': 500,\n}\nsub_model = BaseNet(configs)\n# train and test\nsub_model.train_m()\nsub_model.test_m()\n```\n\n## CNN\n* ResNet\n* AlexNet\n* GoogLeNet\n* DenseNet\n* VGGNet\n* LeNet\n* GAN\n* NiN\n* STN\n* VAE\n\n## RNN\n* RNN\n* LSTM\n* GRU\n* [Neural Network for Time Series](https://github.com/AutuanLiu/Deep-Learning-for-Time-Series)\n\n\n## Related papers\n1. [[1608.03983] SGDR: Stochastic Gradient Descent with Warm Restarts](https://arxiv.org/abs/1608.03983)\n2. [[1506.01186] Cyclical Learning Rates for Training Neural Networks](https://arxiv.org/abs/1506.01186)\n3. [[1704.00109] Snapshot Ensembles: Train 1, get M for free](https://arxiv.org/abs/1704.00109)\n\n## Related references\n1. [Another data science student's blog](https://sgugger.github.io/)\n2. [动手学深度学习 文档](https://zh.gluon.ai/toc.html)\n3. [Understanding LSTM and its diagrams](https://medium.com/mlreview/understanding-lstm-and-its-diagrams-37e2f46f1714)\n4. [吴良超的学习笔记](http://wulc.me/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautuanliu%2Fpytorch-ml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautuanliu%2Fpytorch-ml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautuanliu%2Fpytorch-ml/lists"}