{"id":15027625,"url":"https://github.com/gpleiss/efficient_densenet_pytorch","last_synced_at":"2025-05-15T19:07:05.366Z","repository":{"id":41153184,"uuid":"92970515","full_name":"gpleiss/efficient_densenet_pytorch","owner":"gpleiss","description":"A memory-efficient implementation of DenseNets","archived":false,"fork":false,"pushed_at":"2023-06-01T13:11:51.000Z","size":1144,"stargazers_count":1531,"open_issues_count":12,"forks_count":325,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-04-08T00:37:21.313Z","etag":null,"topics":["deep-learning","densenet","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gpleiss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-05-31T17:19:25.000Z","updated_at":"2025-04-07T07:18:06.000Z","dependencies_parsed_at":"2022-08-10T01:42:48.960Z","dependency_job_id":"75c9cef1-4c84-45f4-b3c6-2a75e87f72f1","html_url":"https://github.com/gpleiss/efficient_densenet_pytorch","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpleiss%2Fefficient_densenet_pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpleiss%2Fefficient_densenet_pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpleiss%2Fefficient_densenet_pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpleiss%2Fefficient_densenet_pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gpleiss","download_url":"https://codeload.github.com/gpleiss/efficient_densenet_pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254404356,"owners_count":22065641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","densenet","pytorch"],"created_at":"2024-09-24T20:06:48.323Z","updated_at":"2025-05-15T19:07:05.345Z","avatar_url":"https://github.com/gpleiss.png","language":"Python","funding_links":[],"categories":["Paper implementations｜论文实现","Paper implementations"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"readme":"# efficient_densenet_pytorch\nA PyTorch \u003e=1.0 implementation of DenseNets, optimized to save GPU memory.\n\n## Recent updates\n1. **Now works on PyTorch 1.0!** It uses the checkpointing feature, which makes this code WAY more efficient!!!\n\n## Motivation\nWhile DenseNets are fairly easy to implement in deep learning frameworks, most\nimplmementations (such as the [original](https://github.com/liuzhuang13/DenseNet)) tend to be memory-hungry.\nIn particular, the number of intermediate feature maps generated by batch normalization and concatenation operations\ngrows quadratically with network depth.\n*It is worth emphasizing that this is not a property inherent to DenseNets, but rather to the implementation.*\n\nThis implementation uses a new strategy to reduce the memory consumption of DenseNets.\nWe use [checkpointing](https://pytorch.org/docs/stable/checkpoint.html?highlight=checkpointing) to compute the Batch Norm and concatenation feature maps.\nThese intermediate feature maps are discarded during the forward pass and recomputed for the backward pass.\nThis adds 15-20% of time overhead for training, but **reduces feature map consumption from quadratic to linear.**\n\nThis implementation is inspired by this [technical report](https://arxiv.org/pdf/1707.06990.pdf), which outlines a strategy for efficient DenseNets via memory sharing.\n\n## Requirements\n- PyTorch \u003e=1.0.0\n- CUDA\n\n## Usage\n\n**In your existing project:**\nThere is one file in the `models` folder.\n - `models/densenet.py` is an implementation based off the [torchvision](https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py) and\n[project killer](https://github.com/felixgwu/img_classification_pk_pytorch/blob/master/models/densenet.py) implementations.\n\nIf you care about speed, and memory is not an option, pass the `efficient=False` argument into the `DenseNet` constructor.\nOtherwise, pass in `efficient=True`.\n\n**Options:**\n- All options are described in [the docstrings of the model files](https://github.com/gpleiss/efficient_densenet_pytorch/blob/master/models/densenet_efficient.py#L189)\n- The depth is controlled by `block_config` option\n- `efficient=True` uses the memory-efficient version\n- If you want to use the model for ImageNet, set `small_inputs=False`. For CIFAR or SVHN, set `small_inputs=True`.\n\n**Running the demo:**\n\nThe only extra package you need to install is [python-fire](https://github.com/google/python-fire):\n```sh\npip install fire\n```\n\n- Single GPU:\n\n```sh\nCUDA_VISIBLE_DEVICES=0 python demo.py --efficient True --data \u003cpath_to_folder_with_cifar10\u003e --save \u003cpath_to_save_dir\u003e\n```\n\n- Multiple GPU:\n\n```sh\nCUDA_VISIBLE_DEVICES=0,1,2 python demo.py --efficient True --data \u003cpath_to_folder_with_cifar10\u003e --save \u003cpath_to_save_dir\u003e\n```\n\nOptions:\n- `--depth` (int) - depth of the network (number of convolution layers) (default 40)\n- `--growth_rate` (int) - number of features added per DenseNet layer (default 12)\n- `--n_epochs` (int) - number of epochs for training (default 300)\n- `--batch_size` (int) - size of minibatch (default 256)\n- `--seed` (int) - manually set the random seed (default None)\n\n## Performance\n\nA comparison of the two implementations (each is a DenseNet-BC with 100 layers, batch size 64, tested on a NVIDIA Pascal Titan-X):\n\n| Implementation | Memory cosumption (GB/GPU) | Speed (sec/mini batch) |\n|----------------|------------------------|------------------------|\n| Naive          |  2.863  | 0.165                  |\n| Efficient      |  1.605  | 0.207                  |\n| Efficient (multi-GPU)      |  0.985  | -                  |\n\n\n## Other efficient implementations\n- [LuaTorch](https://github.com/liuzhuang13/DenseNet/tree/master/models) (by Gao Huang)\n- [Tensorflow](https://github.com/joeyearsley/efficient_densenet_tensorflow) (by Joe Yearsley)\n- [Caffe](https://github.com/Tongcheng/DN_CaffeScript) (by Tongcheng Li)\n\n## Reference\n\n```\n@article{pleiss2017memory,\n  title={Memory-Efficient Implementation of DenseNets},\n  author={Pleiss, Geoff and Chen, Danlu and Huang, Gao and Li, Tongcheng and van der Maaten, Laurens and Weinberger, Kilian Q},\n  journal={arXiv preprint arXiv:1707.06990},\n  year={2017}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgpleiss%2Fefficient_densenet_pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgpleiss%2Fefficient_densenet_pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgpleiss%2Fefficient_densenet_pytorch/lists"}