{"id":13497251,"url":"https://github.com/sacmehta/ESPNetv2","last_synced_at":"2025-03-28T21:32:25.879Z","repository":{"id":43057589,"uuid":"158426519","full_name":"sacmehta/ESPNetv2","owner":"sacmehta","description":"A light-weight, power efficient, and general purpose convolutional neural network","archived":false,"fork":false,"pushed_at":"2021-01-29T00:27:38.000Z","size":44301,"stargazers_count":447,"open_issues_count":2,"forks_count":70,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-10-31T13:34:19.753Z","etag":null,"topics":["cnn","convolutional-neural-networks","deep-learning","efficient","imagenet","lightweight","machine-learning","pytorch","semantic-segmentation"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sacmehta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-20T17:21:15.000Z","updated_at":"2024-10-21T16:34:34.000Z","dependencies_parsed_at":"2022-09-04T15:34:05.348Z","dependency_job_id":null,"html_url":"https://github.com/sacmehta/ESPNetv2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sacmehta%2FESPNetv2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sacmehta%2FESPNetv2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sacmehta%2FESPNetv2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sacmehta%2FESPNetv2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sacmehta","download_url":"https://codeload.github.com/sacmehta/ESPNetv2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246105536,"owners_count":20724326,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","convolutional-neural-networks","deep-learning","efficient","imagenet","lightweight","machine-learning","pytorch","semantic-segmentation"],"created_at":"2024-07-31T20:00:27.466Z","updated_at":"2025-03-28T21:32:22.046Z","avatar_url":"https://github.com/sacmehta.png","language":"Python","funding_links":[],"categories":["SemanticSeg","2.) Lightweight Structures"],"sub_categories":["**[Papers]**"],"readme":"# ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network\n\n**IMPORTANT NOTE 1 (7 June, 2019)**: We have released new code base that supports several datasets and models, including ESPNetv2. Please see [here](https://github.com/sacmehta/EdgeNets) for more details.\n\n**IMPORTANT NOTE 2 (7 June, 2019)**: This repository is obsolete and we are not maintaining it anymore.\n\nThis repository contains the source code of our paper, [ESPNetv2](https://arxiv.org/abs/1811.11431) which is accepted for publication at CVPR'19. \n\n***Note:*** New segmentation models for the PASCAL VOC and the Cityscapes are coming soon. Our new models achieves mIOU of [68.0](http://host.robots.ox.ac.uk:8080/anonymous/DAMVRR.html) and [66.15](https://www.cityscapes-dataset.com/anonymous-results/?id=2267c613d55dd75d5301850c913b1507bf2f10586ca73eb8ebcf357cdcf3e036) on the PASCAL VOC and the Cityscapes test sets, respectively. \n\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003ctd colspan=2 align=\"center\"\u003e\u003cb\u003eReal-time semantic segmentation using ESPNetv2 on iPhone7 (see \u003ca href='https://github.com/sacmehta/EdgeNets'\u003eEdgeNets\u003c/a\u003e  for details)\u003cb\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\n            \u003cimg src=\"https://github.com/sacmehta/EdgeNets/blob/master/images/espnetv2_iphone7_video_1.gif\" alt=\"Seg demo on iPhone7\"\u003e\u003c/img\u003e\n        \u003c/td\u003e\n        \u003ctd\u003e\n            \u003cimg src=\"https://github.com/sacmehta/EdgeNets/blob/master/images/espnetv2_iphone7_video_2.gif\" alt=\"Seg demo on iPhone7\"\u003e\u003c/img\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n## Comparison with SOTA methods\nCompared to state-of-the-art efficient networks, our network delivers competitive performance while being much more **power efficient**. Sample results are shown in below figure. For more details, please read our paper.\n\n  \u003ctable width=\"100%\" align=\"center\" border=1\u003e\n    \u003ctr\u003e\n        \u003ctd width=\"50%\"\u003e\n            \u003cimg src=\"/images/effCompare.png\" width=\"80%\"/\u003e\n        \u003c/td\u003e\n        \u003ctd width=\"50%\"\u003e\n            \u003cimg src=\"/images/powerTX2.png\" width=\"80%\"/\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\n          \u003cp align=\"center\"\u003e\u003cb\u003eFLOPs vs. accuracy on the ImageNet dataset\u003c/b\u003e\u003c/p\u003e\n        \u003c/td\u003e\n        \u003ctd\u003e\n          \u003cp align=\"center\"\u003e\u003cb\u003ePower consumption on TX2 device\u003c/b\u003e\u003c/b\u003e\n      \u003c/td\u003e\n    \u003ctr\u003e\n  \u003c/table\u003e\n\n\n\nIf you find our project useful in your research, please consider citing:\n\n```\n@inproceedings{mehta2019espnetv2,\n  title={ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network},\n  author={Sachin Mehta and Mohammad Rastegari and Linda Shapiro and Hannaneh Hajishirzi},\n  booktitle={CVPR},\n  year={2019}\n}\n\n@inproceedings{mehta2018espnet,\n  title={ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation},\n  author={Sachin Mehta and Mohammad Rastegari and Anat Caspi and Linda Shapiro and Hannaneh Hajishirzi},\n  booktitle={ECCV},\n  year={2018}\n}\n```\n\n## Structure\nThis repository contains source code and pretrained for the following:\n * **Object classification:** We provide source code along with pre-trained models at different network complexities \n for the ImageNet dataset. Click [here](imagenet) for more details.\n * **Semantic segmentation:** We provide source code along with pre-trained models on the Cityscapes dataset. Check [here](segmentation) for more details. \n \n## Requirements\n \nTo run this repository, you should have following softwares installed:\n * PyTorch - We tested with v0.4.1\n * OpenCV - We tested with version 3.4.3\n * Python3 - Our code is written in Python3. We recommend to use [Anaconda](https://www.anaconda.com/) for the same.\n \n ## Instructions to install Pytorch and OpenCV with Anaconda\n \nAssuming that you have installed Anaconda successfully, you can follow the following instructions to install the packeges:\n \n### PyTorch\n```\nconda install pytorch torchvision -c pytorch\n```\n\nOnce installed, run the following commands in your terminal to verify the version:\n```\nimport torch\ntorch.__version__ \n```\nThis should print something like this `0.4.1.post2`. \n\nIf your version is different, then follow PyTorch website [here](https://pytorch.org/) for more details.\n\n### OpenCV\n```\nconda install pip\npip install --upgrade pip\npip install opencv-python\n```\n\nOnce installed, run the following commands in your terminal to verify the version:\n```\nimport cv2\ncv2.__version__ \n```\nThis should print something like this `3.4.3`.\n\n \n## Implementation note\n\nYou will see that `EESP` unit, the core building block of the ESPNetv2 architecture, has a `for` loop to process the input at different dilation rates. \nYou can parallelize it using **Streams** in PyTorch. It improves the inference speed. \n\nA snippet to parallelize a `for` loop in pytorch is shown below:\n```\n# Sequential version\noutput = [] \na = torch.randn(1, 3, 10, 10)\nfor i in range(4):\n    output.append(a)\ntorch.cat(output, 1)\n```\n\n``` \n# Parallel version\nnum_branches = 4\nstreams = [(idx, torch.cuda.Stream()) for idx in range(num_branches)]\noutput = []\na = torch.randn(1, 3, 10, 10)\nfor idx, s in streams:\n    with torch.cuda.stream(s):\n        output.append(a)\ntorch.cuda.synchronize()\ntorch.cat(output, 1)\n```\n\n**Note:** \n * we have used above strategy to measure inference related statistics, including power consumption and run time on a single GPU.\n * We have not tested it (for training as well as inference) across multiple GPUs. If you want to use Streams and facing issues, please use PyTorch forums to resolve your queries. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsacmehta%2FESPNetv2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsacmehta%2FESPNetv2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsacmehta%2FESPNetv2/lists"}