{"id":21389489,"url":"https://github.com/feifeibear/communication-efficient-dnn","last_synced_at":"2025-03-16T12:46:55.614Z","repository":{"id":85016951,"uuid":"129308850","full_name":"feifeibear/Communication-Efficient-DNN","owner":"feifeibear","description":null,"archived":false,"fork":false,"pushed_at":"2018-05-24T22:08:35.000Z","size":85,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-23T00:41:16.822Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feifeibear.png","metadata":{"files":{"readme":"README.md","changelog":"Change.txt","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-12T20:40:43.000Z","updated_at":"2020-01-15T01:25:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"7499bbab-0d7f-4431-9311-6095066c99e0","html_url":"https://github.com/feifeibear/Communication-Efficient-DNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feifeibear%2FCommunication-Efficient-DNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feifeibear%2FCommunication-Efficient-DNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feifeibear%2FCommunication-Efficient-DNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feifeibear%2FCommunication-Efficient-DNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feifeibear","download_url":"https://codeload.github.com/feifeibear/Communication-Efficient-DNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243871652,"owners_count":20361378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T12:26:43.050Z","updated_at":"2025-03-16T12:46:55.586Z","avatar_url":"https://github.com/feifeibear.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Communication Efficient Deep Learning Training Method\nThis is a code repository used to reproduce the results in paper [DEEP GRADIENT COMPRESSION: REDUCING THE COMMUNICATION BANDWIDTH FOR DISTRIBUTED TRAINING](https://arxiv.org/pdf/1712.01887.pdf)\nIt is based off [convNet.pytorch](https://github.com/eladhoffer/convNet.pytorch) with some helpful options such as:\n  - Simulate data parallel senario on multiple machines on one GPU\n  - Gradient pruning\n  - Training on several datasets\n  - Complete logging of trained experiment\n  - Graph visualization of the training/validation loss and accuracy\n  - Definition of preprocessing and optimization regime for each model\n\n## Dependencies\n\n- [pytorch](\u003chttp://www.pytorch.org\u003e)\n- [torchvision](\u003chttps://github.com/pytorch/vision\u003e) to load the datasets, perform image transforms\n- [pandas](\u003chttp://pandas.pydata.org/\u003e) for logging to csv\n\n## Data\n- Configure your dataset path at **data.py**.\n- To get the ILSVRC data, you should register on their site for access: \u003chttp://www.image-net.org/\u003e\n\n## Experiment examples\n```bash\nsh ./submit_pruning.sh\n```\n- See *run_experiments.sh* for more examples\n## Model configuration\n\nNetwork model is defined by writing a \u003cmodelname\u003e.py file in \u003ccode\u003emodels\u003c/code\u003e folder, and selecting it using the \u003ccode\u003emodel\u003c/code\u003e flag. Model function must be registered in \u003ccode\u003emodels/\\_\\_init\\_\\_.py\u003c/code\u003e\nThe model function must return a trainable network. It can also specify additional training options such optimization regime (either a dictionary or a function), and input transform modifications.\n\ne.g for a model definition:\n\n```python\nclass Model(nn.Module):\n\n    def __init__(self, num_classes=1000):\n        super(Model, self).__init__()\n        self.model = nn.Sequential(...)\n\n        self.regime = {\n            0: {'optimizer': 'SGD', 'lr': 1e-2,\n                'weight_decay': 5e-4, 'momentum': 0.9},\n            15: {'lr': 1e-3, 'weight_decay': 0}\n        }\n\n        self.input_transform = {\n            'train': transforms.Compose([...]),\n            'eval': transforms.Compose([...])\n        }\n    def forward(self, inputs):\n        return self.model(inputs)\n\n def model(**kwargs):\n        return Model()\n```\n\n## Author\nJiarui Fang \nfjr14@mails.tsinghua.edu.cn\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeifeibear%2Fcommunication-efficient-dnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeifeibear%2Fcommunication-efficient-dnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeifeibear%2Fcommunication-efficient-dnn/lists"}