{"id":20599653,"url":"https://github.com/adamdad/dery","last_synced_at":"2025-04-09T20:08:42.856Z","repository":{"id":62399924,"uuid":"496473059","full_name":"Adamdad/DeRy","owner":"Adamdad","description":"[NeurIPS2022] Deep Model Reassembly","archived":false,"fork":false,"pushed_at":"2023-08-29T08:02:03.000Z","size":1383,"stargazers_count":253,"open_issues_count":5,"forks_count":11,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-09T20:08:35.467Z","etag":null,"topics":["ai","deep-learning","deep-neural-networks","machine-learning","model-reuse","neural-networks","neurips-2022","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Adamdad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-26T03:52:06.000Z","updated_at":"2025-03-23T14:57:23.000Z","dependencies_parsed_at":"2024-11-16T08:33:52.272Z","dependency_job_id":"554bfb5d-d5c0-4980-8701-0ca1de1148db","html_url":"https://github.com/Adamdad/DeRy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adamdad%2FDeRy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adamdad%2FDeRy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adamdad%2FDeRy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Adamdad%2FDeRy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Adamdad","download_url":"https://codeload.github.com/Adamdad/DeRy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248103873,"owners_count":21048245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","deep-neural-networks","machine-learning","model-reuse","neural-networks","neurips-2022","transfer-learning"],"created_at":"2024-11-16T08:33:41.121Z","updated_at":"2025-04-09T20:08:42.834Z","avatar_url":"https://github.com/Adamdad.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Model Reassembly  🏭 -\u003e 🧱 -\u003e 🏭\n## 😎 Introduction\nThis repository contains the offical implementation for our paper\n\n**Deep Model Reassembly** (NeurIPS2022)\n\n[[arxiv](https://arxiv.org/abs/2210.17409)] [[project page](https://adamdad.github.io/dery/)]\n [[code](https://github.com/Adamdad/DeRy)]\n\n*Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, Xinchao Wang*\n\n\u003e In this work, we explore a novel knowledge-transfer task, termed as Deep Model Reassembly (*DeRy*), for general-purpose model reuse. *DeRy* first dissect each model into distinctive building blocks, and then selectively reassemble the derived blocks to produce customized networks under both the hardware resource and performance constraints.\n\n![pipeline](assets/pipeline.png)\n\n- [x] 2022/12/07 Code Updated for better usage.\n\n## 📚 File Orgnization\n\n```\nDeRy/\n├── blocklize/block_meta.py         [Meta Information \u0026 Node Defnition]\n\n├── similarity/\n│   ├── get_rep.py                  [Compute and save the feature embeddings]\n│   ├── get_sim.py                  [Compute representation similarity given the saved features]\n|   ├── partition.py                [Network partition by cover set problem]\n|   ├── zeroshot_reassembly.py      [Network reassembly by solving integer program]\n\n├── configs/\n|   ├── compute_sim/                [Model configs in the model zoo to compute the feature similarity]\n|   ├── dery/XXX/$ModelSize_$DataSet_$BatchSize_$TrainTime_dery_$Optimizor.py   [Config files for transfer experiments]\n\n├── mmcls_addon/\n|   ├── datasets/                   [Dataset definitions]\n|   ├── models/backbones/dery.py    [DeRy backbone definition]\n\n├── third_package/timm              [Modified timm package]\n```\n    \n\n## 🛠 Installation\nThe model training part is based on [mmclassification](https://github.com/open-mmlab/mmclassification). Some of the pre-trained weights are from [timm](https://github.com/rwightman/pytorch-image-models/tree/master/timm).\n\n    # Create python env\n    conda create -n open-mmlab python=3.8 pytorch=1.10 cudatoolkit=11.3 torchvision -c pytorch -y\n    conda activate open-mmlab\n\n    # Install mmcv and mmcls\n    pip3 install openmim\n    mim install mmcv-full==1.4.8\n    mim install mmcls==0.18.0\n\n    # Install timm\n    pip3 install timm\n\n\n**Note**: Our code needs `torch.fx` to support the computational graph extraction from the torch model. Therefore, please install the `torch \u003e 1.10`.\n\n\n## 🚀 Getting Started\nTo run the code for *DeRy*, we need to go through 4 steps\n\n1. [**Model Zoo Preparation**] Compute the model feature embeddings and representation similarity. We first write model configuration and its weight path, and run the configs in `configs/compute_sim`\n            \n        PYTHONPATH=\"$PWD\" python simlarity/get_rep.py \\\n        $Config_file \\              # configs in `configs/compute_sim`\n        --out $Feature_path \\       # Save feature embeddings in *.pth* files\n        [--checkpoint $Checkpoint]  # download checkpoint if any\n\n    All feature embeddings need to be saved in *.pth* files in the same $Feat_dictionary. We then load them and compute the feature similarity. Similarity will be saved as *net1.net2.pkl* files.\n\n        PYTHONPATH=\"$PWD\" python simlarity/compute_sim.py /\n        --feat_path $Feat_dictionary /\n        --sim_func $Similarity_function [cka, rbf_cka, lr]\n\n    Pre-computed similarity on ImageNet for [Linear CKA](https://drive.google.com/drive/folders/1ebSVwZyKeHdmdOdVlFZF6P9_1PzEMs-J?usp=share_link) and [Linear Regression](https://drive.google.com/drive/folders/1rKmV3iQwETKBO3yYlsXFIxrPfAc7VRRb?usp=share_link).\n\n    We also need to compute the feature size (input-output feature dimensions). It can be done by running\n\n        PYTHONPATH=\"$PWD\" python simlarity/count_inout_size.py /\n        --root $Feat_dictionary\n\n    The results is a json file containing the input-output shape for all network layers, like [MODEL_INOUT_SHAPE.json](https://drive.google.com/file/d/15xDgYOu8Gs866faNHEYI0iHCJkxl-M2h/view?usp=share_link).\n\n2. [**Network Partition**] Solve the cover set optimization to get the network partition. The results is an assignment file in *.pkl*.\n\n        PYTHONPATH=\"$PWD\" python simlarity/partition.py /\n        --sim_path $Feat_similarity_path /\n        --K        $Num_partition /             # default=4\n        --trial    $Num_repeat_runs /           # default=200\n        --eps      $Size_ratio_each_block /     # default=0.2\n        --num_iter $Maximum_num_iter_eachrun    # default=200\n\n3. [**Reassemby**] Reassemble the partitioned building blocks into a full model, by solving a integer program with training-free proxy. The results are a series of model configs in *.py*.\n\n        PYTHONPATH=\"$PWD\" python simlarity/zeroshot_reassembly.py \\\n        --path          $Block_partition_file [Saved in the partition step] \\\n        --C             $Maximum_parameter_num \\\n        --minC          $Minimum_parameter_num \\\n        --flop_C        $Maximum_FLOPs_num \\\n        --minflop_C     $Minimum_FLOPs_num \\\n        --num_batch     $Number_batch_average_to_compute_score \\\n        --batch_size    $Number_sample_each_batch \\\n        --trial         $Search_time \\\n        --zero_proxy    $Type_train_free_proxy [Default NASWOT] \\\n        --data_config   $Config_target_data\n\n4. [**Fune-tuning**] Train the reassembled model on target data. You may refers to [mmclassification](https://github.com/open-mmlab/mmclassification) for the model training.\n\n**Note**: Partitioning and reassembling results may not be identical because of the algorithmatic stochaticity. It may slightly affect the performance.\n \n## 🚛 Other Resources\n1. We use several pre-trained models not included in timm and mmcls, listed in [Pre-trained](assets/pre-trained.md).\n\n## Thanks for the support\n[![Stargazers repo roster for @Adamdad/DeRy](https://reporoster.com/stars/Adamdad/DeRy)](https://github.com/Adamdad/DeRy/stargazers)\n\n## ✍ Citation\n\n```bibtex\n@article{yang2022dery,\n    author    = {Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, Xinchao Wang},\n    title     = {Deep Model Reassembly},\n    journal   = {NeurIPS},\n    year      = {2022},\n}\n```\n# Extensions\n1. Extension on **Efficienr and Parallel Training**: [Deep-Incubation](https://arxiv.org/abs/2212.04129)\n2. Extension for **Efficient Model Zoo Training**:  [Stitchable Neural Networks](https://arxiv.org/abs/2302.06586)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamdad%2Fdery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamdad%2Fdery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamdad%2Fdery/lists"}