Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/icgy96/awesome_model_merging_list

Last synced: 19 days ago
JSON representation
Host: GitHub
URL: https://github.com/icgy96/awesome_model_merging_list
Owner: iCGY96
License: mit
Created: 2024-07-07T14:22:36.000Z (4 months ago)
Default Branch: main
Last Pushed: 2024-07-21T14:01:08.000Z (4 months ago)
Last Synced: 2024-07-22T15:26:00.459Z (4 months ago)
Size: 13.7 KB
Stars: 2
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

README

        # Awesome-Model-Merging-List [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) [![Made With Love](https://img.shields.io/badge/Made%20With-Love-red.svg)](https://github.com/chetanraj/awesome-github-badges)

This repository contains a collection of resources and papers on ***Model Merging***.

## Contributing

Please help contribute this list by [pull request](https://github.com/iCGY96/awesome_model_merging_list/pulls)

Markdown format:

```markdown

  - Paper title. (**Conference Year**) [[pdf]](link) [[code]](link)

```

## Paper

### 2024

- Representation Surgery for Multi-Task Model Merging. (**ICML** 2024) [[paper]](https://arxiv.org/pdf/2402.02705) [[code]](https://github.com/EnnengYang/RepresentationSurgery)

- Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. (**ICML** 2024) [[paper]](https://arxiv.org/abs/2311.03099) [[code]](https://github.com/yule-buaa/mergelm)

- Merging Multi-Task Models via Weight-Ensembling Mixture of Experts. (**ICML** 2024) [[paper]](https://arxiv.org/abs/2402.00433) [[code]](https://github.com/tanganke/weight-ensembling_moe)

- AdaMerging: Adaptive Model Merging for Multi-Task Learning. (**ICLR** 2024) [[paper]](https://openreview.net/pdf?id=nZP6NgD3QY) [[code]](https://github.com/EnnengYang/AdaMerging)

- Adaptive Discovering and Merging for Incremental Novel Class Discovery. (**AAAI** 2024) [[paper]](https://arxiv.org/abs/2403.03382)

- Training Free Pretrained Model Merging. (**CVPR** 2024) [[paper]](https://arxiv.org/abs/2403.01753) [[code]](https://github.com/zju-vipa/training_free_model_merging)

- Arcee's MergeKit: A Toolkit for Merging Large Language Models. (**ArXiv** 2024) [[paper]](https://arxiv.org/abs/2403.13257) [[code]](https://github.com/arcee-ai/mergekit) [[doc]](https://docs.google.com/document/d/1_vOftBnrk9NRk5h10UqrfJ5CDih9KBKL61yvrZtVWPE/edit?pli=1)

### 2023

- Re-basin via implicit Sinkhorn differentiation. (**CVPR** 2023) [[paper]](https://arxiv.org/pdf/2212.12042.pdf) [[code]](https://github.com/fagp/sinkhorn-rebasin)

- Editing Models with Task Arithmetic. (**ICLR** 2023) [[paper]](https://arxiv.org/abs/2212.04089) [[code]](https://github.com/mlfoundations/task_vectors)

- Git Re-Basin: Merging Models modulo Permutation Symmetries. (**ICLR** 2023) [[paper]](https://arxiv.org/abs/2209.04836)

- Dataless Knowledge Fusion by Merging Weights of Language Models. (**ICLR** 2023) [[paper]](https://arxiv.org/pdf/2212.09849.pdf) [[code]](https://github.com/bloomberg/dataless-model-merging)

- REPAIR: REnormalizing Permuted Activations for Interpolation Repair. (**ICLR** 2023) [[paper]](https://arxiv.org/pdf/2211.08403.pdf) [[code]](https://github.com/KellerJordan/REPAIR)

- Population-based evolutionary gaming for unsupervised person re-identification. (**IJCV** 2023) [[paper]](https://arxiv.org/abs/2306.05236)

- ZipIt! Merging Models from Different Tasks without Training. (**ArXiv** 2023) [[paper]](https://arxiv.org/abs/2305.03053)

- Towards Efficient Visual Adaption via Structural Re-parameterization. (**ArXiv** 2023) [[paper]](https://arxiv.org/abs/2302.08106) [[code]](https://github.com/luogen1996/repadapter)

- Deep Model Fusion: A Survey. (**ArXiv** 2023) [[paper]](https://arxiv.org/abs/2309.15698)

### 2022

- Merging Models with Fisher-Weighted Averaging. (**NeurIPS** 2022) [[paper]](https://arxiv.org/abs/2111.09832) [[code]](https://github.com/mmatena/model_merging)

- Deep Model Reassembly. (**NeurIPS** 2022) [[paper]](https://arxiv.org/pdf/2210.17409.pdf) [[code]](https://github.com/Adamdad/DeRy)

- Factorizing Knowledge in Neural Networks. (**ECCV** 2022) [[paper]](https://arxiv.org/pdf/2207.03337.pdf) [[code]](https://github.com/Adamdad/KnowledgeFactor)

- GAN Cocktail: Mixing GANs without Dataset Access. (**ECCV** 2022) [[paper]](https://arxiv.org/pdf/2106.03847.pdf) [[code]](https://github.com/omriav/GAN-cocktail)

- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. (**ICML** 2022) [[paper]](https://arxiv.org/abs/2203.05482) [[code]](https://github.com/mlfoundations/model-soups)

- Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning. (**ICML** 2022) [[paper]](https://proceedings.mlr.press/v162/liu22k/liu22k.pdf) [[code]](https://github.com/Thinklab-SJTU/GAMF)

### 2021

- Amalgamating Knowledge From Heterogeneous Graph Neural Networks. (**ICML** 2022) [[paper]](https://openaccess.thecvf.com/content/CVPR2021/papers/Jing_Amalgamating_Knowledge_From_Heterogeneous_Graph_Neural_Networks_CVPR_2021_paper.pdf) [[code]](https://github.com/ycjing/AmalgamateGNN.PyTorch)

### 2020

- Model Fusion via Optimal Transport. (**NeurIPS** 2020) [[paper]](https://arxiv.org/pdf/1910.05653.pdf) [[code]](https://github.com/sidak/otfusion)

- Collaboration by Competition: Self-coordinated Knowledge Amalgamation for Multi-talent Student Learning. (**ECCV** 2020) [[paper]](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123510630.pdf)

- Multiple Expert Brainstorming for Domain Adaptive Person Re-identification. (**ECCV** 2020) [[paper]](https://arxiv.org/abs/2007.01546) [[paper]](https://github.com/YunpengZhai/MEB-Net)

- Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN. (**CVPR** 2020) [[paper]](https://openaccess.thecvf.com/content_CVPR_2020/papers/Ye_Data-Free_Knowledge_Amalgamation_via_Group-Stack_Dual-GAN_CVPR_2020_paper.pdf) [[paper]](https://github.com/ycjing/Awesome-Model-Merging/blob/main)

### 2019

- Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers. (**IJCAI** 2020) [[paper]](https://arxiv.org/abs/1905.11569)

- Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation. (**ICCV** 2019) [[paper]](https://arxiv.org/pdf/1908.07121.pdf) [[code]](https://github.com/ycjing/Awesome-Model-Merging/blob/main)

- Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More. (**CVPR** 2019) [[paper]](https://arxiv.org/pdf/1904.10167.pdf) [[code]](https://github.com/zju-vipa/KamalEngine)

- Amalgamating Knowledge towards Comprehensive Classification. (**AAAI** 2019) [[paper]](https://arxiv.org/pdf/1811.02796.pdf) [[code]](https://github.com/zju-vipa/KamalEngine)