https://github.com/ennengyang/rankone-moe
秩一专家混合用于多任务学习. 计算机学报, 2025. Mixture of Rank-One Experts for Multi-Task Learning. Chinese Journal of Computers, 2025.
https://github.com/ennengyang/rankone-moe
model-merging multi-task-learning
Last synced: 18 days ago
JSON representation
秩一专家混合用于多任务学习. 计算机学报, 2025. Mixture of Rank-One Experts for Multi-Task Learning. Chinese Journal of Computers, 2025.
- Host: GitHub
- URL: https://github.com/ennengyang/rankone-moe
- Owner: EnnengYang
- License: other
- Created: 2024-11-16T11:11:25.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-05-29T15:22:21.000Z (about 1 year ago)
- Last Synced: 2025-10-31T02:23:54.721Z (9 months ago)
- Topics: model-merging, multi-task-learning
- Language: Python
- Homepage:
- Size: 2.19 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# "秩一专家混合用于多任务学习. 计算机学报，2025."
## 开发环境配置
本项目依赖于[FusionBench-v0.2.4](https://github.com/tanganke/fusion_bench)，可参考该库来配置基本环境，或者按照如下步骤创建开发环境：
第一步：创建一个Conda环境
```bash
conda create --name rankone-moe python=3.12.4
```
第二步：激活Conda环境
```bash
conda activate rankone-moe
```
第三步：安装本项目的依赖的环境
```bash
git clone https://github.com/EnnengYang/RankOne-MoE
cd RankOne-MoE
pip install -e . # install the package in editable mode
```
## 运行实验
> 注意：本项目涉及到的所有数据集和模型权重均可在代码运行时自动下载，请确保您的网络能访问[huggingface](https://huggingface.co/)网站，您也可以考虑手动下载[相关资源](https://huggingface.co/tanganke).
- 实验：我们的RankOne-MoE方法在CLIP-ViT-B/32, CLIP-ViT-B/16, CLIP-ViT-L/14模型下的合并性能
```bash
bash examples/rankone_one/rankone_moe_gridsearch.sh
```
> ViT-B/32(8个任务)的运行结果在[examples/results/clip-vit-base-patch32](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch32)
> ViT-B/16(8个任务)的运行结果在[examples/results/clip-vit-base-patch16](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch16)
> ViT-L/14(8个任务)的运行结果在[examples/results/clip-vit-large-patch14](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-large-patch14)
> ViT-B/32(20个任务)的运行结果在[examples/results/clip-vit-base-patch32_20Tasks](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch32_20Tasks)
- 实验：基线方法们在CLIP-ViT-B/32模型下的合并性能(8个任务)
```bash
bash examples/rankone_one/baseline_b32.sh
```
> ViT-B/32(8个任务)的运行结果在[examples/results/clip-vit-base-patch32](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch32)
- 实验：基线方法们在CLIP-ViT-B/16模型下的合并性能(8个任务)
```bash
bash examples/rankone_one/baseline_b16.sh
```
> ViT-B/16(8个任务)的运行结果在[examples/results/clip-vit-base-patch16](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch16)
- 实验：基线方法们在CLIP-ViT-L/14模型下的合并性能(8个任务)
```bash
bash examples/rankone_one/baseline_l14.sh
```
> ViT-L/14(8个任务)的运行结果在[examples/results/clip-vit-large-patch14](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-large-patch14)
- 实验：基线方法们在CLIP-ViT-B/32模型下的合并性能(20个任务)
```bash
bash examples/rankone_one/baseline_b32_20tasks.sh
```
> ViT-B/32(20个任务)的运行结果在[examples/results/clip-vit-base-patch32_20Tasks](https://github.com/EnnengYang/RankOne-MoE/tree/main/examples/results/clip-vit-base-patch32_20Tasks)