https://github.com/borealisai/dynashare-mtl
PyTorch Implementation of DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning
https://github.com/borealisai/dynashare-mtl
dynamic-neural-networks multi-task-learning pytorch
Last synced: 12 months ago
JSON representation
PyTorch Implementation of DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning
- Host: GitHub
- URL: https://github.com/borealisai/dynashare-mtl
- Owner: BorealisAI
- License: other
- Created: 2024-03-07T21:48:36.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-21T06:25:12.000Z (about 2 years ago)
- Last Synced: 2025-02-16T03:36:09.413Z (about 1 year ago)
- Topics: dynamic-neural-networks, multi-task-learning, pytorch
- Language: Python
- Homepage:
- Size: 6.55 MB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning
## Introduction
In this paper (DynaShare), we extend the task-conditioned parameter sharing approach pioneered by AdaShare, and condition parameter sharing on both the task and the intermediate feature representations. **DynaShare** learns a hierarchical gating policy consisting of a task-specific policy for coarse layer selection and gating units for individual input instances, which work together to determine the execution path at inference time. Experiments on the **NYU v2**, **Cityscapes** and **MIMIC-III** datasets demonstrate the superiority and efficiency of the proposed approach across problem domains.
Here is [the link](https://openaccess.thecvf.com/content/CVPR2023W/ECV/html/Rahimian_DynaShare_Task_and_Instance_Conditioned_Parameter_Sharing_for_Multi-Task_Learning_CVPRW_2023_paper.html) for our CVPR version.
Welcome to cite our work if you find it is helpful to your research.
```
@InProceedings{Rahimian_2023_CVPR,
author = {Rahimian, E. and Javadi, G. and Tung, F. and Oliveira, G.},
title = {DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2023},
pages = {4535-4543}
}
```
## Experiment Environment
Our implementation is in Pytorch. We train and test our model on 1 `Quadro RTX 5000` GPU for `NYU v2 2-task` and `CityScapes 2-task`.
We use `python3.6` and please refer to [this link](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands) to create a `python3.6` conda environment.
Install the listed packages in the virual environment:
```
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
conda install matplotlib
conda install -c menpo opencv
conda install pillow
conda install -c conda-forge tqdm
conda install -c anaconda pyyaml
conda install scikit-learn
conda install -c anaconda scipy
pip install tensorboardX
```
## Datasets
### MIMIC3
This dataset is not public available, so you need to submit a request to work with this data
at [https://mimic.mit.edu/iii/gettingstarted/](https://mimic.mit.edu/iii/gettingstarted/). We followed the
pre-processing steps available
here: [https://github.com/YerevaNN/mimic3-benchmarks](https://github.com/YerevaNN/mimic3-benchmarks) and saved the
results in a folder named `mimic_dataset`.
## Training
### Policy Learning Phase
Please execute train.py for policy learning, using the command:
```
python train.py --config yamls/adashare/mimic_iii_4tasks.yml --gpus 0
```
### Policy Learning Phase new policy learning
```
python train_policy_learning.py --config yamls/adashare/mimic_iii_4tasks_aig.yml --gpus 1
```
## Retrain Phase
After Policy Learning Phase, we sample 8 different architectures and execute re-train.py for retraining.
```
python re-train.py --config --gpus --exp_ids
```
### Retrain Phase AIG
```
python re-train.py --config yamls/adashare/mimic_iii_4tasks_aig.yml --gpus 1 --exp_ids 2
```
, shows the seed index in the config file.
## Test Phase
After Retraining Phase, execute test.py for get the quantitative results on the test set.
```
python test.py --config --gpus --exp_ids
```
and example of using different seed for mimic dataset:
```
python test.py --config yamls/adashare/mimic_iii_4tasks.yml --gpus 0 --exp_ids 2
```
## NYU v2 and CityScapes
For full training and testing access to DynaShare on NYU v2 and Cityscapes access DynaShare folder.
```
cd DynaShare/
```
Please download the formatted datasets for `NYU v2` [here](https://drive.google.com/file/d/11pWuQXMFBNMIIB4VYMzi9RPE-nMOBU8g/view?usp=sharing)
The formatted `CityScapes` can be found [here](https://drive.google.com/file/d/1WrVMA_UZpoj7voajf60yIVaS_Ggl0jrH/view?usp=sharing).
Remember to change the `dataroot` to your local dataset path in all `yaml` files in the `./yamls/`.
## Training
### Policy Learning Phase
Please execute `train.py` for policy learning in **Train_AdaShare** folder, using the command
```
python train.py --config --gpus
```
For example, `python train.py --config yamls/adashare/nyu_v2_2task.yml --gpus 0`.
Sample `yaml` files are under `yamls/adashare`
**Note:** The train phase is exactly the same as Adashare paper [the link](https://arxiv.org/pdf/1911.12423.pdf).
### Retrain Phase
After Policy Learning Phase, we sample 8 different architectures and execute `re-train.py` for retraining.
```
python re-train.py --config --gpus --exp_ids
```
where we use different `--exp_ids` to specify different random seeds and generate different architectures. The best performance of all 8 runs is reported in the paper.
For example, `python re-train.py --config yamls/adashare/nyu_v2_2task_aig.yml --gpus 0 --exp_ids 0`.
## Test/Inference
After Retraining Phase, execute `test.py` for get the quantitative results on the test set.
```
python test.py --config --gpus --exp_ids
```
For example, `python test.py --config yamls/adashare/nyu_v2_2task_test.yml --gpus 0 --exp_ids 0`.