https://github.com/crisostomi/cycle-consistent-model-merging

https://github.com/crisostomi/cycle-consistent-model-merging

Last synced: 12 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/crisostomi/cycle-consistent-model-merging
• Owner: crisostomi
• License: mit
• Created: 2023-08-03T15:08:05.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2024-08-01T10:19:39.000Z (3 months ago)
• Last Synced: 2024-08-01T11:48:52.517Z (3 months ago)
• Language: Jupyter Notebook
• Size: 5.49 MB
• Stars: 6
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Cycle Consistent Model Merging



    

    

    

    

    



Merging models in a cycle-consistent fashion.

## Development installation

Setup the development environment:

```bash

git clone [email protected]:crisostomi/cycle-consistent-model-merging.git

cd cycle-consistent-model-merging

conda env create -f env.yaml

conda activate ccmm

pre-commit install

```

Run the tests:

```bash

pre-commit run --all-files

pytest -v

```

### Update the dependencies

Re-install the project in edit mode:

```bash

pip install -e .[dev]

```

## Usage

All the scripts can be found under `src/scripts/`. Each script has a corresponding configuration file in `conf/matching` where you can change stuff as dataset and model to use. You can train models using `train.py` with a dataset and model of your choice.

### Matching two models

1. get the permutations to align the two models (identified by their seed in the config) by running `match_two_models.py`. The config is `conf/matching.yaml` (see inside the config to see the subconfigs).

2. evaluate the interpolation of the models using `evaluate_matched_models.py` and the same config used for the previous step. Be sure to have `matching.yaml` as config in the script itself.

To change the matching technique, you have to change the `matcher` in `conf/matching/match_two_models.yaml`. Each matcher has its own config file in `conf/matching/matcher/`.

To run all the pairs of models with different seeds, run `shell_scripts/run_all_seeds.sh`.

### Matching multiple models

1. get the permutations to align the models (identified by their seed in the config) by running `match_n_models.py`. The config is `conf/matching_n_models.yaml` (see inside the config to see the subconfigs).

2. evaluate the interpolation of the models using `evaluate_matched_models.py` and the same config used for the previous step.  Be sure to have `matching_n_models.yaml` as config in the script itself.

To change the matching technique, you have to change the `matcher` in `conf/matching/match_n_models.yaml`. Each matcher has its own config file in `conf/matching/matcher/`.

### Merging models

1. get the merged model by running `merge_n_models.py`.

2. evaluate the merged model using `evaluate_merged_model.py` and the same config used for the previous step.

To change the merging technique, you have to change the `merger` in `conf/matching/merge_n_models.yaml`. Each merger has its own config file in `conf/matching/merger/`.