Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/crisostomi/task-vectors-playground
https://github.com/crisostomi/task-vectors-playground
Last synced: 5 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/crisostomi/task-vectors-playground
- Owner: crisostomi
- License: mit
- Created: 2024-04-04T15:01:44.000Z (10 months ago)
- Default Branch: master
- Last Pushed: 2025-01-16T20:15:10.000Z (14 days ago)
- Last Synced: 2025-01-16T21:25:54.092Z (14 days ago)
- Language: Jupyter Notebook
- Size: 10.9 MB
- Stars: 2
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Task Vectors Playground
Playing with those task vectors
## Development installation
Setup the development environment:
```bash
git clone [email protected]:crisostomi/task-vectors-playground.git
cd task-vectors-playground
conda env create -f env.yaml
conda activate tvp
```
## Fine-tuning
First, choose a vision encoder in `{"ViT-B-16", "ViT-B-32", "ViT-L-14"}`. Check that the corresponding WandB artifact is present online (at the time of writing, only `ViT-B-16` is available).
Now choose one or more datasets for which you want to compute the task vectors, and set it in `conf/nn/data/default.yaml` under the `dataset` voice in the defaults. Currently available datasets are `{'svhn', 'mnist', 'cifar100', 'resisc45'}`, but any dataset in `tvp/data/datasets/*` can be used. It is enough to create the corresponding configuration file in `conf/nn/data/dataset/`.
Before fine-tuning the model on the dataset, check if the fine-tuned version is already among the artifacts in WandB. It should be named `__`. If it is not, use `src/scripts/finetune.py` to fine-tune the pretrained model over the chosen dataset. The corresponding configuration is `conf/finetune.yaml`.
## Applying task-vectors
Now the script is `src/scripts/use_task_vectors.py` and the configuration is `conf/task_vectors.yaml`. Task vectors to apply can be chosen in the `task_vectors.to_apply` voice in the config, which expects a list of dataset names.