https://github.com/MadryLab/copriors
Combining Diverse Feature Priors
https://github.com/MadryLab/copriors
Last synced: about 1 year ago
JSON representation
Combining Diverse Feature Priors
- Host: GitHub
- URL: https://github.com/MadryLab/copriors
- Owner: MadryLab
- License: mit
- Created: 2021-10-15T15:41:39.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-10-18T01:35:53.000Z (over 4 years ago)
- Last Synced: 2024-11-15T06:32:52.639Z (over 1 year ago)
- Language: Python
- Size: 24.4 KB
- Stars: 8
- Watchers: 6
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Combining Diverse Feature Priors
This repository contains code for reproducing the results of our paper.
Paper: https://arxiv.org/abs/2110.08220
Blog Post: http://gradientscience.org/copriors/
Important files:
```
Scripts:
pretrain_model.py: a script to pre-train the models on just the labeled data
cotrain.py: a script to co-train pretrained model(s)
sweep_final_models.py: a script to evaluate intermediate eras for a previously run cotrain
File Structure:
datasets:
datasets.py: the definition of the labeled/unlabeled/validation/test sets for our datasets
transforms.py: describes the different prior transforms and spurious tinting
co_training.py: contains the logic for model pre-training and co-training
models:
bagnet_custom.py: the architecture for the bagnets used in this paper
model_utils.py: utilities for loading and building models
```
To generate the pre-trained priors, run:
```
python pretrain_model.py --dataset --data-path --use_val --out-dir --arch --epochs 300 --lr --step_lr --step_lr_gamma --additional-transform
datasets: STLSub10, cifarsmallsub, celebaskewed
data-path: use torchvision datasets from https://pytorch.org/vision/stable/index.html
use-val: determines whether to use validation or test set for tensorboard metrics
arch: vgg16_bn, bagnetcustom32 (bagnet for CIFAR), bagnetcustom96thin (bagnet for celeba/stl10)
lr, step-lr, step-lr-gamma are hyperparameters who's exact values can be found in our appendix.
additional-transform: which prior to use. possibilities are NONE, CANNY, SOBEL (use NONE and a bagnet architecture for the bagnet prior)
Add --spurious TINT to train with a tint (as in the tinted STL-10 experiments)
```
After generating the priors, the models can be self (include one prior directory) or co-trained (include both prior directories) by running:
```
python cotrain.py --dataset --data-path --out-dir --input-dirs --input-dirs --epochs_per_era 300 --fraction 0.05 --eras 20 --epochs 400 --arch vgg16_bn --additional-transform NONE --lr --step_lr --step_lr_gamma --strategy STANDARD_CONSTANT
This command will self/co-train the input prior directories, saving a checkpoint for each era, and then finally train a standard model on the pseudo-labels after the eras are complete.
To use the pure co-training strategy, add --pure
To use tinting as in the STL-10 tinting experiments
```