https://github.com/ermongroup/fairgen
Fair Generative Modeling via Weak Supervision
https://github.com/ermongroup/fairgen
Last synced: 5 months ago
JSON representation
Fair Generative Modeling via Weak Supervision
- Host: GitHub
- URL: https://github.com/ermongroup/fairgen
- Owner: ermongroup
- Created: 2020-06-26T06:49:07.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-11-22T06:29:54.000Z (almost 3 years ago)
- Last Synced: 2025-03-31T16:13:42.414Z (6 months ago)
- Language: Jupyter Notebook
- Size: 92.3 MB
- Stars: 20
- Watchers: 4
- Forks: 8
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Fair Generative Modeling via Weak Supervision
This repo contains a reference implementation for fairgen as described in the paper:
> Fair Generative Modeling via Weak Supervision
> [Kristy Choi*](http://kristychoi.com/), [Aditya Grover*](http://aditya-grover.github.io/), [Trisha Singh](https://profiles.stanford.edu/trisha-singh), [Rui Shu](http://ruishu.io/about/), [Stefano Ermon](https://cs.stanford.edu/~ermon/)
> International Conference on Machine Learning (ICML), 2020.
> Paper: https://arxiv.org/abs/1910.12008
## 1) Data setup:
(a) Download the CelebA dataset here (http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) into the `data/` directory (if elsewhere, note the path for step b). Of the download links provided, choose `Align&Cropped Images` and download `Img/img_align_celeba/` folder, `Anno/list_attr_celeba.txt`, and `Eval/list_eval_partition.txt` to `data/`.
(b) Preprocess the CelebA dataset for faster training:
```
python3 preprocess_celeba.py --data_dir=/path/to/downloaded/dataset/celeba/ --out_dir=../data --partition=train
```
You should run this script for `--partition=[train, val, test]` to cache all the necessary data. The preprocessed files will then be saved in `data/`.
To split the data for multiple attributes, check `notebooks/multi-attribute data and unbiased FID splits.ipynb`.
## 2) Pre-train attribute classifier
For a single-attribute:
```
python3 train_attribute_clf.py celeba ./results/attr_clf
```
For multiple attributes, add the `--multi=True` flag.
```
python3 train_attribute_clf.py celeba ./results/multi_clf -- multi=True
```
Then, the trained attribute classifier will be saved in `./results/attr_clf` (`./results/multi_clf`) and will be used for downstream evaluation for generative model training. Note the path where these classifiers are saved, as they will be needed for GAN training + evaluation.
## 3) Pre-train density ratio classifier
The density ratio classifier should be trained for the appropriate `bias` and `perc` setting, which can be adjusted in the script below:
```
python3 get_density_ratios.py celeba celeba --perc=[0.1, 0.25, 0.5, 1.0] --bias=[90_10, 80_20, multi]
```
Note that the best density ratio classifier will be saved in its corresponding directory under `./data/`.
## 4) Pre-compute unbiased FID scores:
We have provided both (a) biased and (b) unbiased FID statistics in the `fid_stats/` directory.
(a) `fid_stats/fid_stats_celeba.npz` contains the original activations from the *entire* CelebA dataset, as in: https://github.com/ajbrock/BigGAN-PyTorch
(b) `fid_stats/unbiased_all_gender_fid_stats.npz` contains activations from the entire CelebA dataset, where the gender attribute (female, male) are balanced.
(c) `fid_stats/unbiased_all_multi_fid_stats.npz` contains activations from the entire CelebA dataset, where the 4 attribute classes (black hair, other hair, female, male) are balanced.
These pre-computed FID statistics are for model checkpointing (during GAN training) and downstream evaluation of sample quality only, and should be substituted for other statistics when using a different dataset/attribute splits.
## 5) Train generative model (BigGAN)
A sample script to train the model can be found in `scripts/`:
`bash run_celeba_90_10_perc1.0_impweight.sh`
You should add different arguments for different model configurations. For example:
(a) for the multi-attribute setting, append ` --multi 1`
(b) for the equi-weighted baseline, append ` --reweight 0`
(c) for the conditional baseline, append `--conditional 1 --y 1 --reweight 0`
(d) for the importance-weighted model, append `--reweight 1 --alpha 1.0`
Note the argument for `--name_suffix my_experiment`, as you will need it for sampling and computing FID scores.
## 6) Sample from trained model
A sample script to sample from the (trained) model can be found in `scripts/`:
`bash sample_celeba_90_10_perc1.0_impweight.sh`
You can either append the argument `--load_weights name_of_weights` to load a specific set of weights, or pass in the `--name_suffix my_experiment` argument for the script to find the most recent checkpoint with the best FID.
## 7) Compute FID scores
To compute FID scores after running the sampling script, (using the original Tensorflow implementation), run the following:
`python3 fast_fid.py my_experiment --multi=[True,False] --n_replicates=10`
This code assumes that there are 10 sets (`n_replicates`) of 10K samples generated from the model (as per `sample.py`), and will evaluate the samples on both (a) the original FID scores and (b) unbiased FID scores (as per Step #4). `my_experiment` refers to the `--name_suffix my_experiment` parameter from Step 5.
## References
If you find this work useful in your research, please consider citing the following paper:
```
@article{grover2019fair,
title={Fair Generative Modeling via Weak Supervision},
author={Grover, Aditya and Choi, Kristy and Singh, Trisha and Shu, Rui and Ermon, Stefano},
journal={arXiv preprint arXiv:1910.12008},
year={2019}
}
```