{"id":18600839,"url":"https://github.com/autonomousvision/counterfactual_generative_networks","last_synced_at":"2025-04-10T18:31:44.260Z","repository":{"id":41824420,"uuid":"329864628","full_name":"autonomousvision/counterfactual_generative_networks","owner":"autonomousvision","description":"[ICLR'21] Counterfactual Generative Networks","archived":false,"fork":false,"pushed_at":"2021-05-13T06:15:11.000Z","size":33197,"stargazers_count":107,"open_issues_count":3,"forks_count":25,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-25T03:01:37.081Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-15T09:20:20.000Z","updated_at":"2024-10-22T09:22:46.000Z","dependencies_parsed_at":"2022-08-19T02:51:39.625Z","dependency_job_id":null,"html_url":"https://github.com/autonomousvision/counterfactual_generative_networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fcounterfactual_generative_networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fcounterfactual_generative_networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fcounterfactual_generative_networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fcounterfactual_generative_networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/counterfactual_generative_networks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248271772,"owners_count":21075800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T02:06:02.277Z","updated_at":"2025-04-10T18:31:39.253Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Counterfactual Generative Networks #\n\n#### [[Project]](https://sites.google.com/view/counterfactual-generation/home) [[PDF]](https://openreview.net/pdf?id=BXewfAYMmJw) [[Blog]](https://autonomousvision.github.io/cgn/) [[Music Video]](https://youtu.be/JDwaLueR35U) [[Colab]](https://colab.research.google.com/gist/xl-sr/ad93d3918e456fd4cd4aabdcaad87148/generating_counterfactuals.ipynb)\n\u003cimg src=\"media/teaser.gif\" width=\"600\"\u003e\n\nThis repository contains the code for the ICLR 2021 paper \"Counterfactual Generative Networks\" by [Axel Sauer](https://axelsauer.com/) and [Andreas Geiger](http://www.cvlibs.net/).\nIf you want to take the CGN for a spin and generate counterfactual images, you can try out the Colab below.\n\n[![CGN](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/gist/xl-sr/ad93d3918e456fd4cd4aabdcaad87148/generating_counterfactuals.ipynb) \u003cbr\u003e\n\nIf you find our code or paper useful, please cite\n```bibtex\n@inproceedings{Sauer2021ICLR,\n author =  {Axel Sauer, Andreas Geiger},\n title = {Counterfactual Generative Networks},\n booktitle = {International Conference on Learning Representations (ICLR)},\n year = {2021}}\n```\n\n## Setup ##\nInstall anaconda (if you don't have it yet)\n```Shell\nwget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh\nbash Anaconda3-2020.11-Linux-x86_64.sh\nsource ~/.profile\n```\n\nClone the repo and build the environment\n\n```Shell\ngit clone https://github.com/autonomousvision/counterfactual_generative_networks\ncd counterfactual_generative_networks\nconda env create -f environment.yml\nconda activate cgn\n```\n\nMake all scripts executable: ```chmod +x  scripts/*```. Then, download the datasets (colored MNIST, Cue-Conflict, IN-9) and the pre-trained weights (CGN, U2-Net). Comment out the ones you don't need.\n\n```Shell\n./scripts/download_data.sh\n./scripts/download_weights.sh\n```\n\n## MNISTs ##\n\u003cimg src=\"media/mnist_real_cf.png\" width=\"100%\"\u003e\n\nThe main functions of this sub-repo are:\n- Generating the MNIST variants\n- Training a CGN\n- Generating counterfactual datasets\n- Training a shape classifier\n\n### Train the CGN ###\n\nWe provide well-working configs and weights in ```mnists/experiments```. To train a CGN on, e.g., Wildlife MNIST, run\n\n```Shell\npython mnists/train_cgn.py --cfg mnists/experiments/cgn_wildlife_MNIST/cfg.yaml\n```\nFor more info, add `--help`. Weights and samples will be saved in ```mnists/experiments/```.\n\n### Generate Counterfactual Data ###\n\nTo generate the counterfactuals for, e.g., double-colored MNIST, run\n\n```Shell\npython mnists/generate_data.py \\\n--weight_path mnists/experiments/cgn_double_colored_MNIST/weights/ckp.pth \\\n--dataset double_colored_MNIST --no_cfs 10 --dataset_size 100000\n```\n\nMake sure that you provide the right dataset together with the weights. You can adapt the weight-path to use your own weights. The command above generates ten counterfactuals per shape.\n\n### Train the Invariant Classifier ###\n\n The classifier training uses Tensor datasets, so you need to save the non-counterfactual datasets as tensors.\n For DATASET = {colored_MNIST, double_colored_MNIST, wildlife_MNIST}, run\n\n```Shell\npython mnists/generate_data.py --dataset DATASET\n```\n\nTo train, e.g., a shape classifier (invariant to foreground and background) on wildlife MNIST, run,\n\n```Shell\npython mnists/train_classifier.py --dataset wildlife_MNIST_counterfactual\n```\n\nAdd ```--help``` for info on the available options and arguments. The hyperparameters are unchanged for all experiments.\n\n## ImageNet ##\n\nThe main functions of this sub-repo are:\n- Training a CGN\n- Generating data (samples, interpolations, or a whole dataset)\n- Training an invariant classifier ensemble\n\n### Train the CGN ###\n\u003cimg src=\"media/training.gif\" width=\"100%\"\u003e\n\nRun\n\n```Shell\npython imagenet/train_cgn.py --model_name MODEL_NAME\n```\nThe default parameters should give you satisfactory results. You can change them in ```imagenet/config.yml```. For more info, add `--help`. Weights and samples will be saved in ```imagenet/data/MODEL_NAME```.\n\n### Generate Counterfactual Data ###\n\u003cimg src=\"media/cfs.png\" width=\"100%\"\u003e\n\n__Samples__. To generate a dataset of counterfactual images, run\n\n```Shell\npython imagenet/generate_data.py --mode random --weights_path imagenet/weights/cgn.pth \\\n--n_data 100 --weights_path imagenet/weights/cgn.pth --run_name RUN_NAME \\\n--truncation 0.5 --batch_sz 1\n```\n\nThe results will be saved in ```imagenet/data```. For more info, add ```--help```. If you want to save only masks, textures, etc., you need to change this directly in the code (see line 206).\n\n\nThe labels will be stored in a csv file. You can read them as follows:\n```Python\nimport pandas as pd\ndf = pd.read_csv(path, index_col=0)\ndf = df.set_index('im_name')\nshape_cls = df['shape_cls']['RUN_NAME_0000000.png']\n```\n__Generating a dataset to train a classfier__. Produce one dataset with ```--run_name train```, the other\none with ``` --run_name val```. If you have several GPUs available, you can index the name, e.g., ```--run_name\ntrain_GPU_NUM```. The class ```ImagenetCounterfactual``` will glob all these datasets and generate a single, big\ntraining set. Make sure to set ```--batch_sz 1```. With a larger batch size, a batch will be saved as a single png; this is useful for visualization, not for training.\n\n__Interpolations__. To generate interpolation sheets, e.g., from a barn (425) to whale (147), run\n\n```Shell\npython imagenet/generate_data.py --mode fixed_classes \\\n--n_data 1 --weights_path imagenet/weights/cgn.pth --run_name barn_to_whale \\\n--truncation 0.3 --interp all --classes 425 425 425 --interp_cls 147 --save_noise\n```\n\nYou can also do counterfactual interpolations, i.e., interpolating only over, e.g., shape, by setting `--interp shape`. \n\n__Interpolation Gif__. To generate a gif like in the teaser (Sample an image of class $1, than interpolate to shape $2, then background $3, then shape $4, and finally back to $1), run\n```Shell\n./scripts/generate_teaser_gif.sh 992 293 147 330\n```\nThe positional arguments are the classes, see [imagenet labels](https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a) for the available options.\n\n### Train the Invariant Classifier Ensemble ###\n__Training__. First, you need to make sure that you have all datasets in ```imagenet/data/```. Download Imagenet, e.g., from [Kaggle](https://www.kaggle.com/c/imagenet-object-localization-challenge/data), produce a counterfactual dataset (see above), and download the Cue-Conflict and BG-Challenge dataset (via the download script in ```scripts```).\n\nTo train a classifier on a single GPU with a pre-trained Resnet-50 backbone, run\n```Shell\npython imagenet/train_classifier.py -a resnet50 -b 32 --lr 0.001 -j 6 \\\n--epochs 45 --pretrained --cf_data CF_DATA_PATH --name RUN_NAME\n```\nAgain, add ```--help``` for more information on the possible arguments.\n\n__Distributed Training__. To switch to multi-GPU training, run ```echo $CUDA_VISIBLE_DEVICES``` to see if the GPUs are visible. In the case of a\nsingle node with several GPUs, you can run, e.g.,\n```Shell\npython imagenet/train_classifier.py -a resnet50 -b 256 --lr 0.001 -j 6 \\\n--epochs 45 --pretrained --cf_data CF_DATA_PATH --name RUN_NAME \\\n--rank 0 --multiprocessing-distributed --dist-url tcp://127.0.0.1:8890 --world-size 1\n```\nIf your setup differs, e.g., several GPU machines, you need to adapt the rank and world size.\n\n__Visualization__. To visualize the Tensorboard outputs, run ```tensorboard --logdir=imagenet/runs``` and open the local address in your browser.\n\n## Acknowledgments ##\nWe like to acknowledge several repos of which we use parts of code, data, or models in our implementation:\n\n- colored MNIST by [feidfoe](https://github.com/feidfoe/learning-not-to-learn)\n- pre-trained BigGAN by [huggingface](https://github.com/huggingface/pytorch-pretrained-BigGAN)\n- U2-Net by [NathanUA](https://github.com/NathanUA/U-2-Net/)\n- Imagenet training by and with [pytorch](https://github.com/pytorch/examples/tree/master/imagenet)\n- Style-vs-Shape evaluation by [rgeirhos](https://github.com/rgeirhos/texture-vs-shape)\n- BG-Gap evaluation by [MadryLab](https://github.com/MadryLab/backgrounds_challenge)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Fcounterfactual_generative_networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Fcounterfactual_generative_networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Fcounterfactual_generative_networks/lists"}