{"id":13958453,"url":"https://github.com/snap-research/CAT","last_synced_at":"2025-07-21T00:30:57.345Z","repository":{"id":41951869,"uuid":"346117995","full_name":"snap-research/CAT","owner":"snap-research","description":"[CVPR 2021] Teachers Do More Than Teach: Compressing Image-to-Image Models (CAT)","archived":false,"fork":false,"pushed_at":"2022-04-22T16:15:47.000Z","size":82453,"stargazers_count":181,"open_issues_count":6,"forks_count":20,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-05-31T09:47:50.537Z","etag":null,"topics":["compression","cyclegan","deep-learning","gan","gaugan","image-to-image","pix2pix","pytorch"],"latest_commit_sha":null,"homepage":"https://dejqk.github.io/GAN_CAT/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snap-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-09T19:18:29.000Z","updated_at":"2025-02-17T13:31:14.000Z","dependencies_parsed_at":"2022-09-24T04:40:15.518Z","dependency_job_id":null,"html_url":"https://github.com/snap-research/CAT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/snap-research/CAT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2FCAT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2FCAT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2FCAT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2FCAT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snap-research","download_url":"https://codeload.github.com/snap-research/CAT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2FCAT/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266221247,"owners_count":23894964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","cyclegan","deep-learning","gan","gaugan","image-to-image","pix2pix","pytorch"],"created_at":"2024-08-08T13:01:36.318Z","updated_at":"2025-07-21T00:30:52.332Z","avatar_url":"https://github.com/snap-research.png","language":"Python","funding_links":[],"categories":["其他_机器视觉"],"sub_categories":["网络服务_其他"],"readme":"# CAT\n[CVPR](https://openaccess.thecvf.com/content/CVPR2021/html/Jin_Teachers_Do_More_Than_Teach_Compressing_Image-to-Image_Models_CVPR_2021_paper.html) | [arXiv](https://arxiv.org/abs/2103.03467) | [website](https://dejqk.github.io/GAN_CAT) | [Tutorial (Image-to-Image)](https://lensstudio.snapchat.com/guides/machine-learning/lens-templates/) (Our method can be used in mobile devices!)\n\n\u003ctable cellpadding=\"0\" cellspacing=\"0\" \u003e\n  \u003ctr\u003e\n    \u003ctd  align=\"center\"\u003eInput\u003cbr\u003e \u003cimg src=\"images/input.gif\" width=200px\u003e\u003c/td\u003e\n    \u003ctd  align=\"center\"\u003eNight\u003cbr\u003e \u003cimg src=\"images/night.gif\" width=200px\u003e\u003c/td\u003e\n    \u003ctd  align=\"center\"\u003eStyle\u003cbr\u003e \u003cimg src=\"images/style.gif\" width=200px\u003e\u003c/td\u003e\n    \u003ctd  align=\"center\"\u003eAnime\u003cbr\u003e \u003cimg src=\"images/anime.gif\" width=200px\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/comparison.png\" width=400\u003e\n\u003c/p\u003e\n\nPytorch implementation of our method for compressing image-to-image models. \u003cbr\u003e\n[Teachers Do More Than Teach: Compressing Image-to-Image Models](https://arxiv.org/abs/2103.03467)  \n[Qing Jin](https://scholar.google.com/citations?user=X9iggBcAAAAJ\u0026hl=zh-CN)\u003csup\u003e1\u003c/sup\u003e, [Jian Ren](https://alanspike.github.io/)\u003csup\u003e2\u003c/sup\u003e, [Oliver J. Woodford](https://scholar.google.com/citations?user=6czTBiUAAAAJ\u0026hl=en), [Jiazhuo Wang](https://www.linkedin.com/in/jiazhuo-wang-065624102)\u003csup\u003e2\u003c/sup\u003e, [Geng Yuan](https://scholar.google.com/citations?user=tBIAgtgAAAAJ)\u003csup\u003e1\u003c/sup\u003e, [Yanzhi Wang](https://coe.northeastern.edu/people/wang-yanzhi/)\u003csup\u003e1\u003c/sup\u003e, [Sergey Tulyakov](http://www.stulyakov.com/)\u003csup\u003e2\u003c/sup\u003e  \n\u003csup\u003e1\u003c/sup\u003eNortheastern University, \u003csup\u003e2\u003c/sup\u003eSnap Inc. \u003cbr\u003e\nIn CVPR 2021.  \n\n## Overview\n\nCompression And Teaching (CAT) framework for compressing image-to-image models: ① Given a pre-trained teacher generator Gt, we determine the architecture of a compressed student generator Gs by eliminating those channels with smallest magnitudes of batch norm scaling factors. ② We then distill knowledge from the pretrained teacher Gt on the student Gs via a novel distillation technique, which maximize the similarity between features of both generators, defined in terms of kernel alignment (KA).\n\n\n\n\n## Prerequisites\n\n* Linux\n* Python 3\n* CPU or NVIDIA GPU + CUDA CuDNN\n\n## Getting Started\n\n### Installation\n\n- Clone this repo:\n\n  ```shell\n  git clone git@github.com:snap-research/CAT.git\n  cd CAT\n  ```\n\n- Install [PyTorch](https://pytorch.org) 1.7 and other dependencies (e.g., torchvision).\n\n  - For pip users, please type the command `pip install -r requirements.txt`.\n  - For Conda users, please create a new Conda environment using `conda env create -f environment.yml`.\n\n## Data Preparation\n\n### CycleGAN\n\n#### Setup\n\n* Download the CycleGAN dataset (e.g., horse2zebra).\n\n  ```shell\n  bash datasets/download_cyclegan_dataset.sh horse2zebra\n  ```\n\n* Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistic information for several datasets on [Google Drive Folder](https://drive.google.com/drive/folders/1kSH8vPuO6i_oAMzi3V6aVox3UQCtYIiB?usp=sharing).\n\n### Pix2pix\n\n#### Setup\n\n* Download the pix2pix dataset (e.g., cityscapes).\n\n  ```shell\n  bash datasets/download_pix2pix_dataset.sh cityscapes\n  ```\n\n### Cityscapes Dataset\n\nFor the Cityscapes dataset, we cannot provide it due to license issue. Please download the dataset from https://cityscapes-dataset.com and use the script [prepare_cityscapes_dataset.py](datasets/prepare_cityscapes_dataset.py) to preprocess it. You need to download `gtFine_trainvaltest.zip` and `leftImg8bit_trainvaltest.zip` and unzip them in the same folder. For example, you may put `gtFine` and `leftImg8bit` in `database/cityscapes-origin`. You need to prepare the dataset with the following commands:\n\n```shell\npython datasets/get_trainIds.py database/cityscapes-origin/gtFine/\npython datasets/prepare_cityscapes_dataset.py \\\n--gtFine_dir database/cityscapes-origin/gtFine \\\n--leftImg8bit_dir database/cityscapes-origin/leftImg8bit \\\n--output_dir database/cityscapes \\\n--table_path datasets/table.txt\n```\n\nYou will get a preprocessed dataset in `database/cityscapes` and a mapping table (used to compute mIoU) in `dataset/table.txt`.\n\n* Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistics for several datasets. For example,\n\n  ```shell\n  bash datasets/download_real_stat.sh cityscapes A\n  ```\n\n\n## Evaluation Preparation\n\n### mIoU Computation\n\nTo support mIoU computation, you need to download a pre-trained DRN model `drn-d-105_ms_cityscapes.pth` from http://go.yf.io/drn-cityscapes-models. By default, we put the drn model in the root directory of our repo. Then you can test our compressed models on cityscapes after you have downloaded our compressed models.\n\n### FID/KID Computation\n\nTo compute the FID/KID score, you need to get some statistical information from the groud-truth images of your dataset. We provide a script [get_real_stat.py](./get_real_stat.py) to extract statistical information. For example, for the map2arial dataset, you could run the following command:\n\n  ```shell\npython get_real_stat.py \\\n--dataroot database/map2arial \\\n--output_path real_stat/maps_B.npz \\\n--direction AtoB\n  ```\n\nFor paired image-to-image translation (pix2pix and GauGAN), we calculate the FID between generated test images to real test images. For unpaired image-to-image translation (CycleGAN), we calculate the FID between generated test images to real training+test images. This allows us to use more images for a stable FID evaluation, as done in previous unconditional GANs research. The difference of the two protocols is small. The FID of our compressed CycleGAN model increases by 4 when using real test images instead of real training+test images.\n\nKID is not supported for the cityscapes dataset.\n\n\n## Model Training\n\n### Teacher Training\nThe first step of our framework is to train a teacher model. For this purpose, please run the script `train_inception_teacher.sh` under the correponding folder named as the dataset, for example, run \n```shell \nbash scripts/cycle_gan/horse2zebra/train_inception_teacher.sh\n```\n\n### Student Training\nWith the pretrained teacher model, we can determine the architecture of student model under prescribed computational budget. For this purpose, please run the script `train_inception_student_XXX.sh` under the correponding folder named as the dataset, where `XXX` stands for the computational budget (in terms of FLOPs for this case) and can be different for different datasets and models. For example, for CycleGAN with Horse2Zebra dataset, our computational budget is 2.6B FLOPs, so we run \n```shell\nbash scripts/cycle_gan/horse2zebra/train_inception_student_2p6B.sh\n```\n\n### Pre-trained Models\nFor convenience, we also provide pretrained teacher and student models on [Google Drive Folder](https://drive.google.com/drive/folders/1WQbqgqwJdVT6Q-8Bs-vp24jJ3v1dgjU9?usp=sharing).\n\n## Model Evaluation\nWith pretrained teacher and student models, we can evaluate them on the dataset. For this purpose, please run the script `evaluate_inception_student_XXX.sh` under the corresponding folder named as the dataset, where `XXX` is the computational budget (in terms of FLOPs). For example, for CycleGAN with Horse2Zebra dataset where the computational budget is 2.6B FLOPs, please run \n```shell \nbash scripts/cycle_gan/horse2zebra/evaluate_inception_student_2p6B.sh\n```\n\n## Model Export\nThe final step is to export the trained compressed model as onnx file to run on mobile devices. For this purpose, please run the script `onnx_export_inception_student_XXX.sh` under the corresponding folder named as the dataset, where `XXX` is the computational budget (in terms of FLOPs). For example, for CycleGAN with Horse2Zebra dataset where the computational budget is 2.6B FLOPs, please run \n```shell\nbash scripts/cycle_gan/horse2zebra/onnx_export_inception_student_2p6B.sh\n```\nThis will create one .onnx file in addition to log files. \n\n## Citation\n\nIf you use this code for your research, please cite our [paper](https://arxiv.org/abs/2103.03467).\n```\n@inproceedings{jin2021teachers,\n  title={Teachers Do More Than Teach: Compressing Image-to-Image Models},\n  author={Jin, Qing and Ren, Jian and Woodford, Oliver J and Wang, Jiazhuo and Yuan, Geng and Wang, Yanzhi and Tulyakov, Sergey},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={13600--13611},\n  year={2021}\n}\n```\n\n\n## Acknowledgements\n\nOur code is developed based on [AtomNAS](https://github.com/meijieru/AtomNAS) and [gan-compression](https://github.com/mit-han-lab/gan-compression).\n\nWe also thank [pytorch-fid](https://github.com/mseitzer/pytorch-fid) for FID computation and [drn](https://github.com/fyu/drn) for mIoU computation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-research%2FCAT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnap-research%2FCAT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-research%2FCAT/lists"}