{"id":13747191,"url":"https://github.com/WisconsinAIVision/MixNMatch","last_synced_at":"2025-05-09T08:31:58.711Z","repository":{"id":61094504,"uuid":"224086560","full_name":"WisconsinAIVision/MixNMatch","owner":"WisconsinAIVision","description":"Pytorch implementation of MixNMatch","archived":false,"fork":false,"pushed_at":"2020-07-07T23:01:18.000Z","size":21172,"stargazers_count":974,"open_issues_count":12,"forks_count":189,"subscribers_count":55,"default_branch":"master","last_synced_at":"2024-11-10T08:30:42.252Z","etag":null,"topics":["deep-learning","disentangled-representations","fine-grained","gans","image-generation","image-manipulation","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WisconsinAIVision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-26T02:40:22.000Z","updated_at":"2024-10-28T10:56:29.000Z","dependencies_parsed_at":"2022-10-10T11:08:36.605Z","dependency_job_id":null,"html_url":"https://github.com/WisconsinAIVision/MixNMatch","commit_stats":null,"previous_names":["wisconsinaivision/mixnmatch","yuheng-li/mixnmatch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FMixNMatch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FMixNMatch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FMixNMatch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WisconsinAIVision%2FMixNMatch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WisconsinAIVision","download_url":"https://codeload.github.com/WisconsinAIVision/MixNMatch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224842622,"owners_count":17379000,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","disentangled-representations","fine-grained","gans","image-generation","image-manipulation","pytorch"],"created_at":"2024-08-03T06:01:19.754Z","updated_at":"2025-05-09T08:31:58.691Z","avatar_url":"https://github.com/WisconsinAIVision.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation \u003cbr\u003e [[Paper]](https://arxiv.org/abs/1911.11758)\n\n[Yuheng Li](https://github.com/Yuheng-Li),\n[Krishna Kumar Singh](http://krsingh.cs.ucdavis.edu/),\n[Utkarsh Ojha](https://utkarshojha.github.io/),\n[Yong Jae Lee](https://web.cs.ucdavis.edu/~yjlee/)\u003cbr\u003e\nUC Davis \u003cbr\u003e\nIn [CVPR, 2020](https://arxiv.org/abs/1911.11758)\n\n**1/31/2020 update: Code and models released.** \u003cbr\u003e\n\n## Demo Video\n\n[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/w36vnkIbyjs/0.jpg)](https://www.youtube.com/watch?v=w36vnkIbyjs)\n\nThis is our CVPR2020 presentation video [link](https://www.youtube.com/watch?v=MmFL59X-Xwg) \n\n## Web Demo\n\nFor interactive web demo [click here](http://vision1.idav.ucdavis.edu:8005/). This web demo is created by Yang Xue.\n\n## Requirements\n- Linux\n- Python 3.7\n- Pytorch 1.3.1\n- NVIDIA GPU + CUDA CuDNN\n\n## Getting started\n### Clone the repository\n```bash\ngit clone https://github.com/Yuheng-Li/MixNMatch.git\ncd MixNMatch\n```\n### Setting up the data\n\nDownload the formatted CUB data from this [link](https://drive.google.com/file/d/1ardy8L7Cb-Vn1ynQigaXpX_JHl0dhh2M/view?usp=sharing) and extract it inside the `data` directory\n\n### Downloading pretrained models\n\nPretrained models for CUB, Dogs and Cars are available at this [link](https://drive.google.com/drive/folders/1c4NtKyccBNDuh_vqB-KlzZpRv9cQxEI7?usp=sharing). Download and extract them in the `models` directory.\n\n\n## Evaluating the model\nIn `code`\n- Run `python eval.py --z path_to_pose_source_images --b path_to_bg_source_images --p path_to_shape_source_images --c path_to_color_source_images --out path_to_ourput --mode code_or_feature --models path_to_pretrained_models`\n- For example `python eval.py --z pose/pose-1.png --b background/background-1.png --p shape/shape-1.png --c color/color.png --mode code --models ../models  --out ./code-1.png`\n  - **NOTE**:(1) in feature mode pose source images will be ignored; (2) Generator, Encoder and Feature_extractor in models folder should be named as G.pth, E.pth and EX.pth  \n\n## Training your own model\nIn `code/config.py`:\n- Specify the dataset location in `DATA_DIR`.\n  - **NOTE**: If you wish to train this on your own (different) dataset, please make sure it is formatted in a way similar to the CUB dataset that we've provided.\n- Specify the number of super and fine-grained categories that you wish for FineGAN to discover, in `SUPER_CATEGORIES` and `FINE_GRAINED_CATEGORIES`.\n- For the first stage training run `python train_first_stage.py output_name`\n- For the second stage training run `python train_second_stage.py output_name path_to_pretrained_G path_to_pretrained_E`\n  - **NOTE**:  output will be in `output/output_name`\n  - **NOTE**:  `path_to_pretrained_G` will be  `output/output_name/Model/G_0.pth`\n  - **NOTE**:  `path_to_pretrained_E` will be  `output/output_name/Model/E_0.pth`\n- For example `python train_second_stage.py Second_stage ../output/output_name/Model/G_0.pth ../output/output_name/Model/E_0.pth`\n\n\n## Results\n\n### 1. Extracting all factors from differnet real images to synthesize a new image\n\u003cimg src='files/MixNMatch.png' align=\"middle\" width=1000\u003e\n\u003cbr\u003e\n\n### 2. Comparison between the feature and code mode\n\u003cimg src='files/main_result2.png' align=\"middle\" width=1000\u003e\n\u003cbr\u003e\n\n### 3. Manipulating real images by varying a single factor\n\u003cimg src='files/bird_vary.png' align=\"middle\" width=1000\u003e\n\u003cbr\u003e\n\n### 4. Inferring style from unseen data\nCartoon -\u003e image             |  Sketch -\u003e image\n:-------------------------:|:-------------------------:\n\u003cimg src='files/cartoon2img.png' align=\"middle\" width=450\u003e  |  \u003cimg src='files/sketch2img.png' align=\"middle\" width=450\u003e\n\u003cbr\u003e\n\n### 5. Converting a reference image according to a reference video\n\u003cp align=\"center\"\u003e\n\u003cimg src='files/img2gif2.gif' align=\"middle\" width=350\u003e\n\u003c/p\u003e\n\u003cbr\u003e\n\n## Citation\nIf you find this useful in your research, consider citing our work:\n```\n@inproceedings{li-cvpr2020,\n  title = {MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation},\n  author = {Yuheng Li and Krishna Kumar Singh and Utkarsh Ojha and Yong Jae Lee},\n  booktitle = {CVPR},\n  year = {2020}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWisconsinAIVision%2FMixNMatch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FWisconsinAIVision%2FMixNMatch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWisconsinAIVision%2FMixNMatch/lists"}