{"id":13488064,"url":"https://github.com/moatifbutt/color-peel","last_synced_at":"2025-08-16T15:32:45.685Z","repository":{"id":247245771,"uuid":"823611497","full_name":"moatifbutt/color-peel","owner":"moatifbutt","description":"we propose to generate a series of geometric shapes with target colors to disentangle (or peel off ) the target colors from the shapes. By jointly learning on multiple color-shape images, we found that the method can successfully disentangle the color and shape concepts.","archived":false,"fork":false,"pushed_at":"2024-10-07T17:28:31.000Z","size":9280,"stargazers_count":48,"open_issues_count":1,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-30T23:36:23.561Z","etag":null,"topics":["color-learning","colors","diffusion-models","generative-model","prompt-learning","pytorch","stable-diffusion","text-to-image","vae"],"latest_commit_sha":null,"homepage":"https://moatifbutt.github.io/colorpeel/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moatifbutt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-03T11:12:11.000Z","updated_at":"2024-10-23T06:11:56.000Z","dependencies_parsed_at":"2024-07-11T12:04:11.927Z","dependency_job_id":"55430424-5064-4326-9fa3-ee57008360ca","html_url":"https://github.com/moatifbutt/color-peel","commit_stats":null,"previous_names":["moatifbutt/colorpeel_cpl","moatifbutt/color-peel"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moatifbutt%2Fcolor-peel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moatifbutt%2Fcolor-peel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moatifbutt%2Fcolor-peel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moatifbutt%2Fcolor-peel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moatifbutt","download_url":"https://codeload.github.com/moatifbutt/color-peel/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230043979,"owners_count":18164093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["color-learning","colors","diffusion-models","generative-model","prompt-learning","pytorch","stable-diffusion","text-to-image","vae"],"created_at":"2024-07-31T18:01:09.037Z","updated_at":"2024-12-17T00:29:39.790Z","avatar_url":"https://github.com/moatifbutt.png","language":"Python","funding_links":[],"categories":["New Concept Learning"],"sub_categories":[],"readme":"# ColorPeel: Color Prompt Learning with Diffusion Models via  Color and Shape Disentanglement [ECCV 2024]\n\n![teaser](assets/teaser_4.jpg)\n\n***TL;DB*** Given the RGB triplets or color coordinates, ColorPeel generates basic 2D or 3D geometries with target colors for color learning. This facilitates the disentanglement of color and shape concepts, allowing for personalized color usage in image generation.\n\n## ColorPeel: Color Prompt Learning with Diffusion Models via  Color and Shape Disentanglement\n[Muhammad Atif Butt](https://scholar.google.com/citations?user=vf7PeaoAAAAJ\u0026hl=en), [Kai Wang](https://scholar.google.com/citations?user=j14vd0wAAAAJ\u0026hl=en), [Javier Vazquez-Corral](https://scholar.google.com/citations?user=gjnuPMoAAAAJ\u0026hl=en),  [Joost van de Weijer](https://scholar.google.com/citations?user=Gsw2iUEAAAAJ\u0026hl=en)\n\n[[Paper](http://arxiv.org/pdf/2407.07197)] [[arXiv](http://arxiv.org/abs/2407.07197)] [[Project](https://moatifbutt.github.io/colorpeel/)] [[Poster](https://github.com/moatifbutt/color-peel/blob/main/assets/ECCV2024_ColorPeel_.pdf)]\n\n\u003chr\u003e\n\n## Installations (for local execution with PyTorch)\nBefore running the scripts, make sure to install diffusers from source. Note that ColorPeel is developed on **Diffusers 0.17.0**.\nTo install diffusers from source, do the following steps:\n\n```sh\ngit clone https://github.com/huggingface/diffusers\ncd diffusers\npip install -e .\n```\n\nAfter successful installation, download/clone the **ColorPeel** repoistory.\n\n```sh\nhttps://github.com/moatifbutt/color-peel.git\ncd color-peel\npip install -r requirements.txt\n```\n\nAnd initialize an 🤗Accelerate environment with:\n\n```sh\naccelerate config\n```\n\nOr for a default accelerate configuration without answering questions about your environment.\n\n```sh\naccelerate config default\n```\n\n## Dataset\nWe provide two sample datasets for learning colors from 2D and 3D shapes. These datasets are available in data repository along with the `src/concept_json/instances_3d.json` which contain information regarding the class images and their corresponding conditioning prompts.\n\n**Custom Dataset (2D)**: Users can also create their own 2D dataset using the following script.\n\n```sh\npython src/draw_shapes.py 512 --shapes circle rectangle --rgb_values \"235,33,33\" \"33,235,33\" \"33,33,235\" \"235,235,33\" --out data/dataset\n```\n**Custom Dataset (3D)**: We design our 3D shapes in blender. The rendering script will be released soon.\n\n## Train\nNow, we are all set to start training. After setting up the paths in `train/train.sh`, run the following.\n\n```sh\n./src/train/train_colorpeel.sh\n```\n\n## Test\nAfter completing the training, the model will be saved in `models` directory. Run the following command for inference.\n\n```sh\npython src/test.py --exp model_name\n```\n\n# Future Work\nWe have experimented with mapping various colors from color spaces into color prompt embeddings. However, we encountered convergence issues that we are currently unable to resolve. For those interested in learning multiple colors for practical applications, we suggest developing a training scheme based on Textual Inversion, which has demonstrated satisfactory performance. As for the task of mapping color spaces into text embeddings, we leave this as a future research direction for the community to explore.\n\n\n# Citation\n\nIf you like our work, please cite our paper:\n\n```\n@inproceedings{butt2024colorpeel, \n    title={ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement}, \n    author={Muhammad Atif Butt and Kai Wang and Javier Vazquez-Corral and Joost van de Weijer},\n    booktitle={European Conference on Computer Vision}, \n    year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoatifbutt%2Fcolor-peel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoatifbutt%2Fcolor-peel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoatifbutt%2Fcolor-peel/lists"}