https://github.com/xzLcan/CusConcept

CusConcept: Customized Visual Concept Decomposition with Diffusion Models
https://github.com/xzLcan/CusConcept

Last synced: 13 days ago
JSON representation

CusConcept: Customized Visual Concept Decomposition with Diffusion Models

• Host: GitHub
• URL: https://github.com/xzLcan/CusConcept
• Owner: xzLcan
• Created: 2024-09-18T17:17:28.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-11-23T17:35:09.000Z (3 months ago)
• Last Synced: 2024-11-23T18:25:56.421Z (3 months ago)
• Language: Python
• Homepage:
• Size: 75.2 KB
• Stars: 4
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code

README

        
# CusConcept

![License](https://img.shields.io/badge/license-MIT-lightgray)

[![arXiv](https://img.shields.io/badge/arXiv-2410.00398%20-b31b1b)](https://arxiv.org/abs/2410.00398)

This is the official PyTorch codes for the paper:  

[**CusConcept: Customized Visual Concept Decomposition with Diffusion Models**](https://arxiv.org/abs/2410.00398)  

Zhi Xu\*,

Shaozhe Hao\*,

Kai Han†  

*WACV 2025*

## Set-up

Create a conda environment `CUS` using:

```

conda env create -f environment.yml

conda activate CUS

```

## Extracting CLIP text embeddings

To avoid computing the CLIP text embeddings for the entire vocabulary each time, this code is to extract the CLIP embedding for the entire vocabulary once, and save the embeddings such that you can load them for each concept.

```

python save_dictionary_embeddings.py

```

## Training

Create a new folder that contains an image. For example, download [our dataset](https://drive.google.com/drive/folders/1XvPE-UOwkYM7gVVTj9PlcoTQPKKaRbLn?usp=drive_link) and put it under the root path. You can specify any attribute axis and query the LLM to obtain the corresponding attribute vocabulary, then store it. You can change `--train_data_dir` to the image path and change `vocabulary_path` to the vocabulary path in bash file `scripts/run.sh`. You can specify `--output_dir` to save the checkpoints and generated images. 

When the above is ready, run the following to start training:

```

bash scripts/run.sh

```

## Citation

If you use this code in your research, please consider citing our paper:

```bibtex

@misc{xu2024cusconceptcustomizedvisualconcept,

    title={CusConcept: Customized Visual Concept Decomposition with Diffusion Models}, 

    author={Zhi Xu and Shaozhe Hao and Kai Han},

    journal={arXiv preprint arXiv:2410.00398},

    year={2024}

}

```

## Acknowledgements

This code repository is based on the great work of [Conceptor](https://github.com/hila-chefer/Conceptor). Thanks!