https://github.com/soon-yau/visconet

Official Repo of ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet
https://github.com/soon-yau/visconet

Last synced: 3 months ago
JSON representation

Official Repo of ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet

• Host: GitHub
• URL: https://github.com/soon-yau/visconet
• Owner: soon-yau
• License: other
• Created: 2023-12-07T14:34:51.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-10-17T23:07:55.000Z (8 months ago)
• Last Synced: 2024-10-20T10:19:37.030Z (8 months ago)
• Language: Jupyter Notebook
• Homepage: https://soon-yau.github.io/visconet/
• Size: 163 MB
• Stars: 25
• Watchers: 4
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code
• awesome - soon-yau/visconet - Official Repo of ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet (Jupyter Notebook)

README

        
## *ViscoNet*: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet

[Soon Yau Cheong](https://scholar.google.com/citations?user=dRot7GUAAAAJ&hl=en)

[Armin Mustafa](https://scholar.google.com/citations?user=0xOHqkMAAAAJ&hl=en)

[Andrew Gilbert](https://scholar.google.com/citations?user=NNhnVwoAAAAJ&hl=en)

 

 

[![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://youtu.be/3_6Zq3hk86Q)

https://github.com/soon-yau/visconet/assets/19167278/ae58b7ab-fa76-4253-8a10-46656f234b20

### Requirements

A suitable [conda](https://conda.io/) environment named `control` can be created

and activated with:

```

conda env create -f environment.yaml

conda activate control

```

### Files

All model and data files are in [here](https://huggingface.co/soonyau/visconet/tree/main).

Including eval.zip containing all images used in human evaluation.

### Gradio App

[![App](./assets/app.png)](https://youtu.be/3_6Zq3hk86Q)

run ```python gradio_visconet.py```

### Citation

```

@inproceedings{visconet,

        author    = {Cheong, Soon Yau and Mustafa, Armin and Gilbert, Andrew},

        title     = {ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet},

        booktitle = {ECCV Workshop Proceedings}

        month     = {September},

        year      = {2024}}

```