Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/IrisRainbowNeko/DreamArtist-sd-webui-extension
DreamArtist for Stable-Diffusion-webui extension
https://github.com/IrisRainbowNeko/DreamArtist-sd-webui-extension
Last synced: 2 months ago
JSON representation
DreamArtist for Stable-Diffusion-webui extension
- Host: GitHub
- URL: https://github.com/IrisRainbowNeko/DreamArtist-sd-webui-extension
- Owner: IrisRainbowNeko
- License: apache-2.0
- Created: 2022-11-12T12:17:15.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2023-11-08T15:39:19.000Z (about 1 year ago)
- Last Synced: 2024-05-19T16:35:21.871Z (8 months ago)
- Language: Python
- Size: 58.2 MB
- Stars: 687
- Watchers: 14
- Forks: 51
- Open Issues: 30
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-stable-diffusion-webui - DreamArtist-sd-webui-extension - Diffusion-webui extension that introduces new artistic tools and functions into the existing WebUI, enhancing its capabilities and providing a more immersive experience. (GitHub projects)
README
# DreamArtist++
***DreamArtist++ for training lora with just one image has been released, try it now:***
[HCP-Diffusion](https://github.com/7eu7d7/HCP-Diffusion)
All future updates of the DreamArtist series will be released in this new framework.
# DreamArtist (webui Eextension)
Paper: [](https://arxiv.org/abs/2211.11337)
This repo is the official ***Stable-Diffusion-webui extension version** implementation of ***"DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Contrastive Prompt-Tuning"***
with [Stable-Diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui).
Standalone version: [DreamArtist](https://github.com/7eu7d7/DreamArtist-stable-diffusion)
Everyone is an artist. Rome wasn't built in a day, but your artist dreams can be!
With just ***one*** training image DreamArtist learns the content and style in it, generating diverse high-quality images with high controllability.
Embeddings of DreamArtist can be easily combined with additional descriptions, as well as two learned embeddings.
# Setup and Running
Clone this repo to extension folder.
```bash
git clone https://github.com/7eu7d7/DreamArtist-sd-webui-extension.git extensions/DreamArtist
```
## Training and Usage
First create the positive and negative embeddings in ```DreamArtist Create Embedding``` Tab.
### Preview Setting
After that, the ```names``` of the positive and negative embedding (```{name}``` and ```{name}-neg```) should be filled into the
```txt2img Tab``` with some common descriptions. This will ensure a correct preview image.
### Train
Then, select positive embedding and set the parameters and image folder path in the ```DreamArtist Train``` Tab to start training.
The corresponding negative embedding is loaded automatically.
If your VRAM is low or you want save time, you can uncheck the ```reconstruction```.
[Recommended parameters](https://github.com/7eu7d7/DreamArtist-sd-webui-extension#pre-trained-embeddings)
***better to train without filewords***
Remember to check the option below, otherwise the preview is wrong.
### Inference
Fill the trained positive and negative embedding into txt2img to generate with DreamArtist prompt.
### Attention Mask
Attention Mask can strengthen or weaken the learning intensity of some local areas.
Attention Mask is a grayscale image whose grayscale values are related to the learning intensity show in the following table.
| grayscale | 0% | 25% | 50% | 75% | 100% |
|-----------|----|-----|------|------|------|
| intensity | 0% | 50% | 100% | 300% | 500% |
The Attention Mask is in the same folder as the training image and its name is the name of the training image + "_att".
You can choose whether to enable Attention Mask for training.
Since there is a self-attention operation in VAE, it may change the distribution of features.
In the ***Process Att-Map*** tab, it can superimpose the attention map of self-attention on the original Att-Map.
### Dynamic CFG
Dynamic CFG can improve the performance, especially when the data set is large (>20).
For example, linearly from 1.5 to 3.0 (1.5-3.0), or with a 0-π/2 cycle of cosine (1.5-3.0:cos), or with a -π/2-0 cycle of cosine (1.5-3.0:cos2).
Or you can also customize non-linear functions, such as 2.5-3.5:torch.sqrt(rate), where rate is a variable from 0-1.
## Tested models (need ema version):
+ Stable Diffusion v1.4
+ Stable Diffusion v1.5
+ animefull-latest
+ Anything v3.0
+ momoko-e
Embeddings can be transferred between different models of the same dataset.
## Pre-trained embeddings:
[Download](https://github.com/7eu7d7/DreamArtist-stable-diffusion/releases/tag/embeddings_v2)
| Name | Model | Image | embedding length
(Positive, Negative) | iter | lr | cfg scale |
|------------|------------------|--------------------------------------------------------------------|--------------------------------------------|-------|--------|-----------|
| ani-nahida | animefull-latest | 
| 3, 6 | 8000 | 0.0025 | 3 |
| ani-cocomi | animefull-latest | 
| 3, 6 | 8000 | 0.0025 | 3 |
| ani-gura | animefull-latest | 
| 3, 6 | 12000 | 0.0025 | 3 |
| ani-g | animefull-latest | 
| 3, 10 | 1500 | 0.003 | 5 |
| asty-bk | animefull-latest | 
| 3, 6 | 5000 | 0.003 | 3 |
| asty-gc | animefull-latest | 
| 3, 10 | 1000 | 0.005 | 5 |
| real-dog | sd v1.4 | 
| 3, 3 | 1000 | 0.005 | 5 |
| real-sship | sd v1.4 | 
| 3, 3 | 3000 | 0.003 | 5 |
| sty-cyber | sd v1.4 | 
| 3, 5 | 15000 | 0.0025 | 5 |
| sty-shuimo | sd v1.4 | 
| 3, 5 | 15000 | 0.0025 | 5 |
# Style Clone
# Prompt Compositions
# Comparison on One-Shot Learning
# Other Results
