https://github.com/garystafford/sdxl_dreambooth_autotrain_demo
Fine-tuning Stable Diffusion XL on AWS for Generative AI-powered Product Concept Design
https://github.com/garystafford/sdxl_dreambooth_autotrain_demo
amazon-sagemaker aws generative-ai generative-art sdxl stability-ai stable-diffusion
Last synced: 3 months ago
JSON representation
Fine-tuning Stable Diffusion XL on AWS for Generative AI-powered Product Concept Design
- Host: GitHub
- URL: https://github.com/garystafford/sdxl_dreambooth_autotrain_demo
- Owner: garystafford
- License: mit
- Created: 2023-11-25T04:46:20.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-04T00:45:32.000Z (almost 2 years ago)
- Last Synced: 2023-12-04T01:27:29.407Z (almost 2 years ago)
- Topics: amazon-sagemaker, aws, generative-ai, generative-art, sdxl, stability-ai, stable-diffusion
- Language: Jupyter Notebook
- Homepage: https://garystafford.medium.com/fine-tuning-stable-diffusion-xl-on-aws-for-generative-ai-powered-product-concept-design-dae6f4c8c8fa
- Size: 133 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Fine-tuning Stable Diffusion XL on AWS for Generative AI-powered Product Concept Design
Source code files for the blog post: [Fine-tuning Stable Diffusion XL on AWS for Generative AI-powered Product Concept Design](https://garystafford.medium.com/fine-tuning-stable-diffusion-xl-on-aws-for-generative-ai-powered-product-concept-design-dae6f4c8c8fa). For more information, see the blog post. The SageMaker notebook, [SDXL_AutoTrain_DreamBooth.ipynb](SDXL_AutoTrain_DreamBooth.ipynb) contains all the code demonstrated in the post.
## LoRA Weights
To use the PyTorch LoRA (Low-Rank Adaptation of Large Language Models) weights with the SDXL 1.0 model, unzip the `mb_amg_gt_oue_dreambooth.zip` file. The resulting `mb_amg_gt_oue_dreambooth` and inclosed `pytorch_lora_weights.safetensors` file can be used with the SDXL 1.0 base model. The model was fine-tuned with approximately 20 images, each scaled in the longest dimension to 1024 pixels.
## Sample Concept Images
All of the images shown below are unretouched.
### Test of the Fine-tuned Model to Generate `oue car` Images
The photorealistic images below were generated using the following prompts at 768 x 1024 pixels wide:
```python
subject_prompt = subject_prompt = """oue, photo of oue car, sporty, fast, sleek, sexy, aggressive, high performance, daytime, futuristic cityscape"""
subject_negative_prompt = """person, people, human, rider, floating objects, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, monochrome, cropped, cut-off"""
refiner_prompt = """ultra-high-definition, photorealistic, 8k uhd, high-quality, ultra sharp detail"""
refiner_negative_prompt = """low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
### Rough Product Sketches of `oue electric scooter`
The rough product sketches below were generated using the following prompts at 768 x 1024 pixels wide:
```python
subject_prompt = """oue, marker rendering of oue electric scooter, concept art, futuristic cityscape, high contrast, black and white, black marker, marker drawing, sketch, monochromatic illustration, illustrative, graphic, muted, expressive strokes"""
subject_negative_prompt = """person, people, human, rider, floating objects, colors, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, cropped, cut-off, patterned background"""
refiner_prompt = """sharp, crisp, in-focus, uncropped, high-quality"""
refiner_negative_prompt = """photographic, photo, photorealistic, low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
### Color Marker Renderings of `oue electric scooter`
The color marker renderings below were generated using the following prompts:
```python
subject_prompt = """oue, color marker rendering of oue electric scooter, concept art, sporty, fast, sleek, sexy, aggressive, high performance, colors, urban, futuristic cityscape, marker, sketch, black and white lines, illustration, illustrative, marker drawing, expressive strokes, graphic"""
subject_negative_prompt = """person, people, human, rider, floating objects, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, monochrome, cropped, cut-off, patterned background, doubles, repeat elements"""
refiner_prompt = """sharp, crisp, in-focus, uncropped, high-quality"""
refiner_negative_prompt = """photographic, photo, photorealistic, low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
Additional variations of color marker renderings generated with the latest LoRA weights and the following prompts at 768 x 1024 pixels wide:
```python
subject_prompt = """oue, marker rendering of oue electric scooter, concept art, futuristic cityscape, solid color background, bright vibrant colors, marker, sketch, illustration, illustrative, marker drawing, expressive strokes, graphic"""
subject_negative_prompt = """person, people, human, rider, floating objects, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, monochrome, cropped, cut-off, patterned background"""
refiner_prompt = """sharp, crisp, in-focus, uncropped, high-quality"""
refiner_negative_prompt = """photographic, photo, photorealistic, low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
### Photorealistic Images of `oue electric scooter`
The photorealistic images below were generated using the following prompts at 864 x 1536 pixels wide (LinkedIn size):
```python
subject_prompt = """oue, photo of a oue electric scooter, sleek, smooth curves, colorful, daytime, urban, futuristic cityscape"""
subject_negative_prompt = """person, people, human, rider, floating objects, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, monochrome, cropped, cut-off, patterned background"""
refiner_prompt = """ultra-high-definition, photorealistic, 8k uhd, high-quality, ultra sharp detail"""
refiner_negative_prompt = """low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
Additional photorealistic images generated with the latest LoRA weights and the following prompts at 768 x 1024 pixels wide:
```python
subject_prompt = """oue, photo of a oue electric scooter, sleek, smooth curves, colorful, daytime, urban, futuristic cityscape"""
subject_negative_prompt = """person, people, human, rider, floating objects, text, words, writing, letters, phrases, trademark, watermark, icon, logo, banner, signature, username, monochrome, cropped, cut-off, patterned background"""
refiner_prompt = """ultra-high-definition, photorealistic, 8k uhd, high-quality, ultra sharp detail"""
refiner_negative_prompt = """low quality, low-resolution, out of focus, blurry, grainy, artifacts, defects, jpeg artifacts, noise"""
```
