https://github.com/brunoocs/text-to-image-diffusion-model
Text to image diffusion model from scratch with Tiny ImageNet dataset, Pytorch and Web interface for image generation.
https://github.com/brunoocs/text-to-image-diffusion-model
Last synced: about 1 year ago
JSON representation
Text to image diffusion model from scratch with Tiny ImageNet dataset, Pytorch and Web interface for image generation.
- Host: GitHub
- URL: https://github.com/brunoocs/text-to-image-diffusion-model
- Owner: BrunooCS
- Created: 2025-02-26T11:24:02.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-07T11:09:39.000Z (over 1 year ago)
- Last Synced: 2025-03-07T12:22:57.505Z (over 1 year ago)
- Language: Python
- Homepage:
- Size: 132 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Diffusion Model
A powerful conditional diffusion model for image generation from text descriptions.
---
## Overview
This project implements a conditional diffusion model that generates images from text descriptions. The model learns to gradually denoise random Gaussian noise into coherent images, guided by text prompts.
## Key Features
- **Text-to-Image Generation**: Convert textual descriptions into high-quality images
- **Web Interface**: User-friendly interface for image generation
- **Conditional Generation**: Fine-grained control over generated content
- **Interactive Process Visualization**: Watch the denoising process in real-time
- **Multi-Class Support**: Generate images across various categories
## Architecture
The model consists of several key components:
```
├── models/
│ ├── modules.py # Neural network building blocks
│ ├── text_encoder.py # Text embedding module
│ ├── time_encoder.py # Timestep encoding module
│ └── unet.py # Conditional UNet architecture
│
├── output_ImageNet/ # Generated outputs and visualizations
│
├── diffusion.py # Core diffusion model implementation
├── plot_func.py # Visualization utilities
├── preprocess.py # Data loading and preprocessing
└── web.py # Web interface for image generation
```
## Web Interface
The project includes a sleek web interface for easy interaction with the model:
Features:
- Text prompt input
- Multiple image generation
- Process visualization
- Real-time generation progress
## Model Components
### 1. Diffusion Process
- Forward diffusion adds noise gradually
- Reverse diffusion learns to remove noise
- Conditional generation guided by text embeddings
### 2. Architecture Details
- UNet backbone with skip connections
- Text conditioning through cross-attention
- Time embedding using sinusoidal positions
- Batch normalization for stable training
### 3. Dataset
- [Tiny ImageNet](https://paperswithcode.com/dataset/tiny-imagenet), over 1Million images and 200 classes
### 3. Training Process
- Dataset: ImageNet subset
- Text embeddings: SentenceTransformer
- Loss: MSE between predicted and actual noise
- Optimizer: AdamW with gradient scaling
## Sample Generations
Generate images from text descriptions like:
- "red apple"
- "golden retriever"
- "sunset over mountains"
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
---