{"id":29698767,"url":"https://github.com/aatricks/lightdiffusion-next","last_synced_at":"2026-05-09T17:33:13.332Z","repository":{"id":267924353,"uuid":"902761502","full_name":"Aatricks/LightDiffusion-Next","owner":"Aatricks","description":"LightDiffusion SD and Flux refactored. Reassembled to support and be the fastest SD and Flux natively 🚀","archived":false,"fork":false,"pushed_at":"2025-07-21T16:51:58.000Z","size":42555,"stargazers_count":43,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-21T18:52:50.832Z","etag":null,"topics":["ai","artificial-intelligence","deep-learning","diffusion","generative-ai","gui","image-generation","img2img","inpainting","pytorch","stable-diffusion","text2image","torch","txt2img"],"latest_commit_sha":null,"homepage":"https://lightdiffusion.github.io/LightDiffusion-Next/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aatricks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-13T08:06:38.000Z","updated_at":"2025-07-21T16:52:02.000Z","dependencies_parsed_at":"2024-12-13T09:31:04.470Z","dependency_job_id":"b20f2acb-52ba-4d7c-9110-f20d8ab656e4","html_url":"https://github.com/Aatricks/LightDiffusion-Next","commit_stats":null,"previous_names":["lightdiffusion/lightdiffusion-next","aatrick/lightdiffusion-next","aatricks/lightdiffusion-next"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/Aatricks/LightDiffusion-Next","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aatricks%2FLightDiffusion-Next","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aatricks%2FLightDiffusion-Next/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aatricks%2FLightDiffusion-Next/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aatricks%2FLightDiffusion-Next/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aatricks","download_url":"https://codeload.github.com/Aatricks/LightDiffusion-Next/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aatricks%2FLightDiffusion-Next/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266664295,"owners_count":23964933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","deep-learning","diffusion","generative-ai","gui","image-generation","img2img","inpainting","pytorch","stable-diffusion","text2image","torch","txt2img"],"created_at":"2025-07-23T10:38:55.645Z","updated_at":"2026-02-12T18:04:51.545Z","avatar_url":"https://github.com/Aatricks.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Say hi to LightDiffusion-Next 👋\n\n[![demo platform](https://img.shields.io/badge/Play%20with%20LightDiffusion%21-LightDiffusion%20demo%20platform-lightblue)](https://huggingface.co/spaces/Aatricks/LightDiffusion-Next)\u0026nbsp;\n\n**LightDiffusion-Next**  is the fastest AI-powered image generation WebUI, combining speed, precision, and flexibility in one cohesive tool.\n\u003c/br\u003e\n\u003c/br\u003e\n  \u003ca href=\"https://github.com/LightDiffusion/LightDiffusion-Next\"\u003e\n    \u003cimg src=\"./HomeImage.png\" alt=\"Logo\"\u003e\n\n  \u003c/a\u003e\n\u003c/br\u003e\n\u003c/div\u003e\n\n---\n\nAs a refactored and improved version of the original [LightDiffusion repository](https://github.com/Aatrick/LightDiffusion), this project enhances usability, maintainability, and functionality while introducing a host of new features to streamline your creative workflows.\n\n\n## Motivation:\n\n**LightDiffusion** was originally meant to be made in Rust, but due to the lack of support for the Rust language in the AI community, it was made in Python with the goal of being the simplest and fastest AI image generation tool.\n\nThat's when the first version of LightDiffusion was born which only counted [3000 lines of code](https://github.com/LightDiffusion/LightDiffusion-original), only using Pytorch. With time, the [project](https://github.com/Aatrick/LightDiffusion) grew and became more complex, and the need for a refactor was evident. This is where **LightDiffusion-Next** comes in, with a more modular and maintainable codebase, and a plethora of new features and optimizations.\n\n📚 Learn more in the [official documentation](https://aatricks.github.io/LightDiffusion-Next/)\n\n---\n\n## 🌟 Highlights\n\n![image](https://github.com/user-attachments/assets/b994fe0d-3a2e-44ff-93a4-46919cf865e3)\n\n**LightDiffusion-Next** offers a powerful suite of tools to cater to creators at every level. At its core, it supports **Text-to-Image** (Txt2Img) and **Image-to-Image** (Img2Img) generation, offering a variety of upscale methods and samplers, to make it easier to create stunning images with minimal effort.\n\nAdvanced users can take advantage of features like **attention syntax**, **Hires-Fix** or **ADetailer**. These tools provide better quality and flexibility for generating complex and high-resolution outputs.\n\n**LightDiffusion-Next** is fine-tuned for **performance**. Features such as **Xformers** acceleration, **BFloat16** precision support, **WaveSpeed** dynamic caching, **Multi-scale diffusion**, and **Stable-Fast** model compilation (which offers up to a 70% speed boost) ensure smooth and efficient operation, even on demanding workloads.\n\n---\n\n## ✨ Feature Showcase\n\nHere’s what makes LightDiffusion-Next stand out:\n\n- **Speed and Efficiency**:\n  Enjoy industry-leading performance with built-in Xformers, Pytorch, Wavespeed and Stable-Fast optimizations, Multi-scale diffusion, deepcache, AYS (Align Your Steps) scheduler, and automatic prompt caching achieving 30% up to 200% faster speeds compared to the rest of the AI image generation backends in SD1.5 and Flux.\n\n- **Automatic Detailing**:\n  Effortlessly enhance faces and body details with AI-driven tools based on the [Impact Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack).\n\n- **State Preservation**:\n  Save and resume your progress with saved states, ensuring seamless transitions between sessions.\n\n- **Integration-Ready**:\n  Collaborate and create directly in Discord with [Boubou](https://github.com/Aatrick/Boubou), or preview images dynamically with the optional **TAESD preview mode**.\n\n- **Image Previewing**:\n  Get a real-time preview of your generated images with TAESD, allowing for user-friendly and interactive workflows.\n\n- **Image Upscaling**:\n  Enhance your images with advanced upscaling options like UltimateSDUpscaling, ensuring high-quality results every time.\n\n- **Prompt Refinement**:\n  Use the optional Ollama-powered prompt enhancer (defaults to `qwen3:0.6b`) to refine your prompts and generate more accurate and detailed outputs.\n\n- **LoRa and Textual Inversion Embeddings**:\n    Leverage LoRa and textual inversion embeddings for highly customized and nuanced results, adding a new dimension to your creative process.\n\n- **Low-End Device Support**:\n    Run LightDiffusion-Next on low-end devices with as little as 2GB of VRAM or even no GPU, ensuring accessibility for all users.\n\n- **CFG++**:\n    Uses samplers modified to use CFG++ for better quality results compared to traditional methods.\n\n- **Newelle Extension**:\n    LightDiffusion-Next is also available as a backend to the [Newelle LightDiffusion extension](https://github.com/Aatricks/Newelle-Light-Diffusion) permitting to generate images inline during conversations with llms.\n\n---\n\n## ⚡ Performance Benchmarks\n\n**LightDiffusion-Next** dominates in performance:\n\n| **Tool**                           | **Speed (it/s)** |\n|------------------------------------|------------------|\n| **LightDiffusion with Stable-Fast** | 2.8              |\n| **LightDiffusion**                 | 1.9              |\n| **ComfyUI**                        | 1.4              |\n| **SDForge**                        | 1.3              |\n| **SDWebUI**                        | 0.9              |\n\n(All benchmarks are based on a 1024x1024 resolution with a batch size of 1 using BFloat16 precision without tweaking installations. Made with a 3060 mobile GPU using SD1.5.)\n\nWith its unmatched speed and efficiency, LightDiffusion-Next sets the benchmark for AI image generation tools.\n\n---\n\n## 🛠 Installation\n\n\u003e [!NOTE]\n\u003e **Platform Support:** LightDiffusion-Next supports NVIDIA GPUs (CUDA), AMD GPUs (ROCm), and Apple Silicon (Metal/MPS). For AMD and Apple Silicon setup instructions, see the [ROCm and Metal/MPS Support Guide](https://aatrick.github.io/LightDiffusion/rocm-metal-support/).\n\n\u003e [!WARNING]\n\u003e **Disclaimer:** On Linux, the fastest way to get started is with the Docker setup below. Windows users often encounter an `EOF` build error when using Docker; if that happens, set up a local virtual environment instead and install SageAttention inside it.\n\n\u003e [!NOTE]\n\u003e You will need to download the [flux vae](https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors) separately given its gated repo on Huggingface. Drop it in the `/include/vae` folder.\n\n### Quick Start\n\n1. Download a release or clone this repository.\n2. Run `run.bat` in a terminal.\n3. The modern React frontend will launch automatically at `http://localhost:5173` (proxied to the FastAPI backend at `http://localhost:7861`).\n\n**Recommended Launch Command:**\n```bash\n# Start both backend and frontend development server\npython server.py --frontend\n```\n\n**Alternative UIs:**\n- **Streamlit UI**: Run `streamlit run streamlit_app.py` for the legacy clean interface.\n- **Gradio UI**: Run `python app.py` to use the original Gradio interface (mainly for HuggingFace Spaces).\n\n### 🌌 Flux Support\n\nLightDiffusion-Next now features first-class support for **Flux2 Klein**. To get started, you need to download the required model components (Diffusion Model, Text Encoder, and VAE).\n\nWe provide a convenient script to handle this automatically:\n```bash\npython download_flux.py\n```\nThis will download approximately 16GB of weights into the `include/` directory.\n\n### 🐳 Docker Setup\n\nRun LightDiffusion-Next in a containerized environment with GPU acceleration:\n\n\u003e [!IMPORTANT]\n\u003e Confirm you have Docker Desktop configured with the NVIDIA Container Toolkit and at least 12-16GB of memory. Builds expect an NVIDIA GPU with compute capability 8.0 or higher and CUDA 12.0+ support for SageAttention/SpargeAttn.\n\n**Quick Start with Docker:**\n```bash\n# Build and run with docker-compose (recommended - uses Streamlit by default)\ndocker-compose up --build\n\n# Or build and run manually with Streamlit\ndocker build -t lightdiffusion-next .\ndocker run --gpus all -p 8501:8501 -e UI_FRAMEWORK=streamlit -v ./output:/app/output lightdiffusion-next\n\n# To use Gradio instead:\ndocker run --gpus all -p 7860:7860 -e UI_FRAMEWORK=gradio -v ./output:/app/output lightdiffusion-next\n```\n\n**Custom GPU Architecture (Optional):**\n```bash\n# For faster builds, specify your GPU architecture (e.g., RTX 5060 = 12.0)\ndocker-compose build --build-arg TORCH_CUDA_ARCH_LIST=\"12.0\"\n\n# Default builds for: 8.0 (A100), 8.6 (RTX 30xx), 8.9 (RTX 40xx), 9.0 (H100), 12.0 (RTX 50xx)\n```\n\n**Built-in Optimizations:**\nThe Docker image can build the following acceleration paths:\n- ✨ **SageAttention** - 15% speedup with INT8 quantization (all supported GPUs)\n- 🚀 **SpargeAttn** - 40-60% speedup with sparse attention (compute 8.0-9.0 only)\n- ⚡ **Stable-Fast** - Optional UNet compilation for up to 70% faster SD1.5 inference\n\nControl them through build arguments (defaults shown below):\n\n```bash\ndocker-compose build \\\n  --build-arg TORCH_CUDA_ARCH_LIST=\"8.0;8.6;8.9;9.0;12.0\" \\\n  --build-arg INSTALL_STABLE_FAST=1 \\\n  --build-arg INSTALL_OLLAMA=0\n```\n\nSet `INSTALL_STABLE_FAST=1` to enable the compilation step for stable-fast, or `INSTALL_OLLAMA=1` to bake in the prompt enhancer runtime.\n\n\u003e [!NOTE]\n\u003e RTX 50 series (compute 12.0) GPUs currently only support SageAttention.\n\n**Access the Web Interface:**\n- **Streamlit UI** (default): `http://localhost:8501`\n- **Gradio UI**: `http://localhost:7860` (set `UI_FRAMEWORK=gradio` in docker-compose.yml)\n\n**Volume Mounts:**\n- `./output:/app/output` - Persist generated images\n- `./checkpoints:/app/include/checkpoints` - Store model files\n- `./loras:/app/include/loras` - Store LoRA files\n- `./embeddings:/app/include/embeddings` - Store embeddings\n\n\n### Advanced Setup\n\n- **Install from Source**:\n  Install dependencies via:\n  ```bash\n  pip install -r requirements.txt\n  ```\n  Add your SD1/1.5 safetensors model to the `checkpoints` directory, then launch the application.\n\n- **⚡Stable-Fast Optimization**:\n  Follow [this guide](https://github.com/chengzeyi/stable-fast?tab=readme-ov-file#installation) to enable Stable-Fast mode for optimal performance.\n  In Docker environments, set `INSTALL_STABLE_FAST=1` to compile it during the image build or `INSTALL_STABLE_FAST=0` (default) to skip.\n\n- **🚀 SageAttention \u0026 SpargeAttn Acceleration**:\n  Boost inference speed by up to 60% with advanced attention backends:\n\n  **Prerequisites:**\n  - [CUDA toolkit](https://developer.nvidia.com/cuda-toolkit-archive) installed with version compatible with your PyTorch installation\n  \n  **SageAttention (15% speedup, Windows compatible):**\n  ```bash\n  cd SageAttention\n  pip install -e . --no-build-isolation\n  ```\n  \n  **SpargeAttn (40-60% total speedup, requires WSL2/Linux):**\n\u003e [!CAUTION]\n\u003e SpargeAttn cannot be built with the default Windows linker. Use WSL2 or a native Linux environment and set the correct `TORCH_CUDA_ARCH_LIST` before installation.\n  ```bash\n  # On WSL2 or Linux only (Windows linker has path length limitations)\n  cd SpargeAttn\n  export TORCH_CUDA_ARCH_LIST=\"9.0\"  # Or your GPU architecture (8.0, 8.6, 8.9, 9.0)\n  pip install -e . --no-build-isolation\n  ```\n  \n  **Priority System:** SpargeAttn \u003e SageAttention \u003e PyTorch SDPA\n  - Both are automatically detected and used when available\n  - Graceful fallback for unsupported head dimensions\n\n- **🦙 Prompt Enhancer**:\n  Turn on the Ollama-backed enhancer to automatically restructure prompts. By default the app targets `qwen3:0.6b`:\n  ```bash\n  # Local install\n  pip install ollama\n  curl -fsSL https://ollama.com/install.sh | sh\n\n  # Start the Ollama daemon (keep this terminal open)\n  ollama serve\n\n  # New terminal: pull the default prompt enhancer model\n  ollama pull qwen3:0.6b\n  export PROMPT_ENHANCER_MODEL=qwen3:0.6b\n  ```\n  In Docker builds, set `--build-arg INSTALL_OLLAMA=1` (or update `docker-compose.yml`) to install Ollama and pre-pull the model automatically. You can override the runtime model/prefix with the `PROMPT_ENHANCER_MODEL` and `PROMPT_ENHANCER_PREFIX` environment variables. See the [Ollama guide](https://github.com/ollama/ollama?tab=readme-ov-file) for details.\n\n- **🤖 Discord Integration**:\n  Set up the Discord bot by following the [Boubou installation guide](https://github.com/Aatrick/Boubou).\n\n### Third-Party Licenses\n- This project distributes builds that depend on third-party open source components. For attribution details and the full license text, refer to `THIRD_PARTY_LICENSES.md`.\n\n---\n\n🎨 Enjoy exploring the powerful features of LightDiffusion-Next!\n\n\u003e [!TIP]\n\u003e ⭐ If this project helps you, please give it a star! It helps others discover it too.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faatricks%2Flightdiffusion-next","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faatricks%2Flightdiffusion-next","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faatricks%2Flightdiffusion-next/lists"}