{"id":23233445,"url":"https://github.com/pamparamm/sd-perturbed-attention","last_synced_at":"2025-08-19T18:32:38.294Z","repository":{"id":232909280,"uuid":"785436341","full_name":"pamparamm/sd-perturbed-attention","owner":"pamparamm","description":"Perturbed-Attention Guidance for ComfyUI and SD Forge","archived":false,"fork":false,"pushed_at":"2024-04-13T01:02:27.000Z","size":98,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-13T21:44:18.820Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pamparamm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-04-11T21:58:24.000Z","updated_at":"2024-04-15T15:55:44.175Z","dependencies_parsed_at":"2024-04-15T15:55:43.325Z","dependency_job_id":"bca99459-ba60-407d-9d71-751d76a7d881","html_url":"https://github.com/pamparamm/sd-perturbed-attention","commit_stats":null,"previous_names":["pamparamm/sd-perturbed-attention"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pamparamm%2Fsd-perturbed-attention","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pamparamm%2Fsd-perturbed-attention/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pamparamm%2Fsd-perturbed-attention/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pamparamm%2Fsd-perturbed-attention/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pamparamm","download_url":"https://codeload.github.com/pamparamm/sd-perturbed-attention/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230367780,"owners_count":18215325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-19T03:01:42.433Z","updated_at":"2025-08-19T18:32:38.252Z","avatar_url":"https://github.com/pamparamm.png","language":"Python","funding_links":[],"categories":["Workflows (3395) sorted by GitHub Stars","All Workflows Sorted by GitHub Stars"],"sub_categories":[],"readme":"# Various Guidance implementations for ComfyUI / SD WebUI (reForge)\r\n\r\nImplementation of\r\n\r\n- Perturbed-Attention Guidance from [Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance (D. Ahn et al.)](https://ku-cvlab.github.io/Perturbed-Attention-Guidance/)\r\n- [Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention (Susung Hong)](https://arxiv.org/abs/2408.00760)\r\n- Sliding Window Guidance from [The Unreasonable Effectiveness of Guidance for Diffusion Models (Kaiser et al.)](https://arxiv.org/abs/2411.10257)\r\n- [PLADIS: Pushing the Limits of Attention in Diffusion Models at Inference Time by Leveraging Sparsity](https://cubeyoung.github.io/pladis-proejct/) (ComfyUI-only)\r\n- [Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models](https://arxiv.org/abs/2505.21179) (ComfyUI-only)\r\n- [Token Perturbation Guidance for Diffusion Models](https://arxiv.org/abs/2506.10036) (ComfyUI-only)\r\n\r\nas an extension for [ComfyUI](https://github.com/comfyanonymous/ComfyUI) and [SD WebUI (reForge)](https://github.com/Panchovix/stable-diffusion-webui-reForge).\r\n\r\nWorks with SD1.5 and SDXL.\r\n\r\n\u003e [!NOTE]\r\n\u003e `Normalized Attention Guidance` node has its own description inside ComfyUI - you can click on the node and press the button with an encircled question mark.\r\n\r\n\u003e [!NOTE]\r\n\u003e PLADIS isn't working properly right now - for some reason it produces messy images when applied to all cross-attention layers. I'll try to investigate the problem, but it wouldn't be easy considering the lack of official implementation + various typos in the original paper. For debug purposes, I might implement PLADIS as a diffusers pipeline to see if the problem still occurs.\r\n\u003e\r\n\u003e Any help is welcomed!\r\n\r\n## Installation\r\n\r\n### ComfyUI\r\n\r\nYou can either:\r\n\r\n- `git clone https://github.com/pamparamm/sd-perturbed-attention.git` into `ComfyUI/custom-nodes/` folder.\r\n\r\n- Install it via [ComfyUI Manager](https://github.com/ltdrdata/ComfyUI-Manager) (search for custom node named \"Perturbed-Attention Guidance\").\r\n\r\n- Install it via [comfy-cli](https://comfydocs.org/comfy-cli/getting-started) with `comfy node registry-install sd-perturbed-attention`\r\n\r\n### SD WebUI (reForge)\r\n\r\n`git clone https://github.com/pamparamm/sd-perturbed-attention.git` into `stable-diffusion-webui-forge/extensions/` folder.\r\n\r\n### SD WebUI (Auto1111)\r\n\r\nAs an alternative for A1111 WebUI you can use PAG implementation from [sd-webui-incantations](https://github.com/v0xie/sd-webui-incantations) extension.\r\n\r\n## Guidance Nodes/Scripts\r\n\r\n### ComfyUI\r\n\r\n![comfyui-node-pag-basic](res/comfyui-node-pag-basic.png)\r\n\r\n![comfyui-node-pag-advanced](res/comfyui-node-pag-advanced.png)\r\n\r\n![comfyui-node-seg](res/comfyui-node-seg.png)\r\n\r\n### SD WebUI (reForge)\r\n\r\n![forge-pag](res/forge-pag.png)\r\n\r\n![forge-seg](res/forge-seg.png)\r\n\r\n\u003e [!NOTE]\r\n\u003e You can override `CFG Scale` and `PAG Scale`/`SEG Scale` for Hires. fix by opening/enabling `Override for Hires. fix` tab.\r\n\u003e To disable PAG during Hires. fix, you can set `PAG Scale` under Override to 0.\r\n\r\n### Inputs\r\n\r\n- `scale`: Guidance scale, higher values can both increase structural coherence of an image and oversaturate/fry it entirely.\r\n- `adaptive_scale` (PAG only): PAG dampening factor, it penalizes PAG during late denoising stages, resulting in overall speedup: 0.0 means no penalty and 1.0 completely removes PAG.\r\n- `blur_sigma` (SEG only): Normal deviation of Gaussian blur, higher values increase \"clarity\" of an image. Negative values set `blur_sigma` to infinity.\r\n- `unet_block`: Part of U-Net to which Guidance is applied, original paper suggests to use `middle`.\r\n- `unet_block_id`: Id of U-Net layer in a selected block to which Guidance is applied. Guidance can be applied only to layers containing Self-attention blocks.\r\n- `sigma_start` / `sigma_end`: Guidance will be active only between `sigma_start` and `sigma_end`. Set both values to negative to disable this feature.\r\n- `rescale`: Acts similar to RescaleCFG node - it prevents over-exposure on high `scale` values. Based on Algorithm 2 from [Common Diffusion Noise Schedules and Sample Steps are Flawed (Lin et al.)](https://arxiv.org/abs/2305.08891). Set to 0 to disable this feature.\r\n- `rescale_mode`:\r\n  - `full` - takes into account both CFG and Guidance.\r\n  - `partial` - depends only on Guidance.\r\n  - `snf` - Saliency-adaptive Noise Fusion from [High-fidelity Person-centric Subject-to-Image Synthesis (Wang et al.)](https://arxiv.org/abs/2311.10329). Should increase image quality on high guidance scales. Ignores `rescale` value.\r\n- `unet_block_list`: Optional input, replaces both `unet_block` and `unet_block_id` and allows you to select multiple U-Net layers separated with commas. SDXL U-Net has multiple indices for layers, you can specify them by using dot symbol (if not specified, Guidance will be applied to the whole layer). Example value: `m0,u0.4` (it applies Guidance to middle block 0 and to output block 0 with index 4)\r\n  - In terms of U-Net `d` means `input`, `m` means `middle` and `u` means `output`.\r\n  - SD1.5 U-Net has layers `d0`-`d5`, `m0`, `u0`-`u8`.\r\n  - SDXL U-Net has layers `d0`-`d3`, `m0`, `u0`-`u5`. In addition, each block except `d0` and `d1` has `0-9` index values (like `m0.7` or `u0.4`). `d0` and `d1` have `0-1` index values.\r\n  - Supports block ranges (`d0-d3` corresponds to `d0,d1,d2,d3`) and index value ranges (`d2.2-9` corresponds to all index values of `d2` with the exclusion of `d2.0` and `d2.1`).\r\n\r\n## ComfyUI TensorRT PAG (Experimental)\r\n\r\nTo use PAG together with [ComfyUI_TensorRT](https://github.com/comfyanonymous/ComfyUI_TensorRT), you'll need to:\r\n\r\n0. Have 24GB of VRAM.\r\n1. Build static/dynamic TRT engine of a desired model.\r\n2. Build static/dynamic TRT engine of the same model with the same TRT parameters, but with fixed PAG injection in selected UNET blocks (`TensorRT Attach PAG` node).\r\n3. Use `TensorRT Perturbed-Attention Guidance` node with two model inputs: one for base engine and one for PAG engine.\r\n\r\n![trt-engines](res/trt-engines.png)\r\n\r\n![trt-inference](res/trt-inference.png)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpamparamm%2Fsd-perturbed-attention","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpamparamm%2Fsd-perturbed-attention","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpamparamm%2Fsd-perturbed-attention/lists"}