{"id":17908115,"url":"https://github.com/huchenlei/stable-diffusion-ps-pea","last_synced_at":"2025-03-23T20:30:57.976Z","repository":{"id":179939108,"uuid":"662826007","full_name":"huchenlei/stable-diffusion-ps-pea","owner":"huchenlei","description":"Use Stable Diffusion in Photopea!","archived":false,"fork":false,"pushed_at":"2023-11-26T23:27:32.000Z","size":2445,"stargazers_count":86,"open_issues_count":12,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-19T00:25:13.447Z","etag":null,"topics":["generative-ai","photopea","photopea-plugin","stable-diffusion","stable-diffusion-api","stable-diffusion-webui","typescript","vue3"],"latest_commit_sha":null,"homepage":"","language":"Vue","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/huchenlei.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"huchenlei","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2023-07-06T01:18:44.000Z","updated_at":"2025-02-28T05:41:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"a730eb4b-b4bf-4355-8719-d8ba1cc7d8e8","html_url":"https://github.com/huchenlei/stable-diffusion-ps-pea","commit_stats":null,"previous_names":["huchenlei/stable-diffusion-ps-pea"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huchenlei%2Fstable-diffusion-ps-pea","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huchenlei%2Fstable-diffusion-ps-pea/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huchenlei%2Fstable-diffusion-ps-pea/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huchenlei%2Fstable-diffusion-ps-pea/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/huchenlei","download_url":"https://codeload.github.com/huchenlei/stable-diffusion-ps-pea/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245164777,"owners_count":20571180,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-ai","photopea","photopea-plugin","stable-diffusion","stable-diffusion-api","stable-diffusion-webui","typescript","vue3"],"created_at":"2024-10-28T19:14:33.162Z","updated_at":"2025-03-23T20:30:57.305Z","avatar_url":"https://github.com/huchenlei.png","language":"Vue","funding_links":["https://ko-fi.com/huchenlei"],"categories":[],"sub_categories":[],"readme":"\u003ca name=\"readme-top\"\u003e\u003c/a\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n\u003cimg width=\"160\" src=\"https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/b015ba09-3fee-42e6-b907-b957ec1a0e60\"\u003e\n\n\u003ch1 align=\"center\"\u003eStable Diffusion Photopea\u003c/h1\u003e\n\nStable Diffusion plugin for Photopea based on A1111 API.\n\n[Changelog](./CHANGELOG.md) · [Report Bug](issues-url) · [Request Feature](issues-url)\n\n\u003c!-- SHIELD GROUP --\u003e\n[![Discord][discord-shield]][discord-url]\n\n\u003c/div\u003e\n\n\n## Installation\n**Step1: Setup backend service**\nSet following command line arguments in `webui-user.bat`:\n```bat\nset COMMANDLINE_ARGS=--api --cors-allow-origins https://huchenlei.github.io [Rest of ARGS...]\n```\n\nFor SDNext(V1111) users, set following arguments:\n```bat\nset COMMANDLINE_ARGS=--cors-origins https://huchenlei.github.io [Rest of ARGS...]\n```\n\n**Step2: Click Window\u003ePlugin**\n![Step2](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/a91df408-76c6-4300-8987-41f7971379a5)\n**Step3: Search for stable-diffusion-ps-pea**\n![Install](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/35c2b802-4f31-45c2-8a24-e55f621adfae)\n## Features\n\n##### :fire:[New Feature][2023-11-26] Realtime rendering powered by [LCM](https://github.com/luosiallen/latent-consistency-model)\nRecent advancement in LCM(Latent Consistency Model) has significantly increased the speed of\ninference of stable diffusion. The inference time now can be so fast that we can do real-time\nrendering of the canvas.\n\nSome preparations before you start exploring the real-time rendering tab:\n- Make sure to download the latest version of [`config_sharing/huchenlei_configs.json5`](https://github.com/huchenlei/stable-diffusion-ps-pea/blob/main/public/config/huchenlei_configs.json5) and upload it in the config tab. The new config file provides `lcm_base`, `lcm_lora_sd15`, `lcm_sd15_scribble` configs that are necessary.\n- Make sure you have LCM LoRA named `lcm_lora_sd15.safetensor` in A1111. Or you can change the name of LoRA in config `lcm_lora_sd15`. You can download LCM LoRAs [here](https://huggingface.co/collections/latent-consistency/latent-consistency-models-loras-654cdd24e111e16f0865fba6).\n\nAfter these preparations, you can now navigate to the real-time render tab (📹).\n- Select `lcm_base`, `lcm_lora_sd15` in RealtimeConfig.\n- Start drawing on canvas and enjoy!\n\nOther features:\n- If you have any selections on canvas, LCM will only render the selected area.\n- You can add `lcm_sd15_scribble` to RealtimeConfig, which will invoke ControlNet scribble model on canvas content. Make sure you have solid black brush color active when scribbling.\n- You can click `Send to canvas` to send the rendered view to canvas.\n\n![Screen Capture 034 - Photopea - Online Photo Editor - www photopea com](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/0d53c264-6f74-42e2-9581-ba98a6b021ba)\n![scribble](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/fd0a0139-c61f-449d-8e1a-e00575acdda9)\n![apples](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/6a2fed43-1630-4b11-b3f2-548216571b73)\n\n...More documentation work in progress...\n\n**Reference range selection**\nIn A1111 img2img inpaint, one painpoint is that the inpaint area selection is either `WholeImage` or `OnlyMasked`. This\nmight not be an issue when the image is within reasonable size (512x512). Once the image becomes big (1024x1024+), the\ntime and resouce required for inpaint area to be `WhileImage` grows exponentially, which makes this option not viable, but\nsometimes we do want to reference a limited range of surroundings. In this situation, one need to crop the image in an\nimage editor, ask A1111 to only process the cropped image, then put the cropeed image back to the original big image.\n\nThis is a tedious process, but now we have this behaviour as default in `stable-diffusion-ps-pea`. Everytime you do an\nimg2img, optionally you can apply a reference range (%/px), or you can just manually specify the range by creating another\nselection on canvas.\n\n![ref_area](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/951c2420-d5cd-4e65-bde2-45a0880ea73c)\n\n**Scale ratio**\nIn whole body generation, some body parts (hand/face) often becomes bad in quality, because there are just not enough\npixels for the diffusion model to add details to. The diffusion model also performs less well on aspect ratios other\nthan the ratios it was trained on (512x512 for SD1.5, 1024x1024 for SDXL), so doing inpaint in a small area only help\na little. The solution is simple here, when inpainting a small area, we let A1111 target a bigger area closer to diffusion\nmodel's trained aspect ratio and resize the output to put the result image back to the original inpaint spot. The very\npopular extension ADetailer is doing this exact process but using image detection models to automatically detect \nface/hand/body to fix.\n\n![scale_ratio](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/41df7f23-f752-4477-8304-9e06f9725eb3)\n\n**ControlNet**\nMajority of ControlNet models can be applied to a specific part of the image (canny, depth, openpose, etc). However,\nin normal A1111 ControlNet UI, you cannot easily visualize the spatial relationship between each ControlNet unit.\n\nOne example is shown in following video. The author uses openpose to control body pose, and softedge to control hand\ndetail. Noting that he is using a image editor to edit the softedge map to keep only the hand part.\n[![Basic Workflow](http://img.youtube.com/vi/UgVOQTjahxc/0.jpg)](https://www.youtube.com/watch?v=UgVOQTjahxc)\n\nThis type of operation now becomes very easy in `stable-diffusion-ps-pea`. The ControlNet maps can easily overlay\non top of each other. Here I am using a openpose unit and a lineart unit.\n\n![elon](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/1834d24f-e994-41e6-ba19-01a1d0cd1655)\n![cnet](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/5dcb6d6f-5c3e-4cf8-abf6-c5223059a8af)\n![5b57323c40b09034008b45e7](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/1938eac3-4a11-4d6a-95c1-eda23be453ea)\n\nInitial proposal to implement layer control in ControlNet's repo: [Issue #1736](https://github.com/Mikubill/sd-webui-controlnet/issues/1736).\n\n**Config System**\nOne pain point about A1111 is that it is hard to define workflow. There are many configuration I wish can be restored\nlater when I was using A1111. So here I designed a configuration system that let users easily define workflow.\n\nThere are 3 types of config in `stable-diffusion-ps-pea`:\n- Base: The config representing the hardcoded default values for each generation parameters.\n- Default: The default config when each time you enter the UI or click the `refresh` button at the bottom right corner.\nClicking the checkmark will activate the current selected config as default.\n![default_config](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/207672ad-4dcb-4309-8459-be16e029905a)\n- Toolbox: The addon configs that only applied temporarily on the generation triggered by clicking the corresponding\ntoolbox button. This is where you can define you customized workflow.\n![toolbox](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/ae67f328-42e9-4425-8b8e-8d75b1574853)\n\nConfigs are defined as the delta to apply on top of the current UI states. Here are some examples I wrote and you can download `config_sharing/huchenlei_configs.json5` and upload it in config panel to get access to them.\n\n**LamaGenFill:** Use ControlNet's `inpaint_only+lama` to achieve similar effect of adobe's generative fill, and magic eraser. We accept JSON5 as config format, so you can actually add comment in config file.\n```json5\n\"LamaGenFill\": [\n        {\n            \"kind\": \"E\",\n            \"path\": [\n                \"img2imgPayload\",\n                \"denoising_strength\"\n            ],\n            \"lhs\": 0.75,\n            \"rhs\": 1\n        },\n        {\n            \"kind\": \"E\",\n            \"path\": [\n                \"img2imgPayload\",\n                \"inpainting_fill\"\n            ],\n            \"lhs\": 1,\n            \"rhs\": 3, // Inpaint fill is latent nothing.\n        },\n        {\n            \"kind\": \"E\",\n            \"path\": [\n                \"img2imgPayload\",\n                \"inpaint_full_res\"\n            ],\n            \"lhs\": 0,\n            \"rhs\": 0, // Make sure inpaint reference range is whole image.\n        },\n        {\n            \"kind\": \"A\",\n            \"path\": [\n                \"controlnetUnits\"\n            ],\n            \"index\": 0,\n            \"item\": {\n                \"kind\": \"N\",\n                \"rhs\": {\n                    \"batch_images\": \"\",\n                    \"control_mode\": 2,\n                    \"enabled\": true,\n                    \"guidance_end\": 1,\n                    \"guidance_start\": 0,\n                    \"input_mode\": 0,\n                    \"low_vram\": false,\n                    \"model\": \"control_v11p_sd15_inpaint [ebff9138]\",\n                    \"module\": \"inpaint_only+lama\",\n                    \"output_dir\": \"\",\n                    \"pixel_perfect\": false,\n                    \"processor_res\": 512,\n                    \"resize_mode\": 1,\n                    \"threshold_a\": 64,\n                    \"threshold_b\": 64,\n                    \"weight\": 1,\n                    \"linkedLayerName\": \"\"\n                }\n            }\n        }\n    ],\n```\nGenerative Fill using `LamaGenFill` workflow:\n![GenFill1](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/4ae79838-ddeb-452a-a052-2b414381e709)\n![GenFill2](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/f8406fcd-c7a6-4e81-9aeb-290e21755123)\nMagic Eraser using `LamaGenFill` workflow:\n![Eraser1](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/da087193-a343-43fb-acc8-366e8d1d4b78)\n![Eraser2](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/cd1e335e-1513-45bf-8cda-7df789e995dc)\n\n**TileUpscale2x**\nAs previously demoed about scale ratio, this workflow is used to fix hand/face, and add details to the selected region.\n```json\n\"TileUpscale2x\": [\n        {\n            \"kind\": \"E\",\n            \"path\": [\"imageScale\"],\n            \"lhs\": 1,\n            \"rhs\": 2,\n        },\n        {\n            \"kind\": \"A\",\n            \"path\": [\n                \"controlnetUnits\"\n            ],\n            \"index\": 0,\n            \"item\": {\n                \"kind\": \"N\",\n                \"rhs\": {\n                    \"batch_images\": \"\",\n                    \"control_mode\": 0,\n                    \"enabled\": true,\n                    \"guidance_end\": 1,\n                    \"guidance_start\": 0,\n                    \"input_mode\": 0,\n                    \"low_vram\": false,\n                    \"model\": \"control_v11f1e_sd15_tile [a371b31b]\",\n                    \"module\": \"tile_resample\",\n                    \"output_dir\": \"\",\n                    \"pixel_perfect\": false,\n                    \"processor_res\": 512,\n                    \"resize_mode\": 1,\n                    \"threshold_a\": 1,\n                    \"threshold_b\": 64,\n                    \"weight\": 1,\n                    \"linkedLayerName\": \"\"\n                }\n            }\n        }\n    ],\n```\nHere is a video demo using it: https://www.loom.com/share/fb11c0206d7045469b82fe9d6342bd15\n\nOverall, the config system gives users full capability on A1111 API. Even the plugin does not build UI support for some\nextensions, users can still invoke the extensions they want by setting entries of `alwayson_scripts`.\n\n**Interfacing with A1111**: Optionally you can use https://github.com/yankooliveira/sd-webui-photopea-embed to send images between photopea and A1111.\n\n# Development\n## Setup HTTPS\nThe dev server needs to run under HTTPS because the plugin runs in an iframe that is embedded in an HTTPS environment. \nUsing HTTP will make the browser complain about mixing HTTP/HTTPS content on a page.\n\nLinux/Mac bash\n`openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -subj \"/CN=localhost\"`\n\nWindows bash\n`openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -subj \"//CN=localhost\"`\n\n## Setup A1111\nAdd `--cors-allow-origins https://localhost:5173` to WebUI command line args for local development.\n\n## Add dev plugin to photopea plugin store\nI do not make my dev plugin setup public as it might confuse user which plugin to install. So if you are planning to do development of\nthis plugin, I kindly ask every developer to add their own dev plugin to photopea plugin store following these steps:\n\n**Step1: Click Window\u003ePlugin**\n![Step1](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/a91df408-76c6-4300-8987-41f7971379a5)\n**Step2: Click Add Plugin**\n![Step2](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/c559bb60-113d-4a9b-a737-a6a14087c3a8)\n**Step3: Click New**\n![Step3](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/0f47a0dc-5916-44bd-823a-2a7d89c5e331)\n**Step4: Fill the Form**  \n![Step4](https://github.com/huchenlei/stable-diffusion-ps-pea/assets/20929282/6aa8dd13-eba7-46df-bef1-4fe822726b23)\n- File: upload `photopea_dev.json` in project root directory\n- Thumbnail: Use any image link with proper size. I use `https://huchenlei.github.io/stable-diffusion-ps-pea/sd.png`\n- Make sure to check `Make Public`.\n\n**Step5: Install the plugin**  \nYou should be able to find the newly added plugin in the plugin store.\n\n**Step6: Make the plugin private**  \nGo back to Step3 panel, and click `Edit` on the plugin you just added. Uncheck `Make Public`.\n\n## Recommended IDE Setup\n\n[VSCode](https://code.visualstudio.com/) + [Volar](https://marketplace.visualstudio.com/items?itemName=Vue.volar) (and disable Vetur) + [TypeScript Vue Plugin (Volar)](https://marketplace.visualstudio.com/items?itemName=Vue.vscode-typescript-vue-plugin).\n\n## Type Support for `.vue` Imports in TS\n\nTypeScript cannot handle type information for `.vue` imports by default, so we replace the `tsc` CLI with `vue-tsc` for type checking. In editors, we need [TypeScript Vue Plugin (Volar)](https://marketplace.visualstudio.com/items?itemName=Vue.vscode-typescript-vue-plugin) to make the TypeScript language service aware of `.vue` types.\n\nIf the standalone TypeScript plugin doesn't feel fast enough to you, Volar has also implemented a [Take Over Mode](https://github.com/johnsoncodehk/volar/discussions/471#discussioncomment-1361669) that is more performant. You can enable it by the following steps:\n\n1. Disable the built-in TypeScript Extension\n    1) Run `Extensions: Show Built-in Extensions` from VSCode's command palette\n    2) Find `TypeScript and JavaScript Language Features`, right click and select `Disable (Workspace)`\n2. Reload the VSCode window by running `Developer: Reload Window` from the command palette.\n\n## Customize configuration\n\nSee [Vite Configuration Reference](https://vitejs.dev/config/).\n\n## Project Setup\n\n```sh\nnpm install\n```\n\n### Compile and Hot-Reload for Development\n\n```sh\nnpm run dev\n```\n\n### Type-Check, Compile and Minify for Production\n\n```sh\nnpm run build\n```\n\n### Run Unit Tests with [Vitest](https://vitest.dev/)\n\n```sh\nnpm run test:unit\n```\n\n### Run End-to-End Tests with [Nightwatch](https://nightwatchjs.org/)\n\n```sh\n# When using CI, the project must be built first.\nnpm run build\n\n# Runs the end-to-end tests\nnpm run test:e2e\n# Runs the tests only on Chrome\nnpm run test:e2e -- --env chrome\n# Runs the tests of a specific file\nnpm run test:e2e -- tests/e2e/example.ts\n# Runs the tests in debug mode\nnpm run test:e2e -- --debug\n```\n    \n### Run Headed Component Tests with [Nightwatch Component Testing](https://nightwatchjs.org/guide/component-testing/introduction.html)\n  \n```sh\nnpm run test:unit\nnpm run test:unit -- --headless # for headless testing\n```\n\n### Lint with [ESLint](https://eslint.org/)\n\n```sh\nnpm run lint\n```\n\n[discord-shield]: https://img.shields.io/discord/1131685009258987581?style=for-the-badge\u0026logo=discord\n[discord-url]: https://discord.gg/GkaWcUat7R\n[issue-url]: https://github.com/huchenlei/stable-diffusion-ps-pea/issues\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuchenlei%2Fstable-diffusion-ps-pea","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhuchenlei%2Fstable-diffusion-ps-pea","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuchenlei%2Fstable-diffusion-ps-pea/lists"}