{"id":13455691,"url":"https://github.com/varunshenoy/opendream","last_synced_at":"2025-04-08T09:07:29.564Z","repository":{"id":188539006,"uuid":"639687152","full_name":"varunshenoy/opendream","owner":"varunshenoy","description":"An extensible, easy-to-use, and portable diffusion web UI 👨‍🎨","archived":false,"fork":false,"pushed_at":"2023-08-18T06:01:24.000Z","size":34294,"stargazers_count":1668,"open_issues_count":12,"forks_count":72,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-01T07:49:46.128Z","etag":null,"topics":["ai","automatic-1111","diffusion","image-generation","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/varunshenoy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-05-12T02:17:34.000Z","updated_at":"2025-03-19T06:35:04.000Z","dependencies_parsed_at":"2024-02-17T06:32:26.310Z","dependency_job_id":"899803a8-1988-45c1-a5c9-fc5bfd611d6e","html_url":"https://github.com/varunshenoy/opendream","commit_stats":null,"previous_names":["varunshenoy/opendream"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunshenoy%2Fopendream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunshenoy%2Fopendream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunshenoy%2Fopendream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunshenoy%2Fopendream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/varunshenoy","download_url":"https://codeload.github.com/varunshenoy/opendream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809962,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","automatic-1111","diffusion","image-generation","stable-diffusion"],"created_at":"2024-07-31T08:01:09.439Z","updated_at":"2025-04-08T09:07:29.547Z","avatar_url":"https://github.com/varunshenoy.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Opendream: A Web UI For the Rest of Us 💭 🎨\n\nOpendream brings much needed and familiar features, such as layering, non-destructive editing, portability, and easy-to-write extensions, to your Stable Diffusion workflows. Check out our [demo video](https://twitter.com/varunshenoy_/status/1691506322360201216?s=20).\n\n![hero](images/hero.png)\n\n## Getting started\n\n0. *Prerequisites*: Make sure you have Node installed. You can download it [here](https://nodejs.org/en/download).\n1. Clone this repository.\n2. Navigate to this project within your terminal and run `sh ./run_opendream.sh`. After ~30 seconds, both the frontend and backend of the Opendream system should be up and running.\n\n## Features\n\nDiffusion models have emerged as powerful tools in the world of image generation and manipulation. While they offer significant benefits, these models are often considered black boxes due to their inherent complexity. The current diffusion image generation ecosystem is defined by tools that allow one-off image manipulation tasks to control these models - text2img, in-painting, pix2pix, among others.\n\nFor example, popular interfaces like [Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui), [Midjourney](https://midjourney.com/), and [Stability.AI's DreamStudio](https://beta.dreamstudio.ai/generate) only support destructive editing: each edit \"consumes\" the previous image. This means users cannot easily build off of previous images or run multiple experiments on the same image, limiting their options for creative exploration.\n\n### Layering and Non-destructive Editing\n\nNon-destructive editing is a method of image manipulation that preserves the original image data while allowing users to make adjustments and modifications without overwriting previous work. This approach facilitates experimentation and provides more control over the editing process by using layers and masks. When you delete a layer, all layers after it also get deleted. This guarantees that all layers currently on the canvas are a product of other existing layers. This also allows one to deterministically \"replay\" a workflow.\n\nLike Photoshop, Opendream supports non-destructive editing out of the box. Learn more about the principles of non-destructive editing in Photoshop [here](https://helpx.adobe.com/photoshop/using/nondestructive-editing.html).\n\n![layers](images/editing.png)\n\n### Save and Share Workflows\n\nUsers can also save their current workflows into a portable file format that can be opened up at a later time or shared with collaborators. In this context, a \"state\" is just a JSON file describing all of the current layers and how they were created.\n\n![workflow](images/workflow.png)\n\n### Support Simple to Write, Easy to Install Extensions\n\nAs the open-source ecosystem flourishes around these models and tools, extensibility has also become a major concern. While Automatic1111 does offer extensions, they are often difficult to program, use, and install. It is far from being as full-featured as an application like Adobe Photoshop.\n\nAs new features for Stable Diffusion, like ControlNet, are released, users should be able to seamlessly integrate them into their artistic workflows with minimal overload and time.\n\nOpendream makes writing and using new diffusion features as simple as writing a Python function. Keep reading to learn how.\n\n## Extensions\n\nFrom the get-go, Opendream supports two key primitive operations baked into the core system: `dream` and `mask_and_inpaint`. In this repository, extensions for `instruct_pix2pix`, `controlnet_canny`, `controlnet_openpose`, and `sam` (Segment Anything) are provided.\n\nAny image manipulation logic can be easily written as an extension. With extensions, you can also decide how certain operations work. For example, you can override the `dream` operation to use OpenAI's DALL-E instead or call a serverless endpoint on a service like AWS or Replicate. [Here's an example using Baseten](https://gist.githubusercontent.com/varunshenoy/f029c55536bb7e4fac61a595e836d930/raw/f7e693c8aa42a814d05198c28a843a97c8f6a4c6/baseten_stable_diffusion.py).\n\n### Loading an Existing Extension\n\nThere are two ways to load extensions.\n\n1. Install a pre-written one through the Web UI.\n2. _(Manual)_ Download a valid extension file (or write one yourself!) and add it to the `opendream/extensions` folder. Instructions for writing your own extension are below.\n\nHere is a sampling of currently supported extensions. You can use the links to install any given extension through the Web UI.\n\n| **Extension**               | **Link**                                                                                                                                                         |\n| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| OpenAI's DALL-E             | [File](https://gist.githubusercontent.com/varunshenoy/4a9a6bbfedfa7def28178a8f0563320a/raw/d2d10faa0fad8c2d251e599d962b0c7f62c06db0/dalle.py)                    |\n| Serverless Stable Diffusion | [File](https://gist.githubusercontent.com/varunshenoy/f029c55536bb7e4fac61a595e836d930/raw/f7e693c8aa42a814d05198c28a843a97c8f6a4c6/baseten_stable_diffusion.py) |\n| Instruct Pix2Pix            | [File](https://gist.githubusercontent.com/varunshenoy/894c7a723de6b4651380dd7fa2a81724/raw/fa678d8d6c430421fb481f7023ad76898dd27ad6/instruct_pix2pix.py)         |\n| ControlNet Canny            | [File](https://gist.githubusercontent.com/varunshenoy/0b0455449454e5856021fe2971b78352/raw/1c08b376b499c25c84976eade71db9aa355dba47/controlnet_canny.py)         |\n| ControlNet Openpose         | [File](https://gist.githubusercontent.com/varunshenoy/380722906b8ff184569af57e06fd37b7/raw/728832370db0448bc2807ffc9e267635749e6a9f/controlnet_openpose.py)      |\n| Segment Anything            | [File](https://gist.githubusercontent.com/varunshenoy/5fbc883360e5ab2a3c023ce1e286ddd5/raw/efbc92d27ae2209b15948fb52f657e88c185b349/sam.py)                      |\n| PhotoshopGPT                | [Gist](https://gist.github.com/varunshenoy/63054e7a479f256974416ef45a51e6a0)                                                                                     |\n\nNote that extensions may have their own requirements you would need to include in the `requirements.txt` file. For example, you would need to add `openai` if you want to use the DALL-E extension.\n\nFeel free to make a PR if you create a useful extension!\n\n### Writing Your Own Extension\n\nUsers can write their own extensions as follows:\n\n1. Create a new Python file in the `opendream/extensions` folder.\n2. Write a method with type hints and a `@opendream.define_op` decorator. This decorator registers this method with the Opendream backend.\n\nThe method has a few requirements:\n\n- Parameters must have type hints. These enable the backend to generate a schema for the input which is parsed into form components on the frontend. Valid types include: `str`, `int`, `float`, `Layer`, `MaskLayer`, or `ImageLayer`.\n- The only valid return types are a `Layer` or a list of `Layer` objects.\n\n## Contributions and Licensing\n\n_Opendream was built by Varun Shenoy, Eric Lou, Shashank Rammoorthy, and Rahul Shiv as a part of Stanford's [CS 348K](https://cs348k.stanford.edu/)._\n\nFeel free to provide any contributions you deem necessary or useful. This project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarunshenoy%2Fopendream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvarunshenoy%2Fopendream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarunshenoy%2Fopendream/lists"}