{"id":17772923,"url":"https://github.com/radames/real-time-latent-consistency-model","last_synced_at":"2025-05-15T03:08:26.389Z","repository":{"id":204367112,"uuid":"711319439","full_name":"radames/Real-Time-Latent-Consistency-Model","owner":"radames","description":"App showcasing multiple real-time diffusion models pipelines with Diffusers","archived":false,"fork":false,"pushed_at":"2025-05-02T23:22:36.000Z","size":374,"stargazers_count":898,"open_issues_count":3,"forks_count":106,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-05-14T11:10:49.275Z","etag":null,"topics":["diffusers","diffusion-models","latent-consistency-model","machine-learning","mjpeg","mjpeg-stream","real-time","stable-diffusion"],"latest_commit_sha":null,"homepage":"https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radames.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-28T21:52:18.000Z","updated_at":"2025-05-13T04:09:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"6025d194-138e-4855-ba1d-c910f827eb7b","html_url":"https://github.com/radames/Real-Time-Latent-Consistency-Model","commit_stats":null,"previous_names":["radames/real-time-latent-consistency-model"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radames%2FReal-Time-Latent-Consistency-Model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radames%2FReal-Time-Latent-Consistency-Model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radames%2FReal-Time-Latent-Consistency-Model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radames%2FReal-Time-Latent-Consistency-Model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radames","download_url":"https://codeload.github.com/radames/Real-Time-Latent-Consistency-Model/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264771,"owners_count":22041794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusers","diffusion-models","latent-consistency-model","machine-learning","mjpeg","mjpeg-stream","real-time","stable-diffusion"],"created_at":"2024-10-26T21:41:31.551Z","updated_at":"2025-05-15T03:08:21.368Z","avatar_url":"https://github.com/radames.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\ntitle: Real-Time Latent Consistency Model Image-to-Image ControlNet\nemoji: 🖼️🖼️\ncolorFrom: gray\ncolorTo: indigo\nsdk: docker\npinned: false\nsuggested_hardware: a10g-small\ndisable_embedding: true\n---\n\n# Real-Time Latent Consistency Model\n\nThis demo showcases [Latent Consistency Model (LCM)](https://latent-consistency-models.github.io/) using [Diffusers](https://huggingface.co/docs/diffusers/using-diffusers/lcm) with a MJPEG stream server. You can read more about LCM + LoRAs with diffusers [here](https://huggingface.co/blog/lcm_lora).\n\nYou need a webcam to run this demo. 🤗\n\nSee a collecting with live demos [here](https://huggingface.co/collections/latent-consistency/latent-consistency-model-demos-654e90c52adb0688a0acbe6f)\n\n## Running Locally\n\nYou need CUDA and Python 3.10, Node \u003e 19, Mac with an M1/M2/M3 chip or Intel Arc GPU\n\n\n## Install\n\n```bash\nuv venv --python=3.10  \nsource .venv/bin/activate\nuv pip install -r server/requirements.txt\ncd frontend \u0026\u0026 npm install \u0026\u0026 npm run build \u0026\u0026 cd ..\npython server/main.py --reload --pipeline img2imgSDTurbo \n ```\n\nDon't forget to fuild the frontend!!! \n\n```bash\ncd frontend \u0026\u0026 npm install \u0026\u0026 npm run build \u0026\u0026 cd ..\n```\n\n# Pipelines\nYou can build your own pipeline following examples here [here](pipelines),\n\n\n# LCM\n### Image to Image\n\n```bash\npython server/main.py --reload --pipeline img2img \n```\n\n# LCM\n### Text to Image\n\n```bash\npython server/main.py --reload --pipeline txt2img \n```\n\n### Image to Image ControlNet Canny\n\n```bash\npython server/main.py --reload --pipeline controlnet \n```\n\n\n# LCM + LoRa\n\nUsing LCM-LoRA, giving it the super power of doing inference in as little as 4 steps. [Learn more here](https://huggingface.co/blog/lcm_lora) or [technical report](https://huggingface.co/papers/2311.05556)\n\n\n### Image to Image ControlNet Canny LoRa\n\n```bash\npython server/main.py --reload --pipeline controlnetLoraSD15\n```\nor SDXL, note that SDXL is slower than SD15 since the inference runs on 1024x1024 images\n\n```bash\npython server/main.py --reload --pipeline controlnetLoraSDXL\n```\n\n### Text to Image\n\n```bash\npython server/main.py --reload --pipeline txt2imgLora\n```\n\n```bash\npython server/main.py --reload --pipeline txt2imgLoraSDXL\n```\n# Available Pipelines\n\n#### [LCM](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)\n\n`img2img`  \n`txt2img`   \n`controlnet`   \n`txt2imgLora`   \n`controlnetLoraSD15` \n\n#### [SD15](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)\n`controlnetLoraSDXL`   \n`txt2imgLoraSDXL`   \n\n#### [SDXL Turbo](https://huggingface.co/stabilityai/sd-xl-turbo)\n\n`img2imgSDXLTurbo`    \n`controlnetSDXLTurbo`   \n\n\n#### [SDTurbo](https://huggingface.co/stabilityai/sd-turbo)\n`img2imgSDTurbo`   \n`controlnetSDTurbo`   \n\n#### [Segmind-Vega](https://huggingface.co/segmind/Segmind-Vega)\n`controlnetSegmindVegaRT`   \n`img2imgSegmindVegaRT`   \n\n\n### Setting environment variables\n\n\n* `--host`: Host address (default: 0.0.0.0)  \n* `--port`: Port number (default: 7860)  \n* `--reload`: Reload code on change  \n* `--max-queue-size`: Maximum queue size (optional)\n* `--timeout`: Timeout period (optional)\n* `--safety-checker`: Enable Safety Checker (optional) \n* `--torch-compile`: Use Torch Compile\n* `--use-taesd` / `--no-taesd`: Use Tiny Autoencoder  \n* `--pipeline`: Pipeline to use (default: \"txt2img\")  \n* `--ssl-certfile`: SSL Certificate File (optional)\n* `--ssl-keyfile`: SSL Key File (optional)\n* `--debug`: Print Inference time  \n* `--compel`: Compel option  \n* `--sfast`: Enable Stable Fast   \n* `--onediff`: Enable OneDiff\n\nIf you run using `bash build-run.sh` you can set `PIPELINE` variables to choose the pipeline you want to run\n\n```bash\nPIPELINE=txt2imgLoraSDXL bash build-run.sh\n```\n\nand setting environment variables\n\n```bash\nTIMEOUT=120 SAFETY_CHECKER=True MAX_QUEUE_SIZE=4 python server/main.py --reload --pipeline txt2imgLoraSDXL\n```\n\nIf you're running locally and want to test it on Mobile Safari, the webserver needs to be served over HTTPS, or follow this instruction on my [comment](https://github.com/radames/Real-Time-Latent-Consistency-Model/issues/17#issuecomment-1811957196)\n\n```bash\nopenssl req -newkey rsa:4096 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem\npython server/main.py --reload --ssl-certfile=certificate.pem --ssl-keyfile=key.pem\n```\n\n## Docker\n\nYou need NVIDIA Container Toolkit for Docker, defaults to `controlnet``\n\n```bash\ndocker build -t lcm-live .\ndocker run -ti -p 7860:7860 --gpus all lcm-live\n```\n\nreuse models data from host to avoid downloading them again, you can change `~/.cache/huggingface` to any other directory, but if you use hugingface-cli locally, you can share the same cache\n\n```bash\ndocker run -ti -p 7860:7860 -e HF_HOME=/data -v ~/.cache/huggingface:/data  --gpus all lcm-live\n```\n \n\nor with environment variables\n\n```bash\ndocker run -ti -e PIPELINE=txt2imgLoraSDXL -p 7860:7860 --gpus all lcm-live\n```\n\n\n# Demo on Hugging Face\n\n\n* [radames/Real-Time-Latent-Consistency-Model](https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model)  \n* [radames/Real-Time-SD-Turbo](https://huggingface.co/spaces/radames/Real-Time-SD-Turbo)  \n* [latent-consistency/Real-Time-LCM-ControlNet-Lora-SD1.5](https://huggingface.co/spaces/latent-consistency/Real-Time-LCM-ControlNet-Lora-SD1.5)  \n* [latent-consistency/Real-Time-LCM-Text-to-Image-Lora-SD1.5](https://huggingface.co/spaces/latent-consistency/Real-Time-LCM-Text-to-Image-Lora-SD1.5)  \n* [radames/Real-Time-Latent-Consistency-Model-Text-To-Image](https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model-Text-To-Image)  \n\n\n\n\nhttps://github.com/radames/Real-Time-Latent-Consistency-Model/assets/102277/c4003ac5-e7ff-44c0-97d3-464bb659de70\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradames%2Freal-time-latent-consistency-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradames%2Freal-time-latent-consistency-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradames%2Freal-time-latent-consistency-model/lists"}