{"id":28702277,"url":"https://github.com/modelscope/katz","last_synced_at":"2025-06-14T12:32:21.242Z","repository":{"id":293049162,"uuid":"978521684","full_name":"modelscope/Katz","owner":"modelscope","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-21T09:58:39.000Z","size":72,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-21T10:48:45.103Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modelscope.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-06T05:44:49.000Z","updated_at":"2025-05-21T09:58:44.000Z","dependencies_parsed_at":"2025-05-13T12:48:14.867Z","dependency_job_id":"0a723494-2b95-4618-a4ae-3bf191d123f3","html_url":"https://github.com/modelscope/Katz","commit_stats":null,"previous_names":["modelscope/katz"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/modelscope/Katz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FKatz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FKatz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FKatz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FKatz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modelscope","download_url":"https://codeload.github.com/modelscope/Katz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FKatz/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259816213,"owners_count":22915837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-14T12:31:01.686Z","updated_at":"2025-06-14T12:32:21.236Z","avatar_url":"https://github.com/modelscope.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Katz: Efficient Workflow Serving for Diffusion Models with Many Adapters\n\nKatz is a high-performance serving system designed specifically for diffusion model workflows with multiple adapters. It dramatically improves inference efficiency while maintaining image quality.\n\n## 🌟 Key Features\n\n- **ControlNet-as-a-Service**: Decouples ControlNets from the base model for independent scaling.\n- **Bounded Asynchronous LoRA Loading**: Overlaps LoRA loading with base model execution for reduced latency.\n- **Latent Parallelism**: Accelerates base model execution across multiple GPUs.\n- **Performance Gains**: Up to $7.8 \\times$ latency reduction and $1.7 \\times$ throughput improvement.\n\n## 🎬 Demo\n\n![](./assets/katz_demo.gif)\n\n**Prompt**: papercut -subject/scene-a shiba inu wearing a beret and black turtleneck, 4k, clean background\n\n**Negative prompt**: low quality, bad quality, sketches, numbers, letters\n\nThis image was generated with **1 ControlNet** with depth guidance and **1 LoRA** for the papercut style. The depth reference image used for guidance is available [here](./assets/demo_image_depth.png).\n\n## 🚀 Getting Started\n\n### System Requirements\n\n- NVIDIA GPUs (H800 recommended for best performance)\n- CUDA 11.8+\n- Python 3.10+\n\n### 🚧 Environment Setup\n\n```bash\n$ conda create -n katz python=3.10\n$ conda activate katz\n$ pip install -r requirements.txt\n# Install our customized diffusers package\n$ pushd ./diffusers-hf \u0026\u0026 pip install -e . \u0026\u0026 popd\n# Install fast-kernel\n$ pushd ./diffusers-hf/src/fast_kernel/ \u0026\u0026 git submodule update --init --recursive \u0026\u0026 pip install . \u0026\u0026 popd\n```\n\n## 🔥 Quickstart Example\n\nComing soon.\n\n## 🔮 Artifact Evaluation\n\nFor detailed benchmarking instructions and reproducing our results, see the [artifact evaluation](./artifact_evaluation.md) guide.\n\n## 🗄️ Production Trace Analysis\n\nWe provide tools and datasets for analyzing real-world production traces in the [trace](./trace/README.md) directory.\n\n## 📝 Citation\n\nPlease cite our paper if it is helpful to your research.\n\n```bibtex\n@inproceedings{Katz2025,\n  title = {Katz: Efficient Workflow Serving for Diffusion Models with Many Adapters},\n  author = {Li, Suyi and Yang, Lingyun and Jiang, Xiaoxiao and Lu, Hanfeng and An, Dakai and Di, Zhipeng and Lu, Weiyi and Chen, Jiawei and Liu, Kan and Yu, Yinghao and Lan, Tao and Yang, Guodong and Qu, Lin and Zhang, Liping and Wang, Wei},\n  booktitle = {Proc. USENIX ATC},\n  year = {2025}\n}\n```\n\n## 🙏🏻 Acknowledgement\n\nWe thank the contributors of [🤗 Diffusers](https://github.com/huggingface/diffusers) for their foundational work. \n\n## 📬 Contact\n\nFor questions and support, please open an issue or contact the authors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodelscope%2Fkatz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodelscope%2Fkatz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodelscope%2Fkatz/lists"}