{"id":19666715,"url":"https://github.com/open-mmlab/styleshot","last_synced_at":"2025-05-16T06:07:30.573Z","repository":{"id":246880157,"uuid":"822404890","full_name":"open-mmlab/StyleShot","owner":"open-mmlab","description":"StyleShot: A SnapShot on Any Style. 一款可以迁移任意风格到任意内容的模型，无需针对图片微调，即能生成高质量的个性风格化图片!","archived":false,"fork":false,"pushed_at":"2025-02-11T11:00:48.000Z","size":101769,"stargazers_count":372,"open_issues_count":26,"forks_count":23,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-12T04:45:27.754Z","etag":null,"topics":["controllable-generation","style-transfer","text-to-image"],"latest_commit_sha":null,"homepage":"https://styleshot.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/open-mmlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-01T05:08:12.000Z","updated_at":"2025-04-11T20:08:12.000Z","dependencies_parsed_at":"2025-02-26T19:03:47.828Z","dependency_job_id":"9c4f98d0-d4dc-4e44-800a-70903d55eb4c","html_url":"https://github.com/open-mmlab/StyleShot","commit_stats":null,"previous_names":["open-mmlab/styleshot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-mmlab%2FStyleShot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-mmlab%2FStyleShot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-mmlab%2FStyleShot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-mmlab%2FStyleShot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/open-mmlab","download_url":"https://codeload.github.com/open-mmlab/StyleShot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478190,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["controllable-generation","style-transfer","text-to-image"],"created_at":"2024-11-11T16:28:41.803Z","updated_at":"2025-05-16T06:07:25.551Z","avatar_url":"https://github.com/open-mmlab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ___***StyleShot: A SnapShot on Any Style***___\n\n\u003cdiv align=\"center\"\u003e\n\n \u003ca href='https://arxiv.org/abs/2407.01414'\u003e\u003cimg src='https://img.shields.io/badge/arXiv-2407.01414-b31b1b.svg'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n \u003ca href='https://styleshot.github.io/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\u003ca href='https://openxlab.org.cn/apps/detail/lianchen/StyleShot'\u003e\u003cimg src='https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\u003ca href='https://huggingface.co/Gaojunyao/StyleShot'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\u003ca target=\"_blank\" href=\"https://huggingface.co/spaces/nowsyn/StyleShot\"\u003e\n  \u003cimg src=\"https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg\" alt=\"Online Demo in HF\"/\u003e\n\u003c/a\u003e\n\n_**[Junyao Gao](https://jeoyal.github.io/home/), Yanchen Liu, [Yanan Sun](https://scholar.google.com/citations?hl=zh-CN\u0026user=6TA1oPkAAAAJ)\u003csup\u003e\u0026Dagger;\u003c/sup\u003e, Yinhao Tang, [Yanhong Zeng](https://zengyh1900.github.io/), [Kai Chen*](https://chenkai.site/), [Cairong Zhao*](https://vill-lab.github.io/)**_\n\u003cbr\u003e\u003cbr\u003e\n(* corresponding authors, \u003csup\u003e\u0026Dagger;\u003c/sup\u003e project leader)\n\nFrom Tongji University and Shanghai AI lab.\n\n\u003c/div\u003e\n\n## Abstract\n\nIn this paper, we show that, a good style representation is crucial and sufficient for generalized style transfer without test-time tuning.\nWe achieve this through constructing a style-aware encoder and a well-organized style dataset called StyleGallery.\nWith dedicated design for style learning, this style-aware encoder is trained to extract expressive style representation with decoupling training strategy, and StyleGallery enables the generalization ability.\nWe further employ a content-fusion encoder to enhance image-driven style transfer.\nWe highlight that, our approach, named StyleShot, is simple yet effective in mimicking various desired styles, i.e., 3D, flat, abstract or even fine-grained styles, without test-time tuning. Rigorous experiments validate that, StyleShot achieves superior performance across a wide range of styles compared to existing state-of-the-art methods.\n![arch](assets/teasers.png)\n\n## News\n- [2024/8/29] 🔥 Thanks to @neverbiasu's contribution. StyleShot is now available on [ComfyUI](https://github.com/neverbiasu/ComfyUI-StyleShot).\n- [2024/7/5] 🔥 We release [online demo](https://huggingface.co/spaces/nowsyn/StyleShot) in HuggingFace.\n- [2024/7/3] 🔥 We release [StyleShot_lineart](https://huggingface.co/Gaojunyao/StyleShot_lineart), a version taking the lineart of content image as control.\n- [2024/7/2] 🔥 We release the [paper](https://arxiv.org/abs/2407.01414).\n- [2024/7/1] 🔥 We release the code, [checkpoint](https://huggingface.co/Gaojunyao/StyleShot), [project page](https://styleshot.github.io/) and [online demo](https://openxlab.org.cn/apps/detail/lianchen/StyleShot).\n\n## Start\n\n```\n# install styleshot\ngit clone https://github.com/Jeoyal/StyleShot.git\ncd StyleShot\n\n# create conda env\nconda create -n styleshot python==3.8\nconda activate styleshot\npip install -r requirements.txt\n\n# download the models\ngit lfs install\ngit clone https://huggingface.co/Gaojunyao/StyleShot\ngit clone https://huggingface.co/Gaojunyao/StyleShot_lineart\n```\n\n## Models\n\nyou can download our pretrained weight from [here](https://huggingface.co/Gaojunyao/StyleShot). To run the demo, you should also download the following models:\n- [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)\n- [T2I-Adapter Models](https://huggingface.co/TencentARC)\n- [ControlNet models](https://huggingface.co/lllyasviel)\n- [CLIP Model](https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K)\n\n\n## Inference\nFor inference, you should download the pretrained weight and prepare your own reference style image or content image.\n\n```\n# run text-driven style transfer demo\npython styleshot_text_driven_demo.py --style \"{style_image_path}\" --prompt \"{prompt}\" --output \"{save_path}\"\n\n# run image-driven style transfer demo\npython styleshot_image_driven_demo.py --style \"{style_image_path}\"  --content \"{content_image_path}\" --preprocessor \"Contour\" --prompt \"{prompt}\" --output \"{save_path}\"\n\n# integrate styleshot with controlnet and t2i-adapter\npython styleshot_t2i-adapter_demo.py --style \"{style_image_path}\"  --condition \"{condtion_image_path}\" --prompt \"{prompt}\" --output \"{save_path}\"\npython styleshot_controlnet_demo.py --style \"{style_image_path}\"  --condition \"{condtion_image_path}\" --prompt \"{prompt}\" --output \"{save_path}\"\n```\n\n- [**styleshot_text_driven_demo**](styleshot_text_driven_demo.py): text-driven style transfer with reference style image and text prompt.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=assets/text_driven.png\u003e\n\u003cp\u003eText-driven style transfer visualization\u003c/p\u003e\n\u003c/div\u003e\n\n- [**styleshot_image_driven_demo**](styleshot_image_driven_demo.py): image-driven style transfer with reference style image and content image.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=assets/image_driven.png\u003e\n \u003cp\u003eImage style transfer visualization\u003c/p\u003e\n\u003c/div\u003e\n\n- [**styleshot_controlnet_demo**](styleshot_controlnet_demo.py), [**styleshot_t2i-adapter_demo**](styleshot_t2i-adapter_demo.py): integration with controlnet and t2i-adapter.\n\n## Train\nWe employ a two-stage training strategy to train our StyleShot for better integration of content and style. For training data, you can use our training dataset [StyleGallery](#style_gallery) or make your own dataset into a json file.\n\n```\n# training stage-1, only training the style component.\naccelerate launch --num_processes 8 --multi_gpu --mixed_precision \"fp16\" \\\n  tutorial_train_styleshot_stage_1.py \\\n  --pretrained_model_name_or_path=\"runwayml/stable-diffusion-v1-5/\" \\\n  --image_encoder_path=\"{image_encoder_path}\" \\\n  --image_json_file=\"{data.json}\" \\\n  --image_root_path=\"{image_path}\" \\\n  --mixed_precision=\"fp16\" \\\n  --resolution=512 \\\n  --train_batch_size=16 \\\n  --dataloader_num_workers=4 \\\n  --learning_rate=1e-04 \\\n  --weight_decay=0.01 \\\n  --output_dir=\"{output_dir}\" \\\n  --save_steps=10000\n\n# training stage-2, only training the content component.\naccelerate launch --num_processes 8 --multi_gpu --mixed_precision \"fp16\" \\\n  tutorial_train_styleshot_stage_2.py \\\n  --pretrained_model_name_or_path=\"runwayml/stable-diffusion-v1-5/\" \\\n  --pretrained_ip_adapter_path=\"./pretrained_weight/ip.bin\" \\\n  --pretrained_style_encoder_path=\"./pretrained_weight/style_aware_encoder.bin\" \\\n  --image_encoder_path=\"{image_encoder_path}\" \\\n  --image_json_file=\"{data.json}\" \\\n  --image_root_path=\"{image_path}\" \\\n  --mixed_precision=\"fp16\" \\\n  --resolution=512 \\\n  --train_batch_size=16 \\\n  --dataloader_num_workers=4 \\\n  --learning_rate=1e-04 \\\n  --weight_decay=0.01 \\\n  --output_dir=\"{output_dir}\" \\\n  --save_steps=10000\n```\n\n## StyleGallery\u003ca name=\"style_gallery\"\u003e\u003c/a\u003e\nWe have carefully curated a style-balanced dataset, called **StyleGallery**, with extensive diverse image styles drawn from publicly available datasets for training our StyleShot. \nTo prepare our dataset StyleGallery, please refer to [tutorial](DATASET.md), or download json file from [here](https://drive.google.com/drive/folders/10T3t58rQKDmYOLschUYj0tzm6zuOngMd?usp=drive_link).\n\n## StyleBench\nTo address the lack of a benchmark in reference-based stylized generation, we establish a \u003ca href='https://drive.google.com/file/d/1Q_jbI25NfqZvuwWv53slmovqyW_L4k2r/view?usp=drive_link'\u003estyle evaluation benchmark\u003c/a\u003e containing 40 content images and 73 distinct styles across 490 reference images.\n\n## Disclaimer\n\nThis project strives to positively impact the domain of AI-driven image generation. Users are granted the freedom to create images using this tool, but they are expected to comply with local laws and utilize it in a responsible manner. **The developers do not assume any responsibility for potential misuse by users.**\n\n\n## Citation\nIf you find StyleShot useful for your research and applications, please cite using this BibTeX:\n```bibtex\n@article{gao2024styleshot,\n  title={Styleshot: A snapshot on any style},\n  author={Gao, Junyao and Liu, Yanchen and Sun, Yanan and Tang, Yinhao and Zeng, Yanhong and Chen, Kai and Zhao, Cairong},\n  journal={arXiv preprint arXiv:2407.01414},\n  year={2024}\n}\n```\n\n## Acknowledgements\nThe code is built upon \u003ca href='https://github.com/tencent-ailab/IP-Adapter'\u003eIP-Adapter\u003c/a\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-mmlab%2Fstyleshot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopen-mmlab%2Fstyleshot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-mmlab%2Fstyleshot/lists"}